Drupal - What is the correct core update workflow?

I presume that you are using drupal-composer/drupal-project as the basis for your project. If not, take a look at that project, and compare it with yours.

Also, you said that you want to use composer to manage Drupal 8 dependencies, so I presume that you have selected your contrib modules via composer require drupal/devel rather than drush dl devel.

If you are doing all of these things, then you should use composer update to update Drupal core and all of your contrib modules. As long as you retain your composer.lock file, composer install should not change the version of any of your dependencies. You should not use drush pm-update at all. It should not matter to you whether or not files in the core directory are updated or not, as this directory is managed by Composer. You are better off not committing composer-managed directories to your repository, although you can if you wish.

Of course, you should run drush updatedb whenever composer update replaces Drupal core or any module.

To avoid getting development versions, set your minimum stability to 'beta' in your composer.json file using Composer stability flags.

If you are using drupal-composer/drupal-project to manage your site, then all of the root-level files such as README.txt, .htaccess and index.html become owned by your project. That means that you should check them in to your git repository; Composer will not update them, you must update them yourself when they change. These files should change only rarely, but drupal-composer/drupal-project has a script to update these files.

The following is OK for patch releases 8.4.x > 8.4.y, but not OK for minor releases 8.4.x > 8.5.x. Jump to UPDATE 3 below for what I believe is "the answer" for minor release updates.

1- Backup any files that come with Drupal which you have modified, such as .htaccess, robots.txt, etc. (those 2 are the most commonly changed).

2- [I am told delete lock file is wrong, see UPDATE below] Delete the composer.lock file (in the top level folder of your site). This gets recreated in step 5.

3- Check your composer.json (in the top level folder of your site) and make sure that the "drupal:core" is in the require section and not in a replace section, for example

"require": {
"drupal/core": "^8.4"
},

not

"replace": {
"drupal/core": "^8.4"
},

If "drupal/core" is in the replace section, move it to the require section and delete the replace section. If there are other entries in the replace section, just remove the "drupal/core" not the entire replace section - but I think "drupal/core" is normally the only thing there.

Put what version you want to update to in "drupal/core", examples:

"drupal/core": "^8.5" - will update to the latest version of 8.5. "drupal/core": "8.4.6" - will update to version 8.4.6.

5- Run this (in the top level folder of your site):

composer update drupal/core --with-dependencies

6- If no errors, then do the usual, run the updates and clear cache:

drush updatedb
drush cr

Or if not using drush, go to /update.php to run updates, then to admin/config/development/performance and hit the "Clear all caches" button.

7- If you had backed up files in the first step (.htaccess, robots.txt) put them back. But check to see if Drupal made updates to those files and add those changes to yours.

DONE

If there were errors with the composer update in step 5, it is usually due to issues with versions of the stuff in the vendor folder.

This is a great post in dealing with such issues: https://www.jeffgeerling.com/blog/2018/updating-drupalcore-composer-drupal-core-doesnt-update and read Jeff's other 2 posts on Drupal and Composer to get more knowledge about that.

I was told by 2 people on Twitter that composer.lock should not be deleted (Step 2 above). The composer update drupal/core --with-dependencies command recreates the lock file anyways.

In testing this method I find it works fine for 8.4.3 > 8.4.6 (for example) but I get errors for 8.4.6 > 8.5.x. Will report back when I figure it out.

Example of errors:

Your requirements could not be resolved to an installable set of packages.
  Problem 1
    - symfony/yaml 3.4.x-dev conflicts with symfony/console[v3.2.8].
    - symfony/yaml 3.4.x-dev conflicts with symfony/console[v3.2.8].
    - symfony/yaml 3.4.x-dev conflicts with symfony/console[v3.2.8].
    - drupal/core 8.5.0 requires symfony/yaml ~3.4.5 -> satisfiable by symfony/yaml[3.4.x-dev].
    - Installation request for drupal/core 8.5.0 -> satisfiable by drupal/core[8.5.0].
    - Installation request for symfony/console (locked at v3.2.8, required as ~3.2.8) -> satisfiable by symfony/console[v3.2.8].

This post by Jeff Geerling addresses similar issues, but so far no luck for me: https://www.jeffgeerling.com/blog/2018/updating-drupalcore-composer-drupal-core-doesnt-update

So... the only thing that seems to work for me for 8.4.x > 8.5.x is the "nuclear option" that so many others seem to use, which is run composer update.

I guess that is OK so long as you are sure about the module versions in composer.json. Maybe one should lock them down to the current version. For example:

"drupal/address": "1.3"

rather than:

"drupal/address": "^1.3"

But is the the right answer?

OK the answer that seems to be everywhere is to do the "nuclear option":

A. Delete the /vendor folder.

B. Run composer update and simply update your modules along with core. Or, lock down the module versions in composer.json if you don't want to update them.

One person on Drupal Slack said "the whole philosophy of Composer is that you should always be updating packages, as frequently as possible". Packaged includes modules I think. So it makes some sense I guess.

Once I got from 8.4.6 to 8.5.0, this worked fine to get from 8.5.0 to 8.5.1 composer update drupal/core --with-dependencies just as it did for 8.4.3 to 8.4.6.

I am starting to conclude that "the answer" is that deleting the vendor folder and the composer.lock file, then using composer update is fine, and that one should simply make sure the version numbers for dependencies in the composer.json file are what you want. It's not that big a deal to manage what module versions you want to keep or allow to update in composer.json.

For example:

"drupal/admin_toolbar": "1.18", means stick with 1.18

"drupal/admin_toolbar": "^1.18", means go ahead and update but within 1.x (not 2.x)

This is backed up by a comment (General Redneck) on this post: https://www.jeffgeerling.com/blog/2018/updating-drupalcore-composer-drupal-core-doesnt-update "One of the things I've found as I work in support is that locking down the versions of modules and core is a good idea so that you CAN thermonuke the thing when you want because there are times when some of the various plugins even don't want to behave correctly."

By the way, the composer.lock file is no help with composer update because it gets blown away (as opposed to composer install where he lock file is read):

Running composer install will:

• Check if a composer.lock exists
• If not, perform a composer update to create one
• If composer.lock exists, install the specified versions from the lock file

Running composer update will:

• Check composer.json
• Determine the latest versions to install based on your version specs
• Install the latest versions
• Update composer.lock to reflect the latest versions installed

Ref: https://www.engineyard.com/blog/composer-its-all-about-the-lock-file

I see this is mentioned above: https://github.com/drupal-composer/drupal-project. I have used that and it's fine but it is not a requirement for using Composer with Drupal. It is confusing as it sort of "sounds" like it is from the name. When I first started with Drupal 8 I thought it was required, so built my first D8 site with that, thinking it was best practice.

That "version" of Drupal has he docroot in a /web folder, not in the top folder of the project. Also there is a bunch of stuff added to .gitignore compared to normal Drupal:

/drush/contrib/
/vendor/
/web/core/
/web/modules/contrib/
/web/themes/contrib/
/web/profiles/contrib/
/web/libraries/

So, this version of Drupal is really more meant for sites which are using continuous integration to do a new build of Drupal on every deploy, using composer install. If you deploy with a more normal method, you obviously have to commit all the above stuff to your git repo or it will not get deployed to your server [1], and that stuff is all needed for Drupal to run.

[1] if git is involved with your deploy - if you deploy with SFTP, ignore this.