Drupal composer workflow - part 2

In the previous post, we saw how to add and manage modules and module dependencies in Drupal 8 using Composer.

In this post we shall see how to use an exclusive composer based Drupal 8 workflow. Let's start with a vanilla Drupal install. The recommended way to go about it is to use Drupal Composer project.

$ composer create-project drupal-composer/drupal-project:8.x-dev drupal-8.dev

If you are a careful observer(unlike me), you will notice that a downloaded Drupal 8 package ships with the vendor/ directory. In other words, we need not install the composer dependencies when we download it from d.o. On the other hand, if you "git cloned" Drupal 8, it won't contain the vendor/ directory, hence the extra step to run `composer install` in root directory. The top level directory contains a composer.json and the name of the package is drupal/drupal, which is more of a wrapper for the drupal/core package inside the core/ directory. The drupal/core package installs Drupal core and its dependencies. The drupal/drupal helps you build a site around Drupal core, maintains dependencies related to your site and modules etc.

Drupal project takes a slightly different project structure.It installs core and its dependencies similar to drupal/drupal. It also installs the latest stable versions of drush and drupal console.

$ composer create-project drupal-composer/drupal-project:8.x-dev d8dev --stability dev --no-interaction

New directory structure

Everything Drupal related goes in the web/ directory, including core, modules, profiles and themes. Contrast this with the usual structure where there is a set of top level directories named core, modules, profiles and themes.

drush and drupal console(both latest stable versions) gets installed inside vendor/bin directory.The reason Drush and Drupal console are packaged on a per project basis is to avoid any dependency issues which we might normally face if they are installed globally.

How to install Drupal

Drupal can be installed using the typical site-install command provided by drush.

$ cd d8dev/web
$ ../vendor/bin/drush site-install --db-url=mysql://<db-user-name>:<db-password>@localhost/<db-name> -y

Downloading modules

Modules can be downloaded using composer. They get downloaded in the web/modules/contrib directory.

$ cd d8dev
$ composer require drupal/devel:8.1.x-dev

The following things happen when we download a module via composer.

  1. Composer updates the top level composer.json and adds drupal/devel:8.1.x-dev as a dependency.
    "require": {
        "composer/installers": "^1.0.20",
        "drupal-composer/drupal-scaffold": "^2.0.1",
        "cweagans/composer-patches": "~1.0",
        "drupal/core": "~8.0",
        "drush/drush": "~8.0",
        "drupal/console": "~1.0",
        "drupal/devel": "8.1.x-dev"
    },
  1. Composer dependencies(if any) for that module get downloaded in the top level vendor directory. These are specified in the composer.json file of that module. At the time of writing this, Devel module does not have any composer dependencies.
     "license": "GPL-2.0+",
      "minimum-stability": "dev",
      "require": { }
    }

Most modules in Drupal 8 were(are) written without taking composer into consideration. We use the drush dl command every time which parses our request and downloads the appropriate version of the module from drupal.org servers. Downloading a module via composer requires the module to have a composer.json as a minimal requirement. So how does composer download all Drupal contrib modules if they don't have any composer.json? The answer lies in a not so secret sauce ingredient we added in our top level composer.json:

"repositories": [
    {
        "type": "composer",
        "url": "https://packagist.drupal-composer.org"
    }
],

Composer downloads all packages from a central repository called Packagist. It is the npmjs equivalent of PHP. Drupal provides its own flavour of Packagist to serve modules and themes exclusively hosted at Drupal.org. Drupal packagist ensures that contrib maintainers need not add composer.json to their project.

Let's take another module which does not have a composer.json, like Flag(at the time of writing this). Let's try and download flag using composer.

$ composer require drupal/flag:8.4.x-dev
./composer.json has been updated
> DrupalProject\composer\ScriptHandler::checkComposerVersion
Loading composer repositories with package information
Updating dependencies (including require-dev)
  - Installing drupal/flag (dev-8.x-4.x 16657d8)
    Cloning 16657d8f84b9c87144615e4fbe551ad9a893ad75

Writing lock file
Generating autoload files
> DrupalProject\composer\ScriptHandler::createRequiredFiles

Neat. Drupal Packagist parses contrib modules and serves the one which matches the name and version we gave when we ran that "composer require" command.

Specifying package sources

There is one other step you need to do to complete your composer workflow, i.e., switching to the official Drupal.org composer repository. The actual composer.json contains Drupal packagist as the default repository.

"repositories": [
    {
        "type": "composer",
        "url": "https://packagist.drupal-composer.org"
    }
],

Add the Drupal.org composer repo using the following command:

$ composer config repositories.drupal composer https://packages.drupal.org/8

Now, your repositories entry in composer.json should look like this:

"repositories": {
       "0": {
           "type": "composer",
           "url": "https://packagist.drupal-composer.org"
       },
       "drupal": {
           "type": "composer",
           "url": "https://packages.drupal.org/8"
       }
   }

To ensure that composer indeed downloads from the new repo we specified above, let's remove the drupal packagist entry from composer.json.

$ composer config --unset repositories.0

The repositories config looks like this now:

"repositories": {
      "drupal": {
          "type": "composer",
          "url": "https://packages.drupal.org/8"
      }
  }

Now, let's download a module from the new repo.

$ composer require drupal/token -vvv

As a part of the verbose output, it prints the following:

...
Loading composer repositories with package information
Downloading https://packages.drupal.org/8/packages.json
Writing /home/lakshmi/.composer/cache/repo/https---packages.drupal.org-8/packages.json into cache
...

which confirms that we downloaded from the official package repo.

Custom package sources

Sometimes, you might want to specify your own package source for a custom module you own, say, in Github. This follows the usual conventions for adding VCS package sources in Composer, but I'll show how to do it in Drupal context.

First, add your github URL as a VCS repository using the composer config command.

$ composer config repositories.restful vcs "https://github.com/RESTful-Drupal/restful"

Your composer.json will look like this after the above command is run successfully:

"repositories": {
    "drupal": {
        "type": "composer",
        "url": "https://packages.drupal.org/8"
    },
    "restful": {
        "type": "vcs",
        "url": "https://github.com/RESTful-Drupal/restful"
    }
}

If you want to download a package from your custom source, you might want it to take precedence to the official package repository, as order really matters for composer. I haven't found a way to do this via cli, but you can edit the composer.json file and swap both package sources to look like this:

"repositories": {
    "restful": {
        "type": "vcs",
        "url": "https://github.com/RESTful-Drupal/restful"
    },
    "drupal": {
        "type": "composer",
        "url": "https://packages.drupal.org/8"
    }
}

Now, lets pick up restful 8.x-3.x. We can specify a Github branch by prefixing with a "dev-".

$ composer require "drupal/restful:dev-8.x-3.x-not-ready"

Once restful is downloaded, composer.json is updated accordingly.

"require": {
     "composer/installers": "^1.0.20",
     "drupal-composer/drupal-scaffold": "^2.0.1",
     "cweagans/composer-patches": "~1.0",
     "drupal/core": "~8.0",
     "drush/drush": "~8.0",
     "drupal/console": "~1.0",
     "drupal/devel": "8.1.x-dev",
     "drupal/flag": "8.4.x-dev",
     "drupal/mailchimp": "8.1.2",
     "drupal/token": "1.x-dev",
     "drupal/restful": "dev-8.x-3.x-not-ready"
 },

Updating drupal core

Drupal core can be updated by running:

$ composer update drupal/core
> DrupalProject\composer\ScriptHandler::checkComposerVersion
Loading composer repositories with package information
Updating dependencies (including require-dev)                                         
  - Removing drupal/core (8.1.7)
  - Installing drupal/core (8.1.8)
    Downloading: 100%         

Writing lock file
Generating autoload files
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
> DrupalProject\composer\ScriptHandler::createRequiredFiles

As the output reads, we updated core from 8.1.7 to 8.1.8. We will revisit the "Writing lock file" part in a moment. After this step is successful, we have to run drush updatedb to do any database updates. This applies to even updating modules.

$ cd d8dev/web
$ ../vendor/bin/drush updatedb

Updating modules

Updating modules in composer workflow is can be done by the composer update command. For updating a module, say, devel we would do:

$ composer update drupal/devel
> DrupalProject\composer\ScriptHandler::checkComposerVersion
Loading composer repositories with package information
Updating dependencies (including require-dev)                                         
Nothing to install or update
Generating autoload files
> DrupalProject\composer\ScriptHandler::createRequiredFiles

Hmmm. Looks like devel is already the latest bleeding edge version. To quickly revise and ensure what composer related artifacts we need to check in to version control,

Should you check in the vendor/ directory?

Composer recommends that you shouldn't, but there are some environments that don't support composer(ex. Acquia Cloud), in which case you have to check in your vendor folder too.

Should you check in the composer.json file?

By now, you should know the answer to this question :)

Should you check in the composer.lock file?

Damn yes. composer.lock contains the exact version of the dependencies which are installed. For example, if your project depends on Acme 1.*, and you install 1.1.2 and your co-worker runs composer install after a month or so, it might install Acme 1.1.10, which might introduce version discrepancies in your project. To prevent this, composer install will check if a lock file exists, and install only that specific version recorded or "locked" down in the lock file. The only time the lock file changes is when you run a composer update to update your project dependencies to their latest versions. When that happens, composer updates the lock file with the newer version that got installed.