Separating your project and environment settings in Drupal 7

For Drupal 7 sites it’s best that you split up your settings.php in 2 different files: settings.php and env.settings.php.

The logic behind this is:

  • settings.php contains all project-related settings, that are the same for all environments.
  • env.settings.php contains all environment-related settings that are different for all enviroments.

Given this logic, it’s safe to commit the settings.php to our git repository while keeping env.settings.php out of it. The env.settings.php is created manual when setting up an environment.

Examples files

Settings.php contains certain module includes, php ini settings, etc. while env.settings.php contains database connection info, memcache prefixes, reverse proxy configuration, etc.

A simple settings.php file:

$update_free_access = FALSE;
$drupal_hash_salt = 'averysecrethash';

ini_set('session.gc_probability', 1);
ini_set('session.gc_divisor', 100);
ini_set('session.gc_maxlifetime', 200000);
ini_set('session.cookie_lifetime', 2000000);

$conf['memcache_persistent'] = TRUE;
$conf['cache_backends'][] = 'sites/all/modules/contrib/memcache/memcache.inc';
$conf['cache_default_class'] = 'MemCacheDrupal';
## The 'cache_form' bin must be assigned no non-volatile storage.
$conf['cache_class_cache_form'] = 'DrupalDatabaseCache';

$conf['404_fast_paths_exclude'] = '/\/(?:styles)\//';
$conf['404_fast_paths'] = '#\.(?:txt|png|gif|jpe?g|css|js|ico|swf|flv|cgi|bat|pl|dll|exe|asp)$#i';
$conf['404_fast_html'] = '<html xmlns="http:##www.w3.org#1999#xhtml"><head><title>404 Not Found</title></head><body>
<h1>Not Found</h1>
The requested URL "@path" was not found on this server.</body></html>';

$conf['drupal_http_request_fails'] = FALSE;

# environment-specific settings
$settings = DRUPAL_ROOT . '/sites/default/env.settings.php';
if (file_exists($settings)) {
 require_once($settings);
}

An example env.settings.php file:

$databases = array (
 'default' => array (
   'default' => array (
     'database' => 'db_name',
     'username' => 'db_user',
     'password' => '',
     'host' => 'localhost',
     'port' => '',
     'driver' => 'mysql',
     'prefix' => '',
    ),
  ),
);

## We use a memcache prefix as multiple sites use the same bin
$conf['memcache_key_prefix'] = 'prod';
$conf['drupal_http_request_fails'] = FALSE;

## See https://www.karelbemelmans.com/2015/07/reverse-proxy-configuration-for-apache-or-nginx-with-varnish/
$conf['reverse_proxy'] = TRUE;
$conf['reverse_proxy_addresses'] = array('127.0.0.1');
$conf['reverse_proxy_header'] = 'HTTP_X_FORWARDED_FOR';

## Zen-theme option
$conf['theme_narfum_settings']['zen_rebuild_registry'] = 0;

## Caching is on for production
$conf['cache'] = 1;
$conf['block_cache'] = 1;

## Error display is off for production
$conf['error_level'] = 0;

## Turn off js and css aggregation ON
$conf['preprocess_css'] = 1;
$conf['preprocess_js'] = 1;

## GD image quality
$conf['image_jpeg_quality'] = 90;

## Maintenance mode?
$conf['maintenance_mode'] = 0;

Directory structure

How we use this in practise is that settings.php is always deployed with the rest of the Drupal code, while the env.settings.php is kept outside of the website root but linked to with a symbolic link.

Example directory structure:

httpdocs/
  - sites/
    - default/
      - settings.php
      - env.settings.php -> $HOME/shared/env.settings.php
      - files -> $HOME/shared/files
shared/
  - env.settings.php
  - files/

You can see we use symbolic links for the files and env.settings.php file. These links are actually made by our deployment script. The httpdocs directory (= the website root) is actually also a symbolic to a specific release, like this:

httpdocs -> $HOME/releases/release-1.0.3
shared/
  - env.settings.php
  - files/
releases/
  - release-1.0.3/
    - sites/
      - default/
        - settings.php
        - env.settings.php -> $HOME/shared/env.settings.php
        - files -> $HOME/shared/files

If you maintain this logic, setting up an automated deployment is easy and you will never have to do bad stuff like committing passwords or environment specific settings to your Drupal code.

Puphpet - PHP Vagrant development images made easy

Update 2016/07/06: You should probably not be using Vagrant (and Puphpet) anymore now that we live in the wonderful world of Docker containers. I will leave this blog post online for now, but take this information as ‘extremely outdated’.

Puphpet is, beside a horrible word to type, a great tool to create specific Vagrant images that contain a very fine-tuned development stack for PHP development.

The website generates puppet scripts that will provision a Vagrant image with your specific configuration. Using the online configuration tool you have huge selection of things to include in your image:

  • Ubuntu or CentOS base images
  • apache or nginx webservers
  • php versions from early 5 version to php 7 releases candidates and even [HHVM](https://github.com/facebook/hhvm/
  • MySQL or MariaDB
  • PostgreSQL
  • MongoDB
  • Reddis
  • SQLite
  • Mailcatcher
  • Beanstalkd
  • RabbitMQ
  • Elasticsearch
  • Apache Solr (4.x and 5. versions)

Even if you know nothing of server setups this is a great tool to build a production-like environment to develop on. It’s easy to commit these config files in your git repository and let co-workers use them without having to configure one single thing about it.

If you haven’t used Vagrant yet, you are living in the past and should really catch up

Weird errors with Drupal themes containing a node_modules folder

Nowadays frontend developers are totally into the whole compass, grunt, gulp, bower, etc.. thing to automate things as compiling SASS code into css, minimizing javascript, optimizing image sizes and so on.

These tools include using the node package manager, npm, to install a load of modules using a package.json file. These modules most of the time go inside a folder “node_modules” and “bower_components” inside the root folder of your Drupal theme. And that’s where stuff can go wrong.

The error

If you’ve seen this next error after running a drush cc all or flushing caches via your site, you’re probably having the problem I’m going to describe below:

WD menu: PDOException: SQLSTATE[23000]: Integrity constraint violation: 1048 Column 'title' cannot be null: INSERT INTO {menu_router} (path, load_functions, to_arg_functions, access_callback, [error], access_arguments, page_callback, page_arguments, delivery_callback, fit, number_parts, context, tab_parent, tab_root, title, title_callback, title_arguments, theme_callback, theme_arguments, type, description, position, weight, include_file) VALUES ...

This failing menu rebuild action will probably leave your site in a broken state where no menu callbacks work.

After that error message you probably get a var_dump of the router menu code, which is pretty much useless, except that it contains references to files like this:

sites/all/themes/drupaltheme/node_modules/gulp-eslint/node_modules/eslint/node_modules/doctrine/coverage/lcov.info

The cause

See what’s going wrong there? There is a .info file being read by Drupal, thinking it’s a file for a Drupal module, while it actually is a npm module. For some npm modules this doesn’t cause any problems, but sometimes it does like in the example above and then you get vague errors like cache flushes failing and leaving your site in a broken state.

The solution

What’s the solution then? For now: remove the node_modules directory if you are not theming. At Nascom we automatically exclude this folder during builds (just like we exclude folders called .git and .sass-cache). There might be some proper way to tell Drupal 7 to ignore this folder, but I haven’t found it yet.

Edit: For Drupal 8.x there is already an issue to ignore those 2 directories.

Running rsync over ssh on a non-standard port

It seems a lot of people don’t know how to rsync over ssh when the server is running on another port than 22. Here’s the correct command to do that, with the ssh service running on port 9999:

rsync -a -v -e 'ssh -p 9999' user@host:/path/to/files .

Running individual cron commands in Drupal 7

It took me some searching, but it seems this is the best way to run a single cron command, in this case the scheduler cron job to publish and depublish content:

* * * * *  drush --root=$HOME/httpdocs eval "scheduler_cron();"

It’s safe to run this every minute and leave the normal cron job to run every hour:

0 * * * * /usr/bin/flock -w 0 $HOME/.cron.lock drush --quiet --root=$HOME/httpdocs --uri=http://www.example.org cron

In an upcoming blog post I’ll explain why running a full cron every minute is a bad idea.