Infrastructure as code, the first exercise

Disclaimer: This post is a summarization of a learning exercise for the purpose of re-tracing and self-evaluation. It is not a guide or best practice in anyways.

Background and motivation:

Before the start of the exercise, dev.trakr.tech is used as a development/staging environment. However, the utilization of the environment has been very low.

The primary motivation is to only pay for the development environment when we need it vs. paying 24x7x365 infrastructure uptime.

Cost of the server 744 hours — 1 month

Having a development environment that can be quickly provisioned and tossed away will be significantly more cost-effective.

On the surface, it is a trivial exercise to build an on-demand infrastructure to save $10 a month. However, there are many additional benefits that will be explained below.

The existing infrastructure:

The existing server is your pretty standard LAMP stack (Ubuntu 18.04 LTS) that hosts a Drupal instance as the web-app. We also use supervisor for additional process management (communicating with the queue)

The approach:

We selected AWS CloudFormation as our primary tool and infrastructure provider since we already use AWS to host our on-demand backend processes.

Benefits in using AWS CloudFormation and Infrastructure as code in general:

• We can deploy, delete, rinse, and repeat the process to create our development environment.
• No server maintenance needed, updates to packages, etc can be specified from the CFN template itself.
• No need to create elaborate instructions on how to build the server. All of the steps are encapsulated in the template itself.
• The ability to customize server size and other components easily.

See this in action:

CloudFormation Deployment

Once the stack is successfully deployed, you can visit the Drupal instance specified in the Outputs in the AWS console:

CloudFormation stack

Template components:

Our template for the single server LAMP stack consists of 3 main parts:

• Parameters: user credentials such as database username/password, database name, the Drupal app site admin credentials, and other information required by the server.
• Resources: in our extremely simple setup it contains an EC2 instance and a security group to allow for incoming web traffic (80, 443)
• Output: website public DNS for login into the development environment.

Deployment steps:

The deployment (build) process consists of the following:

1- Provisioning the EC2 instance
In the UserData section, we download and install the AWS CFN helper scripts to provide additional capability in the template to install packages and run additional tasks. We also install the supervisor process management tool

EC2 user data

2- Configure CFN helper scripts
We define the configuration for the CFN helper scripts with files. These configuration files take in some builtin-parameters such as the AWS::StackId

CFN helper script configuration file

Once we set the configuration files for the CFN helper scripts, we run a couple of commands to start them up as services:

2- Install the LAMP stack
We can now start to install and configure the LAMP stack with the CFN-helper scripts up and running.

The first part is the equivalent of running apt-get install:

We also prepare configuration files for the LAMP stack such as the apache VHOST configuration below:

Apache VHOST configuration file

We run commands to enable the apache rewrite module (For Drupal clean URL) as well as load our custom VHOST configuration:

Configure apache2

In this section, we can also use sysvinit to start the service for Apache and MySQL

2- Configure the LAMP stack
We still need to create a database to host Drupal and prepare additional dependencies to ultimately install Drupal as the app on the server.

We need to prepare a script to parse the MySQL root password since it is stored in the /etc/mysql/debian.cnf file and use that to create our additional database user and the database. We output the password in the /tmp directory.

<?php
$config_path = '/etc/mysql/debian.cnf';
$output_path = '/tmp/mysqlpass.txt';
$password = null;
$config = file_get_contents($config_path);
$_lines = nl2br($config);
$lines = explode('<br />', $_lines);
foreach ($lines as $line) {
  list($var, $value) = explode('=', $line);
  if(trim($var) == 'password') {
    $password = trim($value);
    break;
    }
  }
if (!empty($password)) {
  file_put_contents($output_path, $password);
}
print('done');

We prepare additional MySQL command to be executed later which will create our database user and the database.

MySQL command

We also prepare a script to install Composer globally which is the preferred dependency manager for PHP and Drupal.

Finally, we issue the commands to configure the MySQL database, install Composer, and install Drush, which will be used to actually install Drupal itself.

LAMP configuration commands

3 - Install Drupal
The Drupal installation step is fairly straight forward, we first prepare a private key to be used to fetch the required repositories:

The rest of it are simply commands to fetch the git repository, perform the code build with “drush make” and install the distribution profile with credentials passed into the template as parameters.

4 - Post-install tasks
We also have scripts that need to be run as processes (daemon) which interacts with the queue service. These are configured after the Drupal installation since they require bootstrapping Drupal.

We prepare our supervisor configuration file here as well as the configuration for our queue processing scripts.

Finally, we issue the command to download the queue processing repository, set up its configuration, and run Supervisor with our configuration.

The Output

We output the public IP address so the user can see it in the CloudFormation stack UI and use this information to log into the website.