Automating Front-End Performance Tests

Setting up the Cloudomation Workspace

With Cloudomation we use Python scripts aka Flows to help us do redundant tasks. Provisioning a Google Cloud Instance for our Webpagetest platform is quite easy. Of course, we have to register for Google Cloud, create a new project and enable billing for it.

In our Cloudomation workspace, under Connections, we store our service account credentials to have access to Google Cloud services. It has to be in yaml format and must follow this structure:

key:
    type: 'service_account'
    auth_uri: 'https://accounts.google.com/o/oauth2/auth'
    client_id: '<your client id>'
    token_uri: 'https://oauth2.googleapis.com/token'
    project_id: '<your Google Cloud project id>'
    private_key: |
        <copy the contents of the private key here>
    client_email: '<your serviceaccount email address>'
    private_key_id: '<your private key id>'
    client_x509_cert_url: '<your cert url for the project>'
    auth_provider_x509_cert_url: 'https://www.googleapis.com/oauth2/v1/certs'

We call the Connection 'gcloud', to remember it easily.
Next, we have to fetch the webpagetest scripts from the Cloudomation Library into our Cloudomation workspace. It's easy, we only have to go to the Admin section and add a new Git Configuration by clicking on the New button on top.
The Remote URL should point to https://github.com/starflows/library.git
The Path mapping should contain:

flow:
    - '**/*.py'
setting:
    - '**/*.yaml'
    - '**/*.yml'

Make sure to enable the Git Configuration by checking Enabled – shortly after, you should see the scripts and files in the Flows and in the Files section of your workspace.

The Flows used to automate the tests are:

• webpagetest.provision
• webpagetest.create.server
• webpagetest.frontend.test
• webpagetest.remove.server

Under Files you should see:

• webpagetest-agent-Dockerfile
• webpagetest-agent-script.sh
• webpagetest-server.json
• webpagetest-server-Dockerfile
• webpagetest-server-locations.ini
• webpagetest-server-template.json
• webpagetest-server-template-startup.sh

We do not need to touch them, we just make sure they are there.

Provision the Webpagetest Platform and Run Tests

There are only 2 things left to do before we can see how our webpage does performance-wise:

At first, we need a Flow to start the provisioning process and the actual test, and then we have to have a small test configuration, to tell the platform which websites we actually want to test.

The Flow script has the following content:

import flow_api
def handler(system: flow_api.System, this: flow_api.Execution):
    this.flow('webpagetest.provision', gcloud_connection='gcloud')
    this.flow('webpagetest.frontend.test', gcloud_connection='gcloud')
    return this.success('all done')

Under the Flows section, create a new Flow, copy-paste this short script above into the code editor and save it with the name start webpagetest.

Please note that the provision is only necessary once. If you do tests regularly (which we encourage), then the line with webpagetest.provision can be commented out.

The only thing left now is to set up a configuration in the Settings section of the Cloudomation workspace. Go to Settings, make a new one and name it webpagetest. The contents of the Setting can contain a set of urls, at least one, and have to have test parameters. (See the full set of parameters on the Webpagetest website). The following configuration should get you started:

url_list:
    - url: 'https://www.pexels.com/search/cat/'
    - url: 'https://europa.eu/european-union/index_en'
    - url: 'https://www.nytimes.com/'
test_params:
    f: json
    runs: '3'
    video: '0'
    private: '1'
    location: 'Test:Firefox'
    priority: '6'

We save this in the newly created Setting webpagetest and we are good to go.

Let's start our Flow start webpagetest from earlier. We go to the Flows section, select the Flow, then hit Run.

Now, we can lean back and watch how our server comes up, gets configured and performs the tests for us. This will take some time. In our configuration we said runs: '3'. Depending on how fast the front-ends render and how long the Webpagetest server needs to compile the results, it can take a couple of minutes. We can see the progress in the Executions section in the Cloudomation workspace. Select Filter Active to see what is going on at the moment.

When the tests are finished, the results can be found in the Files section. We look for webpagetest.result When we open the file, we see a json with the filtered, most important metrics.

If we do another test, we have to remember to make sure that our start webpagetest looks like this.:

import flow_api
def handler(system: flow_api.System, this: flow_api.Execution):
    this.flow('webpagetest.frontend.test', gcloud_connection='gcloud')
    return this.success('all done')

We already have a server provisioned, and a snapshot of it was saved in Google Cloud. When starting a new test, the snapshot will be used and therefore our server is ready much faster.

Bottom Line

Automating performance tests for website with Cloudomation:

• Is set up in minutes.
• Includes provisioning of webpagetest environment with two servers.
• Infrastructure is provisioned on demand and removed after use.
• Can be configured flexibly to test one or many websites and webapps.
• Can be configured to test a range of different metrics.
• Can be scheduled to run regularly.
• Can be extended and modified at will, e.g. to run on AWS or other cloud providers.