Common Conventions

The following are some hopefully-helpful pointers to common conventions we’re using across most automation projects. Their adoption aims to make automation easily repeatable, runnable, more reliable, and with clearer output. This list is neither exhaustive nor authoritative, but we strive to keep it as up-to-date and relevant as possible.

Code Coverage

Purpose: Particularly relevant for our Python packages (which are used themselves in other testing projects and development repos), quite a few of our projects have implemented Coveralls code-coverage/analysis tool.

For a real-world example of code removal that Coveralls caught, see

Dependencies + Virtual Environments


Purpose: helps us manage updating our dependencies and requirements.

There are a few scenarios for its use:

  • Test suite lives in a separate directory within project repo - Example from Socorro - Example from Bouncer
  • Self-contained test suite - Example from Mozillians


Purpose: We use pipenv for managing virtual environments and installing dependencies.

Typically, we:

  1. Pin pipenv with pipenv.txt (Example)
  2. Install directly from the Pipfile (Example)
  3. Ignore generating and using the Pipfile.lock (Example)


Purpose: We use tox to run tests using multiple Python versions, as it automagically takes care of their respective virtual environments.


Purpose: We use Docker to help ensure consistent, portable builds, with dependency/environment control.

In most cases, we build our Docker image directly from its master branch in GitHub, in any one (or combination) of the following: Travis CI, Circle CI, and Jenkins.

Typically, we use two (2) main files, for Docker:

  1. Dockerfile
  2. .dockerignore

For both of these files, see the official docs at


We prefer tests running in Jenkins to use a Jenkinsfile (and Docker, as much as possible).


Purpose: PEP8 and flake8. Other considerations for specific linting are:

  1. flake8-isort
  2. flake8-docstrings
  3. pylint

Travis CI

Purpose: We use Travis CI for linting pull requests/commits.