Common Conventions

The following are some hopefully-helpful pointers to common conventions we’re using across most automation projects. Their adoption aims to make automation easily repeatable, runnable, more reliable, and with clearer output. This list is neither exhaustive nor authoritative, but we strive to keep it as up-to-date and relevant as possible.

Code Coverage

Purpose: Particularly relevant for our Python packages (which are used themselves in other testing projects and development repos), quite a few of our projects have implemented Coveralls code-coverage/analysis tool.

For a real-world example of code removal that Coveralls caught, see https://github.com/mozilla/FoxPuppet/pull/162#issuecomment-373110700

Dependencies + Virtual Environments

pyup

Purpose: pyup.io helps us manage updating our dependencies and requirements.

There are a few scenarios for its use:

  • Test suite lives in a separate directory within project repo - Example from Socorro - Example from Bouncer
  • Self-contained test suite - Example from Mozillians

pipenv

Purpose: We use pipenv for managing virtual environments and installing dependencies.

Typically, we:

  1. Pin pipenv with pipenv.txt (Example)
  2. Install directly from the Pipfile (Example)
  3. Ignore generating and using the Pipfile.lock (Example)

tox

Purpose: We use tox to run tests using multiple Python versions, as it automagically takes care of their respective virtual environments.

Docker

Purpose: We use Docker to help ensure consistent, portable builds, with dependency/environment control.

In most cases, we build our Docker image directly from its master branch in GitHub, in any one (or combination) of the following: Travis CI, Circle CI, and Jenkins.

Typically, we use two (2) main files, for Docker:

  1. Dockerfile
  2. .dockerignore

For both of these files, see the official docs at https://docs.docker.com/engine/reference/builder

Jenkins/Jenkinsfile

We prefer tests running in Jenkins to use a Jenkinsfile (and Docker, as much as possible).

Linting

Purpose: PEP8 and flake8. Other considerations for specific linting are:

  1. flake8-isort
  2. flake8-docstrings
  3. pylint

Travis CI

Purpose: We use Travis CI for linting pull requests/commits.