End-to-End testing with gitlab-ci

May 21, 2022

End-to-End tests require the whole application to be running. This includes frontend, backend, database and possible other services. Then you can use an automated browser or other client, to test the UI.

If you want to run end-to-end tests in a CI-pipeline, you have two choices: Start the whole application as part of the job, or deploy the application to a review-environment in Kubernetes, AWS or any other cloud provider.

For Gachou, I run on a low budget, so I don’t have a Kubernetes cluster. So I have no choice but to start the application as part of the job

Gitlab-CI service containers

Gitlab-CI has the services-feature, which allows you to start additional containers, and I already used this feature to start a database for the backend tests. But e2e-testing is more complicated.

cypress running the test-runner in a cypress/base docker-image
web-ui and backend running the containers that are built by the Gachou repository pipelines.
database running the database, used by the backend

service connections in ci-job

For a long time, I thought that only the main-container was allowed to talk to services. But the backend also has to communicate with the database. I tried this often in the past, and never succeeded.

Feature flag: Network per build

I just found out that Gitlab-CI has a feature flag called FF_NETWORK_PER_BUILD, which creates a dedicated docker-network for the build-job. This allows the different services to be interconnected. Just specify an environment variable FF_NETWORK_PER_BUILD: 1 for the ci-job.

All environment variables to configure the different services just go into the variables section.

Boom. That’s it.

Example:

I now use the following .gitlab-ci.yml for my e2e-build

stages:
  - test

e2e:
  stage: test
  image: docker.io/cypress/base:16.14.2
  services:
    - alias: postgres
      name: docker.io/bitnami/postgresql:14
    - alias: backend
      name: $CI_REGISTRY/gachou/gachou-backend:preview
    - alias: web-ui
      name: $CI_REGISTRY/gachou/gachou-web-ui:preview
  variables:
    ## Feature flag to allow backend to talk to database
    FF_NETWORK_PER_BUILD: 1
    ## Postgres configuration
    POSTGRESQL_DATABASE: gachou
    POSTGRESQL_USERNAME: gachou
    POSTGRESQL_PASSWORD: gachou-dev-pw
    # Backend configuration
    QUARKUS_DATASOURCE_JDBC_URL: jdbc:postgresql://postgres:5432/gachou
    QUARKUS_HTTP_CORS_ORIGINS: "http://web-ui"
    GACHOU_DANGEROUS_TEST_SETUP_ACCESS_TOKEN: e2e-access-token
    # Web-ui configuration
    GACHOU_WEB_UI_CONFIG: '{ "apiBaseUrl": "http://backend:8080" }'
    # Cypress configuration
    API_BASE_URL: "http://backend:8080"
    CYPRESS_BASE_URL: "http://web-ui/"
  before_script:
    # Install dependencies
    - yarn
  script:
    - yarn cypress:run - e2e-test/.yarn/cache
  artifacts:
    when: on_failure
    paths:
      - e2e-test/cypress/screenshots
    expire_in: 1 day

This version is stripped down a little, to demonstrate the “services” part. In the real file, there is also a cache configuration for cypress and yarn, and a rules: property to prevent running the job too often

Gitlab-CI service containers

Feature flag: Network per build

Example:

Related links