Environments Group - GitLab Quality Assurance End-to-End Testing for the Environments group

Maintained by:

Overview

The goal of this page is to document how the Environments group uses the GitLab QA framework (video walkthrough) to implement and run end-to-end tests.

Supporting slides for the above walkthrough

Why do we have them

End-to-end testing is a strategy used to check whether your application works as expected across the entire software stack and architecture, including the integration of all micro-services and components that are supposed to work together.

The testing strategy for this level of the pyramid can be found under the Environments Test Plan epic.

Where are they

In the GitLab repository, the End-to-End tests for the Environments group are located at:

qa/qa/specs/features/api/7_configure/
qa/qa/specs/features/browser_ui/7_configure/
qa/qa/specs/features/ee/api/7_configure/
qa/qa/specs/features/ee/browser_ui/7_configure/

Limitations

End-to-End tests for Environments are running only in staging and staging-canary environments. It is not possible to run these kind of tests against your local GDK without directly tweaking the test code.

FAQ

I triggered the e2e:package-and-test job. Where do I find the tests?

If you have an MR and want to make sure it is running the End-to-End tests, please trigger the manual e2e:package-and-test job on the pipeline of your MR. After the pipeline runs there will be a note on your MR titled "Allure report" with a test report link. It is also possible to see which jobs failed in the e2e:package-and-test pipeline directly.

Note: Currently, the tests most relevant for Environments run only on staging and staging canary environments.

How can I find tests results for staging or staging canary?

Tests running in staging need access to https://ops.gitlab.net/gitlab-org/quality/staging/. The results of test runs can be found also in the #qa-staging Slack channel. The jobs in the staging pipeline that are relevant to Environments are:

ee-qa-api
qa-browser_ui-7_configure

Which resources tests use and where can they be found?

Environments End-to-end tests use the gitlab-qa-resources project in GCP to spin up GKE clusters. The project belongs and it is maintained by the Quality Engineering department.

What's the GitLab Agent for Kubernetes client version in use?

The version in use can be found on the Runtime Environment Variables file.

Troubleshooting

Please reach out to your counterpart SET or in the #quality channel.