Prerequisites Guide

Table of Contents

Overview

This doc is meant to be used by Product QE teams to prepare themselves and their testing repositories to run within OpenShift CI.

There are many things a team needs to prepare for when using OpenShift CI, we'll try to cover everything below.

Please reach out to us on the Slack channel #forum-qe-layered-product to ask questions, brainstorm specific solutions, ..etc.

Prerequisites at a Glance

  1. Resources

  2. Upstream Considerations

  3. GitHub Org Requirements

  4. GitHub User Readiness

  5. Containerized Product Deployment

  6. Containerized Tests

  7. Resolve Internal Dependencies

  8. Valid Test Output

  9. Documented Tests

Resources

  • Identify engineers that are assigned to work on & hold shared responsibility for product onboarding, test execution and ongoing scenario maintenance. See the Scenario Maintenance Policy and this RACI Chart to understand the roles & responsibilities for this onboarding.

Upstream Considerations

Running in OpenShift CI is by default upstream and open to the public. All generated output and artifacts will be made public when run as a Prow job. This means that the following MUST be true about your layered product deployment and test automation:

  • No exposed secrets in the console output of any deployment, config, setup, or test automation.

  • No exposed secrets in an XML generated by a test.

  • No exposed secrets in any artifact generated from product deployment or product testing.

There are two paths for onboarding your layered product scenario into OpenShift CI that you can take

1. Public GitHub repositories (preferred)

  • A GitHub organization that can be used for testing within OpenShift CI.

  • Layered product deployment automation in public GitHub repositories within that org.

  • Layered product testing automation in public GitHub repositories repo within that org.

2. Private GitHub repositories

  • A GitHub organization that can be used for testing within OpenShift CI.

  • Layered product deployment automation in private GitHub repositories within that org (mirrored from your downstream source).

  • Layered product testing automation in private GitHub repositories repo within that org (mirrored from your downstream source).

  • Products prefer to keep all the repositories and images private. GitLab or GitHub repositories are not mirrored.

IMPORTANT:

If you need to use private GitHub repos it is the product QE's responsibility to create the Github org, mirror the downstream repos to Github private repos, and maintain that mirror for the lifetime of the scenario.

GitHub Org Requirements

Invite openshift-ci-robot and openshift-merge-robot to Organization

  1. If you are an owner of the organization, navigate to the organization in GitHub.

  2. Click the "People" header towards the top of the page.

  3. Click the "Invite member" button in the upper right-hand corner.

  4. Search both openshift-ci-robot and openshift-merge-robot and invite them.

  5. The invitations will be accepted within 4 hours via the following jobs:

    • periodic-openshift-release-master-accept-invitations-ci-robot

    • periodic-openshift-release-master-accept-invitations-merge-robot

Add the OpenShift CI GitHub App to the Organization

  1. Navigate to the application: follow this link.

  2. Click the "Configure" button in the upper right-hand corner.

  3. Select the GitHub organization to add the application to.

  4. Ensure "All repositories" is selected.

  5. Click the "Install & Authorize" button towards the bottom of the page.

NOTE: This application is going to replace the openshift-ci-robot and openshift-merge-robot eventually.

NOTE: Documentation from the DPTP (OpenShift CI maintainers) can be found here.

GitHub User Readiness

Containerized Product Deployment

Everything that runs within OpenShift CI runs as a pod on an OpenShift cluster, therefore if you have installation automation that you need to run, you need to first create a container image capable of running your test suite that we can use to create a container within a pod on the OpenShift CI build clusters.

If your product is an operator and does not require any special installation steps, then we can use the existing cli container image and the install-operators ref.

  • The product can be installed via the operator hub (for GA'd layered products).

  • A method to install the operator from the operator hub within a container image.

    • If your product is able to be installed using the steps existing in the operatorhub-subscribe ref than we can make use of that instead.

  • Product install configuration (anything that needs to be done post operator install) executed within a container (maintained by the product QE team in an upstream repo or reachable registry).

  • Identify all environment variables and secrets needed for install.

  • All containerization must also be proven to work within an OpenShift cluster, not just proven in a local or special team based container environment.

See the Container Creation Guide for general guidance when creating your layered product deployment image(s).

Containerized Tests

Again since everything runs as a pod on OpenShift we need a container image for your test suite that can be used to create a container on the OpenShift CI build cluster.

  • Product test environment setup executed within a container image (maintained by product QE team in test repo or reachable registry).

  • A script or command that can be run against the test container to trigger and generate an XML for the tests.

  • Identified all environment variables and secrets needed for testing.

  • All containerization must also be proven to work within an OpenShift cluster, not just proven in a local or special team based container environment.

See the Container Creation Guide for general guidance when creating your scenarios test image(s).

Containerized Product Deployment and Tests

For organizations that do not want to expose their integrated testing to the public, we have a solution. Instead of installing operators and running tests separately, We build test image which includes all the dependencies, push them in a private registry, and then run the tests by starting a pod in a test cluster.

  • The logs reside in a temporary pod which gets destroyed when the test is done. Therefore the logs will not be exposed to the public.

  • The team can decide which information should be included in the output artifacts so that they won’t contain sensitive information to the public. The typical xml file for test results include test name, class name, run time, pass/fail numbers, and error messages for failed test cases.

See the Pod Creation Guide for general guidance when creating pod for your scenarios.

Resolve Internal Dependencies

  • Confirmation that the containerized deployment and testing can be run outside of the company firewall and doesn't rely on internal dependencies.

    • Any dependencies on internal resources will be a blocker.

Valid Test Output

  • Produce JUnit XMLs in a valid format for each test that is executed.

    • This will be needed for reporting results via TestGrid.

    • The string junit_ must prepend the xml filename (i.e. junit_test_results1.xml)

See Report Guide for more information.

Documented Tests

Completion

We define completion of your teams prerequisites as:

All components of the prerequisite guide being complete and most importantly reviewed, verified, and approved by all parties. Once the prerequisites are met we expect that the scenario can be implemented without any blockers. This allows us to plan the implementation appropriately as we attempt to do this for 30+ layered products.

PreviousOnboarding GuideNextKick-off Guide

Last updated