Step Registry - Ref Guide

What is a Ref?

A ref is the most basic step in the step registry, it can be thought of as the building-blocks of other steps. A ref is essentially a BASH script that be used on its own or as part of a chain or workflow. A ref is comprised of five files:

Configuration file: A YAML file used to define the ref's required resources, environment variables, secrets, container, etc.
OWNERS file: A file used by OpenShift GitHub robots as a list of users who are allowed to approve or deny changes to a step.
BASH script: The BASH script that is executed in the ref.
README.md (CSPI requirement): A documentation file written in Markdown. While not required by OpenShift CI, this is a CSPI requirement.
OpenShift CI Metadata file: An autogenerated file used by OpenShift CI internally.

How to Create a Ref

Please follow the steps below to create a ref in the step registry:

Preparation

In the openshift/release repository, create a folder for your ref under ci-operator/step-registry/ if a suitable folder does not exist already.
- Example: ci-operator/step-registry/new-step
Create the following files in your new folder.
- new-step-ref.yaml
- new-step-commands.sh
- OWNERS
- README.md

IMPORTANT:
When creating the .yaml and .sh files in step 2 above, please keep in mind that they must follow a naming standard or OpenShift CI will not accept them. You'll notice the folder I created is named "new-step", both the .yaml and .sh files need to start with that name and end with -ref.yaml and -commands.sh, respectively.
If you create the new folder for your ref in an existing folder under ci-operator/step-registry/, you must include the name of the parent folder in the name as well. For example, if you created the new-step folder under ci-operator/step-registry/existing-folder, the names of your files would be existing-folder-new-step-ref.yaml and existing-folder-new-step-commands.sh.

Populating the Files

`new-step-ref.yaml`

This file is meant to serve as the configuration file for the new ref being created. This file outlines things like:

Resource requirements.
Container the script should be run in.
Required environment variables.
Required secrets.
Some documentation for the autogenerated documentation in OpenShift CI.

Please use the mock-up and guide below to populate your ref's configuration file:

ref:
  as: new-step
  from: some-container
  commands: new-step-commands.sh
  resources:
    requests:
      cpu: '1'
      memory: 3Gi
    limits:
        cpu: `2`
        memory: 6Gi
  credentials:
    - namespace: test-credentials
      name: some-test-credentials
      mount_path: /tmp/secrets/credentials
  env:
  - name: SOME_VARIABLE
    default: "This is the default for SOME_VARIABLE"
    documentation: |-
      A really cool variable used for really cool things.
  documentation: |-
    A sample ref that does something

The ref configuration above is fairly simple. For our purposes (so far) this is about as complicated as a ref should be, barring some additional credentials (secrets) or environment variables.

Here is how we define and get each of the values in the configuration above:

as: This is the name of the ref. You'll notice it is just the name of the configuration file sans the -ref.yaml ending. This is also how you will call this ref to be executed in chains and workflows.
from: This value is the name of the container you'd like this script to be executed in. You can use a custom container or one of the available containers within the OpenShift CI image registry (cli seems to be a popular image to use).
commands: The value for commands simply points to the -commands.sh file that we will populate in the next section. This is telling the ref which bash script to run.
resources: This stanza defines the resources that are needed to run your new ref.
- requests: This stanza defines the optimal resources needed to run this ref.
  - cpu: The number of CPUs requested.
  - memory: The amount of memory requested.
- limits: This stanza defines the maximum resources this ref should be allowed to utilize. Optional
  - cpu: The number of CPUs requested.
  - memory: The amount of memory requested.
credentials: This stanza defines a list of any credentials or secrets the ref may need for execution. Please see the Secrets Guide for more information regarding the usage of this stanza. Optional
- namespace: Defines the credential's namespace defined in Vault.
- name: Defines the name of the credential in Vault.
- mount_path: Defines where the file(s) containing the credentials should be mounted during execution. If you have more than one item in this list, the mount_path values cannot be the same.
env: This stanza defines a list of environment variables needed for this ref to execute. Optional
- name: Defines the name of an environment variable.
- default: Defines the default value of the environment variable if one is not provided. Optional if env: stanza is used
- documentation: A short description of the environment variable and what it is used for.
documentation: The value of this should be a short description of the ref to be used in the autogenerated documentation in OpenShift CI.

`new-step-commands.sh`

This file is just the BASH script that will run during this ref's execution within the container you specified in the from: key of the ref's configuration file.

IMPORTANT:
Just because this file is written in BASH, does not mean the tests or actions you are preforming have to be done in BASH. This file can be as simple as executing a Python script within the container. That being said, the -commands file does need to be a BASH script.

Because this file is a BASH script, this document will not be diving in to the nuances of BASH. It will, however, give you some ideas of what can be accomplished in this script. Here are some examples:

Utilizing the secrets defined in the credentials stanza of the ref's configuration file:

#!/bin/bash

set -o nounset
set -o errexit
set -o pipefail

# Each key/value pair in a Vault secret is given a file in the mount_path defined. 
# This files name is the key value and the contents is a single clear-text line containing the value.
# See the Secrets Guide for more information.

# This line is using the cat command to set the contents of a credential file to a variable.
CREDENTIAL=$(cat /tmp/secrets/credentials/password)

Utilizing the environment variables defined in the env: stanza of the ref's configuration file:

#!/bin/bash

set -o nounset
set -o errexit
set -o pipefail

# The environment variables defined in the env stanza of the config will be set using the "name" key/value pair.
# We can use these variables directly in our BASH script or in a different script (Python, Go, etc.) that we execute in the BASH script.
echo $SOME_VARIABLE

Execute Pytest:

#!/bin/bash

set -o nounset
set -o errexit
set -o pipefail

# If the container you define in the from key/value pair of the configuration document has
# Pytest installed, you could execute test cases in this script and output the results to the SHARED_DIR
pytest /tmp/tests/test_some_test_cases.py -vv --junitxml=${SHARED_DIR}/junit_output.xml

`OWNERS`

See the official Kubernetes documentation.

`README.md`

Follow the instructions in the Step Registry Documentation Policy document.

Finalization

Once you have populated all the files for your ref, make sure to run the make update command in the root of your local clone of the openshift/release repository. This command will create all the necessary metadata files that OpenShift CI uses to execute your new ref. This command will also alert you to any errors in your configuration file and tell you how to fix them.

NOTE:
If running this command returns a permissions error, please try to run it using sudo.

PreviousStep Registry NextStep Registry - Chain Guide

Last updated 2 months ago

OpenShift CI Onboarding