Using Secrets in Workflows

After your secrets have been created, you can reference them when building your workflows in the Fuzzball GUI or within a Fuzzfile to access containers in a private container registry or data within a private S3 bucket.

select

Please select either the GUI or CLI tab to see the appropriate instructions for your environment.

GUI

You can configure a secret to access an S3 bucket by first configuring a volume for your workflow. The image below has configured ephemeral volume test-volume for the workflow. Clicking Add Ingress will bring up a menu where you can configure data ingress for your workflow.

Create a new volume

The image below has set up an ingress configuration which pulls my-input.txt from s3://my-bucket using my-user-s3-secret and stages the file at the root of the ephemeral volume via URI file://my-input.txt. You can save this configuration by clicking OK at the bottom right of the menu.

File ingress configuration

After configuring the ingress for your volume, you can save your volume configuration by clicking save at the bottom right. The images below shows the saved ingress configuration from above.

Save volume configuration

You can configure a secret to access a container in a private registry by first selecting a job’s configuration to edit. In the image below, we have selected job test-job to edit.

Edit test job

You can specify your container from a private registry and secret to access it by navigating to the Environment tab of your job and inputting the URI of the private container you’d like to use, the container name, and tag. You can configure the secret to access this container by using the drop down and selecting the OCI secret you’d like to use. In the image below, we would like to use a container at URI oras://repository.ciq.com/hpc-apps/app-image:tag. In order to access this container, we are using OCI secret ciq-mtn-access-key.

Set OCI secret

CLI

Inside of the YAML workflow specification, secrets are referenced via a URI of the form secret://<scope>/<secret name>. For instance, consider the following for an S3 secret called FBURT_S3_SECRET with user scope:

[snip...]
    ingress:
      - source:
          uri: s3://some-bucket/dir1/compressed-data.tgz
          secret: secret://user/FBURT_S3_SECRET

Of course a secret with the account scope would necessitate the following change in the URI reference.

[snip...]
    ingress:
      - source:
          uri: s3://some-bucket/dir1/compressed-data.tgz
          secret: secret://account/ACCOUNT_REGISTRY_SECRET

For an image registry secret for a user:

[snip...]
    image:
      uri: >-
        oras://us-west1-docker.pkg.dev/ciq-sa/containers/image-name:tag        
      secret: secret://user/FBURT_REGISTRY_SECRET

For an image registry secret for an account:

[snip...]
    image:
      uri: >-
        oras://us-west1-docker.pkg.dev/ciq-sa/containers/image-name:tag        
      secret: secret://account/ACCOUNT_REGISTRY_SECRET

Complete details of secret usage with various storage locations and OCI registries can be found in the Fuzzfile Syntax Guide.