Example 2. Data Ingress and Egress using an Ephemeral Volume

This builds upon the previous example. Instead of using data ingress to pull a single raw file from the GitHub repo, we will download the entire repo (as a .zip file). We will unzip the repo with one job and then append to the README file contents with a second job. Finally, we will use data egress to save the modified file from the ephemeral volume to an AWS s3 bucket.

This example will work for you if you edit the egress destination to point at an s3 bucket or other endpoint where you have write access and modify the secret accordingly.

version: v1
volumes:
  eph_vol_2:
    reference: volume://user/ephemeral
    ingress:
      - source:
          uri: https://github.com/octocat/Hello-World/archive/refs/heads/master.zip
        destination:
          uri: file://Hello-World.zip
    egress:
      - source:
          uri: file://Hello-World-master/README
        destination:
          uri: s3://<your_s3_bucket>/README
          secret: secret://account/<your_s3_secret>
jobs:
  unzip-repo:
    image:
      uri: docker://alpine
    mounts:
      eph_vol_2:
        location: /data
    cwd: /data
    command:
      - /bin/sh
      - '-c'
      - unzip Hello-World.zip
    resource:
      cpu:
        cores: 2
        affinity: NUMA
      memory:
        size: 2GB
  append-to-file:
    requires:
      - unzip-repo
    image:
      uri: docker://alpine
    mounts:
      eph_vol_2:
        location: /data
    command:
      - /bin/sh
      - '-c'
      - echo '(goodbye dear basement)' >>/data/Hello-World-master/README
    resource:
      cpu:
        cores: 2
        affinity: NUMA
      memory:
        size: 2GB

If you add reasonable values to placeholders <your_s3_bucket> and <your_s3_secret> above, you will find a file named README saved on your endpoint with the following content.

Hello World!
(goodbye dear basement)