LogoLogo
  • Welcome to Release
  • Getting started
    • Quickstart
    • Create an account
    • Prepare to use Release
    • Create an application
      • Create custom application
      • Create from template
      • Servers vs runnables
    • Create an environment
  • Guides and examples
    • Domains and DNS
      • Manage domains
      • DNS and nameservers
        • Configure GoDaddy
        • Configure Cloudflare
        • Configure Namecheap
        • Other DNS hosts
      • Routing traffic
    • Example applications
      • Full stack voting app
      • Flask and RDS counter app
      • Static site with Gatsby
      • Golang with Postgres and Nginx
      • WordPress with MySQL
      • Spring and PostgreSQL
      • Terraform and Flask
      • OpenTelemetry demo
      • Load balancer with hostname
      • Static JavaScript service
      • SSH bastion access to services
      • ngrok and OAuth for private tunnels
      • Using OAuth Proxy
      • Hybrid Docker and static site
      • App Imports: Connecting two applications
      • Example library
    • Running instances
      • Cron jobs
      • Jobs
      • Using Helm charts
      • Using terminal
      • Viewing logs
      • Troubleshooting
        • ImagePullBackoff error
        • CrashLoopBackoff error
        • Exit codes
        • OOM: out of memory
    • Advanced guides
      • Containers guide
      • Application guide
      • Kubernetes guide
      • Create a cluster
      • Upgrade a cluster
      • Managing node groups
      • Patch node groups
      • Hostnames and rules
      • Serve traffic on multiple ports
      • Configure access to your K8s cluster
      • Designing for multiple environments
      • Microservices architecture
      • Monitoring your clusters
      • Performance tuning
      • Visibility and monitoring
      • Working with data
        • Container-based data
        • Seeding and migration
        • Cloud-provided data
        • Golden images
        • Third party
      • Pausing Instant Datasets
        • Application pausing schedules
        • Pause/resume environments
      • Infrastructure as code
        • Terraform
  • Reference documentation
    • Account settings
      • Account info
      • Managing users
      • Build settings
        • Build arguments
        • Build SSH keys
      • Add integrations
      • View clusters and cloud integrations
      • Add datasets
      • Environment handles
    • Workflows in Release
      • Stages of workflows
      • Serial deployments
      • Parallel deployments
      • Rolling deployments
      • Rainbow deployments
    • Networking
      • Network architecture (AWS)
      • Network architecture (GCP)
      • Ingresses
      • IP addresses
      • Cloud-provided services
      • Third-party services
    • Release environment versioning
    • Application settings
      • Application Template
        • Schema definition
      • Default environment variables
      • GitHub
      • Pull requests
      • GitOps
      • Just-in-time file mounts
      • Primary App Link
      • Create application FAQ
      • App-level build arguments
      • Parameters
      • Workspaces
    • End-to-end testing
    • Environment settings
      • Environment configuration
      • Environment variables
        • Environment variable mappings
        • Secrets vaults
        • Using Secrets with GitOps
        • Kubernetes Secrets as environment variables
        • Managing legacy Release Secrets
    • Environment expiration
    • Environment presets
    • Instant datasets on AWS
    • Instant datasets on GCP
    • Instant dataset tasks
      • Tonic Cloud
      • Tonic On-Premise
    • Cloud resources
    • Static service deployment
    • Helm
      • Getting started
      • Version-controlled Helm charts
      • Open-source charts
      • Building Docker images
      • Ingress and networking
      • Configuration
    • GitOps
    • The .release.yaml file
    • Docker Compose conversion support
    • Reference examples
      • Adding and removing services
      • Managing service resources
      • Adding database containers to the Application Template
      • Stock Off-The-Shelf Examples
    • Release API
      • Account Authentication
      • Environments API
        • Create
        • Get
        • Setup
        • Patch
      • User Authentication
      • Environment Presets API
        • Get Environment Preset List
        • Get Environment Preset
        • Put Environment Preset
  • Background concepts
    • How Release works
  • Frequently asked questions
    • Release FAQ
    • AWS FAQ
    • Docker FAQ
    • JavaScript FAQ
  • Integrations
    • Integrations overview
      • Artifactory integration
      • Cloud integrations (AWS)
        • AWS guides
        • Grant access to AWS resources
        • AWS how to increase EIP quota
        • Control your EKS fleet with systems manager
        • Managing STS access
        • AWS Permissions Boundaries
        • Private ECR Repositories
        • Using an Existing AWS VPC
        • Using an Existing EKS Cluster
      • Docker Hub integration
      • LaunchDarkly integration
      • Private registries
      • Slack integration
      • Cloud integrations (GCP)
        • GCP Permissions Boundary
      • Datadog Agent
      • Doppler Secrets Manager
      • AWS Secrets Management
    • Source control integrations
      • GitHub
        • Pull request comments
        • Pull request labels
        • GitHub deployments
        • GitHub statuses
        • Remove GitHub integration
      • Bitbucket
      • GitLab
    • Monitoring and logging add-ons
      • Datadog
      • New Relic
      • ELK (Elasticsearch, Logstash, and Kibana)
  • Release Delivery
    • Create new customer integration
    • Delivery guide
    • Release to customer account access controls
    • Delivery FAQs
  • Release Instant Datasets
    • Introduction
    • Quickstart
    • Security
      • AWS Instant Dataset security
    • FAQ
    • API
  • CLI
    • Getting started
    • Installation
    • Configuration
    • CLI usage example
    • Remote development environments
    • Command reference
      • release accounts
        • release accounts list
        • release accounts select
      • release ai
        • release ai chat
        • release ai config-delete
        • release ai config-init
        • release ai config-select
        • release ai config-upsert
      • release apps
        • release apps list
        • release apps select
      • release auth
        • release auth login
        • release auth logout
      • release builds
        • release builds create
      • release clusters
        • release clusters exec
        • release clusters kubeconfig
        • release clusters shell
      • release datasets
        • release datasets list
        • release datasets refresh
      • release deploys
        • release deploys create
        • release deploys list
      • release development
        • release development logs
        • release development start
      • release environments
        • release environments config-get
        • release environments config-set
        • release environments create
        • release environments delete
        • release environments get
        • release environments list
        • release environments vars-get
      • release gitops
        • release gitops init
        • release gitops validate
      • release instances
        • release instances exec
        • release instances logs
        • release instances terminal
  • Release.ai
    • Release.ai Introduction
    • Getting Started
    • Release.ai Templates
    • Template Configuration Basics
    • Using GPU Resources
    • Custom Workflows
    • Fine Tuning LlamaX
    • Serving Inference
Powered by GitBook
On this page
  • Configuring GitHub Actions for E2E testing with Release
  • Building and uploading your tests as a Docker image
  • Creating the Release environment
  • Running the E2E tests
  • Destroying the Release environment
  • Configuring CircleCI for E2E testing with Release
  • Building and uploading your tests as a Docker image
  • Creating the Release environment
  • Running the E2E tests
  • Destroying the Release environment

Was this helpful?

  1. Reference documentation

End-to-end testing

How to integrate third-party CI platforms with Release to do end-to-end testing

PreviousWorkspacesNextEnvironment settings

Last updated 23 days ago

Was this helpful?

(Skip directly to or .)

If you have end-to-end (E2E) tests set up on a platform like GitHub actions or CircleCI, you can configure these tests to run in an ephemeral Release environment. The flow will be:

  1. Create a new Release environment.

  2. Run the E2E tests in this new environment.

  3. Destroy the environment.

Running E2E tests in an ephemeral environment ensures that they run on a "clean slate", which gives you confidence that tests do not rely on any unknown state that might exist in a long-running environment. Testing in an ephemeral environment also reduces costs because the environment is destroyed as soon as it's no longer needed.

Configuring GitHub Actions for E2E testing with Release

To integrate Release with GitHub Actions and run end-to-end tests, you can structure your GitHub Action main.yml file into four jobs:

  1. Build and upload the image.

  2. Create the Release environment.

  3. Run E2E tests.

  4. Delete the Release environment.

We'll cover an example of how to do each step below. If you prefer, you can find the entire example config file in our .

Building and uploading your tests as a Docker image

Our first job, build-and-upload, will build and upload the image of our application to test or (as in the example below) fetch the images from a registry. Our example has a backend and a frontend as separate images, so we'll fetch each of those and save them to a temporary directory. We use the latest Ubuntu image from GitHub for this.

We also have some standard GitHub Actions boilerplate to define the credentials we need and define the trigger.

name: GitHub Actions E2E Testing Demo

on:
  push:
    branches:
      - demo/github-actions

env:
  RELEASE_ACCOUNT: ${{ secrets.RELEASE_ACCOUNT }}
  RELEASE_APP: ${{ secrets.RELEASE_APP }}
  RELEASE_LOGIN: ${{ secrets.RELEASE_LOGIN }}
  RELEASE_TOKEN: ${{ secrets.RELEASE_TOKEN }}

jobs:
  build_and_upload:
    name: Build and upload docker image
    runs-on: ubuntu-latest
    steps:
      - shell: bash
        run: |
          echo "building docker image..."
          echo "uploading docker image..."
          mkdir tmp
          echo "232490755822.dkr.ecr.us-west-2.amazonaws.com/awesome-release/react-express-mongodb/backend:main" > tmp/backend_image.txt
          echo "232490755822.dkr.ecr.us-west-2.amazonaws.com/awesome-release/react-express-mongodb/frontend:main" > tmp/frontend_image.txt
      - name: upload image name
        uses: actions/upload-artifact@v3
        with:
          name: images
          path: tmp/

Creating the Release environment

The next job we define is create_environment, and this is where most of the Release-specific configuration needs to be. We pull a Docker image that includes the Release CLI and use it to create a new Release environment. We use the Release Account ID and Release Application ID defined earlier, pulled from our GitHub Secrets.

  create_environment:
    name: Create release environment
    needs: build_and_upload
    runs-on: ubuntu-latest
    concurrency: ci-${{ github.ref }}
    container: public.ecr.aws/releasehub-com/release-cli
    steps:
      - name: download artifacts
        uses: actions/download-artifact@v3
        with:
          name: images
      - name: create release environment
        shell: bash
        run: |
          BRANCH=e2e-testing
          FRONTEND_IMAGE=$(cat frontend_image.txt)
          BACKEND_IMAGE=$(cat backend_image.txt)
          FRONTEND=frontend
          BACKEND=backend
          release environments create \
            --branch "$BRANCH" \
            --image-overrides "$FRONTEND=$FRONTEND_IMAGE" \
            --image-overrides "$BACKEND=$BACKEND_IMAGE" \
            --output json \
            --wait > res.json
      - name: upload image name
        uses: actions/upload-artifact@v3
        with:
          name: json
          path: res.json

Running the E2E tests

Now that we have the environment, we can run the tests. We'll call the next job run_e2e_tests. It should look as follows:

  run_e2e_tests:
    name: run e2e tests
    needs: create_environment
    runs-on: ubuntu-latest
    steps:
      - name: checkout repository
        uses: actions/checkout@v3
      - name: download artifacts
        uses: actions/download-artifact@v3
        with:
          name: json
      - name: create release environment
        shell: bash
        run: |
          echo "Install dependencies and run E2E tests..."
          FRONTEND_URL=$(jq -r '.environment.hostnames | .[] | select(.target=="frontend").hostname' res.json)
          BACKEND_URL=$(jq -r '.environment.hostnames | .[] | select(.target=="backend").hostname' res.json)
          jq -n --arg baseUrl "https://$FRONTEND_URL" '{ baseUrl: $baseUrl }' > cypress.json
          jq -n --arg backendUrl "https://$BACKEND_URL" '{ backendUrl: $backendUrl }' > cypress.env.json
          sudo apt-get update
          sudo apt-get install libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libgconf-2-4 libnss3 libxss1 libasound2 libxtst6 xauth xvfb
          npm install
          npm run cy:run

Note that you'll need to change the command to install the dependencies you need and run your own tests in this step.

Destroying the Release environment

Our last job is to clean up the Release environment once the tests have run. We call the job delete_environment and it uses the Release CLI image. The delete_environment job needs access to your Release Account ID, App ID, and the Environment ID that was created in the previous step.

  delete_environment:
    name: Delete release environment
    needs: run_e2e_tests
    runs-on: ubuntu-latest
    container: public.ecr.aws/releasehub-com/release-cli
    steps:
      - name: checkout repository
        uses: actions/checkout@v3
      - name: download artifacts
        uses: actions/download-artifact@v3
        with:
          name: json
      - name: delete release environment
        shell: bash
        run: |
          echo "Delete the Release environment..."
          ENVIRONMENT_ID=$(jq -r '.environment.id' res.json)
          release environments delete "$ENVIRONMENT_ID"

Configuring CircleCI for E2E testing with Release

To integrate Release with CircleCI and run end-to-end tests, you can structure your CircleCI config.yml file into four jobs:

  1. Build and upload the image.

  2. Create the Release environment.

  3. Run E2E tests.

  4. Delete the Release environment.

Building and uploading your tests as a Docker image

Our first job, build-and-upload-image, will build the image of our application to test or (as in the example below), fetch these images from a registry. Our example has a backend and a frontend as separate images, so we'll fetch each of those and save them to a temporary directory. We use a Node image from CircleCI to run this.

We also have some standard CircleCI boilerplate to specify the target version.

version: 2.1

jobs:
  # In this example, we are not actually building and pushing docker image
  build-and-upload-image:
    docker:
      - image: circleci/node:13.8.0
    steps:
      - run:
          name: Build and push docker image
          command: |
            mkdir tmp
            echo "Building docker image..."
            echo "Push docker image..."
            echo "232490755822.dkr.ecr.us-west-2.amazonaws.com/awesome-release/react-express-mongodb/backend:main" > tmp/backend_image.txt
            echo "232490755822.dkr.ecr.us-west-2.amazonaws.com/awesome-release/react-express-mongodb/frontend:main" > tmp/frontend_image.txt
      - persist_to_workspace:
          root: .
          paths:
            - tmp/*

Creating the Release environment

The next job we define is create-release-environment, and this is where most of the Release-specific configuration needs to be. We pull a Docker image that includes the Release CLI and use it to create a new Release Environment. For this to work, you'll need to make sure your Release Account ID and Release Application ID are in the appropriate environment variables.

  create-release-environment:
    docker:
      - image: public.ecr.aws/releasehub-com/release-cli
    steps:
      - attach_workspace:
          at: ./
      - run:
          name: Create a new Release environment
          command: |
            BRANCH=demo/circleci
            FRONTEND_IMAGE=$(cat tmp/frontend_image.txt)
            BACKEND_IMAGE=$(cat tmp/backend_image.txt)
            FRONTEND=frontend
            BACKEND=backend
            release environments create \
              --branch "$BRANCH" \
              --image-overrides "$FRONTEND=$FRONTEND_IMAGE" \
              --image-overrides "$BACKEND=$BACKEND_IMAGE" \
              --output json \
              --wait > tmp/res.json
      - persist_to_workspace:
          root: .
          paths:
            - tmp/*

Running the E2E tests

Now that we have the environment, we can run the tests. We'll call the next job run-e2e-tests. It should look as follows:

 run-e2e-tests:
    docker:
      - image: circleci/node:13.8.0
    steps:
      - checkout
      - attach_workspace:
          at: ./
      - run:
          name: Run E2E Tests
          command: |
            echo "Install dependencies and run E2E tests..."
            FRONTEND_URL=$(jq -r '.environment.hostnames | .[] | select(.target=="frontend").hostname' tmp/res.json)
            BACKEND_URL=$(jq -r '.environment.hostnames | .[] | select(.target=="backend").hostname' tmp/res.json)
            jq -n --arg baseUrl "https://$FRONTEND_URL" '{ baseUrl: $baseUrl }' > cypress.json
            jq -n --arg backendUrl "https://$BACKEND_URL" '{ backendUrl: $backendUrl }' > cypress.env.json
            sudo apt-get update
            sudo apt-get install libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libgconf-2-4 libnss3 libxss1 libasound2 libxtst6 xauth xvfb
            npm install
            npm run cy:run

Note that you'll need to change the command to install the dependencies you need and run your own tests in this step.

Destroying the Release environment

Our last job is to clean up the Release environment once the tests have run. We'll call the job delete-release-environment, and it uses the Release CLI image and needs access to your Release Account ID, App ID, and the Environment ID that was created in the previous step.

  delete-release-environment:
    docker:
      - image: public.ecr.aws/releasehub-com/release-cli
    steps:
      - attach_workspace:
          at: ./
      - run:
          name: Delete Release Environment
          command: |
            ENVIRONMENT_ID=$(jq -r '.environment.id' tmp/res.json)
            release environments delete "$ENVIRONMENT_ID"

Finally, we connect all of the jobs into a workflow as follows:

workflows:
  my_workflow:
    jobs:
      - build-and-upload-image:
          filters:
            branches:
              only: demo/circleci
      - create-release-environment:
          requires: [build-and-upload-image]
          filters:
            branches:
              only: demo/circleci
      - run-e2e-tests:
          requires: [create-release-environment]
          filters:
            branches:
              only: demo/circleci
      - delete-release-environment:
          requires: [run-e2e-tests]
          filters:
            branches:
              only: demo/circleci

We also define a expression to prevent multiple jobs from running at once, in case commits come in close to each other.

We'll cover an example of how to do each step below. If you prefer, you can find the entire example config file as part of our .

concurrency
example React, Express, and MongoDB application on GitHub
example React, Express, and MongoDB application on GitHub
Configure GitHub Actions
Configure CircleCI