> This is part 1 of 3 of the full documentation (pages 1–100 of 285).
> The content is paginated: fetch every part to see all of it.
> Next part: https://docs.release.com/llms-full.txt/1
> Page index: https://docs.release.com/llms.txt

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
This site has an agentic ask interface you may use to query the documentation dynamically by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.release.com/index.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

---

# Welcome to Release

Release simplifies development by making the creation of environments easy

## What is Release?

Release makes it easy to manage environments so your team can **focus on building value for your customers**. Release can build environments in our cloud or yours, from the simplest applications (static JavaScript) to the most complex (microservices with many cloud-native dependencies).

* **Eliminate time spent managing environments**

With Release, all of the management and tooling required to build a flexible environment ecosystem comes out of the box. You just connect your repositories, review your environment template and environment variables, and let Release do the rest.

* **Allow for a common, consistent way to define and maintain applications and environments**

Release supports **production and pre-production environments** and every step is completely automated – from code push to environment creation. We track your source changes, then build new preview environments automatically for every feature, freeing you to focus on the things that matter.

* **Allow development teams a simple, easy way to reproduce environments for any reason**

You can create an **infinite** number of development, staging, demo, and production environments without ever having to deal with the underlying infrastructure. These environments can then be automatically generated and updated whenever you push a code change or create a PR. They can also be manually deployed via the Release user interface.

Release enables:

* Parallel environment infrastructure that removes software delivery bottlenecks.
* Easy microservice development.
* Integration of environments into developer workflows for a delightful developer experience.
* DevOps teams to focus on high value work rather than building the same tooling over and over again.

In addition, organizations can create **permanent staging and production environments** and automate all of the steps, from code push to running in production.

A great way to get started with Release is to create ephemeral environments on every pull request to preview changes with stakeholders. Once you're using Release, you'll realize quickly there are many other use cases, including:

* Easy local microservice development using cloud-based environments with microservice dependencies.
* An alternative to your permanent staging environment infrastructure.
* Ephemeral environments to test and preview changes with every pull request.
* Scalable environments for performance testing.
* Environments for developers to experiment with new infrastructure.
* Sales demo environments.
* On-demand environments for large migrations and testing.
* Production environments based on Kubernetes.

When environments are simple to create, your teams will be enabled to move quickly without worrying about environments... ever.

{% hint style="success" %}
Find out more about Release:

* [Request a demo of Release to get started](https://release.com/book-a-demo)
* [Send questions directly to the team](mailto:hello@release.com)
  {% endhint %}

### Static sites

Release supports standalone **static sites** running without Docker backends. These sites are pushed into a CDN and hosted on your behalf. [See Static service deployment](/reference-documentation/static-service-deployment) for more information.

### Docker & Docker Compose

Release builds services running in Docker containers derived from your `docker-compose.yaml` file. Your application can have any combination of one or more services. For more information about creating an application that builds via Docker and Docker Compose, please see these useful reference guides from Docker:

* [Docker overview](https://docs.docker.com/engine/docker-overview/)
* [Getting started with Docker](https://docs.docker.com/get-started/)
* [Building with Dockerfile](https://docs.docker.com/engine/reference/builder/)
* [Overview of Docker Compose](https://docs.docker.com/compose/)

In order to run Docker-based applications in Release, you need a working Docker build and a `docker-compose.yml` file that will run all the services you need.

### Hybrid sites (static sites + Docker containers)

Release supports applications with both **static frontends** and **other Dockerized services**. Static applications are defined in a `package.json` file, while your other services are defined through a `docker-compose.yaml` file. When both these files are present in a repo, Release will automatically detect them and provide you with options to build these services correctly. **Release manages all facets of creating environments and connecting frontends and backends together seamlessly.**

### Connecting to your AWS account

Release supports running environments in your **AWS account** through the creation of a Kubernetes cluster using AWS CloudFormation templates. This cluster will be fully and remotely managed by Release, meaning you don't need to know anything about Kubernetes to use Release.

To integrate your cloud provider with Release, you will need to go to **Account Settings > Integrations** where you will be walked through a simple set up procedure to connect your AWS account to Release.

*Release currently supports AWS and GCP. Full support for other clouds will soon be available.*

## What you need to use Release

### Frontend JavaScript applications

1. A `package.json` file containing a `build` command in the `scripts`.

### Container-based applications

1. A working Docker build.
2. A definition of your application in a `docker-compose.yml`.
3. Your application's source code checked into a VCS such as GitHub or Bitbucket.
4. For OAuth-based VCS integrations (like Bitbucket), you'll need a user who has admin permissions on the repository to install webhooks and deploy keys. For GitHub, we use GitHub Apps and the app authorization handles the management of the repository.
5. Admin access to the AWS account where you'd like your environments to run.

## How Release works

Release environments are built from an Application Template using default environment variables.

The Application Template defines the following high-level concepts:

* The Kubernetes cluster your application is running in.
* The domain where your application will be running.
* Templates used to create ephemeral and permanent environments.
* Hostnames where your application will be publicly accessible.
* Resources allocated to your services.
* Application services
* Application workflows
* Application jobs, such as running migration scripts.

Release can automatically generate these templates, or you can write them yourself from a library of examples for common frameworks. As a starting point, we recommend letting Release analyze your repository and automatically generate templates.

We currently support the following mechanisms to create your application templates and environment variables defaults:

* Write your own templates from a library of examples for common frameworks and services.
* Automatically generate templates by parsing the `docker-compose.yaml` file found in your repository.
* Automatically generate templates by parsing `package.json` files for JavaScript applications.
* Generate environment definitions from your Kubernetes YAML files.
* Generate templates from Helm charts.

Release has been designed with minimal configuration in mind, but every application is unique and the application template allows developers to customize their application to suit their needs.

Developers generally interact with Release by creating a pull request in their VCS (version control system), which automatically creates an environment with the code changes in the PR.

### Webhooks

Webhooks are calls made from your VCS back to Release when certain events happen. This tells Release there is something of interest to pay attention to in your repository, and kicks off automated workflows in Release. Release listens to webhooks for the following actions:

* New pull request.
* Closed pull requests.
* Reopened pull requests.
* Merged pull requests.
* Code pushed to repository.

The user account that creates an organization's Release account must have admin privileges for the application's repository. See [GitHub permissions](/integrations/source-control-integrations/github) and [Bitbucket permissions](/integrations/source-control-integrations/bitbucket) for more information on setting up a repository to allow the proper webhooks for Release.

### Where do applications and environments run?

Applications and environments created with a Release *Starter Account* are hosted in the Release cloud. If you have a *Professional Account* or higher, applications run on a Kubernetes cluster in your AWS account (optional) and static sites are hosted by CloudFront. Release remotely manages the cluster and CDN so your teams can focus on building your applications.

## Comparing ephemeral and permanent environments

**Ephemeral environments** are environments that live for the duration of a pull request or are created manually. These environments will automatically track the branch from your pull request. An ephemeral environment is given a unique URL for each service. This allows developers to share work-in-progress development with stakeholders easily. Ephemeral environments are designed to be close replicas of production environments.

**Permanent environments** are environments that are not destroyed. Permanent environments are typically for more traditional staging or production workloads. These environments get static URLs that are based on the `domain` field in your Application Template. They can be set to track a branch and deployed automatically on a code push to your source control system, or they can be deployed manually.

In general, ephemeral environments and permanent environments are basically the same. However, there are some slight differences between the two that might affect which one you choose to use. These are a few of the differences between ephemeral and permanent environments when using Release:

| Ephemeral environments                                                                                                                                                                                                                                                                                                                    | Permanent environments                                                                                                                                                                                                                                                                                                                    |
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Can be set to expire automatically, can be renewed during push event or manually [Environment expiration](/reference-documentation/environment-expiration)                                                                                                                                                                                | Never expire                                                                                                                                                                                                                                                                                                                              |
| Can be deleted easily                                                                                                                                                                                                                                                                                                                     | Require special checks to delete (contact support to find out how to delete them)                                                                                                                                                                                                                                                         |
| Can be automatically created by pull requests                                                                                                                                                                                                                                                                                             | Must be created manually                                                                                                                                                                                                                                                                                                                  |
| Are located lower down in the "Environment Details" page                                                                                                                                                                                                                                                                                  | Are always listed at the top of the "Environment Details" page                                                                                                                                                                                                                                                                            |
| Cannot use a custom name and cannot be renamed                                                                                                                                                                                                                                                                                            | Must specify a custom name (contact support to rename environments)                                                                                                                                                                                                                                                                       |
| Does not support rainbow deployments                                                                                                                                                                                                                                                                                                      | Supports rainbow deployments                                                                                                                                                                                                                                                                                                              |
| Covered only by normal support SLAs                                                                                                                                                                                                                                                                                                       | Can be included in enhanced support SLAs (contact sales or customer success for more details)                                                                                                                                                                                                                                             |
| Can specify overrides in the Application Template for ephemeral environments ([adding/changing services and default resources](https://docs.releasehub.com/reference-guide/application-settings/application-template/schema-definition#environment-templates)) – this is the same for both, but there is a specific section for each type | Can specify overrides in the Application Template for permanent environments ([adding/changing services and default resources](https://docs.releasehub.com/reference-guide/application-settings/application-template/schema-definition#environment-templates)) – this is the same for both, but there is a specific section for each type |
| Typically have fewer resources and extra container services for databases, Redis, etc.                                                                                                                                                                                                                                                    | Can be set with more resources and usually require more customisations from the default, usually exclude container services in favor of permanent infrastructure (like databases, Redis, queues, buckets, etc.)                                                                                                                           |
| Are created each time from the latest Application Template major version [Release environment versioning](/reference-documentation/release-environment-versioning)                                                                                                                                                                        | Do not change the major version from which they were created and are not updated when the Application Template changes                                                                                                                                                                                                                    |


# Quickstart

Here's a step-by-step guide to adopting Release in your organization. Start with the Release Cloud Sandbox

1. Create a Release account.
2. Play around with the Release Sandbox
3. Choose a Starter Repo or Custom Application

Release Enterprise Quick Start

1. Integrate your cloud provider.
2. Link a domain.
3. Create a cluster.
4. Connect your source control provider.
5. Create your application.

## Step 1: Create a Release account

Visit <https://release.com/signup> and sign up using your email address or a single sign-on provider.

Take a look at our [Create an Account](/getting-started/create-an-account) documentation if you need more information about this step.

## Step 2: Play around with the free Release Cloud Sandbox

When you first signup for Release you are in the Release Cloud and using a Sandbox account which includes 5 free hours of environment time. This is to help you get a feel for the features in Release.

## Step 3: Try a Starter Repo or Custom Application

<figure><img src="https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-ce182a56fa96c4e75362cec12fb17882ec39afe0%2FScreenshot%202025-05-18%20at%209.44.15%E2%80%AFAM.png?alt=media" alt=""><figcaption></figcaption></figure>

There are two ways to try Release. If you just want to see how Release works, start with one of our Starter Repos. The Starter Repos have been setup ahead of time to work out of the box with Release. You just fork the repo and follow the guided setup. This generally takes a few minutes and is the easiest way to get familiar with Release. You can also start with a Custom Application. When you go down this path, you bring code from your own code repositories (Github, Bitbucket or Gitlab). If you just want to see the features in Release, start with a Starter Repo. If you want to bring up your own applications start with Custom Application. We recommend starting with a Starter Repo. You can always bring your own code at anytime once you are familiar with Release.

## Step 4: Upgrade to Release Cloud or Enterprise

Once your 5 free hours has expired you can upgrade to our paid Release Cloud plan which runs on our infrastructure in the Release Cloud or upgrade to Enterprise which allows you to run your environments in your cloud account.

<figure><img src="https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-7ad9bc113d503807636001c1a73106d624aa7a21%2FScreenshot%202025-05-18%20at%2010.00.28%E2%80%AFAM.png?alt=media" alt=""><figcaption></figcaption></figure>

## Release Enterprise Quick Start

If you're on the Release Enterprise paid plan, integrate Release with your cloud provider.

In Account Settings, navigate to **Clusters**, and press **Create new cloud provider integration**. Currently, we support AWS and GCP.

![Create new cloud provider integration](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-204870004be82480616b4f525aa5bc74f389c114%2Fcloud-provider-integration.png?alt=media)

When prompted to log in to your cloud provider, you should use an account with enough privileges to create and destroy clusters, manage permissions, manage DNS settings, and other provider-specific functionality that Release will need to function.

Take a look at the following docs for more help with integrating your cloud provider:

* [AWS Integration](/integrations/integrations-overview/aws-integration)
* [GCP Integration](/integrations/integrations-overview/gcp-integration)

## Link your domain

If you don't already have a domain, you should register one through your cloud provider (for example, Route53 if you're using AWS).

You can also use a domain registered through a third-party provider such as GoDaddy (follow our [external DNS](/guides-and-examples/domains-and-dns/external-dns) guide), but it's easier if you use your cloud provider directly.

To link your domain to Release, go to **Settings** -> **Domains** -> **Create new domain**.

Enter your domain and take note of the nameservers that Release will display. If you're using a third-party domain provider, configure your domain to point to these name servers.

Finally, verify your domain by pressing the **Verify** button.

## Create a cluster

Once you have a linked and verified domain, you can create your first cluster. Navigate to **Settings** -> **Clusters** and choose **Create new cluster** under your cloud provider.

![Create new cluster](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-765c9a418f0963dfc48d660c12576b2e444feb77%2Fonboarding-create-cluster.png?alt=media)

For more help with this, see [Create a cluster](/guides-and-examples/advanced-guides/create-a-cluster).

## Connect your source control provider

Release will pull code from your source control provider (such as GitHub, GitLab, or Bitbucket) to deploy your application.

You may have integrated a source provider account when you signed up for Release using SSO, but you can further [configure your source control provider](/integrations/source-control-integrations) to give Release access to the correct repositories and edit other settings, like whether Release automatically comments on your pull requests.

To get started, click on your profile picture in Release, then on **Manage Profile**.

You can connect GitHub, Bitbucket, or GitLab.

![Source Control Settings](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-05e1545a9c570bf395fcfc79cf9cd134b59a7eca%2Fsource-control-settings.png?alt=media)

If you're connecting GitHub, you'll be given the option to grant Release access to all of your repositories or only specific repositories.

Once you've created the connection, you'll be able to view your repositories when creating a new app from Release and select the one that you want to deploy using Release.

## Create and deploy an application

To test that everything works, you can deploy one of our [ready-to-go example applications](/guides-and-examples/common-setup-examples).

Select an example application and fork it to your integrated source control account. Then press **Create new app** from your Release dashboard and choose the repository you forked.

![New Application](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-b601eb092949d08ca11eca56957845c15c101e5a%2Fapplication-new.png?alt=media)

![Create new app](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-65b424a632ea22b6a6551feaec327530ea3d1349%2Fcreate-new-app-step-1.png?alt=media)

![Analyze Repository](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-0fd0e4f9bf6b820f623f615ab980a850fa1dc7e5%2Fanalyse-step-2.png?alt=media)

![Generate Template](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-329a04f55e9e7686f87279e5662927e9fa8e39d4%2Fgenerate-template.png?alt=media)

## Where to next

Now you can configure and monitor your applications through your Release dashboard. If necessary, you can still access your AWS account and Kubernetes cluster directly, but for most use cases, you can enjoy our simplified controls and sensible defaults.

To get the most out of Release, you should:

* Prepare your applications for use with Release. See [Prepare to use Release](/getting-started/prepare-to-use-release).
* Understand the [Application Template file](/reference-documentation/application-settings/application-template) that lets you configure most aspects of Release.
* Read our [Application guide](/guides-and-examples/advanced-guides/application) to understand the details of how Release works.


# Create an account

You can create a free Release account by using your GitHub, GitLab, or Bitbucket account. If you don't already have an account on one of those services, sign up for a free account on any of them and then return to these steps.

* Visit <https://release.com/signup> in your browser.

![Account signup](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-3e55aef94aa40da5f7b36a99698a7d426c870a7c%2FScreenshot%202025-05-18%20at%209.33.10%E2%80%AFAM.png?alt=media)

* Create an account or signup with your company's Enterprise SSO.
* Enter the name of your company, this will be used as your main Workspace name.

<figure><img src="https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-15b0ed8766c57f4081b468e4a125350b7b2546e3%2FScreenshot%202025-05-18%20at%209.39.57%E2%80%AFAM.png?alt=media" alt=""><figcaption></figcaption></figure>

* You're now in a Release sandbox account on the Release Cloud plan. All environments are hosted in the Release Cloud and while on the free Sandbox trial you have access to resources for 5 environment hours to get a feel for how Release works.
* There are two ways to try Release. If you just want to see how Release works, start with one of our Starter Repos. The Starter Repos have been setup ahead of time to work out of the box with Release. You just fork the repo and follow the guided setup. This generally takes a few minutes and is the easiest way to get familiar with Release. You can also start with a Custom Application. When you go down this path, you bring code from your own code repositories (Github, Bitbucket or Gitlab). If you just want to see the features in Release, start with a Starter Repo. If you want to bring up your own applications start with Custom Application. We recommend starting with a Starter Repo. You can always bring your own code at anytime once you are familiar with Release.

<figure><img src="https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-ce182a56fa96c4e75362cec12fb17882ec39afe0%2FScreenshot%202025-05-18%20at%209.44.15%E2%80%AFAM.png?alt=media" alt=""><figcaption></figcaption></figure>

Your Release account is now up and running and you can log in at any time by visiting <https://web.release.com/login> and signing in.


# Prepare to use Release

Take a look at [what you need to use Release](/#what-you-need-to-use-release) to be sure you meet the requirements.

Release supports building and running complex applications. Release requires a `docker-compose.yml`, `package.json`, or `.release.yaml` file to be in the root of your application's Git repository. Release uses these files to generate the [Application Template](/reference-documentation/application-settings/application-template) that is used as a blueprint to build your environments.

If your application has nested directories where your `docker-compose.yaml` or `package.json` files reside, you can add a `.release.yaml` file to the root of your repository to tell Release where these files are located. See [The `.release.yaml` file](/reference-documentation/release.yaml) reference guide for more information.

We recommend using a [monorepo](https://en.wikipedia.org/wiki/Monorepo) and a directory-per-service at the project root when your application requires two or more services. Here is an example directory structure for three services that require docker builds:

```
├── README.md
├── docker-compose.yml
├── result
│   └── Dockerfile
├── vote
│   └── Dockerfile
└── worker
    └── Dockerfile
```

If you're not using a monorepo, you can create an application for each repository and connect them using environment variables in each of those applications. See [Microservices architecture](/guides-and-examples/advanced-guides/microservices-architecture) for more information.

## Testing Docker Compose for Release

The best way to test if your application will run in Release is to get **Docker Compose** to run your application locally. See [Overview of Docker Compose](https://docs.docker.com/compose/) for more details.

**Release allows you to quickly spin up containerized services without the traditional wait times involved with cloud-native services.**

When running an application locally or in a Release **ephemeral environment**, we recommend that you isolate your application's external dependencies. Consider using *off-the-shelf* Docker images to represent the cloud-native services that your application depends on, such as Postgres, Redis, etc. Visit [Docker Hub](https://hub.docker.com) to find many off-the-shelf Docker images.

You can use environment variables to connect **ephemeral environments** to AWS-native services such as RDS, however, we recommend using these off-the-shelf Docker images to speed up deployment and simplify set up.

This is an example of a `docker-compose.yml` file that builds three Docker images and leverages two off-the-shelf containers for **Redis** and **Postgres**:

```yaml
version: "3"
services:
  vote:
    build: ./vote
    command: python app.py
    volumes:
      - ./vote:/app
    ports:
      - "5000:80"
    depends_on:
      - "worker"

  result:
    build: ./result
    command: nodemon server.js
    volumes:
      - ./result:/app
    ports:
      - "5001:80"
    depends_on:
      - "worker"

  worker:
    build:
      context: ./worker
    depends_on:
      - "redis"
      - "db"

  redis:
    image: redis:alpine
    container_name: redis

  db:
    image: postgres:9.4
    container_name: db
    volumes:
      - "db-data:/var/lib/postgresql/data"

volumes:
  db-data:
```

Release detects many popular services such as **Redis**, **MySQL**, **Postgres**, **RabbitMQ**, and **many more**, and will handle opening standard ports for these services.

Here's an example of adding Redis, RabbitMQ, and Memcached with minimal configuration in Docker Compose:

```yaml
version: "3"
services:
  redis:
    image: "redis:5-alpine"

  rabbitmq:
    image: "rabbitmq:3.7-alpine"

  memcached:
    image: "memcached:1.5-alpine"
```

### Building your application

Release supports most options from [`docker-compose build`](https://docs.docker.com/compose/compose-file/#build). Release looks for all services with a `build` stanza, and executes a **docker build** for each service found.

#### Specify the current working directory:

```yaml
version: "3"
services:
  app:
    build: .
```

#### Specify a different directory to build:

```yaml
version: "3"
services:
  vote:
    build: ./vote
```

#### Specify the Dockerfile and pass in [build arguments](/reference-documentation/account-settings/build-settings/build-args):

```yaml
version: "3"
services:
  mysql:
    build:
      context: .
      dockerfile: Dockerfile-mysql
      args:
        S3_BUCKET: "data.example.com"
        MYSQL_VERSION: "5.7"
```

### Release ports and DNS hostnames

Release uses the [`docker-compose` ports](https://docs.docker.com/compose/compose-file/#ports) (`HOST:CONTAINER`) definition to determine when a service should be exposed to the internet. When Release encounters a service with a `HOST:CONTAINER` definition, it creates a **NodePort**, **load balancer rule**, and **DNS hostname** entry for the service.

```yaml
version: "3"
services:
  frontend:
    build:
      context: .
      dockerfile: Dockerfile-frontend
    ports:
      - "8080:8080"
    command: ["npm", "run", "dev-server-docker"]
```

In this example, Release creates a hostname for the frontend service that exposes port 8080.

**Internal services**, like your **data stores**, **caches**, and **background workers**, should not be exposed to the internet, and should only be available internally to other services defined in your `docker-compose`.

```yaml
version: "3"
services:
  db:
    image: postgres:9.4
    ports:
      - "5432"
```

This example shows **Postgres** opening a container port on **5432** that another service can consume.

### Environment variables

Most applications use **environment variables** to define how services talk to each other and/or change the behavior of an application in a given environment. **Release uses the environment variables provided in your `docker-compose` as a starting point.** All of the environment variables for each service defined in your `docker-compose` are loaded into Release when you initially create an application. Release supports all of the standard `docker-compose` mechanisms for defining environment variables.

You can use [`env` file](https://docs.docker.com/compose/env-file/) to specify an external `env` file:

```yaml
version: "3"
services:
  api:
    build:
      context: ./api
      dockerfile: Dockerfile.django
    ports:
      - "8000:8000"
    env_file: .env
```

You can also set environment variables directly in your `docker-compose` file:

```yaml
version: "3"
services:
  nginx:
    build: .
    ports:
      - "5000:5000"
    environment:
      - AWS_REGION=us-west-2
      - AWS_SECRET_ACCESS_KEY=XXX
      - RESOLVER=8.8.8.8
```

If you have dynamic or secret environment variables, you can read more about [environment-specific environment variables](https://github.com/releasehub-com/docs/blob/main/reference-guide/environment-settings/environment-specific-environment-variables/README.md) in our reference guide.


# Create an application

Learn how to create an application in Release

You can create unlimited applications on Release.

{% hint style="info" %}
**Note:** Free accounts are limited to two environments at a time. If you're on a free account and already have two environments, you'll need to delete one before proceeding with this guide. To create more than two environments, [upgrade your account](https://releasehub.com/pricing).
{% endhint %}

To create an application, first log in to your Release account.

On the bottom-right card, click the **create new app** button.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-33db2a409071288e8018aaf672ca82a033eac15e%2Fcreate-app.png?alt=media)

1. **Create a custom application:** If your application doesn't fit into one of the provided templates or if you prefer to start from scratch, you can create a custom application. This option gives you the most flexibility, but requires you to configure your application manually.

   When creating an application in Release, you have two options:
2. **Create from a template:** Release provides a variety of pre-configured templates for common types of applications. These templates include the necessary configuration and code to get started quickly.

   \## Servers and runnables

There are two types of applications you can create in Release:

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-31fc77c242205e239fcd07cbaaef0a5dd6921564%2Frunnable-and-server.png?alt=media)

1. **Servers:** Servers are applications that provide services over the network. They are continuously running and waiting for requests to process. Examples include web servers, database servers, and application servers. You can also create multiple environments for a given server application, for example, a development environment and a production environment.
2. **Runnables:** Runnables are applications that perform a specific task, run to completion, and then terminate. Examples include scripts, batch jobs, and fine-tuning AI models. You can only create one environment for a given runnable application.

   <div data-gb-custom-block data-tag="hint" data-style="info" class="hint hint-info"><p><strong>Learn more:</strong> To learn more about the differences between servers and runnables, read <a href="/getting-started/create-application/servers-vs-runnables">Servers vs. Runnables</a>.</p></div>

When creating an application, you'll need to specify the type of application you're creating. This helps Release understand how to handle and manage your application.


# Create custom application

Learn how to create a custom application in Release

This guide provides step-by-step instructions for creating a custom application in Release.

## Step 1: Create an application

![The create new custom app button in Release](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-a3e82944e3641d0195a5fba28d8c7b1c3315985a%2Fcreate-app.png?alt=media)

Log in to your Release account and navigate to **Applications**. Click **Create Application** and select **Create Custom** from the dropdown to start the setup process.

Provide a name for your application. This name will be used in hostnames and for identification purposes in Release.

![Select a repo](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-0600edec0170763a9406b562b29f9d89eb42d1b9%2Frepo-selection.png?alt=media)

Next, select the repository and branch that contains your application code. Release will use this repository to analyze and create the initial setup for the application. It’s important to pick the correct branch that holds the latest version of your code.

![Server or runnable](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-5e561f1f211083711248a95ba7fc59830e8541c6%2Fserver-or-runnable.png?alt=media)

Select the execution type for the application. If the application needs to run continuously, choose **Server**. If it only runs on demand, completing tasks as required, select **Runnable**. [Learn more about servers and runnables](https://github.com/releasehub-com/docs/blob/main/getting-started/create-application/servers-vs-runnables.ms).

## Step 2: Analyze the repository

![Choose the files for Release to analyze](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-b77e14cbd71bffd13d4611e0279e52b3099e6f29%2Fanalyze-repo.png?alt=media)

Release can analyze your repository to detect services based on Docker Compose files, or `package.json` for static/node services, or a `.release.yaml` file for an existing Application Template.

Select the branch of the repository and the specific files you want Release to analyze. For example, you might choose the `master` branch and select files like `Dockerfile` and `docker-compose.yml`. These files provide Release with information about your application’s services and dependencies.

Click `Start Analysis` to have Release scan your repository.

Release will now read the files you selected and create a list of services found. Each service has a checkbox, a list of dependencies, and a service type.

![You can now select the services needed and see their dependencies](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-e131551076b242ba3606ac45bfa1badcbba44e6c%2Frepo-analyzed.png?alt=media)

1. Select or deselect a service by clicking the checkbox next to the service's name.
2. Select the service type. Release tries to automatically select the correct service type for each service. This field will only be editable when the service type is ambiguous.
3. Under the `dependsOn` field, you'll see a list of services that a given service depends on. Release tries to deploy services in parallel to save time, but if a service depends on other services, dependencies will be deployed before this service is deployed.
4. Once you've selected your services, click Next step.

## Step 3: Generate a template

![Generate template](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-f9cf31b504f60136466776543800768f9eaffa2f%2Fgenerate-template.png?alt=media)

After analyzing your repository, you’ll need to generate an [Application](https://docs.release.com/reference-documentation/application-settings/application-template) template. Release will automatically create a template based on the services and dependencies identified in your repository. You can customize this template to match your application’s requirements.

Alternatively, you can create a custom template from scratch, or use one of Release’s preset templates. For instance, if you’re working with a Node.js application, choose the Node.js preset template.

Customize the default settings for resources such as CPU, memory, and replicas based on the application's requirements to ensure it runs efficiently. For example, you might allocate 1000m CPU and 1Gi of memory.

You can also define any workflows needed for your application here. Workflows may include setup tasks, deployment processes, or other automation steps.

If you're done editing your Application Template, click the Next step button.

## Step 4: Set environment variables

Environment variables provide information the application needs, like database connection strings, API keys, and other configuration settings.

Release will try to import your default environment variables from your Docker Compose file (if selected previously), and interpolate values from a .env file (if present in your repository).

![Configure the environment variables for the application](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-9ebedacb6e1f3b4eac0c3b200e709735e013e2ee%2Fenv-vars.png?alt=media)

For example, you might add the following environment variables:

* `DB_HOST`: `mysql.example.com`
* `DB_USER`: `root`
* `DB_PASS`: `password123`

Edit the environment variables specification and click Next step to proceed.

{% hint style="info" %}
Learn more about [environment variables](https://docs.release.com/reference-documentation/application-settings/default-environment-variables) in Release.
{% endhint %}

\## Step 5: Set build arguments

Build arguments are passed to the Release build process. Typically, these would be arguments used in the Docker build process.

![Add Build Args](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-de1e318e765136beb0876033d74ab3445d86ed4a%2Fbuild-args.png?alt=media)

To add a new build argument:

* Click on **Add build arg**.
* Enter a **Key**.
* Enter a **Value**.

Repeat these steps for all build arguments you need to add, then click **Next Step**.

## Step 6: Save and deploy

Now you can save and deploy the application.

![Save App template](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-ad46e790a7e0d0eeffe3fe37180dbe7fcf5576e8%2Fsave-and-finish.png?alt=media)

You have the opportunity to go back to any of the preceding steps to edit your application.

You can return to any of the preceding steps to edit the application. When you're ready, click on the **Finish Setup** button.

Release will now save your new application template.

## Create an environment

You can now create an environment for your application. Click on the **Create Environment** button to create an environment for your application.

Go to the [Create an Environment](/getting-started/create-an-environment) guide to learn more on how to create an environment for your application.

![Create an enviroment](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-b504b54d493fd9a8c701fb17bf41a357c061373e%2Fcreate-environment.png?alt=media)

From the environment detail page, you can click on the hosted URLs for this environment to use your application.

## What's next? Push an update to your repository to see how Release builds and redeploys your application

If you push updates to your repository, Release will pull your new code, build your application, and redeploy your environment.

## Learn by doing: Example applications

You can follow along with one of our many [example apps](https://github.com/releasehub-com/docs/blob/main/getting-started/getting-started/common-setup-examples/README.md) or spin up one of the boilerplate apps found in our [example library](https://github.com/releasehub-com/docs/blob/main/getting-started/examples/example-library.md).

{% hint style="info" %}
**Note:** Release creates an initial ephemeral environment from the Create Application process. After your application is created, you can manually [create new environments](/getting-started/create-an-environment) (including permanent environments) from the Environments page.
{% endhint %}


# Create from template

Learn how to create applications from templates in Release

Release simplifies the process of creating and deploying applications by using preset templates. These templates are pre-configured environments tailored to specific use cases, allowing you to get your applications up and running quickly without needing to handle the intricate details of setup and configuration.

Release templates are designed to handle the configurations required for various environments and optimized for specific tasks like running Jupyter Notebooks, deploying language models, or fine-tuning machine learning models. This guide provides step-by-step instructions for creating an application from a template in Release.

## Step 1: Create an application

Log in to your Release account and navigate to **Applications**. Click **Create Application** and select **Create from Template** from the dropdown to start the setup process.

![Create App](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-4efd4d949391a653f0a589dfcf589305a3de933c%2Fcreate-app-template.png?alt=media)

## Step 2: Choose your execution type

Release offers two types of templates: Server and Runnable. The type you choose depends on the nature of your application.

![Select Server or Runnable](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-c2fee58f7cf4f4c4e9647d6d1f25442de0af5dbf%2Fserver-or-runnable.png?alt=media)

* **Server:** Server templates are meant for applications that need a persistent server environment. For instance, running a Jupyter Notebook server with GPU acceleration requires a stable, long-running server. Learn more about [server applications](/getting-started/create-application/servers-vs-runnables).
* **Runnable:** Runnable templates are for applications that can be executed on demand. This is ideal for tasks like fine-tuning models, where the application does not need to run continuously. Learn more about [runnable applications](/getting-started/create-application/servers-vs-runnables).

## Step 3: Select a template

Choose the template that best suits your application requirements. Release offers a range of templates optimized for specific tasks. To learn more about the available templates, check out the [template library](/getting-started/create-application/create-from-template).

![Select a server template](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-9c128810669b64a16dfc874842ae68dcb61632f0%2Fselect-template.png?alt=media)

The stack icons in the top-right corner of each template card represent the core technologies and integrations included in the template.

**Server Templates:**

* **Jupyter:** For GPU-accelerated Jupyter Notebooks.
* **Ollama:** For large language models.
* **Ollama-WebUI:** A feature-rich, self-hosted WebUI.
* **GenAI:** For building GenAI applications.
* **Anything-LLM:** Other language model-based applications.

Alternatively, you can choose a runnable template for your application by selecting the **Runnable** tickbox.

![Select a Runnable Template](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-4adcb3af92d7df794a24201b501467bd91d883a1%2Fselect-template-runnable.png?alt=media)

**Runnable Templates:**

* **NeMo:** For fine-tuning models with NeMo.
* **Pytorch:** A container with PyTorch and NVIDIA GPU drivers.
* **Axolotl:** Another option for fine-tuning models.

To learn more about runnables, check out the [runnable documentation](https://docs.releaseapp.io/getting-started/create-application/create-runnable).

## Step 4: Configure your application

Release will automatically generate a name for your application based on the template you choose. You can change this name to something more descriptive if you wish.

![Configure your Application Template](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-b6c5600c0cdaf1436711ae96953d3709ded902a5%2Fchange-configuration.png?alt=media)

Edit the template configuration settings to suit your specific requirements. These settings include the application’s name, a description, and the resources allocated to it. You can also choose the environment in which the application will run, such as a CPU or GPU environment.

To learn more about configuring your application, check out the [configuration documentation for templates](/getting-started/create-application/create-from-template).

## Step 5: Finish and create environment

Once you have configured your application, click **Finish** to save your app template. You can now create an environment for your application by going to the application’s page and clicking **Create Environment**.

{% hint style="info" %}
Learn more about [creating environments](/getting-started/create-an-environment) in Release.
{% endhint %}


# Servers vs runnables

Learn the difference between servers and runnables

When setting up applications in Release, it's essential to understand the differences between servers and runnables to choose the right execution type for your needs. Here's a detailed explanation of both.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-31fc77c242205e239fcd07cbaaef0a5dd6921564%2Frunnable-and-server.png?alt=media)

## Servers

Servers in Release are designed to run continuously, providing a persistent environment that ensures your application is always available. This makes them ideal for applications that require a stable, long-term setup, such as web servers, databases, or services that need to be operational 24/7.

The primary advantage of using a server is its reliability and stability. Because servers are always running, they can handle tasks that require constant uptime and immediate responsiveness. This is particularly useful for hosting websites, APIs, or backend services where downtime could impact users or business operations.

Another key feature of servers is their ability to maintain state between restarts. This means they can preserve data, sessions, and other critical information, which is essential for applications that need to retain user data or track ongoing transactions.

Scalability is also a strong suit of servers. They can be scaled to handle increased load, ensuring consistent performance as demand grows. This makes them a suitable choice for applications expecting high traffic or usage spikes.

## Runnables

Runnables are designed for on-demand execution. They are perfect for tasks that do not require a continuous run but are triggered to perform specific operations as needed. When a runnable is called, it starts, completes its task, and then shuts down, making it highly efficient for short-lived processes.

The primary benefit of using runnables is resource efficiency. Since runnables only run when needed, they consume fewer resources compared to servers that run continuously. This can lead to cost savings and more effective resource management.

Runnables are typically stateless, meaning they don’t retain information between executions. This makes them ideal for tasks like batch processing, data transformation, or executing microservices that perform specific, isolated functions. Because they don’t need to preserve state, runnables can start up quickly, perform their designated task, and then shut down, ensuring prompt completion of tasks.

In addition to their efficiency, runnables are well-suited for scheduled tasks or automated workflows such as data processing jobs or CI/CD pipelines. This makes them a versatile option for various automated processes that need to run at specific intervals or in response to certain triggers.

## Choosing the execution type

If your application needs to be available continuously, handle persistent data, or serve real-time user requests, a server is the best choice. If your tasks are intermittent and can benefit from resource efficiency, runnables are the better option.

|                    | Servers                                                 | Runnables                                  |
| ------------------ | ------------------------------------------------------- | ------------------------------------------ |
| **Execution Type** | Continuous, always running                              | On-demand, runs as needed                  |
| **Use Case**       | Web servers, persistent services, LLM hosting           | Batch jobs, microservices, LLM fine tuning |
| **Resource Usage** | Higher resource consumption due to continuous operation | Resource efficient, runs only when needed  |
| **Scalability**    | Can be scaled to handle increased load                  | Efficient for short-lived tasks            |
| **Management**     | Suitable for applications needing constant availability | Ideal for scheduled or triggered tasks     |


# Create an environment

When you [create a new Release application](https://github.com/releasehub-com/docs/blob/main/getting-started/broken-reference/README.md), Release automatically creates your first ephemeral environment. To view an application's environments or to create new environments, navigate to the application's **Environments** tab.

![Environments page](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-a9386e467dd0fa4217cb974d1a943a1400e74cd6%2Fcreate-an-environment-environments-page.png?alt=media)

{% hint style="info" %}
Free accounts are limited to two environments at a time. If you're on a free account and already have two environments, you'll need to delete one before proceeding with this guide. To create more than two environments, [upgrade your account](https://releasehub.com/pricing).
{% endhint %}

## How to create a new environment

On the Environments page, click the **Create environment** button. This takes you to the create environment page where you can configure your new environment.

![New environment settings](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-fa34a4867a19a50fd78b881e1dab9da98649c31a%2Fcreate-an-environment-environment-settings.png?alt=media)

Let's take a look at each of these options:

1. Select an **environment type**: Choose whether the new environment should be *ephemeral* (lasting for a limited time – seven days by default) or *permanent* (lasting until manually destroyed). To learn more about ephemeral and permanent environments, see our page about [environment expiration](/reference-documentation/environment-expiration). If you're creating a permanent environment, enter an **environment name**. Release will convert this name to lowercase and remove spaces. Ephemeral environments get an autogenerated name.
2. Select a **branch** from your application's repository to deploy to your new environment.
3. Select if you want the environment to deploy automatically.
4. Select the **cluster** for this environment where this environment will live.
5. Select any other applications that you want to **import** into this environment.
6. Under **Override Build Images**, you can enter specific Docker images to deploy to your services. By selecting a specific Docker image for a service, the image for this service will not be updated when you push new code to your branch. By default, Release will pull the latest image for each service.

Lastly, click **Create environment**.

Release will now create your new environment and redirect you to a page that shows you deployment progress in real time.

## What happens after you create an environment

Release goes through four stages for each new environment.

### 1. Build

During the build stage, Release pulls a branch from your application's repository, then runs `docker build` for each service. After building new Docker images, Release pushes the images to a private container registry.

### 2. Setup

Next, Release sets up your new environment by creating the required nodes and other resources, setting quotas, pulling Docker images, and deploying containers, and starting services with `docker run`.

### 3. Diagnostics

Release then sets up a health checker to monitor your deployed services.

### 4. Traffic routing

Finally, Release routes network traffic to your new environment by updating a routing table for your application.

## How to use your new environment

Your environment should now be deployed and ready to use.

To view the environment's details page, visit your application's Environments page and click on the new environment you created.

From the Environment Details page, you can view URLs for your services, change settings for your environment, and see details about running instances.

{% hint style="info" %}
**Test it out:** To see how Release builds and redeploys your application, push a code update to your repository in the branch tracked by this environment, then visit the **Deployments** tab in Release.
{% endhint %}


# Domains and DNS

To use a custom domain for your Release environments, follow our [manage your domains](/guides-and-examples/domains-and-dns/managing-your-domains) guide.

For an overview of how Release routes traffic to your environments, see our documentation about [routing traffic](/guides-and-examples/domains-and-dns/routing-traffic).


# Manage domains

## View domains

Navigate to **Configuration** and click the **Domains** tab to view all domains associated with your cloud integrations. This page will show you the domain name, the cloud integration it is associated with, name servers, and the status of the domain.

![View and create domains](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-023c3116be02abe82cfe024b50543eb11d7c24d5%2Fview-and-create.png?alt=media)

## Create a domain

Click the purple **Create Domain** button.

In the dialog, select the cloud integration the domain will be associated with and type in the domain name that will be used in clusters for your apps in Release.

![New domain dialog](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-fb43aecdeab21200ff6910b8834e4ff01e8f198d%2Fcreate-form.png?alt=media)

Click **Create Domain**. This will create a DNS record in the chosen cloud integration and return the name servers associated to it.

![Domain details page](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-065e2bd5d86029eea0c08cafa515f7740973901a%2Fdomain-example-org.png?alt=media)

Now you can use the given name servers to add NS and CNAME records to your DNS manager for the top-level domain you have used. In this example, the top-level domain is `example.org`. You can add an NS record to point to the given name servers and a CNAME wildcard record that points to `releasedomain.example.org`. Read [our External DNS guide](/guides-and-examples/domains-and-dns/external-dns) for more information on setting up your external DNS.

## Verify a domain

Once you have updated the DNS records in your host, return to the Domain Info page and click the **Verify** button under the **Actions** dropdown to compare the name servers on Release to those associated with the domain.

![Verify domain](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-38346888a269caed843977cb0a4629c6ec22c838%2Fverify.png?alt=media)

In this example, Release will check to see which name servers are associated with `releasedomain.example.org` using DNS lookups. If the name servers match, the domain will show as verified on the Domain Info page. Verified domains can be used later to create a cluster.

## Remove a domain

To remove a domain, go to the Domains page and click the domain to see the Domain Info page.

![Domains page](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-de0034153ea15e678a1d49989494fda83f777b33%2Fdelete.png?alt=media)

Click the **Delete** button under the **Actions** dropdown to show the delete confirmation dialog. The **Delete** button will be disabled if this domain is currently in use by any running clusters.

{% hint style="danger" %}
You may only remove domains which are not in use with any active clusters. You can tell if any clusters are using a domain by navigating to the Domain Info page looking at the "In Use By" section.
{% endhint %}

![Domain delete confirmation](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-5fe65618fef576f8def75c429dddde66dbd2ced7%2Fdelete-confirm.png?alt=media)

Type in the word `DELETE` and click the **Delete** button. This will remove the domain from Release and attempt to remove any DNS entries created for it.

{% hint style="danger" %}
If the domain was part of an existing DNS entry created outside of Release, it will not be removed from the cloud provider. You may have to manually remove any entries which Release did not explicitly create in those situations.
{% endhint %}


# DNS and nameservers

How to set up external DNS services to work with Release

## Select a domain to use with Release

Before you create a domain in Release, take a moment to decide whether you'll use an entire domain (in other words, all subdomains on a domain) for Release, or a single subdomain of your domain.

### When to use an entire domain name

If you do not yet have a domain name, we recommend registering a new domain through your cloud provider.

Using a domain name dedicated to Release enables launching your live environments on Release and is recommended for applications that will use permanent environments, not only ephemeral environments.

For example, you could launch a permanent environment on `www.example.com`.

### When to use a single subdomain

If you already use an external DNS server for your domain or your domain registrar does not support setting custom nameservers at domain-level (such as Cloudflare Registrar), we recommend using a subdomain for your Release applications.

This way, you can delegate DNS for the subdomain only to Release, while keeping full control over the rest of your domain with your current provider.

For example, if you choose to use `release.example.com` for your Release environments, you still have the option to use `www.example.com` and `api.example.com` for services outside Release.

Your Release environments will have domains that are one level down from your chosen subdomain, for instance, `api-23r4wm-appname.release.example.com`.

## How to get nameservers from Release

When you [create a domain](/guides-and-examples/domains-and-dns/managing-your-domains#create-a-domain) in Release, we automatically create a DNS zone for your domain at your cloud provider. We use Route53 on AWS or Cloud DNS on GCP.

After creating the DNS zone, Release displays the nameservers your domain or subdomain should use.

## How to configure your domain or subdomain to point to Release

Take note of the nameservers provided by Release, then follow one of the guides below to point your domain or subdomain to the Release nameservers:

* [Configure GoDaddy](/guides-and-examples/domains-and-dns/external-dns/godaddy)
* [Configure Namecheap](/guides-and-examples/domains-and-dns/external-dns/namecheap)
* [Configure Cloudflare](/guides-and-examples/domains-and-dns/external-dns/cloudflare) (subdomain only)
* [Other DNS hosts](/guides-and-examples/domains-and-dns/external-dns/generic-dns) (subdomain only)

## Be prepared to wait for propagation

Domain changes sometimes take up to a day or two to propagate. This is due to how DNS resolvers cache records. To check the status of your domain's DNS propagation, use a tool such as [DNS Propagation Checker](https://www.whatsmydns.net/) or [DNSMap](https://dnsmap.io/).

## Verify your domain in Release

Once the DNS change has propagated to most of the internet, you can [verify your domain](/guides-and-examples/domains-and-dns/managing-your-domains#verify-a-domain) in Release and start using it for new environments.


# Configure GoDaddy

If you bought your domain name through GoDaddy, you can still add it to Release.

## In Release, add your domain

To add your domain to Release, follow the steps to [create a domain](/guides-and-examples/domains-and-dns/external-dns). Take note of the nameservers displayed by Release, as you'll use these in the next steps.

## Configuring DNS in GoDaddy

1. Log into your GoDaddy control panel at [godaddy.com](https://godaddy.com)
2. Navigate to **My Products** and then select **DNS** on the domain that you want to use.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-9d3311ab2f00da82e63a81000a7ceae438500bb5%2Fgodaddy-my-products-dns.png?alt=media)

1. Scroll down to the "Nameservers" section and press **Change**.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-94754313fd355c1fd9aee6a8b0b9128a6f2fffa3%2Fgodaddy-change-nameserver.png?alt=media)

1. Click the "Enter my own nameservers (advanced)" link.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-2ad431f87afe73f8a7bd47e58ea0b4c38bb0cfd9%2Fgodaddy-enter-own-nameserver.png?alt=media)

1. Add all the nameservers that you see in your Release control panel after adding the domain there. You'll need to press "Add Nameserver" link to create additional fields.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-a5d5903ac391ed89379f20cc7d5f771b2c214c1e%2Fgodaddy-add-more-nameservers.png?alt=media)

1. Confirm that you want to move your Nameserver management away from GoDaddy.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-1dad9b5d18e79ddd9f4d918cc66d42072033f46a%2Fgodaddy-confirm.png?alt=media)


# Configure Cloudflare

If you registered your domain name through Cloudflare, or your domain points to Cloudflare's nameservers, you can delegate a subdomain to Release's nameservers.

Cloudflare does not currently support external nameservers for domains registered through Cloudflare Registrar, so this guide is limited to using a subdomain on Release only.

## Add your subdomain to Release

To add a subdomain to your Release account, follow the steps to [create a domain](/guides-and-examples/domains-and-dns/external-dns). Take note of the nameservers displayed by Release, as you'll use these in the next step.

## Configuring DNS in Cloudflare

To set up DNS delegation for a subdomain in Cloudflare, log in to your Cloudflare dashboard and select the domain you want to use.

Then follow these steps:

1. Click on **DNS** in the sidebar.
2. Click on **Add record**.
3. Select "NS" from the "Type" dropdown.
4. Enter the subdomain in the "Name" field.
5. Enter a nameserver hostname in the "Nameserver" field.
6. Click **Save**.
7. Repeat steps 2 through 6 for each nameserver.

![Configure Cloudflare DNS](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-7ef0b033a9d78213f171b9ac1e8bc4929e487763%2Fexternal-dns-cloudflare-subdomain.png?alt=media\&token=a9d7eb01-4606-478b-8c05-051d3831e477)

After creating a DNS record for each nameserver, the Cloudflare DNS records should look like this:

![After adding subdomain NS records](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-c73ebfb821c58efdca124ea0e254aa59c283c296%2Fexternal-dns-cloudflare-dns.png?alt=media)

You can find more information and detailed instructions for setting up subdomain delegation in Cloudflare over at Cloudflare's [Delegating Subdomains Outside of Cloudflare](https://support.cloudflare.com/hc/en-us/articles/360021357131) page.


# Configure Namecheap

If you bought your domain name through Namecheap, you can still use your domain with Release.

## In Release, add your domain

To add your domain to Release, follow the steps to [create a domain](/guides-and-examples/domains-and-dns/external-dns). Take note of the nameservers displayed by Release, as you'll use these in the next steps.

## In Namecheap, update your nameservers

Log in to your Namecheap account, click on **Domain List** in the sidebar, then click on **Manage** next to the domain you'd like to update.

![Manage your domain in Namecheap](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-80eb633bdd7a3144ca59c25d2b3d2f162af45bd6%2Fnamecheap-manage-domain.png?alt=media\&token=99162da2-2b80-41b9-89ed-1f7b30d52c54)

1. Select **Custom DNS** from the **Nameservers** dropdown.
2. Add all the nameservers you see in your Release control panel after adding the domain there. Press the "Add Nameserver" link to create additional fields.
3. Click the check mark icon to save your changes.

![Set custom nameservers in Namecheap](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-2329953ff8aa80be112292e92718cc51b12f8905%2Fnamecheap-custom-dns.png?alt=media\&token=d85aa92a-7c11-4919-b2c0-11877fde8336)


# Other DNS hosts

If you use an external DNS server for your domain, you can still use a subdomain for your Release environments.

## Add your subdomain to Release

To add a subdomain to Release, follow the steps to [create a domain](/guides-and-examples/domains-and-dns/external-dns). Take note of the nameservers displayed by Release, as you'll use these in the next step.

## Configuring your existing DNS server

1. Log in to your DNS admin panel.
2. Navigate to the DNS records.
3. Add a new subdomain NS record for each nameserver you received from Release, for example:

```dns-zone-file
release.your-company-name.xyz. IN NS ns-821.awsdns-38.net.
release.your-company-name.xyz. IN NS ns-2002.awsdns-58.co.uk.
release.your-company-name.xyz. IN NS ns-37.awsdns-04.com.
release.your-company-name.xyz. IN NS ns-1189.awsdns-20.org.
```

4. Save your changes.


# Routing traffic

### How Release routes traffic to your environments

When you create environments on Release, we generate hostnames that are [instantly available](#instant-domain-updates), [unique](#how-releasehub-creates-unique-hostnames), and [predictable](#creating-predictable-hostnames-using-environment-handles).

### Instant domain updates

Traditional DNS is too slow for the fast-paced traffic routing required when creating ephemeral environments.

To avoid having to wait for DNS propagation every time you launch a new environment, Release uses a proprietary Lambda\@Edge routing infrastructure that makes domain updates instant.

Each incoming request to CloudFront runs a Lambda\@Edge function to check the request URL against a routing table stored in DynamoDB so that Release can instantly route and reroute traffic to your environments.

For dynamic services such as web servers or APIs, Release creates a routing entry that points requests to the Application Load Balancer for your environments' Kubernetes clusters.

For static sites, Release takes the bucket hostname and path, including a list of any assets written to S3 for the version being deployed, and adds those entries to our routing table.

Release takes care of maintaining this routing table when you create or delete environments.

### How Release creates unique hostnames

If you're a self-hosted user with a custom domain, say, `example.com`, hostnames for your ephemeral environments will follow this pattern:

`service_name-${randomness}-app_name.example.com`

Here is an example hostname for an app called `portal` and a service called `api`:

`api-t9yhp25-portal.example.com`

### Creating *predictable* hostnames using environment handles

You can create [environment handles](/reference-documentation/account-settings/environment-handles) to replace the random string in an environment's hostnames. This enables Release to create, or re-use, unique hostnames from a pre-determined list.

This is useful for services where it is expected that hostnames are known in advance, for example, when manually whitelisting callback domains at your OAuth provider.

When an environment is deleted, the environment handle used for the environment's hostname is returned to the pool of available handles and can be re-used for new environments.


# Example applications

Discover typical Release set-up patterns here.

You can modify your configuration settings in the [**Application Template**](/reference-documentation/application-settings/application-template). Below are some common examples.

{% content-ref url="/pages/-MZOAYIgWoeWGoTjnPDA" %}
[Serve traffic on multiple ports](/guides-and-examples/advanced-guides/serve-traffic-on-multiple-ports)
{% endcontent-ref %}

{% content-ref url="/pages/D8hCepfi0rpphTPAToj6" %}
[Spring and PostgreSQL](/guides-and-examples/common-setup-examples/spring-postgresql)
{% endcontent-ref %}

{% content-ref url="/pages/-MZOAYIiq\_8OyK6xJZqy" %}
[Load balancer with hostname](/guides-and-examples/common-setup-examples/load-balancer-with-hostname)
{% endcontent-ref %}

{% content-ref url="/pages/-MZOAYIjRqcm4aq-0Smv" %}
[Static JavaScript service](/guides-and-examples/common-setup-examples/define-a-static-javascript-service)
{% endcontent-ref %}

{% content-ref url="/pages/-MZOAYJO\_gsDMOatm\_xJ" %}
[Microservices architecture](/guides-and-examples/advanced-guides/microservices-architecture)
{% endcontent-ref %}

{% content-ref url="/pages/-MZOAYJQr\_wf1iJJOYK6" %}
[SSH bastion access to services](/guides-and-examples/common-setup-examples/adding-ssh-bastion-access-to-services)
{% endcontent-ref %}

{% content-ref url="/pages/YGAPzI6Neu6tBMpRNaar" %}
[ngrok and OAuth for private tunnels](/guides-and-examples/common-setup-examples/using-ngrok-+-oauth-for-private-tunnels)
{% endcontent-ref %}


# Full stack voting app

## Overview

In this tutorial, we'll deploy a containerized full-stack application to Release. Our example voting application will allow users to vote for their favorite category and then view the results of the votes. We'll use multiple stacks and frameworks for our app to illustrate the breadth and flexibility of deployments in Release.

Our codebase will comprise three services that will be containerized and managed by a `docker-compose` file. Additionally, we will use Postgres as a database and Redis as a message broker to offload some of the computational load to the `worker` service. Our services will be:

* **Vote:** A frontend and some server-side code that will push the vote made by a user to Redis. This will be built in Python, using the Flask framework.
* **Result:** A frontend that uses a websocket API to poll data from its server-side implementation to provide real-time updates of votes. This will be a Node.js application that uses Express to serve an Angular frontend. The frontend will use [Socket.IO](http://socket.io) to manage the websocket connection.
* **Worker:** The background task processor that reads from Redis and creates entries in our Postgres database to represent the results of the votes. The worker will be implemented using Java.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-c33e6a5de3b1fd0f6881556455e4f888347600eb%2F1-architecture-voting-app.png?alt=media\&token=d397d299-ea09-4459-a8fb-52a0897677b7)

Our completed voting application will look like this:

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-44b68a40c89b3ad9b2a4275d274fad482fac8032%2F2-screenshot-voting-app.png?alt=media\&token=bf61889d-6433-4c6c-b175-5e1757e78f3d)

And the result application will look like this:

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-d28d57f7ef3f30cb955482bfb51467a49242de31%2F3-screenshot-voting-app.png?alt=media\&token=250c5186-5725-4d7e-b2f0-6f8077a3d247)

You can find the completed code for the project [here](https://github.com/awesome-release/release-example-voting-app).

## Fork and clone the repository

Create a fork of the [repository](https://github.com/awesome-release/release-example-voting-app) on your version-control hosting provider (GitHub, GitLab, or Bitbucket). Ensure that the provider you fork the repository to is the provider you have [integrated with Release](https://docs.releasehub.com/integrations/source-control-integrations).

Once you have forked the repository, you can clone it to your development machine to get started.

{% hint style="info" %}
You do not need to have the repository clone to deploy the application to Release, but it is useful to be able to work through the code and understand the codebase.
{% endhint %}

## Project structure

### Vote application

Our vote application is a small Flask web app that accepts POST requests from the `index.html` file it bundles and serves statically via a GET request to `/`.

If you take a look at the `vote/app.py` file, you should see the code below:

```jsx
@app.route("/", methods=['POST','GET'])
 def hello():
     voter_id = request.cookies.get('voter_id')
     if not voter_id:
         voter_id = hex(random.getrandbits(64))[2:-1]

     vote = None

     if request.method == 'POST':
         redis = get_redis()
         vote = request.form['vote']
         data = json.dumps({'voter_id': voter_id, 'vote': vote})
         redis.rpush('votes', data)

     resp = make_response(render_template(
         'index.html',
         option_a=option_a,
         option_b=option_b,
         hostname=hostname,
         vote=vote,
     ))
     resp.set_cookie('voter_id', voter_id)
     return resp
```

This code has a few responsibilities:

* When an HTTP request is received, we assign a voter ID to the caller, if one is not already present as a cookie on the request.
* If the HTTP request is a POST request, we connect to Redis, and push a JSON payload containing voter data onto a Redis queue called `votes`.
* If the HTTP request is a GET request, we return the `index.html` template file with a few parameters.

The most important parameters provided to our template are `option_a` and `option_b`.

```jsx
option_a = os.getenv('OPTION_A', "Cats")
option_b = os.getenv('OPTION_B', "Dogs")
```

These are the categories that a user can vote for. If the environment variables for `OPTION_A` and `OPTION_B` aren’t set, the default options will be "Cats" and "Dogs".

### Worker application

The worker application is purely a backend service, written in Java.

On startup, it establishes a connection to Redis and the PostgreSQL database.

```jsx
...
class Worker {
  public static void main(String[] args) {
    try {
      Jedis redis = connectToRedis("redis");
      Connection dbConn = connectToDB("db");
...
		}
	}
}
```

As part of the connection to our database, the worker application also *creates* the necessary database tables.

```jsx
...
PreparedStatement st = conn.prepareStatement(
        "CREATE TABLE IF NOT EXISTS votes (id VARCHAR(255) NOT NULL UNIQUE, vote VARCHAR(255) NOT NULL)");
      st.executeUpdate();
...
```

It then watches the Redis queue called `votes` for new items.

```jsx
while (true) {
        String voteJSON = redis.blpop(0, "votes").get(1);
        JSONObject voteData = new JSONObject(voteJSON);
        String voterID = voteData.getString("voter_id");
        String vote = voteData.getString("vote");

        System.err.printf("Processing vote for '%s' by '%s'\n", vote, voterID);
        updateVote(dbConn, voterID, vote);
      }
```

When a new item is found, it calls a method called `updateVote`, which handles writing the result of a vote to the PostgreSQL database.

```jsx
static void updateVote(Connection dbConn, String voterID, String vote) throws SQLException {
    PreparedStatement insert = dbConn.prepareStatement(
      "INSERT INTO votes (id, vote) VALUES (?, ?)");
    insert.setString(1, voterID);
    insert.setString(2, vote);

    try {
      insert.executeUpdate();
    } catch (SQLException e) {
      PreparedStatement update = dbConn.prepareStatement(
        "UPDATE votes SET vote = ? WHERE id = ?");
      update.setString(1, vote);
      update.setString(2, voterID);
      update.executeUpdate();
    }
  }
```

### Result application

The result application, in a similar fashion to the vote application, serves an `index.html` file via its `/` route.

More interestingly, it exposes a websocket API using [Socket.IO](http://socket.io).

```jsx
io.sockets.on('connection', function (socket) {

  socket.emit('message', { text : 'Welcome!' });

  socket.on('subscribe', function (data) {
    socket.join(data.channel);
  });
});
```

On startup, the result application establishes a connection to the PostgreSQL database.

```jsx
async.retry(
  {times: 1000, interval: 1000},
  function(callback) {
    pool.connect(function(err, client, done) {
      if (err) {
        console.error("Waiting for db");
      }
      callback(err, client);
    });
  },
  function(err, client) {
    if (err) {
      return console.error("Giving up");
    }
    console.log("Connected to db");
    getVotes(client);
  }
);
```

Once a connection has been successfully established, it calls a `getVotes()` function using the database client.

This function reads the vote results from the database (which were written to it via the worker) and publishes them to a [Socket.IO](http://socket.io)-managed channel called `scores`.

```jsx
function getVotes(client) {
  client.query('SELECT vote, COUNT(id) AS count FROM votes GROUP BY vote', [], function(err, result) {
    if (err) {
      console.error("Error performing query: " + err);
    } else {
      var votes = collectVotesFromResult(result);
      io.sockets.emit("scores", JSON.stringify(votes));
    }

    setTimeout(function() {getVotes(client) }, 1000);
  });
}
```

Our client-side code (anchored at `result/views/app.js`) reads from the `scores` channel and updates the result application’s frontend accordingly.

```jsx
...
var updateScores = function(){
    socket.on('scores', function (json) {
       data = JSON.parse(json);
       var a = parseInt(data.a || 0);
       var b = parseInt(data.b || 0);

       var percentages = getPercentages(a, b);

       bg1.style.width = percentages.a + "%";
       bg2.style.width = percentages.b + "%";

       $scope.$apply(function () {
         $scope.aPercent = percentages.a;
         $scope.bPercent = percentages.b;
         $scope.total = a + b;
       });
    });
  };
...
```

### Docker Compose

Each of the applications described above is containerized using a `dockerfile` in their respective directories. We can use `docker-compose` to coordinate and run our applications together, as well as run containerized versions of Redis and PostgreSQL.

Below is the complete `docker-compose.yml` file required to build and run our applications.

```yaml
version: "3"

services:
  vote:
    build: ./vote
    command:
      - python
      - app.py
    ports:
      - "5000:80"
    depends_on:
      - "redis"
      - "db"
  
  result:
    build: ./result
    command:
      - nodemon
      - server.js
    ports:
      - "5001:80"
    depends_on:
      - "redis"
      - "db"
  
  worker:
    build:
      context: ./worker
    depends_on:
      - "redis"
      - "db"
  
  redis:
    image: redis:alpine
    ports:
      - "6379"
    volumes:
      - redis:/data
  
  db:
    image: postgres:14
    ports:
      - "5432"
    volumes:
      - postgres-data:/var/lib/postgresql/data
    environment:
      POSTGRES_PASSWORD: postgres
      POSTGRES_USER: postgres

volumes:
  postgres-data: {}
  redis: {}
```

## Run the project locally

To run our project locally, ensure you have Docker installed, and run the following command in the root of the project:

```jsx
docker-compose up 
```

The vote application will be accessible via port `5001` on `localhost` and the Result application will be available via port `5002`.

## Deploy to Release

Once we’ve created the applications and set up our `docker-compose.yaml` file, we’re ready to deploy our app to Release.

Ensure you’ve **forked** the repository before we get started. The instructions to deploy our example voting app are [here](https://docs.release.com/getting-started/create-an-application).

After deployment, we can click on the hostname URL for the vote application to tinker with making votes. You can share this URL with other people to vote, too.

To view the results in real-time, you can navigate to the hostname URL for the result application.

### Changing environment variables

Our default environment variables for `OPTION_A` and `OPTION_B` were set to “Cats” and “Dogs”, but perhaps we’d like our users to choose between “Python” and “JavaScript”.

To modify this, we can navigate back to our Application Dashboard and click on our ephemeral environment.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-b9f5a7ed7dd03471f6a9b85c2a6228dd95abb69f%2F14-config-voting-app.png?alt=media\&token=33c5999d-8796-4863-a1a7-350fb4b7bc59)

From there click on the **Settings** tab and click the **Edit** button for the **Environment Variables** section.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-d1d8e73bf8c55b214b3aeb9c304908763317f3b9%2F15-env-vars-voting-app.png?alt=media\&token=2d399d52-8bee-47a2-b06a-b33f9c8cd768)

From here, we can modify the environment variables for the environment. We will add two variables, for `OPTION_A` and `OPTION_B` respectively. Then click **Save As New Version** and then **Apply**.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-eeacba962adb3f6cd50765d4017486cbf0e6fcbd%2F16-env-config-voting-app.png?alt=media\&token=50b857b7-3300-4cbd-acd3-3f243f2dc7ac)

This will apply the latest configuration changes to our live environment and redeploy it.

Once our deployment is complete, we should be able to navigate back to the vote application and see our new voting categories in action!

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-bef6c55e54f5bbd13317ba5339c59e1fbb29dd6e%2F17-updated-voting-app.png?alt=media\&token=ff8f2bce-ebcb-416b-87c3-85d390a17d16)

### Next steps

In this tutorial, we’ve learned how to set up and deploy a non-trivial project with multiple services using Release. We’ve also looked at how to configure databases using Docker on Release. Additionally, we learned how to modify environment configurations and redeploy afterward.

A good next step might be to create an environment specifically for a `development` branch of the project so that you can iterate on your project without impacting a production deployment.


# Flask and RDS counter app

Let's walk through an end-to-end example of building a basic counting app using:

* Flask
* PostgreSQL
* RDS (AWS Relational Database Service)

We'll deploy the Flask application into our Release cluster and configure a security group to allow the nodes in the cluster to communicate with the RDS database. You can use a similar method to configure access to other cloud services, such as Redshift, Elasticache, or Redis.

Our final setup will look like this:

![Counter app and RDS in AWS VPC](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-e3f3bbb673ea27a2e0b38f07eba1af4cee4974f6%2Fflask-rds-overview-diagram.png?alt=media\&token=df2b3c39-42bc-48d4-b898-38a21533e98c)

Note how the database and node group exist in the same VPC and use a common security group to communicate.

To follow along, you should have a Release account configured with AWS, with a domain name and cluster already running. Follow the [quickstart guide](/getting-started/quickstart) if you don't have these in place yet.

## Set up an RDS instance

We'll create a small, free-tier RDS instance for this demonstration, but for a production application, you'll want to tweak the settings regarding the number of instances, multi-region deployments, backups, and so on. If you already have an RDS database, you'll just need to configure it to:

* Share a VPC with your Release node group.
* Share a security group with your Release node group.

### Identify the security group of your node group in AWS

When you set up a new cluster and node group in Release, an EC2 launch template was automatically created for you with a default security group. We'll need to find this security group that your worker nodes already use so we can assign it to the database.

Log in to your Release account, and take note of your cluster context and node group name, as both of these are used in the name of the security group. You can see how to find each in the image below.

![Find the cluster identifier and node group name](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-6fb91eee5712fa17649377f08f7a79276571f54e%2Fflask-rds-cluster-nodegroup-name.png?alt=media)

Also note which **region** your cluster is running in.

Now log in to your AWS account and navigate to the EC2 dashboard for the same region as your Release cluster. You should see at least one (and probably more) EC2 instances that are worker nodes for your cluster. You can identify them as they have the same cluster context and node group name that we noted in the Release control panel in their name, followed by the suffix `-Node`.

Select one of these worker nodes and navigate to **Actions** -> **Security** -> **Change Security Groups**, as shown below.

![Navigate to security groups for EC2 instance](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-f511969e17baafb311de11af8373a5c954331719%2Fflask-rds-get-security-group-ec2.png?alt=media)

Don't make any changes here, but note the name of the security group and then press **Cancel**.

![Note the security group name](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-0e6955080eca8608971523f53c83ce37754592a2%2Fflask-rds-security-group-name.png?alt=media\&token=d32bd7ef-1a33-417d-bdad-766353e1703f)

Finally, back on the main EC2 page, note the VPC ID of the worker node too. You can find it as shown in the image below.

![Get the VPC ID of an EC2 instance](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-527a7e76ecac864cfe12552cc11b96496937c6b1%2Fflask-rds-ec2-vpc-id.png?alt=media\&token=0b1b0f21-1f13-41cd-a96a-ff27d2fd4676)

### Create the RDS database

Now navigate to the RDS dashboard and choose to create a new database.

![Create a new RDS database](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-812394d6c6e2c249c58b1feb0e2933c8715bad37%2Fflask-rds-create-database.png?alt=media\&token=b0e97567-9f6e-46c6-bfcb-66c0667d9c98)

The most important settings to change are:

* The VPC - Make sure you create the database in the same VPC as your Release clusters.
* The security group - Add the same security group as the one you noted earlier.

For this demo, we left most of the default settings for a free-tier Postgres database, as detailed below.

Leave the default database creation method as **Standard create** and choose **PostgreSQL** as the database type.

![Choose Postgres](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-4f42834ddd5682bbd31ab3d529becf4daea773ea%2Fflask-rds-create-database-choose-postgres.png?alt=media\&token=eb72bac0-fcc7-44ef-90c5-2a691854c6b7)

Choose **Free tier**, name your database something meaningful, and choose a strong password (note this somewhere safe).

![Choose free tier and set name and password](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-1c3267c62c96349c197a5032137b383dc4f62fd5%2Fflask-rds-create-database-free-tier-password.png?alt=media\&token=2032f279-8a3f-44fa-ab2b-f283dced1323)

Set the VPC and security group to match the settings from your Release EC2 worker node, which we noted in the previous section.

![Set VPC and security group](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-9ceda8cea1d42d1aefda3e69144d9707d1a91d4f%2Fflask-rds-create-database-vpc-security-group.png?alt=media\&token=1a910b84-3562-4cae-b8a6-09c7715c6f09)

Scroll to the bottom and click **Create database**.

![Confirm create](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-f9be687ff82aaf1a783df461e5948d53ad06946d%2Fflask-rds-create-database-confirm-create.png?alt=media\&token=28e2eca4-f9ab-4354-a596-728447b2c8b7)

The database will take a few minutes to initialize. Once it's ready, navigate to it from the RDS dashboard and note the endpoint, as shown below.

![Note RDS endpoint](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-64a057494c5606c4c603f2a9e148cb738284cf47%2Fflask-rds-get-rds-endpoint.png?alt=media\&token=94e46d74-b06c-4060-9578-26cdff5e2648)

You now have an RDS database that your Release cluster can communicate with. To complete the example, we'll build a basic Flask application that uses this database and deploy it into our cluster.

## Build the application

For the demo, we'll build a very basic "counter" application that counts the number of times the **Count** button has been pressed. It'll look like this when it's up and running:

![An application showing a count button and the total recorded counts](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-73f2dfa6825873c9bb1a0d8ac3b3c238b8ded49c%2Fflask-rds-counter-result.png?alt=media\&token=99255cf3-992a-477b-aa8d-a3e9c076ee17)

Our full project will have the following directory structure. We'll use a Docker Compose file even though we're only running a single container to make it easier to extend the app. We've called our app `count`, and placed all the application code in an `app.py` file and the frontend code in `templates/index.html`.

![Directory structure as tree output](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-2f86bda099b44dd1c3df2b837c2e6109b7fb47be%2Fflask-rds-directory-structure.png?alt=media\&token=4033b03b-962e-4d40-9afe-dc67e1d5f1a8)

Create a directory for the overall project, a subdirectory called `count` for the Python application, and a subdirectory in the `count` directory called `templates` for the frontend. In the `count` directory, create blank files for `app.py`, `Dockerfile`, and `requirements.txt`. In the templates directory, create a blank file called `index.html`. In the outer project directory, create a blank file called `docker-compose.yml`.

### Write the `app.py` file

Our `app.py` file uses Flask-SQLAlchemy as an ORM. First we get credentials from our environment (we'll set these up later), and then we define a basic `counts` object, which has an automatic primary key and a single field that always stores `1`.

We then initialize the database and define a `hello()` method. When a user visits the page, we'll count the number of records in the database. If the user presses the **Count** button, we'll create a new count object, which will have the effect of incrementing the total.

Finally, we run the application on port 80 with debug turned on.

Add the following code to the `count/app.py` file:

```python
from flask import Flask, render_template, request
from flask_sqlalchemy import SQLAlchemy
import traceback
import os

user = os.environ.get("POSTGRES_USER")
pw = os.environ.get("POSTGRES_PASSWORD")
host = os.environ.get("POSTGRES_HOST")
db = os.environ.get("POSTGRES_DATABASE_NAME")
DB_URL = f"postgresql+psycopg2://{user}:{pw}@{host}/{db}"
print(DB_URL)

app = Flask(__name__)

app.config["SQLALCHEMY_DATABASE_URI"] = DB_URL
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False

db = SQLAlchemy(app)


class counts(db.Model):
    id = db.Column("count_id", db.Integer, primary_key=True)
    value = db.Column(db.Integer)


try:
    db.create_all()
except Exception as e:
    print(e)
    print("couldn't initialize db")
    traceback.print_exc()


@app.route("/", methods=["POST", "GET"])
def hello():
    if request.method == "POST":
        c = request.form["countbutton"]
        count = counts(value=1)
        db.session.add(count)
        db.session.commit()

    total_counts = counts.query.count()
    return render_template("index.html", total=total_counts)


if __name__ == "__main__":
    app.run(host="0.0.0.0", port=80, debug=True, threaded=True)
```

Note that we've taken some liberties here in terms of best practices to keep the code simple. Specifically, for production settings, you should:

* Turn debug mode off in the last line.
* Run `db.create_all()` in a separate once-off script rather than in the web application code.
* Properly handle the exception rather than only logging the error if the database connection fails.

### Write the `templates/index.html` file

Our frontend contains a basic form with a single button and a templated variable `{{count}}` to display the total. Add the following code to `templates/index.html`:

```html
<!DOCTYPE html>
<html>
  <head>
  </head>
  <body>
        <h3>Count</h3>
        <p>Total counts: {{total}}</p>
        <form id="countform" name='form' method="POST" action="/">
          <input id="countbutton" type="submit" name="countbutton" value="Count"></input>
        </form>
  </body>
</html>
```

### Write the Dockerfile

For our Dockerfile, we'll extend the Python3 Alpine Docker image, as this has most of the packages we need while still being very slim. Add the following code to the `count/Dockerfile` file:

```docker
# Using official Python runtime base image
FROM python:3-alpine

# Set the application directory
WORKDIR /app

# Install our requirements.txt
ADD requirements.txt /app/requirements.txt

RUN apk add build-base

RUN apk add --no-cache supervisor \
    && python -m pip install --upgrade pip \
    && pip install -r requirements.txt

# Copy our code from the current folder to /app inside the container
ADD . /app

# Make port 80 available for links and/or publish
EXPOSE 80

# Define our command to be run when launching the container
CMD ["gunicorn", "app:app", "-b", "0.0.0.0:80", "--log-file", "-", "--access-logfile", "-", "--workers", "4", "--keep-alive", "0"]
```

This brings in a few more dependencies that we need to correctly install the `psycopg2binary` database driver, copies in our application code, installs our Python requirements, and runs the application using Gunicorn.

### Write the `requirements.txt` file

Our application needs Flask (our web framework), Gunicorn (to act as a gateway between Flask and our webserver), Flask-SQLAlchemy (as an ORM to easily create objects in our database), and `psycopg2-binary` (as a driver to talk to PostgreSQL).

```bash
Flask
gunicorn
flask-sqlalchemy
psycopg2-binary
```

### Write the `docker-compose.yaml` file

Finally, we'll initialize everything using Docker Compose. This step is not essential in our case as we have only a single Docker file, but real-world applications will usually tie together a few different containers so it's useful to have it in place. Add the following code to the top-level `docker-compose.yml` file:

```
version: "1"

services:
  count:
    build: ./count
    command: 
    - python 
    - app.py
    ports:
      - "5000:80"
```

This creates a single service (`count`) and builds the Docker file in our `count` directory.

We now have a full-stack application running. If you have Docker and Postgres running locally, you can set the expected environment variables and run `docker-compose up` to test that everything is working, otherwise we can skip straight to deploying it on Release and debugging any issues there.

## Deploy the application to Release

To deploy the application to Release, you need to:

* Push the application up to GitHub or other provider that you've integrated with Release.
* Create a new application in Release.
* Configure the environment variables to connect to the database.

### Create the application

You can find detailed steps for doing this in our [create your application guide](https://github.com/releasehub-com/docs/blob/main/examples/broken-reference/README.md), but we'll cover the most important steps below.

In your Release dashboard, choose to create a new application.

![Create new app](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-021ac5caf31473958989a19850820cd8d478ef94%2Fflask-rds-create-app.png?alt=media\&token=9b3132ea-c285-4de7-9a22-a71b1cc18619)

Select the repository that holds your application code and proceed to the next step. You can leave all the defaults in the generated Application Template.

In the "Build & Runtime Configurations" section, choose to **Edit** "Default Environment Variables", and add in the connection settings for the database.

![Edit environment variables](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-8233bc0a7ab081fbdd22df220e6fd621124a31bd%2Fflask-rds-edit-env-variables.png?alt=media\&token=a2076b9d-8505-4f3d-804e-4651a36ab299)

The bottom of the file (after the comments) should look as follows (substituting in your own database endpoint and password that you set in AWS, and username and database name if you changed those from their defaults).

```yaml
defaults: []
services:
  count:
  - key: POSTGRES_DATABASE_NAME
    value: postgres
  - key: POSTGRES_USER
    value: postgres
  - key: POSTGRES_PASSWORD
    secret: true
  - key: POSTGRES_HOST
    secret: true
```

Note how we define these under the `count` service (named in our `docker-compose.yml` file) so that these are only available to our main application service. We've also set the `secret` flag on the password and host fields to `true`, so these will be encrypted by Release at rest.

Click **Start Build & Deploy** and wait for the environment to come up. Once it's ready, you can click the URL in the section "Hosted URLs" to navigate to your running application.

![View hosted URL](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-d86738b68bbc40a0b1ab16877bf06a6d52b92a06%2Fflask-rds-hosted-url.png?alt=media\&token=752b7591-829f-493a-af35-1643417ff547)

If everything worked as expected, you should see the basic counting application running and be able to add to the count by clicking on the button.

![Final app](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-73f2dfa6825873c9bb1a0d8ac3b3c238b8ded49c%2Fflask-rds-counter-result.png?alt=media\&token=99255cf3-992a-477b-aa8d-a3e9c076ee17)

If you see an error or a blank page, [use the log viewer](https://github.com/releasehub-com/docs/blob/main/guides-and-examples/running-instances/viewing-logs.md) to look for any errors, or [use the terminal](https://github.com/releasehub-com/docs/blob/main/guides-and-examples/running-instances/using-terminal.md) to double check that the environment variables are set correctly by running, for example, `echo $POSTGRES_USER`.


# Static site with Gatsby

## Overview

You can use Release to deploy static sites to the Release CDN. Release can detect services inside your application that are eligible for [static service deployment](/reference-documentation/static-service-deployment) and you can choose to build and deploy these to the Release CDN, ensuring fast performance at the edge.

In this example, we'll use [Gatsby](https://www.gatsbyjs.com) to deploy a simple static site to the CDN. Gatsby is an open-source frontend framework designed to help you build fast, scalable, and SEO-optimised static sites. It's often used to run company blogs or websites.

Here's a look at the site we'll deploy to Release:

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-e8f53d6ca6325703238e34288176c2b8cc43c767%2Fgatsby-site-example.png?alt=media\&token=686f7808-b6de-4d1c-b974-b4fb45ec81ce)

We'll also take a look at how to manage and update the site once it is deployed.

## Get the example project

You can get the code for this project by forking the repository on [GitHub](https://github.com/awesome-release/gatsby-starter-theme). By forking the repository containing the code, you create a copy of it on your account. You can then edit this copy as you wish without needing permissions on the original project.

Once you’ve forked the repository to your account, clone your copy to your development machine.

```bash
git clone https://github.com/awesome-release/gatsby-starter-theme
```

## Source control integration

Once you’ve forked and cloned the repository, you can use the Release UI to integrate directly with your source-control provider (either GitHub or GitLab in this case).

To start, ensure that you are [signed in](https://app.releasehub.com/auth/login-page) to your Release account with the SSO provider for your source control integration.

When signing in, Release will ask for read/write access to the provider API so that it can list and view your repositories.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-855c1818a612f62dba246f2b6ce61c2985777842%2Frelease-authorisation.png?alt=media\&token=2900e8de-9679-4ac2-8703-dd08bf06e558)

{% hint style="info" %}
If you use the same email address across Bitbucket, GitLab, and GitHub, Release will automatically list a combined view of repositories for you. Each integration will appear separately on your profile page.
{% endhint %}

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-574a8ba54f04dbdd260701bb60edb3ff3be56c7b%2Frelease-source-control-integrations.png?alt=media\&token=4c04b83a-8a5e-4d6c-bbcd-c43839a2c826)

Once you’ve signed in, click the **Create New App** button on the [home page](https://app.releasehub.com/admin).

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-5a3f738bc90d7e0522f124833a023549577ed2eb%2Frelease-applications-view.png?alt=media\&token=54da9539-5eb2-4c9c-9ab2-899d56bb96be)

This will open a six-step form to help you create your application. The steps are as follows:

1. Create your application: Enter a name and select your repository.
2. Analyze your repository: Let Release scan your repository for services, then select which services to add to your application.
3. Generate a template: Based on the services you selected from your repository, Release generates a template for your application.
4. Set environment variables: Set the default environment variables for your new application.
5. Set build arguments: Set arguments to use when Release builds your application's Docker images.
6. Save and deploy: Deploy your new application.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-57a457f7a91669e49b4e0689488e2ea7119470a2%2Fcreate-application.png?alt=media\&token=2cd90778-f57d-4f2f-aa45-83e372af7b20)

Give the application a name and pick a repository and then click the **Next Step** button.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-c360f8015be196026ca77b5fb770fc62ef7b94c6%2Fanalyse-repository.png?alt=media\&token=99aee49c-3a70-4680-a8a2-818353264a3c)

Ensure that you have selected the correct branch for your repository to be built and deployed from (in this case, it is the `master` branch). From the **Select files** dropdown, select the `package.json` file for Release to analyze and click **Start Analysis**. Release will now read the file you selected and create a list of services found.

Release will **automatically detect** that your project contains static assets that are eligible for static asset deployment.

Double-check that Release has inferred the correct configuration for the project.

The service type dropdown should be set to **Static** with the Build Command being `yarn build`. The build **Output Directory** should be set to `public`. Click **Next Step** to move to the template generation step.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-bd01a8cc1cba2531f8e2aa31473159cd4643bc0e%2Fapp-template.png?alt=media\&token=62cad49d-bd54-4433-b57b-42b712f7797c)

The **Application Template** acts as a blueprint for new environments in this application. Release creates a template based on the services you selected in step two. You can edit the template or proceed to the next step.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-baf5182f7fa8820da1fb8b07f69ef28e55478248%2Fenv-variables.png?alt=media\&token=b7507535-6ec3-492c-8a4c-458bdb9dced0)

In this step, you can set default environment variables that will be available for all environments for your application. These environment variables consist of a key, a value, and an optional secret flag. For this application, you can proceed without setting environment variables.

In the next step, you can set build arguments. Build arguments are passed to Release's build process. You can proceed without setting build arguments for this application.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-bffdf299de4230432d265c4753ef8a487dbfb3b7%2Fsave-deploy.png?alt=media\&token=e8dc34c2-6fc5-492f-bb50-feb90db98eb4)

You've created your application and it is now time to save and deploy the application.

## The App Dashboard status page

While your application builds and deploys, you will be redirected to the App Dashboard status page.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-db5f8f1e641b2357daca0bf794a48f22cb4b4bdb%2Frelease-environments-view.png?alt=media\&token=aca0999f-8c46-400d-a493-b3396fe2fdca)

Here you can explore environments, builds, and deploys related to your application. You can also modify application-specific settings.

Click on the **Builds** tab to see a successful build for your Gatsby application.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-8b99cc48adc64e642b898ff31403d9de3009d266%2Frelease-view-builds.png?alt=media\&token=307804bb-b5a0-4e9e-91ad-0d2263d97700)

Once an ephemeral environment has been created and successfully deployed for your application, you should see a prompt.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-3cd2ddca4698bf2144afd601a0d29e976d1be3a7%2Frelease-view-environments.png?alt=media\&token=6796ef9d-4d23-4509-9e65-fde0459dc060)

You can also view this environment by clicking on the **Environments** tab.

## View environment details

Click on the name of an environment in the **Environments** tab to be taken to the Environment Details page.

Here you can view granular information about a particular environment, including the application instances deployed into it and the hostname URLs that exist.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-0e320b26055c3681ed1aef16fd07a44508c72371%2Frelease-deployed-environment.png?alt=media\&token=2405cfab-ef0d-4bdc-bf17-94c2aeaa27c2)

Click the hostname URL to go to your static website hosted on Release.

## Modify the application and deploy changes

In the real world, we would probably want to make changes to our application and redeploy it. Let’s add a new blog post to our application to see how this works.

Change directory into the repository we cloned earlier and open it in a text editor.

Create a new file in the `content/post` directory called `hello-release.mdx`. Add the following content to your newly created file:

```jsx
---
title: Hello Release
date: 2022-06-01
---

Hello, Release! This is a demo post illustrating how to make a change and redeploy a static website on Release.
```

Commit and push these changes to your remote repository:

```jsx
git add content/
git commit -m "Add a new post"
git push
```

Once you’ve pushed the changes, Release will automatically start a new build and deployment based on the latest commit in your repository.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-7ead519e5037d9f08f0567ff45dedfcd277c4c2b%2Frelease-build-after-change.png?alt=media\&token=44693697-e002-4636-a1b0-0990c5901884)

Once the deployment is complete, you should be able to see the updated blog at the same hostname URL you accessed earlier.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-52ced6ec229cdd4ea2e899d48bd537479c35fd55%2Frelease-updated-site.png?alt=media\&token=9fddbe18-235c-443d-b6f3-60c0bc40f54c)

## Deploy to a different branch

You probably don't want to deploy new changes directly to a production environment. Instead, you may have a development branch where you can send changes for review. Release allows you to create a new environment for this.

Let’s create a development branch in our repository and push a change to it.

```jsx
% git checkout -b develop
Switched to a new branch 'develop'
```

Create a new file in `content/posts` called `staged-post.mdx` and add the following content to it:

```jsx
---
title: Hello Release - Staged
date: 2022-06-01
---

This is a staged post that isn't in production yet.
```

Push the newly created `develop` branch remotely, including the new staged post:

```jsx
% git add .
% git commit -m "add new staged post"
[develop 3a00048] add new staged post
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 content/posts/staged-post.mdx
% git push --set-upstream origin develop
```

Navigate to the **Environments** tab in the App Dashboard of your application.

Click on **Create New Environment** and specify the `develop` branch.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-04a45ae4eb636b212020dca9ae426ea4f42b69ba%2Frelease-new-environment.png?alt=media\&token=254616ce-86f7-4959-83ce-4b3865070b0e)

When you click **Create Environment**, Release will automatically build and deploy from the development branch.

View the deployed application by navigating to the hostname URL specified for this environment on its Environment Details page. You can now iterate and make changes on this branch without impacting your main environment.


# Golang with Postgres and Nginx

{% hint style="danger" %}
Unfortunately, as of October 2022, this example application has been reported to us as broken. We will fix the example presented here to work with the new software release(s) shortly.
{% endhint %}

## Overview

In this walkthrough, we’ll look at the code and configuration required to deploy a Go server that connects to a Postgres database, running behind an Nginx proxy on Release.

You can find the completed code for the project [here](https://github.com/awesome-release/nginx-golang-postgres).

## Fork the project

[Fork the project](https://github.com/awesome-release/nginx-golang-postgres/fork) on GitHub to create a copy of the repository. Your fork of the project will allow you to integrate it with Release using your account, and experiment and make changes without affecting the original project.

## Project structure

Our example project uses `docker-compose` to orchestrate our backend, Nginx, and Postgres.

### The backend

The backend is a simple Go server, using the [Gorilla web toolkit](https://github.com/gorilla) to serve JSON content over HTTP. It connects to a PostgreSQL database, which stores the data that is served at `/` via the backend.

We have a simple route handler, anchored at `/`, which executes a `blogHandler` function.

```jsx
func blogHandler(w http.ResponseWriter, r *http.Request) {
	db, err := connect()
	if err != nil {
		w.WriteHeader(500)
		return
	}
	defer db.Close()

	rows, err := db.Query("SELECT title FROM blog")
	if err != nil {
		w.WriteHeader(500)
		return
	}
	var titles []string
	for rows.Next() {
		var title string
		err = rows.Scan(&title)
		titles = append(titles, title)
	}
	json.NewEncoder(w).Encode(titles)
}

func main() {
	...
	r := mux.NewRouter()
	r.HandleFunc("/", blogHandler)
	log.Fatal(http.ListenAndServe(":8000", handlers.LoggingHandler(os.Stdout, r)))
}
```

The blog handler queries the database, and returns the query result as JSON.

Additionally, on startup, the backend will bootstrap the required tables and data into the database.

### Nginx

Our Nginx server is configured to listen on port `80` and forward any requests to the backend service.

```jsx
server {
    listen       80;
    server_name  localhost;
    location / {
        proxy_pass   http://backend:8000;
    }

}
```

We build a Docker image for our Nginx server with a straightforward Dockerfile.

```jsx
FROM nginx:1.13-alpine
COPY conf /etc/nginx/conf.d/default.conf
```

### Docker Compose

Our backend, Nginx, and PostgreSQL are orchestrated via `docker-compose`.

```jsx
services:
  backend:
    build: backend
    secrets:
      - db-password
    depends_on:
      - db
  db:
    image: postgres
    restart: always
    secrets:
      - db-password
    volumes:
      - db-data:/var/lib/postgresql/data
    environment:
      - POSTGRES_DB=example
      - POSTGRES_PASSWORD_FILE=/run/secrets/db-password
    
  proxy:
    build: proxy
    ports:
      - 80:80
    depends_on: 
      - backend
volumes:
  db-data:
secrets:
  db-password:
    file: db/password.txt
```

## Run the project locally

To run the codebase on your development machine, ensure you have Docker installed.

Run the following command via the terminal:

```jsx
docker-compose up
```

Once the application has started, you can run the following command to check if everything is working:

```jsx
$ curl localhost:80
["Blog post #0","Blog post #1","Blog post #2","Blog post #3","Blog post #4"]
```

Lastly, to stop and remove the containers created by `docker-compose` , run:

```jsx
docker-compose down
```

## Deploy to Release

To deploy this application to Release, log in to your Release account and navigate to the dashboard.

From the dashboard, select the **Create New App** button. This will take you to the Create Your Application screen.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-1094a46bc812cb7ebc7313184aa32c3b5db2fb54%2F1-create-app-go-nginx-new.png?alt=media\&token=8cbf0e8f-b779-4626-a954-c88fb0d05dcf)

Refresh the repository list, and select the forked version of the [`nginx-golang-postgres`](https://github.com/cishiv/nginx-golang-postgres) repository from the dropdown.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-4586f0bd3f2a761ad865c8c5c56153c53eaae38b%2F2-repo-select-go-nginx-new.png?alt=media\&token=3f114e13-afb4-4579-b7fd-82ab4d47f9de)

Using the `docker-compose` configuration, Release will be able to automatically detect the services in the codebase.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-2e18e3c636b4e7c29f14499342f2e34007651aee%2F3-app-config-go-nginx.png?alt=media\&token=e8ea928a-e4a3-4d83-bd34-ad37cdbc02a8)

Select a name for your application, and click **Generate App Template**. It's a good idea to keep the name short but descriptive, as it will be used in your hostnames for this application.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-1220a3b0b2f3b42a8978d48d582afedb095a2322%2F4-app-name-go-nginx.png?alt=media\&token=a405f0c3-f71d-4791-86e7-88ee42887a9e)

Release will then allow you to modify the Application Template that was generated. For the purposes of this application, nothing needs to be modified here, so click **Save & Continue**.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-3687a623493b34832c3182362482429b13529aae%2F5-app-template-go-nginx.png?alt=media\&token=4b0a015d-ddb8-4efe-b120-870edc8f7ae9)

Lastly, before building and deploying the application, you can configure any build arguments, environment variables, or Just-In-Time File Mounts. Once again, we don't need to make any changes here, so click **Start Build & Deploy**.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-c9b3098f39dca3f5153e180d04af2ef82525ea9b%2F6-build-conf-go-nginx.png?alt=media\&token=9f449f42-9200-4273-86b0-be20eab2753f)

Release will kick off a build and deployment for your application. This shouldn’t take more than a few minutes for this particular example.

Once our application has been successfully deployed into an environment, we can hover over the `proxy` badge on the environments tab. This badge indicates that the `proxy` service configured in our `docker-compose.yaml` file has a host name URL that we can access.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-0afac4f9b3a2c145c6603120cf612d10f5f71f35%2F7-deployed-go-nginx.png?alt=media\&token=7e6c91cc-1a7b-44f6-b2b2-3d80856dcd9d)

Clicking on this badge will take us to our live application, where we should see the response from our backend being routed via Nginx.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-8be7a6f0504d78c57219003cf73e324cdfa692ea%2F8-result-go-nginx.png?alt=media\&token=247a5ff6-749d-443c-9d8c-7ede1e8836af)


# WordPress with MySQL

In this tutorial, you'll learn how to deploy WordPress and MySQL on Release.

## Docker Compose and Release for WordPress development

Using Docker Compose during WordPress theme or plugin development ensures that your development and build environments can be recreated easily on your team's workstations and staging servers.

By adding your WordPress repository to Release, you can create environments per branch, create staging environments, test contributors' pull requests, and share demo links with clients.

## WordPress and Docker Compose

For this tutorial, we'll start with the [WordPress with MySQL](https://github.com/docker/awesome-compose/tree/master/wordpress-mysql) example from [Awesome Compose](https://github.com/docker/awesome-compose/). To simplify getting started, we've created a repository with only the `wordpress-mysql` folder from the Awesome Compose repository: [wordpress-mysql](https://github.com/awesome-release/wordpress-mysql).

## Requirements

[Create a Release account](/getting-started/create-an-account) and [integrate your source control](/integrations/source-control-integrations) (GitHub, Bitbucket, or GitLab) account.

Besides a web browser, you won't need anything specific installed on your computer for this guide.

## Fork the `wordpress-mysql` example repository

Fork the [`wordpress-mysql`](https://github.com/awesome-release/wordpress-mysql) repository to a private or public repository in your version control hosting service.

## An overview of the example application

The `docker-compose.yaml` file lists two services for this application: `db` and `wordpress`.

The service called `db` runs a specific `mysql` Docker image, while the `wordpress` service runs the latest `wordpress` Docker image.

## Create a new application on Release

Release can import applications based on Docker Compose, so we won't need to change anything in the repository to add this application to Release.

Firstly, log into Release, and [create an application](https://github.com/releasehub-com/docs/blob/main/examples/broken-reference/README.md) by clicking on **Create new app**.

![Create an application](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-b7e0ab96d89a633193dfb37b6616dd8431dc8699%2Fwordpress-mysql-create-application.png?alt=media\&token=e6328b14-fdc0-4127-9eda-701ee4845bbf)

## Name your application and select your repository

Enter a unique name for your application, and pick the forked repository you created earlier, then click **Next step**.

![Enter a name and pick your repository](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-4fdf88eb4c5eae87377562f6a7a6b4230bdc03c4%2Fwordpress-mysql-create-app-settings.png?alt=media\&token=88cc3038-f36b-4b21-8327-d02b0c40ed20)

## Pick your services

1. Pick **Analyze the repository**, so Release can create services from `docker-compose.yaml`.
2. Select the branch from your repository you'd like to track in this application.
3. Select the `docker-compose.yaml` file.
4. Click **Start analysis**.

![Pick your services](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-56fddb23f5aec215a8d86a4411cd1644194cb387%2Fwordpress-mysql-pick-services.png?alt=media\&token=33672a38-1594-4eab-a6c0-15ecdab8c831)

Release will read your Docker Compose file, and list the services found.

![Pick your services showing analysis result](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-5833d6e788a9040c8a1d921f541b42a7e0669165%2Fwordpress-mysql-services-analyzed.png?alt=media\&token=282373b0-1cc9-4e00-86c2-ca6f6e15dc07)

Click **Next step**.

## Generate a template

Release will generate a template based on the services from your `docker-compose.yaml` file.

![Generate an application template](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-5668f9d3f94c59c1b3fd748d57658aeb8c7536f4%2Fwordpress-mysql-app-template.png?alt=media\&token=15a383e4-1693-4233-86dc-7d30a31806e8)

Release will use this template to create new environments later on, and the defaults should work as expected for most Docker Compose applications.

### Expose container ports for the database service

{% hint style="info" %}
Release does not currently convert `expose` directives when importing services from Docker Compose. Add a `container_port` for each port in your Docker Compose file. Read more about [ports](/reference-documentation/application-settings/application-template/schema-definition#ports) in our template schema.
{% endhint %}

For this example application, we'll make one important change to the template to make sure the `db` service exposes two ports for MySQL.

Find the `db` service in the application template YAML, and add two container ports by making the following changes:

```diff
  services:
  - name: db
+   ports:
+   - type: container_port
+     port: '3306'
+   - type: container_port
+     port: '33060'
# ...
```

Release supports two different port types: `node_port` and `container_port`. We're adding a container port to the `db` service because this port should only be accessible on your application's private network. This enables the `wordpress` service to connect to the `db` service without exposing your MySQL database to the internet.

After editing the template, click **Next step**.

## Environment variables

The `docker-compose.yaml` file from our repo lists the environment variables for each service.

To see how each service's container will use these environment variables, we can consult the Docker images documentation.

### [`mysql`](https://hub.docker.com/_/mysql) Docker image variables

#### `MYSQL_ROOT_PASSWORD`

Specifies the `root` MySQL user's password. On a production server, this should be treated as secret.

#### `MYSQL_DATABASE`

If this variable is set, MySQL will create a database with this variable's value as a name. For our example, we'll set this to `wordpress`.

We can expect the MySQL server to create a database called `wordpress` on startup.

#### `MYSQL_USER` and `MYSQL_PASSWORD`

MySQL uses these two variables together to set a username and password for a new MySQL user on startup. This user will be granted superuser privileges (`GRANT ALL`) on the database specified by `MYSQL_DATABASE`.

For the example application, we'll set both to `wordpress`.

On startup, we can expect MySQL to create a new user called `wordpress` with the password `wordpress`, and grant the user all privileges on the `wordpress` database.

### [`wordpress`](https://hub.docker.com/_/wordpress) Docker image variables

WordPress uses a `wp-config.php` file to set database access strings and other settings. Usually, you would edit `wp-config.php` manually and set the database hostname, username, and password as PHP variables.

In Docker, we do this differently. The WordPress Docker image uses a [custom `wp-config.php` file](https://github.com/docker-library/wordpress/blob/master/wp-config-docker.php) that enables you to configure WordPress through environment variables.

For more information about the benefits of using environment variables for configuration (not just in Docker), we recommend reading [The Twelve-Factor App III: Config](https://12factor.net/config).

Let's take a look at the four environment variables in our example application.

#### `WORDPRESS_DB_HOST`

The WordPress Docker image uses this variable to set the database hostname. For our example, we'll set this to `db`.

Release creates a private network for your application, and by default, any service in your application can discover any other service in the application by querying a service's name.

This means the `wordpress` service can reach the `db` service's container on the private network by querying the `db` hostname.

#### `WORDPRESS_DB_USER`, `WORDPRESS_DB_PASSWORD`, and `WORDPRESS_DB_NAME`

The WordPress Docker image uses these three variables to know which database to use and to authenticate with the MySQL database server.

For this example, we'll set all three to `wordpress` so that they match the equivalent variables set for the `db` service.

### Set your environment variables

![Set your environment variables](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-e81a318849451118e0c2d8386e17becb45f8a745%2Fwordpress-mysql-env-vars.png?alt=media\&token=c497805a-1e94-42c3-9b64-f37b16909840)

Edit the YAML environment variables in Release to look like this:

```yaml
---
defaults: []
services:
  db:
  - key: MYSQL_ROOT_PASSWORD
    value: somewordpress
    secret: true
  - key: MYSQL_DATABASE
    value: wordpress
  - key: MYSQL_USER
    value: wordpress
  - key: MYSQL_PASSWORD
    value: wordpress
    secret: true
  wordpress:
  - key: WORDPRESS_DB_HOST
    value: db
  - key: WORDPRESS_DB_USER
    value: wordpress
  - key: WORDPRESS_DB_PASSWORD
    value: wordpress
    secret: true
  - key: WORDPRESS_DB_NAME
    value: wordpress
```

### Secret environment variables

You'll notice that we marked the `MYSQL_ROOT_PASSWORD`, `MYSQL_PASSWORD`, and `WORDPRESS_DB_PASSWORD` variables as secret by setting the `secret` flag to true for each of them.

Release saves [secret variables](/reference-documentation/environment-settings/environment-specific-environment-variables/managing-secrets) in an encrypted vault, and exposes these secrets to your running containers at startup.

Click **Next step** to save your environment variables.

## Build arguments

This application does not use any build arguments, so we can click **Next step**.

## Save and deploy

Click **Deploy your app** to create your application.

![Deploy your app](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-a51b94c1e949d2e509fe8962cde748a3f7a1051a%2Fwordpress-mysql-deploy.png?alt=media\&token=195b529d-0d21-47ad-a923-6043638e021e)

Release will now pull two Docker images, `mysql` and `wordpress`, then start two services, `db` and `wordpress`.

Your browser should redirect to a deployment status page, showing the progress of the setup workflow.

After Release's set-up and deployment workflow completes, navigate to your new environment and click on the URL for the `wordpress` service.

![Service hosted URL](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-f8463a3f7cb1d2c1c782aae184cd5c25fd457052%2Fwordpress-mysql-detail.png?alt=media\&token=67de8800-eb86-4d6e-82e9-2f45ed0730ab)

## Install WordPress

You should now see the WordPress setup screen.

![WordPress' language selection screen](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-260c31827a63c67472aa085f03ea42266cf777ef%2Fwordpress-mysql-result.png?alt=media\&token=dd31c503-4dc5-4982-92c1-b5e2e19596a3)

Select your language and click **Continue**.

![Install WordPress](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-985d078a57e5d310eba3c3174c419940fc72b45c%2Fwordpress-mysql-wordpress-setup.png?alt=media\&token=813abf98-443e-4377-be80-d23a8557e414)

Create a user and install WordPress by following these steps:

1. Enter a site title.
2. Enter a username, which you'll use later to log into WordPress.
3. Copy and save the recommended password. You'll use this later to log into WordPress.
4. Enter your email address.
5. Tick the **search engine visibility** checkbox. We recommend this to discourage search engines from crawling websites in development.
6. Click **Install WordPress**.

Your WordPress service will now run WordPress's installation script, which connects to your MySQL database and adds the required tables and configuration. WordPress also creates an admin user based on the username and password you entered.

![WordPress installed successfully](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-dde0fe78b1c298babac0b36db1e9c74d5853455b%2Fwordpress-mysql-success.png?alt=media\&token=801fff4c-dec2-4392-bff9-30dc8ca4b094)

Click **log in**.

## Log into WordPress

Enter your username and password, then click **Log in**.

![Log into WordPress](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-4deedc8f6ca085135c98ac4bafd85abb22f5abe3%2Fwordpress-mysql-login.png?alt=media\&token=9734d520-cde1-47dc-8f96-69c025b6fcc9)

## View your WordPress site

On the WordPress dashboard, hover over your site's name and click **visit site**.

![View your WordPress site](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-f4b67379a3a7e437e3274e44805727d5c13c73ef%2Fwordpress-mysql-view-site.png?alt=media\&token=cb78bce8-29ec-4f75-bfe0-4afbd043ea88)

You should see a sample page on your WordPress site now.

![WordPress sample page](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-9218f377bb104fc75c2da1459bd55012623fcbbe%2Fwordpress-mysql-site.png?alt=media\&token=1b1ab692-8077-466b-9006-095e701d34ff)

## Troubleshooting

If you don't see the WordPress language selector after the deployment workflow above, you can view the logs for each service in your environment to see whether either service logged any errors.

To view a service's logs, navigate to your environment's details page, scroll down to the list of services, and click on **logs**.

### Debugging database connection errors

The most common problem when first installing WordPress is a database connection error.

![WordPress database error](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-3358d752e4e906d54cb8b49fb512e3d0856dae74%2Fwordpress-mysql-db-error.png?alt=media\&token=c03d7e58-332e-4be6-b7dc-583ab08054d5)

On Release, this error can be solved by working through the following steps:

1. Make sure the corresponding environment variables from both services match.

   Values for the variables in each row below should match:

   | `db` service     | `wordpress` service     |
   | ---------------- | ----------------------- |
   | `MYSQL_DATABASE` | `WORDPRESS_DB_NAME`     |
   | `MYSQL_USER`     | `WORDPRESS_DB_USER`     |
   | `MYSQL_PASSWORD` | `WORDPRESS_DB_PASSWORD` |
2. Make sure the `WORDPRESS_DB_HOST` environment variable matches the name of your database service.
3. Make sure your database service exposes the correct ports. MySQL on Docker uses ports `3306` and `33060`.

## Further reading

Docker is most useful when developing WordPress plugins and themes, rather than just configuring a *WordPress site*.

We recommend pre-installing your themes and plugins if you're using Docker for WordPress development.

### Pre-installed themes and plugins

To learn more about how you can pre-install WordPress plugins using Docker Compose, we recommend reading [Include pre-installed themes / plugins](https://github.com/docker-library/docs/tree/master/wordpress#include-pre-installed-themes--plugins) from the WordPress Docker image documentation.

### Persistent volumes for uploads

In the example above, only the database service mounts a persistent volume. This means that the database data for each environment will remain after the application restarts or re-deploys.

The WordPress service does not use a persistent volume.

Images and other files uploaded to your WordPress site will be deleted when you re-deploy your application.

To learn about how to mount a persistent volume for WordPress using Docker Compose, we recommend reading [Quickstart: Compose and WordPress](https://docs.docker.com/samples/wordpress/).

To learn how to mount a persistent volume on a Release service, see the [Volumes](/reference-documentation/application-settings/application-template/schema-definition#volumes-1) section in our template schema documentation.


# Spring and PostgreSQL

In this tutorial, we'll learn how to deploy a Java application on Release, using the Spring application framework with PostgreSQL as its database.

We'll start with the `spring-postgres` example application from [Awesome Compose](https://github.com/docker/awesome-compose/). To help you focus on Spring and PostgreSQL only, we've created a repository with just the [`spring-postgres`](https://github.com/awesome-release/spring-postgres) folder copied from Awesome Compose.

Because Release supports creating applications using Docker Compose, we won't need to change anything in the example repository to add this application to Release.

We'll take a look at how Release converts the settings from the application's `docker-compose.yaml` to a template for new environments, then we'll explore how we can change the Release application slightly to follow best practices.

## Requirements

Before getting started, [create a Release account](/getting-started/create-an-account) and [integrate your source control](/integrations/source-control-integrations) (GitHub, Bitbucket, or GitLab) account.

## Fork the `spring-postgres` example repository

Fork the [`spring-postgres`](https://github.com/awesome-release/spring-postgres) repository to a private or public repository in the version control service you integrated with your Release account.

## An overview of the application

Let's take a look at the contents of the repository, and try to figure out how this application is installed and started.

The repository looks like this:

```
.
├── backend
│   ├── src
│   │   └── ... (Java app source)
│   ├── Dockerfile
│   └── pom.xml
├── db
│   └── password.txt
├── README.md
└── docker-compose.yaml
```

Two folders, `backend` and `db`, correspond to the two services defined in `docker-compose.yaml`:

```yaml
services:
  backend:
    build: backend
    #...
  db:
    image: postgres
    #...
#...
```

Let's look at the two services in more detail.

### The backend service

Excerpt from `docker-compose.yaml`:

```yaml
backend:
  build: backend
  ports:
    - 8080:8080
  environment:
    - POSTGRES_DB=example
  networks:
    - spring-postgres
```

You'll notice that the `backend` folder contains a Dockerfile, which, combined with the `build: backend` directive from `docker-compose.yaml`, indicates that a Docker image needs to be built for the `backend` service before starting a container.

The service joins the `spring-postgres` network, and forwards port `8080` on the host to port `8080` on the container.

This service has one environmental variable, `POSTGRES_DB`, which will be used as the PostgreSQL database name.

In the `backend/src` folder, you'll find the source code and other resources for the Java application. Of special interest is the `backend/src/main/resources/application.properties` file, where Spring boot looks for settings such as database connection strings.

### The `db` service

Excerpt from `docker-compose.yaml`:

```yaml
db:
  image: postgres
  restart: always
  secrets:
    - db-password
  volumes:
    - db-data:/var/lib/postgresql/data
  networks:
    - spring-postgres
  environment:
    - POSTGRES_DB=example
    - POSTGRES_PASSWORD_FILE=/run/secrets/db-password
  expose:
    - 5432
```

The `db` (short for database) service starts a container based on the [`postgres`](https://hub.docker.com/_/postgres) docker image.

This service needs a volume to save data in, has two environment variables (`POSTGRES_DB` and `POSTGRES_PASSWORD_FILE`), and exposes the container port `5432` on the `spring-postgres` network.

The `db` folder contains only one file, `password.txt`, which the `db` service can access as `/run/secrets/db-password` at runtime.

### How Docker Compose runs this app

We won't run this application locally for this tutorial, but if we were to run `docker-compose up` from the root of this folder, we would expect Docker to go through the following steps:

1. Read `backend/Dockerfile` and pull dependencies from a Docker registry.
2. Build a `backend` Docker image based on the steps in `backend/Dockerfile`, and store the image locally.
3. Pull the `postgres` image from a Docker registry and store it locally.
4. Start a `backend` container by running the `backend` image built in step 2.
5. Start a `db` container by running the `postgres` image retrieved in step 3.
6. Create a network called `spring-postgres` and connect both running containers to this network.

## Create a new application on Release

Now that we have an idea of how Docker Compose would run this application, let's see how to add the application to Release.

Log into Release, and [create an application](https://github.com/releasehub-com/docs/blob/main/examples/broken-reference/README.md) by clicking on **Create new app**.

![Create an application](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-39b54bf307966baba57d2897edfd515e48e4a5c6%2Fspring-postgres-create-application.png?alt=media\&token=e0f178aa-266b-4041-b18e-7dcbc566080d)

## Name your application and select your repository

Enter a unique name for your application, and pick the forked repository you created earlier, then click **Next step**.

![Enter a name and pick your repository](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-a3e486cbb029ce9b1bd0bfdefe7ac18e2ec3d0ee%2Fspring-postgres-name-and-repo.png?alt=media\&token=3b26404e-b4dc-4497-a4d8-30ad913d4880)

## Pick your services

1. Pick **Analyze the repository**, so Release can create services from `docker-compose.yaml`.
2. Select the branch from your repository you'd like to track in this application.
3. Select the `docker-compose.yaml` file.
4. Click **Start analysis**.

![Pick your services](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-4aad5c62898999be6dd9fc85f6c232cc941a0e86%2Fspring-postgres-pick-services.png?alt=media\&token=45048fa3-f379-421f-bd62-406cb9cb20ab)

Release will read your Docker Compose file, and list the services found:

![Pick your services showing analysis result](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-ea2379f1ba13660e9c43535907374b11f023822a%2Fspring-postgres-pick-services-analyzed.png?alt=media\&token=bc5d2c44-f504-4db8-b211-a65168d8274a)

Click **Next step**.

## Generate a template

Release will generate a template based on the services from your `docker-compose.yaml` file.

![Generate an application template](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-fd71a28f2173baa3480bb53af9c312f66821cce2%2Fspring-postgres-application-template.png?alt=media\&token=4011dab7-e926-4f68-a9c4-516ae780344c)

This template will be used to create new environments, and the defaults should work as expected for most Docker Compose applications.

For this example application, we'll need to make two changes to the template.

### Start `db` before `backend`

The first change makes sure the `db` service always starts before the `backend` service.

The Spring app we're installing runs database initialization scripts on startup. If there's no database to initialize, the `backend` service will fail to start.

The section of the template we want to edit is under `workflows.setup` and `workflows.patch`. Instead of running setup steps in parallel, we want them to follow a set order.

Change `workflows.setup` and `workflows.patch` to look like this:

```yaml
workflows:
- name: setup
  order_from:
    - services.db
    - services.backend
- name: patch
  order_from:
    - services.db
    - services.backend
```

Change `parallelize` to `order_from`, then list the tasks in the order we want Release to execute them: `services.db`, then `services.backend`.

### Expose a container port on `db`

{% hint style="info" %}
Release does not currently convert `expose` directives when importing services from Docker Compose. Add a `container_port` for each port in your Docker Compose file. Read more about [ports](/reference-documentation/application-settings/application-template/schema-definition#ports) in our template schema.
{% endhint %}

Add a container port to the `db` service by adding the lines below:

```diff
  services:
  - name: db
+   ports:
+   - type: container_port
+     port: '5432'
```

This change exposes port `5432` on the database container to the private network between your services.

You can read more about [workflow parallelization](/reference-documentation/application-settings/application-template/schema-definition#workflow-parallelization) in the template schema documentation.

After editing the template, click **Next step**.

## Environment variables

The `docker-compose.yaml` file from our repo lists the environment variables needed for each service. Both services use `POSTGRES_DB`, and the `db` service has one extra variable, `POSTGRES_PASSWORD_FILE`.

### Setting the PostgreSQL database password via an environment variable

The [`postgres`](https://hub.docker.com/_/postgres) Docker image can read a password from a file specified in the `POSTGRES_PASSWORD_FILE` environment variable, or it can read the `POSTGRES_PASSWORD` environment variable directly.

Since Release supports adding [secrets as environment variables](/reference-documentation/environment-settings/environment-specific-environment-variables/secrets), let's use a `POSTGRES_PASSWORD` environment variable instead of a password file.

There's no database password environment variable in the `backend` service from `docker-compose.yaml`, so we'll need to find out how the `backend` service knows the database password.

The Spring Boot application reads the database password from the `backend/src/main/resources/application.properties` file.

In our example application, this file has a database password setting, `spring.datasource.password`, that looks like this:

```bash
spring.datasource.password=${POSTGRES_PASSWORD:db-wrz2z}
```

The line above means that Spring will try to read the database password from the `POSTGRES_PASSWORD` environment variable, but will use the hard-coded value `db-wrz2z` if the variable `POSTGRES_PASSWORD` does not exist.

We can see that `db-wrz2z` matches the password from `db/password.txt` in our repository.

Instead of using this hard-coded password, we can set the `POSTGRES_PASSWORD` environment variable.

You'll notice that both the `db` and `backend` services can read this same environment variable to get the database password.

Let's set it to a random string, like `xUnlL9U9J1Dh`.

### Set your environment variables

![Set your environment variables](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-26f196aeb0fdf13cc545dba438fed5ff53ac645b%2Fspring-postgres-pick-services-env-vars.png?alt=media\&token=b2798d1c-b3e2-4d87-bfc8-c4fa7d3dd07c)

Edit the YAML environment variables in Release to look like this:

```yaml
---
defaults:
  - key: POSTGRES_DB
    value: example
  - key: POSTGRES_PASSWORD
    value: UNwmSZpmo9Aj
    secret: true
services:
  backend: []
  db: []
```

Because both environment variables are used by both services, we don't need to specify them individually for each service. All services get all environment variables listed under the `defaults` section.

You'll notice that we marked `POSTGRES_PASSWORD` as secret. This means that `POSTGRES_PASSWORD` will be saved to an encrypted vault, and hidden from the Release interface after you save your changes.

Click **Next step** to save your environment variables.

## Build arguments

This application does not use any build arguments, so we can click **Next step**.

## Save and deploy

Click **Save and deploy** to create your application.

Release will now pull your repository and run `docker build` to build a Docker image of your `backend` service, before setting up and deploying your two services, `db` and `backend`.

## View the resulting application

After the build, setup, and deployment workflow completes, navigate to your new environment and click on the URL for the backend service.

![Service hosted URL](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-f4d2798f640c6b922f645720dfe22efd3e013aae%2Fspring-postgres-hostname.png?alt=media\&token=34205252-e931-4433-802d-9d3b70d4915f)

This request runs the `controllers.HomeController.showHome` Java function on the `backend` service, which reads a value from the PostgreSQL database running on the `db` service, and returns the string `Hello from Docker!`.

![Hello from Docker!](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-e451f52601cf7d0d5cb366da54523594204b3ef3%2Fspring-postgres-result.png?alt=media\&token=dc099039-ca92-4af7-902f-b754eed73cd5)

## Troubleshooting

If you don't see the result above, you can view the logs for each service in your environment to see whether either service logged any errors.

To view a service's logs, navigate to your environment's details page, scroll down to the list of services, and click on **logs**.

![Service logs](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-e743eccd9a5bf84580c46e4ef220659611207bbb%2Fspring-postgres-service-logs.png?alt=media\&token=9d1302f1-5b7a-4a08-83c5-ab4d7257e08e)

## Further reading

If you would like to learn more about running Spring applications using Docker, view the [Spring Boot with Docker](https://spring.io/guides/gs/spring-boot-docker/) guide from Spring.


# Terraform and Flask

In this tutorial, we'll learn how to deploy a Python Flask application that queries a Redis database hosted by AWS ElastiCache.

Instead of manually configuring Redis on ElastiCache, we'll use Terraform with Release's infrastructure runners to provision a Redis cluster.

We'll start with the [`awesome-release/release-terraform-flask-redis`](https://github.com/awesome-release/release-terraform-flask-redis) application.

## Step 1: Fork the GitHub repo

Fork [`awesome-release/release-terraform-flask-redis`](https://github.com/awesome-release/release-terraform-flask-redis) on GitHub using the account you have authenticated with Release.

## Step 2: Create a new Release application

![Create a new Release application, step one](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-df9e8dbc749e08e654699c42f4c383f20023589b%2Fterraform-example-create-application-one.png?alt=media\&token=892c3ffa-d0be-401b-aa67-23c001c222a2)

1. Enter a unique name for your application.
2. Select the fork you created in step 1.
3. Click **Next Step**.

![Create a new Release application, step two](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-4b476e4d3473ed1f685a291ea61ffc2ee7e74559%2Fterraform-example-create-application-two.png?alt=media\&token=b9b3f89e-3665-4f45-881b-60f20982d07c)

1. Select the `main` branch.
2. Select `compose.yaml`.
3. Click **Start Analysis**.

![Create a new Release application, step three](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-b3553b9d58bda85ff96f3df5226a431e9f526257%2Fterraform-example-create-application-three.png?alt=media\&token=2e5607a9-da3a-4b8a-8319-279b2b60ad24)

1. Click **Next Step**.

![Create a new Release application, step four](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-f0fc67e825ad96652dac2823f3986a5de67c7c2b%2Fterraform-example-create-application-four.png?alt=media\&token=702cd2df-d1e5-4cf2-aafb-22ee850afcd8)

1. Select the `main` branch.
2. Select `.release/application_template.yaml`.
3. Make sure the application's context and domain are correct.
4. Click **Next Step**.

![Create a new Release application, step five](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-e7805ec3ac84b893e00ba28b429857be5c2dda84%2Fterraform-example-create-application-five.png?alt=media\&token=39150af8-832d-41d6-980f-7104257f96a7)

1. Select `.release/environment_variables.yaml`.
2. Click **Next Step**.

![Create a new Release application, step six](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-e2bda2f67cbf49a46e2183b9e6da48cef356e714%2Fterraform-example-create-application-six.png?alt=media\&token=3b2d0091-6da0-4adc-bc6e-7bddb6a984ad)

1. Click **Deploy your app!**

## Step 3: Follow the deployment

![Release deployment terraform](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-de5d3de9a90e453c5abb754b699d8f75ee9ca428%2Fterraform-example-deploy.png?alt=media\&token=0f4dc262-928f-4f98-a31e-2aef684b8416)

Release will build and deploy your application. Open the infrastructure runner step to see output from Terraform.

## Step 4: View the environment

![Release environment detail page](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-f1587c84786d89a32efa88b475b0b2c08824b02f%2Fterraform-example-environment-detail.png?alt=media\&token=b21f3a8f-b7ab-48a9-8743-b9b7909bd388)

View the environment detail page and click on the web URL.

## Result

![Release result](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-1460aad46a69de75d137a6c912fb81d7246a0042%2Fterraform-example-result.png?alt=media\&token=957e5edc-3e71-4dc5-83b8-f3ff0043253b)

You should see HTML output showing the number of times this application has been viewed.

If the resulting view count is more than you anticipated, keep in mind that Release's health checker makes an HTTP request, which counts as a page view in our example.

## Further reading

* [Get started with Terraform](https://docs.releasehub.com/guides-and-examples/advanced-guides/infrastructure/terraform) – Release documentation about using Terraform
* [Infrastructure as code (IaC)](https://docs.releasehub.com/guides-and-examples/advanced-guides/infrastructure) – Release documentation about IaC


# OpenTelemetry demo

Observability is an essential part of any microservice architecture. But when services are built using many divergent technologies, consistent observability becomes a challenge.

OpenTelemetry is a collection of tools that enables developers and DevOps to simplify and standardize the collection of telemetry data.

To demonstrate how OpenTelemetry works for a complex microservices-based application, the OpenTelemetry developers maintain an example application called the [OpenTelemetry Astronomy Shop Demo](https://github.com/open-telemetry/opentelemetry-demo).

The demo is in constant development, with contributors regularly adding more services. This may seem overengineered, but this is the point: To provide a realistic demonstration of how to improve observability of a complex distributed system.

In this tutorial, we'll look at how to host a snapshot of this application on Release.

## The Release fork of the OpenTelemetry Astronomy Shop Demo

We created a [fork of the OpenTelemetry Astronomy Shop Demo](https://github.com/awesome-release/opentelemetry-demo), hosted in our [Awesome Release](https://github.com/awesome-release/) GitHub organization.

We modified this fork by including the following files to simplify adding the demo as a Release application:

| Filename                              | Purpose                                               |
| ------------------------------------- | ----------------------------------------------------- |
| `.release/application_template.yaml`  | A complete application template, generated by Release |
| `.release/environment_variables.yaml` | Default environment variables, generated by Release   |
| `.release.yaml`                       | This file indicates where to find the templates above |

## Adding the demo app using template files

The simplest way to add this demo application to Release is to use the pre-generated application template and corresponding default environment variables.

You'll need an existing [Release account](/getting-started/create-an-account) with a [cluster](/guides-and-examples/advanced-guides/create-a-cluster) and a [domain](/guides-and-examples/domains-and-dns/managing-your-domains).

Follow the steps below to deploy the demo application:

1. Add a new Release application.

![Select your repo and branch](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-923ddd6f24c9d3697d770175d20ea798e6613c52%2Fotel-demo-select-repo.png?alt=media\&token=b26e7cab-bbb7-470d-8986-41f5b1e1d772)

1. Under **Analyze your repository**, skip to the **Next Step**.

![Skip the analysis function](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-e6e7b9639f6c654946d7c3c821d3db2380d0bf9f%2Fotel-demo-skip-analyze.png?alt=media\&token=52ebb32f-1a8f-4acd-80e7-768fe4c7e541)

1. Select `.release/application_template.yaml` from the **Select release template file** dropdown.

![Select the application template](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-933debc8a1b35483045fc2fb8b034a2045bb7fc0%2Fotel-demo-template.png?alt=media\&token=52715234-4bee-4e3e-871b-a199ba06a2e0)

1. Double-check your `context` and `domain` values, then click **Next Step**.
2. Select `.release/environment_variables.yaml` from the **Select release env template file** dropdown, then click **Next Step**.

![Select environment variables template](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-986872f45666c687a968a57f517162d537f547e4%2Fotel-demo-environment-vars.png?alt=media\&token=8a5bbbda-6a2a-4e5b-b316-4287859972dc)

1. Click **Next Step** again to skip over the build arguments form.
2. Click **Deploy your app!**

![Deploy your application](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-99d45fb64c834249416913ae94495c17358e12d7%2Fotel-demo-deploy.png?alt=media\&token=0cb97042-f9ac-4324-9e31-1e17d68ef1f3)

1. Release will now build and deploy your OpenTelemetry demo. This could take up to 20 minutes the first time.
2. Visit the environment detail page.
3. Click the **frontend proxy** URL to see the demo shop.

Reach the demo's public services by visiting the URLs below (substitute your frontend proxy hostname):

* Webstore: `https://your-frontend-proxy-hostename.tld/`
* Grafana: `https://your-frontend-proxy-hostename.tld/grafana/`
* Feature Flags UI: `https://your-frontend-proxy-hostename.tld/feature/`
* Load Generator UI: `https://your-frontend-proxy-hostename.tld/loadgen/`
* Jaeger UI: `https://your-frontend-proxy-hostename.tld/jaeger/ui/`

## Detailed steps: How to prepare a Release Application Template from scratch

We provide the steps we used to generate a new `application_template.yaml` file for this application as reference.

Release can create an application based on an existing Docker Compose file. At the time of writing, Release can import the OpenTelemetry app file, requiring only a few minor changes to the resulting application template:

1. Service names should not contain underscores.
2. Service names are used by Release to create service-specific environment variables. These variables sometimes clash with expected variables from existing apps.

### Use the Docker Compose file to create an application in Release

1. In Release, create a new application.
2. Select your repository and the branch containing your new `compose.yaml` file. Click **Next Step**.

![Select your repo and branch](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-923ddd6f24c9d3697d770175d20ea798e6613c52%2Fotel-demo-select-repo.png?alt=media\&token=b26e7cab-bbb7-470d-8986-41f5b1e1d772)

1. From the **Select files** dropdown, pick only `docker-compose.yaml`.

![Select docker-compose.yaml](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-b57329718b6e1a1d34cdb952a3fcfeedc596b6d4%2Fotel-demo-docker-compose-yaml.png?alt=media\&token=6dbc6f12-92ca-4bcb-b1ab-22ad32bf604b)

1. Click **Start Analysis**. Release will read the `docker-compose.yaml` file and show a list of services.
2. Add `otelcol` as a dependency to the `kafka` service, then click **Next Step**.

![Select services](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-25537fc6d064b64ca26300ef2cb8e086594f3835%2Fotel-demo-kafka-depends-otelcol.png?alt=media\&token=e33d0728-0467-4391-87de-26d4e1b40cc7)

1. Release will create a complete Application Template. Double-check the `context` and `domain` values for your account.

![Generated application template](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-84fac8f6956d3ad04f890eab8eb25ae1577b056d%2Fotel-demo-new-app-template.png?alt=media\&token=a85df1b6-d5d6-476c-8d50-ffb84b1a4d3d)

1. Change `kafka` to `kafkaservice` for each occurrence, except the image name.
2. Change all occurrences of `ffs_postgres` to `ffspostgres`.
3. Add container port 5432 to the `ffspostgres` service:

   ```yaml
   ports:
   - type: container_port
     port: '5432'
     loadbalancer: false
   ```
4. Add container ports 9092 and 9093 to the `kafkaservice` service:

   ```yaml
   ports:
   - type: container_port
     port: '9092'
     loadbalancer: false
   - type: container_port
     port: '9093'
     loadbalancer: false
   ```
5. Fix the `kafkaservice` health check by changing `kafkaservce` hostname to `localhost`:

   ```yaml
   command:
   - nc
   - "-z"
   - kafka
   - localhost
   - '9092'
   ```
6. Remove the extra current directory selector (`./`) from `configmap` repo paths.
7. Mount extra Grafana configuration files:

   ```yaml
   - mount_path: "/etc/grafana/provisioning/dashboards/general.yaml"
   repo_path: "./src/grafana/provisioning/dashboards/general.yaml"
   name: grafana-general-yaml
   - mount_path: "/etc/grafana/provisioning/dashboards/general/demo-dashboard.json"
   repo_path: "./src/grafana/provisioning/dashboards/general/demo-dashboard.json"
   name: grafana-general-demo-dashboard-json
   - mount_path: "/etc/grafana/provisioning/dashboards/general/opentelemetry-collector.json"
   repo_path: "./src/grafana/provisioning/dashboards/general/opentelemetry-collector.json"
   name: grafana-general-otel-collector-json
   - mount_path: "/etc/grafana/provisioning/datasources/default.yaml"
   repo_path: "./src/grafana/provisioning/datasources/default.yaml"
   name: grafana-datasources-default-yaml
   - mount_path: "/etc/grafana/provisioning/datasources/jaeger.yaml"
   repo_path: "./src/grafana/provisioning/datasources/jaeger.yaml"
   name: grafana-datasources-jaeger-yaml
   ```
8. Click **Next Step**.
9. Edit the generated environment variables file, by removing `PUBLIC_OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` key and value from the `frontend` service.
10. Add a new entry for `PUBLIC_OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` under the `mapping` key:

    ```yaml
    mapping:
      PUBLIC_OTEL_EXPORTER_OTLP_TRACES_ENDPOINT: "${OTELCOL_INGRESS_URL}/v1/traces"
    ```
11. Change the value of `FEATURE_FLAG_SERVICE_HOST` to `featureflagservice`.
12. Click **Next Step**.

![Generated environment variables](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-2a9571bc0b2ef1cbc22003f61358e262fa490730%2Fotel-demo-new-environment-vars.png?alt=media\&token=1c44ec9d-b873-4248-9571-b61b7f13102b)

1. Skip the build arguments form by clicking **Next Step** again.
2. Click **Deploy your app!**

![Deploy your application](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-99d45fb64c834249416913ae94495c17358e12d7%2Fotel-demo-deploy.png?alt=media\&token=0cb97042-f9ac-4324-9e31-1e17d68ef1f3)

Release will now build and deploy your demo application.

### View your running application

Reach the demo's public services by visiting the URLs below (substitute your frontend proxy hostname):

* Webstore: `https://your-frontend-proxy-hostename.tld/`
* Grafana: `https://your-frontend-proxy-hostename.tld/grafana/`
* Feature Flags UI: `https://your-frontend-proxy-hostename.tld/feature/`
* Load Generator UI: `https://your-frontend-proxy-hostename.tld/loadgen/`
* Jaeger UI: `https://your-frontend-proxy-hostename.tld/jaeger/ui/`


# Load balancer with hostname

Exposing HTTP and non-HTTP-based services to the internet

### Introduction

Release will automatically handle services that are HTTP-based and generate hostnames for the backend services. But not all services are HTTP: many services listen for other types of traffic, like TCP or UDP traffic. There are also cases where you may want to avoid using a CDN or bypass the Nginx Ingress for even HTTP services. There are several cases where a TCP, UDP, or HTTP/GRPC service may not be suitable for regular load balancers and ingresses. In AWS, these specific load balancers are implemented with Network Load Balancers (NLB) and in GCP with the L4 Load Balancer.

In order to expose a service to the internet, you will need to define a `node_port`. A `node_port` requires both a `target_port` and a `port` number.

* `target_port`: This is the port on the pod that the request gets sent to internally. Your application needs to be listening for network requests on this port for the service to work.
* `port`: The specified port within the cluster that exposes the service externally. The service is visible on this port, and other pods and services will send requests to this port to reach the service. The load balancer directive will listen on this port and forward the requests to the `target_port`.
* `tls_enabled`: Whether the load balancer will negotiate (and possibly offload) TLS encryption on the frontend.
* `backend_protocol` should be set to either `tcp` for plaintext backend requests or `tls` for encrypted backend requests.
* Release doesn't define a fixed `NodePort` in Kubernetes, which allows Kubernetes to allocate a random port from the host node. However, for external communications, the fixed port is applied to the load balancer. This allows services that run on the same port number to coexist with other applications on the same physical nodes without conflict.

Once you have defined a `node_port` for your service, you can define a `hostname`. Release uses variable interpolation for `env_id` and `domain`.

There are detailed instructions in the [reference documentation](/reference-documentation/application-settings/application-template/schema-definition#loadbalancer).

### HTTP/S, and GRPC/s Services

Typically, you can expose HTTP services via the default ingress and CDN options in Release. However, several cases exist where these cannot be implemented well. For example, an HTTP service that requires extremely large payloads (10+MiB) for uploads or downloads may cause the CDN or NGINX ingress to timeout or reject the request. Another use case is an HTTP server that responds in minutes and could leave connections open for hours. Another example that is becoming increasingly common is a GRPC endpoint that is not compatible with typical HTTP-only load balancers.

#### HTTP example

In this case, you can specify an HTTP service load balancer with the following configuration. Of particular note, you will see that the `node_port` is mapped from the standard 80 to 3000 running in the container. The load balancer port will listen on 80 and send the request to the container running on port 3000. The annotations are given to show specific overrides that are available to users in either AWS or GCP.

```yaml
  services:
    - backend:
      image: "..."
      ports:
        - type: node_port
          target_port: "3000"
          port: "80"
      loadbalancer:
        type: http
        visibility: private
        hostnames:
        - backend-${env_id}.${domain}
        - api-${env_id}.${domain}
        annotations:
          service.beta.kubernetes.io/aws-load-balancer-healthcheck-protocol: TCP # AWS
          service.kubernetes.io/healthcheck: k8s2-pn2h9n5f-l4-shared-hc # GKE
```

#### HTTPS with offload example

In another case, you can specify an HTTPS TLS offload service. This allows the load balancer to listen on a TLS port with your custom certificate and pass the unecrypted traffic to the container. This section is identical to above but you will see `tls_enabled: true` and `port: 443` are different.

```
services:
    - backend:
      image: "..."
      ports:
        - type: node_port
          target_port: "3000"
          port: "443"
      loadbalancer:
        type: http
        visibility: private
        tls_enabled: true
        hostnames:
        - backend-${env_id}.${domain}
        - api-${env_id}.${domain}
```

#### GRPCS end-to-end encrypted example

You may want to enable end-to-end encryption for secured communication by setting the `backend_protocol: tls` parameter.

```
services:
    - backend:
      image: "..."
      ports:
        - type: node_port
          target_port: "3000"
          port: "443"
      loadbalancer:
        type: grpc
        visibility: private
        tls_enabled: true
        backend_protocol: tls
        hostnames:
        - backend-${env_id}.${domain}
        - api-${env_id}.${domain}
```

#### Generic Layer7 Example

Lastly, you may want to take a more hands-off approach so the load balancer does not perform an ALPN or other HTTP negotiation on your behalf. Simply use the `type: layer7` load balancer.

```
services:
    - backend:
      image: "..."
      ports:
        - type: node_port
          target_port: "3000"
          port: "80"
      loadbalancer:
        type: layer7
        visibility: private
        hostnames:
        - backend-${env_id}.${domain}
        - api-${env_id}.${domain}
```

### Non-HTTP services

Here is an example of exposing a Minecraft service to the internet:

```yaml
services:
- name: minecraft
  image: dustyspace/docker-minecraft-server/minecraft
  has_repo: true
  ports:
  - type: node_port
    target_port: '25565'
    port: '25565'
  loadbalancer:
    type: layer4
    visibility: public-direct
    hostname: minecraft-${env_id}-${domain}
```

The Minecraft service listens on port 25565, creates a load balancer, and generates a hostname.

The code below is an example of exposing a postgres database service accessible privately via your VPC (either with other services outside your cluster, or using VPC peering, or even available via a VPN tunnel into your account):

```yaml
services:
- name: db
  image: postgres:9.4
  ports:
  - type: node_port
    target_port: '5432'
    port: '5432'
  loadbalancer:
    type: layer4
    visibility: private
    hostname: postgres-db-${env_id}-${domain}
```


# Static JavaScript service

How to define a new static JavaScript service or convert an existing Docker service to use a static JavaScript build deployed to a CDN

## Define a static JavaScript service

Creating a static JavaScript service in Release requires the following parameters:

* `static`: `true` for a static build, `false` to switch back to a Docker build.
* `build_base`: the path to the JavaScript project in your repository.
* `build_command`: your JavaScript build command.
* `build_output_directory`: the directory the build command outputs the build to.

Here's an example static build:

```yaml
services:
- name: frontend
  static: true
  build_command: GENERATE_SOURCEMAP=false yarn build
  build_base: frontend
  build_output_directory: build/
```

### Converting an existing Docker service to static JavaScript

Release allows you to define a service with both static JavaScript and Docker service parameters. You can also easily convert any single page JavaScript Docker service to a static JavaScript build.

Here is a Docker-based JavaScript frontend that was generated from a `docker-compose` by Release:

```yaml
services:
- name: frontend
  image: acme-co/awesome-project/frontend
  ports:
  - type: node_port
    target_port: '3000'
    port: '3000'
  has_repo: true
  memory:
    limits: 4Gi
    requests: 100Mi
```

You can update this to use static builds by adding the same parameters for defining a static JavaScript service as outlined above. Use the `static` parameter to switch the service between a static JavaScript build and running in a Docker container.

```yaml
services:
- name: frontend
  image: acme-co/awesome-project/frontend
  ports:
  - type: node_port
    target_port: '3000'
    port: '3000'
  has_repo: true
  memory:
    limits: 4Gi
    requests: 100Mi
  static: true
  build_command: yarn build
  build_base: frontend
  build_output_directory: build/
```

### Setting the NodeJS Javascript version

During builds, you can control the version of NodeJS that is used by supplying the version information in a `.nvmrc` file. We recommend setting the file at the root of your project, but you can also create the file at the subtree where your project lives. You can read the [documentation](https://github.com/nvm-sh/nvm#nvmrc) to find more details. This is a simple example that will use the version that you specify from your local version:

```shell
cd /project/root
node -v > .nvmrc
git add .nvmrc
git commit -m "Set javascript version"
git push
```

## Deprecated

This section is deprecated and may be removed at a later date.

### Adding basic authentication to static sites

If you want the benefits of a static CDN site but don’t want your site to be public, Release has a simple way to add [basic authentication](https://en.wikipedia.org/wiki/Basic_access_authentication) to your static content via our proprietary Edge Routing System.

{% hint style="warning" %}
Static site basic authentication does not allow for multiple users with their own passwords. It is only suitable for a single shared user and password scheme. It is not suitable for production environments.
{% endhint %}

{% hint style="danger" %}
Basic authentication does not use encryption to secure usernames and/or passwords. It is not suitable for applications running in production.
{% endhint %}

If you would like to add basic authentication to your site, set up a static application as shown in the previous section. Then, find the `hostnames` entry for your static application as shown in this example:

```yaml
hostnames:
  - api-${env_id}-${domain}
  - web-client-${env_id}-${domain}
```

Replace the `web-client-...` line with the following example:

```yaml
hostnames:
  - api-${env_id}-${domain}
routes:
- name: web-client
  hostname: web-client-${env_id}-${domain}
  auth:
    username: admin
    password: letmein
  rules:
  - service: web-client
    path: "/"
```


# SSH bastion access to services

Access namespace services like pods or databases for maintenance

You can add SSH bastion to an environment to allow team members to securely access resources in the environment, for example, to run utilities accessing backend services like database containers or to perform administrative commands like starting and stopping jobs on private containers not connected to the public internet.

{% hint style="danger" %}
We do not recommend using bastion access in critical environments, such as staging or production. SSH access is usually unaudited and has elevated powers that could be used to cause harm to your services and environments. Most customers do not need SSH bastion access for their environments.

It is a mistake to believe that VPNs and SSH bastions make access to your environments more secure. In reality, these connections may expose your environments to additional risk, and you should keep this in mind if you choose to add SSH bastion access to your application or environment.
{% endhint %}

## Create bastion service

Navigate to the Application Template settings to create a bastion service that will run an SSH image. This example creates a service for you:

```yaml
- name: bastion
  image: binlab/bastion
  command:
  - sh
  - "-c"
  - >-
    cp /var/lib/bastion/public-key /var/lib/bastion/authorized_keys && 
    chmod 600 /var/lib/bastion/authorized_keys &&
    chown bastion:bastion /var/lib/bastion/authorized_keys &&
    bastion
  ports:
  - type: node_port
    target_port: '22'
    port: '22'
    loadbalancer: true
  hostname: bastion-${env_id}-${domain}
```

Let's take a look at these configuration directives:

* `name` is the name of the service.
* `image` refers to [binlab's public SSH container](https://github.com/binlab/docker-bastion).
* `command` provides a series of bash commands that will be run to copy the keys from a known location (keys will be uploaded in the next step) and start the bastion service.
* `ports` specifies that the service will listen on port `22`, which is standard for SSH.
* `hostname` describes the hostname that will be generated for the bastion service.

## Create and upload public keys to gain access

Next we'll use a [just-in-time file mount](/reference-documentation/application-settings/file-mounts) to upload the public keys that will be used to access the bastion.

Create a text file on your computer called `public-key` with no file extension. Add the list of public SSH keys to this file, placing each key on a new line. An example file with two keys might look like this:

```
ssh-rsa AAAAB3Nza...abcd== User1
ssh-rsa AAAAB3Nza...uvwxyz User2
```

Now navigate to App Settings and scroll down to the "Just-in-time File Mounts" section. Upload the `public-key` file with the file directory `/var/lib/bastion/` and make sure you select the `bastion` service checkbox. You do not need to select **Secret**, because this file only contains public keys which are not secrets.

![Create the file mount with the public keys and save the file](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-c78ded26c98059b86db472452a3b25be582d750f%2Fjit_file_mount_example.png?alt=media\&token=0efd0f61-3cd0-4790-870c-38f7140173e3)

## Connect to the bastion

Once you have applied the services to deploy a new environment or update and existing environment, navigate to the environment's details page. You'll find the hostname for the bastion service in the "Hostname URLs" section.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-e4c0323e6ba4ed494d4d101ccda39815cdc86ea2%2Fenvironment-details-app-import.png?alt=media\&token=dfe1b991-8708-42e4-9d09-0f1580936efe)

Copy the hostname for the bastion service. You can now use an SSH terminal to connect as the user `bastion` as follows:

```
$ ssh bastion@bastion-staging-releaseapp.io
The authenticity of host 'bastion-staging-releaseapp.io (XX.YY.ZZZ.WWW)' can't be established.
ECDSA key fingerprint is SHA256:KKTfemSDp1s.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'bastion-staging-releaseapp.io,XX.YY.ZZZ.WWW' (ECDSA) to the list of known hosts.
Welcome to Bastion!

bastion-c4dc7-cx:~$
```

You can now execute commands on the bastion to reach hosts beyond the bastion server.

## Optional: Use the bastion as a jump host

The SSH bastion supports a local configuration you can enable to proxy through the bastion transparently. If you would like to learn more using a proxy jump host, take a look at Tecmint's [How to Access a Remote Server Using a Jump Host](https://www.tecmint.com/access-linux-server-using-a-jump-host/).


# ngrok and OAuth for private tunnels

Protect private services in Release with ngrok

You may need a simple way to keep the applications you deploy to Release ephemeral environments private. Perhaps you're deploying a basic disposable admin tool, for example, or you would like to share a link with a potential client or vendor outside your organization. In such cases, you don't want to set up expensive access controls that need time-consuming configuration.

The solution is to create a proxy layer for your Release application environment.

## ngrok vs. open-source proxy servers

We have a guide available for an [open source OAuth2 server](/guides-and-examples/common-setup-examples/opensource-oauth-proxy).

Alternatively, take a look at this [list of open source tunneling solutions](https://github.com/anderspitman/awesome-tunneling#open-source-at-least-with-a-reasonably-permissive-license) you can investigate.

We chose ngrok for this guide because it offers a simple, lightweight, and cost-effective way to securely expose an application without too much additional configuration or infrastructure maintenance.

## What you'll need

Here's what you'll need to follow along:

* **An ngrok account.** A free account may work for once-off environments. A business license is better for multiple accounts in a wildcard setup. Visit [ngrok's pricing page](https://ngrok.com/pricing) for more details.
* **An ngrok token.** Preferably a token generated specifically for this use case so it can be revoked or changed without impacting a regularly used token somewhere else.
* **A Release application** running in your Release account.
* **A service that you would like to expose via the proxy tunnel.**
* **An OAuth, SAML, or OIDC service provider,** such as Google Workspaces – this is optional, as it may incur vendor charges.
* **A custom domain name for your private pages,** this is also optional, as it may incur vendor charges.

## Set up

Say you want to expose the following service via the proxy tunnel:

```yaml
services:
- name: docsbuild
  image: mycoolrepo/mycoolapp/docsbuild
  registry: local
  has_repo: true
  ports:
  - type: container_port
    port: '8080'
  build:
    context: "."
    dockerfile: docs/Dockerfile
```

The `docsbuild` service is not for public consumption, but we would like to view it along with our code deployments in pre-production settings. The main application is configured with authentication and access controls, but the documentation is a set of web pages and would be difficult to protect. If you added a regular `node_port:` and `hostnames:` section to your application, the documentation would be exposed to the internet.

Let's create a private tunnel for an additional layer of protection instead.

### Add the ngrok secure tunnel

We can expose the application via ngrok using a [simple container](https://hub.docker.com/r/wernight/ngrok/). Instantiate this service alongside your other services in the application template as follows:

```yaml
services:
- name: docsbuild-tunnel
  image: wernight/ngrok
```

Next, add the following to your application's environment variables template:

```yaml
services:
  docbuild-tunnel:
  - key: NGROK_AUTH
    value: mysecretauthtoken
    secret: true
  - key: NGROK_LOOK_DOMAIN
    value: docsbuild
  - key: NGROK_PORT
    value: '8080'
  - key: NGROK_BINDTLS
    value: "true"
```

Here, the `NGROK_AUTH` key has the value of your ngrok token. We use `NGROK_LOOK_DOMAIN` to identify the service to expose via the ngrok tunnel, in this case it is `docsbuild`. The port is specified as `8080`. By default ngrok will use http (you could specify TLS or TCP as well). Therefore the full url to your internal service in this example would be `http://docsbuild:8080`. You could consider adding a [sidecar](https://docs.microsoft.com/en-us/azure/architecture/patterns/sidecar) and pointing to `http://localhost` for even more privacy. The `NGROK_BINDTLS` option tells ngrok to redirect from HTTP to HTTPS and only support `https:` traffic externally.

Add both services to your `setup` workflow, but don't add the tunnel service to your patch flow, since it doesn't need to be restarted during deploys:

```yaml
workflows:
- name: setup
  paralellize:
  - step: docs
    tasks:
    - services.docsbuild
    - services.docsbuild-tunnel
    wait_for_finish: false
- name: patch
  paralellize:
  - step: docs
    tasks:
    - services.docsbuild
    wait_for_finish: false
```

### Find the tunnel address

Exposing the tunnel service logs will reveal the one-time URL in ngrok.io similar to the following:

```
ngrok by @inconshreveable

Tunnel Status                 online
Version                       2.0/2.0
Web Interface                 http://localhost:4040
Forwarding                    https://abc.ngrok.io -> docsbuild:80

Connnections                  ttl     opn     rt1     rt5     p50     p90
                              0       0       0.00    0.00    0.00    0.00
```

Point your browser to the `Forwarding` URL to view your documentation pages.

{% hint style="info" %}
You can also expose the ngrok dashboard that runs on port 4040 by using `hostnames:`, or by using another tunnel pointed at the ngrok tunnel server. The ngrok tunnel dashboard could be used for various development or debugging services, but the dashboard itself will be exposed on the internet and may need to be protected too.
{% endhint %}

## Release's tips for protecting private services

Here are our tips for securing private tunnels.

### 1. Use an easy subdomain

You may want to use a well-known custom domain name for your application tunnel endpoint, but this could incur additional costs and require an upgrade to your ngrok account.

You could instead add a subdomain directive to your environment variable template:

```yaml
services:
  docsbuild-tunnel:
  - key: NGROK_SUBDOMAIN
    value: mydocs
```

Which yields something like this output:

```
ngrok by @inconshreveable

...
Forwarding                    https://mydocs.ngrok.io -> docsbuild:80
```

You can also use [Environment variable mappings](/reference-documentation/environment-settings/environment-specific-environment-variables/environment-variable-mappings)to create a link to the tunnel port like this

```
mapping:
  NGROK_SUBDOMAIN: RELEASE_ENV_ID
```

You will also need to add [Hostnames and rules](/guides-and-examples/advanced-guides/hostnames-and-rules) section like the following to your application template so that you can easily click the link in the UI or pull request comments.

```
hostnames:
- ${env_id}.ngrok.io
```

### 2. Use a custom branded domain

For an even better domain name option, use a fully customised hostname. Please note this probably requires an upgrade to a paid ngrok account and features.

#### Prerequisites

A quick diversion here is required since the custom domain names will not be managed by Release (we have no way of knowing what the ngrok tunnel addresses will be). You will need to setup CNAME record(s) in your cloud DNS provider, and a custom domain and a TLS certificate in the ngrok dashboard.

Make sure you follow [ngrok's set up instructions](https://ngrok.com/docs/guides/how-to-set-up-a-custom-domain).

#### Setup example

```yaml
services:
  docsbuild-tunnel:
  - key: NGROK_HOSTNAME
    value: mydocs.private.example.com
```

Which yields something like this output:

```
ngrok by @inconshreveable

...
Forwarding                    https://mydocs.private.example.com -> docsbuild:80
```

You can also follow the same instructions as above for a mapping and hostnames section. This will avoid having to hardcode the values in your configuration file. These two sections should help guide you.

```
mapping:
  NGROK_HOSTNAME: DOCSBUILD_INGRESS_HOST
```

```
domain: example.com
hostnames:
- ${env_id}.private.${domain}
```

### 3. Use password protection

Add a basic authentication password:

```yaml
services:
  docsbuild-tunnel:
  - key: NGROK_USERNAME
    value: "myuser"
  - key: NGROK_PASSWORD
    value: "mypassword"
    secret: true
```

### 4. Use OAuth protection

Please note that these features are an upgrade in the ngrok product feature and may require additional costs. Follow the ngrok documentation to [configure an Edge](https://ngrok.com/docs/cloud-edge#creating-an-edge), then associate the `NGROK_AUTH` token with an ACL to attach to the custom Edge. You can add various Edge modules, including Mutual TLS, IP restrictions, OIDC, SAML, OAuth, and more.


# Using OAuth Proxy

[OAuth2 Proxy](https://oauth2-proxy.github.io/oauth2-proxy/) is:

> A reverse proxy and static file server that provides authentication using Providers (Google, GitHub, and others) to validate accounts by email, domain or group.

You can deploy OAuth2 Proxy to secure your services in Release. This is handy if a service doesn't have its own authentication mechanism.

## Configuring your Application Template

We'll use the Release [`apache-php` example](https://github.com/awesome-release/apache-php) from the `awesome-release` library to demonstrate.

After import, Release creates an Application Template containing the following:

```yaml
---
repo_name: awesome-release/apache-php
hostnames:
- web: web-${env_id}-${domain}
services:
- name: web
  has_repo: true
  static: false
  ports:
  - type: node_port
    target_port: '80'
    port: '80'
  build:
    context: app
```

We can add `oauth2-proxy` to this as a service alongside our `web` service.

```yaml
- name: oauth2-proxy
  image: quay.io/oauth2-proxy/oauth2-proxy
  ports:
  - type: node_port
    port: 4180
    loadbalancer: false
```

Replace the current hostname that points directly to the web service, and have it point to `oauth2-proxy` instead. Find the following:

```yaml
hostnames:
- web: web-${env_id}-${domain}
```

And change it to the following:

```yaml
hostnames:
- oauth2-proxy: web-${env_id}-${domain}
```

The Application Template should now look like this:

```yaml
---
repo_name: awesome-release/apache-php
hostnames:
- oauth2-proxy: web-${env_id}-${domain}
services:
- name: web
  has_repo: true
  static: false
  ports:
  - type: node_port
    target_port: '80'
    port: '80'
  build:
    context: app
- name: oauth2-proxy
  image: quay.io/oauth2-proxy/oauth2-proxy
  ports:
  - type: node_port
    port: 4180
    loadbalancer: false
```

## Configuring your default environment variables

Next, we'll configure OAuth by setting some environment variables.

Find all the available settings in the [OAuth2 Proxy documentation](https://oauth2-proxy.github.io/oauth2-proxy/docs/configuration/overview/#environment-variables).

In this example, we'll use Google as our OAuth2 provider but configuring any other service will be nearly the same.

```yaml
services:
  oauth2-proxy:
  - key: OAUTH2_PROXY_PROVIDER
    value: google
  - key: OAUTH2_PROXY_OIDC_ISSUER_URL
    value: https://accounts.google.com
  - key: OAUTH2_PROXY_CLIENT_ID
    value: << YOUR_CLIENT_ID >>
  - key: OAUTH2_PROXY_CLIENT_SECRET
    value: << YOUR_CLIENT_SECRET >>
    secret: true
  - key: OAUTH2_PROXY_COOKIE_SECRET
    value: << YOUR_COOKIE_SECRET >>
    secret: true
  - key: OAUTH2_PROXY_COOKIE_DOMAINS
    value: ".release.com"
  - key: OAUTH2_PROXY_EMAIL_DOMAINS
    value: release.com
  - key: OAUTH2_PROXY_WHITELIST_DOMAINS
    value: ".release.com"
  - key: OAUTH2_PROXY_HTTP_ADDRESS
    value: 0.0.0.0:4180
  - key: OAUTH2_PROXY_UPSTREAMS
    value: http://web:80
  - key: OAUTH2_PROXY_SKIP_PROVIDER_BUTTON
    value: true
mapping:
  OAUTH2_PROXY_REDIRECT_URL: ${OAUTH2_PROXY_INGRESS_URL}/oauth2/callback
```

Generate `YOUR_COOKIE_SECRET` by running `python -c 'import os,base64; print(base64.urlsafe_b64encode(os.urandom(16)).decode())'`.

Follow [the OAuth2 Proxy docs to set up Google OAuth2 credentials](https://oauth2-proxy.github.io/oauth2-proxy/docs/configuration/oauth_provider#google-auth-provider) to get `YOUR_CLIENT_ID` and `YOUR_CLIENT_SECRET`.

When you set up your Google OAuth2 credentials, you will be asked for the authorized JavaScript origin and redirect URI.

```
Authorized redirect URIs is the location of oauth2/callback ex: https://internal.yourcompany.com/oauth2/callback
```

As each Release environment generates its own domain, you can handle this using environment handles or by updating environment variables.

### Using environment handles

Release [environment handles](https://docs.releasehub.com/reference-documentation/account-settings/environment-handles) is probably the best bet for truly ephemeral environments with OAuth2 support.

You can predefine a handful of environment names like `dev1`, `dev2`, `dev3`, and so on, or `jupiter`, `mars`, `venus`. Then you can predefine your OAuth2 credentials for each of these predefined environments and your domains will match up correctly.

### Updating environment variables

If you don't want to use environment handles, you can deploy a new environment in Release as normal and once that new environment is created, you can plug the generated domain into your OAuth2 settings, update the environment variables for that specific environment, and redeploy it.

The environment will restart with the correct OAuth2 settings for that temporary domain. The settings will work until that environment disappears.

## Accessing your service

Once you have configured OAuth2, you can access the generated URL (in our case, `https://web-jupiter-mydomain.com/`) and you will be automatically redirected to the Google Login screen.

Log in with an email that belongs to one of the domains that you configured your OAuth2 credentials for, and you should be redirected to your service.

## Further Configuration

Take a look at all the [configuration options](https://oauth2-proxy.github.io/oauth2-proxy/docs/configuration/overview#command-line-options) available for OAuth2.


# Hybrid Docker and static site

In this guide, we'll create a Release application that combines two Docker services with a JavaScript SPA (single-page application).

## Fork the repository on GitHub

1. On GitHub, navigate to our [awesome-release/react-express-mysql](https://github.com/awesome-release/react-express-mysql) example repository.
2. Click **Fork** in the top-right corner to create your fork.

![A screenshot of GitHub with the fork button highlighted](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-1dce9407db54b1d0eb676147a33c5eca90360530%2Fhybrid-app-github-fork.png?alt=media\&token=31ed123c-8053-4203-b280-7b4189b3d264)

## Create and configure a new application in Release

1. Log in to your Release account.
2. Click **Create New App** on the Getting Started page.

![A screenshot of the Release Getting Started page, with the create new app button highlighted](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-12fe8cd7c2301e17e951e4e27f60bdcfa5d8eb69%2Fhybrid-app-create-application.png?alt=media\&token=66e1ac7b-47a7-45a0-832f-3ca3ea6b64f7)

### Create your application

1. In the **Name your application** field, enter a name for your new application.
2. Under **Pick your repository**, select your forked repository.
3. Click **Next Step**.

![A screenshot of the first page of the Release new application workflow](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-f39f284cc9dfdf6785aa9bb9880280d2941bc05b%2Fhybrid-app-create-name.png?alt=media\&token=ec72c9d8-5420-4fab-84aa-520ef3f1bbaa)

### Analyze your repository

1. Under **Select branch**, select your main branch.
2. Click the **Select files** dropdown.
3. Select the `docker-compose.yaml` file.
4. Select the `frontend/package.json` file.
5. Click **Start Analysis**.

![A screenshot of the repository analysis step in the Release new application workflow](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-927f8af155d785d88dc37924f8ef4c834804a27c%2Fhybrid-app-analyze.png?alt=media\&token=4eb8a916-d77b-4bac-b564-33f4dd00fe83)

### Update frontend service type

Once the repository analysis completes, a list of services will appear.

1. Click on the service type selector for the frontend service.
2. Select **Static** from the list of service types.
3. Click **Next Step**.

![A screenshot of the Release new application workflow, showing how to select a static service type](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-c664a41fba33aa7a6f1781877ac2d7d189909893%2Fhybrid-app-services.png?alt=media\&token=7fa94e03-ba34-43b8-90d5-b68743347164)

### Add a port to the db service in the application template

Release creates an application template based on the services you selected by reading the `docker-compose.yaml` and `package.json` files.

The `docker-compose.yaml` file did not specify a port for the `db` service, so we'll need to add one to the Application Template.

Edit the Application Template and add the following to the `db` service.

```yaml
  ports:
  - type: container_port
    port: 3306
```

![A screenshot of the Release application template editor](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-6dc3a2541f3e8a70abe08442a9295a5b18a9b023%2Fhybrid-app-port.png?alt=media\&token=6b14d8de-9458-48c6-b240-14a96f77f5ef)

This setting opens a port on the `db` service to the private network for your environment so that the `backend` service can reach the MySQL database running on `db`.

Click on **Next Step**.

### Update the default environment variables

Release generates a default environment variables file for environments in your application.

The single-page application's frontend build process references an environment variable `REACT_APP_BACKEND_URL`. This is to let the frontend SPA know where to make backend API calls.

Using the `mapping` section in the default environment variables file, we can map existing environment variables to new names.

One of the default environment variables that Release creates for all environments is an ingress URL for each service with node ports.

In this application, we want to map the `backend` ingress URL to a new name: `REACT_APP_BACKEND_URL`.

Do this by updating the `mapping` stanza as follows:

```yaml
mapping:
  REACT_APP_BACKEND_URL: BACKEND_INGRESS_URL
```

![A screenshot of the Release environment variables editor](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-ccb303a55d9ce8b7d1ef33c2fd2f0280dae68336%2Fhybrid-app-environment-variables.png?alt=media\&token=2bb3b36e-536b-4ea2-a68b-58e0d0771b01)

Click on **Next Step**.

Skip over the Build Arguments page by clicking **Next Step** again.

### Deploy your application

Click on the **Deploy your app** button.

![A screenshot of the Release deploy button](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-04e01d9a9376ffa8c2774216c41b5e0326a696b1%2Fhybrid-app-deploy.png?alt=media\&token=8f864f32-8f74-4b94-baf7-2b7204362413)

Release will redirect to a deployment progress screen, where it will build and deploy your application.

You may notice that the `db` service fails to deploy.

If you inspect the `db` service's logs, you'll find a line indicating that the database can't find a password file:

```
/usr/local/bin/docker-entrypoint.sh: line 36: /run/secrets/db-password: No such file or directory
```

## Add a database password file called `db-password`

1. On your local machine, create a plain text file called `db-password` with a short random string to act as the MySQL root password.
2. Save the `db-password` file on your machine.

### Add the `db-password` file as a just-in-time file mount on Release

Click on **Settings** in the application menu, then scroll down to the **Just-in-time File Mounts** section.

![A screenshot of Release's JIT file mount settings](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-215b144dc48eff87c3f0bae7651aa52467b976ed%2Fhybrid-app-jit-file-mount.png?alt=media\&token=53905225-f4db-450b-a5df-31d92791f32a)

1. Click on the **Just-in-time File Mounts** section heading to expand the settings.
2. In the **mount path** field, enter `/run/secrets/`.
3. Click the **Select file** button and select the `db-password` file from your computer.
4. Tick the **backend** and **db** service checkboxes.
5. Tick the **Secret** checkbox.
6. Click the **Upload** button.

## Create a new environment

A file mount can only be added to newly created environments, so we'll need to delete the existing environment that Release created and create a new one.

1. Navigate to **Environments**.
2. Click the **delete** icon.
3. In the popup, click on the **Yes, delete environment** button.

![A screenshot showing how to delete an environment on Release](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-f127a02d25aa3f9947edddca786a1c362889be0a%2Fhybrid-app-delete-environment.png?alt=media\&token=9fb7ee6a-3d06-4050-a4d2-bd597f9e4b05)

Next, we'll create a new environment. Click on **Environments** and then click **Create new environment**.

In the popup:

1. Select the `main` branch.
2. In the **Environment type** select box, select **Ephemeral**.
3. Click the **create environment** button.

![A screenshot of the new environment settings popup on Release](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-8ce55e33ded6b08aa901c2909465178f849c76ae%2Fhybrid-app-create-environment.png?alt=media\&token=6a998b36-d047-44cf-a122-bd9d4806cc4e)

Release will build and deploy a new environment.

## View your app

When the deployment completes, click on the **go to environment** button in the popup.

![A screenshot of the popup that shows a new deployment was successful](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-e6d0efffdff08d98bcdeced926179a401eebb8ce%2Fhybrid-app-deployed.png?alt=media\&token=e359846e-9269-40c9-bb75-cb6e948d7d38)

On the environment details page, click on the frontend URL.

![A screenshot of the environment details page, with the frontend URL highlighted](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-aa12b13ec4fbd809f97f3f47786657ad8a508622%2Fhybrid-app-environment-details.png?alt=media\&token=53779f05-8b26-4a8d-9d55-51f32c5941a9)

You should now see the single page app with a response from the backend that reads "Hello from MySQL 8.0.19".

![A screenshot of the resulting application](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-3345a933e12c40dce4a27a6f4250f0c9e520e5c2%2Fhybrid-app-result.png?alt=media\&token=4aae74e6-369f-4d56-a7c5-d3442ca85f24)


# App Imports: Connecting two applications

Release's App Imports functionality allows you to connect two applications with code in different repositories. You can read more about how App Imports is implemented in our [microservices architecture](/guides-and-examples/advanced-guides/microservices-architecture) documentation.

Here we'll show you an example of App Imports in action, using two repositories from our [awesome-release](https://github.com/awesome-release) directory:

* The [docker-create-react-app](https://github.com/awesome-release/docker-create-react-app) will act as our frontend application.
* The [rails\_postgres\_redis](https://github.com/awesome-release/rails_postgres_redis) will act as our backend, with worker process provided by [sidekiq](https://github.com/mperham/sidekiq).

### Release's matching branches heuristic

Let's take a moment to understand how Release chooses which branch to use when automatically creating an environment for connected apps with different repositories.

Release will try to match a branch in the imported repository to the name of the importing repository. If there is no match, Release will fall back to either the [`tracking_branch`](/reference-documentation/application-settings/application-template/schema-definition#tracking_branch) or the default branch. You can also use the [`tracking_tag`](/reference-documentation/application-settings/application-template/schema-definition#tracking_tag).

Here's an example to demonstrate. Say we have RepoA, RepoB, and RepoC with the following branches:

| Repository | Default Branch | Tracking Branch | Other Branches       |
| ---------- | -------------- | --------------- | -------------------- |
| RepoA      | main           | main            | feature-1, feature-2 |
| RepoB      | develop        |                 | feature-2            |
| RepoC      | v5             | bug-1           | feature-1, bug-2     |

* If an automatic environment request for RepoA on `main` is created, Release will select the `develop` default branch for RepoB and the `bug-1` tracking branch for RepoC.
* If an automatic environment request for RepoA on `feature-1` is created, Release will select the `develop` default branch for RepoB and the matching `feature-1` branch for RepoC.
* If an automatic environment request for RepoA on `feature-2` is created, Release will select the `feature-2` matching branch for RepoB and the `bug-1` tracking branch for RepoC.
* If an automatic environment request for RepoB or RepoC is created, the results will be similar depending on how the separate imported apps are designed. For instance, your frontend may always import the backend and a middleware proxy, but the backend will only import the proxy automatically, and not the frontend.

In our example, RepoA imports RepoB and RepoC, but it is not clear how RepoB or RepoC imports the others. If you're importing bi-directionally (or meshed, as in our three-way example), the same heuristic would apply to RepoB, RepoC, and RepoA.

## Fork the repositories on Github

Navigate to the [Release GitHub repository for our frontend application](https://github.com/awesome-release/docker-create-react-app).

Click the **Fork** button in the top right corner of the page to fork the repository.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-3089188cf1faf7b636e0b804bd40858bc1c0e295%2Ffork.jpg?alt=media\&token=df4d9d7f-527c-4153-b196-130829bb2f70)

Do the same for the [GitHub repository for our backend application](https://github.com/awesome-release/rails_postgres_redis).

You will need to grant access to the newly forked repositories. Follow the instructions in [connect your source control provider](/getting-started/quickstart#connect-your-source-control-provider) to do so.

## Create the backend application

The first application we'll create will be from the `rails_postgres_redis` repository.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-4e2a22b5ccaca75f94d7f6f216fd560d380d536f%2Frails_postgres_redis_service_selection.png?alt=media\&token=a241a862-fb21-42da-b975-5dbacc6a27cd)

When you've selected the repository, Release presents options for how the services should be run. The `api` service has options for both **Static** (because a `package.json` file was found) and **Docker** (because a Dockerfile was found). Since this is a Rails server, we'll pick **Docker**.

Notice that Release specifies `443` under "Public Port" for our `api` service. We'll use this later when we connect our two applications.

Next we'll name the application. You can give the application any name, but since this repository is the backend API for our example, we'll name it "backend".

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-346fbb13ec9289f238f98f173b8a4d97ff01d1d7%2Frails_postgres_redis_backend_name.png?alt=media\&token=6f3daf0b-5fda-436b-b597-ac901439ac86)

Click the **Generate App Template** button to move on to the next screen.

On the Application Template screen, we don't need to make any changes, so you can go ahead and click the **Save & Continue** button.

We don't need to make any changes in the Build & Runtime Configurations page either, so you can click the **Start Build & Deploy** button to finish setting up the backend.

## Create the frontend application

Now we can create an application from the `docker-create-react-app` repository.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-e9a96c5983334a8684f02e4ae6ec4154830afc5f%2Fdocker-create-react-app-service-selection.png?alt=media\&token=2620a58e-496f-4940-9707-5da797f4ea41)

Select the repository and Release will present the options for how the services will be run. This repository has only one service named `app`, which is configured to run in Docker, so we select **Docker**.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-0874f171fc2f8673478e5d49dd5aa674d40ad48a%2Fdocker-create-react-app-name-frontend.png?alt=media\&token=a3bb342d-0b9e-4156-89ae-bd37c340481d)

Name the app – we've named it "frontend" – then click the **Generate App Template** button.

This time, we'll modify the Application Template to connect our applications using the App Imports feature. Click **Edit** to open the Application Template.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-ca66bd20adf451126ba2824646d2beeab2ff0245%2Fapp-imports-backend.png?alt=media\&token=0d962b70-1b33-473e-abb0-18a076bab69f)

At the bottom of the Application Template we've added:

```
app_imports:
- name: backend
```

This tells Release that when we create the `frontend` application, the `backend` application must be imported.

Click **Save & Continue**.

### Add the environment variable

Next we need to add an environment variable to ensure the two applications can talk to each other.

Release automatically generates environment variables in the form of `<SERVICE_NAME>_INGRESS_URL` when the service is exposed to the internet.

When we created our `backend` application, we selected **Docker** for our `api` service, and Release specified that the service is exposed on public port 443. This means we'll have an environment variable in our backend named `API_INGRESS_URL`.

When we use App Imports to connect the applications, the ingress environment variables for `backend` are shared with `frontend`. We need to add a mapping from `API_INGRESS_URL` to the correct environment variable on the frontend. We'll map to the `REACT_APP_BACKEND_API` environment variable, which can be found in the [`api.js`](https://github.com/awesome-release/docker-create-react-app/blob/main/src/api.js#L5) file of the repository.

{% hint style="info" %}
If you'd like to learn more about mapping environment variables, read the section about it in our [guide to environment variables](/reference-documentation/application-settings/default-environment-variables#mapping).
{% endhint %}

To add the environment variable, click the **Edit** button in the "Default Environment Variables" section.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-6b7372f02bfd5bffaf0cdcc43f4301641ea0dda2%2Fenvironment-variable-mapping-frontend-backend.png?alt=media\&token=c0d8259b-981b-4154-973e-94f3ebb0c42d)

We added

```
mapping:
  REACT_APP_BACKEND_API: API_INGRESS_URL
```

Now our frontend application will receive the value of `API_INGRESS_URL` as the environment variable `REACT_APP_BACKEND_API`. Click **Save** to close the Default Environment Variables modal and click **Start Build & Deploy**.

## View the deploy

On the App Dashboard page, we can see that a deploy is in progress.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-30df5cde70c3dc3e73abf5a462b1118586f1a3c4%2Fdeploy-activity.png?alt=media\&token=3df145a8-5b10-487a-bba8-15c9488006a8)

Click on `Deploy 3345` to go to the Deploy Info page where you'll see the progress.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-0dd142b41b47be9664297edeef688c9381a2dec2%2Fdeploy-page-app-import.png?alt=media\&token=923d8d6d-3b79-4d26-8edb-af9a768e8464)

Notice that there are four stages to this deployment: Two for the `backend` application and two for the `frontend` application. This tells us that our App Import has worked.

When deployment is complete, you can click on the **Go To Environment** button to view the URLs and ensure that the `frontend` application can talk to the `backend` application.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-9abc418ae0a1742cef7f955a192a454474df5c06%2Fsucessful-deploy.png?alt=media\&token=35dd5524-8a87-4622-8d55-4bb3f0b98509)

On the Environment Details page, we can see the environment for our backend application in the "Namespace" section, and the URL for our frontend application in the "Hostname URLs" section.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-e4c0323e6ba4ed494d4d101ccda39815cdc86ea2%2Fenvironment-details-app-import.png?alt=media\&token=dfe1b991-8708-42e4-9d09-0f1580936efe)

Click on the `app` link to go to the live Release environment.

## View the live application

There isn't much to see when you first visit the live application, but what it's doing is making an API call to the backend to fetch users from the database and display them. Since we don't have any users yet, there are none to display.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-93b764620e77c64b93cebdb4e7c290ae27eaecdb%2Fcreate-react-app-no-users.png?alt=media\&token=6ae55461-3975-4e51-b52b-bc821e3893ee)

Click on the **Create New Users** button to trigger a background job using Sidekiq and Redis to create users. After a short wait, the API to retrieve the users is called again and we can see that the users were successfully created.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-9176f11f94a6733ccfda2d00540f281a3b58b54d%2Fcreate-react-app-users.png?alt=media\&token=fd78f695-172e-44f8-9076-0891b8e0cfdd)


# Example library

Awesome-Release GitHub repo with tons of working starter projects and examples

We've put together the [Awesome-Release organization in GitHub](https://github.com/awesome-release) with tons of working examples of frameworks that work out of the box in Release. We also wrote about [Awesome-Release on our blog](https://releaseapp.io/blog/awesome-release-docker-compose-examples-working-in-release-and-kubernetes), which you can read for more information.

## Examples

* [apache-php](https://github.com/awesome-release/apache-php) - Apache with PHP example.
* [aspnet-mysql](https://github.com/awesome-release/aspnet-mssql) - ASP.NET with MS SQL example.
* [angular](https://github.com/awesome-release/angular) - Angular example.
* [django](https://github.com/awesome-release/django) - Django example.
* [docker-minecraft-server](https://github.com/awesome-release/docker-minecraft-server) - A working Release-powered Minecraft server based on itzg's docker-minecraft-server project.
* [elasticsearch-logstash-kibana](https://github.com/awesome-release/elasticsearch-logstash-kibana) - Elasticsearch, Logstash, Kibana (ELK stack) example.
* [example-voting-app](https://github.com/awesome-release/release-example-voting-app) - The multi-service Docker example voting app. Also explained in our walkthrough tutorial [Docker-only example](/guides-and-examples/common-setup-examples/example-voting-app).
* [flask](https://github.com/awesome-release/flask) - Flask example.
* [gatsby-starter-theme](https://github.com/awesome-release/gatsby-starter-theme) - Standalone Gatsby example, also explained in our walkthrough tutorial [Simple static site example](/guides-and-examples/common-setup-examples/simple-static-build-example-gatsby-starter-theme).
* [gitea-postgres](https://github.com/awesome-release/gitea-postgres) - Gitea with Postgres example.
* [nextcloud-postgres](https://github.com/awesome-release/nextcloud-postgres) - NextCloud with Postgres example.
* [nextcloud-redis-mariadb](https://github.com/awesome-release/nextcloud-redis-mariadb) - NextCloud with Redis and MariaDB example.
* [nginx-flask-mysql](https://github.com/awesome-release/nginx-flask-mysql) - Nginx with Flask and MySQL example.
* [nginx-flask-mongo](https://github.com/awesome-release/nginx-flask-mongo) - Nginx with Flask and MongoDB example.
* [nginx-golang](https://github.com/awesome-release/nginx-golang) - Nginx Golang example.
* [nginx-golang-postgres](https://github.com/awesome-release/nginx-golang-postgres) - Nginx with Golang and Postgres example.
* [nginx-golang-mysql](https://github.com/awesome-release/nginx-golang-mysql) - Nginx with Golang and MySQL example.
* [prometheus-grafana](https://github.com/awesome-release/prometheus-grafana) - Prometheus with Grafana example.
* [rails-postgres-redis](https://github.com/awesome-release/rails_postgres_redis) - Rails 6 with Postgres and Redis.
* [react](https://github.com/awesome-release/docker-create-react-app) - React example.
* [react-express-mongo](https://github.com/awesome-release/react-express-mongodb) - React with Express and MongoDB example.
* [react-express-mysql](https://github.com/awesome-release/react-express-mysql) - React Express MySQL example, also explained in our [Hybrid Docker and static site example documentation](/guides-and-examples/common-setup-examples/hybrid-docker-+-static-site-example).
* [react-java-mysql](https://github.com/awesome-release/react-java-mysql) - React with Java and MySQL example.
* [react-rust-postgres](https://github.com/awesome-release/react-rust-postgres) - React with Rust and Postgres example.
* [sparkjava](https://github.com/awesome-release/sparkjava) - Spark standalone example.
* [sparkjava-mysql](https://github.com/awesome-release/sparkjava-mysql) - Spark with a MySQL database example.
* [spring-postgres](https://github.com/awesome-release/spring-postgres) - Java application with the Spring framework using Postgres example.
* [vuejs](https://github.com/awesome-release/vuejs) - Vue.js example.
* [wordpress-mysql](https://github.com/awesome-release/wordpress-mysql) - WordPress using MySQL as a database example.


# Running instances

How to manage instances running in your environments

Each environment can consist of one or more running instances. Although Release will manage most aspects of these instances for you, it can sometimes be helpful to interact with them directly. For example, you might want to open a terminal to a specific instance to help you debug or troubleshoot an issue.


# Cron jobs

How to create cron jobs

## Understanding cron jobs

Cron jobs let you run scripts and other jobs periodically, either based on an interval (for example, every ten minutes) or on a schedule (for example, every Tuesday at 03:00 AM).

## Creating a cron job in your environment

To create cron jobs in Release, edit your "Environment Configuration" under **Settings**.

![Edit environment configuration](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-3b9aec97240fdd6463e468e92b1e8930c69ffd9f%2Fjobs-edit-evironment-configuration.png?alt=media)

1. At the end of the YAML, add a section called `cron_jobs` and define any jobs you want to run following the example below.

```yaml
# ...
cron_jobs:
  - name: hello-cron
    schedule: "0/5 * * * *"
    image: busybox
    command:
      - sh
      - "-c"
      - "echo 'hi from cron'"
```

2. Press **Save as new version**.

![Save new configuration version](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-00b626d305ab82f2ad4d2db2edd699374746bffa%2Fcronjobs-cron-configuration-block.png?alt=media)

3. Click **Deploy** -> **Deploy Patch** to apply this new configuration to your environment. This will redeploy your existing instances, and also add a new one to run the cron job you defined.

![Apply latest configuration](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-5fc1384287683ba6e467d6c58dad2cd8a438344d%2Fjobs-apply-latest-config.png?alt=media)

Wait for the new environment to deploy. If you used the configuration from the example above, you should see a new instance come up every five minutes. Viewing the logs of that instance will confirm that the job ran as expected.

![Viewing the CronJob log](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-d2b5562a50d074582463516ccc6bf0c94a0a9a2f%2Fcronjobs-cron-log-success.png?alt=media)

## The difference between cron jobs and jobs

Cron jobs are distinct from jobs in that:

* Cron jobs are run on a schedule or timer.
* Jobs are run manually or based on events, for example, after a backend service comes up.

## The cron jobs schema

Instead of choosing an `image` to base your cron job on, you can define `from_services` to base your cron job on an existing service. You can find more examples and the full schema in the [schema docs](/reference-documentation/application-settings/application-template/schema-definition#cron-jobs).


# Jobs

How to create jobs

### Jobs

Jobs are used to run a command that ultimately terminates. They are distinct from services (which stay running) and cron jobs (which run on a schedule).

Some common use cases for jobs include:

* Set-up scripts
* Database migrations
* Compiling assets
* Validation scripts

### Creating jobs on Release

To create jobs in Release, edit your "Environment Configuration" under **Settings**.

![Edit environment configuration](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-3b9aec97240fdd6463e468e92b1e8930c69ffd9f%2Fjobs-edit-evironment-configuration.png?alt=media)

At the end of the YAML, add a section called `jobs` and define any jobs you want to run following the example below.

```yaml
jobs:
- name: migrate
  command:
  - "./run-migrations.sh"
  from_services: backend
- name: setup
  command:
  - "./run-setup.sh"
  from_services: backend
  cpu:
    limits: 100
    requests: 100
  memory:
    limits: 1Gi
    requests: 1Gi
```

Press **Save as new version**.

![Save new configuration version](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-2fb419d8e1904e4e8e8f6d86cb0027679025c09a%2Fjobs-job-configuration-block.png?alt=media)

Click **Deploy** to apply this new configuration to your environment. This will redeploy your existing instances, and also add a new one to run the job you defined.

![Apply latest configuration](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-5fc1384287683ba6e467d6c58dad2cd8a438344d%2Fjobs-apply-latest-config.png?alt=media)

Wait for the new environment to deploy. If you used the configuration from the example above, you should see a new instance come up every five minutes. Viewing the logs of that instance will confirm that the job ran as expected.

### The jobs schema

Instead of choosing a `from_services` to base your job on, you can define `image` to base your job on a Docker image. You can find more examples and the full schema in the [schema docs](/reference-documentation/application-settings/application-template/schema-definition#jobs).


# Using Helm charts

How to use Helm charts with your Release instances

### Using Helm charts on your instances

Helm is a package manager for Kubernetes. Helm charts are used to describe related collections of Kubernetes resources. You can use [Helm with Release](/reference-documentation/helm).

You can configure Helm by adding a `values.yaml` file to your project repository. As an example, you can see our [Helm Ingress](https://github.com/releasehub-com/helm-ingress) repository on GitHub, which configures a Helm chart for generating an ingress for your application.

### Viewing the instances deployed by Helm

One Helm chart might deploy several instances. You can see these in the same way that you view other instances in an environment. Visit your environment and scroll down on the "Details" page to see all your running instances, including those deployed by Helm.

### Viewing Helm charts directly on your Release cluster

If you want to view the Helm charts that are deployed to your Release cluster, download and configure [Kubectl](https://kubernetes.io/docs/reference/kubectl/kubectl/) on your local machine. This will give you access to the `helm` command, which you can use to see your deployed charts, for example:

```bash
helm list
```

and

```bash
helm get all jenkins -n jenkins
```

### Injecting Release environment variables into your Helm charts

You can inject any environment variables defined in your Release configuration into your Helm `values.yaml` file. Here's an example:

```yaml
web:
  image:
    repository: ${RELEASE_REGISTRY_PATH}
    name: web@sha256
    tag: ${WEB_REGISTRY_IMAGE_SHA}
  hostname: ${WEB_INGRESS_HOST}
  url: "jdbc:postgresql://${WEB_RDS_DB_POOL_HOST}:5432/web"
```

For more information, take a look at the [reference documentation for Helm](/reference-documentation/helm/configuration).


# Using terminal

How to open a terminal to a specific running container

In any of your environments, under **Resources** you can see a list of instances associated with that environment.

![Opening a Release terminal](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-387a64308480db2cb74c39012df350b4edce992e%2Fusing-terminal-open-terminal.png?alt=media)

Click on the `>_` button to the right of a specific instance to open a web-based terminal that allows you to run commands directly on that instance.

![Running a terminal command](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-f47473fe21b4739f0ca22a8f16a198f8079ecbd5%2Fusing-terminal-run-command.png?alt=media)

You can use this to check the status of specific processes or run any commands to help you diagnose and fix issues.


# Viewing logs

How to view logs for a specific instance or container

In any of your environments, you can see a list of instances associated with that environment.

![Opening the Release log viewer](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-da94901df954096a9875bec0f6a9c8e88e71c2bb%2Fview-instance-logs.png?alt=media)

Click on the **Logs** button to the right of a specific instance to get a web-based log viewer showing the latest logs from that instance.

From this log viewer, you can:

* **Download** the logs: This will download a `.txt` version of the logs.
* **Clear** the logs: Clear the current output to see newer entries more easily.
* **Reset** the logs: If you have previously cleared the output, resetting the viewer will bring back the previously cleared output.

![Log viewer controls](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-6336f1b8061c094cb040cf1faa71f8d2d66aab7e%2Fviewing-logs-log-controls.png?alt=media\&token=a358cfa8-4247-48e0-b684-d311e9c0bdc0)


# Troubleshooting

Troubleshooting your running instances

You may run into problems or unexpected behavior with your running instances. Here are some common problems and how to troubleshoot them.


# ImagePullBackoff error

How to debug an ImagePullBackOff error

An ImagePullBackOff error means that an instance could not start because the image to build that instance could not be pulled from the image registry.

This is often a temporary error, so restarting the instance will be attempted. After each unsuccessful start, the delay before the next attempt to restart is increased.

## Steps to debug and fix

An ImagePullBackOff error is usually the result of one of the following issues:

**The build failed**

Try to kick off a new build. You can trigger a new build by clicking **Apply** under "Apply Latest Configuration" in the **Settings** tab.

**The Docker Hub integration failed**

If you're pulling images from Docker Hub, you might not have set your credentials for Docker Hub. Make sure you've followed the correct steps to [integrate Release with DockerHub](/integrations/integrations-overview/docker-hub-integration).

**There is a typing error in the service definition**

Double-check your service definitions. A typing error or incorrect name can cause an ImagePullBackOff error if the specified image does not exist.

**A network glitch**

This is rare, but it can happen. Try again after some time and the problem may resolve itself.

**Using underscores in the Release YAML file**

Use dashes instead of underscores in your Release YAML file.


# CrashLoopBackoff error

How to troubleshoot a CrashLoopBackOff error

A CrashLoopBackOff error in Kubernetes means that an instance is constantly crashing. After each crash, the instance restarts, but this does not fix the problem, and it crashes again, leading to a loop.

#### Troubleshooting a CrashLoopBackOff error

The underlying cause of a CrashLoopBackOff error is usually somewhere in your application, so it can be hard to track down.

Take the following steps:

* Click on the instance in the Release UI and look at the crash reason.
* Look at the exit codes, which might describe what caused the crash.
* Look at the logs before the crash to see if anything else went wrong.

If the container dies too quickly and you can't debug effectively, you can modify the run command for that instance and add a `sleep` step. This will force the container to stay alive and give you time to figure out the problem.


# Exit codes

How to troubleshoot exit codes

Exit codes give you information about terminated containers, specifying why the container terminated. In general, a `0` exit code is successful and indicates a purposefully stopped container, while a `1` exit code indicates an error. Other exit codes (larger than 1) also indicate an error state, and contain more information about what kind of error occurred.

## Finding the exit code of a failing container

You can see the latest output of your instances, including exit codes and error messages, by viewing the logs for that instance.

## Common exit codes

* `137`: OOM (out of memory) indicates that your container did not have enough RAM allocated to process what it was trying to do. Sometimes this can be resolved by assigning more resources to your instances, but if it is the cause of a memory leak or similar, you might need to debug the root cause in your application.


# OOM: out of memory

How to troubleshoot OOM errors

An out of memory (OOM) error occurs when an instance runs out of RAM.

The first thing to check if you encounter this error is how much RAM is assigned to the problem instance. Do this by going to the details view of the instance and scrolling down to the "Limits" and "Requests" sections.

![Checking RAM allocation](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-77638d9f191955ea0bb2f1ee4221ff0545930792%2Ftroubleshooting-check-ram.png?alt=media)

* The **Requests** amount defines the amount of memory that is guaranteed to be available to the instance.
* The **Limits** amount defines the amount of memory that the instance may not exceed.

You can read more about how resources are defined [in the resources reference documentation](/reference-documentation/application-settings/application-template/schema-definition#resources).

To increase the amount of memory resources available for your instance, edit your Environment Configuration settings under the **Settings** tab.

### View currently running processes using the top command

If you are debugging an OOM error, you can see what processes are running on an instance and how much memory each process is using by connecting to a terminal for that instance and running the `top` command.

![Checking RAM allocation](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-6f57a53914588616da45d30500020429af796a7d%2Fcurrently-running-terminal.png?alt=media)

![The top command](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-b242424c725e144fc5603aad3ceb64df4ddd0fc3%2Ftroubleshooting-top.png?alt=media\&token=eb84a8b2-8308-40d0-ab84-a2e44ea0ad14)


# Advanced guides


# Containers guide

By now you should have a good understanding of which services (and repositories) are required to successfully model your services in Release. For each of these services, we recommend running through the following steps:

* Create a Dockerfile to create a container.
* Create a `docker-compose` to run the container.

OR

* Create an Application Template file and an Environment Variables Template file in your repository.

Every application is different, so we cannot give you pre-canned files to use. However, in this guide we will walk you through using containers with Release at a high level.

## Create a Dockerfile

In Release, services that are not static need to be run in containers. This means writing a [Dockerfile](https://docs.docker.com/engine/reference/builder/), a set of instructions on how to build your service within a container. Here are some resources to help you write a Dockerfile:

* [Docker's Orientation and setup page](https://docs.docker.com/get-started/)
* [Get started with Docker Compose](https://docs.docker.com/compose/gettingstarted/)
* [Docker Tutorial: Get Going From Scratch](https://stackify.com/docker-tutorial/)

For the best experience, your containers should be isolated and configurable.

### Isolated containers

A container should contain everything it needs to run. This means that it should not be necessary to mount files from your local filesystem into the container to make it work. You need to be able to bring up a container with no reference to local files, so that everything required is built inside the container.

### Configurable containers

In Release, you’ll use the same container image in multiple environments. For this reason, you need to be able to pass in configuration information, for example, the hostname of the generated database to use.

#### Environment variables

Traditionally, this configuration is done through **environment variables**, key value pairs set within a UNIX shell. An example of this might be:

```
DATABASE_HOST=&lt;generated hostname>
```

#### Environment files

Release supports environment variables as the primary way to pass in configuration information. Some frameworks and systems will automatically look for and apply environment variables, but in some cases you may have to:

1. Change your source code to look for environment variables and fall back to defaults if not found.
2. Write an `/entrypoint.sh` wrapper script that runs whenever your container starts, which will write any required configuration files to the container and start your service.

#### Secrets

Secrets should never be checked into your source control (except in rare cases where you can encrypt them securely). Release offers a powerful way to input secrets into your Environment Variables Template, and this ensures that secrets are encrypted at rest and in flight during setup.

Alternatively, since Release runs in your own cloud provider account, you can also write code to integrate with your cloud-native services secrets manager, or use an open-source or managed service such as Vault.

#### Review

Sometimes your code needs to behave differently depending on where its running. For example, you may want to test your email sending functionality in your test environment, but not actually email all your users whenever you run tests. Your code might therefore need to be aware of its surroundings, and adapt at execution time. But there is no single set of rules for when or how your application needs to adapt to its environment.

One final tip: If you’re using an off-the-shelf framework such as Django, Rails, or Next.js, someone has already solved every problem you will ever encounter in Docker. You can google your way out of almost any common issue!

## Docker Compose

Once you have a working Dockerfile, the next step is to build a `docker-compose` file. This is a built-in orchestration tool provided by Docker, and it allows you to model out multiple containers to be brought up in a specific order. A typical example would be a Django backend that requires Redis and Postgres to operate properly.

In our Django backend example, you would define the dependencies and provide the environment variables in your `docker-compose.yaml` file.

Here are some tutorials that will help you create a `docker-compose` file:

* [Define services in a Compose file](https://docs.docker.com/compose/gettingstarted/#step-3-define-services-in-a-compose-file)
* [Getting started with Docker Compose](https://www.educative.io/blog/docker-compose-tutorial#compose)

When you create an application with Release, we'll scan your Git repository for a `docker-compose` file and automatically model out the services it defines.

In the same way containers built from your Dockerfile must be isolated, your final `docker-compose` configuration must be completely self-contained and not require the mounting of files or import of anything external.

## Release Application Template and Environment Variables Template

If you are an advanced user, or you are already familiar with Release, you can create or copy an Application Template and an Environment Variables Template into your repository and Release will automatically pull those files in during the scan process. If you have GitOps enabled, the whole process will be completely automated on every check-in of your codebase too. You can read more about [using GitOps with Release](https://docs.releasehub.com/reference-guide/gitops) in our documentation.


# Application guide

The application is a top-level Release concept. Let's take a look at some high-level information about applications and environments to help you model your services in Release.

Here's some terminology you'll encounter often:

| Term        | Explanation                                                                                            |
| ----------- | ------------------------------------------------------------------------------------------------------ |
| Application | An application is where you define the configuration required to deploy one or more services           |
| Environment | An environment is an instance of an application (you can have many environments within an application) |
| App Imports | App Imports is the mechanism by which we connect applications together to form “virtual applications”  |

## Overview

In Release, it’s possible to model your existing environments in a number of different ways, and it’s important to perform this mapping before you start building Release applications.

An application within Release can run multiple containers in a single Kubernetes namespace. A typical application might be called "backend", map to your backend repository, and run your backend service, as well as supporting containers such as Postgres and Redis.

It’s possible to make an application build and pull containers from multiple repositories, and then run all those containers in a single application. Sometimes this is necessary in a monorepo environment, but generally we recommend that you model out smaller applications that can be connected together later in the process. This gives you additional flexibility when modeling an environment.

For example, this kind of flexibility is essential if you would like to:

* Test your backend as a standalone environment with just the database, but not the associated frontend, and
* Deploy the associated backend whenever you test the frontend, because you can't test the frontend in isolation.

Release's App Imports feature allows this flexibility.

When you set up the configuration for an application, you list a single repository to track (for the purposes of matching builds for updating running environments), so we recommend you follow the model of one application per repository.

### Application environments

Traditionally, the word "environment" means the superset of everything required for your system to run. You might have a staging environment that contains every service you have.

In Release, each application can have any number of "environments" running. When you create an ephemeral environment, you select a branch to track and the environment is created using the templates you define when configuring the application. Once an environment is created, it is no longer affected by changes to the template; you can think of it as a complete fork at the time of creation.

### The Release App Imports feature and virtual applications

If you use our App Imports feature, you may have multiple environments connected to form a virtual environment that contains everything you need.

Using App Imports, you can trigger the creation of environments in multiple applications when an environment for one application is triggered. Take a look at our [App Imports example to connect two applications](/guides-and-examples/common-setup-examples/app-imports-connecting-two-apps-together) to learn more about this feature.

Let's imagine you have three applications:

* Backend
* Frontend
* Admin

Each application maps to its own repository on GitHub, which has the same name as the application.

In the frontend application, you can configure App Imports for backend and admin when you create an environment. For example, if you create an environment for a branch called `feature/new` in the frontend repository, Release will:

1. Look for matching `feature/new` branches in the backend and admin repositories.
   * If there are matching branches in the backend and frontend repositories, Release will build the necessary containers.
   * If there aren't matching branches in the backend and frontend repositories, Release will fall back on your default branch (`master` or `main`).
2. Kick off new environments in backend, admin, and frontend.
3. Merge all the containers into a single Kubernetes namespace (so they can all talk to each other).

When this is done, you’ll be able to connect to the URL you see for the frontend, and the frontend will be able to talk to the URLs created for the backend and admin as necessary, giving you a complete virtual environment containing all the services you need to test the pull request on the frontend.

### How traffic is routed to applications

Release can route traffic to your applications publicly (so that anyone can connect if they have the URL) or privately (you can only access the URLs from your AWS network). This is currently an account-wide setting, so talk to your Release TAM if you need to make this change.

More sophisticated routing requirements are handled by Kubernetes Ingress, a shared, cluster-wide resource. Read more about Ingress in the [Kubernetes documentation](https://kubernetes.io/docs/concepts/services-networking/ingress/).

Here's what you need to know about Ingress in Release:

1. Some aspects of an Ingress are configurable, for example, timeouts and body-sizes, among others.
2. Release supports rules-based routing for complex scenarios, for example, allowing you to route your `/api` path to one service and all other paths to another.

Ingress is mostly handled by magic, and you’ll never need to think about TLS certificates or basic configuration. We map your configured [routing rules](https://github.com/releasehub-com/docs/blob/main/application-settings/application-template/schema-definition.md#hostnames-or-rules) into Ingress configuration for you. This configuration will only need to be adjusted if an application has very long requests, large POST/GET bodies, or other unique needs. Speak to your Release TAM if you need to investigate responses at the Ingress layer and find appropriate tuning values.

## Building containers

The foundation of any service in Release is the container. When creating applications, the repository link is used to determine which containers need to be built and how they need to be built.

Release supports two kinds of builds:

### Docker builds

Most builds in Release are Docker builds.

You define a build argument for a service and Release will look for the corresponding Dockerfile and build the container. We can build in two locations: a Release-managed cluster, and your cluster for enhanced security.

Once containers are built (in either environment), they are pushed to a container registry in your cloud account.

### Static builds

Release also provides a build mechanism primarily used for static node-based frontends.

For example, you may have a frontend that builds and pushes to a cloud bucket (typically S3) and is served through a CDN for performance reasons. Release's static build replicates this pattern, allowing you to build CDN-served frontends.

This build option may not be suitable if you need runtime information, such as dynamically generated URLs to connect to. If you find you need more information about the environment when the frontend is built, a solution may be to use a Docker build then run a build when the container starts, ensuring it has all the environment information.

## Initializing an application

The steps you'll follow to create an application in Release are:

1. Create a new application from the Release dashboard.
2. Connect the application to a version control system repository, for example, GitHub or GitLab.
3. Review and customize the automatically generated Application Template.
4. Provide the necessary environment variables and secrets required to run the application.
5. Start the first build and deployment.

When the Application Template is first generated, Release will check out your application’s source code from your repository, attempt to detect the various services that compose your application, and determine how to build them.

The most effective way for Release to discover your application’s services is through a well-constructed Docker Compose file. For tips on creating a Docker Compose file, refer to our [containers guide](/guides-and-examples/advanced-guides/containers).

Follow our [step-by-step guide to create an application](https://github.com/releasehub-com/docs/blob/main/getting-started/advanced-setup/broken-reference/README.md).

### Workflows

A deployment to an environment will follow one of a predefined set of workflows depending on how it is triggered.

Workflows can be customized to run user-defined jobs or customize which services are started or recreated in which order in response to changes to the environment.

| **Workflow** | **Triggered by**                                                                         | **Useful for**                                                                                                                            |
| ------------ | ---------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| *Setup*      | <ul><li>Initial environment creation</li><li>Environment configuration changes</li></ul> | <ul><li>One-time jobs, like database initialization</li><li>External resource creation</li><li>Service startup</li></ul>                  |
| *Patch*      | <ul><li>Push to environment’s tracking branch</li></ul>                                  | <ul><li>Recreating service instances with images based on new code</li><li>Running database migration jobs to update the schema</li></ul> |
| *Teardown*   | <ul><li>Environment deletion</li></ul>                                                   | <ul><li>External resource cleanup</li></ul>                                                                                               |

See our [workflow schema definition](https://github.com/releasehub-com/docs/blob/main/application-template/schema-definition/README.md#workflows) for more details on configuring your application's workflows.

### Connecting data sources

Ephemeral environments work best when they have enough useful data for your teams to test. Our Instant Datasets feature allows you to track a snapshot of an existing database in your cloud account and generate ephemeral databases that are attached to the environment at creation time.

Here's how the Release Instant Datasets feature works:

* Release must be deployed into the same cloud account and region as the database you wish to track.
* Each night, any unattached databases will be replaced with fresh clones from the most recent snapshot.

Note that you will have additional cloud costs for maintaining a pool of databases for ephemeral environments.

Read more about creating and attaching datasets in our [guide to Instant Datasets](https://github.com/releasehub-com/docs/blob/main/reference-documentation/instant-datasets.md).

## Advanced application configuration

Let's take a brief look at our advanced application configuration that supports a richer set of functionalities.

### Builds

In addition to mapping one repository to one application, with Release, you can also connect multiple repositories to one application to ensure multiple containers are built as necessary to be used in the services section.

### Environment variables

In Release, services are primarily configured by providing appropriate environment variables, meaning your application should be capable of reading environment variables and configuring itself.

### Services

Here are some advanced services available in Release:

#### Health checks

Health checks are an optional feature that help to ensure your application is monitored and restarted appropriately.

Kubernetes supports two kinds of health checks: liveness probes and readiness probes. These are documented in the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/), as well as our [reference documentation](/reference-documentation/application-settings/application-template/schema-definition#readiness-and-liveness-probes). Release supports both `exec` and `httpGet` style probes.

A liveness probe is a test to see if your container is working. This may be a simple GET on `/` to see if we get a `200`. If a container fails the liveness probe, Kubernetes will delete the container and start a new one. Liveness probes should be used sparingly and only when you can tolerate containers being killed and restarted automatically.

A readiness probe is a test to see if your container is ready for traffic. This is useful if your containers take a while to start, or perform work before they are ready for incoming requests, for example, if they have required services or dependencies that must be ready before subsequent services start. Without a readiness probe, your service may refuse service requests for a long period of time before being ready.

Health checks are not required in Release, but we strongly recommend them because they help the underlying Kubernetes cluster understand if your services are healthy or not. However, we advise you add health checks carefully and only after you are done testing the functionality and initial configuration of your application. Readiness probes are one of the last steps you can add to your application before considering it ready to be used.

#### Init containers

Init containers are a way to perform one-time tasks at container startup. They are often used for tasks such as database migrations, or perhaps copying static assets from S3 into the container at runtime. You can read more about how to use init containers in our [documentation](/reference-documentation/application-settings/application-template/schema-definition#init-containers) and in the [Kubernetes documentation](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/).

The most important thing to know about init containers is that they must run successfully to completion before your main container will be started. Otherwise, if the init container exits with an error code, the pod will be deleted and a new one will be created to try again. Often, this results in the dreaded CrashLoopBackoff if not managed correctly.

#### Sidecar containers

Sidecar containers are a way to run multiple containers together in one pod. The benefits of this model include running non-essential or duplicative tasks in a separate container from the application code and processes. For example, you may have a loging monitor daemon and a security monitoring system you wish to bundle with your production web server. You can include these processes, configurations, and binaries in your production web container, or -- with a sidecar -- bundle the extra components in separate containers that run alongside your web server container. This allows you to separate out the concerns (e.g., security, monitoring, logging, etc.) from your codebase and business logic. See more in our [documentation](/reference-documentation/application-settings/application-template/schema-definition#sidecars-1) and the [Kubernetes documentation](https://kubernetes.io/docs/concepts/workloads/pods/sidecar-containers/).

### Jobs

Jobs are a one-time service. They start a container and run once, and then exit. Jobs are great for various setup tasks and other one-time work. By default, they do not block other services from coming up, and their failure will not cause your services to fail. You can manage how jobs and services interact and how they handle failure modes in the workflow. You can read about how to use jobs in our [documentation](https://github.com/releasehub-com/docs/blob/main/application-template/schema-definition.md#jobs).

#### Migrations

Jobs are commonly used for migrations. A good example of this is when you run Postgres as a service in Release, but want to run a command to run migrations after it’s up. This is similar to the init container, but jobs won’t block the start of a service unless you want them to stop the deployment.

## GitOps

Release supports GitOps configuration management by checking your Application Template and environment variables in to your Git repository. This is a beta feature that you'll need the Release support team to enable for your Release account. Once your account has been enabled for GitOps, you will be able to make Release configuration changes via Git commits and pushes to your repository. If you are interested in learning more about using GitOps in Release, please reach out to <support@releasehub.com>.


# Kubernetes guide

Originally designed by Google, Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications.

While Release dramatically simplifies operating Kubernetes for our users, it is important to have at least a basic understanding of some of its key concepts and terms as they relate to Release.

* **Pod**: One or more containers that run with shared storage and networking.
* **Node**: The cloud computing instances that run your Pods (for example, an AWS EC2 instance).
* **Cluster**: Manages Pods and related resources across a set of Nodes.
* **Ingress**: Routes HTTP and HTTPS traffic from outside the cluster to services within it.

Release manages the creation and administration of Kubernetes clusters within your cloud account. Configuration of your cluster, such as Node instance types, autoscaling of Nodes, Kubernetes version upgrades, and deployment of resources into your cluster, are all managed within the Release platform.

You can learn more about Kubernetes from [their documentation](https://kubernetes.io/docs/home/).

## Node sizing

To ensure best performance and to keep your cloud costs in check, it’s important to size the Nodes within your Kubernetes cluster appropriately for the applications they’ll be running. This involves selecting a cloud instance type that fits the needs of your workloads.

Some factors to consider include:

* How much memory does your application need, under your expected usage?
* Does your application tend to be more memory- or CPU-bound?
* Does your application need access to high-performance local storage?
* How sensitive are the environments within this cluster to variance in performance?
* What are the minimum and maximum number of environments that will be running at a time?

Overlaying these considerations onto your budget will help you choose an appropriate instance type for your Nodes. This is also something that can and should be tuned as your needs or usage patterns change.

For more information on Node sizing, see the [AWS EKS documentation](https://docs.aws.amazon.com/eks/latest/userguide/choosing-instance-type.html). [This third-party EC2 instance comparison tool](https://instances.vantage.sh/) may also be helpful, particularly with regards to pricing.

## Autoscaling

At a high level, there are two major categories of autoscaling to consider within a Kubernetes cluster:

* Node autoscaling
* Pod autoscaling

### Node autoscaling

Release manages Node autoscaling on your behalf. If the Kubernetes cluster does not have sufficient resources available to run a workload, then new Nodes will be automatically provisioned and added to the cluster, up to a set maximum. Likewise, Nodes will be automatically terminated and removed from the cluster, down to a set minimum, as environments are removed.

You can set a comfortable minimum and maximum number of Nodes and adjust it over time.

### Pod autoscaling

Pod autoscaling is a little more complicated, because of the many application-specific factors that need to be taken into account. To give a few examples:

* How can we tell that the system is in a state that it needs more or fewer Pods?
* Which types of Pods do we need more or fewer of to respond to the changing load?
* How can we avoid overwhelming limited resources (for example, DB connections)?

Release can support most popular Pod autoscaling solutions that exist in the Kubernetes ecosystem, but they are configured at the Kubernetes layer and require additional effort and expertise to effectively integrate them into your application. If you’d like more information on Pod autoscaling, contact us.

Typically, Pod autoscaling is something that most end users will not worry about or even consider until they deploy to production environments. Some customers may also want to configure and test a production-like environment for performance testing and tuning. These are the most common use cases for Pod autoscaling.

## Resource management

Kubernetes provides capabilities to ensure that, when an application environment is created, its services are run on Nodes that have sufficient memory and CPU available. Kubernetes generally refers to this type of configuration as a resource *request*. It is also possible to monitor ongoing resource usage and restart Pods if they exceed the configured *limit*. Requests are can be thought of as a minimum set of guaranteed resources, while limits are can be thought of as the maximum advisable values.

Tuning resource requests and limits is key to efficiently managing your cluster’s resources and maintaining the stability and performance of your applications. Since resource configuration is heavily dependent on your application’s unique needs, this is something that should be carefully considered and adjusted over time. For more information on service resource tuning, refer to our documentation on [managing service resources](https://docs.releasehub.com/reference-guide/reference-examples/managing-service-resources).


# Create a cluster

## Prerequisites

{% hint style="info" %}
An important prerequisite to creating a cluster is to create at least one domain to use with your clusters. See [Manage domains](/guides-and-examples/domains-and-dns/managing-your-domains). If you do not want to create a custom domain with your own DNS, you can always generate a shared namespace under our customer-shared domain name. See [Using a Release-generated domain](#using-a-release-generated-domain) for more details.
{% endhint %}

{% hint style="warning" %}
Generally, the prerequisites for creating a cluster in AWS and GCP are low, but we will identify key items in a warning box like this. Pay attention to these warnings and see if they apply to you.
{% endhint %}

## Create a new cluster

To create a cluster, navigate to **Configuration** and then to the **Clusters** tab. Press **Create cluster** and select the existing cloud integration that you want to create the cluster in. Choose a region, an IP range, the Kubernetes version, domain to use with this cluster, and a cluster name.

![Create cluster](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-2aeaec0c744cd29c383e8f8301161740b219d031%2Fcreate-cluster.png?alt=media)

{% hint style="warning" %}
If you are using AWS and want to deploy this new cluster into an existing VPC, please see [Using an Existing AWS VPC](/integrations/integrations-overview/aws-integration/using-an-existing-vpc).
{% endhint %}

{% hint style="info" %}
Note that only account owners may create new clusters.
{% endhint %}

Click **Create Cluster** button at the bottom right. Note that only account owners may create new clusters.

This will spin up a new Kubernetes cluster in your cloud provider (for example, using EKS if you've integrated with AWS). Note that this will create compute resources in your cloud provider account, which you will be billed for.

## Choosing an IP range or CIDR block

If you're not sure which IP range to use, you can choose any one from the dropdown.

{% hint style="warning" %}
If you are using an existing VPC and networking layout, the CIDR block will be in use or might not be available in the dropdown. If you know what you are doing, type the existing CIDR block range into the field to override the Release suggestions.
{% endhint %}

Release environments run inside your cloud provider account, and these might need to connect to existing services and applications that run there. Conversely, you might need to fully isolate your Release environments from existing services.

To keep Release environments from interfering with existing applications, by default, Release selects a network address or CIDR block that is distinct from the other network addresses you use in your enterprise. If you know which CIDR block to use, go ahead and select that. Release will build a virtual private cloud (VPC) with the unique CIDR block you provide.

## Managing your cluster and node groups

After creating your cluster, you'll automatically have a default node group. If you want to reduce costs (for example, if you're still experimenting with Release), you can set the minimum, maximum, and desired node size to 1 (or even 0).

![Edit cluster node group](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-332abe8bac21036617683bed244f308f65afb95f%2Fcluster-edit-node-group.png?alt=media)

From the cluster settings page, you can also upgrade the cluster, set it as default, or add more node groups.

## Choosing a node group size and type

Note that you can set a minimum and maximum number of nodes and the type of node that your cloud provider will use to host the workloads.

The size of cluster you need will depend on your application stack, use case, and team size. Unless you know for sure how many environments you will be running at once and what each workload will be, the best option is to choose the defaults (min: 3, max: 10, type: 16GiB/4 CPUs).

With Release, you can seamlessly upgrade, increase, decrease, or resize your cluster later without any interruption, so you can always change this later if you need more compute power or want to reduce your cloud bill.

{% hint style="warning" %}
In both AWS and GCP, there may be resource limits and quotas that apply. If you are using a new account or project dedicated specifically to Release, you may not need to adjust the defaults. However, if you are sharing resources in an existing account or project, you may run into an issue and receive an error message that hopefully sheds light on the problem. You will need to adjust or correct any problems before continuing. We will gladly help you, so feel free to reach out to your support contact for help.
{% endhint %}

## Using a Release-generated domain

You can choose to use a Release-generated domain by selecting "Generate a Subdomain" when you create a cluster. This will generate a subdomain under one of our domains that will be used for the URLs generated for your environments.

![Create cluster with subdomain](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-aaa60c56442de5b85c22aa174d93fe2f6480dcaa%2FDeepinScreenshot_select-area_20221221143911.png?alt=media)

## Creating a cluster with elevated permissions

By default, clusters are created with no extra permissions or roles. If you require elevated permissions on your cluster, you can turn on the "Use elevated permissions when creating clusters " setting on your Organization Settings page.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-8fbebfdcee88146405b7afd6c48d4bb0d403fe50%2Felevated-permissions.png?alt=media)

When this setting is on, any new clusters will be created with the `AdministratorAccess` role attached.

## Migrating apps to a new cluster

When creating a new cluster, any existing environments will need to be recreated fresh or they will continue to try and talk to the old cluster that no longer exists.


# Upgrade a cluster

You can upgrade the version of Kubernetes that runs your clusters directly from your Release control panel. Note that this will put your cluster into maintenance mode for 30-60 minutes. Downtime is not expected, but you should not perform upgrades during critical or peak hours.

## Steps

1. From **Configuration**, navigate to **Clusters** and select the cluster you want to upgrade.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-93460ab767e1c7031787699ccd1767dc88ee85a5%2Fupgrade-cluster-button.png?alt=media)

2. On the Cluster Info page, you can view the current version of the cluster. If an upgrade is available, you will see the **Upgrade Cluster** button under the **Actions** dropdown. Press **Upgrade**.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-2702f8c3767aae7d2c6479b7b1430555408b6184%2Fupgrade-cluster-version.png?alt=media)

3. You'll see a confirmation dialog. Press **Confirm**.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-8cef304542950ab46927a939f9eb018ad65a3812%2Fupgrade-cluster-confirm.png?alt=media)

4. Your cluster will go into maintenance mode and then automatically go back into "Running" once the upgrade is complete.

## Troubleshooting and notes

Note that you can only upgrade one version at a time. In order to upgrade, for example, from 1.19 to 1.21, you will first have to upgrade to 1.20.

While the cluster is in maintenance mode, you may not be able to perform deployments or builds in that cluster, but your services should continue running.

If the **Upgrade Cluster** button is grayed out, it means we do not currently support an automatic upgrade path from your current cluster version. In this case, you can continue using your current version or reach out to us to help you upgrade your cluster.

For AWS clusters, we support AWS EKS versions as closely as possible. You can read more about the supported versions and end-of-life dates [on AWS](https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html).

If your cluster stays in maintenance mode for more than one hour or shows any unexpected errors, please contact Release support.

{% hint style="info" %}
Note that users with "member" permissions cannot perform cluster upgrades. They will not see the **Upgrade Cluster** button and will need to ask a user with "owner" permissions to perform the upgrade instead.
{% endhint %}


# Managing node groups

## Edit a node group

You should rarely need to set the minimum and maximum values for a node group. If you find you need to adjust these values often, it could be an indication of another issue that needs to be addressed. Please reach out to our support team if you have any questions or concerns.

{% hint style="warning" %}
You cannot change the instance type of an existing node group, only the number of instances. To change the instance type, you'll need to create a new node group and delete the old one during a maintenance window. Our support team can assist you with any questions or concerns.
{% endhint %}

Once you have created a cluster, you can view the cluster info. You can also view the associated node groups below the cluster details.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-d9617da6f3f833e24248fb2afe45fa095f8c7915%2Fview-node-details.png?alt=media)

Click the **Edit** button to show the edit dialog. Here you can adjust the minimum, maximum, and desired node size for this node group.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-f23584944a816c68dc1b22590736231338dc9794%2Fedit-node-group.png?alt=media)

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-af86ad854cb8ba2dd5411f93afac1e7b637ce590%2Fedit-node-group-form.png?alt=media)

Click **Update Node Group** to save your changes.

Keep in mind that the "Desired Nodes" value is only set initially and may change later, for example, as workloads are removed or deployed. The cluster will automatically adjust this number according to the workload present during a scheduler cycle. You may wish to increase the number in advance of a large new deployment, or when you know in advance more capacity is needed. In general, your desired value will be closer to the minimum number than the maximum number for most workloads.

To find out what value to use for "Desired Nodes", check the existing number of nodes you have running in your cluster and set it to the same value (or higher). To enforce the "Desired Nodes" value as the baseline, set "Minimum Size" to be the same as "Desired Nodes". To enforce that the "Desired Nodes" value is never exceeded, set "Maximum Size" to be the same as "Desired Nodes". Get in touch with our support team if you have any questions or concerns.

{% hint style="danger" %}
Setting "Desired Nodes" and/or "Maximum Size" to a value less than the number of nodes you are currently running could cause an outage. For example, if you are currently running five nodes in your node group and you set either "Desired Nodes" or "Maximum Size" to four, you could cause some downtime in your application by forcing one node to be terminated. You should always schedule a maintenance window for this operation if you have workloads that end users will rely on.
{% endhint %}

{% hint style="danger" %}
Setting the minimum and maximum sizes to 0 will scale your node group down and could cause an outage.
{% endhint %}

## Adding a node group

To create a new node group for a cluster, click the **Add Node Group** button on the Cluster Info page.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-1bcc0ef2f96c3607139e4f9ce7e3f4d03abed857%2Fadd-node-group.png?alt=media)

This will open the Add Node Group dialog.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-53c09892c62faca0169e6f2553ad7576ba762068%2Fadd-node-group-form.png?alt=media)

The node group you create here will be associated to the current cluster.

Although the minimum and maximum size values are useful to estimate how your workloads will be distributed, keep in mind that having two or more different node groups with different instance types will not guarantee your workloads will scale the correct resources without additional configuration. The Release support team can provide more information about this.

{% hint style="danger" %}
Setting "Maximum Size" to 0 will not allow any pods to run on the new node group, effectively disabling it.
{% endhint %}

In very rare cases (for example, Windows workloads, machine learning instances, and GPU-enabled instances), you can create a node group with a "Minimum Size" value of 0. In such a case, no new nodes will be created until the appropriate pods are scheduled and the workloads are running. This is called "scale up and down from zero" and typically requires extra configuration to enable. The most common way to scale from zero is to use [node selectors](/reference-documentation/application-settings/application-template/schema-definition#service-node-selectors).

We recommend having at least three nodes of any type available, even for an empty cluster, since there are service-level pods that need to run for even small workloads in the cluster. In rare cases where you are running very small workloads, you can set the minimum to two. We do not recommend having less than two nodes available in the cluster.

## **Advanced Options**

**Volume Size**:

This setting will determine the amount of disk space assigned to the physical node. Depending on your cloud provider, this volume is probably a block device attached across the network, as opposed to being a local disk attached to the instance. However, this block device is indistinguishable from local disks for most intents and purposes. This volume will be used by the node for any local operations that may require disk space to support Kubernetes. In general, it is *not* used by workloads, but rather by the node itself. The most likely reason to increase this volume size is to host large docker image caches. The Kubernetes nodes most likely do not need a lot of disk space other than for large amounts of docker images, but you can tune it above or below 100GB at your preference.

#### Policy Identifiers (AWS):

Additional (comma-separated) policy identifier(s) you want to add to the default nodegroup IAM role. This feature is *additive*, meaning, the policy(ies) you add here will provide additional *default* policy(ies) access for all workloads running on this nodegroup. As an example, you may want all your workloads to be able to read from an SQS queue or a DynamoDB table (or other cloud-service) without explicitly adding a [service account](/reference-documentation/application-settings/application-template/schema-definition#service_accounts) and role.

{% hint style="info" %}
Policies can block access to services with explicit deny which take effect "last". This is similar but not exactly the same as Permissions Boundaries (see next section). It is recommended to sticking with additive permissions in the policy(ies) and removing or filtering explicitly allowed permissions with the Permissions Boundary.
{% endhint %}

This feature is only currently implemented in AWS but GCP is coming soon.

**Permissions Boundary (AWS)**:

{% hint style="info" %}
You can read more about permissions boundaries from the write-up on cloud integration permissions here in our section [here](/integrations/integrations-overview/aws-integration/how-to-use-the-permissions-boundary-to-restrict-release-access).
{% endhint %}

This setting will allow you to apply a [permissions boundary](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_boundaries.html) to the nodes that will restrict access to certain cloud functions by default from workloads running on the nodes. A permissions boundary acts as a final seal of approval for what permissions are allowed to pass through from the woker node to your AWS IAM control plane. Follows is a list of default permissions allowed for worker nodes in AWS as of approximately March 2021:

| Category                 | IAM permissions                                                                       | Reason                                                                                   |
| ------------------------ | ------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
| Default EKS requirements | AmazonEKSWorkerNodePolicy, AmazonEC2ContainerRegistryReadOnly, AmazonEKS\_CNI\_Policy | Basic work that an EKS node performs (do not change)                                     |
| Storage                  | s3:\*                                                                                 | Push and pull files from S3; can be scoped to specific buckets/paths                     |
| KMS secrets              | kms:key/\*                                                                            | Decrypt/encrypt secrets for EKS cluster; can be scoped to specific keys                  |
| SSM                      | ssm:\<various>, ssmmessages:\<various>                                                | Allow SSM agent to run (optional, but recommended)                                       |
| Vault Secrets            | dynamodb:table/vault-\*                                                               | DynamoDB backing for Vault secrets (optional -- not all customers use this)              |
| Various EC2 permissions  | loadbalancer:\*; ec2:tags, ec2:describe\*                                             | Permissions to tag resources and add ingress loadbalancers (this may be deprecated soon) |

The above settings can be scoped down to fit your security profile and needs. As one example, if you needed to restrict which S3 buckets were being accessed, you could follow the following steps to add a permissions boundary:

1. Locate and copy the existing permissions from the IAM policy named `release-nodegroup-policy-<suffix>` (the suffix will be different for each nodegroup and cluster)
2. Create a new IAM policy with permissions based on all of the policy contents in step 1
3. Restrict any access you are uncomfortable with (for example, substitute `s3:my_safe_bucket*` for the s3 permissions). Just remember that cutting too many permissions too closely may cause things to break. Make sure you test these changes before affecting any live workloads!
4. Also remember that the permissions boundary is *positive*: if you add a specific permission it will be allowed, but if you remove or are missing a permission from the permissions boundary, then it will be filtered/blocked. Unfortunately, this means you cannot add a negative condition (like "*all s3 buckets except*..."). You must add a positive condition (like "*these specific buckets and prefixes*"). See the AWS documentation for more examples and resources.
5. Save your new policy and copy the ARN for use when creating the nodegroup.
6. Make sure you test it!

## Deleting a node group

{% hint style="danger" %}
DANGER: deleting a node group could cause downtime for running pods. You should always schedule a pod deletion during off-hours or inside a maintenance window.
{% endhint %}

After adding a new node group, you may want to delete the old node group. This can be to save on resources by removing old nodes, or it can help freshen the lifecycle of your instances. Either way, you will want to schedule a maintenance window with your customers and then click the **Delete** button to delete the node group.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-6614a841326cffc8ef9643f0da2bd291d3f0d5ea%2Fdelete-node.png?alt=media)

Release will perform a [cordon-and-drain](https://kubernetes.io/docs/tasks/administer-cluster/safely-drain-node/) operation to smoothly move pods from the old node group on to the newer available node group(s). You should ensure you have adequate resources on the new node group(s) to be able to move the pods over. If you have any questions or concerns, feel free to reach out to our support team. After approximately 10-30 minutes, your old node group should be deleted and the new node groups will take over.

While deleting and draining nodes should not cause an outage on a scaled production cluster, many customers have no fault-resilience or extra replicas setup on a pre-production cluster. This means that this operation ***could cause downtime*** and needs to be scheduled when it will impact your end users the least. Even some production environments have a single point of failure for non-essential services, so you will need to ensure your application can survive and repair itself if any non-resilient service goes down temporarily.

{% hint style="success" %}
Release always recommends you schedule a maintenance window with your customers when performing any action that could impact the performance of your application or stability of your site.
{% endhint %}


# Patch node groups

Learn how to patch node groups to update their AMI or Kubernetes version

This guide will show you how to update your Kubernetes node groups for a Release cluster.

This function is useful after you upgrade your cluster's Kubernetes version or when you want to update the Amazon Machine Images (AMI) running on your node groups.

## Step-by-step guide

Log in to your Release account, then navigate to your cluster settings:

1. Click on the **account settings icon**, top-left.
2. Click on **Clusters** in the left menu.
3. Open the relevant cluster by clicking the cluster name.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-93460ab767e1c7031787699ccd1767dc88ee85a5%2Fupgrade-cluster-button.png?alt=media)

4. Scroll down to **Node Groups** and click on the **Patch All** button.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-1d51bc113ceaa32b07782f8f5dcdf53af4e8f9b7%2Fpatch-node-group.png?alt=media)

5. In the confirmation dialog, click **Confirm**.

The node group upgrade process should start now and can take up to 30 minutes to complete.

## Services should remain online

Release drains each node of its pods, one node at a time, by scheduling pods to run on other nodes. This ensures that your services remain online.

Nevertheless, as with any system update, we recommend running this process during a maintenance window to avoid unforeseen downtime during times of high load.

## What could cause an update to fail

Before updating nodes, Release sets `ForceUpdateEnabled` to `false`, which prevents forcing the update if the Kubernetes scheduler fails to drain a node's pods.

The scheduler might fail to drain a node if there is a shortage of available nodes. To work around this shortage, temporarily increase the maximum nodes on affected groups by following the steps in our guide on [managing your node groups](/guides-and-examples/advanced-guides/managing-your-node-groups#edit-a-node-group). Readjust your group size after patching.

## How to see patching progress

You can view the output of the patching process from the **Logs** section. The process is complete when the cluster log displays `nodegroup successfully upgraded`.


# Hostnames and rules

Customize the URLs used by each of your services for easier discovery and collaboration

## Hostnames

Each public service in each environment has a unique hostname that can be built from components, including:

* The service name (for example, "frontend")
* The application name (for example, "voting app")
* The environment ID (for example, "production" for a permanent environment or "ted8a14" for an ephemeral environment)
* Your domain (for example, example.com or .rls.sh)

Configure your hostnames in the Application Template file. For example, you might want longer hostnames to make it easier to identify from the name exactly which application and environment you're viewing, or shorter hostnames for easier readability and sharing.

To change the default hostname for an application, navigate to the **Settings** tab for the application. Look for the `hostnames` key, as shown below.

![Look for the hostnames key](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-f3bcdb9c80b0e82c6dd79bff8004b10e887b2135%2Fapplicaton-settings-hostnames.png?alt=media)

The default hostname looks something like this: `<service.name>-<example-app>-<env_id>-<domain>`.

The hostname starts with the service name followed by the application name (for example, `frontend-voting-app`), then a unique random environment ID (for example, `ted1bc4` for an ephemeral environment or `production` for a permanent environment), and finally, your domain name (for example, `example.com`).

A service called "frontend" that is part of an application called "voting-app" running in an environment with the ID "ted1bc4" on the free tier of Release would look like this: `https://frontend-voting-app-ted1bc4-.rls.sh`.

You can customize your hostname by adding hard-coded strings or by adding or removing the available templated variables. For example, if you don't want the environment ID in the hostname, you can change the template string in the Application Template file to `frontend-voting-app.${domain}`.

Click **Save as new version** and the next time you deploy, you'll see the updated custom hostname.

Note that customizing the hostname URL in the default Application Template file will affect the default hostname of every other environment that is instantiated from this global default.

If you have a starter account, you will need to leave the domain variable in place so that you can view your environments on the `rls.sh` domain.

## Rules

Rules provide an alternative way to define entry points to your services and let you assign multiple endpoints for a single service. Rules are based on the [NGINX Ingress Controller](https://kubernetes.github.io/ingress-nginx/user-guide/basic-usage/).

When you define rules in your Release configuration, Release will automatically set up an NGINX Ingress deployment to handle routing for you.

Rules consist of a **service**, one or more **hostnames**, and a **path**. The full schema is shown [here](/reference-documentation/application-settings/application-template/schema-definition#rules-1).

As an example, you could set up the following rules for an application with a backend authentication system served from `/auth/` and a frontend GraphQL server from `/graphql` using the following rules:

```yaml
 rules:
  - service: backend
    hostnames:
    - backend-${env_id}.${domain}
    path: "/auth/"
  - service: frontend
    hostnames:
    - frontend-${env_id}.${domain}
    path: "/graphql"
```

You could also "mix and match" visibility rules which allow you to host private URLs:

<pre><code> rules:
  - service: backend
    hostnames:
    - backend-${env_id}.${domain}
<strong>    path: "/"
</strong><strong>    visibility: public
</strong>  - service: admin
    hostnames:
    - admin-${env_id}.${domain}
    path: "/"
    visibility: private
</code></pre>

## Choosing between hostnames and rules

If you only want to configure how your hostnames look and behave, changing the hostname template is the simplest option. Use rules for more complicated cases, such as when you need to assign more than a single endpoint to a specific service.

Note that you can't use both hostnames and rules at the same level of your configuration. For example, you cannot define both rules and hostnames in your default Application Template file, but you can define default hostnames and then override them with rules inside the `environment_templates` section.


# Serve traffic on multiple ports

How to set up a service that accepts traffic on two different ports

Release currently only supports serving HTTP traffic from the first port defined by your service. In the following example, the service RCON listens on port 4326 for the web user interface, and port 4327 is used for websocket traffic from the web application.

```yaml
hostnames:
  - rcon: rcon-${env_id}-${domain}

services:
- name: rcon
  image: itzg/rcon
  ports:
  - type: node_port
    target_port: '4326'
    port: '4326'
  - type: node_port
    target_port: '4327'
    port: '4327'
```

By default, Release will ignore the second port and route all HTTP traffic to port 4326. You can work around this limitation by duplicating the service in Release.

```yaml
hostnames:
  - rcon: rcon-${env_id}-${domain}
  - websocket: websocket-${env_id}-${domain}

services:
- name: rcon
  image: itzg/rcon
  ports:
  - type: node_port
    target_port: '4326'
    port: '4326'

- name: websocket
  image: itzg/rcon
  ports:
  - type: node_port
    target_port: '4327'
    port: '4327'
```

This workaround requires your application container to be stateless, and you will need to configure your environment variables to let the web UI know which URL to talk to for websocket traffic. See [Environment Variables](/reference-documentation/environment-settings/environment-specific-environment-variables) for more information about environment-specific environment variables.


# Configure access to your K8s cluster

Learn how to create access controls and view your cluster using kubectl, K9s, and eksctl

This document walks you through giving a user access to a Release EKS cluster, accessing the cluster, and viewing the cluster using K9s and `kubectl`.

## Definitions

* AWS: Amazon Web Services
* IAM: Identity and Access Management
* EKS: Elastic Kubernetes Service
* ARN: Amazon Resource Name

## How to add an IAM user to your self-hosted EKS cluster

### Prerequisites

To add an IAM user to your cluster, you will need the following:

* Administrator privileges to the EKS cluster.
* The ARN for the user you're granting access to (looks like `aws:arn:iam::ACCTID:user/USERNAME`).
* A kubeconfig file for the EKS cluster. If you do not have an existing kubeconfig file, generate one by following the steps in the [create the kubeconfig file](#create-the-kubeconfig-file) section below.

We recommend you [install K9s](https://github.com/derailed/k9s).

### Steps

You can grant a user access to a cluster in two ways: Using the K9s visual editor or the command line.

#### Using the K9s visual editor

1. Start up K9s and use the `:namespace` command to access the `kube-system` namespace as shown below:

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-c3fe9b65d883cc64fc2d53d6ff9600cd2a34e3ac%2Fimage6.png?alt=media)

2. Use the `:configmap` command to access the `aws_auth` configuration:

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-d2a64925d6f3ab7b88c1c770456bedf12b3596e0%2Fk9s_aws_auth.png?alt=media)

3. Find `aws_auth` and hit the `e` command to `edit` the file. Insert the user as shown below:

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-910950fa6f89a5fd996c101a2cdab5653a93bbd7%2Fk9s_add_aws_user.png?alt=media)

4. Copy and paste the section outlined in red above to create a new user. Be careful to edit the ARN correctly to allow the user to access the system. In this example, the users are administrators, but you can consult the [documentation for Kubernetes](https://kubernetes.io/docs/reference/access-authn-authz/rbac/%23default-roles-and-role-bindings) to define default roles like viewers and ops users.
5. Save the file and then verify the changes by using the `d` (describe) command to view the document that was applied.

### Using the CLI

Follow the [AWS documentation](https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html) to complete the same procedure done visually above. The steps are the same:

1. Download the existing `aws_auth` configmap from the `kube-system` namespace.
2. Edit the `mapUsers` field and add the user.
3. Save the file.
4. Apply the changes to the cluster.
5. Verify the changes have been made.

## How to access the cluster

### Prerequisites

To access the cluster once you have been added to the configmap, you'll need:

* Your AWS IAM credentials for the account where the EKS cluster is running.
* The EKS cluster name and region.
* Command line utility `eksctl` installed. You can follow these [installation instructions](https://docs.aws.amazon.com/eks/latest/userguide/eksctl.html).
* Command line utility `kubectl` installed. You can follow these [installation instructions](https://kubernetes.io/docs/tasks/tools/install-kubectl/).

We recommend you [install K9s](https://github.com/derailed/k9s).

### Steps

Once you have been added to the cluster configmap and you have the prerequisites installed, you can gain access to the cluster to view status and logs, and to perform other tasks you have permissions for.

#### Create the kubeconfig file

Have your AWS credentials available in configuration files, in your environment variables, or in named profiles.

Follow the steps in the AWS [configuration basics guide](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html). The `eksctl` binary respects the usual configuration directives that the AWS CLI uses. This document assumes the default credentials are available. If you wish to specify a set of credentials other than default, you will need to specify them appropriately.

Your credentials will authenticate you as a user or role in the account and region where the EKS cluster is available. You may have a user role configured in a different account and then assume a role in the EKS cluster account, or you may have a very complicated setup with [OAuth](https://oauth.net/2/) or [SAML](https://en.wikipedia.org/wiki/Security_Assertion_Markup_Language) integrations, which is beyond the scope of this document.

To generate your kubeconfig file, type the following where your `eksctl` binary is available and your AWS credentials are specified by default:

```
eksctl utils write-kubeconfig --cluster CLUSTERNAME --region REGION
```

#### K9s instructions

We recommend that you use the K9s interface for visualization and viewing logs and status. Administrating the cluster from the K9s interface is also possible. Here are a few use cases we’ve found useful.

**View application namespaces**

You can use the `:namespaces` command and filter with the `/release` search to list applications running from Release environments as shown below:

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-47a829298ca057269cd600e31c108d1c8ed734f8%2Fimage8%20\(3\)%20\(1\).png?alt=media)

**View pods for a Release environment**

You can then either click on a namespace or type the `:pods` command to view the applications in the Release environment as shown below:

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-140f2604cd88d785ba386afc7145b52aaec64476%2Fimage1%20\(3\)%20\(1\).png~merged?alt=media)

**View logs for an application container in a Release environment**

You can use the `l` (or logs) command to view what is happening in your application:

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-ab2a5c37445f4317018c2a19035facb012a92cc9%2Fk9s_view_logs.png?alt=media)

**Access the container system (if available)**

If you have sufficient privileges and configuration, use the `s` (or shell) command to enter the running container, if available:

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-e1182f297a4f2c63830027ea37af8a080f1bab30%2Fimage7%20\(2\)%20\(1\).png?alt=media)

**Exit K9s**

Use the familiar VI controls to `:quit` the K9s application:

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-0a5d6740953b64b85d2306decf0739a942f4fbe8%2Fk9s_exit.png?alt=media)

#### CLI instructions

Use CLI commands to examine the state of the cluster, but we generally don't recommend using them to change settings or start or stop pods or services, as this should be handled by the Release website or CLI tool.

You can find great [kubectl documentation](https://kubernetes.io/docs/reference/kubectl/cheatsheet/) in the Kubernetes documentation.

**`kubectl get namespaces`**

Remember that a namespace in Kubernetes maps to a Release environment.

**`kubectl get pods -n RELEASEENV`**

Remember that a pod in Kubernetes maps to a Release service in the environment.

**`kubectl get logs RELEASESERVICE`**


# Restrict cluster API access

Lock down your cluster's Kubernetes API server to a specific list of source IP addresses (allowed hosts) for AWS EKS and GCP GKE clusters, without losing Release's management access.

By default, your cluster's Kubernetes API server (the control plane) accepts connections from any source IP address. **Allowed hosts** lets you restrict that access to an explicit allowlist of source IPs and CIDR ranges — so only you and Release can reach the API server.

Use this when you want a tighter, private-networking or compliance-oriented posture around your cluster's control plane.

## What it does

When you enable allowed hosts, the cluster's Kubernetes API endpoint stops accepting connections from any source that isn't on your list. `kubectl`, `k9s`, CI runners, and any other API clients must connect from an allowed address, or they are denied at the cloud provider's control plane.

## How Release access is preserved

Release's static egress address, **`35.247.5.67/32`**, is **always** merged into your allowlist automatically. You never type it, and it can't be removed. This guarantees Release never loses management access to your cluster, no matter what you enter.

Whatever list you submit is applied as *your entries plus Release's egress*.

## Before you start

* Your cluster must be **AWS (EKS)** or **GCP (GKE)**, **Release-managed**, and in the **running** state.
* Have your own current public IP address ready. You can find it by running:

  ```bash
  curl ifconfig.me
  ```
* Entries must be **IPv4** addresses or CIDR ranges. A bare IP (for example `203.0.113.4`) is treated as a `/32`. IPv6 is not supported.

## How to enable or edit allowed hosts

1. Open your cluster's detail page in the Release dashboard.
2. Open the **Actions** menu (top right).
3. Click **Enable allowed hosts** (or **Edit allowed hosts** if a list is already applied). This opens the **Restrict cluster API access** dialog.
4. Enter one IP address or CIDR range per row:
   * A bare IP such as `203.0.113.4` becomes `203.0.113.4/32`.
   * A range such as `203.0.113.0/24` is used as-is.
   * Use **+ Add another** to add more rows. Invalid entries are flagged inline and you can't apply until every row is valid.
5. Click **Apply allowlist**.

Release's egress (`35.247.5.67/32`) is shown at the top of the dialog and is always included — you don't need to add it.

> The **Actions** menu item is only available on running, managed AWS or GCP clusters. It doesn't appear on imported clusters and is disabled while the cluster is busy with another operation.

## ⚠️ Avoiding lockout

**Every source that isn't on your allowlist is denied — including you.** If you apply a list that doesn't contain your own current public IP, you'll immediately lose direct `kubectl` / API access to the cluster.

Before applying:

1. Find your current public IP: `curl ifconfig.me`
2. Add it to the list (as a `/32`, or as a wider CIDR that covers it).
3. If you connect from behind a NAT, VPN, or corporate gateway, allowlist that egress range instead of your local address.

Release access is always preserved regardless, so Release itself is never locked out — but **your** direct access depends entirely on including your own address.

### The "Release access only" option

If you submit an **empty** list (no entries of your own), the dialog requires you to explicitly check **"Allow Release access only — I understand this may remove my own direct kubectl/API access."**

Only check this if you intend for Release's egress to be the *only* thing that can reach the API server. This is a deliberate choice, not something you can do by accident.

## Recovering from lockout

The allowlist lives at your cloud provider's control plane, so even if you lock yourself out of the Release UI action, you can fix it directly with your cloud CLI (using credentials that don't depend on cluster API access).

**AWS (EKS)** — add your IP to the public-access CIDR list:

```bash
aws eks update-cluster-config \
  --name <CLUSTER_NAME> \
  --region <REGION> \
  --resources-vpc-config \
    endpointPublicAccess=true,publicAccessCidrs=35.247.5.67/32,<YOUR_IP>/32
```

Include **every** address you want to keep — this call replaces the current public-access CIDR list, so list Release's `35.247.5.67/32` alongside your own.

**GCP (GKE)** — update the master authorized networks:

```bash
gcloud container clusters update <CLUSTER_NAME> \
  --region <REGION> \
  --enable-master-authorized-networks \
  --master-authorized-networks=35.247.5.67/32,<YOUR_IP>/32
```

As with EKS, this replaces the existing list, so include Release's `35.247.5.67/32` along with your own address.

Because Release's egress stays on the list regardless, **Release support can also re-open access** for you if needed.

Once you can reach the cluster again, re-run **Edit allowed hosts** in the Release dashboard so the UI and the cloud provider stay in sync.

## What happens during apply

Applying an allowlist is asynchronous. When you click **Apply allowlist**:

1. The cluster briefly enters a **maintenance** state (typically \~15–30 seconds) while the change is pushed to your cloud provider's control plane.
2. Once the provider confirms the change, the cluster returns to **running** and the new allowlist is shown as applied.

This brief maintenance state is expected — it is not an error. The allowlist only reads as "applied" after the provider confirms it, so a change that hasn't finished (or that failed) never shows as active.

## AWS (EKS): worker nodes and the private endpoint

This section applies only to AWS (EKS) clusters. It explains a behavior Release handles for you automatically so a restrictive allowlist doesn't disconnect your worker nodes.

### Why EKS needs special handling

By default an EKS cluster's Kubernetes API endpoint is **public-only** (`endpointPrivateAccess: false`). In that configuration your worker nodes reach the control plane over the **public** endpoint, using your VPC's NAT/internet gateway egress IP addresses.

The allowed-hosts list (`publicAccessCidrs`) gates that public endpoint — it does not affect the private endpoint. So on a public-only cluster, a restrictive allowlist that doesn't include your node egress IPs would cut the nodes' kubelets off from the control plane, and the nodes would go **NotReady**.

### What Release does automatically

When you apply a restrictive allowlist to a public-only EKS cluster, Release **automatically enables the EKS private endpoint** (`endpointPrivateAccess: true`) in the same update. Node-to-control-plane traffic then routes **inside your VPC**, so the nodes stay connected regardless of the public-access allowlist. This is AWS's recommended durable configuration.

You do **not** need to add your NAT/egress IPs to the allowlist to keep nodes healthy — Release takes care of the private-endpoint side for you. (The modal shows a note reminding you of this whenever you're about to apply a restrictive list on an AWS cluster.)

Some guarantees worth knowing:

* Release **only ever enables** the private endpoint — it never disables a private endpoint you already have on.
* Reverting the allowlist to open (`0.0.0.0/0`) does **not** change the private-endpoint setting; once on, it stays on.
* Release's own egress CIDRs are always merged into the allowlist, so Release retains access either way.

> **Note — custom VPCs:** Enabling the EKS private endpoint requires the cluster's VPC to have `enableDnsHostnames` and `enableDnsSupport` set to `true` (EKS creates a Route 53 private hosted zone for the endpoint). Standard Release-provisioned VPCs already satisfy this. If you run the cluster in a custom VPC, confirm both settings are enabled.

### GKE is different

On GCP (GKE), the cluster's own internal IP ranges are preset access sources for the control-plane endpoint, so restricting authorized networks does **not** sever your nodes. There is no equivalent auto-enable step on GKE, and none is needed.

## The active-restriction banner

Whenever an allowlist is applied, the cluster page shows an **"API access is restricted"** banner listing the currently allowed sources. If you lose direct access, use the banner as a reminder to add your current IP via **Actions → Edit allowed hosts**, or follow the recovery steps above.

## Supported clusters and limits

* **Providers:** AWS (EKS) and GCP (GKE) only. Azure and imported clusters are not supported.
* **Cluster state:** must be Release-managed and running.
* **Address family:** IPv4 only. IPv6 is rejected.
* **GKE limit:** GKE caps authorized networks at **50 entries** total (your entries plus Release's egress).


# Designing for multiple environments

When running your application in different environments, like those provided by Release, you may need to make some minor changes to your code to support multi-variations. You may need to adapt your code depending on how your framework handles [CORS](https://en.wikipedia.org/wiki/Cross-origin_resource_sharing), or you may want to support frontends pointing to different instantiations of a backend. Release provides several environment variables to use in your application to assist you to properly configure your application to work in multiple environments.

Release provides the following environment variables:

* `RELEASE_ENV`: set to `true` for all Release environments
* `SERVICE_INGRESS_URL`: where `SERVICE` is the name of your service, including `https://`
* `SERVICE_INGRESS_HOST`: where `SERVICE` is the name of your service, no protocol added
* `RELEASE_RANDOMNESS`: the random unique portion of the URL

For example, the following code configures CORS in Django:

```python
"""
Django settings for backend project.
"""
import os
import re
import logging, logging.config

IS_RELEASE_ENV = bool(os.environ.get('RELEASE_ENV', None))
API_INGRESS_HOST = os.environ.get('API_INGRESS_HOST', '')
API_INGRESS_URL = os.environ.get('API_INGRESS_URL', '')


if DEBUG:
    ALLOWED_HOSTS = ['*']
elif IS_RELEASE_ENV:
    logger.info('RELEASE ENV: allowing ingress host: %s' % (API_INGRESS_HOST))
    ALLOWED_HOSTS = [API_INGRESS_HOST]
else:
    ALLOWED_HOSTS = ['example.com']

if IS_RELEASE_ENV:
    BASE_RETURN_URL = API_INGRESS_URL
else:
    BASE_RETURN_URL = 'https://example.com'
```

## Applications with a frontend and backend

If your app uses a JavaScript framework, such as React, on the frontend, and Rails on the backend, you will need to configure the Rails backend to allow traffic from the frontend. You will also need to configure the frontend to talk to the backend.

In the following example, a Rails backend service named `backend` is configured to allow traffic from a React frontend service named `frontend`:

```ruby
module MySweetApp
  class Application < Rails::Application

    frontend_ingress_host = ENV.fetch('FRONTEND_INGRESS_HOST') { "localhost:4000" }

    # CORS
    config.middleware.insert_before 0, Rack::Cors do
      allow do
        origins frontend_ingress_host

        resource "*",
          headers: :any,
          methods: [:get, :post, :put, :patch, :delete, :options, :head],
          credentials: true
      end
    end
  end
end
```

Release will generate the following environment variables to use in your application:

* `FRONTEND_INGRESS_URL`
* `FRONTEND_INGRESS_HOST`
* `BACKEND_INGRESS_URL`
* `BACKEND_INGRESS_HOST`

Now we need to configure the frontend service to talk to the backend. Here we configure Axios to talk to our backend:

```javascript
import axios from "axios";
import config from "../config";

const client = axios.create({
  baseURL: process.env.BACKEND_INGRESS_URL || "http://localhost:3000",
  withCredentials: true,
  headers: {
    "Content-Type": "application/json"
  }
});
```


# Microservices architecture

Creating applications from multiple repositories

Release works well with both large complex applications and many smaller applications all talking to each other. If you fall into the latter category, our solution to a microservices architecture is the App Imports feature. App Imports creates single-direction dependencies from one application to another with all services being brought up in the same namespace. This means there is one hard requirement for using App Imports: you cannot have two services with the same name. See [Unique service names](#unique-service-names) for further explanation.

A step-by-step example of using App Imports can be found at [App Imports: Connecting two apps](/guides-and-examples/common-setup-examples/app-imports-connecting-two-apps-together).

## Setting up App Imports

Here's an example of setting up App Imports in the Application Template:

```
app_imports:
- name: backend
  branch: new-branch
  exclude_services:
  - name: redis
```

The documentation for the schema can be viewed in the [Application Template guide](/reference-documentation/application-settings/application-template#app-imports). In our example, an application `frontend` uses App Imports with an application named `backend`.

## Single-directional and bi-directional imports

If our `frontend` app adds an `app_imports` section to its template to import `backend`, then whenever an environment for `frontend` is created, an environment for `backend` will also be created. This is the single-direction of the import. If someone were to create an environment for `backend`, only the environment for `backend` would be created.

To create bi-directional imports, both applications need to have an `app_imports` section in their template.

If bi-directional imports are set up, **you will not create an infinite loop**. You might think that `frontend` would import `backend` which would import `frontend` and create an infinite loop, but the import will only occur once with `frontend` importing `backend` and finishing. This is because:

{% hint style="warning" %}
App Imports are not recursive. So if A imports B and B imports A, you will **not** get ABABABAB. If A imports B and B imports C, you will get AB when you deploy A, and you will get BC when you deploy B, and you will get C when you deploy C. You will **not** get ABC in any scenario.
{% endhint %}

## Branch selection

Branch selection has a fallback order that includes three different options. They are:

1. The branch is set in the Application Template `app_imports` section via the `branch` key as described above. If set, all environments will **always** use this branch. Remember that the branch attribute is optional in the template.
2. A matching branch on GitHub. If `frontend` is on branch `example`, then Release will query GitHub and see if the repository used for `backend` also has a branch named `example`. If there is a match, that branch will be used.
3. The `tracking_branch` attribute set in the Application Template. Remember that `tracking_branch` is optional so it may not be set.
4. The default (or "main") branch that is set for the repository on GitHub.

## Ignoring Deployments

Imports can be set to ignore deployments for specific refs via the `ignore_deployment_refs` key. An example would be

```
app_imports
- name: backend
  ignore_deployment_refs:
    - main
```

In this scenario, if the branch selected for `backend` is `main` then whenever a deployment is created, it will be automatically aborted. There is support for negation and path globs as well.

```
app_imports
- name: backend
  ignore_deployment_refs:
    - "!main"
    - releases/**
```

In this scenario, if the branch for `backend` was anything other than `main` or something like `releases/10/20` then the deployment would be automatically aborted.

## Ingress environment variables

Release autogenerates environment variables in the form of `<SERVICE_NAME>_INGRESS_URL` when the service is exposed to the internet. If, for example, `backend` has a service named `api`, then Release will generate an environment variable named `API_INGRESS_URL`. If our `frontend` app has a service named `web`, then Release will generate the `WEB_INGRESS_URL` environment variable too.

When using App Imports, the ingress environment variables are shared between between all environments. This means that `frontend` will have access to `API_INGRESS_VARIABLE` and its own `WEB_INGRESS_URL`, and the same will apply for `backend`. Release shares all the environment variables in case two or more imported apps need to be able to communicate.

## Mapping environment variables

Given that the ingress environment variables are available, you may need to map those to another environment variable your application is expecting. If `frontend` was using the environment variable `REACT_APP_BACKEND_API` to talk to its API, then we would want to map the `API_INGRESS_URL` to it. This can be achieved using [environment variable mapping](/reference-documentation/environment-settings/environment-specific-environment-variables/environment-variable-mappings).

![Environment Variable Mapping Example](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-14b08ff72ac298c38cda210e2e4f1f5d60805534%2Fenv_var_mapping%20\(1\).png?alt=media)

## Unique service names

{% hint style="danger" %}
Warning: Unexpected behavior is caused by using services with the same name.
{% endhint %}

If `frontend` has a service named `api` and `backend` has a service named `api`, Release will try to deploy two services named `api` into the same namespace. By default, Kubernetes will remove the first service and replace it with the second – this is a function of how Kubernetes works and is not Release-specific. To avoid naming conflicts, we suggest adding the name of the application as a prefix to the service name to differentiate between the services in each app, for example, `${app_name}-api`.

## Viewing services in the namespace

When using App Imports, all the services are placed into the same namespace. This means that when viewing the "Instances" table on an environment's details page, you will be able to see the services from all the applications.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-2d06fea550ca6517e9243da8c54b01b4b911b20b%2Fnamespace_instances.png?alt=media)

If you look under the `Namespace` section in the top left, you'll see that the environment `jer-k-main` from the application `backend` is imported into this namespace. The `backend` application defines the `api`, `db`, `redis`, and `sidekiq` services, which can see seen in the "Instances" table. The `frontend` application defines the `app` service, which can also seen.

## Managing App Imports via the UI

While the App Imports feature is able to be fully controlled through the templates, you can also manage them through the Release UI.

From the Environments page, when creating a new Environment, expand the Advanced Options section to see which Apps are available to use. Apps that are imported in the Application Template show up as preselected with additional Apps listed below. If `branch` is specified in the template, the branch selector will be pre-filled. Use the selector if a specific branch is required, otherwise leave it blank and the branching logic described above will be used.

<figure><img src="https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-23660d927aacb1903327fc51bc340aaf4e659454%2Fcreate-environment-app-imports.png?alt=media" alt=""><figcaption><p>App Imports during Environment creation</p></figcaption></figure>

After the Environment has been created, visit the App Imports tab to make changes to which Apps are imported.

<figure><img src="https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-b4f58cf5a3d6f9aba00e7449df2bd17c9ea96f9d%2Fenvironment-app-imports.png?alt=media" alt=""><figcaption><p>Managing App Imports after Environment creation</p></figcaption></figure>

After selecting a new App to import, this time with a branch and clicking the Save button, the selected App will be saved into the template and move to the top of the list.

<figure><img src="https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-1d78e71c75dcd52b158737096769e02a3eb7bfff%2Fenvironments-app-imports-updated.png?alt=media" alt=""><figcaption><p>Saving a new App Import</p></figcaption></figure>

If we view the Environment Template, we will see

```yaml
app_imports:
- name: rails-postgres-redis
- name: angular
  branch: main
```

which matches the visual representation from the UI. After successfully updating, use the Deploy Setup button to start a new deployment with the new imported Apps.


# Monitoring your clusters

In this guide, we'll look at how you can use Datadog to collect metrics and logs, monitor clusters, and receive alerts for events in your Release environments.

Observability is a crucial part of modern DevOps and software development. By collecting logs and stack traces in a central place, developers can keep an eye on how their distributed systems function. Having access to logs and system metrics in a single view can speed up debugging, help identify bottlenecks or performance issues, and alert developers about possible security and scaling issues before they happen.

At Release, we use Datadog to monitor our systems. Datadog automatically collects system metrics, monitors selected metrics, and can be configured to send alerts for events.

Datadog is especially suited for collecting metrics from Kubernetes clusters, which is why we recommend adding Datadog monitoring to your Release clusters.

## Setting up your Datadog integration

To get started, you'll need a Datadog account, and you'll need to activate the Datadog agent on your Kubernetes clusters in Release.

### Set up a Datadog account

If you don't have a Datadog account yet, you can follow our [guide on setting up a new Datadog account](/integrations/integrations-overview/datadog-agent#create-a-datadog-account). Keep in mind you'll need to use the same region for Datadog (for example US1), that you use for your Release clusters (for example us-east-1).

### Enable the Datadog agent in Release

If you haven't set up your integration yet, follow our guide on [integrating Datadog in Release](/integrations/integrations-overview/datadog-agent#set-up-the-datadog-integration-in-releasehub). Make sure to [enable the Datadog agent](/integrations/integrations-overview/datadog-agent#enable-or-disable-the-datadog-agent-in-your-clusters) on any new clusters you create in Release.

## How the Datadog agent works in Release

When you activate the Datadog agent on a Release cluster, Release installs and starts a [Datadog cluster agent](https://docs.datadoghq.com/containers/cluster_agent/) to record cluster metrics, and runs a [Datadog agent](https://docs.datadoghq.com/agent/) on each node in your cluster.

Release automatically configures your Datadog agents with the API key you provided, and sets the Datadog site to the appropriate region based on your cluster's region.

## Metrics tracked by Datadog

Release configures Datadog agent running on nodes to collect all standard system metrics (such as CPU usage, memory usage, and disk usage). Node agents also collect any logs that your container images write to `stdout` or `stderr`.

The Datadog cluster agent records metrics about the entire cluster, such as the number of active pods, Kubernetes network throughput, dropped packets, etc.

The best way to get a full view of all the metrics recorded, is to log in to Datadog and use the metrics explorer.

## Setting up a monitor with alerts in Datadog

Before we get started, make sure you're logged in to your [Datadog account](https://app.datadoghq.com/).

### Decide what to monitor by viewing a dashboard

If you don't have a specific statistic in mind yet, or to get an overview of how metrics can be combined to form meaningful statistics, start by viewing the dashboards that Datadog creates automatically.

For instance, if you are interested in monitoring the state of Kubernetes pods, navigate to the *Kubernetes Pods Overview* dashboard.

![Find the Kubernetes Pods Overview dashboard in Datadog](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-2f8e00834acdeb3fa6762d56c692162341c29c35%2Fdatadog-dashboard-kubernetes-pods-overview.png?alt=media\&token=3dbd8153-d8bc-48c1-b43f-c486520ed862)

Dashboards created by Datadog have filters to help you view only relevant statistics. On the Kubernetes Pods Overview, click the *namespace* dropdown.

![Kubernetes namespaces in Datadog](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-3cc8da9f2fc93de2a490a3c4f1733750885a651c%2Fdatadog-dashboard-namespaces.png?alt=media\&token=c9a750fb-4386-4136-a038-003fa350ecaf)

Here you'll notice that Release creates a Kubernetes namespace per environment, and a separate namespace for supporting services, called `kube-system`.

It is useful to get an overview of everything Release runs on your cluster, but you might be interested to filter your statistics by environment later on.

Since we're interested in the state of pods, scroll down to the *Pods* section on the dashboard.

The *Pods in Bad State by Namespace* statistic shows pods that are not in the ready state at any given time. This would be a useful statistic to keep an eye on.

![Metric details in Datadog](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-151ad50110bdf9591556558c5794531568452a68%2Fdatadog-metric-details.png?alt=media\&token=7e737717-24e6-4f1b-8634-74954eb35e8f)

To see how this statistic is calculated:

1. Click on the *Pods in Bad State by Namespace* statistic for one of the namespaces.
2. Datadog displays the metric used for this statistic.
3. Click on the **Copy** icon to copy the metric to your clipboard.

In this case, the metric that shows pods in a bad state is:

`exclude_null(sum:kubernetes_state.pod.ready{condition:false,!pod_phase:succeeded})`

### Create a metric monitor

![Navigation steps to create a new metric monitor in Datadog](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-76c4fdce438e98abe5b30c103e3a54ce961ab1f0%2Fdatadog-new-monitor-navigation.png?alt=media\&token=831ee3f8-8b57-4dd8-bfde-7e32d9bc7053)

To create a monitor based on our chosen metric:

1. Click on **monitors** in the sidebar.
2. Click on **new monitor**.
3. Finally, click on **metric**.

Now you can configure your new monitor.

![Configure a new monitor in Datadog](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-19db201dc5d70bc5910e403b0dea7659b98ae1de%2Fdatadog-new-monitor-one.png?alt=media\&token=88ea83b3-96a7-4b99-8bab-e26c78ad1111)

1. Under *Choose the detection method*, select **threshold alert**.
2. Under *Define the metric*, click source.
3. Paste the metric you copied earlier:

   `exclude_null(sum:kubernetes_state.pod.ready{condition:false,!pod_phase:succeeded})`

   To exclude Release's `kube-system` namespace, add `!kube_namespace:kube-system` to the list of filters. The final metric should look like this:

   `exclude_null(sum:kubernetes_state.pod.ready{condition:false,!pod_phase:succeeded,!kube_namespace:kube-system})`
4. Under *Set alert conditions*, select the option **above or equal to**.
5. Enter the number `1` as the alert threshold.

![Set alert message on Datadog](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-0131beb8d5d3b32d17e0d00473c7e9b0195095ec%2Fdatadog-new-monitor-two.png?alt=media\&token=ead5104d-d1ce-4cb4-8d80-e4a3cc9599b1)

1. Under *Notify your team*, enter a subject for this alert.
2. Enter an alert message. Note that you can use variables from a monitor in the monitor's alert message. Type `{{` in this field to see suggested variables.
3. Select Datadog users or groups of users who should be notified.
4. Click **create** to save your new monitor.

Your new monitor will alert your team when the alert threshold is exceeded.

### Datadog's recommended monitors.

Datadog offers many monitor examples that are ready to use with data collected from your clusters.

![Datadog's recommended monitors](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-3bc19f937d0e6990a2ed23ae006a68d9e7ec20c5%2Fdatadog-recommended-monitors.png?alt=media\&token=f2781bd4-9a8a-450b-8d6e-0d012f0e4c15)

To use a recommended monitor:

1. Click on **monitors** in the sidebar.
2. Click **new monitor**.
3. Select **recommended** from the tabs at the top of the page.

## Viewing Release application logs in Datadog

Metrics offer a clear indication of *when* unexpected events take place, while logs often provide the answer to *why* a metric had an unexpected value.

The Datadog agents installed by Release are configured to collect logs from your clusters and nodes automatically.

To view logs in Datadog, click on **logs** in the sidebar, then on **search**.

![Search logs in Datadog](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-23d618e5701c55d28b251d4afc80cac801786eac%2Fdatadog-logs-search.png?alt=media\&token=a0d0bc94-8440-4670-be76-86abefb21916)

Logging in a Kubernetes environment can be quite verbose, so it is helpful to filter by service or container when viewing logs.

## Troubleshooting

### No metrics in Datadog

There might be a communication problem between the Datadog agents in your cluster and Datadog's servers, or the agent might not be installed correctly.

See our [Datadog integration troubleshooting](/integrations/integrations-overview/datadog-agent#troubleshooting-and-common-misconfigurations) steps for more help.

### Missing application logs in Datadog

If you see metrics in Datadog, such as resource usage statistics, but no or only some application logs, make sure your Docker containers write all logs to `stdout` or `stderr`.

Read more about logging in Docker's guide on [viewing logs for a container or service](https://docs.docker.com/config/containers/logging/).

Of particular interest from Docker's guide, are the two examples of applications that always log to files: The official `nginx` Docker image and Apache's `https` Docker image. To output logs to `stdout` and `stderr`, these images either create symbolic links from their log files to `/dev/stdout` and `/dev/stderr`, or write logs directly to `/proc/self/fd/1` and `/proc/self/fd/2`.


# Performance tuning

When it comes to performance tuning, having the right tools to troubleshoot problems with your applications will save you time, money, and headaches. Trying to troubleshoot multiple complex issues with only your expertise and imagination can take days or weeks.

There is no shortage of options for application performance monitoring (APM) software, but here at Release, we use Datadog and highly recommend it to our customers.

Datadog provides a 360-degree view of your infrastructure and applications and can quickly identify bottlenecks, freeing you to focus on more important things.


# Visibility and monitoring

Monitoring your applications is essential to stay on top of different aspects of your business's success, including but not limited to:

* Performance (for example, slow database queries)
* Security (for example, noticing a brute force attack)
* Incident response (for example, figuring out what caused a specific issue)
* Optimization (for example, identifying bottlenecks in a distributed system)

While you can monitor top-level infrastructure directly in your Release dashboard (such as what status a specific pod is in), you can integrate with a number of third-party monitoring providers for deeper analytics across all of your system's custom metrics and logs.

## Supported third-party integrations

Naturally, Release doesn't limit how your system integrates with the third-party provider of your choice, but we have existing flows to help you integrate with the following providers. Speak to your TAM for assistance, and let us know if there are specific providers you would like to see better support for.

### [Datadog](/integrations/monitoring-and-logging-add-ons/datadog)

Datadog provides monitoring and analytics, often at an *infrastructure* level. It can help you get set up with:

* Real-time distributed tracing for all your applications.
* Security and compliance monitoring.
* General monitoring of servers, databases, networks, and other tools.
* Analyzing and filtering logs.
* Playing back sessions to troubleshoot end-user issues.

Release uses Datadog internally and happily recommends it for gaining full insight into your applications.

### [New Relic](/integrations/monitoring-and-logging-add-ons/new-relic)

New Relic and Datadog offer overlapping functionality, but New Relic focuses more on *application*-level monitoring instead of infrastructure. New Relic is an observability platform that helps you with:

* Visualizing application metrics.
* Troubleshooting software.
* Tracking performance and response time metrics.

For example, you might use New Relic to analyze the web browser response times of your users, grouped by their geography and browser type.

### [ELK (Elasticsearch, Logstash, Kibana)](/integrations/monitoring-and-logging-add-ons/elasticsearch-logstash-and-kibana)

ELK describes the common stack of Elasticsearch, Logstash, and Kibana used together. Logstash aggregates server-side logs from multiple sources and passes these to Elasticsearch in a common format. Kibana lets you build charts to visualize this data.

This overlaps with the functionality provided by Datadog and New Relic. If you're looking for an opinionated, pre-built monitoring solution, we would recommend either Datadog or New Relic. If you want to add search and custom visualisations to your existing application logs, then ELK may be easier and faster to set up. ELK doesn't require either financial or time buy-in to a specific monitoring solution like Datadog or New Relic.


# Working with data

Ephemeral environments need some form of data to be usable for testing purposes. With Release there are multiple ways to bring data into an ephemeral environment:

* [Container-based data](/guides-and-examples/advanced-guides/data/container-based-data)
* [Data seeding or data migration](/guides-and-examples/advanced-guides/data/seeding-migration)
* [Instant Datasets](https://github.com/releasehub-com/docs/blob/main/advanced/data/broken-reference/README.md)
* [Cloud database services](/guides-and-examples/advanced-guides/data/cloud-provided)
* [Golden image](/guides-and-examples/advanced-guides/data/golden-images)
* [Third-party platforms](/guides-and-examples/advanced-guides/data/third-party)


# Container-based data

Container-based data is simply the act of creating a container that has the data pre-baked into it, generally loaded into the same database technology that you use in production.

There are a lot of ways to model this, but most databases that have publicly available containers (like MySQL and Postgres) have documentation on where to add files to the container to have them automatically imported on startup.

An example for MySQL would be a Dockerfile that looks like this:

```
FROM mysql
COPY backup.sql /docker-entrypoint-initdb.d/backup.sql
```

This container, built in Release, will automatically load `backup.sql` into the database when it starts up.

A more sophisticated model for larger datasets would include a custom startup script using Docker's COPY in the container that runs `aws s3 cp` to download and restore a large backup.

{% hint style="info" %}
We generally recommend \[Instant Datasets]\(../reference-guide/instant-datasets.md) over baking data into a container, due to the improved ability to keep this data updated. With a container-based approach, you will have to generate your \`.sql\` file and commit it to the repository on a regular basis.
{% endhint %}


# Seeding and migration

Data seeding or data migration involves adding data to ephemeral environments by running a process (either as a separate job or within your service) to populate the database with data that's usable.

### Data seeding

Data seeding generally involves running a job alongside your services that connects to the database to create the appropriate contents. This could be as simple as running `mysql < seed.sql`, or as complex as generating and injecting fake data.

You will need to develop a script that can be run within a job container (most frequently the same container as your service) that reads the appropriate environment variables for your database host, user, and password, and then connects and adds data.

There are standard frameworks for this in most languages.

### Data migrations

Much like data seeding, data migration can be handled in a variety of ways, but it's commonly done within the framework you're writing software in (Django, Rails, etc.). Data migrations must be run at startup, so that the first thing your backend service does is run its own migrations into the empty database to populate it.

{% hint style="danger" %}
We commonly find that nobody has run migrations from scratch in years in the real world, and they often fail from empty databases if not exercised on a regular basis. You may have to do some repair work to make migrations work!
{% endhint %}


# Cloud-provided data

Release's [Instant Datasets](https://github.com/releasehub-com/docs/blob/main/advanced/data/broken-reference/README.md) is a feature that is currently limited to MySQL and Postgres databases.

If your ephemeral environments rely on cloud database services such as [Amazon DynamoDB](https://aws.amazon.com/dynamodb/) or [Azure Cosmos DB](https://azure.microsoft.com/en-us/services/cosmos-db/), you are responsible for creating and maintaining the data that is associated with them.

There are a number of ways you can do this:

### Data automation

In the data automation approach to managing your data, one of the first steps in deploying an ephemeral environment would be to run Terraform or Pulumi to create the cloud resource and populate the data. Once this is done, you will be able to use this resource in your other services.

### Data pooling

In the data pooling approach to managing your data, you would pre-create some number (N) of databases and create an [environment handle](/reference-documentation/account-settings/environment-handles) to round-robin distribute the environment variables that reference these databases to environments. When an environment is created, it "checks out" a handle and uses the associated credentials.

You are responsible for the automation required to routinely reset the data in these databases on some periodic basis so that they are usable over time.


# Golden images

If you don't have any existing databases to snapshot, or you wish to manipulate the data before providing it to a testing environment (for example, making certain private data anonymous), you can provide data to ephemeral environments from a golden image Amazon RDS instance through [Instant Datasets](https://github.com/releasehub-com/docs/blob/main/advanced/data/broken-reference/README.md).

To do this you need to:

* Setup an Amazon RDS instance – you can do this manually or using a tool like Terraform or Pulumi.
* Set this instance to backup daily.
* Update the data in the database frequently.
* Create an [Instant Dataset](https://github.com/releasehub-com/docs/blob/main/advanced/data/broken-reference/README.md) pool in the Release UI.


# Third party

You can use a third-party platform to manage and provide the appropriate data to your Ephemeral environments. We encourage you to discuss your use case with your TAM to establish how Release can help.


# Pausing Instant Datasets

How to pause, resume, or set schedules for Instant Datasets

If your application uses [Instant Datasets](/reference-documentation/instant-datasets-aws), you can pause them to reduce costs. You can set application-wide or environment-specific schedules or both, or pause and resume Instant Datasets manually.

[Application pausing schedules](/guides-and-examples/advanced-guides/pausing-instant-datasets/application-pausing-schedules)

[Pause/resume environments](/guides-and-examples/advanced-guides/pausing-instant-datasets/pausing-resuming-environments)


# Application pausing schedules

How to set, change, or remove default application pause and resume schedules for Instant Datasets

You can set an application-wide default schedule to pause and resume environments. This schedule will only affect **non-production environments** and can be overridden by individual environments

{% hint style="warning" %}
Keep in mind that you can only pause ephemeral environments.
{% endhint %}

## Create or edit a schedule

To create or edit a schedule, go to your **Application Settings**.

Scroll to the **Advanced** section and click **Pausing Schedule**.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-24d37afdff872bd75e77cba59bca991db4ec7784%2Fpausing-schedule-button.png?alt=media)

Add pause and resume times. To keep environments paused during weekends, check the "Stay paused on Saturdays and Sundays" box. Click **Create Pausing Schedule**.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-51b6b837116d7278727280bdd1cfca773510a9a3%2Fcreate.png?alt=media)

The schedule will be set for all **ephemeral** environments for this application.

## Remove a schedule

To remove a schedule, go to **Settings** -> **Pausing Schedule**. Click **Remove Existing Schedule** and your schedule will be removed.

![Remove a schedule](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-c8009e94caeb13594fd41e09d84b8ae459759995%2Fremove.png?alt=media)


# Pause/resume environments

How to pause and resume Instant Datasets in individual environments

Instant Datasets can drive up the costs of your clusters, especially if your data requires larger instances. However, since environments are not used at all times, you can pause them to reduce costs.

{% hint style="warning" %}
Keep in mind that you can only pause ephemeral environments.
{% endhint %}

The following operations are done from the environment **Details** page.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-3cffd71aba18ac33761cfd5b8851f86d34a9746e%2Fenv-details.png?alt=media)

## Manually pause and resume ephermeral environments

Under the **Action** dropdown, click **Pause**.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-7d6e79f023643fa3b567ba99ba90ad2fe0a5d57f%2Fmanual-pause.png?alt=media)

Note that it may take several minutes for this operation to complete.

To resume the environment, click **Resume** under the **Action** dropdown.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-81c298b9595ad6fe7e10a6b3b30c8be1ff603823%2Fmanual-resume.png?alt=media)

## Set up a pause and resume schedule

You can set up a schedule to pause and resume environments daily.

1. In the environment **Settings** page, click **Pausing Schedule**.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-0a52939a06c174b0e8a9eaaa67501dc452296f32%2Fpause-settings-button.png?alt=media)

2. Select the times the environment should pause and resume in the correct fields. To pause the environment over weekends, check the box for "Stay paused on Saturdays and Sundays".

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-d9dd6a107d736085b303a7725c6fd02738ee786f%2Fpausing-form.png?alt=media)

{% hint style="info" %}
To pause ephemeral environments over weekends, leave the pause and resume time fields blank and check the "Stay paused on weekends" box.
{% endhint %}

## Remove a pause and resume schedule

To remove a schedule, go to **Settings** -> **Pausing Schedule**. Click **Remove Existing Schedule** and your schedule will be removed.

![Remove a schedule](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-c8009e94caeb13594fd41e09d84b8ae459759995%2Fremove.png?alt=media)

3. Click **Create Pausing Schedule**.

## Remove a default schedule

{% hint style="info" %}
This feature is only available for applications with a default schedule. Read more about [default schedules](https://docs.release.com/guides-and-examples/advanced-guides/pausing-instant-datasets/application-pausing-schedules).
{% endhint %}

If your application has a default schedule, ephemeral environments will pause and resume according to that schedule.

To remove the default application schedule, on the **Pausing Schedule** page click **Remove default schedule**.

![Remove a schedule](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-c8009e94caeb13594fd41e09d84b8ae459759995%2Fremove.png?alt=media)

To set a new schedule specific to this environment, follow the instructions under [#Set-up-a-pause-and-resume-schedule](#Set-up-a-pause-and-resume-schedule "mention").


# Infrastructure as code

Release's infrastructure runner applies your infrastructure-as-code (IaC) plans as part of your application's workflows.

## Why use infrastructure as code with Release

Release creates Kubernetes clusters in your cloud account (currently AWS or GCP) to host your application's development, staging, and production environments. By deploying your application with Kubernetes, you can start up most of the services you need when the environment is created. But your application may also rely on external services that aren't a good fit for Kubernetes.

Stateful services such as databases, file storage, content delivery networks, and DNS configuration are typically provided by cloud providers such as AWS RDS, S3, CloudFront, and Route53.

Configuring external services manually each time you create a new environment would slow down your workflow and likely cause your development, staging, and production environments to diverge should you start using different services for different environments.

You can use an IaC tool alongside your Release environments to overcome this problem. With minimal configuration, an IaC tool can start, update, and destroy external cloud services for your environments.

## Terraform

Terraform is the first IaC tool supported by Release's infrastructure runner, with Pulumi on the roadmap.

See our [Terraform with Release](/guides-and-examples/advanced-guides/infrastructure/terraform) documentation to start using Terraform for your environments.

## Using Pulumi or other IaC tools

If you use an IaC tool besides Terraform, please contact your technical account manager for help. Release's job runners can run any scripts, including IaC runners, during your application's workflows.


# Terraform

[Terraform by HashiCorp](https://developer.hashicorp.com/terraform/intro) is a popular infrastructure-as-code (IaC) tool that you can use to define cloud infrastructure in a concise configuration language called HashiCorp Configuration Language (HCL). Release can apply your Terraform configuration as part of your application's workflow steps.

Terraform is extensible and can plan and execute infrastructure updates for many cloud and on-premises resources. To keep this guide simple, we'll focus on AWS resources only.

Jump to the section you need:

* [How Terraform works](#how-terraform-works)
* [How Release works with Terraform](#how-release-works-with-terraform)
* [Authenticate Release to execute Terraform](#authenticate-release-to-execute-terraform)
* [How to define infrastructure as code using Terraform](#how-to-define-infrastructure-as-code-using-terraform)
* [How to add Terraform runners to your application in Release](#how-to-add-terraform-runners-to-your-application-in-release)
* [How to provide values to Terraform](#how-to-provide-values-to-terraform)
* [Use AWS SSM parameters to store output values from Terraform](#use-aws-ssm-parameters-to-store-output-values-from-terraform)
* [How to use Terraform modules from private GitHub repositories](#how-to-use-terraform-modules-from-private-github-repositories)
* [Step-by-step summary](#step-by-step-summary)

## How Terraform works

Terraform's workflow is commonly described in three distinct stages:

1. **Write**: You add Terraform configuration to a repository, specifying what your infrastructure should look like.
2. **Plan**: You run Terraform's plan script. Terraform compares live infrastructure with the configuration to output a plan to update infrastructure to represent your configuration.
3. **Apply**: You run Terraform's apply script to approve and execute Terraform's plan.

Each time you change (**write** or update) your Terraform configuration, you will need to run the **plan** and **apply** scripts again to update existing infrastructure to be in line with your desired configuration.

When working with ephemeral infrastructure, we recommend you add a less-often described *fourth* stage to your workflow: **Destroy**. When you no longer need the infrastructure created by Terraform, you can run Terraform's destroy script, which executes a plan that destroys the infrastructure.

{% hint style="warning" %}
Keep in mind that Release skips over the `plan` stage in the standard Terraform workflow and changes are applied without manual approval. This could start up services that incur a cost, so it is important to review Terraform configuration changes before merging with a branch tracked by Release.

To avoid surprise costs, limit dynamic configuration determined by environment variables as much as possible. For example, rather than reading an environment variable to determine the size of an RDS database in Terraform, add an explicit value for this to your Terraform configuration.
{% endhint %}

## How Release works with Terraform

Release adds override files to partially override Terraform's `backend` setting and two blocks in your Terraform configuration: the AWS `assume_role` and `region`.

The `backend` override file looks like this:

```hcl
terraform {
  backend "local" {}
}
```

The AWS `assume_role` and `region` override file is as follows:

```hcl
provider "aws" {
  assume_role {
    # ARN for the AWS role with elevated permissions
    role_arn     = "arn:aws:iam::111111111111:role/release/release-elevated-perms-role"

    # AWS session name is set to your environment ID
    session_name = "RELEASE_ENV_ID"
  }

  # AWS region for your application's cluster
  region = "us-west-2"
}
```

[Terraform override files](https://developer.hashicorp.com/terraform/language/files/override) are merged with existing configuration instead of completely replacing configuration.

Release does not override any other sections besides AWS `assume_role`, AWS `region`, and Terraform `backend`. This means that you can still, for example, add tagging to your AWS provider.

## Authenticate Release to execute Terraform

To create services in your AWS account, Terraform needs to know how to authenticate as a user on your AWS account.

To begin using Terraform for AWS with Release, contact your Release technical account manager to request that they add an AWS role with elevated privileges to your account.

Release's infrastructure runner will assume this privileged role when executing your Terraform configuration.

### How to authenticate with non-AWS Terraform providers

You can add authentication details for other providers to your Terraform configuration. We recommend using [environment variables](/reference-documentation/application-settings/default-environment-variables) to store authentication strings such as usernames and passwords and marking passwords and other auth tokens as secret.

See [How to provide values to Terraform](#how-to-provide-values-to-terraform) for an overview of how to use environment variables in your Terraform configuration.

## How to define infrastructure as code using Terraform

By default, during the plan or apply stages, Terraform looks for files with the `.tf` extension in the current folder.

This simple Terraform configuration example uses the `aws` provider to add a VPC called `example` to the user's AWS account in the `us-east-1` region:

```hcl
# Specify which providers to use
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.0"
    }
  }
}

# Configure the AWS provider (simplified for this example)
provider "aws" {
  region = "us-east-1"
}

# Create a VPC
resource "aws_vpc" "example" {
  cidr_block = "10.0.0.0/16"
}
```

You could add this file to your repository, as `main.tf`, then run `terraform plan` (Terraform will output the plan to update your AWS account), followed by `terraform apply` (Terraform will update your AWS account).

## How to add Terraform runners to your application in Release

When using Terraform with your Release environments, Release can run `terraform plan`, `terraform apply`, and `terraform destroy` on your behalf.

This enables you to start up cloud infrastructure specific to your Release environments when an environment is created, update infrastructure when an environment is patched, or destroy infrastructure when Release tears down an environment.

Modify your Application Template to instruct Release to run Terraform:

* Add an [`infrastructure` section](/reference-documentation/application-settings/application-template/schema-definition#infrastructure).
* Add steps to your `workflows` section to execute the Terraform runners.

### Add an `infrastructure` section to the Application Template

Here's an example `infrastructure` section, with two Terraform runners:

```yaml
infrastructure:
- name: dynamodb-table1
  type: terraform
- name: dynamodb-table2
  type: terraform
  directory: "./dynamodb"
  values: ".release/dynamodb2_values.tfvar"
```

View the infrastructure schema in our [schema documentation](/reference-documentation/application-settings/application-template/schema-definition#infrastructure).

### Add a `workflow` section to the Application Template

Release's infrastructure runner is only executed during one of Release's three predefined workflow stages.

Release workflows consist of three stages:

1. **Setup**: When an environment is created or updated.
2. **Patch**: When your code (or in this case, Terraform configuration) is changed.
3. **Teardown**: When an environment is destroyed.

During setup and patch, Release executes `terraform apply -auto-approve`.

During teardown, Release executes `terraform destroy -auto-approve`.

Here is an example of workflow steps that run infrastructure tasks:

```yaml
workflows:
- name: setup
  parallelize:
  - step: apply_infrastructure
    halt_on_error: true
    tasks:
    - infrastructure.dynamodb-table1
    - infrastructure.dynamodb-table2
- name: patch
  parallelize:
  - step: apply_infrastructure
    halt_on_error: true
    tasks:
    - infrastructure.dynamodb-table1
    - infrastructure.dynamodb-table2
- name: teardown
  parallelize:
  - step: destroy_infrastructure
    halt_on_error: true
    tasks:
    - infrastructure.dynamodb-table1
    - infrastructure.dynamodb-table2
  - step: remove_environment
    tasks:
    - release.remove_environment
```

## How to provide values to Terraform

Terraform modules often take input variables and you can specify input variables for your Terraform configuration.

You can pass variables to Terraform in three ways:

1. Using a custom `values` file.
2. Using environment variables prefixed with `TF_VAR_` in your Release environments.
3. Using Release-specific environment variables.

Let's look at each method in more detail:

### Method 1: Using a custom `values` file

If you declare a `values` file as part of your infrastructure runner, Release will substitute environment variables with matching names in the `values` file before running Terraform.

In the following infrastructure runner declaration, the `values` key points to a file in the repository at `.release/dynamodb2_values.tfvar`:

```hcl
infrastructure:
- name: dynamodb-table2
  type: terraform
  directory: "./dynamodb"
  values: ".release/dynamodb2_values.tfvar"
```

The contents of the `values` file could look as follows:

```hcl
table_name = "test-${RELEASE_ENV_ID}-2"
```

In this example, before running Terraform, Release will replace `${RELEASE_ENV_ID}` with the current Release environment ID and run Terraform with the `-var-file=` option pointing to this `values` file.

### Method 2: Using environment variables prefixed with `TF_VAR_` in your Release environments

Terraform can access environment variables that start with `TF_VAR_`.

Add environment variables with the `TF_VAR_` prefix in their names to your Release environments. Then add Terraform `variable` blocks for these variables to your configuration, omitting `TF_VAR_` from the names.

For example, to use an environment variable called `TF_VAR_TEST_TAG`, add a Terraform `variable` block to your configuration:

```hcl
variable "TEST_TAG" {
  type        = string
  description = "Custom tag for testing"
}
```

You can use this input when configuring resources, for example:

```hcl
# Create a VPC with test tag
resource "aws_vpc" "example" {
  cidr_block = "10.0.0.0/16"
  tags = {
    "testing/test-tag" = "${var.TEST_TAG}"
  }
}
```

### Method 3: Using Release-specific environment variables

For convenience, Release automatically prefixes all environment variables that start with `RELEASE_` with `TF_VAR_`. Access these variables by adding Terraform `variable` blocks.

This means you can reference all environment variables that start with `RELEASE_` by adding them to a `values` file or by declaring `variable` blocks for them.

## Use AWS SSM parameters to store output values from Terraform

To save values from Terraform for use in your applications, we recommend using AWS SSM parameters.

The example below saves a Redis cluster address to an SSM parameter, prefixed with your Release environment's ID:

```hcl
# Create an SSM parameter for the Redis address
resource "aws_ssm_parameter" "redis_address" {
  name        = "/${var.RELEASE_ENV_ID}/redis/address"
  description = "Redis address"
  type        = "String"
  value       = aws_elasticache_cluster.redis.cache_nodes[0].address
}
```

## How to use Terraform modules from private GitHub repositories

Release can download Terraform modules from private GitHub repositories if you have added the Release GitHub app and given it access to the repositories.

To learn how to authorize Release to access your repositories, read our documentation about [integrating GitHub with Release](/integrations/source-control-integrations/github).

In the Terraform configuration example below, Release will automatically add an access token to the Git configuration and download your private repository:

```hcl
module "dynamodb" {
  source = "github.com/YOUR-GH-ACCOUNT/private-repository//dynamodb"
}
```

## Step-by-step summary

To summarize, add Terraform to your application on Release by following these steps:

1. Add your Terraform configuration to your repository.
2. Add an `infrastructure` section with a `terraform` runner to your Application Template.
3. Create workflow steps in the application template to execute your infrastructure runners.
4. Map any environment variables you would like to use with Terraform in your Release application's default environment variables file.
5. Deploy a new environment to test your Terraform configuration.

## Example application with Terraform

To see Terraform in action, work through our [Terraform, Flask, and Redis example](/guides-and-examples/common-setup-examples/terraform-flask-redis).


# Account settings

### Administration of your Release account

* [Account info](/reference-documentation/account-settings/account-info)
* [Managing users](/reference-documentation/account-settings/managing-users)
* [Build arguments](/reference-documentation/account-settings/build-settings/build-args)
* [Add integrations](/reference-documentation/account-settings/add-integrations)
* [View clusters and cloud integrations](/reference-documentation/account-settings/view-clusters)
* [Add datasets](/reference-documentation/account-settings/add-datasets)
* [Environment handles](/reference-documentation/account-settings/environment-handles)

### Accessing Account Settings

Access your Account Settings page by clicking on the slider icon in the top right corner of the Release interface.

![Account Settings Shortcut for the Release Account](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-5b6d4f68fa7de5fa1eb7044e0f4e6ed2ea77c189%2Faccount_settings_btn.png?alt=media)

### Switching accounts

If you are part of more than one organization in your Release account, switch your account view by clicking the account avatar in the top right corner of the Release interface and selecting the account you want to view details for from the dropdown menu.

![Account switching](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-8e5761714787e7edd91fb9a8f08c8f6fdf2b95fb%2Faccount_switching.png?alt=media\&token=58e3b79f-7ef9-4d65-a527-996662427116)


# Account info

Edit your account-level information by clicking the Settings button in the left navigation bar.

Edit general information about your workspace in the "Details" sub menu.

* The **Owner** can change the name or avatar of the workspace.
* Professional plan account holders can edit the domain name.

![Workspace details Example](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-874dfd1d0d97982a736051c7fea6ee7238fc7064%2Fworkspace-details.png?alt=media)

## Workspace settings

Under **Settings > General** you can configure properties such as when stale environments are destroyed or recreated, and how public access to your ephemeral environments works.

![General settings](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-ae5ea45e5482764a903ac07a0e9af27ae2a4152c%2Fworkspace-settings.png?alt=media)

### Environment settings

* **Create Environments when Pull Requests are opened:** If this setting is on, Release will create a new ephemeral environment for each new pull request made on your integrated GitHub repositories. We recommend leaving this on so your team can view preview environments for each proposed change to your code base. This can be [overridden](/reference-documentation/application-settings/github-1) at the app level as needed.
* **Recreate Pull Request environments on push:** If your environments are set to expire, you can recreate them when new changes are pushed to a branch. When this setting is on, Release will check for related expired environments for each change pushed. If it finds a match, it will recreate the environment with the latest code changes.
* **Create ephemeral environments for users who don't have Release accounts:** If you have users who will open PRs to your GitHub repository but **don't** have Release accounts, turn this setting on so that Release can create an ephemeral environment for them.
* **Destroy ephemeral environments after no activity:** If your team opens a high volume of pull requests and your cluster is filling up quickly, then it's a good idea to destroy your ephemeral environments as they become stale. Switch this feature on to specify how long your ephemeral environments will stay alive. If you have a low volume of pull requests, then you can leave this off and manually clean up environments as they are no longer needed. If you need an old ephemeral environment that has expired, you can easily re-create it by closing the stale pull request and re-opening it, or by turning on the Recreate feature described above.
* **Period to wait before marking an environment as inactive:** Choose how long you want your environments to stay around for even if they are not being used. A longer time period is more convenient, while a shorter one saves on costs. The default of 7 days works for many people.
* **Enable unauthenticated environments page:** In some cases, it is useful for non-technical staff to access preview environments. For example, your Product Managers, UI/UX designers, and other managerial staff may want to check out new changes and offer design feedback or approval. Turn this feature on to allow your ephemeral environments to be viewed publicly using a `<magic_string>` slug that only your team knows about. You can edit and save your team's slug.

### Cluster settings

* **Use elevated permissions when creating clusters:** This uses our elevated permissions role which might be required in some cases, such as running administrative services. Contact us to find out the details of what permissions are supplied in the elevated role permissions and how to access the ARN for assuming that role, but usually leave this setting off.

### Miscellaneous

* **Enable GitOps:** GitOps is a framework for using your git repository to manage your infrastructure and app configuration files. Read more about [GitOps](/reference-documentation/gitops), but if you prefer configuring your environment settinsg through the web console then leave this setting off.

## Source control settings

If you've integrated your Release account with source control, the Source Control settings page allows you to set how Release automatically posts information back to your source control provider.

![Account Settings Example](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-a6f631d787d29a9e1f7cb5b73d1571b02b2c3117%2Faccount-settings-source-control.png?alt=media)

* **Pull request comments:** Release will automatically comment on a pull request with a link to the running environment for that PR. More info [here](/integrations/source-control-integrations/github/pull-request-comments).
* **Pull request status updates:** This setting integrates with [GitHub's status check features](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/collaborating-on-repositories-with-code-quality-features/about-status-checks) to push information about the PR status and any checks from Release to GitHub. More info [here](/integrations/source-control-integrations/github/github-statuses).
* **Pull request deployment updates:** This setting integrates with [GitHub's deployment features](https://docs.github.com/en/rest/deployments) to push information from Release to GitHub. More info [here](/integrations/source-control-integrations/github/github-deployments).
* **Create Environments when the following label is applied to a Pull Request:** This setting integrates with [GitHub's labeling features](https://docs.github.com/en/issues/using-labels-and-milestones-to-track-work/managing-labels) so that you can create environments in Release by adding labels to PRs in GitHub. More info [here](/integrations/source-control-integrations/github/pull-request-labels). These labels can be overridden by the [application settings](/reference-documentation/application-settings/github).


# Managing users

Expand your team and share your environments

## Inviting Team Members

To add team members to your Release account, navigate to **Settings** -> **Members**. Click **Send Invites** to bring up the form for inviting a new member.

Enter in their email address and select a role (see below). The team member will receive an invitation and upon logging in or signing up with Release, they will be added to your Release account.

You can change a user's access level under the "Users" section. Select either "Member" to give them limited permissions or "Owner" to give them full permissions.

![Member Permissions](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-8c37bb3311f4ccd1335e9057880ee64ddae2ca84%2Fmembers-permissions.png?alt=media)

## User permissions

Users with "Owner" permissions can configure all aspects of your Release account, including changing the account information such as the name and avatar, and removing existing integrations such as domain names, cloud provider integrations, and clusters.

Users with "member" permissions can view all settings and can change some account-level settings, but cannot access destructive settings for projects that they do not own.

Users with "instance\_accessor" permessions have the same permissions as members, but additionally can access the Logs and Terminal for services in environments they did not create.

For each category of settings, the permissions apply as follows:

### Account settings

* Members **can** change Account Settings, including when ephemeral environments are created and destroyed.
* Members **can** change GitHub settings, including whether comments are sent from Release to GitHub PRs, etc.

### Users

* Members **can** *view* other members, including their roles and their email addresses.
* Members **cannot** invite new members or remove existing members.

### Build settings

* Members **can** view, edit, or delete build arguments.

### Integrations

* Members **can** add new integrations.

### Domains

* Members **can** add new domains and delete or verify existing ones.

### Clusters

* Members **can** create new clusters and cloud provider integrations.
* Members **can** set the default cluster, add new node groups, and edit existing node groups.
* Members **cannot** upgrade existing clusters.

### Datasets

* Members **can** create new datasets.

### Environment handles

* Members **can** create new environment handles.
* Members **can** duplicate or delete existing environment handles.


# Build settings

Pass build arguments and SSH keys to builds in Release.

* [Build arguments](/reference-documentation/account-settings/build-settings/build-args)
* [Build SSH keys](/reference-documentation/account-settings/build-settings/build-ssh-keys)


# Build arguments

Docker build arguments and static service environment variables

Build arguments are key-value pairs that are used to initialize a build with either `docker-compose` or static service environment variables. A static service is any service that doesn't require a container. Visit [our guide to static service deployment](https://docs.releaseapp.io/static-service-deployment) for more details.

Build arguments can be global and used for all your applications, or specific to a single application.

You can add global build arguments to your global Configuration under the **Builds** tab. Add secret environment variables only relevant to a single application to App Settings in the "Advanced Settings" section to prevent exposing them to other containers.

## Add account-level build arguments

Navigate to **Configuration**. Select **Build Arguments**.

![Build Arguments Setting](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-44c407f8aee4eefdb4284e10909a41397e49532b%2Fbuild-args.png?alt=media)

An example use case for global build arguments is an API key for a static build.

## Using secrets as build arguments

{% hint style="success" %}
Read about how secrets are referenced from SSM or the Release Secrets Manager in [Secrets vaults](/reference-documentation/environment-settings/environment-specific-environment-variables/secrets#referencing-cloud-secrets).
{% endhint %}

{% hint style="warning" %}
Environment variables and environment-specific variables are not accessible in the `docker build` step, so you need to use build arguments as explained in this guide.
{% endhint %}

In Docker, regular build arguments using the `ARG=SOMEVAL` format are stored in the Docker image and can be retrieved if someone has access to your image or repository. This may not be a problem for private images unless you store private keys or sensitive information that should not be exposed, even internally.

To safely access secrets in Docker builds, Release recommends using Docker secrets. To access the secret, you need to make some changes to your Dockerfile.

* [ ] You can enter your secret directly as a string (Release stores all values encrypted in the database for you), but it's preferable to create a secret or use an existing secret in one of the supported [secrets managers](/reference-documentation/environment-settings/environment-specific-environment-variables/secrets). Secrets stored in secrets managers cannot be retrieved from the UI, which adds a layer of security over your secrets.
* [ ] Click the padlock icon to convert the secret or secret reference to a Docker secret. Release will mount the value at a file location you can specify later.
* [ ] The Docker secret will be available with the same ID as the key you specified, and will be mounted under `/var/secrets/ID` in the file system. For example, in your Dockerfile, locate your build argument access code, which will look something like this:

```docker
ARG API_TOKEN
RUN API_TOKEN=$API_TOKEN start_server.sh
```

Now change it to look like this:

```docker
RUN --mount=type=secret,id=API_TOKEN API_TOKEN=$(cat /run/secrets/API_TOKEN) \
    start_server.sh
```

* [ ] (Optional) In some cases, startup scripts can automatically read build arguments as environment variables if you use the `_FILE` convention. In this [documentation example](https://docs.docker.com/compose/use-secrets/#advanced) from Docker, the startup script knows to access the value of `API_TOKEN` from the file location pointed to by `API_TOKEN_FILE`. However, this convention only applies under certain circumstances, so be sure to check if it will work in your case.

## Add application-level build arguments

Navigate to the Settings tab within an application. Select **Build Arguments**. The application-specific build arguments will override the account-wide build argument settings.

![Navigating to App-Level Build Args under Application Advanced Settings](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-0659a4d8981d93ddc900f55cebf2e9d39fc04776%2Fapp-level-build-args.png?alt=media)

Notice how `USERNAME` is inherited from the account-level build arguments, while `API_TOKEN` is overridden by this application.


# Build SSH keys

Securely pass SSH keys to the Docker build command

With Release, you can pass SSH keys to the `docker build` command as part of the build process.

You might need to pass SSH keys to a `docker build` command if the build process pulls code or other dependencies from a private Git repository hosted on a remote server. To access the repository, the build process will authenticate with the server using SSH keys.

## Add the SSH key to a Dockerfile

Start by updating your Dockerfile to use the passed-in SSH keys.

Use `RUN --mount=type=ssh,id=your_key_name` to mount the SSH key in your Dockerfile. Read more about SSH keys and Docker builds in the [Docker documentation](https://docs.docker.com/engine/reference/commandline/buildx_build/#ssh).

If you plan to use only one SSH key in your organization, you can use `RUN --mount=type=ssh` without the ID, and name the SSH key `default` in the Release UI.

## Add the SSH key to Release

In Release, you can set build SSH keys at the account level. These SSH keys will be passed to all the builds in your account.

Access the build SSH keys by going to **Configuration** -> **SSH Keys**

![Add a new SSH key](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-f673b8e427b01f6592ae592b22ec53be66d9f108%2FSSH-Keys.png?alt=media)

The following fields are required to add a new SSH key:

* **Name:** The name of the SSH key. Release uses this name to pass the SSH key when building the Dockerfile. The SSH key name should match the `id` of the SSH key in your Dockerfile.
* **Secret Identifier:** This can be either the full private SSH key (including `BEGIN` and `END` tags) or the secret ID of the SSH key stored in your cloud secret storage and configured in Release.

### Example Dockerfile and Release SSH key

The following Dockerfile example clones a GitHub repo and uses the SSH key with the name `myssh`:

{% code title="Dockerfile" %}

```docker
# start from a base image
FROM ubuntu:20.04

# install dependencies
RUN apt-get update && apt-get install -y git
RUN sed /^StrictHostKeyChecking/d /etc/ssh/ssh_config; \
  echo StrictHostKeyChecking no >> /etc/ssh/ssh_config
  
# clone the target repository
RUN --mount=type=ssh,id=myssh git clone git@github.com:release/example_privaterepo.git
```

{% endcode %}

![Note the name of the SSH key matches the ID in the Dockerfile](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-adbda219c9ba95535ea3d6d05ef7d05736a8ecd5%2Fdocker-ssh.png?alt=media)

## SSH keys and secrets

By default, the SSH keys you add in the Release Build Settings UI are stored in the Release Secrets Manager. However, you can use SSH keys stored in AWS SSM, AWS Secrets Manager, or GCP Secret Manager to pass to Docker builds, as long as these secrets managers are configured with Release. Learn more about configuring your secrets manager with Release in our [importing secrets from external secrets managers](/reference-documentation/environment-settings/environment-specific-environment-variables/secrets) documentation.


# Add integrations

Connect your account with an external service to get the most out of Release features

Find the full documentation on the [integrations overview](/integrations/integrations-overview) page.


# View clusters and cloud integrations

If you have a professional account that is integrated with a cloud account, you can view the clusters that are in use by your Release environments on the Clusters page.

This page is a read-only overview showing the names, identifiers, and regions of clusters currently connected in Release.

![Screen showing the clusters and cloud integration accounts](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-e9e07b0622b9b493b80b3f3f19fe873375f10d7e%2Fview-clusters.png?alt=media)


# Add datasets

Create production data replicas for accurate testing in your environments

You can add Instant Datasets to your account so that your environments can access seed data to simulate a production environment. This feature is available to Professional accounts.

Refer to the detailed [Instant Datasets](/reference-documentation/instant-datasets-aws) page for more information on how to set up and configure an Instant Dataset.

To add a dataset, click on the **Datasets** tab to navigate to the Instant Datasets page.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-189094c3125cc0600a6154b1290b81742aa9aa69%2Fmanage-datasets-button.png?alt=media)

Click on the **Create Dataset** button to open a modal form to fill out. Release will provide the available clusters and snapshots you can pick from.

![Dataset Creation Form](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-51469fbc2a0e7d131da62e632eaa724af263d020%2Fcreate-1.png?alt=media)

Give your database a name and create a user password, then click **Create Dataset**.

![Giving the database a name and password](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-7dfacb37a0f4e71463e9191317561a2e6d40d36b%2Fcreate-2.png?alt=media)

Your dataset will now be visible in the Datasets view. It will show "Pending" until the RDS instances are available.

![View Datasets](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-50c15b7f21d8c55fb17252a8fcb09c1642f3efae%2Fdatasets-view.png?alt=media)


# Environment handles

Create custom, unique resource pools for your environments.

An environment "handle" is a pre-generated, unique identifier that can be used for deterministic resource pools, such as environment URLs. For example, you can use an environment handle to create deterministic URLs for your environments for OAuth applications.

In Release, Professional account-holders can create a custom handle name and specify the number of resource handles to use it with. One of these resource handles can be "checked out" by an environment, which will then display the handle in its domain name.

For example, the end result of your environment handle being inserted into an environment's domain URL would look something like this:

`https://service-${env_id}-appname.releaseapp.io`

This is no different to a regular ephemeral environment space, except that your environment handle `${env_id}` is allocated from the handle pool.

## Create an environment handle

Select the **Env Handles** tab in Account Settings. In the Environment Handles page, click the **New** button.

![List of Environment Handles](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-a19cc8545b1f5d9b772498e0b224fac6b3131fdc%2Fenv_handle_example.png?alt=media\&token=f8ff4f36-a354-4978-be0a-baf9a64163a7)

In the modal window that opens, enter a name for your handle and how many handles you need. Click **Create**.

![Create Handle modal window](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-738b268044887bde8a2d3eda34bafc4efaac30e2%2Fenv_handle_create.png?alt=media\&token=d2c6c8c8-c369-47b5-94f3-eb73d6c0b8c5)

## View environment handle details

In the Environment Handle Details page, you can see the name of each resource handle and which environment currently has it checked out. Notice how each resource handle name begins with the base handle name, followed by an incrementing index number. This is what will appear in the environment's URL.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-4cbccaa11bff392770382d124cee5daba4eb3848%2Fenv-handles.png?alt=media\&token=9f658581-ecfd-4dbf-8df0-3275c4a08d15)

## Add the environment handle to your Application Template

In order for an environment to check out a unique handle, add the following to your [Application Template](https://docs.releasehub.com/reference-documentation/application-settings/application-template):

```
environment_templates:
- name: ephemeral
  datasets:
  - name: handle_name
```

{% hint style="info" %}
URL handle pools are only applied during environment creation and cannot be changed or applied retroactively to an existing environment.
{% endhint %}

## Environment-specific environment variables

Environment handles allow you to set specific environment variables for each individual environment. For example, you may have a set of third-party service tokens that require a unique ID for each environment for testing purposes. This is a common pattern for services like Firebase or Twilio. You can provide each environment with a specific set of environment variables to be applied to that environment.

To add an environment variable to a handle, expand the section below the environment handle and click the plus sign. Insert the key and value in the relevant fields and click the check mark to save your changes.

![Adding a specific environment value to the "llama" environment handle.](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-919a5db95e65603e1b46965ae5e244c5526ca0eb%2Fenvironment-handles-variables.png?alt=media\&token=ed6e169f-1027-4acb-b616-874f2f9ae2a2)

Remove an environment variable from a handle by clicking the trash sign next to it.


# Workflows in Release


# Stages of workflows

There are currently four stages in the Release workflow:

## The build stage

The build stage in Release is when the `docker build` commands are run. It's the first step that must take place before any other stages can be executed. You can't configure the build stage in Release.

## The setup stage

The setup stage is the beginning of the lifecycle of an environment. In this stage, Release issues `docker run` commands, creates the infrastructure and objects that are necessary for the fully functioning environment to be constructed, and then deploys the code.

This initial code deployment takes place during the setup stage so that the environment is fully functional when the setup stage is complete.

The setup stage must run at least once before the patch stage, and again whenever the Application Template or Environment Variables Template is changed to apply changes.

The setup stage can be run manually to rebuild an existing environment that is out of sync with the Release version, which may occur due to some unforeseen failure or restart in Kubernetes.

## The patch stage

The patch stage is only run when the build stage and setup stage have been completed. This is the stage at which new code is deployed or updated.

A patch can be triggered from a Git push to the same tracking branch for the environment, or it can be triggered manually using the **Deploy** button on the Environment Details page or the Build Details page.

The patch stage is somewhat like a traditional "deploy" stage, when code is updated and jobs are run. The key difference between Release's patch stage and a traditional deploy stage is that the patch stage does not need to do any of the heavy lifting of setting up necessary infrastructure, as this will have been completed in the setup stage. For example, in the patch stage, there is no need to initialise databases or create load balancers.

## The teardown stage

The teardown stage is run when an environment is either expired or deleted. You can use the teardown stage to remove any extra infrastructure, permanent entries, or storage created elsewhere that Release is not aware of. For example, you can use the teardown stage to destroy infrastructure created with Terraform or Pulumi, or to delete or drop tables in a shared database or storage system.

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-76d475e6df15b31bbb2f236850165eeab9019b5b%2FWorkflow%20stages.jpg?alt=media)


# Serial deployments

In Release, deployments are serial by default: all charts, services, and jobs execute in the order they are defined in the Application Template.

There are some cases in which the order of deployment may not be obvious, for example, whenever the keywords `charts.all`, `services.all`, or `jobs.all` are referenced. However, the order of each step in the array is normally well-defined and you can count on the execution order whenever the steps are explicit. The `*.all` keywords are used as a catch-all to ensure that no services or jobs are forgotten.

Here's an example to visualize the workflow:

```yaml
workflows:
  - name: setup
    order_from:
      - jobs.migrate
      - services.all
      - jobs.setup
  - name: patch
    order_from:
      - jobs.migrate
      - services.backend
      - services.frontend
  - name: teardown
    order_from:
    - release.remove_environment
```

This would yield the following steps:

![Serial Workflow example](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-704472c61210c2ea9c5709ed721b39066fe5b235%2Fserial_deployment_workflow.jpg?alt=media)


# Parallel deployments

You may find that Release's default serial deployment is slow. A series of 30-second deployments can add up to several minutes when executed serially.

To improve on deployment time, you can break your Release workflow up into **steps** and allow steps to run **tasks** in parallel.

Let's take a look at how parallel deployment works.

## Steps

Steps run sequentially.

You can break your workflow up into serial-dependent steps to ensure that prerequisites are set up first, before dependent services or databases are deployed. This is the same way the default serial deployment works.

## Tasks

You can then group tasks together under the workflow steps, and each step can run in parallel as long as there are no interdependencies between tasks in a step.

## Errors

Errors are typically ignored during tasks because Kubernetes can automatically restart and keep trying until a service succeeds.

For example, if service B depends on service A, and you mistakenly run service B before service A, service B may fail to start. Kubernetes will retry service B and if service A is now functioning, then service B will start properly.

This may not be the behaviour you want, however. A job in a task might be critical to some dependency further on in the tasks or steps, and you may prefer the whole chain to fail as soon as a failure is detected. In such a case, you can use the `halt_on_error: true` key to stop the stage and abort the deployment. Any services or jobs successfully started in a previous step or task may still be running or have run, though, so we recommend you assess how you will clean up on the next run.

## Waiting

You may not want to wait for a task to succeed completely before moving on. This may be the case, for example, if a frontend takes a long time to deploy and has no particular dependencies in the steps. You can start the task in an earlier step and use the `wait_for_finish: false` key to let it run in parallel with the other steps.

## Visualizing parallel deployments

Here's an example workflow definition:

```yaml
workflows:
  - name: setup
    parallelize:
    - step: frontend
      tasks: [services.frontend]
      wait_for_finish: false
    - step: migrate
      tasks: [jobs.migrate, jobs.scrub]
      halt_on_error: true
    - step: backend
      tasks: [services.backend]
```

These steps would deploy like this:

![](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-c1ad0a04a882f54f98f7a9de32bf8968088d8416%2FParallel%20Workflow.jpg?alt=media)


# Rolling deployments

Release automatically handles most aspects of rolling deployments for you. This means that when you push code changes, your applications will not face any downtime.

A rolling update does not apply code changes to all of your nodes at once, but rather works as follows:

* Nodes in your cluster are terminated one at a time and replaced with new nodes running the new version of your software.
* While each node is down, traffic is routed only to the healthy nodes. For a short while, some users may see the new version of your software while others still see the old version.
* The update process for each node is started only after the previous node comes up and is confirmed healthy.

As you can see in the diagram below, during a rolling update, the majority of nodes are available to serve traffic at all times.

![Rolling Updates Diagram](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-da5876dc5004ab1d7ceb65d5af64759483133e48%2Frolling-deployments-diagram.png?alt=media\&token=a2b0ef83-f88c-46b2-85e7-cf3cbf55bf0c)

You can read more about how rolling deployments work [in the Google Cloud documentation](https://cloud.google.com/kubernetes-engine/docs/how-to/updating-apps), but note that Release handles this for you automatically and there is no need for you to follow the steps listed on that page.

## Considerations when using rolling deployments

While rolling deployments work well for many use cases, they may not always be appropriate. For example, you may need to be sure that all your users see the same version of your software. With rolling updates, it can be difficult to know which version of your application users are accessing at a given point in time.

Database migrations can also be tricky with rolling deployments. If you need to add or rename a database column, for example, you'll need to be careful with your application code and the order your changes are applied to ensure that your database logic can handle both the old and the new application code.

## Blue-green and rainbow deployments

An alternative to rolling deployments is blue-green deployments. With this model, you have two copies of your environment, one "blue" and one "green". Only a single environment is user-facing at any given time. When you deploy changes, you deploy them to the non-user-facing environment, and only once all changes are completed and confirmed working do you switch all users to the new environment simultaneously.

This can result in cleaner updates, but it also means that any bugs or issues will affect all of your users. Traditionally, the blue-green deployment model is also associated with higher costs, as you have to maintain two full instances of your complete infrastructure.

### Rainbow deployments

Because Release gives you environments as primitive building blocks, you can extend the blue-green deployment model further. Instead of having only two environments, you can add as many "colors" as you need, having a completely new environment for every change. We call these rainbow deployments.

Please contact us if you would like to use rainbow deployments with your account, as this is currently a manual feature that we activate for you.


# Rainbow deployments

{% hint style="info" %}
This is an opt-in Enterprise feature. Please contact us at <support@release.com> if you want to enable rainbow deployments on your account.
{% endhint %}

Rainbow deployments are an extension of the popular blue-green deployment pattern.

* With blue-green deployments, you have two copies of your production environment, "blue" and "green". Either the blue or green environment is the active production environment at any given time. When you deploy changes, you deploy them to the environment that is not currently active and then switch the recently changed environment to active. This means that your development and production environments remain closely mirrored by default.
* With Release, you have easy access to unlimited environments, so you can extend the blue-green pattern to infinite colors of the rainbow. Each deployment happens in a **namespace**, which is a copy of your production environment. Each namespace gets a color, and you can have as many colors as you need.

You can think of these deployments as lanes on a highway: Two lanes are better than one, but infinite lanes are better than two. Let's say you are currently running version 1 of your software in the "blue" environment and you deploy version 2 to the "green" environment. You switch "blue" over to your main environment and direct user traffic there, while keeping "green" running to complete a few asynchronous jobs that still need to complete. Now you discover a problem with version 2 and deploy a hotfix. If you're stuck with only two environments, blue and green, you have to choose between killing the jobs running on green or pushing new code directly to your live environment. If you have infinite colors available, you can keep both the previous environments running while starting a new environment, "yellow", for example.

![Rainbow environments](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-6a73734afa45be2efda0df69a24677b658c296cc%2Frainbow-deployments-overview.png?alt=media\&token=b3d8a07f-c6b2-4ab5-b91d-2c08ed8d6583)

We use rainbow deployments internally at Release, so we'll be happy to help you decide if they are a good fit for you and advise you on using them effectively in your specific case.

## How to use rainbow deployments

After the rainbow deployment feature has been activated in your account, you will be able to configure rainbow deployments through your Application Template file. Note that it only works for **permanent** environments, not ephemeral ones. You'll specify a timeout period and how many active environments you want, as described in the next section.

If you activate rainbow deployments in an environment, you'll see a rainbow icon to indicate that it is active for that environment, as shown below.

![rainbow icon](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-80f1b0149bd899cc56a19e8005768e9f2a1d7192%2Frainbow-deployments-rainbow-icon.png?alt=media)

This will also activate the **Namespaces** tab, where you can see a number of subenvironments, each with its own color, as shown below.

![Namespaces tab](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-fa14ef793b899a90dcbece338f9d575ff6299d1b%2Frainbow-deployments-namespaces-tab.png?alt=media\&token=068fba7d-6ecb-4c61-acd8-d85aed546efe)

A new production deployment will automatically go to a new namespace, get a new color, and immediately become the active namespace in that environment. The previous namespace will stay alive for a configured timeout period.

This timeout period is useful in case you have long-running jobs or other tasks that need to complete before the environment disappears.

* After the timeout, the old namespace will be removed.
* If you redeploy exactly the same build, it will still create a new namespace and expire the previous one, but the color won't change.

## Configuring rainbow deployments

To turn rainbow deployments on in a specific environment, add the following block to your Application Template YAML file:

```yaml
rainbow_deployment_configuration:
    type: infinite
    termination_threshold: 180.minutes
```

* The `type` value defines the kind of rainbow deployment. Currently, we only support `infinite` for this value, which means that there is no limit to how many namespaces can be created in an environment.
* The `termination_threshold` value specifies the timeout for environments to stay alive after a new environment is deployed. You can set this in seconds, minutes, or hours, and you should choose a value greater than the longest-running job or process in that namespace to avoid losing any in-progress work.

Note that the `termination_threshold` is applied when the namespace is **created**, so if you update it and redeploy your build, then your existing namespace will still terminate after the previously defined value has elapsed.

## State and state changes for rainbow deployments

A namespace can take on one of several states. These are:

* **Active**: The currently active environment, also indicated by an arrow in the list of namespaces.
* **Pending**: The namespace is initializing with your configuration. This state should only last a few seconds before moving into **Activating**.
* **Activating**: The namespace that will be active next. As soon as it is ready, it will become active and the previously active namespace will become **Inactive**.
* **Inactive**: Any previously active namespace that has not yet been destroyed. You'll see a countdown timer next to inactive namespaces indicating when they are due for termination.
* **Destroyed**: Any previously active namespace that has passed its `termination_threshold`. You can still see information such as the build number, configuration version, and creation time for these environments.

There are two icons with arrows to the left of the namespaces that show the currently active environment and (if applicable) the next environment that will activate. You can see these in the image below.

![statuses](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-489054857df563e5dfabe9aa3ecdae2763875dd8%2Frainbow-deployments-statuses.png?alt=media\&token=d4fd2fdb-c287-4285-8bfa-a0d3a05315e7)

Inactive environments show a timer indicating when they will be destroyed. You can see this below.

![expiring](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-07d0ecea94af46b25ea01447728d8cc870bd3851%2Frainbow-deployments-expiring.png?alt=media\&token=ca88feeb-8f1c-4aca-8b18-bfb5c973665f)

## Namespace details

By clicking on the color for any namespace, you'll see a Namespace Details page that shows the same information (name, time until removal, deployment ID, and version) as you can see on the overview page, as well as a list of instances running in that namespace.

![namespace details](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-22757471f1e0f85586f1d0739ba5b1280abca034%2Frainbow-deployments-namespace-details.png?alt=media\&token=9ab4119b-35af-4ef6-acb7-e9fc79f0dcaa)

## What lives inside rainbow environments?

In an ideal world, rainbow deployments mean creating infinite copies of all resources that make up a given environment. In reality, it is not practical to have infinite copies of resources such as your production database or Apache Kafka instances. Therefore, resources like this often live "outside" of this setup.

You'll have to structure your systems carefully so that you don't, for example, have conflicts when several expired but not destroyed environments are competing for the same external resource.

## Choosing colors

In case you wonder how the colors for environments are chosen, we use the short Git hash (conveniently, a six-character hex number) and translate it directly to a hex color code.


# Networking


# Network architecture (AWS)

Release environments depend on networking components from both Kubernetes and cloud providers (for example, AWS and GCP). Here's an overview of our architecture from a networking perspective, and an example of how traffic flows through each step.

This document uses AWS names for simplicity. If you're using GCP, in most cases the architecture is the same but uses GCP equivalents to the AWS components.

## Architecture overview

The main cloud components used in Release's architecture are as follows:

* **Route 53** for public DNS mapping and to assign unique URLs for each of your permanent and ephemeral environments.
* **DynamoDB** to keep a table mapping each URL to a specific environment or S3 bucket.
* **CloudFront** and **Lambda\@Edge** to route incoming traffic to the correct environment or S3 bucket.
* **S3** to host static websites and assets.
* **Internet Gateway** to allow incoming traffic into a VPC.
* **Elastic Load Balancer (ELB)** to route traffic to the Kubernetes Ingress Controller.
* **Elastic Kubernetes Service (EKS)** to run Kubernetes node groups.

When a user visits a URL, the request passes through Route 53. We look up the URL in the DynamoDB table and route the request either to S3 (for static websites) or to a load balancer (for containerized services). In the case of containerized services, the request then passes through to Kubernetes Ingress Controller, and then to the specific Kubernetes pod running the associated service.

Let's take a closer look at how this works with an example.

## Following a network request

Let's assume we're running three services, each in their own environment.

* `env-123.example.com` runs a static website.
* `env-456.example.com` runs a static website.
* `env-789.example.com` runs a containerized service with a frontend JavaScript application and a backend API.

We'll follow a user's request from the first DNS request until it hits S3 (for the static websites) or a Kubernetes pod (for the containerized service).

### 1. Resolving DNS and URL mapping

The first service we hit is Route 53, which handles global DNS. Route 53 routes the request to AWS CloudFront. Specifically, Release does this using AWS *Alias* records. An Alias record is a Route 53–specific extension to standard DNS functionality, which can route traffic to specific AWS resources, including CloudFront, instead of to an IP address or another domain.

Usually, CloudFront is used as CDN to serve static files from machines close to your end user's location, but Release also uses it to intelligently route traffic based on environment type. It does this by using a Lambda\@Edge function to look up the URL in a DynamoDB table. The DynamoDB table contains a record of every Release environment, including the URL and the Release service it maps to.

For example, the simplified DynamoDB table below shows that environments 123 and 456 should be routed to specific S3 buckets, while 789 should be routed to a specific load balancer.

```
Domain              | Target
--------------------+--------------------------------
env-123.example.com | s3://acme-static-sites/env-123
env-456.example.com | s3://acme-static-sites/env-456
env-789.example.com |  arn:aws:elasticloadbalancing..
-----------------------------------------------------
```

Requests for static websites are immediately served from S3. If the user's request maps to a containerized application, it will be routed to an Internet Gateway and then onwards as described in the next section.

You can see an overview of this in the diagram below.

![Networking DNS and DynamoDB](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-f0538e9a50765e42e21a7e4d48d7a56bda4f8352%2Fnetworking-1.png?alt=media\&token=a5962aec-ff2e-4a0c-9bf8-dd888b57f966)

## 2. Load balancing and ingress

Your Kubernetes clusters are run in a private subnet in your VPC, and are therefore mainly inaccessible from the outside world. The request for environment 789 is sent through an Internet Gateway to an elastic load balancer, and then passed on to a Kubernetes Ingress Controller, which will send the request on to a specific service running in a pod.

More specifically, the request first passes through AWS Internet Gateway to get into your VPC. From here, it hits an Elastic Load Balancer which lives in a public subnet of your VPC. The Elastic Load Balancer passes the request to EKS, which runs in a private subnet of your VPC.

First the request hits the Nginx Kubernetes Ingress Controller, and then gets routed to a specific Release service, running within a Kubernetes namespace.

In this example, our service consists of a frontend application running on port 8080 and a backend application running on port 3000. The Kubernetes Ingress Controller routes the request to our frontend port, which then communicates with the backend service. The backend service runs any application logic, and then passes data back to the frontend, from where the result can be passed back through to the end user.

The actual Kubernetes pods are hosted on AWS EC2 instances, which are controlled by EKS.

You can see an overview of this in the diagram below. The internet gateway at the top is the same as the one shown in the bottom left of the previous diagram.

![Networking DNS and DynamoDB](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-bc4c7de503c0d87dde515e4f2052589147a9621c%2Fnetworking-2.png?alt=media\&token=cf6f6c9a-d6aa-4ec1-ae1b-f0d6de6d2d41)

## Considerations and simplifications

This is a high-level overview of Release's networking architecture. In reality, there are a few other components at play too, including:

* **Security groups.** Each AWS service, including each underlying EC2 instance, has one or more security groups, which act as stateful firewalls. While Release automatically creates and assigns the required security groups for these components to communicate with each other, you may have to manually configure access to other internal services (such as your RDS database) or external services (such as Stripe or Twilio). You can read more about how to integrate these in the documentation for [cloud-provided services](/reference-documentation/networking/cloud-provided-services) and [third-party services](/reference-documentation/networking/third-party-services).
* **Other load balancer types.** Release may spin up other load balancer types in certain situations. Usually, this will not affect you, but contact us if you require further information.
* **Availability zones.** By default, Release creates resources in multiple availability zones within each region. This mitigates adverse effects on your environments if there are problems in a specific AWS data center.


# Network architecture (GCP)

When you create environments, Release sets up network access from the internet to services in your environments. For environments hosted on GCP, connectivity is handled through GCP's networking services.

This is an overview of how Release sets up GCP to allow visitors to access your applications.

## High-level network overview

Release configures the following GCP services to direct traffic to your applications:

* **Cloud DNS** points subdomains at load-balancer frontend IP addresses so that visitors' web browsers know which endpoints to connect to.
* **Google Cloud Load Balancer** acts as a Layer 7 load balancer that terminates visitors' HTTPS requests and routes traffic to backends based on URL maps.
* **URL maps** contain a set of rules against which incoming requests are matched so that requests are routed to the appropriate backends.
* **Load-balancer backends** connect the load balancer to network endpoint groups for your applications.
* Pods in Kubernetes clusters connect to the default project-wide **virtual private cloud (VPC) network** for your Google Cloud project.
* **VPC firewall rules** allow traffic from the external load balancer to IP addresses in your VPC.
* **Google Kubernetes Engine** connects incoming requests to pods within your Kubernetes clusters through an Nginx proxy.

When a visitor opens a URL for one of your services in a web browser, the visitor's browser sends a DNS query for a subdomain in your zone in Google Cloud DNS. A DNS resolver returns the IP address for a GCP load-balancer frontend.

The visitor's browser then connects to the load balancer via HTTPS, and sends a request.

The load balancer forwards this request on to Kubernetes, which serves a response from one of your applications.

![GCP network diagram](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-277bd9209bc364549fdb7eb322bb655c74d3e3cc%2Fnetworking-gcp-diagram.png?alt=media\&token=f1445294-b208-4424-aa3e-52f05f6f6cbd)

We'll use an example to see how GCP networking routes traffic.

## Following a network request to a dynamic service

Our visitor wants to connect to an application running in our Release environment. Let's follow their request to explore Release's network configuration of GCP in more detail.

In a Release environment, each service that is accessible from the internet has a unique URL. You can find the URL for a service by visiting the Environment Details page in Release, as shown below.

![Find a service URL in Release](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-f8463a3f7cb1d2c1c782aae184cd5c25fd457052%2Fwordpress-mysql-detail.png?alt=media\&token=67de8800-eb86-4d6e-82e9-2f45ed0730ab)

To start the request, our visitor clicks on a link or enters the URL in their browser, for example, `https://app-service.staging-env.example.com`.

### 1. DNS resolution

The visitor's web browser queries their DNS resolver for the hostname `app-service.staging-env.example.com`. The visitor's DNS resolver, through a series of DNS servers, queries one of GCP's authoritative name servers for the hostname.

Google Cloud DNS responds to the visitor's DNS resolver with the IP address.

{% hint style="info" %}
**How Release configures this DNS record**: When we created our environment, Release added an `A` record associating `app-service.staging-env.example.com` with the IP address of a load-balancer frontend, for example, `216.239.32.108`, to [Google Cloud DNS](https://cloud.google.com/dns). The hostname used is based on the [hostname templates](/guides-and-examples/advanced-guides/hostnames-and-rules#hostnames) configured for our application.
{% endhint %}

### 2. Load balancing and URL maps

The visitor's browser completes a TLS handshake with Google's load-balancer frontend, so that all further traffic between the browser and the load balancer is encrypted.

The domain-verified SSL certificate used for this connection is created in advance by Release by verifying the domain with a certificate authority via DNS-based verification.

After the connection is secured, the browser sends the request:

```http
GET / HTTP/2
Host: app-service.staging-env.example.com
```

The load balancer now matches the request against a URL map, which associates the request with the appropriate load-balancer backend.

In our app's load balancer, the URL map would look like this simplified example:

```json
{
  "hosts": [
    "app-service.staging-env.example.com",
    "*.app-service.staging-env.example.com"
  ],
  "paths": [
    "/",
    "/*"
  ],
  "backend": "release-kn0rp0iie0hndklwzmg9f97ukyz4cd7o"
}
```

Since the host and path in our visitor's request both match this URL map, the load balancer directs the visitor's traffic to the load-balancer backend called `release-kne0hndklwzmg90rp0iif97ukyz4cd7o`.

If for some reason, a request does not match any URL map rules, Release will forward the request to `https://none-such.releasehub.com`.

{% hint style="info" %}
**How Release configures this URL map**: When we created our environment, Release created a [URL map](https://cloud.google.com/load-balancing/docs/url-map-concepts) associating the service's hostname and paths with a load-balancer backend, and added the URL map to the [load balancer](https://cloud.google.com/load-balancing/docs/load-balancing-overview).
{% endhint %}

### 3. Backend services

GCP's load balancer now directs the visitor's traffic to a [backend service](https://cloud.google.com/load-balancing/docs/backend-service).

A load-balancer backend in GCP is responsible for connecting the load balancer to network endpoints. In our example, the network endpoint is a pod in our Kubernetes cluster.

GCP simplifies connecting to multiple endpoints at once by adding endpoints to [network endpoint groups](https://cloud.google.com/load-balancing/docs/negs).

{% hint style="info" %}
Release configures a network endpoint group (NEG) per network zone (for example, `us-west-1-a`, `us-west-1-b`, and `us-west-1-c`) for each environment and adds the internal VPC network IP addresses for servers running in these environments to each group.

If you configure [session affinity](https://docs.releasehub.com/reference-documentation/application-settings/application-template/schema-definition#affinity) for your applications, the backend service keeps track of affinity at the network-zone level. The backend service uses health checks to keep track of healthy endpoints that can handle requests.
{% endhint %}

### 4. VPC network firewall

Because the endpoints in our clusters are connected to a private network, the project-wide default VPC, our backend service needs to pass through the VPC firewall.

The firewall also needs to allow health checks from the load balancer.

{% hint style="info" %}
Release configures VPC firewall rules to accept traffic from Google's load balancing IP ranges: `130.211.0.0/22` and `35.191.0.0/16`.
{% endhint %}

### 5. Kubernetes ingress

Once the visitor's request reaches our Kubernetes cluster, a Kubernetes Nginx ingress controller routes the request to the correct pod to handle the request.

### 6. Request processing and response

For this example, suppose the service at `https://app-service.staging-env.example.com` listens on port 8000. Let's call this service the page-rendering service.

The page-rendering service will handle and process the visitor's request, make any required external requests, and respond with an HTTP response.

To render the page for our visitor, the service may have to communicate with other services in the Kubernetes cluster or cloud-provided services, such as databases.

For example, the page-rendering service may connect to a Redis cache that runs as a service in your Release environment and to a PostgreSQL database hosted on Google Cloud SQL.

### 7. Kubernetes networking

To connect to the Redis cache, Release provides the connection strings, such as hostname and password, to the page-rendering service as environment variables.

Connecting to other services within a single environment follows the same workflow as connecting to services in a single Kubernetes namespace. Services in a namespace are added to a Kubernetes network and can address other services directly by name.

In our example, the Redis cache service might be called `redis`, so the page-rendering service will access Redis directly by connecting to the hostname `redis`.

### 8. Connecting to cloud-hosted databases

If our Cloud SQL database is in the same GCP project and shares the same VPC network as our cluster, no extra steps in GCP are required.

If, however, the database is in a different VPC, we need to set up [VPC network peering](https://cloud.google.com/vpc/docs/vpc-peering) in GCP. VPC network peering allows connections between two different VPCs without requiring public IP addresses for your database.

## Network requests for static files

Release can also build and host static resources on Google Cloud Storage.

Requests for resources served from Google Cloud Storage follow the same DNS and load-balancing steps as dynamic requests, but instead of routing to a Kubernetes ingress controller, requests are routed to Cloud Storage.


# Ingresses

An *ingress* is the main point of entry to your cluster from the public internet. By default, Release configures an Nginx ingress for your cluster, and in many cases, you can use it with the default settings. For advanced cases, you can configure your ingress in your Application Template file, along with your other environment settings.

Some common cases where you might need to configure your ingress manually include:

* Setting specific cookie configuration, such as expiration times.
* Setting affinity sessions or "stickiness" to ensure visitors are predictably served by the same nodes.
* Changing the default buffer sizes and timeout parameters to serve larger files.

## Sticky sessions and affinity

Usually, incoming requests are served by any available node. This means that subsequent requests by the same visitor might be served by different nodes. For many applications, this doesn't matter, as all nodes are running exactly the same application code. If you perform A/B testing or are rolling out a feature gradually to all users, it might be important that users have a consistent experience, even across multiple visits.

In these cases, you can configure *affinity* in your ingress settings. This creates a pairing between unique visitors and specific nodes by setting a random cookie for each new visitor. Returning visitors will automatically share the cookie along with their request, and your ingress will direct them to the same node they were previously served by.

In the diagram below, the visitor automatically sends the previously randomly generated cookie on their second visit, allowing the ingress to say they should be served by node 1 again.

![Affinity cookie](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-0b01ae818e242ae64a652faac8db7a54a49cbc10%2Faffinity.png?alt=media\&token=ae43a0c6-95dc-4945-aeaf-356ec56f6a9f)

You can customize exactly how specific situations are handled by setting `affinity_mode` to either `balanced` or `persistent`. If you are too strict about always assigning users to specific nodes, you can lose the advantages of running a cluster in the first place. For example, the node might be down or overloaded. By using `balanced`, some users might be reassigned to other nodes if the cluster is scaled up. By using `persistent`, users will always be given their assigned node until the cookie expires.

An example of setting up your ingress with cookie-based affinity is shown below, which will ensure that all users are served by the same node for 24 hours. If a request fails, then the affinity will be ignored and the user will be served by a healthy node.

```yaml
ingress:
  affinity: "cookie"
  affinity_mode: "persistent"
  session_cookie_name: "custom-cookie-name"
  session_cookie_path: "/"
  session_cookie_max_age: 86440
  session_cookie_change_on_failure: true
```

You can read more about how Kubernetes handles affinity in their [affinity docs](https://kubernetes.github.io/ingress-nginx/examples/affinity/cookie/).

## Buffer settings and large web requests

By default, the ingress is optimized to serve many smaller files to many different users. This works well for most web applications that consist of many pages and other resources such as JavaScript, CSS, and images. If your application serves larger files, you'll probably want to configure the ingress buffers and timeouts.

With larger buffers, the ingress will do more of the work, buffering data received from the application. This means that you can serve your users in fewer total requests, but with the tradeoff of putting more stress on the hardware responsible for your ingress.

An example of an ingress configured to serve larger web requests is shown below.

```yaml
ingress:
  proxy_body_size: 30m
  proxy_buffer_size: 64k
  proxy_buffering: true
  proxy_buffers_number: 4
  proxy_max_temp_file_size: 1024m
  proxy_read_timeout: "180"
  proxy_send_timeout: "180"
```

This creates a larger temporary file, so that the ingress can store data locally even if the buffer sizes are exceeded. It also sets longer timeouts of three minutes, and creates more buffers to read the initial parts of the response from the application servers.

You can find the full schema definition for configuring ingresses [in the schema docs](/reference-documentation/application-settings/application-template/schema-definition#ingresses).


# IP addresses

When you create a cluster through Release, it will handle allocating IP addresses and defining IP address ranges for you. In most cases, you can use the defaults, but in more complicated cases it might be necessary to configure these manually.

This information might be important if:

* You are running software that spans multiple cloud accounts (for example, using VPC peering in AWS).
* You are running hundreds or thousands of nodes and run out of IP addresses.

## How IP addresses work with Kubernetes

The underlying Kubernetes cluster will have an assigned range of IP addresses that it can assign to **nodes** and **pods**. A node is a virtual compute server, which can run multiple pods. Each pod can run multiple containers.

IP addresses are unique in a specific context. Global IP addresses, reachable on the public internet, are globally unique. In the context of your cluster, you will use internal IP addresses that are also used by other clusters and internal networks. This means that there is a chance you will have two services conflicting on the same IP address if your software spans multiple clusters running in different accounts.

## Choose a unique range for your cluster

When you create a new cluster, you'll be given the option to choose which range of IP addresses to use. In most cases, you can choose any range. If you are integrating your software with services running in a different AWS account (for example, by using [AWS VPC peering](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/peer-with-vpc-in-another-account.html)), then you may have multiple services competing for the same internal IP address. In this case, it is important to choose an IP range that does not overlap with any ranges that are already in use.

The example below shows two peered AWS accounts, each with unique (global) IP addresses for ingress to their respective clusters, but which use an overlapping IP range for subnets. This will only be a problem if the accounts are peered.

![Conflicting IP ranges](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-c56e93670f5a92edf52284012639bab2cebebd20%2Fnetworking-peer-conflict.png?alt=media\&token=c16879c9-4f21-4dd1-9e41-9f83a509568f)

## Running out of IP addresses (AWS)

If you are running lots of microservices, you might exhaust the available IP addresses. You can fix this by spreading your services out over more separate Release clusters, or by optimizing IP allocation directly in your AWS account. For example, you can configure the `WARM_PREFIX_TARGET` and `WARM_IP_TARGET` parameters. See some examples [in this doc](https://github.com/aws/amazon-vpc-cni-k8s/blob/master/docs/prefix-and-ip-target.md). Release sets `WARM_PREFIX_TARGET=1` by default on clusters for v1.21 and above.

You can find more details on how to optimize your IP address allocation directly in your cloud provider at the following links:

* [Optimizing IP address allocation with GCP](https://cloud.google.com/kubernetes-engine/docs/how-to/flexible-pod-cidr)
* [Optimizing IP address allocation with AWS](https://aws.amazon.com/blogs/containers/optimize-ip-addresses-usage-by-pods-in-your-amazon-eks-cluster/)


# Cloud-provided services

You can run almost anything you need directly in your Release-managed Kubernetes cluster. But sometimes you'll also want to integrate with other cloud services that are not within the cluster, such as database services (RDS, CloudSQL), data stores (Elasticache, Memorystore), data analytics solutions (Redshift, BigQuery), and others.

To allow your Release clusters to communicate with other cloud services, you need to:

* Ensure that the other services are running in the same VPC as your Release clusters.
* Ensure that you have the correct security groups (AWS) or firewall rules (GCP) set up.

This means that any customer or public traffic will enter your VPC through your ingress, and your software running in your cluster will be able to connect to other services, as shown in the diagram below.

![diagram showing ingress and security group](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-3306599175e67a649b40c1da112a8a0145431633%2Fcloud-native-diagram.png?alt=media\&token=e8744ffc-309c-4b5d-a5c2-3f64263fd618)

## Find the correct AWS security group and assign it to new cloud services

When Release creates a new node group in your AWS account, it also creates a default security group that all nodes in that group belong to.

You can find this security group using the following steps:

* Log in to your Release account and navigate to the Cluster Info page.
* Note the context, node group name, and region of your cluster.

You can see where to find the cluster context and node group name in the image below.

![Get cluster and node group name](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-6fb91eee5712fa17649377f08f7a79276571f54e%2Fflask-rds-cluster-nodegroup-name.png?alt=media)

Log in to your AWS console, and:

* Navigate to the EC2 dashboard for the same region as your cluster.
* Use the cluster context and node group name to find a worker node.
* Go to **Actions** -> **Security** -> **Change security groups** and look for the security group that also contains the cluster context.

You can see an example of a worker node and how to navigate to the security groups in the image below.

![Finding the security group](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-f511969e17baafb311de11af8373a5c954331719%2Fflask-rds-get-security-group-ec2.png?alt=media)

## Create a new cloud service

When you create a new cloud service, you need to:

* Set the VPC to the same one used by your Release cluster.
* Assign the security group you identified above.

Because the security group allows traffic from any services with the same group assigned, the code running in your cluster will be able to connect to these services.

## Create new node groups

Note that whenever you create a new node group through Release, all nodes in that group will be assigned to a **new** security group. You'll therefore have to follow these steps and add the new security group to any cloud services each time you create a new node group in Release.


# Third-party services

When you host your software with Release, you can continue to integrate with third-party services as you usually would. In some cases, third-party services might ask you to allowlist (or whitelist) the IP addresses that your software will call out to their software from. Because Release automatically creates clusters and other networking infrastructure in your cloud account, you need to know how to find your main egress IP address.

You might need this to integrate with external services, including but not limited to the following:

* SendGrid
* Stripe
* Twilio
* Salesforce
* Office 365

## Finding your outgoing IP for AWS

When you create a cluster with Release and AWS, Release automatically sets up Kubernetes using EKS and a **NAT gateway**. A NAT gateway is the primary gate between the outside world and your clusters, so external requests from your software will appear to originate from this gateway.

To find your NAT gateway address, follow these steps (if you have Release clusters in multiple regions, repeat the steps for each region):

1. Log in to your AWS account.
2. Select the region your Release cluster runs in.
3. Navigate to the **VPC dashboard**.
4. Select **NAT gateways** on the left sidebar.
5. Identify the NAT gateway associated with your Release cluster and look for "Elastic IP Address".

This is the IP address you send to any third-party providers that need to allowlist your IP ranges.

![Get NAT IP address](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-c19e0e5b666b2b718d5e945a8be8f0d24ebec091%2Fnetworking-get-nat-ip.png?alt=media\&token=35d3dcfa-5b1a-42ef-b3e8-5e96831a7c06)

You can read more about how NAT gateways work in AWS [in their docs](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-nat-gateway.html).


# Release environment versioning

How Release versions environments

Environments in Release are configured based on your **default** environment configuration and your **environment-specific** configuration.

Each environment in Release is associated with a specific version string, which has the format of `v{default}.{specific}`. For example, you might see that a given environment is on version `v1.2`, which means it is based on the first modification of the **default** environment and the second modification of the **specific** environment.

* The first digit refers to the version of the default environment configuration.
* The second digit refers to the version of the currently deployed environment-specific configuration.

## Find the version of a specific environment

Under each environment for a specific application, you can view the environment version listed as part of the summary, as shown in the image below.

![Find an environment version](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-b3a0bceedb0a16865088181eb85f19010d4ed91e%2Fenvironment-versions-summary.png?alt=media)

In the **Settings** tab for the application, under **Environment Configuration**, you can view the environment configuration version, as shown below.

![Environment configuration version](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-26009b962e0e48f2778247e95932533adfb490ae%2Fenvironment-versions-major.png?alt=media)

## Default configuration version

The default environment configuration's version number will increment whenever you save either the environment configuration or the default environment variables.

Once you have created an environment, this digit in the environment's version number will **not** change if you update the default environment, as it refers to the default configuration the environment-specific configuration was originally based on.

## Environment history

Release keeps a history of all changes made to the environment configuration.

In the **Settings** tab for any environment, under **History**, you can view the history of changes made to the environment configuration. This includes the date and time of the change, the user who made the change, and the version number of the configuration.

![View environment version history](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-ff4e9bbb147a0bcd3155a5040dc1f2a81e4a5af5%2Fenvironment-version-history.png?alt=media)

## Deploy the latest environment version

After changing the environment configuration, you may wish to redeploy your application. You can redeploy the application in an environment by clicking the **Deploy** dropdown in the top right corner of the environment page and then selecting **Deploy Patch**.

![Deploy new version](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-158de43e3ea638503786bfcdd81fc3bfef57bf53%2Fdeploy-patch.png?alt=media)


# Application settings

Settings and default configuration for your application's environments

Release deploys your application's code to multiple environments.

We make it quick and easy to create new environments by helping you skip manual configuration every time you create a new environment.

When you create new environments, Release generates each new environment's configuration files from two files in your application's settings:

1. [Application Template](/reference-documentation/application-settings/application-template) — the template for each new environment's [Environment Configuration](/reference-documentation/environment-settings/environment-configuration).
2. [Environment Variables](/reference-documentation/application-settings/default-environment-variables) — the template for each new environment's [Environment Variables](/reference-documentation/environment-settings/environment-specific-environment-variables).

![Application Settings](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-dc46d2c46f451c705a2b51afca825fd324bfb380%2Fapplication-settings-main.png?alt=media)

These two files are the starting points for generating environment-specific configuration for each environment you create.

After an environment is created, you can edit the environment's configuration and variables at any time. Environment-specific configuration is useful if, for instance, an environment for a feature branch needs environment variables that don't apply to other branches.


# Application Template

How the Release Application Template works

Release automatically generates an Application Template based on the `docker-compose.yml` file from your repository.

The Application Template defines everything a new environment would need to deploy and run your application: services, jobs, which ports to open, hostnames, deployment workflows, and resource limits.

When you create a new environment, Release uses the Application Template as a starting point to generate an [environment configuration](/reference-documentation/environment-settings/environment-configuration).

The default Application Template generated by Release rarely requires any editing because it is based on your `docker-compose.yml`.

To view or edit the Application Template, navigate to **App Settings** and click the **Edit** button in the “Application Template” section.

![How to edit the Application Template](https://585411240-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M1neGLLQ0sDXeK6ooSo%2Fuploads%2Fgit-blob-4f13d4403386a3082a26737da1e89f73df626d0e%2Fapplication-template.png?alt=media\&token=1399663e-df33-49ed-9ba2-380c27034610)

{% hint style="info" %}
Changes to the Application Template will not affect *existing* environments. The template is only used to configure new environments.
{% endhint %}

The `environment_templates` section in the Application Template describes differences between your `ephemeral` and `permanent` environment configurations. You will select one of these options when creating new environments.

See the [Application Template Schema Definition](/reference-documentation/application-settings/application-template/schema-definition) for a detailed description of each section in the Application Template.


# Schema definition

## Schema Definition

### Application Template Schema

This configuration template is the basis for all environments you will create for this application. Each of the sections and directives file help to create the configuration for your specific environment. The **environment\_templates** section describes differences between your ephemeral and permanent templates and you select one of these when creating an environment. Each section and directive will be described in detail below in this document.

```yaml
---
app:
  type: String
  required: false
  description: Name of your app, can't be changed.
auto_deploy:
  type: Boolean
  required: false
  description: If true, environments will auto deploy on a push
context:
  type: String
  required: false
  description: Cluster context
domain:
  type: String
  required: false
  description: Used to create hostnames
execution_type:
  type: String
  required: false
  description: Determines whether the app creates Server or Runnable Environments
git_fetch_depth:
  type: Integer
  required: false
  description: Git fetch depth
mode:
  type: String
  required: false
  description: Deprecated
parallelize_app_imports:
  type: Boolean
  required: false
  description: Parallelize the deployment of all the apps
repo_name:
  type: String
  required: false
  description: Name of the repository, can't be changed.
tracking_branch:
  type: String
  required: false
  description: Default branch for environments to track
tracking_tag:
  type: String
  required: false
  description: Default tag for environments to track
app_imports:
  type: Array
  required: false
  description: Connect multiple apps together
builds:
  type: Array
  required: false
  description: Defines how Release should build images.
cron_jobs:
  type: Array
  required: false
  description: Cron Jobs
custom_links:
  type: Array
  required: false
  description: Additional Custom Links that will be presented with each Environment
development_environment:
  type: Hash
  required: false
  description: Set of services configured for remote development
environment_templates:
  type: Array
  required: true
  description: Templates for creating environments
hostnames:
  type: Array
  required: false
  description: Hostnames for services
infrastructure:
  type: Array
  required: false
  description: Infrastructure as code runners.
ingress:
  type: Hash
  required: false
  description: Ingress
jobs:
  type: Array
  required: false
  description: Arbitrary jobs, scripts to run.
node_selector:
  type: Array
  required: false
  description: Node Selector
notifications:
  type: Array
  required: false
  description: Define your notifications.
parameters:
  type: Array
  required: false
  description: Key-Values that you can define and use in your templates and containers
resources:
  type: Hash
  required: true
  description: Default cpu, memory, storage and replicas.
routes:
  type: Array
  required: false
  description: For defining multiple entry points to a service and routing rewrites
    and auth
rules:
  type: Array
  required: false
  description: For defining multiple entry points to a service
s3_volumes:
  type: Array
  required: false
  description: Volumes from S3 buckets.
secrets:
  type: Array
  required: false
  description: Config for utilizing specific secrets managers
service_accounts:
  type: Array
  required: false
  description: Service Accounts
services:
  type: Array
  required: false
  description: List of services needed for you application
shared_volumes:
  type: Array
  required: false
  description: Volumes that are accessed by multiple services
sidecars:
  type: Array
  required: false
  description: Reusable sidecar definitions
workflows:
  type: Array
  required: true
  description: Definitions for deploying config and code updates
workspaces:
  type: Array
  required: false
  description: Collection of data sources for your containers.
```

### auto\_deploy

If true, environments will deploy whenever you push to the corresponding repo and tracking branch.

### context

This value is used by your application to deploy to a specific cluster. If you have your own EKS cluster through Release you can change this value to match that cluster/s, but if not use the generated value.

### domain

The domain name where your applications will be hosted. These domains must be AWS Route 53 hosted domains. Release supports first and second level domains. (i.e. domain.com or release.domain.com)

### execution\_type

Determines whether the app creates Server or Runnable Environments

### git\_fetch\_depth

Configures the fetch depth for Git operations in builds. Defaults to fetching the repository's complete Git history. Setting this to 1 will result in a shallow clone and can speed up builds for larger repositories. Defines s3 buckets that can be mounted as volumes in your services.

### mode

Mode is a configuration directive that you can use (it is set as an environment variable in your containers) if useful. e.x 'development' or 'production' or 'test'

### parallelize\_app\_imports

If there are no dependencies for the order in which in the apps deploy, use `parallelize_app_imports` to deploy all the apps at the same time.

### tracking\_branch

By default this will be the default branch of your repository, but it can be changed to any branch you would like to track with you environments.

### tracking\_tag

A specific git tag that you want your environments to track. You must unset tracking\_branch if you use tracking\_tag.

### app\_imports

App Imports are a way to connect multiple apps together. When you create an environment on one application, the apps that you import will also get environments created in the same namespace. Click [here](#app-imports) for more info.

### builds

You can specify builds at the top level to be pulled in during the services sections. See the [builds](#builds) section for details.

### cron\_jobs

Cron Jobs are Jobs that run on a schedule. Cron jobs allow you to periodically execute commands within a namespace. They can be used for warming up caches, running database maintenance, etc. Click [here](#cron-jobs) for more info.

### custom\_links

Custom Links are defined as an array of key/value pairs, under the custom\_links directive. The key is a name, and the value is any URL you want. The values can be hardcoded URLs or utilize variable substitution.

### development\_environment

This allows you to connect from a local machine to the remote environment and sync files and folders. Click [here](#development-environments) for more info.

### environment\_templates

These templates are used when creating an environment. They allow you to override or change any of the defaults in this file for particular type of environments: ephemeral or permanent. Click [here](#environment-templates) for more info.

### hostnames

Hostnames are defined as an array of key/value pairs, under the hostnames directive. The key is a service name, and the value is the hostname. These can be hardcoded hostnames or utilize variable substitution. They are auto-generated for any service with a node\_port or static build.

### infrastructure

Infrastructure runners are specialized jobs that execute infrastructure as code plans during deployment.

### ingress

Ingress settings that can control the behavior and functionality of the NGINX ingress controller to access HTTP services in your cluster.

### jobs

Jobs are like services except they run to completion. Examples include database migrations, asset compilation, etc. The inherit the image from a service, and run a command that ultimately terminates. Click [here](#jobs-1) for more info.

### node\_selector

Node Selectors allow you to assign workloads to particular nodes based on common labels such as `kubernetes.io/os=windows` and `kubernetes.io/arch=arm64`. Click [here](#service-node-selectors) for more information.

### notifications

Allow you to define where and which events to be notified. Click [here](#notifications) for more info.

### parameters

These parameters allows you to collect info from the user at deploy time. You may interpolate these in your configuration and/or use them as inline environment variables for your service, jobs, etc. Click [here](#parameters-1) for more info.

### resources

Default resources for all of your services. The structure and values are based on <https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/>. Click [here](#resources-1) for more info. For examples of why and how to override these defaults check out [Managing Service Resources](#../../reference-examples/managing-service-resources).

### routes

Routes allow an easy way define multiple endpoints per service. Routes allow for edge routing rewrites, authentication and provide full support for ngnix ingress rules.

### rules

Rules are based on <https://kubernetes.github.io/ingress-nginx/user-guide/basic-usage/> and allow an easy way define multiple endpoints per service. They consist of three parts: service name, hostnames, and a path. Release will take this configuration and create an Nginx Ingress deployment to handle your routing automatically.

### s3\_volumes

Defines s3 buckets that can be mounted as volumes in your services

### secrets

Secrets is an array of configs for utilizing specific secret managers. Release supports both secret stores from AWS and Doppler. Release supports specific types of behavior depending on which secrets managers you are using. Click [here](#secrets-1) for more info.

### service\_accounts

Allow you to define service accounts that can be used to control the cloud permissions assumed by your workloads (services, jobs, cron jobs, etc.)

### services

These services define the most important part/s of your application. The can represent services Release builds from your repositories, off-the-shelf containers (postgres, redis, elasticsearch, etc), external services you need to connect to or even services from other applications you have that are needed in this application also. Click [here](#services-1) for more info.

### shared\_volumes

Shared Volumes creates a PersistentVolumeClaim that is written to/read from by multiple services.

### sidecars

Top level sidecar definitions allow you to create reusable containers that can be applied to several services defined with in your application. These are useful for log aggregation. Click [here](#sidecars-2) for more info.

### workflows

Workflows are an ordered list of what must be done to deploy new configuration or code to your environments. They are a combination of services and jobs (if you have them). Click [here](#workflows-1) for more info.

There are three workflows Release supports by default: `setup`, `patch`, and `teardown`. When a new environment is created `setup` is ran, and when code is pushed a `patch` is run against that environment. Whenever a an environment is destroyed the `teardown` workflow is run.

Release also supports user defined workflows that can be run in a one-off manner.

### workspaces

Workspaces allow you to assemble multiple repositories or other data sources into a directory tree that can be shared by any number of containers in your environment. Click [here](#workspaces-3) for more info.

## Hostnames or Rules

Hostnames or Rules can be both be used to define entry points to your services. They can not be used together at the same level in the config. In other words, you can't have default hostnames and rules, but you could have default hostnames and then use rules inside the environment\_templates section of the file.

#### Hostnames

Hostnames are defined as an array of key/value pairs, under the hostnames directive. The key is a service name, and the value is the hostname. These can be hardcoded hostnames or utilize variable substitution. They are auto-generated for any service with a **node\_port** or static build.

```yaml
---
hostnames:
- frontend: frontend-${env_id}-${domain}
- docs: docs-${env_id}-${domain}
- backend: backend-${env_id}-${domain}
```

By default, Hostnames are generated using two variables **env\_id** and **domain**. env\_id is a randomly generated string for ephemeral environments or the name of the environment for permanent ones. Using some amount of random values allows Release to bring up any number of ephemeral environments on the same domain without conflicts. Domain is taken directly from your configuration file.

#### Rules

Rules are based on <https://kubernetes.github.io/ingress-nginx/user-guide/basic-usage/> and allow an easy way define multiple endpoints per service. They consist of three parts: service name, hostnames, and a path. Release will take this configuration and create an Nginx Ingress deployment to handle your routing automatically. The `visibility` parameter will determine how the URL is reachable on the Internet (or privately via a VPC)

```yaml
service:
  type: String
  required: true
  description: Service name from your config
hostnames:
  type: Array
  required: true
  description: Same as hostnames above
path:
  type: String
  required: true
  description: Entry point for hostnames
keepalive_timeout:
  type: Integer
  required: false
  description: |
    Set the keepalive_timeout for CDN-based options (`visibility: public` only), defaults to 5 (seconds).
    This value should be set much lower than read_timeout and other values, typically much less than 60 (seconds).
read_timeout:
  type: Integer
  required: false
  description: |
    Set the read_timeout for CDN-based options (`visibility: public` only), defaults to 60 (seconds).
    Keep it small, usually should not (or cannot) be set more than 300 (seconds, 5 minutes)
visibility:
  type: String
  required: false
  description: |
    Describes the reachability of the URL, one of:
      `public`        (publicly accessible CDN)
      `public-direct` (publicly accessible load balancer)
      `private`       (privately accessible load balancer)
```

> Rules Schema

```yaml
 rules:
  - service: admin
    hostnames:
    - admin-${env_id}.internal.example.com
    path: "/"
    visibility: private
  - service: backend
    hostnames:
    - backend-${env_id}.${domain}
    path: "/auth/"
    visibility: public-direct
  - service: frontend
    hostnames:
    - frontend-${env_id}.${domain}
    path: "/"
    read_timeout: 45
    visibility: public
```

> Rules Example

## App Imports

App Imports are optional and not present in the Application Template by default.

```yaml
---
branch:
  type: String
  required: false
  description: Setting the branch pins all created Environments to that branch
name:
  type: String
  required: true
  description: Name of App you want to import. The imported App from must exist in
    your account.
development_environment:
  type: Hash
  required: false
  description: Optional overrides of development environment configuration. See examples
    below.
exclude_services:
  type: Array
  required: false
  description: If you have a services in your imported app that would be a repeat,
    say both apps have Redis, you can exclude them
ignore_deployment_refs:
  type: Array
  required: false
  description: Ignores deployments when the ref matches the Environment's ref; can
    use negation and path globs
pull_request_labels:
  type: Array
  required: false
  description: For environments created from a pull request, if specified, the app
    import will only be added if the pull request has at least one of the given labels.
    This is ignored for environments created outside of a pull request.
```

```yaml
app_imports:
  - name: backend
    branch: new-branch
    exclude_services:
      - name: redis
```

> Example: App Imports excluding a service

```yaml
app_imports:
  - name: backend
    ignore_deployment_refs:
      - main
      - "!development"
      - releases/**
```

> Example: App Imports ignoring deployments

```yaml
parallelize_app_imports: true
app_imports:
  - name: backend
  - name: upload-service
  - name: worker-service
  - name: authentication-service
```

> Example: App Imports with many apps utilizing the parallel deploys

You can optionally customize the order in which the current application is deployed with the special name `$self`. If not present, the current app is always deployed last.

```yaml
app_imports:
  - name: $self # references the current application
  - name: backend
  - name: upload-service
  - name: worker-service
  - name: authentication-service
```

> Example: App Imports with custom ordering for the current application

You can optionally make certain app imports conditional upon whether or not a pull request has one of the specified labels.

```yaml
app_imports:
  - name: backend
    pull_request_labels:
    - import-all
    - import-backend
  - name: upload-service
    - import-all
    - import-upload
  - name: worker-service
    - import-all
    - import-worker
  - name: authentication-service
    - import-all
    - import-authentication
```

> Example: App Imports with pull request label filtering

You can optionally override development environment configuration for a specific app import. For example, if you know your imported app will be located in the "frontend" subdirectory of your root app you can specify that:

```yaml
app_imports:
  - name: frontend
    development_environment:
      local_root_path: "./frontend"
```

> Example: App Imports with development environment overrides

### Exclude Services

Allows the removal of duplicate services during App Imports

```yaml
---
name:
  type: String
  required: true
  description: Name of service you want to exclude
```

## Builds

Top-level builds section for docker builds. Can be used for special docker images, especially jobs, init containers, and sidecars. If you are using build directives for a single service, please see the [build](#build) section.

```yaml
---
context:
  type: String
  required: false
  description: The working directory for the docker build context.
dockerfile:
  type: String
  required: false
  description: The location of the dockerfile to use for a build.
name:
  type: String
  required: false
  description: Name of the docker image to build.
repo_branch:
  type: String
  required: false
  description: Combined with `repo_url`, use to target a specific branch
repo_commit:
  type: String
  required: false
  description: Combined with `repo_url`, use to target a specific commit
repo_url:
  type: String
  required: false
  description: If you want to create a Build from a different repository
target:
  type: String
  required: false
  description: If a specific build stage should be targeted
args:
  type: Array
  required: false
  description: Args passed into the build command
image_scan:
  type: Hash
  required: false
  description: Release can scan your built images for known security vulnerabilities
    **This feature is deprecated and will be removed -- contact support**
```

```yaml
builds:
- name: my-init
  context: sysops/dockerfiles
  dockerfile: init.Dockerfile
services:
- name: app
  image: acme/api/app
  init:
    name: my-init
    has_repo: true
    image: acme/api/my-init
  build:
    context: app
    dockerfile: Dockerfile
  has_repo: true
  command:
  - sh
  - "-c"
  - "/usr/local/bin/entrypoint.sh"
```

> Example: A builds stanza for an [init container](#init-containers). Please notice the use of `has_repo` and `name` fields in particular.

## Cron Jobs

Cron Job containers allow you to define additional workloads run on a schedule. Cron Jobs can be used for many different tasks like database maintenance, reporting, warming caches by accessing other containers in the namespace, etc.

```yaml
---
completions:
  type: Integer
  required: false
  description: Minimum Required Completions For Success
  default: 1
concurrency_policy:
  type: String
  required: false
  description: Policy On Scheduling Cron jobs
  default: Forbid
from_services:
  type: String
  required: false
  description: Service To Use For Job Execution
has_repo:
  type: Boolean
  required: false
  description: Repository is local
image:
  type: String
  required: false
  description: Docker Image To Execute
name:
  type: String
  required: true
  description: A Name
parallelism:
  type: Integer
  required: false
  description: Amount Of Parallelism To Allow
  default: 1
schedule:
  type: String
  required: true
  description: Cron Expression
args:
  type: Array
  required: false
  description: Arguments
command:
  type: Array
  required: false
  description: Entrypoint
```

Each cron job entry has a mutually exclusive requirement where either `image` or `from_services` must be present.

```yaml
cron_jobs:
  - name: poll-frontend
    schedule: "0 * * * *"
    image: busybox
    command:
      - sh
      - "-c"
      - "curl http://frontend:8080"
  - name: redis-test
    schedule: "*/15 * * * *"
    from_services: redis
    command:
      - sh
      - "-c"
      - "redis-cli -h redis -p 6390 ping"
```

> Example cron job definitions to poll the frontend service and ping Redis

`parallelism`, `completions`, and `concurrency_policy` are ways to control how many pods will be spun up for jobs and how preemption will work. By default, a minimum of one job will run successfully to be considered passing. Also by default, we set `concurrency_policy` to equal `Forbid` rather than the default Kubernetes setting of `Allow`. We have found that the default of `Allow` creates problems for long running jobs or jobs that are intensive and need to be scheduled on a smaller cluster. For example, if a job runs for ten minutes but is scheduled every five minutes, then Kubernetes will gladly keep starting new jobs indefinitely because it does not think the job is finished. This can quickly overwhelm resources. You can use `Forbid` to prevent rescheduling jobs that should not be rescheduled even if they are not run or fail to start.

A few examples follow.

```yaml
cron_jobs:
  - name: poll-frontend
    concurrency_policy: "Forbid"
    parallelism: 1
    completions: 1
    schedule: "0 * * * *"
    image: busybox
    command:
      - sh
      - "-c"
      - "curl http://frontend:8080"
```

> An example of the default settings (same as leaving them blank).

```yaml
cron_jobs:
  - name: poll-frontend
    concurrency_policy: "Replace"
    parallelism: 2
    completions: 2
    schedule: "*/10 * * * *"
    image: busybox
    command:
      - sh
      - "-c"
      - "curl http://frontend:8080"
```

> An example of a job that will run two polling jobs roughly simultaneously every ten minutes. Two jobs must succeed for the job to be marked complete; if it does not finish within 10 minutes, then the `Replace` policy will kill the previous job and start a new one in its place.

```yaml
cron_jobs:
- name: sync-data-lake
  concurrency_policy: "Allow"
  parallelism: 3
  completions: 6
  schedule: "@daily"
  image: busybox
  command:
    - sh
    - "-c"
    - "backup db"
```

> An example of a queue-pulling job that will run 3 threads of self-synchronising pods and usually takes six runs to complete. The setting of `Allow` will ensure the job starts again if the scheduler decides the jobs did not finish or started late due to resource constraints on the cluster. *Please note: `completion_mode` is not available until v1.24 is supported.*

### completions

Integer amount greater than zero to indicate how many successful runs should be considered finished. Usually you would set this value equal or greater than `parallelism` but it might be possible to set it less if you do not care about wasted pods being scheduled. Depending on `parallelism` and `concurrency_policy`, the combination of settings may cause jobs to be run multiple times in excess of this value. See the [Kubernetes documentation](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#concurrency-policy)

### concurrency\_policy

One of `Allow`, `Forbid`, or `Replace`. Kubernetes defaults to `Allow` which allows jobs to be rescheduled and started if they have failed or haven't started or haven't finished yet. We prefer to set `Forbid` because it prevents pods from being started or restarted again, which is much safer. The option to use `Replace` means that if a job is failed or stalled then the previous job start will be killed (if it is still running) before being started on a new pod.

### from\_services

A reference to the service name to use as the basis for executing the cron job. Parameters from the service will be copied into creating this cron job.

### has\_repo

Use an internal repository built by Release, or not.

### image

A reference to the docker image to execute the job, use if `from_services` is not a good fit

### name

What's in a name? That which we call a rose/By any other name would smell as sweet.

### parallelism

Integer amount of number of pods that can run in parallel. Set to 0 to disable the cron job. See the [Kubernetes documentation](https://kubernetes.io/docs/concepts/workloads/controllers/job/#parallel-jobs). This controls how many pods are potentially running at the same time during a scheduled run.

### schedule

A string representing the schedule when a cron job will execute in the form of `minute` `hour` `dayofmonth` `month` `dayofweek` or `@monthly`, `@weekly`, etc. Read the [Kubernetes docs](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule)

### args

An array of arguments to be passed to the entrypoint of the container.

### command

An array of arguments to be passed to override the entrypoint of the container.

### Service Command

Pods running in Kubernetes typically have a default command that is run upon container start. The value specified in the `command` will override the supplied Docker `ENTRYPOINT`. Please note there is a lot of confusion for Kubernetes commands and Docker commands since they use similar and overlapping meanings for each. The `command` is specified as an `EXECV` array, not a string.

You can specify the override command that a shell will start with.

```yaml
service:
- name: frontend
  command:
  - "/bin/sh"
  - "-c"
  - "sleep 3600"
```

## Development Environments

Development Environment allows you to configure an environment to be used for remote development. This allows you to connect from a local machine to the remote environment and sync files and folders.

```yaml
---
local_root_path:
  type: String
  required: false
  description: Optional path to root directory for file syncing. All `local_path`
    fields in `sync` configurations will be relative to this path.
services:
  type: Array
  required: true
  description: Set of services which will allow remote development
```

Each service entry describes:

* `image` to use, if not using the same as the one defined on the `service`
* `command` to run on the image, if not using the one defined on the `service`
* `sync` which files and folders to sync from a local machine to the remote container
* `port_forwards` which ports to forward from the local machine to the remote container

```yaml
development_environment:
  services:
  - name: api
    command: "yarn start"
    image: releasehub
    sync:
      - remote_path: "/app/src/api"
        local_path:  "./src/api"
    port_forwards:
      - remote_port: '4000'
        local_port: '4000'
  - name: frontend
    command: "bash"
    sync:
      - remote_path: "/app/src/frontend"
        local_path:  "./src/frontend"
    port_forwards:
      - remote_port: 4000
        local_port: 4000
      - remote_port: 4001
        local_port: 4001
```

> Development Environment Example

### Development Environment Services

Development Environment allows you to configure an environment to be used for remote development. This allows you to connect from a local machine to the remote environment and sync files and folders.

```yaml
---
command:
  type: String
  required: false
  description: Command to run on container start. Overrides any `command` specified
    for the `service`.
image:
  type: String
  required: false
  description: The image to use for the container. Overrides any `image` specified
    for the `service`.
name:
  type: String
  required: true
  description: Name of the service to use for remote development.
port_forwards:
  type: Array
  required: false
  description: Specify which ports are forwarded.
sync:
  type: Array
  required: false
  description: Specify which files and folders are synchronized.
```

### Port Forwards

Forward ports allows you to configure which local port(s) are mapped to the remote port(s) on your container.

```yaml
---
local_port:
  type: Integer
  required: true
  description: The local port
remote_port:
  type: Integer
  required: true
  description: The remote port
```

### Sync

Sync allows you to configure which files and folders are synchronized between a local machine and a remote container.

```yaml
---
local_path:
  type: String
  required: true
  description: The full path or the relative path assumed from the current working
    directory.
remote_path:
  type: String
  required: true
  description: The full path on the container.
```

## Environment Templates

There are two types of allowed and required templates: ephemeral and permanent. When creating a new environment, either manually or through a pull request one of these templates will be used to construct the configuration for that particular environment. If the template is empty you get the defaults contained in your Application Template, but these templates allow you to override any of the defaults.

The schema for these is a duplicate of the entire default configuration as it allows you override anything contained in this file for that particular template. As such, we won't detail the schema twice, but there are examples contained here showing how to override default configuration in your templates.

{% hint style="info" %}
[Instant Datasets](#instant-datasets) are unique in that they are not allowed at the root of the default config and can only be added under environment\_templates. Since Instant Datasets allow you to use instances of RDS databases (often snapshots of production, but they could be snapshots of anything) having this be the default could result in unwanted behavior for you permanent environments.

Release requires you to be explicit on which template/s you would like to (by default) use Instant Datasets. Once you have created an environment you may add Instant Datasets to your environments through the Environment Configuration file if you don't want all environments of a particular type to use datasets.
{% endhint %}

## Infrastructures

Infrastructure runners are specialized jobs that execute infrastructure as code plans during deployment.

```yaml
---
directory:
  type: String
  required: false
  description: Relative path to directory containing infrastructure module.
name:
  type: String
  required: true
  description: Unique name to use when referencing the infrastructure runner.
type:
  type: String
  required: true
  description: 'Infrastructure runner type, one of: `terraform` (supported), `pulumi`
    (in development)'
values:
  type: String
  required: false
  description: Relative path to a file containing configuration values to pass to
    the infrastructure module.
```

The example below shows two infrastructure runners:

```yaml
infrastructure:
- name: dynamodb-table1
  type: terraform
- name: dynamodb-table2
  type: terraform
  directory: "./dynamodb"
  values: ".release/dynamodb2_values.tfvar"
```

## Ingresses

Ingress settings that can control the behavior and functionality of the NGINX ingress controller to access HTTP services in your cluster

```yaml
---
affinity:
  type: String
  required: false
  description: Nginx affinity
affinity_mode:
  type: String
  required: false
  description: The mode for affinity stickiness
backend_protocol:
  type: String
  required: false
  description: Protocol to use on the backend
proxy_body_size:
  type: String
  required: false
  description: Proxy Body Size maximum
proxy_buffer_size:
  type: String
  required: false
  description: Proxy Initial Buffer Size
proxy_buffering:
  type: Boolean
  required: false
  description: Enable or Disable Proxy Buffering
proxy_buffers_number:
  type: Integer
  required: false
  description: Proxy Initial Buffer Count
proxy_connect_timeout:
  type: String
  required: false
  description: Proxy Connection Timeout
proxy_max_temp_file_size:
  type: String
  required: false
  description: Proxy Max Temp File Size
proxy_read_timeout:
  type: String
  required: false
  description: Proxy Read Timeout
proxy_send_timeout:
  type: String
  required: false
  description: Proxy Send Timeout
session_cookie_change_on_failure:
  type: Boolean
  required: false
  description: Session Cookie Change on Failure
session_cookie_max_age:
  type: Integer
  required: false
  description: Session Cookie Maximum Age in Seconds
session_cookie_name:
  type: String
  required: false
  description: Session Cookie Name
session_cookie_path:
  type: String
  required: false
  description: Session Cookie Path
wafv2_acl_arn:
  type: String
  required: false
  description: Web Application Firewall Version 2 Access Control List Amazon Web Services
    Resource Name
ip_allow_list:
  type: Array
  required: false
  description: Allowed client IP ranges
ip_deny_list:
  type: Array
  required: false
  description: Denied client IP ranges
```

```yaml
ingress:
  proxy_body_size: 30m
  proxy_buffer_size: 64k
  proxy_buffering: true
  proxy_buffers_number: 4
  proxy_max_temp_file_size: 1024m
  proxy_read_timeout: "180"
  proxy_send_timeout: "180"
```

> Example proxy buffer settings for large web requests

```yaml
ingress:
  affinity: "cookie"
  affinity_mode: "persistent"
  session_cookie_name: "my_Cookie_name1"
  session_cookie_path: "/"
  session_cookie_max_age: 86440
  session_cookie_change_on_failure: true
```

> Example settings for stickiness settings using a cookie

```yaml
ingress:
  wafv2_acl_arn: arn:aws:wafv2:us-west-2:xxxxx:regional/webacl/xxxxxxx/3ab78708-85b0-49d3-b4e1-7a9615a6613b
```

> Example settings for applying a WAF ruleset to the ALB in (AWS-only)

### Ingress settings schema

### affinity

Type of the affinity, set this to `cookie` to enable session affinity. See <https://kubernetes.github.io/ingress-nginx/examples/affinity/cookie/>

### affinity\_mode

The affinity mode defines how sticky a session is. Use `balanced` to redistribute some sessions when scaling pods or `persistent` for maximum stickiness.

### backend\_protocol

Which backend protocol to use (defaults to HTTP, supports HTTP, HTTPS, GRPC, GRPCS, AJP and FCGI)

### proxy\_body\_size

Sets the maximum allowed size of the client request body.

### proxy\_buffer\_size

Sets the size of the buffer used for reading the first part of the response received from the proxied server. This part usually contains a small response header.

### proxy\_buffering

Enables or disables buffering of responses from the proxied server.

### proxy\_buffers\_number

Sets the number of the buffers used for reading the first part of the response received from the proxied server.

### proxy\_connect\_timeout

Sets the timeout in seconds for establishing a connection with a proxied server or a gRPC server. Most CDN and loadbalancer timeout values are set to 60 seconds, so you should use a value like 50 seconds to ensure you do not leave stranded connections. It should be noted that this timeout cannot exceed 75 seconds.

### proxy\_max\_temp\_file\_size

When buffering of responses from the proxied server is enabled, and the whole response does not fit into the buffers set by the proxy\_buffer\_size and proxy\_buffers directives, a part of the response can be saved to a temporary file. This directive sets the maximum size of the temporary file.

### proxy\_read\_timeout

Sets the timeout in seconds for reading a response from the proxied server. The timeout is set only between two successive read operations, not for the transmission of the whole response.

### proxy\_send\_timeout

Sets the timeout in seconds for reading a response from the proxied server. The timeout is set only between two successive read operations, not for the transmission of the whole response.

### session\_cookie\_change\_on\_failure

When set to `false` nginx ingress will send request to upstream pointed by sticky cookie even if previous attempt failed. When set to `true` and previous attempt failed, sticky cookie will be changed to point to another upstream.

### session\_cookie\_max\_age

Time in seconds until the cookie expires, corresponds to the `Max-Age` cookie directive.

### session\_cookie\_name

Name of the cookie that will be created (defaults to `INGRESSCOOKIE`).

### session\_cookie\_path

Path that will be set on the cookie (required because Release Ingress paths use regular expressions).

### wafv2\_acl\_arn

The ARN for an existing WAF ACL to add to the load balancer. AWS-only, and must be created separately.

### ip\_allow\_list

An array of source client CIDRs to allow access (e.g. 10.0.0.0/24, 172.10.0.1) at the ingress.

See <https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#whitelist-source-range>

### ip\_deny\_list

An array of source client CIDRs to deny access (e.g. 10.0.0.0/24, 172.10.0.1) at the ingress.

See <https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#denylist-source-range>

## Jobs

Jobs allow you to run arbitrary scripts during a deployment. This allows you to do anything before or after a service is deployed that is needed to setup your environment. A common example is database migrations before your backend comes up, but after you have deployed your database. Another good example might be running asset compilation. These tasks and any others can be accomplished using jobs.

```yaml
---
completed_timeout:
  type: Integer
  required: false
  description: How long (in seconds) Release will wait until the job is considered
    timed out and raise an error
  default: 1200
completions:
  type: Integer
  required: false
  description: Minimum Required Completions For Success
  default: 1
from_services:
  type: String
  required: false
  description: 'Name of service to inherit image from. Please note: this does NOT
    import or inherit the settings or options from the other service. You should explicitly
    set the job parameters on the job itself.'
halt_on_error:
  type: Boolean
  required: false
  description: When set to `true`, the deployment will be aborted with an error if
    the job fails.
image:
  type: String
  required: false
  description: The image to use for the job
name:
  type: String
  required: true
  description: Unique name to use when referencing the job
parallelism:
  type: Integer
  required: false
  description: Amount Of Parallelism To Allow
  default: 1
service_account_name:
  type: String
  required: false
  description: Runs the job using the given service account
annotations:
  type: Hash
  required: false
  description: Add any annotations which will appear on the pod this job runs on.
args:
  type: Array
  required: false
  description: Arguments that are passed to command on container start
build:
  type: Hash
  required: false
  description: Instructions for Release to build an image.
command:
  type: Array
  required: false
  description: Command to run on container start. Overrides what is in the Dockerfile
cpu:
  type: Hash
  required: false
  description: Same as resources, but for this job only. If not specified the default
    resources will be used. Can include units like `milli`, `centi`, etc.
memory:
  type: Hash
  required: false
  description: Same as resources, but for this job only.  If not specified the default
    resources will be used. Include the units in Gibibytes or Mebibytes.
node_selector:
  type: Array
  required: false
  description: Node Selector
nvidia_com_gpu:
  type: Hash
  required: false
  description: Specify the limits value for gpu count on this job. Do not specify
    `requests`. Must be an integer and cannot be overprovisioned or shared with other
    containers.
secrets_from:
  type: Array
  required: false
  description: References secrets configs by name. Click [here](#secrets-1) to see
    examples of how to define secrets configs.
volumes:
  type: Array
  required: false
  description: List of volumes and mount points
workspaces:
  type: Array
  required: false
  description: Attach workspaces to the container associated with this service.
```

Each job entry has a mutually exclusive requirement where either `image` or `from_services` must be present.

```yaml
jobs:
- name: migrate
  completed_timeout: 600
  command:
  - "./run-migrations.sh"
  from_services: backend
- name: setup
  parallelism: 0 # disabled
  command:
  - "./run-setup.sh"
  from_services: backend
  cpu:
    limits: 100
    requests: 100
  memory:
    limits: 1Gi
    requests: 1Gi
- name: mljob
  completed_timeout: 3600
  parallelism: 3
  completions: 3
  command:
  - "./run-ml-batch.sh"
  from_services: backend
  node_selector:
    key: "nvidia.com/gpu"
    value: "true"
  nvidia_com_gpu:
    limits: 1
```

> Jobs Example

### completions

Integer amount greater than zero to indicate how many successful runs should be considered finished. Usually you would set this value equal or greater than `parallelism` but it might be possible to set it less if you do not care about wasted pods being scheduled. Depending on `parallelism` and `concurrency_policy`, the combination of settings may cause jobs to be run multiple times in excess of this value. See the [Kubernetes documentation](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#concurrency-policy)

### parallelism

Integer amount of number of pods that can run in parallel. Set to 0 to disable the job. See the [Kubernetes documentation](https://kubernetes.io/docs/concepts/workloads/controllers/job/#parallel-jobs). This controls how many pods are potentially running at the same time during a scheduled run.

### service\_account\_name

See [service accounts](#service-accounts)

### node\_selector

Node Selectors allow you to assign workloads to particular nodes based on common labels such as `kubernetes.io/os=windows` and `kubernetes.io/arch=arm64`. Click [here](#service-node-selectors) for more information.

### secrets\_from

You need only reference the names of the secrets configs to utilize those secrets in your service:

```yaml
services:
  - backend:
    image: "..."
    ports:
      - type: node_port
        target_port: "5001"
        port: "5001"
    secrets_from:
    - default-secrets
    - backend-secrets
```

### workspaces

Click [here](#jobs-and-services-workspace-mounts) for more info on how to attach specific workspaces to this service.

### Service Command

Pods running in Kubernetes typically have a default command that is run upon container start. The value specified in the `command` will override the supplied Docker `ENTRYPOINT`. Please note there is a lot of confusion for Kubernetes commands and Docker commands since they use similar and overlapping meanings for each. The `command` is specified as an `EXECV` array, not a string.

You can specify the override command that a shell will start with.

```yaml
service:
- name: frontend
  command:
  - "/bin/sh"
  - "-c"
  - "sleep 3600"
```

## Notifications

Notifications allows you to configure how and where we notify you of events for an environment. The events for 'build', 'deploy', and 'pull\_request' subscribe to all variants for those events. If you want only a message when a build fails use "build.errored".

Here are the supported values:

* build - all build events
* build.started - only build started events
* build.completed - only build completed events
* build.errored - only build errored events
* deploy - all deploy events
* deploy.started - only deploy started events
* deploy.completed - only deploy completed events
* deploy.errored - only deploy errored events
* pull\_request - all pull\_request events
* pull\_request.created - only pull\_request created events

> NOTE: Slack integration *must* be configured to send any notifications to Slack.

```yaml
---
channel:
  type: String
  required: true
  description: Slack channel to post messages. Only valid if type is 'slack' and slack
    integration is be configured.
event:
  type: String
  required: true
  description: Define the event to be notified.
type:
  type: String
  required: true
  description: Currently only 'slack' is supported.
```

```yaml
notifications:
- type: slack
  event: build.errored # only when builds error
  channel: build-failures
- type: slack
  event: deploy # all deploy events
  channel: deploys
```

> Notifications Example

## Parameters

Parameters allow you to change inputs to your environments or jobs when deploying. These parameters give you deploy time customization of your deployments giving you flexibility and simplifying you configurations.

```yaml
---
advanced:
  type: Boolean
  required: false
  description: If true, this param will be in the advanced section
default:
  type: Any
  required: false
  description: Default value for parameter
description:
  type: String
  required: true
  description: Description of your parameter
name:
  type: String
  required: true
  description: Name of your parameter
optional:
  type: Boolean
  required: false
  description: If true, the parameter will be marked optional in the UI
placeholder:
  type: String
  required: false
  description: Placeholder or hint about what kind of values are valid for your param
type:
  type: String
  required: true
  description: The type of parameter
```

Examples of how to using parameters are below

```yaml
parameters:
  - name: name
    type: string
    description: name of runnable app
  - name: model-location
    type: string
    description: location of model
    placeholder: s3://...
  - name: training-dataset-location
    type: string
    description: location of training dataset
    placeholder: s3://...
  - name: validation-dataset-location
    type: string
    optional: true
    description: optional validation dataset location
    placeholder: s3://...
  - name: test-dataset-location
    type: file
    optional: true
    description: optional test dataset
  - name: fine-tuned-model-location
    type: string
    placeholder: s3://...
    description: Fine tuned model output location
    optional: true
    advanced: true
  - name: hyperparams
    type: text
    description: Hyper param values
    advanced: true
    default: |
      trainer.devices=4
      trainer.num_nodes=1
      trainer.precision=bf16
      trainer.val_check_interval=20
      trainer.max_steps=50
      model.megatron_amp_O2=False
      ++model.mcore_gpt=True
      model.tensor_model_parallel_size=4
      model.pipeline_model_parallel_size=1
      model.micro_batch_size=1
      model.global_batch_size=4
      model.data.train_ds.num_workers=0
      model.data.validation_ds.num_workers=0
      model.data.train_ds.concat_sampling_probabilities=[1.0]
      model.peft.peft_scheme="lora"
```

> Example: Various parameters including string and text types. Also, it shows how to use the advanced flag.

### type

The allowed types are: string, integer, list, text, boolean, git-repo, git-ref, and file

## Resources

Resources are service level defaults. They represent the resources allocated for each service. Storage is different in that not every container needs storage, so while you can specify defaults, not every container will use storage.

**Requests** define resource guarantees. Containers are guaranteed the request amount of resource. If not enough resources are available the container will not start.

**Limits**, on the other hand, make sure a container never goes above a certain amount of resource. The container is never allowed to exceed the limit.

`memory`: Limits and requests for memory are measured in bytes. You can express memory as a plain integer or as a fixed-point integer using one of these suffixes: E, P, T, G, M, K. You can also use the power-of-two equivalents: Ei, Pi, Ti, Gi, Mi, Ki. For example, the following represent roughly the same value: 128974848, 129e6, 129M, 123Mi

`nvidia_com_gpu`: Limits for Nvidia GPU units. (Do not specify `requests:`). GPU limits can only be integer values and cannot be shared concurrently with other containers. You must also specify a [node\_selector](#service_node_selectors) to schedule a job or service on the correct worker node(s).

`cpu`: Limits and requests for cpu are represented in millicpu. This is represented by '{integer}m', e.x: 100m (guarantees that service will receive 1/10 of 1000m, or 1/10 of 1 cpu). You can also represent the cpu resources as fractions of integers, e.x. 0.1, is equivalent to 100m. Precision finer than '1m' is not allowed.

`replicas`: This number is the number of containers that will run during normal operation. This field is an integer, e.x: 5, that would make 5 of each service.

`storage`: Consists of two values size and type. Size accepts the same values as memory and type is the type of storage, whether persistent/nfs, empty\_dir, shmem, or host\_path.

```yaml
---
replicas:
  type: Integer
  required: true
  description: Number of containers, per service
  default: 1
cpu:
  type: Hash
  required: true
  description: Limits and requests for cpus
  default: '{"requests"=|"100m"}'
memory:
  type: Hash
  required: true
  description: Limits and requests for memory
  default: '{"limit"=|"1Gi", "requests"=|"100Mi"}'
nvidia_com_gpu:
  type: Hash
  required: false
  description: Limits for nvidia.com/gpu tagged nodes
storage:
  type: Hash
  required: false
  description: Size and type definition
```

## Secrets

Release supports advanced features for AWS and Doppler for secrets Management.

If you would like to read about our Doppler integration click [here](https://docs.release.com/integrations/integrations-overview/doppler-secrets-manager#setting-up-doppler-in-release).

If you would like to read about our AWS integration click [here](https://docs.release.com/integrations/integrations-overview/aws-secrets-management).

```yaml
---
config:
  type: String
  required: false
  description: Doppler Config for release to use
name:
  type: String
  required: true
  description: Name to use when referring to the secrets in the config, using [secrets_from]().
namespace:
  type: String
  required: false
  description: Used to tell Release you have installed the Doppler operator in some
    other namespace than the default (doppler-operator-system).
pattern:
  type: String
  required: false
  description: Path with a asterisk for Release to follow in AWS Secrets Manager or
    AWS Parameter Store
project:
  type: String
  required: false
  description: Doppler Project for release to use
type:
  type: String
  required: true
  description: The two types supported are doppler and aws
```

Examples:

```yaml
secrets:
# defines the secrets for the Rails project
- name: development
  type: doppler
  project: rails  # project in doppler
  config: dev     # config in doppler

# defines the secrets for the AI project
- name: development-ai
  type: doppler
  project: ai
  config: dev
```

> Using two different doppler projects in Release

```yaml
secrets:
# defines the secrets for the Rails project
- name: development
  type: doppler
  project: rails  # project in doppler
  config: dev     # config in doppler

# defines the secrets for the AI project
- name: default-aws
  type: aws
  pattern: /Release/secrets/defaults/*
```

> Using doppler and aws. The pattern defines how Release will match secrets you want.

## Service Accounts

Service accounts allow you to control the cloud permissions granted to your workloads (services, jobs, infrastructure runners, etc.)

To apply a service account to a workload, set its `service_account_name` field to its `name`.

```yaml
---
cloud_role:
  type: String
  required: false
  description: |
    Cloud role to assume.

    On AWS, this is the IAM Role's ARN. On GCP this is the service account's email address.

    Optionally allows expansion of a `${cloud_account_id}` variable with the environment's
    cloud provider account ID.
name:
  type: String
  required: true
  description: Unique name to use when referencing the service account from `service_account_name`
```

```yaml
service_accounts:
- name: custom-role
  cloud_role: arn:aws:iam::111111111111:role/MyCustomRole
services:
- name: aws-cli
  image: amazon/aws-cli
jobs:
- name: aws-whoami
  from_services: aws-cli
  args:
  - sts
  - get-caller-identity
  service_account_name: custom-role
```

> Example: Assuming a custom IAM role from a job

```yaml
service_accounts:
- name: custom-role
  cloud_role: arn:aws:iam::${cloud_account_id}:role/MyCustomRole
```

> Example: Cloud account ID expansion

## Services

Services contain descriptions of each of your containers. They include many fields from your docker-compose and fields auto-generated by Release upon application creation. For each service you can define:

* Static javascript builds
* Open and map any number of ports
* Creates mounts and volumes
* Use ConfigMaps to modify config at run-time for off-the-shelf containers
* Override default resources
* Pin particular services to particular images
* Create liveness and readiness probes and set other k8s config params (e.x. max\_surge)
* Create stateful services
* External DNS entries for cross namespace services

```yaml
---
build_base:
  type: String
  required: false
  description: Path to the Javascript application if it does not reside at the root
build_command:
  type: String
  required: false
  description: Command to create the static Javascript build.
build_destination_directory:
  type: String
  required: false
  description: Directory to copy the generated output to
build_output_directory:
  type: String
  required: false
  description: Directory where the generated output is located
build_package_install_command:
  type: String
  required: false
  description: Command to install packages such as `npm install` or `yarn`. Defaults
    to `yarn`
completed_timeout:
  type: Integer
  required: false
  description: Time (in seconds) to wait for container to reach completed state
  default: 600
has_repo:
  type: Boolean
  required: false
  description: If we should reference an image built by Release
image:
  type: String
  required: false
  description: Name of or path to image
max_surge:
  type: String
  required: false
  description: K8s max_surge value (as a percentage from 0 to 100)
  default: 25
name:
  type: String
  required: true
  description: Name of your service
pinned:
  type: Boolean
  required: false
  description: Pin service to particular image
ready_timeout:
  type: Integer
  required: false
  description: Time (in seconds) to wait for container to reach ready state
  default: 180
replicas:
  type: Integer
  required: false
  description: Same as resources, but for this service only
service_account_name:
  type: String
  required: false
  description: Runs the service using the given service account.
stateful:
  type: Boolean
  required: false
  description: Deploys the service as a StatefulSet rather than a Deployment.
static:
  type: Boolean
  required: false
  description: When true, Release will create a static Javascript build. Review the
    following build_* attributes
annotations:
  type: Hash
  required: false
  description: Add any annotations which will appear on the pod this job runs on.
args:
  type: Array
  required: false
  description: Arguments that are passed to command on container start
autoscale:
  type: Hash
  required: false
  description: 'Autoscale configuration. See the #autoscale section below.'
scaled_object:
  type: Hash
  required: false
  description: 'Event-driven autoscaling with KEDA. See the #scaled_object section below.
    Mutually exclusive with autoscale and replicas.'
build:
  type: Hash
  required: false
  description: Instructions for Release to build an image.
command:
  type: Array
  required: false
  description: Command to run on container start. Overrides what is in the Dockerfile
configmap:
  type: Array
  required: false
  description: Specify a local file in your repository to mount inside a container.
cpu:
  type: Hash
  required: false
  description: Same as resources, but for this service only
depends_on:
  type: Array
  required: false
  description: List of service that must be deployed before
init:
  type: Array
  required: false
  description: List of containers to be invoked before the primary service
liveness_probe:
  type: Hash
  required: false
  description: Test of proper container operation
loadbalancer:
  type: Hash
  required: false
  description: 'per service loadbalancer configuration. See the #loadbalancer section
    below.'
memory:
  type: Hash
  required: false
  description: Same as resources, but for this service only
node_selector:
  type: Array
  required: false
  description: Node Selector
nvidia_com_gpu:
  type: Hash
  required: false
  description: Specify the limits value for GPU count on this service.
ports:
  type: Array
  required: false
  description: Set the ports which will be exposed for the service
readiness_probe:
  type: Hash
  required: false
  description: Test for proper container start-up
secrets:
  type: Array
  required: false
  description: Inline secrets
secrets_from:
  type: Array
  required: false
  description: References secrets configs by name. Click [here](#secrets-1) to see
    examples of how to define secrets configs.
sidecars:
  type: Array
  required: false
  description: List of containers along side the primary service
startup_probe:
  type: Hash
  required: false
  description: Allow more time for services which take longer to startup before the
    liveness_probe is applied.
storage:
  type: Hash
  required: false
  description: Same as resources, but for this service only
volumes:
  type: Array
  required: false
  description: List of volumes and mount points
workspaces:
  type: Array
  required: false
  description: Attach workspaces to the container associated with this service.
```

You can define how to build and deploy a static service or a Docker container not defined in your `docker-compose` file, for example, here we define a JavaScript static build:

```yaml
service:
- name: frontend
  build_base: my-app
  build_command: yarn build
  build_output_directory: build
  static: true
```

### service\_account\_name

See [service accounts](#service-accounts)

### autoscale

Read more details at the [Horizontal Pod Autoscaler docs](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#support-for-resource-metrics). The following example shows how you can scale based on both CPU and Memory targets:

```yaml
services:
- frontend:
  image: "..."
  autoscale:
    min_replicas: 1
    max_replicas: 5
    metrics:
    - resource:
        name: cpu
        average_utilization: 60
    - resource:
        name: memory
        average_value: 50Mi
```

You can also specify a custom metric that is reported by your own resource or helm chart. This is the [pods metric specification](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/#autoscaling-on-multiple-metrics-and-custom-metrics) which can scale pods based on some custom metric that is aggregated across the pods in question. For example if the `packets_per_second` metric is available in your cluster and it exceeds an average value across several pods then a scale up will be triggered. You can combine them with other metrics including `CPU` and `memory` as above.

```yaml
services:
- frontend:
  image: "..."
  autoscale:
    min_replicas: 1
    max_replicas: 5
    metrics:
    - pods:
        metric: packets_per_second
        average_value: 1K
```

### scaled\_object

`scaled_object` configures **event-driven autoscaling** for a service using [KEDA](https://keda.sh/) (Kubernetes Event-Driven Autoscaling). Where `autoscale` (the Horizontal Pod Autoscaler) scales on CPU/memory only, `scaled_object` can scale a service on external signals — a Kafka topic's lag, a Prometheus query, a cloud queue's depth, a cron schedule, and [any other KEDA scaler](https://keda.sh/docs/latest/scalers/). Release renders a KEDA `ScaledObject` into every environment's namespace automatically, so you configure it once instead of applying one per environment. See the [Event-driven autoscaling with KEDA](/reference-documentation/reference-examples/event-driven-autoscaling-with-keda) guide for a full walkthrough.

`scaled_object` is mutually exclusive with `autoscale` and `replicas` — KEDA manages the replica count itself.

{% hint style="info" %}
KEDA must be installed on the target cluster before you can use `scaled_object`, and the feature must be enabled for your account. If either is missing, Release rejects the configuration with a clear error. Contact Release to enable KEDA for your account. See the [KEDA guide](/reference-documentation/reference-examples/event-driven-autoscaling-with-keda#prerequisites).
{% endhint %}

The following example scales a `worker` service on a Kafka topic's consumer lag, between 1 and 10 replicas:

```yaml
services:
- name: worker
  image: "..."
  scaled_object:
    min_replica_count: 1
    max_replica_count: 10
    triggers:
    - type: kafka
      metadata:
        bootstrapServers: kafka.svc:9092
        consumerGroup: my-group
        topic: my-topic
        lagThreshold: '50'
```

#### Options

| Field               | Type    | Description                                                                                                                                              |
| ------------------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `min_replica_count` | Integer | Minimum replicas KEDA scales down to. Must be `>= 0`. When set, `max_replica_count` is required. Defaults to KEDA's default (`0`) when omitted.          |
| `max_replica_count` | Integer | Maximum replicas KEDA scales up to. Must be `> 0` and `>= min_replica_count`.                                                                            |
| `polling_interval`  | Integer | How often (in seconds) KEDA checks each trigger. Must be `> 0`. Defaults to KEDA's default (`30`) when omitted.                                          |
| `cooldown_period`   | Integer | Seconds to wait after the last trigger fires before scaling back to `min_replica_count`. Must be `> 0`. Defaults to KEDA's default (`300`) when omitted. |
| `triggers`          | Array   | Required. One or more scaling triggers. At least one is required.                                                                                        |

#### Triggers

Each entry in `triggers` describes one KEDA scaler:

| Field                | Type   | Description                                                                                                                                                |
| -------------------- | ------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `type`               | String | Required. The KEDA scaler type, e.g. `cpu`, `memory`, `kafka`, `prometheus`, `cron`. See the full [list of scalers](https://keda.sh/docs/latest/scalers/). |
| `metadata`           | Hash   | Required. Scaler-specific settings. Passed through to KEDA verbatim — see each scaler's docs for its keys.                                                 |
| `metric_type`        | String | Optional. One of `Utilization`, `AverageValue`, or `Value`. **Required for `cpu` and `memory` triggers** (KEDA 2.18+ moved this out of `metadata`).        |
| `authentication_ref` | Hash   | Optional. References a KEDA `TriggerAuthentication` object (`{ name: <name> }`) you supply on the cluster for scalers that need credentials.               |

{% hint style="info" %}
`metadata` is opaque — Release passes its keys and values to KEDA unchanged, so you can use any current or future KEDA scaler without waiting on a Release schema change. Refer to the [KEDA scaler docs](https://keda.sh/docs/latest/scalers/) for the exact `metadata` keys each scaler expects.
{% endhint %}

**CPU / memory triggers**

CPU and memory ("resource") triggers require `metric_type`:

```yaml
services:
- name: web
  image: "..."
  scaled_object:
    min_replica_count: 1
    max_replica_count: 5
    triggers:
    - type: cpu
      metric_type: Utilization
      metadata:
        value: '60'
```

**Scale-to-zero on a schedule**

A `cron` trigger scales deterministically on a schedule — useful for scaling a batch worker up during business hours and down (to `min_replica_count`) overnight:

```yaml
services:
- name: batch
  image: "..."
  scaled_object:
    min_replica_count: 0
    max_replica_count: 5
    triggers:
    - type: cron
      metadata:
        timezone: UTC
        start: 0 8 * * *
        end: 0 18 * * *
        desiredReplicas: '3'
```

### configmap

You can use configmaps to mount a file from your repository into a path specified inside the container. You can only reference files this way, not directories.

```yaml
services:
  grafana:
  - name: grafana
    image: grafana/grafana:9.1.0
    has_repo: false
    configmap:
    - name: grafana-ini
      mount_path: "/etc/grafana/grafana.ini"
      repo_path: "./src/grafana/grafana.ini"
    ports:
    - type: container_port
      port: '3000'
```

### loadbalancer

The following example shows explicitly set default values for a public node\_port loadbalancer with a hostname:

```yaml
services:
  - backend:
    image: "..."
    ports:
      - type: node_port
        target_port: "5001"
        port: "5001"
    loadbalancer:
      type: layer4
      visibility: public-direct
      hostname: backend-${env_id}.${domain}
      tls_enabled: false
      annotations:
```

The following example shows how to create a http internal loadbalancer with a hostname with TLS offload:

```yaml
services:
  - backend:
    image: "..."
    ports:
      - type: node_port
        target_port: "80"
        port: "443"
    loadbalancer:
      type: http
      visibility: private
      hostnames:
      - backend-${env_id}.${domain}
      - api-${env_id}.${domain}
      tls_enabled: true
      annotations:
        service.beta.kubernetes.io/aws-load-balancer-healthcheck-protocol: TCP # AWS
        service.kubernetes.io/healthcheck: k8s2-pn2h9n5f-l4-shared-hc # GKE
```

One of `hostname` (single) or `hostnames` (array) is the only required field to set the alias DNS for the loadbalancer as a list or single string

`type` supports one of `layer4` (default, TCP or UDP), `layer7` (optional generic passthrough), `http` (legacy HTTP/1.1 or HTTP/2), `http2`, or `grpc` (HTTP/2, especially the alias GRPC). Depending on your cloud provider and custom annotations, these settings may be identical.

`visibility` supports one of `public-direct` (default), or `private` for internet-facing or internal VPC addressing, respectively. (To be compatible with visibility rules, you can use `public` and `unmanaged` as synonyms for `public-direct`. The latter will not create a DNS entry.)

`tls_enabled` is one of `true` (enabled TLS negotiation and certificate offload) or `false` (default, for no TLS). You can also provide custom annotations to expand the coverage of the TLS configuration options.

`backend_protocol` is one of `tcp` (default, no TLS) or `tls` (to enable end-to-end encryption)

`annotations` is a hash of custom annotations to apply to the service. Depending on your cloud provider(s), these may not be portable between controllers. Mixing two cloud providers' annotations is not tested.

### node\_selector

Node Selectors allow you to assign workloads to particular nodes based on common labels such as `kubernetes.io/os=windows` and `kubernetes.io/arch=arm64`. Click [here](#service-node-selectors) for more information.

### nvidia\_com\_gpu

`limits:` must be an integer value. Do not specify `requests:`. GPU processors cannot be overprovisioned or shared with other containers.

### secrets\_from

You need only reference the names of the secrets configs to utilize those secrets in your service:

```yaml
services:
  - backend:
    image: "..."
    ports:
      - type: node_port
        target_port: "5001"
        port: "5001"
    secrets_from:
    - default-secrets
    - backend-secrets
```

### sidecars

For more info on sidecars click [here](#sidecars-2)

### workspaces

Click [here](#jobs-and-services-workspace-mounts) for more info on how to attach specific workspaces to this service.

### Stateful Sets and Deployments

`stateful` provides a StatefulSet which creates guarantees about the naming, ordering and uniqueness of a service.

* Stable, unique network identifiers.
* Stable, persistent storage.
* Ordered, graceful deployment and scaling.
* Ordered, automated rolling updates.

If an application doesn’t require any stable identifiers or ordered deployment, deletion, or scaling, you should either set stateful to **false** or remove it.

### Build

Instructions for Release to build an image. This needs to be combined with `has_repo: true`.

```yaml
---
context:
  type: String
  required: false
  description: Path to the files if they do not reside at the root. Defaults to '.'
dockerfile:
  type: String
  required: false
  description: Name of the Dockerfile to use. Defaults to 'Dockerfile'
name:
  type: String
  required: false
  description: Name of build
repo_branch:
  type: String
  required: false
  description: Combined with `repo_url`, use to target a specific branch
repo_commit:
  type: String
  required: false
  description: Combined with `repo_url`, use to target a specific commit
repo_url:
  type: String
  required: false
  description: If you want to create a Build from a different repository
target:
  type: String
  required: false
  description: If a specific build stage should be targeted
args:
  type: Array
  required: false
  description: Args passed into the build command
image_scan:
  type: Hash
  required: false
  description: Release can scan your built images for known security vulnerabilities
    **This feature is deprecated and will be removed -- contact support**
```

You can specify parameters that affect how your docker image is built.

```yaml
service:
- name: frontend
  context: dockerfiles/base
  dockerfile: Dockerfile-base
  target: web
  args:
  - MYVAR=somevalue
  - MYARG=somethingelse
```

### Build Image Scan

Release allows for scanning your images for vulnerabilities. If any are found, the build is marked as an error. You are able to designate what level of severity will cause an error and also whitelist specific vulnerabilities to ignore.

```yaml
---
severity:
  type: String
  required: true
  description: Level of severity which
whitelist:
  type: Array
  required: false
  description: List of vulnerabilities to ignore
```

```yaml
build:
  context: .
  image_scan:
    severity: high
    whitelist:
      - name: CVE-123
        description: "Release created this CVE"
        reason: "This CVE doesn't exist!"
```

> Example image scan that will fail the build if any CVE's with a severity level of high are found. The scan also skips over CVE-123 because it is known that Release created that fake CVE in this documentation.

### Service Command

Pods running in Kubernetes typically have a default command that is run upon container start. The value specified in the `command` will override the supplied Docker `ENTRYPOINT`. Please note there is a lot of confusion for Kubernetes commands and Docker commands since they use similar and overlapping meanings for each. The `command` is specified as an `EXECV` array, not a string.

You can specify the override command that a shell will start with.

```yaml
service:
- name: frontend
  command:
  - "/bin/sh"
  - "-c"
  - "sleep 3600"
```

### Service Resources

Resources can be overwritten on a service by service basis. The `resources` key is removed and each directive `cpu`, `memory`, `storage`, and `replicas` can be defined individually. If they are not specified the defaults will be used.

`cpu`, `memory`, `nvidia_com_gpu`, and `storage` define resource guarantees. The service definition for cpu, memory, nvidia\_com\_gpu, and storage overrides the values in resource\_defaults. In the case of `nvidia_com_gpu`, Kubernetes recommends setting `limits:` but not `requests:`, unless they are the same. You can use the service definition to more finely tune the amount cpu, memory, nvidia\_com\_gpu, and storage for each service.

`replicas` allows you to specify different amount of pods that will be deployed for your particular service.

### Init Containers

Init containers allow you define additional containers that share volume mounts from the primary service. These can be used to perform setup tasks that are required for the main service to run. Init containers should run to completion with an exit code of zero. Non zero exit codes will result in a CrashLoopBackoff.

```yaml
---
has_repo:
  type: Boolean
  required: false
  description: If we should reference an image built by Release
image:
  type: String
  required: false
  description: Name of or path to image
name:
  type: String
  required: true
  description: Name of the init container
args:
  type: Array
  required: false
  description: Arguments that are passed to command on container start
command:
  type: Array
  required: false
  description: Command to run on container start. Overrides what is in the Dockerfile
volumes:
  type: Array
  required: false
  description: List of volumes and mount points
```

```yaml
service:
- name: backend
  image: fred/spaceplace/backend
  init:
  - name: sync-seed-data
    command:
    - rsync
    - "-avzh"
    - fred@savage.com:/home/fred/seed-data
    - /app/seed-data
  - name: build-static-assets
    command:
    - rake assets:precompile
```

> Example init container which inherits image from the main service

You can also define init containers using off the shelf images like busybox. This can be useful for performing additional operations which don't require the main service image or require binaries not included in the primary service.

```yaml
- name: backend
  image: fred/spaceplace/backend
  init:
  - name: wait-for-my-other-service
    image: busybox:
    command:
    - sh
    - '-c'
    - while ! httping -qc1 http://myhost:myport ; do sleep 1 ; done
```

> Example init container using busybox to wait for another service to startup

#### volumes

See [Volumes](#volumes-1)

### Service Command

Pods running in Kubernetes typically have a default command that is run upon container start. The value specified in the `command` will override the supplied Docker `ENTRYPOINT`. Please note there is a lot of confusion for Kubernetes commands and Docker commands since they use similar and overlapping meanings for each. The `command` is specified as an `EXECV` array, not a string.

You can specify the override command that a shell will start with.

```yaml
service:
- name: frontend
  command:
  - "/bin/sh"
  - "-c"
  - "sleep 3600"
```

### Startup, Readiness, and Liveness Probes

`startup_probe`, `liveness_probe`, and `readiness_probe` are used to check the health of your service. When your code is deployed via a rolling deployment, the readiness\_probe will determine if the service is ready to serve traffic before adding it to the load balancer. Release will convert the docker-compose healthcheck to a liveness\_probe and readiness\_probe. Both liveness\_probe and readiness\_probe allow for more advanced configuration beyond the docker-compose healthcheck definition. A `startup_probe` may be used if an application takes a long time during startup.

```yaml
---
 services:
  # HTTP health check with customer header
  - name: frontend
    image: davidgiffin/spacedust/frontend
    command:
    - "./start.sh"
    completed_timeout: 240
    ready_timeout: 1200
    registry: local
    has_repo: true
    ports:
    - type: node_port
      target_port: '4000'
      port: '4000'
    startup_probe:
      exec:
        command:
        - curl
        - "-Lf"
        - http://localhost:4000
      failure_threshold: 90
      period_seconds: 30
      timeout_seconds: 10
    liveness_probe:
      exec:
        command:
        - curl
        - "-Lf"
        - http://localhost:4000
      failure_threshold: 30
      period_seconds: 30
      timeout_seconds: 10
    readiness_probe:
      exec:
        command:
        - curl
        - "-Lf"
        - http://localhost:4000
      failure_threshold: 30
      period_seconds: 30
      timeout_seconds: 10
    cpu:
      limits: 2000m
      requests: 100m
    memory:
      limits: 4Gi
      requests: 100Mi
    static: true
    build_command: GENERATE_SOURCEMAP=false yarn build
    build_base: frontend
    build_directory: build/
  - name: web
    readiness_probe:
      http_get:
        path: /healthz
        port: 8080
        http_headers:
          - name: Custom-Header
            value: Awesome
      initial_delay_seconds: 5
      period_seconds: 10
  # TCP health check
  - name: redis
    readiness_probe:
      tcp_socket:
        port: 6379
      initial_delay_seconds: 10
      period_seconds: 30
  # Command / shell health check
  - name: worker
    readiness_probe:
      exec:
        command:
          - cat
          - /tmp/healthy
      initialDelaySeconds: 5
      periodSeconds: 5
```

> In this example we show the various types of probes that you can define for services along with overrides for resources and timeouts, while also defining static builds.

### Service Node Selectors

Node Selector allows pods to chose specific nodes to run on. The most common use case is for selecting nodes with different OS (like Windows) or different architecture (like ARM64, GPUs), but could also select specific cloud provider settings such as AWS AZ (like us-east-1c).

```yaml
---
key:
  type: String
  required: true
  description: Label Key
value:
  type: String
  required: true
  description: Label value
```

```yaml
services:
  - name: frontend
    image: mcr.microsoft.com/windows/servercore:ltsc2019
    node_selector:
    - key: kubernetes.io/os
      value: windows

# Top level default
node_selector:
  - key: "topology.kubernetes.io/zone"
    value: "us-east-1c"
```

> Example Windows 2019 server selector for a specific zone

```yaml
services:
  - name: mlapp
    image: "registry.k8s.io/cuda-vector-add:v0.1"
    replicas: 4
    nvidia_com_gpu:
      limits: 1 # Must be an integer
    node_selector:
    - key: nvidia.com/gpu
      value: "true"
```

> Example Nvidia server selector for GPU processors

### Notice

Please note the top-level node\_selector will be treated as a global default that can be overridden inside each service or job.

### Automatic tolerations

Release will automatically apply tolerations for the following well-known selectors:

* `kubernetes.io/os=windows`
* `kubernetes.io/arch=arm64`
* `nvidia.com/gpu="true"`

This allows linux/amd64 workloads to mix with Windows, ARM64, and GPU nodegroup workloads.

### Helm examples

Release does not manage any Helm charts to use node\_selectors, so you will need to add your own YAML and tolerations as follows for Windows as an example:

```yaml
# My own helm yaml
# kind: deployment
# ...
nodeSelector:
  kubernetes.io/os: windows
  node.kubernetes.io/windows-build: '10.0.17763'
tolerations:
  - key: "os"
    operator: "Equal"
    value: "windows"
    effect: "NoSchedule"
```

> Example Helm chart addition to schedule Windows nodes.

```yaml
# My own helm yaml
# kind: deployment
# ...
    resources:
      limits:
        nvidia.com/gpu: 1
nodeSelector:
  nvidia.com/gpu: "true"
tolerations:
- key: nvidia.com/gpu
  operator: Exists
  effect: NoSchedule
```

> Example Helm chart addition to schedule GPU nodes.

```yaml
# My own helm yaml
# kind: deployment
# ...
nodeSelector:
  kubernetes.io/arch: arm64
tolerations:
  - key: "arch"
    operator: "Equal"
    value: "arm64"
    effect: "NoSchedule"
```

> Example Helm chart addition to schedule ARM64 nodes.

### Further Reading

[Kubernetes Taints Documentation](https://kubernetes.io/docs/setup/production-environment/windows/user-guide-windows-containers/#ensuring-os-specific-workloads-land-on-the-appropriate-container-host) [Kubernetes GPU Documentation](https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/#clusters-containing-different-types-of-gpus)

#### key

The kubernetes label key to select on.

#### value

The value of the kubernetes label to match on.

### Ports

Ports can be one of two types `container_port` or `node_port`.

`container_port` is used to define a port that another service will consume. Internal services like your data stores, caches and background workers should not be exposed to the internet and available only internally to other services.

`node_port` is used to define a service that you want to expose to the Internet. target\_port is the port on the pod that the request gets sent to from a load balancer or internally. Your application needs to be listening for network requests on this port for the service to work. port exposes the service on the specified port internally within the cluster.

You can set an optional `loadbalancer` flag to create a separate load balancer that can be used access the service over the Internet. loadbalancer is useful for exposing a TCP based service to the Internet that doesn't support HTTP/HTTPS traffic.

`node_port` will also define an ingress rule to allow HTTP/HTTPS traffic to be routed to a service. See the documentation on hostnames and rules to understand how to define ingress rules and mount your service at a custom path, etc.

```yaml
---
 services:
  # create an ingress rule on port 8080
  - name: frontend
    image: example-org/web-app/frontend
    has_repo: true
    ports:
      - type: node_port
        target_port: "8080"
        port: "8080"
  # create an ingress rule on port 8080 and listen locally on port 4572
  - name: localstack
    image: example-org/web-app/localstack
    has_repo: true
    ports:
      - type: container_port
        port: "4572"
      - type: node_port
        target_port: "8080"
        port: "8080"
  # create a load balancer that listens on port 6000
  - name: worker
    image: example-org/web-app/frontend
    has_repo: true
    ports:
      - type: node_port
        target_port: "6000"
        port: "6000"
        loadbalancer: true
```

> Container and Node Ports together

### Secrets

You can use secrets to mount a file with the contents of a secret into a path specified inside the container.

```yaml
---
mount_path:
  type: String
  required: true
  description: An unused directory name where you would like the secrets to appear.
name:
  type: String
  required: true
  description: Name of the volume in Kubernetes
items:
  type: Array
  required: true
  description: List of secret values to project into files.
```

Each entry in the `secrets` stanza requires a mount path, which needs to be an unused directory. For each mount path, a list of items is used to populate the files with the secret values. The `value` must be sourced from Release's Secrets Manager. The following YAML results in two secrets being mounted, one at `/etc/foo/my-secrets/my-secret.conf` and another at `/run/secrets/db-password`.

```yaml
services:
- name: frontend
  secrets:
  - name: conf-secrets
    mount_path: "/etc/foo"
    items:
    - key: config-secret
      value: "$secrets.rsm.secret-config-file"
      path: my-secrets/my-secret.conf
  - name: db-secrets
    mount_path: "/run/secrets"
    items:
    - key: db-password
      value: "$secrets.rsm.postgres-password"
      path: db-password
```

As noted above, the mount path has to be an unused directory. If you need the file to be mounted into a directory that is already populated, we suggest moving the files in the startup command for the service. An example is shown below moving the files inline, however this same result could be achieved using a script.

```yaml
services:
  - name: frontend
    secrets:
      - name: conf-secrets
        mount_path: "/etc/foo"
        items:
          - key: config-secret
            value: "$secrets.rsm.secret-config-file"
            path: my-secrets/my-secret.conf
    command:
      - bash
      - "-c"
      - cp ./etc/foo/my-secrets/my-secret.conf ./app/secrets/my-secret.conf && yarn start
```

### Secret Items

Attributes of the `items` in the `secrets` attribute

```yaml
---
key:
  type: String
  required: true
  description: The key in which the secret value is stored in the Kubernetes `Secret`
    object.
path:
  type: String
  required: false
  description: The file path in which the secret will be mounted.
value:
  type: String
  required: true
  description: The value stored in the corresponding `key`. `value` must be sourced
    from Release's Secrets Manager.
```

### Storage

*Usage of storage is deprecated* See [volumes](#volumes-1) and use `size`. If `storage` is present as well as `size` under `volumes`, the`volumes` attributes will take precedence

Use `storage` in combination with `volumes` to create persistent storage for your service.

```yaml
---
size:
  type: String
  required: true
  description: Size of storage
type:
  type: String
  required: false
  description: Type of storage. Accepts aws-efs or nfs as values.
```

```yaml
services:
- name: db
  image: postgres:latest
  volumes:
    - type: persistent
      name: postgres-data
      mount_path: "/var/lib/postgresql/data"
  ports:
    - type: container_port
      port: '5432'
  storage:
    size: 10Gi
    type: aws-efs
```

> Example storage definition with volumes to retain Postgresql data on container restarts

#### size

Measured in bytes. Defaults to 1Gi. You can express size as a plain integer or as a fixed-point integer using one of these suffixes: E, P, T, G, M, K. You can also use the power-of-two equivalents: Ei, Pi, Ti, Gi, Mi, Ki. For example, the following represent roughly the same value: 128974848, 129e6, 129M, 123Mi

### Volumes

Volumes provide a means to storing information on disk. This ensures no data is lost if the Pod restarts or as a means to sharing files between Pods.

Further reading can be found on the Kubernetes documentation for [Volumes](https://kubernetes.io/docs/concepts/storage/volumes)

```yaml
---
bucket:
  type: String
  required: false
  description: S3 bucket name referenced in the `s3_volumes`. Required for the type
    `s3`.
claim:
  type: String
  required: false
  description: Supplying a claim allows for connecting to a shared volume
mount_path:
  type: String
  required: true
  description: The location at which the volume will be mounted inside the container
name:
  type: String
  required: false
  description: Name of the volume
path:
  type: String
  required: false
  description: Deprecated.
server:
  type: String
  required: false
  description: Deprecated.
size:
  type: String
  required: false
  description: Size of storage in bytes.
type:
  type: String
  required: false
  description: What type of volume gets created. Allowed types are 'persistent', 'empty_dir',
    'host_path'.
```

```yaml
service:
- name: db
  image: postgres
  volumes:
  - type: persistent
    name: postgres-data
    mount_path: "/var/lib/postgresql/data"
    size: 10Gi
```

> Example `persistent` volume to retain Postgresql data on container restarts

```yaml
service:
- name: app
  volumes:
  - type: empty_dir
    name: empty-dir-volume
    mount_path: demo-volume
```

> Example `empty_dir` volume

```yaml
service:
- name: NeMo megatron launcher
  volumes:
  - type: shmem
    name: shmem
    size: 8Gi
    mount_path: /dev/shm
```

> Example `shmem` volume

```yaml
service:
- name: app
  volumes:
  - type: host_path
    name: host-path-volume
    mount_path: demo-volume
```

> Example `host_path` volume

```yaml
service:
- name: app
  volumes:
  - type: s3
    bucket: bucket-name
    mount_path: /bucket
```

> Example `s3` volume

#### claim

See [Shared Volumes](#shared-volumes) for more information.

#### name

* type: persistent - `name` is only reference for your team. Release auto generates the name of the PersistentVolumeClaim
* type: empty\_dir - The name is added to Kubernetes
* type: shmem - The name is added to Kubernetes
* type: host\_path - The name is added to Kubernetes

#### path

This field is deprecated, please do not use. It will be removed in future versions.

#### server

This field is deprecated, please do not use. It will be removed in future versions.

#### size

See the Kubernetes documentation for [Memory Resource Units](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#meaning-of-memory)

#### type

The allowed types

* persistent - Creates a PersistentVolumeClaim and a PersistentVolume. See the Kubernetes documentation for [Persistent Volumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/)
* empty\_dir - An emptyDir volume is initially empty. See the Kubernetes documentation for [emptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir)
* shmem - An emptyDir volume backed by memory. The memory is deducted from the container memory. You must set requests/limits appropriately. See the empty\_dir documentation above.
* host\_path - A hostPath volume mounts a file or directory from the host node's filesystem. See the Kubernetes documentation for [hostPath](https://kubernetes.io/docs/concepts/storage/volumes/#hostpath)
* s3 - An S3 volume present in the `s3_volumes` configuration. See [s3\_volumes](#s3_volumes) for information on how to create them.

### Jobs and Services Workspace Mounts

Jobs and Services can specify workspaces they would like mounted explicitly instead of using `auto_mount`. Auto mount will mount the workspace in every container and job. If you would rather control which workspaces are mounted in which jobs and/or services use the `workspaces` key word in your service and/or job definition is a way to do that. You are able to reference a workspace and and create a unique mount path for each job and/or service.

If you would like to read about workspaces in general click [here](#workspaces-3).

```yaml
---
name:
  type: String
  required: true
  description: Name of the declared workspace
path:
  type: String
  required: false
  description: Path to the workspace in the container
```

Examples:

```yaml
app: workspaces.ai
mode: development
environment_templates:
- name: ephemeral
- name: permanent
workspaces:
- name: ws1
  path: "/user/test"
  auto_attach: true
  mounts:
  - path: "/mnt2"
    source_url: s3://release-handsup-us-east-1-static-builds
- name: ws2
  path: "/user2/test"
  mounts:
  - path: "/mnt2"
    source_url: s3://fake/one
  - path: "/mnt"
    source_url: git://fake/two
- name: ws3
  path: "/user/test3"
  auto_attach: true
  mounts:
  - path: "/mountain"
    source_url: s3://release-handsup-us-east-1-static-builds
services:
- name: svc1
  image: nginx
  workspaces:
  - name: ws3
    path: "/user/test3"
jobs:
- name: echo
  from_services: svc1
  command:
  - echo
  - test
  args:
  - less
  workspaces:
  - name: ws2
    path: "/user/test2"
```

> Attach workspace: ws3 to service: svc1 and ws2 to job: echo. Workspace: ws1 is attached to all jobs and services

## Shared Volumes

Shared Volumes are optional. Each Shared Volume creates a single PersistentVolumeClaim and PersistentVolume. To connect a service to the shared volume, define a volume which sets its `claim` attribute to the `name` attribute of the shared volume.

```yaml
---
name:
  type: String
  required: true
  description: The name of the shared volume
path:
  type: String
  required: false
  description: Deprecated.
server:
  type: String
  required: false
  description: Deprecated.
size:
  type: String
  required: true
  description: Size of storage in bytes.
type:
  type: String
  required: true
  description: Type of storage. Defaults to `persistent`.
```

```yaml
shared_volumes:
 - name: ostrich
   size: 8Gi
   type: persistent
services:
  - name: frontend
    volumes:
      - claim: ostrich
        mount_path: /app/frontend
  - name: backend
    volumes:
      - claim: ostrich
        mount_path: /app/backend
```

> Example of shared volumes using persistent

### path

This field is deprecated, please do not use. It will be removed in future versions.

### server

This field is deprecated, please do not use. It will be removed in future versions.

### size

See the Kubernetes documentation for [Memory Resource Units](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#meaning-of-memory)

### type

The only currently supported value is `persistent`. Shared volumes are implemented using NFS in the cloud provider.

## Sidecars

It’s a generally accepted principle that a container should address a single concern only. Sidecar containers allow you to define additional workloads for a given service. Sidecar containers share resources with the main service container so they are great for shared filesystems, networking ports, routing process spaces, etc..

```yaml
---
has_repo:
  type: Boolean
  required: false
  description: If we should reference an image built by Release
image:
  type: String
  required: false
  description: Name of or path to image
name:
  type: String
  required: true
  description: Name of your sidecar
args:
  type: Array
  required: false
  description: Arguments that are passed to command on container start
command:
  type: Array
  required: false
  description: Command to run on container start
cpu:
  type: Hash
  required: false
  description: Same as resources, but for this sidecar only
liveness_probe:
  type: Hash
  required: false
  description: Test for container health / liveness
memory:
  type: Hash
  required: false
  description: Same as resources, but for this sidecar only
readiness_probe:
  type: Hash
  required: false
  description: Test for proper container start-up
startup_probe:
  type: Hash
  required: false
  description: Allow more time for services which take longer to startup before the
    liveness_probe is applied.
volumes:
  type: Array
  required: false
  description: List of volumes and mount points
```

```yaml
services:
  - name: frontend
    image: kornkitti/express-hello-world:master
    ports:
      - type: node_port
        target_port: "80"
        port: "80"
    volumes:
      - name: nginx-logs
        type: empty_dir
        mount_path: /var/log/ngnix
    sidecars:
      - name: nginx
        image: nginx:1.7.9
      - name: logtail
        from: logtailer
sidecars:
  - name: logtailer
    image: docker.elastic.co/logstash/logstash:7.10.1
    command:
      - tail
      - "-f"
      - /var/log/*
```

> Example sidecar definition with a reusable logstash container and nginx for serving static assets

### Service Command

Pods running in Kubernetes typically have a default command that is run upon container start. The value specified in the `command` will override the supplied Docker `ENTRYPOINT`. Please note there is a lot of confusion for Kubernetes commands and Docker commands since they use similar and overlapping meanings for each. The `command` is specified as an `EXECV` array, not a string.

You can specify the override command that a shell will start with.

```yaml
service:
- name: frontend
  command:
  - "/bin/sh"
  - "-c"
  - "sleep 3600"
```

## Workflows

By default there are three workflows : `setup`, `patch`, and `teardown`. `setup` is what creates your environment from scratch. `patch` is the workflow for deploying your code to an already setup environment. `teardown` is the workflow for deleting your environment and removing any cloud native resources that have been created during setup. These are auto-generated, but you can add your own jobs and change the order, add tasks, etc.

You may define your own workflows that you run on a one-off basis. These workflows allow you to run jobs inside the namespace of the deployed environment. Whenever, you click the Deploy button you will be given the option to run your one-off workflows.

<figure><img src="https://drive.google.com/uc?export=download&#x26;id=1yvNX9KW3lh08I5NALPIC-4q07mWQYa29" alt=""><figcaption><p>Deploy Button showing all runnable workflows</p></figcaption></figure>

```yaml
---
name:
  type: String
  required: true
  description: Name of the workflow.
wait_for_all_tasks_to_complete:
  type: Boolean
  required: false
  description: If true, will wait at the end of the workflow, for all tasks to finish
order_from:
  type: Array
  required: false
  description: Jobs and Services involved in the workflow
parallelize:
  type: Array
  required: false
  description: Alternative to order_from that runs your workflows in parallel
```

`wait_for_all_tasks_to_complete` is set to true by default. This means that when the deployment finishes it will wait for all the tasks to finish. If it's set to false, the stage will not wait for everything to finish and the next stage will run immediately.

```yaml
workflows:
  - name: setup
    parallelize:
    - step: services-0
      tasks:
      - services.chroma
      - services.ollama
    - step: services-1
      tasks:
      - services.api
      - jobs.pull-model
    - step: services-2
      tasks:
      - services.front-end
  - name: patch
    parallelize:
    - step: services-0
      tasks:
      - services.api
  - name: gitbook-ingest
    parallelize:
    - step: gitbook-ingest
      tasks:
      - jobs.gitbook-ingest
  - name: teardown
    parallelize:
    - step: remove-environment
      tasks:
      - release.remove_environment
```

> Example: Example showing setup, patch, teardown and a custom workflow that ingests docs from gitbook

### Workflow Parallelization

Parallelization of your workflows can result in significant decrease in the time it takes to deploy your environment. But, it may not be as simple as just telling Release to run all of your services and/or jobs in parallel. In some cases you may need certain services to wait before a migration job has run, for example. You can design your application around this issue, but in some cases it will be unavoidable and `parallelize` gives you both options.

```yaml
---
halt_on_error:
  type: Boolean
  required: false
  description: If halt_on_error is true, the deployment will stop if any of the tasks
    in this step fail.  If you don't include it, it's false by default.
step:
  type: String
  required: true
  description: A name for this step in the workflow
tasks:
  type: Collection of String or Hash
  required: true
  description: List of jobs and/or services to run in parallel
wait_for_finish:
  type: Boolean
  required: false
  description: Some steps you want to start first, but not wait for.  A long running
    static job is a good example of something you may want to run near the beginning
    of your workflows, but not hold up your backends for.
metadata:
  type: Array
  required: false
  description: Data and params you can pass to tasks
```

`wait_for_finish` allows you to customize the behavior of each step. By default everything in a step is run in parallel, but the workflow will not transition to the next step until every task in the step has finished. By setting it to `false`, the workflow will start it and the progress to the next step.

The most obvious use-case for it being false, is a long running frontend static build. You want to start it at the beginning of the workflow and make `wait_for_finish: false`, this way it will not hold up things that don't depend on it and by default the workflow will wait at the end for all jobs to finish, before transitioning to the next workflow.

`metadata` allows you to customize tasks by passing directives or data to them. It allows you to have multiple tasks in your task list, but send different kinds of data or params to specific tasks. See the schema for metadata in [Release Tasks](#releasehub-tasks-beta)

```yaml
workflows:
  - name: setup
    parallelize:
    - step: frontend
      tasks: [services.frontend]
      wait_for_finish: false
    - step: migrate
      tasks: [jobs.migrate]
    - step: backend
      tasks: [services.backend]
    - step: post-setup-deployment
      tasks: [jobs.setup]
  - name: patch
    - step: frontend
      tasks: [services.frontend]
      wait_for_finish: false
    - step: migrate
      tasks: [jobs.migrate]
    - step: backend
      tasks: [services.backend]
  - name: teardown
    - step: remove_environment
      tasks: [release.remove_environment]
```

> Example: Same services and jobs, but the frontend will be started first, but everything else will be allowed to run without waiting for it to finish. Only after everything else is done, will the workflow wait for `services.frontend` to finish.

### Release Tasks (beta)

Release has created a few tasks that you can reference and use in your workflows. These tasks are built into release and you can parameterize them and utilize them in your workflows.

* `pod_exec`: This task allows you to run any arbitrary commands on the pods of your choosing. This is very useful for sending messages to the service/s you have running on each pod. A good use case is you need to send a signal to all of your queue workers. If you used a `Job` (k8s jobs) you would need to find all the pods, etc to run the command on. This way Release does that for you and executes your command on each Pod.
* `pod_checker`: This task will check the states of the pods and not finish until at least one of those states is found on every pod. This is very useful if you would like to do something to, or run something on, or against those pods, but only after they have transitioned to a specific state or one of a list of states.
* `remove_environment`: This task is required in the `teardown` workflow. When it runs, Release will remove your environment from the UI and remove the namespace along with all corresponding objects from Kubernetes.

#### Pod Exec Metadata Schema

When you are using metadata to parameterize your `pod_exec` job, only a subset of the metadata directives are available.

```yaml
task_name:
  value: 'release.pod_exec'
  type: String
  description: complete name of the task this metadata is for
  required: true
wait:
  type: Integer
  description: |
    How long this task will wait to finish
  required: true
command:
  type: Array
  description: |
    Command you wish to run
  required: true
for_pod:
  type: String
  description: exact pod name to run against, only available when using helm.  Cannot be used with for_service.
  required: false
for_service:
  type: String
  description: service name to run the command against
  required: false
namespace:
  type: String
  default: current
  description: |
    Can be either `previous or current`, by default it is current.  This defines which namespace to run against.  This is most useful when using rainbow deploys because you have multiple namespaces in k8s per environment.
  required: false
```

#### Pod Exec Example

```yaml
workflows:
  - name: setup
    parallelize:
    - step: frontend
      tasks: [services.frontend]
      wait_for_finish: false
    - step: migrate
      tasks: [jobs.migrate]
    - step: backend
      tasks: [services.backend]
    - step: send-backend-pod-setup
      tasks: [release.pod_exec]
      metadata:
      - task_name: release.pod_exec
        command: ["ruby ./bin/backend_pod_setup.rb"]
        wait: 300
        for_service: backend
    - step: post-setup-deployment
      tasks: [jobs.setup]
  - name: patch
    - step: frontend
      tasks: [services.frontend]
      wait_for_finish: false
    - step: migrate
      tasks: [jobs.migrate]
    - step: backend
      tasks: [services.backend]
  - name: teardown
    - step: remove_environment
      tasks: [release.remove_environment]
```

> Pod Exec Example: We are going to `ruby ./bin/backend_pod_setup.rb` on each pod for the backend service.

**Full Metadata Schema**

```yaml
task_name:
  type: String
  description: complete name of the task this metadata is for
  required: true
wait:
  type: Integer
  description: |
    How long this task will wait to finish
  required: true
command:
  type: Array
  description: |
    Command you wish to run
  required: false
for_service:
  type: String
  description: service name to run the command against
  required: false
namespace:
  type: String
  default: current
  description: |
    Can be either `previous or current`, but default it is current.  This defines which namespace to run against.  This is most useful when using rainbow deploys because you have multiple namespaces in k8s per environment.
  required: false
states:
  type: Array
  default: [running,terminated]
  description: |
    These are pod states that the `pod_checker` task will check each pod for.
  required: false
exclude:
  type: Array
  description: |
    Services to exclude when running the pod checker
  required: false
include:
  type: Array
  description: |
    Services to include when running the pod checker
  required: false
```

**More Examples**

```yaml
workflows:
  - name: setup
    parallelize:
    - step: frontend
      tasks: [services.frontend]
      wait_for_finish: false
    - step: migrate
      tasks: [jobs.migrate]
    - step: backend
      tasks: [services.backend]
    - step: check-backend-pods
      tasks: [release.pod_checker]
      metadata:
      - task_name: release.pod_checker
        wait: 300
        states: ['running']
        include: [services.backend]
    - step: send-backend-pod-setup
      tasks: [release.pod_exec]
      metadata:
      - task_name: release.pod_exec
        wait: 300
        command: ["ruby ./bin/backend_pod_setup.rb"]
        for_service: backend
    - step: post-setup-deployment
      tasks: [jobs.setup]
  - name: patch
    - step: frontend
      tasks: [services.frontend]
      wait_for_finish: false
    - step: migrate
      tasks: [jobs.migrate]
    - step: backend
      tasks: [services.backend]
  - name: teardown
    - step: remove_environment
      tasks: [release.remove_environment]
```

> Pod Checker and Pod Exec Combined Example: We are checking the backend pods to make sure they are in the `running` state before running `ruby ./bin/backend_pod_setup.rb` on each pod.

## Workspaces

Workspaces are collections of data sources and/or repositories that are made available for your containers. Workspaces allow you access to the data you need for your workflows located locally or in the cloud (S3, Cloud Storage, etc). Release can mount S3 buckets, Cloud Storage Buckets, and Git Repositories.

<figure><img src="https://drive.google.com/uc?export=download&#x26;id=1M_Yab0tpoQhW9SGf5juPXuvWbJfei6Ah" alt=""><figcaption><p>Diagram of workspace with mounts</p></figcaption></figure>

You can make workspaces more dynamic by combining Parameters and Workspaces to change mounts or paths at deploy time.

<figure><img src="https://drive.google.com/uc?export=download&#x26;id=1P-3smk6dimkdiD8u2V0vNbC-fUsYQjIs" alt=""><figcaption><p>Diagram of workspace with mounts use parameters</p></figcaption></figure>

```yaml
---
auto_attach:
  type: Boolean
  required: false
  description: If true, this workspace will be attached to every job and/or service
name:
  type: String
  required: true
  description: Name of the workspace.
path:
  type: String
  required: true
  description: Workspace path in container
mounts:
  type: Array
  required: false
  description: List of data sources and where you would like to mount them
```

Examples:

```yaml
workspaces:
- name: ws1
  path: "/workspace"
  auto_attach: true
  mounts:
  - path: "/data"
    source_url: https://github.com/aicompany/data
- name: ws2
  path: "/user/test2"
  auto_attach: true
  mounts:
  - path: "/mountain"
    source_url: s3://release-handsup-us-east-1-static-builds
```

> Mount github repo at /data in the workspace and an S3 bucket in second workspace

```yaml
parameters:
- name: foo
  description: "for testing only"
  type: string
  default: schnell
workspaces:
- name: ws1
  path: /ws-path
  mounts:
  - path: mnt-path
    source_url: ${parameters.foo}
```

> Use parameters to change source URL upon environment creation

### Workspace Mounts

A list of mount paths and source urls for datasources you would like mounted in your containers and/or jobs.

```yaml
---
path:
  type: String
  required: true
  description: Path in the workspace where you would like something mounted
source_url:
  type: String
  required: true
  description: Workspace mounts must have a 'source_url' defined as a valid URL for
    s3, git, or https.
```

Examples:

```yaml
workspaces:
- name: ws1
  path: "/workspace"
  auto_attach: true
  mounts:
  - path: "/data"
    source_url: https://github.com/aicompany/data
- name: ws2
  path: "/user/test2"
  auto_attach: true
  mounts:
  - path: "/mountain"
    source_url: s3://release-handsup-us-east-1-static-builds
```

> Mount github repo at /data in the workspace and an S3 bucket in second workspace




---

[Next Page](https://docs.release.com/llms-full.txt/1)