Using an Existing EKS Cluster
Last updated
Last updated
By default, Release creates a new dedicated cluster from scratch when you create a new EKS cluster. However, you can create an EKS cluster using an existing EKS cluster, which is useful if you have specific requirements for your clusters, like specific security policies, internal processes and best practices, or existing infrastructure-as-code configurations.
To successfully use an existing EKS cluster with Release, take note of the requirements.
Take a look at the [eksctl documentation on non eksctl-created clusters] to understand the requirements and get a sense of what is supported.
Follow the instructions in the AWS EKS authentication documentation to give Release access to your cluster.
In summary, you need:
An existing EKS cluster and node groups with associated VPC, routing requirements, and subnets.
An updated aws-auth
ConfigMap
giving Release access to your Kubernetes cluster.
Supporting add-ons or Helm charts – listed below.
If you have an existing EKS cluster, you should already have all the networking, routing, and security settings configured correctly.
The following table outlines the minimum configuration Release requires for your existing EKS cluster to ensure workloads deployed to it operate correctly.
AWS Resource | Min | Recommended | Notes |
---|---|---|---|
This section is optional if you used AWS-native tools like eksctl or the AWS Console.
You must apply tags to each subnet when you create a cluster to tell Release what the subnet's function is (public or private) and which subnets to deploy to.
Be sure the name of the cluster you create matches the variable <cluster_name>
wherever it is used.
You need at least two private subnets. Although public subnets are optional, you need at least two if you have any.
Note that tags for private and public subnets are different.
Release requires several add-ons and Helm charts to function properly, please go through the following table to identify if anything needs to be added to your cluster to function properly with Release. If you have any questions or concerns, feel free to reach out to us.
ConfigMap
Release uses a "Console Role" to access your cloud resources and identify itself to access your EKS cluster. You will need to get the console role ARN from the Cloudwatch resources output. It will look something like releasehub-integration-ConsoleRole-XYZ
in AWS IAM.
The following documentation walks you through the same steps as outlined in the AWS EKS authentication documentation. Read that document and refer to it as we go along here.
To access your cluster from the control plane, Release needs the Amazon Resource Name (ARN) for its role.
To find the ARN, go to the AWS Cloudformation template called release-integration
. In the Resources tab, find the link to the ConsoleRole
and follow it to AWS IAM. Copy the ARN, as you'll need it in the next steps.
For a cluster named my-cluster
in region-code
, the role ARN might look something like this: arn:aws:iam::111122223333:role/release/release-integration-ConsoleRole-xxxx
.
To use the ARN to grant Release permission to deploy to your cluster, you need to remove the /release
path.
Remember: The ARN MUST NOT have any path prefixes after the role. For example, arn:aws:iam:xyz:role/my/long/path/role-name
needs to be shortened to arn:aws:iam:xyz:role/role-name
. You also cannot use an STS or assumed role; you must use the original role. For example, you cannot use arn:aws:sts:assume-role/role-name/role-session
.
We can now grant permission to Release to deploy to your cluster by adding an entry to its aws-auth
ConfigMap
using the shortened ARN:
Please note that this example creates a role that has administrative access to your cluster. This is usually acceptable because Release needs to perform administrative actions in the cluster, like creating namespaces or installing Helm charts. If you need to restrict the role, let us know what level of permissions you would like to restrict and we can work with you to verify the permissions will work with our deployment processes.
Now that the cluster information is readily available and Release has been granted access, you will be able to import the cluster as you can see in the dialog box:
View the configuration of your cluster with the following command:
It will look something like this:
You can also navigate to Settings -> Clusters -> Cluster and click the Verify Cluster button. If the cluster status remains "Pending" (or worse, "Errored"), verify the configurations and settings described above. If you get no errors, your cluster is ready to go!
Tag Key | Tag Value | Example | Notes |
---|---|---|---|
Add-on or Helm Chart | Required? | Min. version | Notes |
---|---|---|---|
Name of Field | Required Information | Example | Notes |
---|---|---|---|
VPC CIDR mask
/19
/16
A VPC should be created with a CIDR block big enough to support the workloads and services deployed to it. We recommend using a /16 size for all workload types.
Private subnets
3
3
At least three subnets in three Availability Zones.
Public subnets
3
3
At least three subnets in three Availability Zones.
Internet gateway
1
1
Pods and workloads need to be able to access the internet. Creating a completely isolated private cluster is not tested at this time.
NAT gateway
1
1
Only one NAT gateway with an EIP is recommended; having more is supported if you require high availability for your production workloads.
Node type
t3a.large
t3a.xlarge preprod, m5a.2xlarge prod
Instance sizes are highly dependent on your workload count and type.
Node count
3
>3
We recommend using a minimum of three and a maximum of ten nodes, with a steady state of three or four. (Autoscaling is supported if you install the cluster autoscaler.)
Spot Instances
0
0
You can enable spot workloads in a separate node group. We recommend using at least one non-spot workload node group for cluster support services.
kubernetes.io/cluster/<cluster_name>
shared
kubernetes.io/cluster/production: "shared"
In our testing, it didn't matter whether using "owned" or "shared".
kubernetes.io/role/internal-elb
1
kubernetes.io/role/internal-elb: "1"
This tag should only be applied to each PRIVATE subnet.
kubernetes.io/role/elb
1
kubernetes.io/role/elb: "1"
This tag should only be applied to each PUBLIC subnet.
CoreDNS
yes
latest
CoreDNS provides intra- and inter-cluster DNS lookups.
Amazon VPC CNI
yes
latest
Required to connect to Amazon networking resources.
Amazon EBS CSI
yes
latest
Allows pods and workloads to mount EBS volumes; not required if you never use storage volumes.
OIDC provider
yes
N/A
The OIDC provider can be added at cluster-create time or later; it is recommended you take advantage of service accounts and IAM roles in your cluster.
External DNS
yes
latest
External DNS is a Helm chart that allows Release to add external or internal DNS entries for services and load balancers. Highly recommended but not necessary if you never use Route53 DNS.
Datadog
no
latest
Highly recommended to capture logs and metrics. Release offers a Datadog integration but unmanaged clusters are not included with our integration unless you request it.
Cloud Provider Integration
The cloud integration tied to your AWS account
production
This is a drop-down that you cannot edit, so you must create it beforehand and make sure it is attached to the same account with the existing resources.
Region
The region where the existing resources exist
us-east-1
This is a drop-down and must match the region where the existing resources are created.
Cluster Name
The name of the existing cluster
prod-cluster
This is a drop-down and must match the name of your cluster.
Domain
The name of the subdomain to use
release.example.com
This domain must be created as part of the cluster prerequisites or choose a Release-supplied domain name.