LogoLogo
  • Welcome to Release
  • Getting started
    • Quickstart
    • Create an account
    • Prepare to use Release
    • Create an application
      • Create custom application
      • Create from template
      • Servers vs runnables
    • Create an environment
  • Guides and examples
    • Domains and DNS
      • Manage domains
      • DNS and nameservers
        • Configure GoDaddy
        • Configure Cloudflare
        • Configure Namecheap
        • Other DNS hosts
      • Routing traffic
    • Example applications
      • Full stack voting app
      • Flask and RDS counter app
      • Static site with Gatsby
      • Golang with Postgres and Nginx
      • WordPress with MySQL
      • Spring and PostgreSQL
      • Terraform and Flask
      • OpenTelemetry demo
      • Load balancer with hostname
      • Static JavaScript service
      • SSH bastion access to services
      • ngrok and OAuth for private tunnels
      • Using OAuth Proxy
      • Hybrid Docker and static site
      • App Imports: Connecting two applications
      • Example library
    • Running instances
      • Cron jobs
      • Jobs
      • Using Helm charts
      • Using terminal
      • Viewing logs
      • Troubleshooting
        • ImagePullBackoff error
        • CrashLoopBackoff error
        • Exit codes
        • OOM: out of memory
    • Advanced guides
      • Containers guide
      • Application guide
      • Kubernetes guide
      • Create a cluster
      • Upgrade a cluster
      • Managing node groups
      • Patch node groups
      • Hostnames and rules
      • Serve traffic on multiple ports
      • Configure access to your K8s cluster
      • Designing for multiple environments
      • Microservices architecture
      • Monitoring your clusters
      • Performance tuning
      • Visibility and monitoring
      • Working with data
        • Container-based data
        • Seeding and migration
        • Cloud-provided data
        • Golden images
        • Third party
      • Pausing Instant Datasets
        • Application pausing schedules
        • Pause/resume environments
      • Infrastructure as code
        • Terraform
  • Reference documentation
    • Account settings
      • Account info
      • Managing users
      • Build settings
        • Build arguments
        • Build SSH keys
      • Add integrations
      • View clusters and cloud integrations
      • Add datasets
      • Environment handles
    • Workflows in Release
      • Stages of workflows
      • Serial deployments
      • Parallel deployments
      • Rolling deployments
      • Rainbow deployments
    • Networking
      • Network architecture (AWS)
      • Network architecture (GCP)
      • Ingresses
      • IP addresses
      • Cloud-provided services
      • Third-party services
    • Release environment versioning
    • Application settings
      • Application Template
        • Schema definition
      • Default environment variables
      • GitHub
      • Pull requests
      • GitOps
      • Just-in-time file mounts
      • Primary App Link
      • Create application FAQ
      • App-level build arguments
      • Parameters
      • Workspaces
    • End-to-end testing
    • Environment settings
      • Environment configuration
      • Environment variables
        • Environment variable mappings
        • Secrets vaults
        • Using Secrets with GitOps
        • Kubernetes Secrets as environment variables
        • Managing legacy Release Secrets
    • Environment expiration
    • Environment presets
    • Instant datasets on AWS
    • Instant datasets on GCP
    • Instant dataset tasks
      • Tonic Cloud
      • Tonic On-Premise
    • Cloud resources
    • Static service deployment
    • Helm
      • Getting started
      • Version-controlled Helm charts
      • Open-source charts
      • Building Docker images
      • Ingress and networking
      • Configuration
    • GitOps
    • The .release.yaml file
    • Docker Compose conversion support
    • Reference examples
      • Adding and removing services
      • Managing service resources
      • Adding database containers to the Application Template
      • Stock Off-The-Shelf Examples
    • Release API
      • Account Authentication
      • Environments API
        • Create
        • Get
        • Setup
        • Patch
      • User Authentication
      • Environment Presets API
        • Get Environment Preset List
        • Get Environment Preset
        • Put Environment Preset
  • Background concepts
    • How Release works
  • Frequently asked questions
    • Release FAQ
    • AWS FAQ
    • Docker FAQ
    • JavaScript FAQ
  • Integrations
    • Integrations overview
      • Artifactory integration
      • Cloud integrations (AWS)
        • AWS guides
        • Grant access to AWS resources
        • AWS how to increase EIP quota
        • Control your EKS fleet with systems manager
        • Managing STS access
        • AWS Permissions Boundaries
        • Private ECR Repositories
        • Using an Existing AWS VPC
        • Using an Existing EKS Cluster
      • Docker Hub integration
      • LaunchDarkly integration
      • Private registries
      • Slack integration
      • Cloud integrations (GCP)
        • GCP Permissions Boundary
      • Datadog Agent
      • Doppler Secrets Manager
      • AWS Secrets Management
    • Source control integrations
      • GitHub
        • Pull request comments
        • Pull request labels
        • GitHub deployments
        • GitHub statuses
        • Remove GitHub integration
      • Bitbucket
      • GitLab
    • Monitoring and logging add-ons
      • Datadog
      • New Relic
      • ELK (Elasticsearch, Logstash, and Kibana)
  • Release Delivery
    • Create new customer integration
    • Delivery guide
    • Release to customer account access controls
    • Delivery FAQs
  • Release Instant Datasets
    • Introduction
    • Quickstart
    • Security
      • AWS Instant Dataset security
    • FAQ
    • API
  • CLI
    • Getting started
    • Installation
    • Configuration
    • CLI usage example
    • Remote development environments
    • Command reference
      • release accounts
        • release accounts list
        • release accounts select
      • release ai
        • release ai chat
        • release ai config-delete
        • release ai config-init
        • release ai config-select
        • release ai config-upsert
      • release apps
        • release apps list
        • release apps select
      • release auth
        • release auth login
        • release auth logout
      • release builds
        • release builds create
      • release clusters
        • release clusters exec
        • release clusters kubeconfig
        • release clusters shell
      • release datasets
        • release datasets list
        • release datasets refresh
      • release deploys
        • release deploys create
        • release deploys list
      • release development
        • release development logs
        • release development start
      • release environments
        • release environments config-get
        • release environments config-set
        • release environments create
        • release environments delete
        • release environments get
        • release environments list
        • release environments vars-get
      • release gitops
        • release gitops init
        • release gitops validate
      • release instances
        • release instances exec
        • release instances logs
        • release instances terminal
  • Release.ai
    • Release.ai Introduction
    • Getting Started
    • Release.ai Templates
    • Template Configuration Basics
    • Using GPU Resources
    • Custom Workflows
    • Fine Tuning LlamaX
    • Serving Inference
Powered by GitBook
On this page
  • Set up an Instant Dataset in Release
  • Create an Instant Dataset
  • Set up your application to use Instant Datasets
  • Deploy your application to use your Instant Dataset
  • Modify an existing Instant Dataset
  • Change the target number of available databases
  • Change the Cloud Sql instance type
  • Advanced Modifications
  • Changing the Instant Dataset master password

Was this helpful?

  1. Reference documentation

Instant datasets on GCP

How to use Instant Datasets in Release on GCP

PreviousInstant datasets on AWSNextInstant dataset tasks

Last updated 10 months ago

Was this helpful?

Access to data can be a challenge when developing software.

Seed data allows you to generate a simple and consistent dataset, but such a dataset is unlikely to be reflective of what your application will be accumulating in production. As your application changes, access to data becomes yet another problem to manage.

A more optimal solution to the problem of accessing data would be to have a pool of data resources in various forms – such as in a database or cache, in search infrastructure, and so on – ready to be used immediately by an environment for the duration of the environment's life.

This is the solution Instant Datasets provide. With minimal set up, you can have production-replica data available to any of your environments, instantly, regardless of dataset size.

Release currently supports the Instant Datasets feature for databases on Google Cloud and .

Set up an Instant Dataset in Release

Each Instant Dataset is limited to a single account.

Create an Instant Dataset

Log on to Release and click on Configurations. Open the Datasets tab under the "Deploy" heading on the sidebar to go to the Datasets screen. Click the Create Dataset button to create a new Instant Dataset.

Complete the fields as follows:

  • Dataset Name: Give your dataset a name. You can choose any name that will help you remember what this dataset contains.

  • Select a Snapshot: These are the snapshots available and accessible by the cluster you selected.

  • Database Name: This is the name of the database that will be created in the Instant Dataset. This is the name an application will use to connect to the underlying database in the Instant Dataset.

  • User Password: This is the password an application will use to connect to the underlying database in the Instant Dataset.

  • Number of available Instant Databases: This is the target number of available databases in the Instant Dataset pool that are free to checkout at any time. Each time an application environment is created that uses one of the instances, the system starts the creation of another database in the set. The ideal size will be the number of "spare" instances you need to keep up with environment creation. For instance, if you create two environments within a few minutes of each other before the pool is replenished, you would likely want at least two instances available at any given time.

We generally recommend starting with two available Instant Databases if you unsure. You can increase this number later, if you need to. We do not recommend setting the pool size to one, unless you are sure that you will not run out of new database instances for a new environment while a replacement is being generated.

If you ever find that environments are delayed or fail to be created during Instant Dataset checkouts, you will need to increase the number of available Instant Databases. If you find that you have a lot of database instances sitting idle all day, you can reduce the number of available Instant Databases, but take care not to make the number too small.

Click Create to begin the process of initializing the database instances. This process could take a few minutes to a few hours to complete, depending on the size of the database.

Once the database is created, it takes some time to restore the data from the snapshot, depending on the size of the snapshot.

Once the dataset is ready to be used, it will transition to an Active state and environments can use it when they are deployed.

Click on a dataset to see more details about it.

Set up your application to use Instant Datasets

You will need to modify the Application Template and environment configuration for environments that will use Instant Datasets.

If an Instant Dataset was not set up to work with every ephemeral environment through its default configuration, you can add it explicitly to an environment you create. The syntax is the same whether Release automatically creates the environment or you do it manually.

environment_templates:
- name: ephemeral
  datasets:
  - name: release-dataset

In the environment configuration, create a mapping using the "Generated Environment Variables" section in the Instant Dataset details page.

---
mapping:
  DATABASE_HOST: RELEASE_PROD_FOR_DEVELOPMENT_RDS_DB_POOL_HOST
  DATABASE_PASSWORD: RELEASE_PROD_FOR_DEVELOPMENT_RDS_DB_POOL_PASS
  DATABASE_USER: RELEASE_PROD_FOR_DEVELOPMENT_RDS_DB_POOL_USER

Your environment variables are on the left side of the colon, and the generated environment variables are on the right.

Notice: These changes will only be propagated to newly created environments. You cannot apply a new Instant Dataset configuration retroactively to an existing environment. We are working on fixes to update this experience in the future.

Deploy your application to use your Instant Dataset

Your application is now ready to use your Instant Dataset!

  • Whenever you deploy an ephemeral environment, it will checkout one of the available (active) databases to be used while the environment exists.

  • The Instant Dataset will then create an additional database to maintain the available database count.

Modify an existing Instant Dataset

Change the target number of available databases

To change the target size of an Instant Dataset, go to the Datasets tab in account settings and click on the edit button on the right (pencil icon) for the dataset that you want to modify.

Reducing the number of available databases will not destroy any databases currently in use by an environment.

Change the Cloud Sql instance type

Changing the instance type will destroy ALL standby databases in the Instant Dataset. You might not be able to create a new environment during this window, because there will be no database instances available to checkout. You should only perform this step during off-hours, when you are unlikely to make new environments.

To change the instance type for an Instant Dataset, go to the Datasets tab in account settings and click on the edit button on the right (pencil icon) for the dataset that you want to modify.

Advanced Modifications

Expanding the Advanced drop-down allows you to change:

  • Which snapshot to be used for the next refresh

  • Which snapshot refresh strategy to be used for the next refresh

  • How many days, if any, to delay the snapshot selection. This defaults to zero which means no delay is applied by default. This only applies when Latest is the selected strategy.

The default snapshot refresh strategy is Latest.

By default there is no snapshot delay.

The snapshot refresh strategy can be set to one of the following:

  • Latest: use the most recently available snapshot, taking into account any delay, and apply that during refresh

  • Static: use the currently selected snapshot and do not update it during refresh.

Changing the Instant Dataset master password

On the Edit Dataset page, you can change the master password for the Instant Dataset. This will change the password for all databases in the Instant Dataset.

In order to create Instant Datasets in Release, you'll need a snapshot to restore from. Ensure you have automatic snapshots set up for your Cloud SQL database. Take a look at the Google Cloud SQL document to set up automatic snapshots.

Cluster: Instant Datasets must be assigned to a cluster. You can select from a list of created clusters in the dropdown. If you have not created a cluster yet, read the to learn how to create one.

Instance Type: This refers to the database instance type in GCP Cloud Sql. for additional information.

In the , add a datasets line. Give it a name property, with the name of your Instant Dataset as the value:

About Cloud SQL backups
clusters documentation
Refer to this document
Application Template
AWS
High-level diagram of using Instant Datasets
Instant Dataset creation prompt
List of Instant Datasets
Instant Dataset details
Linking an Instant Dataset to an ephemeral environment in the application's template config
Linking an Instant dataset to an ephemeral environment in the application's environment config
Go to the dataset details page
Change the number of datasets
Go to the dataset details page
Instant Dataset modification prompt
Instant Dataset modification prompt
Password change prompt