> For the complete documentation index, see [llms.txt](https://docs.release.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.release.com/release.ai/release.ai-gpu-resources.md).

# Using GPU Resources

Kubernetes, the backbone of Release AI, provides mechanisms to allocate GPUs to your applications. This is crucial for running compute-intensive tasks such as machine learning model training, deep learning inference, and high-performance data processing.

## Example Configurations for GPU Usage

### Selecting GPUs for Your Service

To start, you need to specify that your service requires GPUs. This can be done by setting the nvidia.com/gpu limits. Below is an example configuration:

```yaml
services:
 - name: mlapp
   image: "registry.k8s.io/cuda-vector-add:v0.1"
   replicas: 4
   nvidia_com_gpu:
     limits: 1 # Number of GPUs, must be an integer
   node_selector:
     - key: nvidia.com/gpu
       value: "true"
```

In this example:

* The `mlapp` service is configured to run with an image that leverages CUDA.
* The `replicas` field is set to `4`, meaning four instances of this service will run.
* The `nvidia_com_gpu.limits` is set to `1`, specifying that each instance will use one GPU.
* The `node_selector` ensures that the service is scheduled on nodes with GPU resources.

### Selecting an Instance Type Known to Have GPUs

Choosing the right instance type is important for your application to access the necessary GPU resources. Here’s how you can specify an instance type with GPUs:

```yaml
node_selector:
  - key: nvidia.com/gpu.product
    value: A10G
  - key: beta.kubernetes.io/instance-type
    value: g5.12xlarge
```

In this example:

* The `nvidia.com/gpu.product` key is set to `A10G`, specifying that the nodes should have NVIDIA A10G GPUs.
* The `beta.kubernetes.io/instance-type` key is set to `g5.12xlarge`, which is a known instance type that includes GPU resources.

### Complete Configuration Example

Combining the GPU selection and instance type configuration, here is the complete setup:

```yaml
services:
 - name: mlapp
   image: "registry.k8s.io/cuda-vector-add:v0.1"
   replicas: 2
   nvidia_com_gpu:
     limits: 2 
   node_selector:
     - key: nvidia.com/gpu
       value: "true"
     - key: nvidia.com/gpu.product
       value: A10G
     - key: beta.kubernetes.io/instance-type
       value: g5.12xlarge
```

In this example:

* The `services` section defines the application services in your deployment.
* The `mlapp` service runs a Docker image optimized for CUDA operations.
* The `replicas` field specifies the number of instances to run.
* The `nvidia_com_gpu.limits` ensures each instance gets `2` GPU.
* The `node_selector` filters the nodes to only those with the specified GPU (`A10G`) and instance type (`g5.12xlarge`).


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.release.com/release.ai/release.ai-gpu-resources.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.