> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tensorcost.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Agent installation

> Install the unified TensorCost agent — IMDSv2 auto-detect, HMAC-SHA256 authentication for the streaming transport, and console-issued credentials.

# Agent installation

The unified TensorCost agent runs inside your compute footprint and reports GPU + workload metrics back to TensorCost. This page is the customer-side install playbook.

<Note>
  If a step here doesn't match what you see in the console, or a command fails in a way this page doesn't cover, page support — that's a documentation bug we want to fix.
</Note>

## What the agent does

* Detects EC2 / EKS / GKE / AKS / bare-metal context automatically via IMDSv2 (where available).
* Reports GPU health, MIG topology, NVML samples, container/pod context, and Spot interruption events back to TensorCost.
* Talks to TensorCost over plain HTTPS by default. A long-lived gRPC stream (TCP/50051, TLS-terminated) is available as an opt-in transport for lower-latency delivery and inbound commands — see [transport modes](#transport-modes) below.
* Receives commands (start/stop/scale, drain, run-policy) only when both the gRPC transport is enabled and the tenant admin has explicitly granted the agent execution scope.

## Requirements

| Item              | Detail                                                                                                                                                                                     |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Outbound network  | HTTPS (443) to your TensorCost API host by default. If you enable the gRPC transport, also allow TCP/50051 outbound to the regional gRPC endpoint the console shows you.                   |
| Privileges        | Read-only NVML access (host `/usr/lib/x86_64-linux-gnu/nvidia` or `--gpus all` in Docker); container or pod metadata; no write privileges required                                         |
| Cloud IAM         | Not required for the agent itself. Read-only cost/billing access (Bedrock, CUR, Cost Explorer) is a separate connection, configured under **Integrations**, not part of the agent install. |
| Container runtime | Docker or containerd (the published image); a systemd/bare-metal path is on our roadmap but isn't documented here yet — ask support if Docker isn't an option for you.                     |
| Architecture      | linux/amd64 and linux/arm64 images published                                                                                                                                               |

## Get your credentials

The agent authenticates with per-agent credentials minted from the console — there's no CloudFormation stack or IAM role involved in the agent install itself (that pattern is used for the separate [Bedrock connection](/bedrock-integration), not for the GPU agent).

<Steps>
  <Step title="Open the agent page">
    In the console, go to **Integrations → Agents → Add agent**.
  </Step>

  <Step title="Pick a hostname and mint credentials">
    Name the agent (typically the host or node name) and confirm. The console mints a tenant ID, agent hostname, key ID, API key, and HMAC signing value.

    <Warning>
      The API key and HMAC value are shown **once**. Copy the generated install command before closing the dialog — TensorCost stores only a hash and cannot show you the plaintext again. If you lose it, rotate the credential from the same page.
    </Warning>
  </Step>

  <Step title="Copy the install command">
    The console shows a ready-to-run command with your credentials already filled in. Copy it — that's everything you need for the next step.
  </Step>
</Steps>

## Install

### Docker (recommended)

The agent image is published publicly on Amazon ECR Public — no login required to pull it.

```bash theme={null}
docker run -d \
  --name tensorcost-agent \
  --restart=unless-stopped \
  -e BACKEND_API_URL='https://api.tensorcost.com' \
  -e TENANT_ID='<from the console>' \
  -e AGENT_HOSTNAME='<from the console>' \
  -e AGENT_KEY_ID='<from the console>' \
  -e AGENT_API_KEY='<from the console>' \
  -e AGENT_HMAC_PEPPER='<from the console>' \
  public.ecr.aws/g2c6m0v1/agent:latest
```

For GPU access, add `--gpus all` (or mount the NVIDIA driver libraries read-only if you're not using the NVIDIA Container Toolkit).

<Note>
  The image is currently published under the `latest` tag only — there isn't a pinned semver tag to reference yet. Re-pulling `latest` and restarting the container is today's upgrade path; see [upgrades](#upgrades).
</Note>

### Kubernetes

There's no published Helm chart yet — a `helm repo add` for TensorCost doesn't resolve today. Until one ships, run the same public image as a DaemonSet using your own manifest, mounting the credentials from the console as a Secret and setting the same environment variables as the Docker example above. Ask support if you'd like a starting-point manifest.

### Terraform

Not currently offered. If Terraform-managed rollout matters to your team, tell us — it'll inform whether we build and publish a module.

## Transport modes

By default the agent talks to TensorCost over HTTPS, authenticated with `AGENT_API_KEY`. This works through standard corporate egress rules and is the simplest option.

Setting `COMM_MODE=grpc` (or `both`, to run HTTPS and gRPC side by side) additionally opens a long-lived gRPC stream to a regional endpoint (shown in the console when you enable it) on TCP/50051, authenticated by an HMAC-SHA256-signed `AgentHello` built from `AGENT_KEY_ID` and `AGENT_HMAC_PEPPER`. gRPC is what enables inbound commands (pause/resize/drain) and is lower-latency, but it needs an extra egress rule most HTTPS-only environments don't have open by default.

```mermaid theme={null}
sequenceDiagram
  participant Agent
  participant NLB as Regional gRPC endpoint (TLS)
  participant GPU as TensorCost backend
  participant R as Redis

  Agent->>Agent: IMDSv2 — fetch instance ID, region, account (where available)
  Agent->>NLB: TLS handshake
  Agent->>GPU: AgentHello { tenantId, agentId, keyId, nonce, ts, hmac }
  GPU->>GPU: ±300s clock-skew check
  GPU->>R: SETNX nonce:{tenantId}:{keyId}:{nonce} TTL 600s — replay guard
  GPU->>GPU: timingSafeEqual(hmac, expected)
  GPU-->>Agent: connection accepted
```

Credential rotation today is manual and customer-driven: rotate from **Integrations → Agents**, then update the running container's environment and restart it. There's no fixed rotation schedule enforced automatically yet.

For the full ingress design, see [agent ingress](/soc2-readiness-guide#agent-ingress) in the SOC 2 readiness guide.

## Verification — did the agent connect?

<Steps>
  <Step title="Open Agents">
    **Integrations → Agents**. Your agent should appear with a recent last-seen timestamp shortly after startup.
  </Step>

  <Step title="Confirm metrics flow">
    Sidebar → **GPU fleet**. GPU utilization, memory, and temperature should populate within a few minutes.
  </Step>

  <Step title="Check the container logs">
    ```bash theme={null}
    docker logs -f tensorcost-agent
    ```

    Startup logs report which monitors initialized (NVML, cloud discovery, gRPC if enabled) and log a warning naming anything that failed to start — that's the fastest way to tell "not connecting" from "connected but nothing to report."
  </Step>
</Steps>

## Configuration reference

The agent reads environment variables. These are the ones you're most likely to need beyond what the console gives you:

### Core

| Env var               | Default        | Purpose                                  |
| --------------------- | -------------- | ---------------------------------------- |
| `BACKEND_API_URL`     | (required)     | Your TensorCost API host                 |
| `TENANT_ID`           | (required)     | Your tenant ID                           |
| `AGENT_API_KEY`       | (required)     | The console-minted API key               |
| `AGENT_HOSTNAME`      | hostname       | Human-readable name in the Agents view   |
| `AGENT_INSTANCE_ID`   | auto-generated | Stable per-host identity across restarts |
| `COMM_MODE`           | `http`         | `http`, `grpc`, or `both`                |
| `MONITORING_INTERVAL` | `300`          | Seconds between sync cycles              |
| `LOG_LEVEL`           | `INFO`         | `DEBUG`, `INFO`, `WARNING`, `ERROR`      |

### gRPC transport (only when `COMM_MODE=grpc` or `both`)

| Env var             | Purpose                                                                     |
| ------------------- | --------------------------------------------------------------------------- |
| `GRPC_TARGET`       | Regional gRPC endpoint, shown in the console when you enable this transport |
| `AGENT_KEY_ID`      | Console-minted key ID                                                       |
| `AGENT_HMAC_PEPPER` | Console-minted HMAC signing value                                           |

### NVML / GPU sampling

| Env var                   | Default | Purpose                                             |
| ------------------------- | ------- | --------------------------------------------------- |
| `NVML_ENABLED`            | `false` | Enable local GPU sampling — set `true` on GPU hosts |
| `NVML_SAMPLE_INTERVAL`    | `10`    | Sampling cadence, seconds                           |
| `NVML_IDLE_GPU_THRESHOLD` | `10`    | Utilization% below this counts as idle              |
| `NVML_IDLE_DURATION`      | `120`   | Seconds idle before flagging                        |

### Cloud discovery (optional)

Enable the clouds you want the agent to tag instances against. On EC2 with an instance profile, IMDS supplies most of this automatically.

| Env var                                   | Purpose                                 |
| ----------------------------------------- | --------------------------------------- |
| `AWS_ENABLED` / `AWS_REGION`              | Enable AWS discovery / override region  |
| `AZURE_ENABLED` / `AZURE_SUBSCRIPTION_ID` | Enable Azure discovery                  |
| `GCP_ENABLED` / `GCP_PROJECT_ID`          | Enable GCP discovery                    |
| `K8S_ENABLED`                             | Enable Kubernetes pod/namespace tagging |

## Upgrades

There's no versioned release channel today — the published image is `public.ecr.aws/g2c6m0v1/agent:latest`. To upgrade:

```bash theme={null}
docker pull public.ecr.aws/g2c6m0v1/agent:latest
docker stop tensorcost-agent && docker rm tensorcost-agent
# re-run the docker run command from Install, above
```

We'll document a pinned-tag upgrade path once semver releases ship. Critical security patches are announced via your tenant's notification channel.

## Common day-1 failures

| Symptom                                                              | Likely cause                                                                                                   | Remediation                                                                                                                         |
| -------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
| Agent never appears in **Integrations → Agents**                     | Container isn't running, or `BACKEND_API_URL` / `TENANT_ID` / `AGENT_API_KEY` are missing or mistyped          | `docker logs tensorcost-agent`; re-check the env vars against what the console generated                                            |
| `agent gRPC handshake: HMAC mismatch` (`COMM_MODE=grpc`/`both` only) | Pasted `AGENT_HMAC_PEPPER` or `AGENT_KEY_ID` with leading/trailing whitespace                                  | Re-copy from the console; rotate if you're not sure which value is current                                                          |
| Agent shows connected but with stale last-seen                       | Outbound HTTPS (and TCP/50051, if gRPC is enabled) blocked by a firewall or corporate proxy                    | Allowlist the relevant egress; confirm with `curl -vI <BACKEND_API_URL>` from the host                                              |
| `NVML not available` in logs                                         | NVIDIA drivers missing, or the container lacks `--gpus all` / a device mount                                   | Add `--gpus all`, or mount `/usr/lib/x86_64-linux-gnu/nvidia` read-only; on Kubernetes ensure the node has the NVIDIA device plugin |
| Metrics flow but look wrong (0 GPUs, wrong region)                   | `NVML_ENABLED` not set, or cloud discovery flags (`AWS_ENABLED` etc.) don't match where the host actually runs | Set the flags for your actual environment                                                                                           |

When stuck, page support via your shared Slack Connect channel, or email [support@tensorcost.com](mailto:support@tensorcost.com) with the container logs.

## Data the agent collects

| Class                 | Cadence                             | Examples                                                    |
| --------------------- | ----------------------------------- | ----------------------------------------------------------- |
| Inventory             | Periodic                            | Instance ID, type, region, GPU model, MIG profile, tags     |
| GPU metrics           | 10s NVML sample by default, batched | Utilization, memory, temperature, power, clock, ECC errors  |
| Container/pod context | On change                           | Namespace, pod name, deployment, requested vs allocated GPU |
| Spot signals          | On event                            | Interruption notice, hibernation, capacity rebalance        |
| Health                | Periodic                            | Uptime, queue depth, error counts                           |

Outbound traffic is TLS-encrypted. No customer data is written to disk beyond the agent's own rotating local log file.