Agent installation

The unified TensorCost agent runs inside your compute footprint and reports GPU + workload metrics back to TensorCost. This page is the customer-side install playbook.

If a step here doesn’t match what you see in the console, or a command fails in a way this page doesn’t cover, page support — that’s a documentation bug we want to fix.

What the agent does

Detects EC2 / EKS / GKE / AKS / bare-metal context automatically via IMDSv2 (where available).
Reports GPU health, MIG topology, NVML samples, container/pod context, and Spot interruption events back to TensorCost.
Talks to TensorCost over plain HTTPS by default. A long-lived gRPC stream (TCP/50051, TLS-terminated) is available as an opt-in transport for lower-latency delivery and inbound commands — see transport modes below.
Receives commands (start/stop/scale, drain, run-policy) only when both the gRPC transport is enabled and the tenant admin has explicitly granted the agent execution scope.

Requirements

Item	Detail
Outbound network	HTTPS (443) to your TensorCost API host by default. If you enable the gRPC transport, also allow TCP/50051 outbound to the regional gRPC endpoint the console shows you.
Privileges	Read-only NVML access (host `/usr/lib/x86_64-linux-gnu/nvidia` or `--gpus all` in Docker); container or pod metadata; no write privileges required
Cloud IAM	Not required for the agent itself. Read-only cost/billing access (Bedrock, CUR, Cost Explorer) is a separate connection, configured under Integrations, not part of the agent install.
Container runtime	Docker or containerd (the published image); a systemd/bare-metal path is on our roadmap but isn’t documented here yet — ask support if Docker isn’t an option for you.
Architecture	linux/amd64 and linux/arm64 images published

Get your credentials

The agent authenticates with per-agent credentials minted from the console — there’s no CloudFormation stack or IAM role involved in the agent install itself (that pattern is used for the separate Bedrock connection, not for the GPU agent).

Open the agent page

In the console, go to Integrations → Agents → Add agent.

Pick a hostname and mint credentials

Name the agent (typically the host or node name) and confirm. The console mints a tenant ID, agent hostname, key ID, API key, and HMAC signing value.

The API key and HMAC value are shown once. Copy the generated install command before closing the dialog — TensorCost stores only a hash and cannot show you the plaintext again. If you lose it, rotate the credential from the same page.

Copy the install command

The console shows a ready-to-run command with your credentials already filled in. Copy it — that’s everything you need for the next step.

Install

Docker (recommended)

The agent image is published publicly on Amazon ECR Public — no login required to pull it.

docker run -d \
  --name tensorcost-agent \
  --restart=unless-stopped \
  -e BACKEND_API_URL='https://api.tensorcost.com' \
  -e TENANT_ID='<from the console>' \
  -e AGENT_HOSTNAME='<from the console>' \
  -e AGENT_KEY_ID='<from the console>' \
  -e AGENT_API_KEY='<from the console>' \
  -e AGENT_HMAC_PEPPER='<from the console>' \
  public.ecr.aws/g2c6m0v1/agent:latest

For GPU access, add --gpus all (or mount the NVIDIA driver libraries read-only if you’re not using the NVIDIA Container Toolkit).

The image is currently published under the latest tag only — there isn’t a pinned semver tag to reference yet. Re-pulling latest and restarting the container is today’s upgrade path; see upgrades.

Kubernetes

There’s no published Helm chart yet — a helm repo add for TensorCost doesn’t resolve today. Until one ships, run the same public image as a DaemonSet using your own manifest, mounting the credentials from the console as a Secret and setting the same environment variables as the Docker example above. Ask support if you’d like a starting-point manifest.

Terraform

Not currently offered. If Terraform-managed rollout matters to your team, tell us — it’ll inform whether we build and publish a module.

Transport modes

By default the agent talks to TensorCost over HTTPS, authenticated with AGENT_API_KEY. This works through standard corporate egress rules and is the simplest option. Setting COMM_MODE=grpc (or both, to run HTTPS and gRPC side by side) additionally opens a long-lived gRPC stream to a regional endpoint (shown in the console when you enable it) on TCP/50051, authenticated by an HMAC-SHA256-signed AgentHello built from AGENT_KEY_ID and AGENT_HMAC_PEPPER. gRPC is what enables inbound commands (pause/resize/drain) and is lower-latency, but it needs an extra egress rule most HTTPS-only environments don’t have open by default. Credential rotation today is manual and customer-driven: rotate from Integrations → Agents, then update the running container’s environment and restart it. There’s no fixed rotation schedule enforced automatically yet. For the full ingress design, see agent ingress in the SOC 2 readiness guide.

Verification — did the agent connect?

Open Agents

Integrations → Agents. Your agent should appear with a recent last-seen timestamp shortly after startup.

Confirm metrics flow

Sidebar → GPU fleet. GPU utilization, memory, and temperature should populate within a few minutes.

Check the container logs

docker logs -f tensorcost-agent

Startup logs report which monitors initialized (NVML, cloud discovery, gRPC if enabled) and log a warning naming anything that failed to start — that’s the fastest way to tell “not connecting” from “connected but nothing to report.”

Configuration reference

The agent reads environment variables. These are the ones you’re most likely to need beyond what the console gives you:

Core

Env var	Default	Purpose
`BACKEND_API_URL`	(required)	Your TensorCost API host
`TENANT_ID`	(required)	Your tenant ID
`AGENT_API_KEY`	(required)	The console-minted API key
`AGENT_HOSTNAME`	hostname	Human-readable name in the Agents view
`AGENT_INSTANCE_ID`	auto-generated	Stable per-host identity across restarts
`COMM_MODE`	`http`	`http`, `grpc`, or `both`
`MONITORING_INTERVAL`	`300`	Seconds between sync cycles
`LOG_LEVEL`	`INFO`	`DEBUG`, `INFO`, `WARNING`, `ERROR`

gRPC transport (only when `COMM_MODE=grpc` or `both`)

Env var	Purpose
`GRPC_TARGET`	Regional gRPC endpoint, shown in the console when you enable this transport
`AGENT_KEY_ID`	Console-minted key ID
`AGENT_HMAC_PEPPER`	Console-minted HMAC signing value

NVML / GPU sampling

Env var	Default	Purpose
`NVML_ENABLED`	`false`	Enable local GPU sampling — set `true` on GPU hosts
`NVML_SAMPLE_INTERVAL`	`10`	Sampling cadence, seconds
`NVML_IDLE_GPU_THRESHOLD`	`10`	Utilization% below this counts as idle
`NVML_IDLE_DURATION`	`120`	Seconds idle before flagging

Cloud discovery (optional)

Enable the clouds you want the agent to tag instances against. On EC2 with an instance profile, IMDS supplies most of this automatically.

Env var	Purpose
`AWS_ENABLED` / `AWS_REGION`	Enable AWS discovery / override region
`AZURE_ENABLED` / `AZURE_SUBSCRIPTION_ID`	Enable Azure discovery
`GCP_ENABLED` / `GCP_PROJECT_ID`	Enable GCP discovery
`K8S_ENABLED`	Enable Kubernetes pod/namespace tagging

Upgrades

There’s no versioned release channel today — the published image is public.ecr.aws/g2c6m0v1/agent:latest. To upgrade:

docker pull public.ecr.aws/g2c6m0v1/agent:latest
docker stop tensorcost-agent && docker rm tensorcost-agent
# re-run the docker run command from Install, above

We’ll document a pinned-tag upgrade path once semver releases ship. Critical security patches are announced via your tenant’s notification channel.

Common day-1 failures

Symptom	Likely cause	Remediation
Agent never appears in Integrations → Agents	Container isn’t running, or `BACKEND_API_URL` / `TENANT_ID` / `AGENT_API_KEY` are missing or mistyped	`docker logs tensorcost-agent`; re-check the env vars against what the console generated
`agent gRPC handshake: HMAC mismatch` (`COMM_MODE=grpc`/`both` only)	Pasted `AGENT_HMAC_PEPPER` or `AGENT_KEY_ID` with leading/trailing whitespace	Re-copy from the console; rotate if you’re not sure which value is current
Agent shows connected but with stale last-seen	Outbound HTTPS (and TCP/50051, if gRPC is enabled) blocked by a firewall or corporate proxy	Allowlist the relevant egress; confirm with `curl -vI <BACKEND_API_URL>` from the host
`NVML not available` in logs	NVIDIA drivers missing, or the container lacks `--gpus all` / a device mount	Add `--gpus all`, or mount `/usr/lib/x86_64-linux-gnu/nvidia` read-only; on Kubernetes ensure the node has the NVIDIA device plugin
Metrics flow but look wrong (0 GPUs, wrong region)	`NVML_ENABLED` not set, or cloud discovery flags (`AWS_ENABLED` etc.) don’t match where the host actually runs	Set the flags for your actual environment

When stuck, page support via your shared Slack Connect channel, or email support@tensorcost.com with the container logs.

Data the agent collects

Class	Cadence	Examples
Inventory	Periodic	Instance ID, type, region, GPU model, MIG profile, tags
GPU metrics	10s NVML sample by default, batched	Utilization, memory, temperature, power, clock, ECC errors
Container/pod context	On change	Namespace, pod name, deployment, requested vs allocated GPU
Spot signals	On event	Interruption notice, hibernation, capacity rebalance
Health	Periodic	Uptime, queue depth, error counts

Outbound traffic is TLS-encrypted. No customer data is written to disk beyond the agent’s own rotating local log file.

Getting Started

Architecture

Setup

Cloud cost accounts

Features

FinOps Console

Reference

Agent installation

Agent installation

What the agent does

Requirements

Get your credentials

Install

Docker (recommended)

Kubernetes

Terraform

Transport modes

Verification — did the agent connect?

Configuration reference

Core

gRPC transport (only when `COMM_MODE=grpc` or `both`)

NVML / GPU sampling

Cloud discovery (optional)

Upgrades

Common day-1 failures

Data the agent collects

​Agent installation

​What the agent does

​Requirements

​Get your credentials

​Install

​Docker (recommended)

​Kubernetes

​Terraform

​Transport modes

​Verification — did the agent connect?

​Configuration reference

​Core

​gRPC transport (only when COMM_MODE=grpc or both)

​NVML / GPU sampling

​Cloud discovery (optional)

​Upgrades

​Common day-1 failures

​Data the agent collects

Agent installation

What the agent does

Requirements

Get your credentials

Install

Docker (recommended)

Kubernetes

Terraform

Transport modes

Verification — did the agent connect?

Configuration reference

Core

gRPC transport (only when `COMM_MODE=grpc` or `both`)

NVML / GPU sampling

Cloud discovery (optional)

Upgrades

Common day-1 failures

Data the agent collects