Deploy CosmicAC

Deploy the CosmicAC Docker Compose stack on your host machine. After deployment, CosmicAC connects to your GPU Kubernetes cluster. For the services in the stack and how they interact, see Deployment architecture.

Prerequisites

The recommended host is Ubuntu 22.04 or 24.04 on an x86_64 CPU. Before you start, make sure that you have the following.

Docker Engine and Docker Compose v2.
Task.
Node.js.
The jq and kubectl command-line tools.
Access to the private CosmicAC deployment repository, which contains the Compose files and deployment scripts. To get access, contact the CosmicAC team.
GitHub Container Registry (GHCR) credentials for the private CosmicAC images in ghcr.io/tetherto.
- Your GitHub username.
- A classic GitHub personal access token with the read:packages scope. If the tetherto organization enforces single sign-on (SSO), approve the token for the organization. See Managing your personal access tokens.
A GPU Kubernetes cluster that meets the requirements. CosmicAC connects to the cluster but doesn't create it.
A kubeconfig for the cluster.

Steps

Verify the prerequisites

Enable Docker, and check that each required tool is installed.

sudo systemctl enable --now docker
docker compose version
task --version
jq --version
node --version
kubectl version --client

Set up the deployment

Clone the deployment repository and change to its directory.

git clone <deployment-repo-url>
cd <deployment-repo>

Create the .env file from the example file.

cp .env.example .env

In .env, set the variables that are required before the first deployment. For every supported variable, see Deployment configuration.

If GITHUB_PAT and GITHUB_USER aren't set in .env, bootstrap prompts for them before it pulls the private images from GHCR.

Add the kubeconfig

Get the kubeconfig from your cluster administrator. The kubeconfig must meet the kubeconfig requirements.

Create a file on the host, paste the kubeconfig into it, and save the file.

nano ~/kubeconfig

Get the absolute path of the file.

realpath ~/kubeconfig

In .env, set KUBECONFIG_SRC to that path.

KUBECONFIG_SRC=/home/<user>/kubeconfig

Check that the file isn't empty and that the kubeconfig reaches your cluster.

test -s ~/kubeconfig && echo ok
kubectl --kubeconfig ~/kubeconfig config current-context
kubectl --kubeconfig ~/kubeconfig cluster-info

Run the bootstrap

The task bootstrap command deploys the whole stack with the TAG value in .env. For what bootstrap runs, see Task deployment commands.

Get the path of the deployment directory, and copy it.

pwd

Open a root shell.

sudo -i

A root shell starts in the root home directory. Change back to the deployment directory.

cd <deployment-directory>

Run the bootstrap.

task bootstrap

Run later commands as root

Run all later task commands as root. Bootstrap and the running services create the deployment's configuration and state files as root, so commands such as task backup and task update fail for other users.

Verify the deployment

Check the services and the API.

task ps
curl -s4 -o /dev/null -w '%{http_code}\n' http://127.0.0.1:5173/
curl -s4 "http://127.0.0.1:5173/api/auth/servers?overwrite_cache=true"
curl -s4 "http://127.0.0.1:5173/api/auth/jobs?page=1&pageSize=10"

These requests don't need a token, because a default deployment has authentication turned off.

The deployment is ready when task ps shows every service as Up and the web interface returns HTTP 200. The servers request lists your GPUs. On a new deployment, the jobs request returns an empty list.

Open the web interface

In your browser, go to http://<server-ip>:5173. If you changed UI_PORT, use that port instead.

A default deployment has authentication turned off, so anyone who can reach this port has full access without signing in. Restrict network access to the deployment. For the authentication settings, see Deployment configuration.

Help and troubleshooting

Bootstrap fails with rm: cannot remove ...: Permission denied

Bootstrap resets the HyperMQ stores in ./services/*/store before it deploys. The stack's containers create those files as root, so a non-root user can't delete them.

To fix the error, run bootstrap again from a root shell in the deployment directory.

sudo -i
cd <deployment-directory>
task bootstrap

Bootstrap fails with credentials still cannot access or error from registry: denied

The token in .env can't read the CosmicAC images. Before bootstrap pulls the images, it logs in again with the values in .env, so you don't need to remove earlier Docker credentials. For more information, see task ensure-login.

To fix the error, do the following as root in the deployment directory.

In .env, check that GITHUB_PAT is a classic personal access token that meets the following conditions.
- It has the read:packages scope.
- It hasn't expired.
- It's authorized for the tetherto organization, if the organization enforces single sign-on (SSO).
In .env, check that TAG names a release that exists. A misspelled tag fails with the same error.
Run bootstrap again.
```
task bootstrap
```
Bootstrap logs in with the corrected values and pulls the images.

Bootstrap fails with docker login failed for ghcr.io

GHCR rejected the GITHUB_USER or GITHUB_PAT value in .env.

To fix the error, do the following as root in the deployment directory.

In .env, correct GITHUB_USER and GITHUB_PAT.
Run bootstrap again.
```
task bootstrap
```
Bootstrap logs in with the corrected values and pulls the images.

If the login still fails with correct values, open the Docker configuration file, ~/.docker/config.json by default, and look for a credsStore or credHelpers entry. Either entry makes Docker store credentials in a separate helper program, and a missing or broken helper makes every login fail.

Next steps

After you deploy CosmicAC, add a model master for each model that you want to serve. Then install the CLI and create your first job.

Deploy CosmicAC

On this page