Examples

See also Register Each Agent, mac test, Configuring Switch Access, Default Login and password recovery.

Switch to a Private Mirror

Mirror the MAC and SDK images, run docker login as the user who pulls, then run:

sudo mac env set --global MAC_REGISTRY=<registry-host>/<mac-path>
sudo mac env set --global SDK_AGENT_IMAGE=<registry-host>/<sdk-image>:<tag>
mac pull all
sudo mac restart all

Prometheus, Loki and vendor exporter images still come from their usual registries.

Back Up Prometheus Before mac clean

umask 077
docker stop mac-prometheus
docker cp mac-prometheus:/prometheus/. - > prometheus-backup.tar
docker start mac-prometheus
tar tf prometheus-backup.tar

This holds metrics only. A full backup also needs the API data, mac-loki-data, configuration and token files. Store backups in a private directory.

Restore Prometheus

warning

These commands erase the current metrics. Validate the archive first and use them only on the default MAC volume.

If the container was removed, start the master once to create it. Then run:

docker stop mac-prometheus
docker run --rm --pull=never --user 0 -v mac-prometheus-data:/data --entrypoint sh \
  "$(docker inspect -f '{{.Image}}' mac-prometheus)" -c 'rm -rf /data/* /data/.[!.]*'
docker run --rm -i --pull=never --user 0 -v mac-prometheus-data:/data --entrypoint sh \
  "$(docker inspect -f '{{.Image}}' mac-prometheus)" -c 'tar xpf - -C /data' < prometheus-backup.tar
docker start mac-prometheus

Back Up and Restore Accounts and Enrollment

On the master, back up the API data, which includes the key that decrypts stored BMC credentials:

umask 077
docker stop mac-api-server
docker cp mac-api-server:/app/data/. - > api-data-backup.tar
docker start mac-api-server
tar tf api-data-backup.tar

warning

Restoring erases current accounts, enrollment and other API data, including changes made after the backup. Validate the archive first and use the commands only on the default MAC volume.

If the container was removed, start the master once to create it. Then run:

docker stop mac-api-server
docker run --rm --pull=never --user 0 -v mac-api-data:/app/data --entrypoint sh \
  "$(docker inspect -f '{{.Image}}' mac-api-server)" -c 'rm -rf /app/data/* /app/data/.[!.]*'
docker run --rm -i --pull=never --user 0 -v mac-api-data:/app/data --entrypoint sh \
  "$(docker inspect -f '{{.Image}}' mac-api-server)" -c 'tar xpf - -C /app/data' < api-data-backup.tar
docker start mac-api-server

Check login, enrolled nodes and saved BMC connections.

Reload Agent After a Driver Upgrade

After upgrading the GPU driver, run mac restart agent and mac test --expect agent, then check that GPU metrics appear.