Skip to main content

LLMBoost

Issues with the Services → LLMBoost integration.

"Services → LLMBoost" page shows Disconnected​

The link is always in the sidebar. Disconnected means MAC could not reach a serving API — no registered agent had a LLMBoost cluster on port 30080 and no endpoint override is set.

Auto-discovery probes the management UI at http://<node-ip>:30080/manage/ (derived from the :30080/api base), not the inference API. It checks /manage/ deliberately: the management UI is up whenever the cluster is, whereas /v1/models returns 503 when no models are deployed. Two consequences worth knowing:

  • A running cluster with no models deployed still shows Connected — an empty models list is not a connection fault.
  • A standalone engine that serves only the root OpenAI API (:30080/v1/models, no /manage/) is never auto-discovered, even though it is up. Configure it manually (see below).

Steps:

  • Reachability check (a reasonable smoke test, though not what auto-discovery probes): curl http://<node-ip>:30080/v1/models (or .../api/v1/models for Helm-chart deployments).
  • If the serving cluster is standalone or on a subnet the agents can't reach, set the endpoint explicitly — either paste it into the connection card at the top of the LLMBoost page (admin only, in-memory), or persist it on the master:
~$ mac env set LLMBOOST_API_ENDPOINT=http://<node-ip>:30080
~$ mac restart master

Endpoint mismatch (/api/ vs root)​

The official LLMBoost Helm chart serves the API at /api/ (e.g. http://<ip>:30080/api/v1/models). Standalone deployments serve at the root (e.g. http://<ip>:30080/v1/models). Set the endpoint accordingly:

~$ mac env set LLMBOOST_API_ENDPOINT=http://<ip>:30080/api    # Helm chart
# OR
~$ mac env set LLMBOOST_API_ENDPOINT=http://<ip>:30080        # standalone

Models show but charts stay empty​

The charts come from the serving engine's Prometheus /metrics. If the models list works but every chart is flat/empty, check that <endpoint>/metrics is reachable from the master and returns sglang:-, vllm:-, or llmboost:-prefixed metrics.

Engine-identity chips are missing​

The engine-identity chips on the serving hero (engine version, parallelism, spec-decode, KV dtype, architecture) come from SGLang's /get_server_info and /get_model_info introspection endpoints. Engines that don't implement them (e.g. vLLM) simply show fewer chips — that is expected, not a fault. The core serving metrics still render from the Prometheus /metrics scrape regardless of engine.

Collecting a Support Bundle​

When opening a ticket, please include the output of:

~$ mac status
~$ mac test sanity
~$ docker logs mac-api-server --tail 500
~$ docker logs mac-prometheus --tail 200
~$ curl -s http://localhost:29090/api/v1/targets
~$ uname -a
~$ dpkg -l mango-ai-center

Send it to contact@mangoboost.io along with a description of what you expected and what you observed.