Skip to main content

LLMBoost

"Services → LLMBoost" page shows Disconnected​

not_configured means no discovery path is set. Run mac llmboost detect on serving nodes and set the ports on the master (LLMBoost Auto-Discovery). Discovery queries /v1/models at each endpoint's origin, ignoring paths such as /api; it must return a model list, not HTML.

Kubernetes services or models are missing​

Check the connection and token permissions, and apply mac-llmboost-rbac.yaml. Namespace discovery needs cluster-wide read access to Services.

Models show but charts stay empty​

Check that the master can reach each engine's /metrics. SGLang, vLLM, and LLMBoost metrics are supported. Engines other than SGLang show fewer identity details.

Collecting a Support Bundle​

mac status -v
mac test -v
docker logs mac-api-server --tail 500
dpkg -l mango-ai-center        # RPM hosts: rpm -q mango-ai-center

Before sending it to contact@mangoboost.io, remove credentials, tokens and sensitive workload data. Include the expected and observed behavior and your discovery settings.