The system operates as a single container exposing three interfaces—web UI, REST API, and MCP agent tools—that share validations and backend logic, enabling both agent-driven and browser-driven cluster operations without data-path involvement in training or inference jobs.

HyperPod InstantStart is an open-source control plane that merges AWS EKS orchestration with SageMaker HyperPod capabilities, as described in an AWS Machine Learning Blog post published on the same day.

Advanced features include managed training and inference operators, managed tiered checkpointing, and managed autoscaling, with explicit mappings between UI toggles and backend dependencies; tiered checkpointing relies on an identity chain comprising a Kubernetes service account, IAM role/policy, OIDC trust, and bindings.

InstantStart creates HyperPod clusters with automatic node recovery and health checks, plus GPU/Elastic Fabric Adapter stress testing and managed /20 subnets for large accelerator fleets; autoscaling is Karpenter-based under AWS management and limited to HyperPod instance groups.

Training paths include a HyperPod training operator and standard KubeRay, with a recipe layer for PyTorch scripts, LLaMA-Factory, MS-Swift, and VER RL; logs stream to the browser via WebSocket and metrics can be reported to managed MLflow on SageMaker.

The MCP server provides 38 tools covering lifecycle, instance groups, storage, inference, jobs, and node operations; mutating tools expose a status tool to ensure mutations complete before polling.

Operational boundaries emphasize read-only diagnostic capabilities, with state-changing commands treated as suggestions; security hinges on IAM/Kubernetes controls and backend validation, while the agent extends control-plane access without elevating privileges.

Deployment uses a CloudFormation template to provision the management environment, shared S3 bucket, and IAM roles; the web UI is served from the container on port 3099 and accessed via AWS Systems Manager port-forwarding.

Inference paths include a managed HyperPod inference operator with tiered KV caching and routing, plus a self-managed path for deploying a user-selected serving container (e.g., vLLM, SGLang) with multiple service shapes and a warm GPU model pool; multi-replica SGLang can employ a router with cache-aware routing and EDA autoscaling.

The MCP tools wrap the control-plane REST APIs, providing a unified validation layer for both agent and web interfaces; clusters can be configured via natural-language agent commands or through a structured web form, with the agent orchestrating steps from EKS control-plane creation to storage setup.