VCF AI Atlas: Designing a Governed Private AI World on VMware Cloud Foundation 9.1

TL;DR

The VCF AI Atlas is a mental model for designing private AI as a governed set of territories rather than one oversized cluster. Training, inference, sovereign data, tenant isolation, Kubernetes services, and disaster recovery have different placement, security, capacity, latency, and operational requirements. VMware Cloud Foundation provides the infrastructure and control foundation, but the architecture only becomes useful when teams define the borders, routes, ownership, evidence, and recovery expectations for each workload class.

The practical takeaway is simple: do not begin by asking where to install an AI stack. Begin by defining how workloads are classified, where data and models are allowed to live, which services may communicate, how accelerators are allocated, what telemetry proves health, and who owns each decision. The atlas turns those questions into an operating model that can be automated, observed, audited, and tested.

Introduction

Enterprise AI architecture is often presented as a vertical stack. Hardware sits at the bottom, virtualization and Kubernetes sit above it, model services appear near the top, and governance is added as a side panel. That view is useful for identifying components, but it is weak at explaining how a real platform behaves across teams, regions, data classifications, latency requirements, failure domains, and recovery obligations.

The image behind this article uses a better metaphor. It presents VMware Cloud Foundation as an atlas containing distinct AI worlds: a continent for GPU training, cities for inference, sovereign data regions, tenant nations, Kubernetes industrial zones, and mirrored recovery environments. Around those worlds sit the controls that decide where workloads can run and the operational panels that show whether the environment remains healthy.

That metaphor is powerful because enterprise AI is fundamentally a placement and governance problem. The challenge is not only providing GPUs or deploying a model endpoint. The harder work is deciding which resources a workload may consume, what data it may reach, where its artifacts may move, which network boundaries apply, what service level it must meet, how it is recovered, and who is accountable when the assumptions change.

This article turns the atlas into a practical architecture and operating model for VMware Cloud Foundation 9.1.

The Atlas Is a Control Model, Not a Product Diagram

An atlas does not replace a bill of materials or a validated design. It provides the shared mental model that architects, platform engineers, security teams, data owners, AI engineers, and operations teams can use before they argue about individual products or cluster counts.

In this model:

  • Territories represent workload and service classes with distinct operating requirements.
  • Borders represent identity, network, data, compliance, and administrative boundaries.
  • Routes represent model movement, data access, API traffic, replication, telemetry, and recovery flows.
  • Coordinates represent the placement criteria that determine where a workload belongs.
  • Control towers represent automation, operations, policy enforcement, observability, and capacity management.
  • Owners represent the teams accountable for platform, model, data, security, application, and recovery decisions.

The image is intentionally conceptual. Names such as GPU Training Continent or Tenant Nations are not official VMware Cloud Foundation object types. The utilization percentages, model counts, availability figures, and health indicators are also illustrative. They should be replaced with measured values from the actual platform and service telemetry.

There is also a naming guardrail. The image shows VCF Operations, VCF Automation, and Aria Suite together. In a current VCF 9.1 design, use the product and service names from the current documentation and support matrix. Treat the Aria label as lineage and capability context, not as permission to mix legacy and current names without verification.

Scope, Assumptions, and Decision Criteria

The atlas assumes an enterprise private AI environment built on VMware Cloud Foundation 9.1, with virtual machine and Kubernetes workload patterns, accelerator-backed compute, model and container artifacts, software-defined networking, policy-driven automation, operations telemetry, and recovery services.

It does not assume that every AI workload must remain on premises. A hybrid design may still be the correct answer for burst capacity, specialized services, global distribution, or lower-risk experimentation. The atlas is useful because it makes those exceptions explicit instead of allowing cloud placement to happen by accident.

The design also assumes that AI workloads are not homogeneous. A nightly training job, a regulated retrieval service, an internal coding assistant, and a customer-facing inference API should not inherit the same placement, network, data, and recovery policy simply because they all use machine learning.

The main decision question is:

Which governed territory should host this AI workload, under which controls, with what evidence, and with which recovery path?

The following diagram shows the model at a glance. The important point is that the territories sit above a common foundation, while policy, automation, operations, and recovery provide the controls that make the shared substrate safe.

The Foundation Under Every Territory

The value of VMware Cloud Foundation in this model is not that every AI service becomes identical. Its value is that different workload classes can consume a consistent infrastructure and governance foundation while retaining the controls they genuinely require.

vSphere Provides the Compute Abstraction

vSphere provides the virtualized compute foundation for management components, AI service virtual machines, data services, supporting middleware, and accelerator-backed workloads. The design question is not simply whether a GPU is present. It is how that accelerator is exposed, scheduled, monitored, reserved, maintained, and recovered within the supported configuration.

GPU attachment and sharing choices affect consolidation, mobility, maintenance, failure handling, and lifecycle operations. Architects should validate the supported accelerator, server, firmware, driver, hypervisor, and guest combinations before defining a standard placement class. A conceptual atlas does not override the compatibility matrix.

vSAN Provides the Data and Artifact Substrate

AI environments create multiple storage behaviors at once. Training can generate high-throughput reads, large checkpoint writes, and temporary staging data. Inference services may depend on compact model artifacts but require predictable latency. Retrieval-augmented generation can add vector indexes, source documents, embedding pipelines, and frequent refresh activity.

vSAN should therefore be designed around workload profiles, not around a single generic capacity target. Capacity planning must include usable data, replica or erasure-coding overhead, rebuild headroom, snapshots, checkpoints, registry growth, temporary pipelines, and recovery copies. Storage policy becomes part of the workload passport.

NSX Defines Borders and Routes

NSX gives the atlas enforceable borders. It can separate management, training, inference, data services, tenant projects, registries, recovery services, and external integration paths. The purpose is not to create as many segments as possible. The purpose is to make allowed communication explicit and observable.

A mature design treats east-west policy, egress control, load balancing, service exposure, DNS dependencies, and administrative access as one connected security model. An inference endpoint that is private at the front door but can reach unrestricted external destinations is not meaningfully private.

VKS Supports Kubernetes-Based AI Services

VMware vSphere Kubernetes Service provides a Kubernetes operating model for containerized AI components and supporting services. It is a logical fit for model-serving runtimes, operators, API gateways, retrieval services, data-processing pipelines, and platform services that benefit from declarative deployment and independent scaling.

Kubernetes does not remove infrastructure concerns. Cluster topology, node placement, storage classes, network policy, registry access, secrets, upgrade sequencing, GPU operators, and workload quotas still have to align with the VCF foundation. VKS is a territory within the atlas, not a separate universe with different governance.

VCF Automation Turns Placement into a Repeatable Contract

Without automation, the atlas remains a diagram. VCF Automation can expose approved catalogs, templates, infrastructure policies, and lifecycle workflows that translate architecture decisions into repeatable consumption patterns.

A useful catalog item does more than deploy a virtual machine or cluster. It should collect the workload classification, data residency, tenant, model registry, accelerator profile, network zone, service level, backup class, recovery destination, and ownership metadata needed to place and operate the service correctly.

VCF Operations Makes the Map Observable

VCF Operations provides the health, capacity, performance, inventory, and operational context needed to manage the shared platform. AI service telemetry should be integrated with this infrastructure view rather than isolated in a separate dashboard that operators only consult after an incident.

The atlas becomes operational when a team can move from a degraded AI service to the affected model endpoint, Kubernetes workload or virtual machine, GPU, host, datastore, network path, policy change, capacity condition, and recovery state without losing context.

The Six AI Territories

The territories in the image should be treated as reusable workload profiles. Each profile has different optimization goals and different failure modes.

TerritoryPrimary objectiveDominant constraintsEvidence required
GPU Training ContinentComplete training and tuning jobs efficientlyAccelerator capacity, data locality, checkpoint throughput, batch schedulingJob duration, GPU utilization, queue time, checkpoint success, cost allocation
Inference CitiesServe model responses predictablyLatency, concurrency, availability, model version, network pathRequest latency, error rate, saturation, model version, service SLO
Sovereign Data RegionKeep regulated data and models inside approved boundariesResidency, egress, identity, encryption, audit, approved servicesPolicy state, access logs, data lineage, artifact provenance, exception record
Tenant NationsIsolate teams, applications, and business unitsIdentity scope, quotas, network policy, chargeback, noisy-neighbor controlOwnership, entitlement, quota usage, cost, policy exceptions
Kubernetes Industrial ZonesRun containerized AI platform servicesCluster lifecycle, registry, operators, storage, network policy, GitOpsDeployment state, image provenance, drift, cluster health, rollout evidence
Disaster Recovery Mirror WorldsRestore the complete AI service, not only computeDependency order, data consistency, artifact availability, identity, network, testingRPO, RTO, recovery test result, dependency validation, runbook ownership

GPU Training Continent

Training is usually the most resource-intensive territory. It may require exclusive accelerator access, large memory footprints, sustained storage throughput, high east-west data movement, and long-running jobs that need reliable checkpointing.

The placement decision should consider more than free GPU capacity. Data gravity may make a slightly smaller local accelerator pool more effective than a remote pool connected through a slower or more expensive path. The platform should also distinguish interactive experimentation from scheduled production training. Mixing both without quotas and scheduling policy creates unpredictable contention.

A production training profile should define accelerator type, sharing model, minimum memory, CPU and memory ratios, local or remote data sources, checkpoint destination, maximum run duration, preemption behavior, network class, observability, and cost owner.

Inference Cities

Inference behaves more like a digital service than a batch job. It must meet request latency, throughput, error-rate, and availability objectives while handling model loading, scaling, version promotion, rollback, and traffic management.

An inference city should separate service tiers. A customer-facing real-time endpoint may require reserved capacity, multiple failure domains, controlled rollout, and strict recovery objectives. An internal summarization service may tolerate queueing, shared accelerators, and slower restoration. Both use models, but they should not inherit the same architecture.

The service boundary matters. Infrastructure availability does not prove that the model endpoint is healthy. Synthetic requests, model-response validation, queue depth, time to first response, throughput, saturation, and dependency health must be measured at the service level.

Sovereign Data Region

A sovereign region is defined by enforceable constraints, not by a label on a cluster. It must specify where data is stored, processed, backed up, replicated, and observed; which identities can access it; which model artifacts are permitted; and which external services are prohibited or allowed through an exception.

Sovereignty also applies to operational evidence. Logs, traces, prompts, outputs, embeddings, backup copies, support bundles, and model evaluation data may carry sensitive information. A design that protects the primary data store but exports detailed telemetry to an unapproved destination has an incomplete boundary.

The architecture should document the exact regulatory, contractual, or organizational requirement being addressed. “Sovereign” is too broad to serve as a technical control by itself.

Tenant Nations

Tenant isolation converts a shared AI platform into a service that multiple teams can trust. Tenancy may align to business units, application portfolios, regulated domains, projects, development stages, or customer environments.

Each tenant needs explicit ownership, identity boundaries, quotas, accelerator entitlements, network segments, namespace or project structure, registry scope, data permissions, cost allocation, and exception handling. Isolation should be strong enough to protect workloads while avoiding a separate snowflake platform for every team.

The most common failure is confusing logical organization with security isolation. A folder, tag, namespace, or dashboard view may help administration but may not create an enforceable trust boundary. The architecture must state which control performs the isolation.

Kubernetes Industrial Zones

Kubernetes zones host the repeatable factories that build, deploy, expose, and update AI services. These zones may include model-serving runtimes, retrieval services, data preparation, evaluation pipelines, agent services, API gateways, and supporting observability components.

The platform team should define approved cluster classes rather than allowing every project to invent its own topology. A cluster class can include node pools, accelerator profiles, storage classes, registry policy, network policy, secrets integration, admission controls, logging, backup, upgrade cadence, and service exposure patterns.

The key operating principle is lifecycle consistency. A model service is not production-ready because it runs once. It must be reproducibly deployed, upgraded, observed, rolled back, recovered, and retired.

Disaster Recovery Mirror Worlds

AI disaster recovery must restore the service chain. Recovering a virtual machine while losing the model registry, vector index, prompt configuration, secrets, certificates, network policy, identity mapping, or source data does not restore the AI service.

Recovery plans should identify dependency order and consistency requirements. The plan may need to recover platform management, identity, DNS, network services, registries, model artifacts, data stores, Kubernetes clusters, virtual machines, policy definitions, and application endpoints in a controlled sequence.

RPO and RTO values should be assigned to the AI service, not copied blindly from the underlying cluster. Recovery tests should include a representative request and evidence that the correct model and data version are serving after failover.

The Coordinates That Decide Placement

The left panel in the image lists the architectural controls that turn the atlas into a decision system. These are the coordinates that should be captured during intake and enforced through policy.

CoordinateDecision questionFailure when ignored
GPU placementWhich accelerator profile and sharing model does the workload require?Capacity exists but the workload cannot meet memory, latency, isolation, or support requirements
Model residencyWhere may model artifacts be stored, loaded, cached, backed up, and replicated?Approved data stays local while model intellectual property moves outside the boundary
Data gravityWhere is the authoritative data, and what movement is technically, legally, and economically acceptable?Expensive transfers, slow pipelines, duplicate data, and stale retrieval results
Network segmentationWhich services, users, models, data stores, and external endpoints may communicate?Broad trust, uncontrolled egress, lateral movement, and difficult incident containment
Latency zoneWhat end-to-end response or pipeline latency must the service meet?Infrastructure appears healthy while the application misses its service objective
Compliance boundaryWhich obligations and evidence requirements apply to the workload?Controls are assumed rather than mapped to implementation and proof
Capacity forecastWhat is the expected growth in GPUs, memory, storage, throughput, tokens, users, and models?Scarce capacity is exhausted by successful adoption
Recovery destinationWhat must be restored, where, in what order, and within which RPO and RTO?Compute recovers but the AI service remains unusable

These coordinates should be evaluated together. A low-latency inference service may appear to belong near users, but data residency may restrict the allowed location. A training job may fit the available GPU pool, but its source data may make movement impractical. A shared accelerator profile may improve utilization, but the tenant isolation or support requirement may demand a different placement.

The atlas therefore needs a decision record, not only a scheduler. The record should preserve the workload classification, assumptions, selected territory, rejected alternatives, control requirements, owner approvals, and review trigger.

Borders, Routes, and Passports

A world map becomes useful when it shows what may cross each border. The same is true for private AI.

Identity Is the Passport

Human administrators, developers, data scientists, applications, pipelines, model services, and agents all need identities with bounded permissions. Non-human identities should not inherit broad administrator credentials simply because automation is difficult to configure.

The platform should define who may request a workload, who may approve it, who may operate it, who may access its data, who may promote a model, and who may change its network or recovery policy. Break-glass access should be explicit, logged, reviewed, and separated from normal operations.

NSX Policy Is the Border Control

Network policy should be derived from service intent. Training jobs may need data-source access but no inbound user traffic. Inference services may accept requests from approved application tiers while reaching only specific data services. Management planes should be isolated from workload networks. Recovery traffic should use defined paths and controls.

Default-deny is meaningful only when teams can understand and maintain the allowed flows. Policy ownership, naming, change review, logging, and troubleshooting are as important as the firewall rule itself.

Provenance Is the Customs Record

Models, container images, prompts, retrieval indexes, and policy bundles are software supply-chain artifacts. The platform should know where they came from, who approved them, which version is running, what data influenced them, and how to roll them back.

A model registry entry without promotion evidence is not enough. Production workflows should connect artifact identity to evaluation results, risk acceptance, deployment history, runtime telemetry, and retirement status.

The Operating Model Behind the Map

The atlas will fail if every decision is assigned to the platform team. Private AI spans infrastructure, data, models, applications, security, risk, and operations. Ownership must be distributed while controls remain consistent.

RoleAccountable decisionsRequired evidence
Platform ownerSupported service classes, capacity, lifecycle, automation, shared SLOsPlatform health, capacity forecast, compatibility baseline, change records
Model ownerModel selection, evaluation, version promotion, rollback, retirementEvaluation results, model card, approval, runtime performance
Data ownerData classification, residency, access, retention, lineageClassification, lineage, access review, retention and deletion evidence
Security and governance ownerTrust boundaries, policy, exceptions, audit, risk acceptanceControl mapping, policy state, exception record, audit evidence
Application ownerBusiness outcome, request pattern, service SLO, dependency behaviorUser-impact metrics, synthetic tests, dependency map, incident history
Operations or SRE ownerMonitoring, incident response, recovery, maintenance, capacity actionAlerts, runbooks, recovery tests, error budget, post-incident actions

A useful governance model separates standards from exceptions. Standards define the approved territories and placement classes. Exceptions document why a workload cannot use them, which compensating controls apply, who accepts the risk, and when the exception expires.

Turning the Atlas into Policy

The following YAML is an illustrative policy contract. It is not a native VMware Cloud Foundation API schema. Its purpose is to show the information a service catalog, policy engine, or orchestration workflow should collect before deployment.

apiVersion: dtd.example/v1alpha1
kind: AIPlacementPolicy
metadata:
  name: sovereign-inference-gold
spec:
  workloadClass: inference
  environment: production

  ownership:
    businessService: claims-assistance
    platformTeam: private-ai-platform
    modelOwner: applied-ai
    dataOwner: regulated-data
    operationsOwner: ai-sre

  tenancy:
    organization: regulated-services
    project: claims-ai
    costCenter: cc-4820

  placement:
    region: primary-private-region
    failureDomain: dual-zone
    runtime: vks
    acceleratorProfile: shared-gpu-inference
    capacityReservation: required

  data:
    residency: in-region
    allowedClassifications:
      - confidential
    externalEgress: deny
    backupResidency: in-region

  model:
    registry: approved-enterprise
    versionPolicy: pinned
    promotionGate: required
    rollbackVersion: previous-approved

  network:
    segment: ai-inference-production
    ingress: private
    eastWestPolicy: explicit-allow
    administration: privileged-access-zone

  observability:
    serviceSlo: inference-gold
    syntheticTest: required
    retainAuditDays: 365
    capacityForecast: enabled

  recovery:
    class: regional
    destination: secondary-private-region
    rpoMinutes: 15
    rtoMinutes: 60
    testFrequency: quarterly

  approvals:
    - model-owner
    - data-owner
    - security-governance
    - service-owner

The fields should map to actual platform constructs and workflows. The region may resolve to an approved VCF instance or workload domain. The runtime may resolve to a VKS cluster class or virtual machine blueprint. The segment may resolve to an NSX network and policy group. The accelerator profile may resolve to a supported hardware and scheduling configuration. The recovery class may trigger backup, replication, and test requirements.

Successful execution should produce more than deployed resources. It should produce a traceable record containing the request, selected placement, policy decisions, approvals, generated configuration, artifact versions, ownership metadata, and validation results.

Observability Must Describe the Service

The image shows a compelling operations center, but real observability cannot stop at green infrastructure tiles. AI services fail in ways that traditional infrastructure dashboards may not detect.

A practical telemetry model includes several layers:

Telemetry layerWhat to measure
InfrastructureHost health, CPU, memory, GPU utilization, GPU memory, storage latency, network loss, capacity headroom
PlatformVKS cluster health, node pools, operators, registries, automation failures, policy drift, certificate state
Model serviceRequest latency, time to first response, throughput, queue depth, error rate, saturation, model load time
Model and dataActive model version, evaluation status, data freshness, retrieval quality, index state, provenance
GovernanceDenied requests, policy exceptions, privileged access, egress events, unapproved artifacts, audit completeness
EconomicsAccelerator allocation, idle capacity, cost by tenant, cost by service, cost per successful request or job
RecoveryBackup success, replica lag, artifact completeness, last recovery test, measured RPO and RTO

The dashboard should help an operator decide what to do next. It should expose ownership, dependencies, recent changes, capacity risk, policy state, and recovery readiness. A wall of percentages without thresholds, service context, or action paths is decoration rather than observability.

A Practical Build Sequence

The atlas should be implemented incrementally. A large private AI platform does not need to be designed in one program before value appears.

Define the Coordinates and Owners

Begin with two or three representative workloads. Capture their data classification, model lifecycle, accelerator needs, request pattern, network flows, service objectives, recovery requirements, and owners. Use these examples to define the first placement coordinates and decision records.

The exit criterion is not a completed diagram. It is agreement on the minimum information required to approve and operate a workload.

Create a Small Set of Territory Classes

Define a limited set of supported workload classes, such as batch training, production inference, regulated retrieval, shared development, and platform services. Each class should have a documented infrastructure profile, policy profile, telemetry package, and recovery expectation.

Avoid creating a unique class for every application. The purpose is to standardize the common path while preserving a controlled exception process.

Codify the Classes in Catalogs and Policies

Translate each class into catalog items, templates, tags, naming standards, NSX policy, storage policy, VKS cluster or namespace patterns, accelerator profiles, backup requirements, and ownership metadata.

Validate that a request can be fulfilled consistently and that the resulting environment contains the evidence needed for operations and audit.

Build Service-Level Telemetry

Connect infrastructure telemetry to model-service telemetry. Define service-level indicators, thresholds, alert ownership, runbooks, and capacity triggers. Test whether operators can trace a synthetic failure from the user-facing symptom to the responsible infrastructure, platform, model, data, or network component.

Test Lifecycle and Recovery

Exercise model promotion, rollback, certificate rotation, platform maintenance, cluster upgrade, capacity exhaustion, node failure, registry outage, data unavailability, and regional recovery. Document which assumptions failed during the test and update the territory profile.

Recovery testing is especially important because AI services often depend on artifacts and data pipelines that are not visible in a conventional virtual machine recovery plan.

Scale Through Onboarding

Once the first territory classes are proven, onboard additional teams through a defined readiness process. Measure time to onboard, exception rate, policy failures, resource utilization, service reliability, and support demand.

The platform is ready to scale when repeated onboarding becomes easier without weakening governance.

Conclusion

The VCF AI Atlas is useful because it changes the design conversation. Instead of treating private AI as one stack deployed into one cluster, it frames the platform as a governed set of territories with distinct borders, routes, coordinates, owners, evidence, and recovery paths.

VMware Cloud Foundation provides the common substrate through vSphere, vSAN, NSX, VKS, automation, operations, and protection services. VMware Private AI Foundation with NVIDIA and NVIDIA AI Enterprise can extend that substrate with AI-specific capabilities. The architecture becomes defensible only when those capabilities are connected to workload classification, data and model residency, accelerator policy, service-level telemetry, lifecycle controls, and accountable ownership.

The practical next step is not to build every territory shown in the image. Select a small number of representative AI services, define their placement coordinates, establish their owners, codify the approved patterns, and prove that the platform can deploy, observe, update, contain, and recover them. Once those controls work, the atlas can expand without turning the private AI environment into an ungoverned collection of expensive experiments.

External References

Keep exploring

Choose your next step

Continue with the path that best matches the architecture or operating challenge in front of you.

Leave a Reply

Discover more from Digital Thought Disruption

Subscribe now to keep reading and get access to the full archive.

Continue reading