If you manage VMware infrastructure, you already know the pain. A simple question like “are there any old snapshots?” shouldn’t require opening three consoles, running PowerCLI scripts, and correlating information across different tools. But it does — every single time.
I wanted to change that. So I built a working lab integration between Microsoft Copilot Studio and my on-premises vCenter, allowing me to ask infrastructure questions in plain English and get real answers from live data.
This post walks through how I did it, what worked, what didn’t, and what I learned along the way.
🔗 Source code: github.com/vidingeb/operational-intelligence
The problem I was trying to solve
My day-to-day operational questions are pretty straightforward:
- Which workloads are running where?
- Is this cluster healthy?
- Are there operational risks right now?
- Do we have old snapshots?
- Are VMware Tools outdated?
- Which hosts need remediation?
Simple questions — but getting answers means navigating the vSphere Client, maybe jumping into Aria Operations, checking a PowerCLI script output, or asking a colleague who happens to know where that information lives. It’s slow, inconsistent, and creates knowledge silos where only certain people can answer certain questions.
I figured: what if I could just ask the infrastructure directly?
The idea: intent-centric operations
The concept is simple. Instead of me navigating to the right tool and knowing the right clicks or commands, I type what I want to know:
“How many VMs are running on this cluster?”
“Which host is carrying the highest load?”
“Do we have snapshots older than 30 days?”
“Give me a health summary of this environment.”
The AI figures out which API to call, gets the data, and summarizes the answer conversationally. I’m shifting from tool-centric operations (where I need to know how to get the answer) to intent-centric operations (where I just express what I want to know).
How the architecture works
There are five components that make this work, and getting them to talk to each other cleanly was the real challenge:
Microsoft Copilot Studio sits at the top as the conversational interface. It handles understanding what I’m asking, picking the right action to take, and summarizing the response. This is where the AI reasoning happens.
A Custom Connector defines the available actions in a format Copilot Studio understands. It’s essentially an OpenAPI spec that maps my API endpoints to tools the AI can invoke. One important lesson here: Swagger 2.0 was the only format that worked reliably. I initially tried OpenAPI 3.x, but it caused parsing errors and rendering issues in Copilot Studio. Switching to Swagger 2.0 fixed everything immediately.
The On-Premises Data Gateway bridges the gap between Microsoft’s cloud services and my local lab. This is the magic piece that lets a cloud AI service reach a server running on my local network without exposing anything to the internet.
A Python FastAPI service runs locally and acts as the middleware between the connector and VMware. It handles authentication to vCenter, exposes clean REST endpoints, and translates requests into pyVmomi calls.
vCenter via pyVmomi is the source of truth. Every answer ultimately comes from the VMware API.
The request flow
When I ask a question, here’s what happens:
| |
The whole round-trip typically takes 2–5 seconds depending on the complexity of the vCenter query.
Building it: what actually happened
Step 1 — The VMware API layers
I didn’t stop at vCenter. I ended up building three separate API modules, each wrapping a different VMware platform:
vCenter API — The core operational layer using pyVmomi:
| |
VCF Operations API — For platform-level health, capacity, and lifecycle insights from VMware Cloud Foundation’s operations manager. 29 operations covering alerts, symptoms, notifications, compliance, and more.
VCF Network Insight API — For network visibility, flow analysis, and micro-segmentation context. 18 operations covering entity search, NSX segments, Tier-1 routers, alerts, and infrastructure nodes.
In total, that’s 77 operations across three connectors — all exposed as tools the AI can invoke conversationally.
The pattern is the same for all three: Python code defines the API calls, you map them in a Swagger spec, and then enable them as tools in your Copilot Studio agent. Each module runs as its own FastAPI service on a separate port (8080, 8081, 8082) with its own connector definition.
The response structures are kept flat and descriptive across all three — the AI does a much better job summarizing when it doesn’t have to parse deeply nested objects.
Step 2 — Authentication and credentials
The API services need credentials to talk to vCenter, VCF Operations, and Network Insight. These are stored as environment variables on the server — never hardcoded in the Python files. This keeps secrets out of version control.
On Windows (PowerShell as admin), set them as machine-level variables so they persist across reboots:
| |
After setting these, restart your PowerShell session (or the services) for them to take effect. The Python code reads them at startup via os.getenv().
Step 3 — Testing locally
Before connecting anything to the cloud, I validated every endpoint locally:
| |
This seems obvious, but it saved me hours of debugging later. When something doesn’t work through the gateway, you want to be 100% sure the local API is solid.
Step 4 — Hybrid connectivity
The On-Premises Data Gateway installation was straightforward — Azure identity registration, gateway setup, recovery key, region selection. Microsoft’s docs cover this well.
Step 5 — The connector (where I hit walls)
This is where I spent the most time debugging. Two lessons worth sharing:
Host resolution caught me off guard. I initially configured the connector to point to map:8080 (my machine’s hostname). It didn’t work. The fix was using localhost:8080 instead — because the gateway runs on the same machine as my FastAPI service, so from its perspective, it’s connecting locally. This seems obvious in hindsight, but it wasn’t documented clearly anywhere.
The OpenAPI version matters more than you’d expect. I started with a proper OpenAPI 3.x spec because, well, it’s the current standard. But Copilot Studio’s connector framework had issues parsing it — actions wouldn’t render correctly, parameters were missing, and some endpoints just didn’t appear. Downgrading to Swagger 2.0 fixed every single issue. If you’re building a custom connector for Copilot Studio today, save yourself the trouble and start with Swagger 2.0.
Step 6 — Bringing it to life in Copilot Studio
Once the connectors were working, I enabled the actions as tools inside a Copilot Studio agent. No topic authoring or rigid conversation flows — just tool definitions and the AI’s ability to match intent to action.
One thing worth calling out: enabling actions in Copilot Studio is tedious. You have to enable each action individually in the agent configuration. With 77 operations across three connectors, this was a significant time investment. There’s no “enable all” button — it’s one by one, clicking through each action, confirming it. Hopefully Microsoft improves this workflow in the future.
The deployment workflow (the ugly truth)
Here’s something that doesn’t make it into architecture diagrams: the developer experience of iterating on this setup is painful.
My lab isn’t directly reachable from my workstation. Every time I need to change or add an API endpoint, the workflow looks like this:
- Write or update the Python code locally
- Connect to my lab via VPN
- RDP into my jump server
- RDP from there into my MCP server (which has L2 connectivity to the lab)
- Manually copy the updated Python files to the MCP server
- Restart the FastAPI service
- Update the Swagger spec in my custom connector definition
- Re-test through Copilot Studio
That’s a lot of friction for what should be a simple code change. There’s no CI/CD pipeline here — it’s copy-paste through RDP sessions. If I typo something in the Swagger spec, I don’t find out until I test it through the full chain.
Update: I’ve since solved this with a git-based workflow. The MCP server now has a clone of the project’s GitHub repo, with a scheduled task that pulls every 5 minutes. My new workflow is:
- Edit Python code or Swagger specs on my Mac
git pushto GitHub- The MCP server auto-pulls within 5 minutes — code is deployed
No more RDP chain, no more copy-paste. The Swagger specs live in the same repo under a swagger/ folder, so they’re version-controlled alongside the Python code. When I update an endpoint, I change both files in the same commit — single source of truth.
The only remaining manual step is pasting an updated Swagger spec into the Copilot Studio connector when I add new endpoints. That’s a Power Platform limitation I haven’t automated yet.
What it actually looks like in practice
Here’s what surprised me most: the AI doesn’t just return raw data. It reasons about what it sees.
When I asked for a cluster health summary, it didn’t just list CPU and memory percentages. It noticed that workloads were unevenly distributed across hosts, flagged that some VMs had snapshots over 30 days old, identified outdated VMware Tools versions, and gave me an overall health assessment — all from one conversational question.
It handled inventory queries, capacity analysis, platform state checks, and hygiene reporting naturally. The responses felt like talking to a knowledgeable colleague who happens to have instant access to every vCenter metric.
Here’s what it looks like in Copilot Studio with live vCenter data:
From advisory to action
I started read-only by design — no destructive actions, just queries. That was the right approach for building confidence in the system and understanding how the AI interprets intent before giving it any real power.
But since this is my lab and not production, I eventually removed that protection and enabled write operations too. The AI can now take actions like powering VMs on/off, creating and removing snapshots, and triggering vMotion — not just answering questions about the environment.
In a real-world scenario, this is where the security conversation becomes critical. You need to think carefully about:
- Who can trigger actions? Role-based access controls so not every user of the chatbot can reboot a host.
- What requires approval? A
confirm=truepattern where the AI proposes an action and a human explicitly approves before execution. - What’s the blast radius? Distinguishing between low-risk actions (list VMs) and high-risk ones (enter maintenance mode) with different governance levels.
- Audit trail. Every action the AI takes should be logged with who asked, what was done, and when.
The write capabilities I’ve enabled in my lab:
- VM lifecycle: power on/off, reboot, graceful shutdown, snapshot create/delete
- Host operations: enter/exit maintenance mode, reboot, shutdown
- Workload mobility: vMotion, Storage vMotion
- Remediation: remove stale snapshots, upgrade VMware Tools, lifecycle actions
For production, I’d recommend the phased approach: start read-only, build trust, then gradually enable write operations behind approval gates. The technology works — the question is governance, not capability.
Where this can go
The architecture isn’t specific to vCenter. I’ve already extended it to VCF Operations and Network Insight, and the same pattern — natural language → connector → gateway → local API → infrastructure — works for anything with an API:
More VMware surfaces: VCF / SDDC Manager lifecycle, NSX policy and security, Aria Automation, vSAN Health monitoring.
Broader infrastructure: HPE iLO and Dell iDRAC for hardware management, firmware lifecycle tooling, storage platforms, backup systems.
Enterprise workflows: ServiceNow integration for approval flows, CMDB enrichment from live data, automated incident remediation triggered by conversational triage.
Fully on-premises AI: The most interesting next step might be removing the cloud dependency entirely. My FastAPI layer already does the real work — Copilot Studio is just the AI reasoning on top. Replace it with a local LLM (Llama, Mistral, or similar) running on-prem with direct tool-calling access to the same APIs, and you get a fully self-contained solution. No gateway, no connector, no cloud round-trip. I’m exploring this as a follow-up project — stay tuned.
Wrapping up
This isn’t a concept or a mockup. It’s a working implementation running against real vCenter, VCF Operations, and Network Insight infrastructure in my lab. The prototype already delivers genuine operational value — I use it regularly to check on my environment without opening a single console.
The bigger picture is what excites me: we’re at the beginning of a shift where infrastructure operations move from “know which tool to open and which buttons to click” to “just ask what you need to know.” That’s a fundamental change in how we interact with the platforms we manage.
If you’re interested in building something similar, the stack is straightforward: Python, FastAPI, pyVmomi, the On-Premises Data Gateway, and Copilot Studio. Start with read-only endpoints and a Swagger 2.0 spec, and you can have something working in a weekend.