NSX-T Traceflow and Port Mirroring: Deep Dive for Troubleshooters

Introduction

Troubleshooting in NSX-T Data Center 4.x is a core skill for modern network engineers and architects. This guide explores two essential tools—Traceflow and Port Mirroring—that give you unparalleled insight into virtual and physical network paths, firewall policy effects, and traffic analysis.

You’ll get real-world troubleshooting scenarios, step-by-step technical walkthroughs, actionable code, and diagrams. All code is tested for NSX-T 4.x, and both UI and automation are covered.

---

1. NSX-T Traceflow

How Traceflow Works

Traceflow lets you inject and trace synthetic packets through the NSX-T fabric, visualizing every hop—distributed firewall, logical switches, Tier-0/1 routers, edge nodes—and highlighting where packets are delivered or dropped.

Use Cases:

• Validate firewall policy behavior
• Troubleshoot VM-to-VM connectivity (overlay and VLAN)
• Diagnose misconfigurations or routing issues

---

Traceflow in Single-Site Topology

Diagram

---

Traceflow via NSX Manager UI

1. Navigate to Networking > Tools > Traceflow.
2. Click Start New Traceflow.
3. Select the Source and Destination (VM, interface, or IP).
4. Set protocol, port, and options.
5. Click Start and analyze the displayed path and inspection results for drops or routing issues.

---

Traceflow via REST API

Python Example:

import requests
from requests.auth import HTTPBasicAuth

nsx_manager = "https://nsx-manager.local"
username = "admin"
password = "yourpassword"

traceflow_url = f"{nsx_manager}/api/v1/traceflows"

payload = {
    "resource_type": "TraceflowConfig",
    "source": {"vm_id": "vm-123"},
    "destination": {"vm_id": "vm-456"},
    "packet_type": "IPv4",
    "tcp_flags": "SYN",
    "destination_port": 80
}

resp = requests.post(
    traceflow_url,
    auth=HTTPBasicAuth(username, password),
    json=payload,
    verify=False
)
print(resp.json())

Use the returned traceflow ID to poll status and fetch hop-by-hop results.

---

PowerShell Example

Connect-NsxtServer -Server nsx-manager.local -User admin -Password 'yourpassword'
$Traceflow = Start-NsxtTraceflow -SourceVMId 'vm-123' -DestinationVMId 'vm-456' -Protocol 'TCP' -Port 80
$Traceflow | Get-NsxtTraceflowResult

---

Real-World Traceflow: Multi-Site with Edge

Diagram

---

Interpreting Traceflow Results

• Green check: Packet delivered, all hops successful
• Red X: Drop detected, with reason (firewall rule, routing loop, interface down)
• Hop-by-hop: DFW, logical switch, router, edge, and uplink actions
• Common Issues:
  • Misapplied DFW rules
  • Incorrect overlay-to-physical mapping
  • Edge node routing or NAT issues

---

2. NSX-T Port Mirroring

Overview

Port mirroring copies selected network traffic to an analysis destination, typically a packet analyzer VM.

Types:

• Local (same host)
• Remote (different hosts or sites)
• Logical Switch (virtual switch level)
• Distributed (across fabric via overlay)

---

Port Mirroring Topology Example

Diagram

---

Setting Up Port Mirroring

1. Go to Networking > Port Mirroring.
2. Add a new Port Mirroring Session.
3. Define name, source, destination (Analyzer VM), session type (local or remote).
4. Apply the session and validate mirrored traffic at the Analyzer VM.

---

Automation Examples

REST API:

curl -k -u admin:yourpassword -X POST "https://nsx-manager.local/api/v1/port-mirroring-sessions" -H "Content-Type: application/json" -d '{
  "display_name": "TestMirror",
  "source": {"logical_port_id": "lp-123"},
  "destination": {"logical_port_id": "lp-321"},
  "mirror_type": "REMOTE"
}'

PowerShell:

Connect-NsxtServer -Server nsx-manager.local -User admin -Password 'yourpassword'
New-NsxtPortMirroringSession -DisplayName "TestMirror" -SourceLogicalPortId "lp-123" -DestinationLogicalPortId "lp-321" -MirrorType "REMOTE"

---

Real-World Scenario

Set up remote port mirroring from a production VM segment to an Analyzer VM. Use Wireshark or tcpdump to inspect mirrored packets, analyze for packet loss, retransmissions, or traffic anomalies.

---

3. Use Cases and Troubleshooting

Use Case	Traceflow	Port Mirroring
Validate firewall policy	Excellent	Good
Packet loss diagnosis	Excellent	Excellent
Deep payload analysis	Limited	Full
Latency/jitter troubleshooting	Good	Excellent
Automated CI/CD test validation	Excellent	Good

---

Example: Troubleshooting Inter-Site Latency

• Use Traceflow to confirm correct routing and policy between sites.
• Use Port Mirroring to capture actual packets and diagnose network delay, loss, or retransmissions.

---

4. Best Practices, Limitations, and Gotchas

• Traceflow is synthetic and cannot analyze actual production payloads.
• Port Mirroring impacts resource usage, especially at scale.
• Analyzer VMs should be isolated and protected.
• Automate repeated analysis tasks with API/CLI tools.
• Validate edge routing and VLAN mapping for multi-site scenarios.

---

5. Conclusion

Traceflow and Port Mirroring are essential for root cause analysis in NSX-T 4.x. Together, they offer comprehensive visibility, Traceflow for hop-by-hop analysis, Port Mirroring for packet-level forensic investigation.

---

Disclaimer

The views expressed in this article are those of the author and do not represent the opinions of VMware, my employer, or any affiliated organization. Always refer to the official VMware documentation before production deployment.