Logging Every Packet: NSG Flow Logs and Integration with Microsoft Sentinel

Introduction

In the evolving landscape of cloud security, network visibility is more critical than ever. As enterprises adopt hybrid and cloud-native architectures, understanding east-west and north-south traffic becomes essential for both operational insight and threat detection.

In Microsoft Azure, Network Security Group (NSG) Flow Logs offer powerful telemetry by logging metadata about network traffic traversing NSG rules. When integrated with Microsoft Sentinel, these logs unlock proactive security monitoring, anomaly detection, and compliance analytics across your cloud estate.

This article dives deep into NSG Flow Logs, how to enable them, and how to stream them to Microsoft Sentinel for actionable insights.

---

What Are NSG Flow Logs?

NSG Flow Logs are a feature of Azure Network Watcher that captures metadata on allowed and denied traffic through NSG rules. These logs are stored in Azure Storage or streamed to Log Analytics, and they play a pivotal role in identifying misconfigurations, unauthorized access attempts, and traffic patterns.

Key Metadata Captured

Field	Description
srcIp_s	Source IP address
destIp_s	Destination IP address
srcPort_s	Source port
destPort_s	Destination port
protocol_s	Transport protocol (TCP/UDP)
flowState_s	Direction and status (e.g., Initiated)
action_s	Allowed or Denied
ruleName_s	NSG rule applied
mac_s, interface_s	Network interface and NIC details

Flow Log Versions

Version	Description	Recommended
V1	Basic 5-tuple metadata per flow	No
V2	Includes rule name, byte counts, MACs	Yes

Important Notes:

• NSG Flow Logs do not capture packet payloads.
• Logs are aggregated in 5-minute intervals.
• Requires Network Watcher to be enabled in the region.

---

Enabling NSG Flow Logs

You can enable NSG Flow Logs via the Azure Portal, PowerShell, CLI, or automation templates.

Prerequisites

• NSG deployed on subnet or NIC
• Network Watcher enabled
• Azure Storage Account or Log Analytics Workspace

Azure Portal

1. Navigate to Network Watcher > NSG Flow Logs
2. Select the desired NSG
3. Enable logging and choose:
   • Flow Log version (V2 recommended)
   • Retention settings
   • Destination (Log Analytics or Storage)
4. Click Save

PowerShell Example

Set-AzNetworkWatcherConfigFlowLog -NetworkWatcherName "NW-EastUS" `
  -ResourceGroupName "RG-Network" `
  -TargetResourceId "/subscriptions/xxx/resourceGroups/xxx/providers/Microsoft.Network/networkSecurityGroups/NSG-Web" `
  -EnableFlowLog $true `
  -TrafficAnalyticsInterval 10 `
  -LogAnalyticsWorkspaceId "/subscriptions/xxx/resourceGroups/xxx/providers/Microsoft.OperationalInsights/workspaces/SentinelWorkspace" `
  -FormatVersion 2

Bicep Template Snippet

resource flowLog 'Microsoft.Network/networkWatchers/flowLogs@2022-05-01' = {
  name: 'nsgFlowLog'
  location: resourceGroup().location
  properties: {
    targetResourceId: nsg.id
    enabled: true
    format: {
      type: 'JSON'
      version: 2
    }
    storageId: storageAccount.id
    flowAnalyticsConfiguration: {
      networkWatcherFlowAnalyticsConfiguration: {
        enabled: true
        workspaceId: logAnalytics.id
        trafficAnalyticsInterval: 10
      }
    }
  }
}

---

Streaming Logs to Microsoft Sentinel

To visualize and query NSG Flow Logs within Microsoft Sentinel, configure diagnostic settings to stream logs into a Log Analytics Workspace connected to Sentinel.

Setup Overview

1. Ensure NSG Flow Logs point to the same workspace linked to Sentinel.
2. In Sentinel:
   • Go to Settings > Data Connectors
   • Find and configure NSG Flow Logs
3. Validate logs appear in AzureDiagnostics table.

Verifying Log Ingestion

Use this query in Sentinel:

AzureDiagnostics
| where Category == "NetworkSecurityGroupFlowEvent"
| limit 50

---

Analyzing Traffic in Sentinel

Common KQL Queries

Top Talkers by Source IP

AzureDiagnostics
| where Category == "NetworkSecurityGroupFlowEvent"
| summarize Count = count() by srcIp_s
| top 10 by Count

Allowed vs Denied Traffic

AzureDiagnostics
| where Category == "NetworkSecurityGroupFlowEvent"
| summarize count() by action_s

Port Scanning Detection

AzureDiagnostics
| where Category == "NetworkSecurityGroupFlowEvent"
| summarize PortCount = dcount(destPort_s) by srcIp_s
| where PortCount > 20

Workbooks and Dashboards

Use Sentinel’s built-in NSG Analytics Workbook or create custom visualizations:

• Heatmaps of denied traffic
• NSG rule effectiveness
• Inactive NSG rule detection

---

Use Cases

Use Case	Description
Lateral Movement Detection	Identify unusual internal flows not aligned with zero-trust model
Shadow IT Discovery	Discover unauthorized services or hosts
Firewall Optimization	Detect ineffective or overly permissive NSG rules
Compliance Audits	Show traffic history aligned with HIPAA, PCI, etc.
Cost-Saving	Eliminate unused rules or over-provisioned segments

---

Best Practices

• Use Version 2 of NSG Flow Logs for richer telemetry
• Centralize flow logs in a shared Log Analytics workspace
• Configure retention policies based on compliance requirements
• Integrate with Microsoft Defender for Cloud for enriched alerts
• Use KQL saved functions to reduce repetitive query overhead
• Automate deployment using Bicep or Terraform
• Set alert rules for unusual denied flows or port scans
• Build dashboards for SOC teams
• Periodically review NSG rules using flow log analysis
• Cross-reference with Threat Intelligence feeds

---

Bonus: Architecture Diagram

---

Final Thoughts

NSG Flow Logs are a foundational layer in Azure’s native network observability stack. When combined with Microsoft Sentinel, they offer unparalleled visibility into allowed and denied traffic patterns, empowering security teams to detect misconfigurations, threats, and anomalous behavior.

Proactive monitoring using KQL queries, alerts, and dashboards built on these logs can significantly reduce the mean time to detect (MTTD) and respond (MTTR) to network-based threats.

Next Steps:

• Integrate Azure Firewall logs
• Connect to Threat Intelligence (TI) feeds
• Explore custom Machine Learning detections in Sentinel

*The thoughts and opinions in this article are mine and hold no reflect on my employer*