Ingest network security group (NSG) flow logs from Microsoft Azure Network Watcher for use in Cortex XDR network stories.
Note
Ingesting Logs from Azure Event Hub requires a Cortex XDR Pro per GB license.
To receive network security group (NSG) flow logs from Azure Network Watcher, you must configure data collection from Microsoft Azure Network Watcher using an Azure Function provided by Cortex XDR. This Azure Function requires a token that is generated when you configure your Azure Network Watcher Collector in the Collection Integration settings in Cortex XDR . After you set up data collection, Cortex XDR begins receiving new logs and data from the source.
When Cortex XDR begins receiving logs, the app creates a new dataset (MSFT_Azure_raw) that you can use to initiate XQL Search queries. For example queries, refer to the in-app XQL Library. For enhanced cloud protection, you can also configure Cortex XDR to ingest network flow logs as Cortex XDR network connection stories, which you can query with XQL Search using the xdr_dataset dataset with the preset called network_story. Cortex XDR can also raise Cortex XDR alerts (Analytics, Correlation Rules, IOC, and BIOC) when relevant from Azure Network Watcher flow logs. While Correlation Rules alerts are raised on non-normalized and normalized logs, Analytics, IOC, and BIOC alerts are only raised on normalized logs.
Enhanced cloud protection provides:
Normalization of cloud logs
Cloud logs stitching
Enrichment with cloud data
Detection based on cloud analytics
Cloud-tailored investigations
Be sure you do the following tasks before you begin configuring data collection from Azure Network Watcher.
Ensure that your NSG flow logs in Azure Network Watcher, conform to the requirements as outlined in the Microsoft documentation. For more information, see Introduction to flow logging for network security groups.
Configure the Azure Network Watcher collection in Cortex XDR.
Configure the Azure Network Watcher collection in Cortex XDR.
Select → → → .
In the Azure Network Watcher configuration, click Add Instance to begin a new configuration.
Set these parameters.
Name—Specify a descriptive name for your log collection configuration.
Normalize and enrich flow logs—(Optional) For enhanced cloud protection, you can Normalize and enrich flow logs by selecting the checkbox. If selected, Cortex XDR ingests network flow logs as Cortex XDR network connection stories, which you can query with XQL Search using the
xdr_datasetdataset with the preset callednetwork_story.
Save & Generate Token. The token is displayed in a blue box, which is blurred out in the image below.
Click the copy icon next to the key and record it somewhere safe. You will need to provide this key when you configure the Azure Function and set the XDR Token value. If you forget to record the key and close the window, you will need to generate a new key and repeat this process. When you are finished, click Done to close the window.
In the Integrations page for the Azure Network Watch Collector that you created, select Copy api url and record it somewhere safe. you will need to provide this URL when you configure the Azure Function and set the XDR Host value.
Configure the Azure Function provided by Cortex XDR.
Open the Azure Function provided by Cortex XDR.
Click Deploy to Azure.
Set these parameters, where some fields are mandatory to set and others are already populated for you.
Subscription—Specify the Azure subscription that you want to use for the App Configuration. If your account has only one subscription, it is automatically selected.
Resource group—Specify or create a resource group for your App Configuration store resource.
Region—Specify the Azure region that you want to use.
App Name—Specify the name of the function app. In the Azure Portal, this will be the name that appears in the list of resources.
App Service Plan—Select the applicable service plan. If you select Service Plan (default), an App Service plan is created and you are billed accordingly. If you select Consumption, you are billed based on the Consumption Plan.
App Service Plan Tier—When setting a Service Plan, you must select the applicable App Service Plan Tier from the list of options (Free (default), Shared, Basic, Standard, Premium, and PremiumV2). Otherwise, leave the default option configured.
App Service Plan Name—When setting a Service Plan, you must set the App Service Plan Name, which must match the Service Plan Tier.
App Service Plan Capacity—When setting a Service Plan, specify how many instances do you want to set for the upper limit or leave the default as 2. For example, when configuring an Standard Tier Service Plan, S2, set a value from 1 to 10.
Github Repo URL—Specify the URL of the repo that contains the function app source. Leave the default as https://github.com/PaloAltoNetworks/AzureNetworkWatcherNSGFlowLogsConnector.git or specify your fork's address here.
Github Repo Branch—Specify the name of the branch containing the code you want to deploy. Leave the default as master or specify the applicable branch.
Nsg Source Data Connection—Specify your storage account connection string for your Azure Network Watcher.
From the Microsoft Azure Console, open the Storage accounts page, and select the storage account that contains the connection string for the Azure Network Watcher you have configured for data collection by Cortex XDR .
Select → , and click Show keys.
Copy the applicable Connection string and paste it in the Nsg Source Data Connection field.
Output Binding—Select where you want to send you logs to either xdr (default) or eventhub.
XDR Host—Specify the API URL that you recorded when you configured the Azure Network Watcher collection in Cortex XDR.
XDR Token—Specify the token you received.
Click Review + Create to confirm your settings for the Azure Function.
Click Create. It can take a few minutes for the deployment to complete.
Once events start to come in, a green check mark appears underneath the Azure Network Watcher configuration that you created in Cortex XDR with the amount of data received.