Ingest network flow logs from Microsoft Azure Network Watcher - Administrator Guide - Cortex XDR - Cortex - Security Operations

Cortex XDR Documentation
Product
Cortex XDR
Creation date
2024-03-06
Last date published
2024-10-01
Category
Administrator Guide
Abstract

Ingest network security group (NSG) flow logs from Microsoft Azure Network Watcher for use in Cortex XDR network stories.

To receive network security group (NSG) flow logs from Azure Network Watcher, you must configure data collection from Microsoft Azure Network Watcher using an Azure Function provided by Cortex XDR. This Azure Function requires a token that is generated when you configure your Azure Network Watcher Collector in Cortex XDR. After you set up data collection, Cortex XDR begins receiving new logs and data from the source.

When Cortex XDR begins receiving logs, the app creates a new dataset (MSFT_Azure_raw) that you can use to initiate XQL Search queries. For example queries, refer to the in-app XQL Library. For enhanced cloud protection, you can also configure Cortex XDR to ingest network flow logs as Cortex XDR network connection stories, which you can query with XQL Search using the xdr_dataset dataset with the preset called network_story. Cortex XDR can also raise Cortex XDR alerts (Analytics, Correlation Rules, IOC, and BIOC) when relevant from Azure Network Watcher flow logs. While Correlation Rules alerts are raised on non-normalized and normalized logs, Analytics, IOC, and BIOC alerts are only raised on normalized logs.

Enhanced cloud protection provides:

  • Normalization of cloud logs

  • Cloud logs stitching

  • Enrichment with cloud data

  • Detection based on cloud analytics

  • Cloud-tailored investigations

Danger

  • Ensure that your NSG flow logs in Azure Network Watcher conform to the requirements as outlined in the Microsoft documentation. For more information, see Introduction to flow logging for network security groups.

  • Enable NSG flow logs in the Microsoft Azure Portal.

  • Ensure that you have an Azure subscription with user role permissions to deploy ARM templates and create the required resources.

  • Perform this procedure in the order shown below, because you need to save a token and a URL from Cortex XDR in earlier steps, and use them in Azure in later steps.

  1. Configure the Azure Network Watcher collection in Cortex XDR.

    1. Select Settings → Configurations → Data Collection → Collection Integrations.

    2. In the Azure Network Watcher configuration, click Add Instance to begin a new configuration.

    3. Set these parameters:

      • Name: Specify a meaningful name for your log collection configuration.

      • Enhanced Cloud Protection: (Optional) For enhanced cloud protection, you can normalize and enrich flow logs by selecting the Use flow logs in analytics checkbox. If selected, Cortex XDR ingests network flow logs as Cortex XDR network connection stories, which you can query with XQL Search using the xdr_dataset dataset with the preset called network_story.

    4. Click Save & Generate Token. The token is displayed in a popup.

      Click the copy icon next to the key and save the copy of this token somewhere safe. You will need to provide this token when you configure the Azure Function and set the Cortex Access Token value. If you forget to record the token and close the window, you will need to generate a new one and repeat this process. When you are finished, click Done to close the window.

    5. On the Integrations page for the Azure Network Watch Collector that you created, click the Copy API URL icon and save a copy of the URL somewhere safe. You will need to provide this URL when you configure the Azure Function and set the Cortex Http Endpoint value.

  2. Configure the Azure Function provided by Cortex XDR.

    1. Open the Azure Function provided by Cortex XDR.

    2. Click Deploy to Azure.

    3. Log in to Azure, and if necessary, complete authentication procedures.

    4. Set these parameters, where some fields are mandatory to set and others may already be populated for you.

      • Subscription: Specify the Azure subscription that you want to use for the App Configuration. If your account has only one subscription, it is automatically selected.

      • Resource group: Specify or create a resource group for your App Configuration store resource.

      • Region: Specify the Azure region that you want to use.

      • Unique Name: Provide a unique name for the function app. In the Azure Portal, this will be the name that appears in the list of resources.

      • Cortex Access Token: Cortex HTTP authorization key that you recorded when you configured the Azure Network Watcher collection in Cortex XDR in an earlier step.

      • Target Storage Account Name: Specify the name of the Azure Storage Account from which you want to capture the log blobs.

      • Target Container Name: Name of the container that contains the logs you want to forward (might be filled automatically).

      • Location: The region where all the resources will be deployed (leave blank to use the same region as the resource group).

      • Cortex Http Endpoint: Specify the API URL that you recorded when you configured the Azure Network Watcher collection in Cortex XDR.

      • Remote Package: The URL of the remote package ZIP file containing the Azure Function code (filled automatically).

    5. Click Review + Create to confirm your settings for the Azure Function.

    6. Click Create. It can take a few minutes until the deployment is complete.

    After events start to come in, a green check mark appears underneath the Azure Network Watcher configuration that you created in Cortex XDR, and the amount of data received is displayed.