Ingest logs from Microsoft Azure Event Hub with an option to ingest audit logs to use in Cortex XDR authentication stories.
Note
Ingesting Logs from Azure Event Hub requires a Cortex XDR Pro per GB license.
Cortex XDR can ingest different types of data from Microsoft Azure Event Hub using the Microsoft Azure Event Hub data collector. To receive logs from Azure Event Hub, you must configure the settings in Cortex XDR based on your Microsoft Azure Event Hub configuration. After you set up data collection, Cortex XDR begins receiving new logs and data from the source.
When Cortex XDR begins receiving logs, the app creates a new dataset (MSFT_Azure_raw
) that you can use to initiate XQL Search queries. For example, queries refer to the in-app XQL Library. For enhanced cloud protection, you can also configure Cortex XDR to normalize Azure Event Hub audit logs, including Azure Kubernetes Service (AKS) audit logs, with other Cortex XDR authentication stories across all cloud providers using the same format, which you can query with XQL Search using the cloud_audit_logs
dataset. For logs that you do not configure Cortex XDR to normalize, you can change the default dataset. Cortex XDR can also raise Cortex XDR alerts (Analytics, IOC, BIOC, and Correlation Rules) when relevant from Azure Event Hub logs. While Correlation Rules alerts are raised on non-normalized and normalized logs, Analytics, IOC, and BIOC alerts are only raised on normalized logs.
Enhanced cloud protection provides:
Normalization of cloud logs
Cloud logs stitching
Enrichment with cloud data
Detection based on cloud analytics
Cloud-tailored investigations
The following table provides a brief description of the different types of Azure audit logs you can collect.
Note
For more information on Azure Event Hub audit logs, see Overview of Azure platform logs.
Type of Data | Description |
---|---|
Activity logs | Retrieves events related to the operations on each Azure resource in the subscription from the outside in addition to updates on Service Health events. NoteThese logs are from the management plane. |
Azure Active Directory (AD) Activity logs and Azure Sign-in logs | Contain the history of sign-in activity and audit trail of changes made in Azure AD for a particular tenant. NoteEven though you can collect Azure AD Activity logs and Azure Sign-in logs using the Azure Event Hub data collector, we recommend using the Office 365 data collector as it's easier to configure. In addition, ensure that you don't configure both collectors to collect the same types of logs as you'll be creating duplicate data in Cortex XDR. |
Resource logs, including AKS audit logs | Retrieves events related to operations that were performed within an Azure resource. NoteThese logs are from the data plane. |
Prerequisite Steps
Be sure you do the following tasks before you begin configuring data collection from Azure Event Hub.
Create an Azure Event Hub. For more information, see Quickstart: Create an event hub using Azure portal.
Ensure the format for the logs you want collected from the Azure Event Hub is either JSON or raw.
Configure the Azure Event Hub collection in Cortex XDR.
In the Microsoft Azure console, open the Event Hubs page, and select the Azure Event Hub that you created for collection in Cortex XDR.
Record the following parameters from your configured event hub, which you will need when configuring data collection in Cortex XDR.
Your event hub’s consumer group.
Select
→ , and select your event hub.Select
→ , and select your event hub.In the Consumer group table, copy the applicable value listed in the Name column for your Cortex XDR data collection configuration.
Your event hub’s connection string for the designated policy.
Select
→ .In the Shared access policies table, select the applicable policy.
Copy the Connection string-primary key.
Your storage account connection string required for partitions lease management and checkpointing in Cortex XDR.
Open the Storage accounts page, and either create a new storage account or select an existing one, which will contain the storage account connection string.
Select Show keys.
→ , and clickCopy the applicable Connection string.
Configure diagnostic settings for the relevant log types you want to collect and then direct these diagnostic settings to the designated Azure Event Hub.
Open the Microsoft Azure console.
Your navigation is dependent on the type of logs you want to configure.
Log Type
Navigation Path
Activity logs
Select +Add diagnostic setting.
→ → , andAzure AD Activity logs and Azure Sign-in logs
Select
→ .Select +Add diagnostic setting.
→ , and
Resource logs, including AKS audit logs
Search for Monitor, and select → .
From your list of available resources, select the resource that you want to configure for log collection, and then select +Add diagnostic setting.
Note
For every resource that you want to confiure, you'll have to repeat this step, or use Azure policy for a general configuration.
Set the following parameters:
Diagnostic setting name—Specify a name for your Diagnostic setting.
Logs Categories/Metrics—The options listed are dependent on the type of logs you want to configure. For Activity logs and Azure AD logs and Azure Sign-in logs, the option is called Logs Categories, and for Resource logs it's called Metrics.
Log Type
Log Categories/Metrics
Activity logs
Select from the list of applicable Activity log categories, the ones that you want to configure your designated resource to collect. We recommend selecting all of the options.
Administrative
Security
ServiceHealth
Alert
Recommendation
Policy
Autoscale
ResourceHealth
Azure AD Activity logs and Azure Sign-in logs
Select from the list of applicable Azure AD Activity and Azure Sign-in Logs Categories, the ones that you want to configure your designated resource to collect. You can select any of the following categories to collect these types of Azure logs.
Azure AD Activity logs:
AuditLogs
Azure Sign-in logs:
SignInLogs
NonInteractiveUserSignInLogs
ServicePrincipalSignInLogs
ManagedIdentitySignInLogs
ADFSSignInLogs
Note
There are additional log categories displayed. We recommend selecting all the available options.
Resource logs, including AKS audit logs
The list displayed is dependent on the resource that you selected. We recommend selecting all the options available for the resource.
Destination details—Select Stream to event hub, where additional parameters are displayed that you need to configure. Ensure that you set the following parameters using the same settings for the Azure Event Hub that you created for the collection.
Subscription—Select the applicable Subscription for the Azure Event Hub.
Event hub namespace—Select the applicable Subscription for the Azure Event Hub.
(Optional) Event hub name—Specify the name of your Azure Event Hub.
Event hub policy—Select the applicable Event hub policy for your Azure Event Hub.
Save your settings.
Configure the Azure Event Hub collection in Cortex XDR.
Select
→ → → .In the Azure Event Hub configuration, click Add Instance to begin a new configuration.
Set these parameters.
Name—Specify a descriptive name for your log collection configuration.
Event Hub Connection String—Specify your event hub’s connection string for the designated policy.
Storage Account Connection String—Specify your storage account’s connection string for the designated policy.
Consumer Group—Specify your event hub’s consumer group.
Log Format—Select the log format for the logs collected from the Azure Event Hub as Raw, JSON, CEF, LEEF, Cisco-asa, or Corelight.
Note
When you Normalize and enrich audit logs, the log format is automatically configured. As a result, the Log Format option is removed and is no longer available to configure (default).
CEF or LEEF: The Vendor and Product defaults to Auto-Detect.
Note
For a Log Format set to CEF or LEEF, Cortex XDR reads events row by row to look for the Vendor and Product configured in the logs. When the values are populated in the event log row, Cortex XDR uses these values even if you specified a value in the Vendor and Product fields in the Azure Event Hub data collector settings. Yet, when the values are blank in the event log row, Cortex XDR uses the Vendor and Product that you specified in the Azure Event Hub data collector settings. If you did not specify a Vendor or Product in the Azure Event Hub data collector settings, and the values are blank in the event log row, the values for both fields are set to unknown.
Cisco-asa: The following fields are automatically set and not configurable.
Vendor—Cisco
Product—ASA
Cisco data can be queried in XQL Search using the
cisco_asa_raw
dataset.Corelight: The following fields are automatically set and not configurable.
Vendor—Corelight
Product—Zeek
Corelight data can be queried in XQL Search using the
corelight_zeek_raw
dataset.Raw or JSON: The following fields are automatically set and are configurable.
Vendor—Msft
Product—Azure
Raw or JSON data can be queried in XQL Search using the
msft_azure_raw
dataset.
Vendor and Product—Specify the Vendor and Product for the type of logs you are ingesting.
The Vendor and Product are used to define the name of your Cortex Query Language (XQL) dataset (
<vendor>_<product>_raw
). The Vendor and Product values vary depending on the Log Format selected. To uniquely identify the log source, consider changing the values if the values are configurable.Note
When you Normalize and enrich audit logs, the Vendor and Product fields are automatically configured, so these fields are removed as available options (default).
Normalize and enrich audit logs—(Optional) For enhanced cloud protection, you can Normalize and enrich audit logs by selecting the checkbox (default). If selected, Cortex XDR normalizes and enriches Azure Event Hub audit logs with other Cortex XDR authentication stories across all cloud providers using the same format. You can query this normalized data with XQL Search using the
cloud_audit_logs
dataset.
Click Test to validate access, and then click Enable.
Once events start to come in, a green check mark appears underneath the Azure Event Hub configuration with the amount of data received.