Datasets and presets - Administrator Guide - Cortex XDR - Cortex

Datasets and presets - Administrator Guide - Cortex XDR - Cortex - Security Operations

Cortex XDR Documentation

Product

Cortex XDR

Creation date

2024-03-06

Last date published

2025-03-20

Datasets

The standard, built-in data source that is available in every Cortex XDR instance is the xdr_data dataset. This is a very large dataset with many available fields. For more information about this dataset, see Cortex XDR XQL Schema Reference. Cortex Query Language (XQL) supports using different languages for dataset and field names. In addition, the dataset formats supported are dependent on the data retention offerings available in Cortex XDR according to whether you want to query hot storage (default) or cold storage. For more information, see XQL Language Structure.

This dataset is comprised of both raw Endpoint Detection and Response (EDR) events reported by the Cortex XDR agent, and of logs from different sources such as third-party logs. To help you investigate events more efficiently, Cortex XDR also stitches these logs and events together into common schemas called stories. These stories are available using the Cortex XDR Presets.

When building queries in XQL, keep the following in mind with respect to datasets:

Use the dataset keyword to specify a dataset on your query.
Create custom datasets using the target stage.
Dataset names can use uppercase characters, but in queries dataset names are always treated as if they are lowercase. In addition, dataset names are supported using different languages, numbers (0-9), and underscores (_). Yet, underscores cannot be the first character of the name.
Upon ingestion, all fields are retained even fields with a null value. You can also use XQL to query parsing rules for null values.

Depending on your integrations, you can have the following datasets available for queries:

Data	Dataset
Active Directory via Cloud Identity Engine	`pan_dss_raw` Note To set up this Cloud Identity Engine (previously called Directory Sync Service (DSS)) dataset, you need to set up a Cloud Identity Engine. Otherwise, you will not have a `pan_dss_raw` dataset. For more information, see Set up Cloud Identity Engine.
Alerts table in Cortex XDR	alerts Note `INFO` alerts are not included in this dataset. The alert fields included in this dataset are limited to certain fields available in the API. For the full list, see Get Alerts Multi-Events v2 API.
Amazon S3	Audit logs All logs: `aws_s3_raw` Normalize and enrich audit logs: `cloud_audit_logs` Generic logs `<Vendor>_<Product>_raw` Network flow logs All logs: `aws_s3_raw` Normalize and enrich flow logs: `xdr_dataset` dataset with a preset called `network_story`
Authentication logs (subset of xdr_data)	Authentication logs, such as Okta: `auth_logs` Note The fields contained in this dataset are a subset of the fields in the `xdr_data` dataset.
AWS CloudTrail and Amazon CloudWatch	`<Vendor>_<Product>_raw`
Azure Event Hub	All logs: `MSFT_Azure_raw` Normalize and enrich audit logs: `cloud_audit_logs`
Azure Network Watcher	All logs: `MSFT_Azure_raw` Normalize and enrich flow logs: `xdr_dataset` dataset with a preset called `network_story`
BeyondTrust Privilege Management Cloud	`beyondtrust_privilege_management_raw`
Box	Events (admin_logs) `box_admin_logs_raw` Box Shield Alerts `box_shield_alerts_raw` Users `box_users_raw` Groups `box_groups_raw`
Checkpoint FW1/VPN1	`<Vendor>_<Product>_raw`
Cisco ASA	Cisco ASA firewalls or Cisco AnyConnect VPN `cisco_asa_raw`
Collector status change audit for collection integrations, custom collectors, and marketplace collectors.	`collection_auditing`
Corelight Zeek	`corelight_zeek_raw`
Correlation rule executions	`correlations_auditing`
Cortex Data Lakes	`xdr_data`
Cortex XDR Collectors	`panw_xdrc_raw`
Cortex XDR Host Firewall enforcement events	`host_firewall_events`
CSV files in shared Windows directory	Custom datasets: Select from pre-existing user-created datasets or add a new dataset.
Database data (MySQL, PostgreSQL, MSSQL, and Oracle)	`<Vendor>_<Product>_raw`
Data ingestion health metrics	Datasets: `metrics_source`
Dropbox	Events `dropbox_events_raw` Member Devices `dropbox_members_devices_raw` Users `dropbox_users_raw` Groups `dropbox_groups_raw`
Elasticsearch Filebeat	`<Vendor>_<Product>_raw`
Elasticsearch Winlogbeat	`<Vendor>_<Product>_raw` Note If the vendor and product are not specified in the Winlogbeat profile’s configuration file, Cortex XDR creates a default dataset called `microsoft_windows_raw`.
Errors related to Parsing Rules	`parsing_rules_errors`
Forcepoint DLP	`forcepoint_dlp_endpoint_raw`
Fortinet Fortigate	`<Vendor>_<Product>_raw`
GlobalProtect access authentication logs	`xdr_data` Note To ensure GlobalProtect access authentication logs are sent to Cortex XDR, verify that your PANW firewall’s Log Settings for GlobalProtect has the Cortex Data Lake checkbox selected.
Google Cloud Platform (GCP) logs	All log types: `google_cloud_logging_raw` Normalize and enrich audit and flow logs: `cloud_audit_logs` Audit logs: `cloud_audit_logs` Network flow logs: `xdr_dataset` dataset with a preset called `network_story`
Google Kubernetes Engine (GKE)	`<Vendor>_<Product>_raw`
Google Workspace	Google Chrome: `google_workspace_chrome_raw` Admin Console: `google_workspace_admin_console_raw` Google Chat: `google_workspace_chat_raw` Enterprise Groups: `google_workspace_enterprise_groups_raw` Login: `google_workspace_login_raw` Rules: `google_workspace_rules_raw` Google drive: `google_workspace_drive_raw` Token: `google_workspace_token_raw` User Accounts: `google_workspace_user_accounts_raw` SAML: `google_workspace_saml_raw` Alerts: `google_workspace_alerts_raw` Emails: `google_gmail_raw`
Host Inventory and Vulnerability Assessment	Datasets `host_inventory` `va_cves` `va_endpoints` Presets `host_inventory` `host_inventory_accessibility` `host_inventory_applications` `host_inventory_auto_runs` `host_inventory_cpus` `host_inventory_daemons` `host_inventory_disks` `host_inventory_drivers` `host_inventory_endpoints` `host_inventory_extensions` `host_inventory_groups` `host_inventory_kbs` `host_inventory_mounts` `host_inventory_services` `host_inventory_shares` `host_inventory_users` `host_inventory_volumes` `host_inventory_vss`
Incidents table in Cortex XDR	`incidents`
JSON or text logs from third-party source over HTTP	`<Vendor>_<Product>_raw`
Login logs (subset of xdr_data)	Login logs, such as WEC: `login_logs` Note The fields contained in this dataset are a subset of the fields in the `xdr_data` dataset.
Logs from third party source over FTP, FTPS, or SFTP	`<Vendor>_<Product>_raw`
Microsoft 365 (email)	`msft_o365_emails_raw` `msft_o365_users_raw` `msft_o365_groups_raw` `msft_o365_devices_raw` `msft_o365_mailboxes_raw` `msft_o365_rules_raw` `msft_o365_contacts_raw`
Microsoft Office 365	Microsoft Office 365 audit events from Management Activity API: Azure AD Activity Logs: `msft_o365_azure_ad_raw` Exchange Online: `msft_o365_exchange_online_raw` Sharepoint Online: `msft_o365_sharepoint_online_raw` DLP: `msft_o365_dlp_raw` General: `msft_o365_general_raw` Microsoft Office 365 emails via Microsoft’s Graph API: `msft_o365_emails_raw` Azure AD authentication events from Microsoft Graph API: `msft_azure_ad_raw` Azure AD audit events from Microsoft Graph API: `msft_azure_ad_audit_raw` Alerts from Microsoft Graph Security API: `msft_graph_security_alerts_raw`
NetFlow	`ip_flow_ip_flow_raw` (default) When configured, uses the format `<Vendor>_<Product>_raw`
Network Share logs	`<Vendor>_<Product>_raw`
Okta	`okta_sso_raw`
OneLogin	Log collection `onelogin_events_raw` Directory `onelogin_users_raw` `onelogin_groups_raw` `onelogin_apps_raw`
PANW EDR	`xdr_data`
PANW IOT Security	Alerts `panw_iot_security_alerts_raw` Devices `panw_iot_security_devices_raw`
PANW NGFW	`panw_ngfw__raw` Supports the following logs. Authentication Logs: `panw_ngfw_auth_raw` Configuration Logs: `panw_ngfw_config_raw` File Data Logs: `panw_ngfw_filedata_raw` Global Protect Logs: `panw_ngfw_globalprotect_raw` Hipmatch Logs: `panw_ngfw_hipmatch_raw` System Logs: `panw_ngfw_system_raw` Threat Logs: `panw_ngfw_threat_raw` Traffic Logs: `panw_ngfw_traffic_raw` URL Logs: `panw_ngfw_url_raw` User ID Logs: `panw_ngfw_userid_raw` These datasets use the query field names as described in the Cortex schema documentation.
PingFederate	`ping_identity_pingfederate_raw`
PingOne for Enterprise	`pingone_sso_raw`
Prisma Access Browser	`panw_prisma_access_browser_raw`
Prisma Cloud	`prisma_cloud_raw`
Prisma Cloud Compute	`prisma_cloud_compute_raw`
Proofpoint Targeted Attack Protection	`proofpoint_tap_raw`
ServiceNow CMDB	A ServiceNow CMDB dataset is created for each table configured for data collection using the format `servicenow_cmdb_<table name>_raw`.
Salesforce.com	`salesforce_connectedapplication_raw` `salesforce_permissionset_raw` `salesforce_profile_raw` `salesforce_groupmember_raw` `salesforce_group_raw` `salesforce_user_raw` `salesforce_userrole_raw` `salesforce_document_raw` `salesforce_contentfolder_raw` `salesforce_attachment_raw` `salesforce_contentdistribution_raw` `salesforce_tenantsecuritylogin_raw` `salesforce_useraccountteammember_raw` `salesforce_tenantsecurityuserperm_raw` `salesforce_account_raw` `salesforce_audit_raw` `salesforce_login_raw` `salesforce_eventlogfile_raw`
Syslog/CEF	`<CEFVendor>_<CEFProduct>_raw`
USB devices connect and disconnect events reported by the agent	`xdr_data` Note You can query in XQL for this data and build widgets based on the `xdr_data` dataset or using the preset `device_control`. To view in an XQL query these events, the Device Configuration of the endpoint profile must be set to Block. Otherwise, the USB events are not captured. The events are also captured when a group of device types are blocked on the endpoints with a permanent or temporary exception in place. For more information, see [Ingest Connect and Disconnect Events of USB Devices] in Device control.
VPN logs (subset of `xdr_data`)	VPN logs, such as GlobalProtect: `vpn_logs` Note The fields contained in this dataset are a subset of the fields in the `xdr_data` dataset.
Windows Endpoints using Cortex XDR Forensics Add-on	`forensics_amcache` `forensics_application_resource_usage` `forensics_arp_cache` `forensics_background_activity_monitor` `forensics_chrome_history` `forensics_cid_size_mru` `forensics_command_history` `forensics_dns_cache` `forensics_edge_anaheim_history` `forensics_edge_spartan_history` `forensics_event_log` `forensics_file_access` `forensics_file_listing` `forensics_firefox_history` `forensics_handles` `forensics_hosts_file` `forensics_internet_explorer_history` `forensics_jumplist` `forensics_last_visited_pidl_mru` `forensics_log_me_in` `forensics_net_sessions` `forensics_network` `forensics_network_connectivity_usage` `forensics_network_data_usage` `forensics_open_save_pidl_mru` `forensics_port_listing` `forensics_prefetch` `forensics_process_execution` `forensics_process_listing` `forensics_psreadline` `forensics_recent_files` `forensics_recentfilecache` `forensics_recycle_bin` `forensics_registry` `forensics_remote_access` `forensics_seven_zip_folder_history` `forensics_shellbags` `forensics_shimcache` `forensics_team_viewer` `forensics_typed_paths` `forensics_typed_urls` `forensics_user_access_logging` `forensics_user_assist` `forensics_windows_activities` `forensics_winrar_arc_history` `forensics_word_wheel_query`
Windows event logs via Cortex XDR Windows agents	`microsoft_windows_raw`
Windows Event Collector (WEC)	`xdr_data` `microsoft_windows_raw`
Windows DHCP using Elasticsearch Filebeat	`microsoft_dhcp_raw`
Windows DNS Debug using Elasticsearch Filebeat	Raw Data `microsoft_dns_raw` Normalized Stories `xdr_data` with the preset called `network_story`.
Workday	`workday_workday_raw`
Zscaler Cloud Firewall	ZIA Firewall logs: `zscaler_nssfwlog_raw` Web logs: `zscalar_nssweblog_raw` ZPA `zscaler_zpa_raw`

Presets

Presets offer groupings of xdr_data fields that are useful for analyzing specific areas of network and endpoint activity. All of the fields available for a preset are also available on the larger xdr_data dataset, but by using the preset your query can run more efficiently. Presets are sorted at random by the first one million results found.

Two of the available presets are stories. These contain information stitched together from Cortex XDR agent events and log files to form a common schema. They are authentication_story and network_story.

You use the preset keyword to specify a dataset in your query.

Datasets

Note

Note

Note

Note

Note

Note

Note

Note

Presets