XQL Search - Administrator Guide - Cortex XSIAM - Cortex

XQL Search - Administrator Guide - Cortex XSIAM - Cortex - Security Operations

Cortex XSIAM Administrator Guide

Product

Cortex XSIAM

Creation date

2024-07-16

Last date published

2024-12-12

Tip

For dataset queries, it's possible to change the default dataset in the Dataset Management page. For more information, see Manage Datasets.

XQL queries can contain different components depending on the type of query you want to build. For a complete list of the syntax options available with example queries, see the Cortex XSIAM XQL Language Reference Guide.

Important

Forensic datasets are not inlcuded by default in XQL Search query results, unless the dataset query is explicitly defined to use a forensic dataset.

Cortex Data Model

The Cortex Query Language (XQL) supports a single Cortex Data Model (XDM), which is a normalized data structure. Datasets are mapped to the XDM in 3 different ways:

Automatic default mappings, including the following:
- The xdr_data dataset is automatically mapped to the XDM with some data mapping exceptions.
- Next-Generation Firewall (NGFW) network log data are mapped to the XDM from the following datasets:
  - panw_ngfw_traffic_raw
  - panw_ngfw_threat_raw
  - panw_ngfw_url_raw
  - panw_ngfw_filedata_raw
  - panw_ngfw_globalprotect_raw
  - panw_ngfw_hipmatch_raw
Out-of-the-box mappings of the datasets as part of the Data Model Rules via the Marketplace. For more information, see Marketplace.
You can create your own mappings by creating your own Data Model Rules. For more information, see Create Data Model Rules.

For more information on the XDM Schema, specifically the fields, fieldsets, fields designated as ENUMS (CONST), and aliases, see the Cortex XSIAM Data Model Schema.

XDM Queries

The basic syntax structure for querying the Cortex Data Model (XDM) is:

datamodel dataset = <dataset name>...
    | <STAGE> ...
    | <STAGE> ...
    | <STAGE> ...

datamodel dataset in (<dataset name>,...)...
    | <STAGE> ...
    | <STAGE> ...
    | <STAGE> ...

In an XDM query using the datamodel dataset command, you must specify the datasets mapped to the XDM that you want to run your query against. These datasets contain log information ingested by Cortex XSIAM. You can also install Marketplace Content Packs, or map an ingested dataset into the XDM, to query additional datasets.

Adding a wildcard suffix (*) is supported in the <dataset_name>, which matches all datasets that are mapped to the data model and begin with the specified text. For example, datamodel dataset = xdr* or datamodel dataset in (xdr*).

When querying the XDM, fields that are not mapped to the XDM are accessible by <dataset>.<field>. They can be used at any stage of a datamodel query.

When creating XDM queries, auto-suggestions are available, according to the existing XDM fields.

Dataset Queries

In a dataset query, unless otherwise specified, the query runs against the xdr_data dataset, which contains all log information that Cortex XSIAM collects from all Cortex product agents, including EDR data, and PAN NGFW data. In a dataset query, if you are running your query against a dataset that has been set as default, there is no need to specify a dataset. Otherwise, specify a dataset in your query. The Dataset Queries lists the available datasets, depending on system configuration.

Note

Users with different dataset permissions can receive different results for the same XQL query.
An administrator or a user with a predefined user role can create and view queries built with an unknown dataset that currently does not exist in Cortex XSIAM. All other users can only create and view queries built with an existing dataset.
When you have more than one dataset or lookup, you can change your default dataset by navigating to Settings → Configurations → Data Management → Dataset Management, right-click on the appropriate dataset, and select Set as default. For more information about setting default datasets, see Dataset Management.

The basic syntax structure for querying datasets that are not mapped to the XDM is:

dataset = <dataset name> 
    | <stage1> ...
    | <stage2> ... 
    | <stage3> ...

dataset in (<dataset name>)
    | <stage1> ...
    | <stage2> ...
    | <stage3> ...

You can specify a dataset using one of the following formats, which is based on the data retention offerings available in Cortex XSIAM.

Hot Storage queries use the format dataset = <dataset name>. This is the default option. For example:
```
dataset = xdr_data
```
Cold Storage queries use the format cold_dataset = <dataset name>. For example:
```
cold_dataset = xdr_data
```
Note
You can build a query that investigates data in both a cold dataset and a hot dataset in the same query. In addition, as the hot storage dataset format is the default option and represents the fully searchable storage, this format is used throughout this guide for investigation and threat hunting. For more information on hot and cold storage, see Dataset Management.

When using the hot storage default format, this returns every xdr_data record contained in your Cortex XSIAM instance over the time range that you provide to the Query Builder user interface. This can be a large amount of data, which may take a long time to retrieve. You can use a limit stage to specify how many records you want to retrieve.

There is no practical limit to the number of stages that you can specify. See Stages Commands Reference for information on all the supported stages.Stages Commands Reference

In the xdr_data dataset, every user ﬁeld included in the raw data for network, authentication, and login events has an equivalent normalized user ﬁeld associated with it that displays the user information in the following standardized format:

<company domain>\<username>

For example, the login_data ﬁeld has the login_data_dst_normalized_user ﬁeld to display the content in the standardized format. To ensure the most accurate results, we recommend that you use these normalized_user ﬁelds when building your queries.