Create an XQL Query - Administrator Guide - Cortex XDR - Cortex - Security Operations

Cortex XDR Pro Administrator Guide

Product
Cortex XDR
License
Pro
Creation date
2023-10-31
Last date published
2024-03-18
Category
Administrator Guide
Abstract

Learn how to create queries using the Cortex Query Language (XQL).

Use XQL Search to analyze raw log data stored in Cortex XDR. The following task demonstrates how to create a query that uses the coalesce function to derive a single username by examining multiple field names.

The XQL Language Reference guide provides more information about valid commands, such as the ones used in this example, and general Cortex Query Language (XQL) syntax.

  1. From Cortex XDR, select Incident ResponseInvestigationQuery BuilderXQL Search.

  2. Determine the type of XQL query you are creating.

    • To create an XQL query from scratch, use the default XQL query field to build your query.

    • To create an XQL query based on a Splunk query, move the toggle to Translate to XQL, where both a SPL query field and XQL query field are displayed. You can easily add a Splunk query in the SPL query field, which is converted to XQL in the XQL query field after you click the arrow (translate-to-spl-arrow.png). This option is disabled by default, so only the XQL query field is displayed. When building a Splunk query, skip to step #6.

  3. (Optional) Specify a dataset.

    You only need to specify a dataset if you are running your query against a dataset that you have not set as default. For more information, see how to manage datasets. See the XQL Language Reference for a list of datasets that are available to you, depending on your configuration.

    Note

    An administrator can create and view queries built with an unknown dataset that currently does not exist in Cortex XDR. All other users can only create and view queries built with an existing dataset.

    You can specify a dataset using one of the following formats, which is based on the data retention offerings available in Cortex XDR.

    • Hot Storage queries are performed on a dataset using the format dataset = <dataset name>. This is the default option.

      dataset = xdr_data
    • Cold Storage queries are performed using the format cold_dataset = <dataset name>.

      cold_dataset = xdr_data

    Note

    You can also build a query that investigates data in both a cold_dataset and hot dataset in the same query. In addition, since the hot storage dataset format is the default option and represents the fully searchable storage, for investigation and threat hunting, this format is used throughout this guide. For more information on hot and cold storage, see Dataset Management.

    From the first letter that you type, the query field provides you with suggestions of commands and their definitions.

    When you select a command, you will see available operators.

    After selecting the operator, the query field presents available values.

  4. Hit the return key and enter a pipe (|) followed by the first stage of your query.

    This stage uses the fields command to declare which fields are returned in the results. If you use this stage, then the following stages can only operate on the fields specified in it.

  5. Continue adding stages until your query is complete.

    This stage uses the function coalesce to return the first value that is not NULL out of the given fields and the alter stage command to assign that value to the field username.

  6. Specify the time period against which you want to run your query.

    The options are last 24H (hours), last 7D (days), last 1M (month), or select a Custom time period.

  7. Choose when to run the query.

    Select the calendar icon to schedule a query to run on or before a specific date, Add as BIOC to save the query as a BIOC rule (if compatible), or Run the query immediately.

    While the query is running, you can always navigate away from the page and a notification is sent when the query completes. You can also Cancel the query or run a new query, where you have the option to Run only new query (cancel previous) or Run both queries.

  8. (Optional) After your query is complete, you can save the query as one of the following rules.

    • BIOC RuleSave asBIOC Rule. The XQL query must at a minimum filter on the event_type field in order for it to be a valid BIOC rule that you can save. For more information, see Working with BIOCs.

    • Correlation RuleSave asCorrelation Rule. For more information, see Working with Correlation Rules.

  9. After running your query, review the Query Results.

    Alternate between the following display options to investigate your query results:

    • Table (xql-table-view.png)—Displays results in rows and columns according to the entity fields.

      From the table-settings.png menu, you can change the table layout and the following sections are displayed with some more configuration options:

      • In the Appearance section, you can Show line breaks for any text field in the Query Results. By default, the text in these fields are wrapped unless the Show line breaks option is selected. In addition, you can change the way rows and columns are displayed.

      • In the Log Format section, you can change the way that logs are displayed:

        • RAW—Raw format of the entity in the database.

        • JSON—Condensed JSON format with key value distinctions. Null values are not displayed.

        • TREE—Dynamic view of the JSON hierarchy with the option to collapse and expand the different hierarchies.

      • In the Search column section, you can find a specific column; enable or disable display of columns using the checkboxes.

    • Graph (xql-graph-view.png)—Use the Chart Editor to visualize the query results.

    • Advanced (xql-advanced-view.png)—Displays results in a table format aggregating the entity fields into one column. You can change the layout, decide whether to Show line breaks for any text field in the results table, and change the log format from the table-settings.png menu.

      Select Show more to pivot an Expanded View of the event results that include null values. You can toggle between the JSON and Tree views, search, and Copy to clipboard.

    You can also perform the following additional actions on the results displayed.

    • Export to File (export-to-file.png)—Exports the results to a TSV (Tab-separated values) file.

    • Refresh (refresh.png)—Refreshes the query results.

    • Free text search (free-text-search.png)—Searches the query results for the text that you specify in the free text search. Click the Free text search icon to reveal the text Type your search here.

    • Filter (xql-query-results-filter.png)—Enables you to filter a particular field in the interface that is displayed to specify your filter criteria.

    Note

    We recommend for Integer, Boolean, and timestamp, such as _Time, fields that you use the Filter as opposed to the Free text search to retrieve the most accurate query results.

    For Table and Advanced displays, Cortex XDR provides a Fields menu on the left side of the query results that you use to filter the results. To quickly set a filter, Cortex XDR displays the top 10 results from which you can choose to build your filter. From within the Fields menu, click on any field (excluding JSON and array fields) to see a histogram of all the values found in the result set for that field. This histogram includes a count of the total number of times a value was found in the result set, the value's frequency as a percentage of the total number of values found for the field, and a bar chart showing the value's frequency. In order for Cortex XDR to provide a histogram for a field, the field must not contain an array or a JSON object.

  10. (Optional) Save the query to your personal query library.

  11. (Optional) Continue investigation in the Causality View or Timeline View.

    Right-click the event and select the desired view. This option is available for the following types of events: process (except for those with an event sub-type of termination), network, file, registry, injection, load image, system calls, event logs for Windows, and system authentication logs for Linux. For network stories, you can pivot to the Causality View only. For cloud Cortex XDR events and Cloud Audit Logs, you can only pivot to the Cloud Causality View, while software-as-a-service (SaaS) related alerts for audit stories, such as Office 365 audit logs and normalized logs, you can only pivot to the SaaS Causality View.

  12. (Optional) Add a file path to your existing Malware Profile allowed list.

    Right-click a <path> fields, for example, target_process_path, file_path, or os_parent_path, and select Add <path type> to malware profile allow list.

  13. (Optional) Visualize Query Results.