Using Data Enrichment - Administrator Guide - Cortex XSIAM - Cortex - Security Operations

Cortex XSIAM Administrator Guide

Product
Cortex XSIAM
Creation date
2024-02-26
Last date published
2024-04-16
Category
Administrator Guide
Abstract

Learn about data-enriched fields and their limitations.

Note

Data enrichement is a beta feature, supported by the Enterprise Plus license only.

Cortex XSIAM automatically enriches your Cortex Data Model (XDM) data with additional information and context. Some examples of the types of data that are enriched include:

Note

For a complete list of auto-enriched fields, see the Cortex Data Model Schema Guide.

  • IP addresses are enriched with geolocation information.

  • User data is normalized.

  • If DSS exists, it is also enriched.

These enrichments are important for cyber analytics, rule detection, and investigations. Since these fields are enriched automatically by default, they do not have to be mapped manually in Data Model Rules. Note that enrichment is not performed when the input fields needed for enrichment are not available.

Enriched data is calculated by the system upon ingestion, and is saved for future queries. Keep in mind that some data may change over time, such as IP addresses that may change geolocation.  Therefore, checking the same IP address in external systems at a later time might return a different geolocation result.

Overriding Data Enrichment

We do not recommend overriding enriched fields. However, if enriched fields are not desired, they can be overridden by mapping data to fields that are usually enriched. For example:

[MODEL: dataset=okta_sso_raw]
| alter xdm.source.ip = actor->ip_address,
      xdm.source.location.country = actor->country,
      xdm.source.location.city = actor->geo.city;

When overriding enriched fields, ensure the following:

  • The overridden data should be normalized. 

  • All relevant enriched fields should be overridden (for example, all location fields), and empty values should be filled with “unknown” (or with NULL, if calculated enrichments are desired). These actions will prevent data mismatch and conflicts.

Important

When manually mapping ASN fields that are enriched, such as xdm.source.asn.as_number, with other ISP and domain fields that are not enriched, such as xdm.source.asn.isp and xdm.source.asn.domain, it's possible to receive incorrect XDM query results due to the misalignment between the overridden enrichement and system enrichment fields.

Limitations
  • Geolocation limitations

    • Some values will be NULL if the log country doesn't match the country detected by an external geolocation tool.

    • There might be discrepancies when some data come from the log and other data from the enrichment. For example, log country data versus enrichment longitude data.

  • Data enrichment is not performed for EDR events.

  • This feature is not supported in cold storage.

Backward compatibility

Data ingested by versions prior to Cortex XSIAM version 1.3 will not be enriched, because enrichment is calculated at the time of ingestion.

By default, enrichment is performed for NULL values only (non-NULL values are not overridden). Therefore, some existing mapping rules may need to be updated, in order to prevent mapping data to the enriched fields. Contact Customer Support for assistance with converting custom modeling rules and saved queries.