The Parsing Rules file consists of multiple sections of three types, which also represent the custom syntax specific to Parsing Rules.
Note
Only a user with Cortex Account Administrator or Instance Administrator permissions can access Parsing Rules.
The Parsing Rules file consists of multiple sections of these three types, which also represent the custom syntax specific to Parsing Rules.
INGEST: This section is used to define the resulting dataset.COLLECT(Optional): This section defines a rule that enables data reduction and data manipulation at the Broker VM to help avoid sending unnecessary data to the Cortex XSIAM server and reduce traffic, storage, and computing costs. In addition, theCOLLECTsection is used to manipulate, alter, and enrich the data before it’s passed to the Cortex XSIAM server. While this rule is optional to configure, once added this rule runs before theINGESTsection.CONST(Optional): This section is used to define strings and numbers that can be reused multiple times within Cortex Query Language (XQL) statements in otherINGESTsections by using$constName.RULE(Optional): Rules are part of the XQL syntax, which are tagged with a name, and can be reused in the code in theINGESTsections by using[rule:ruleName].EXTEND(Optional): This section is used to chain your Parsing Rules logic to extend your existing defaultRULEsections, which are added by a Content Package you installed from the Marketplace. AnEXTENDsection runs immediately after the defaultRULEsection that it extends and enables data manipulation without overriding or interfering with the existing vendor Parsing Rules.
The order of the sections is unimportant. The data of each section type gets grouped together during the parsing stage. Before any action takes place all COLLECT, CONST, RULE, EXTEND, and INGEST objects are grouped together and collected to the same list.
The syntax used in the Parsing Rules file is derived from XQL, but with a few modifications. This subset of XQL is called XQL for Parsing (XQLp).
Note
For more information on the XQL syntax, see Cortex XQL Language Reference.
The COLLECT, CONST, INGEST, RULE, and EXTEND syntax is derived from XQL, but with the following modifications for XQLp:
A statement never starts with a dataset or preset selection. The query's data source is meaningless. It is transparent to the user where the raw logs are coming from, fully handled by the system.
Only the following XQL stages are permitted: alter, fields, filter, and join. In addition, a new
callstage is supported, which is used to invoke another rule.Note
An
innertype ofjoinstage is only supported inCONST,INGEST, andRULEsections and is not supported in aCOLLECTsection.You cannot
callaRULEsection that exists in Default Rules from the User Defined Rules section.
Only the following XQL functions are permitted in all sections: parse_timestamp, parse_epoch, and regexcapture.
Note
The regexcapture function is only supported in Parsing Rules and cannot be used in any other XQL query.
No output stages are supported.
A
Ruleobject can only contain a single statement.A
join innerquery is restricted to using a lookup as a data source and is only supported in XQLp stages.There is no default lookup, so all
join innerqueries must start withdataset=<lookup> | ....CONSTreference ($MY_CONST) is supported.An
INcondition can only take a sequence list, such asdevice_name in (“device1”, “device2”, “device3”)and not another XQL or XQLpinnerqueries.You can't create parsing rules for Next-Generation Firewall (NGFW) datasets that are in the format
panw_ngfw_<text>_raw, and the Observability dataset calledpanw_observability_raw.
Comments in C programming language can be used anywhere throughout the Parsing Rules file:
// line comment /* inner comment */
Note
Every statement in the Parsing Rules file must end with a semicolon (;).