Use DT for various context related functions in Cortex XSIAM. DT is a query language for JSON objects, similar to JSONQuery.
Cortex XSIAM Transform Language (commonly referred to as DT) is used for various context related functions in Cortex XSIAM. DT is a query language for JSON objects, similar to JSONQuery.
Context Example
The following sample context data shows the various ways DT can access, aggregate, and mutate data.
{
"URL": {
"Data": "google.com"
},
"IP": {
"Hostname": "google-public-dns-a.google.com",
"RecordedFuture": {
"FirstSeen": "2010-04-27T12:46:51.000Z",
"Criticality": "None",
"LastSeen": "2019-02-06T11:13:38.139Z"
},
"Address": "8.8.8.8",
"Geo": {
"Country": "United States",
"Location": "37.751, -97.822"
},
"ASN": 15169
},
"DBotScore": [{
"Vendor": "urlscan.io",
"Indicator": "google.com",
"Score": 0,
"Type": "url"
}, {
"Vendor": "ipinfo",
"Indicator": "8.8.8.8",
"Score": 0,
"Type": "ip"
}, {
"Vendor": "Recorded Future",
"Indicator": "8.8.8.8",
"Score": 0,
"Type": "ip"
}, {
"Vendor": "VirusTotal",
"Indicator": "8.8.8.8",
"Score": 1,
"Type": "ip"
}],
"URLScan": {
"URL": "google.com",
"Country": ["IE"],
"Certificates": [{
"SubjectName": "<span>www.google</span>.com",
"ValidFrom": "2018-12-19 08:16:00",
"ValidTo": "2019-03-13 08:16:00",
"Issuer": "Google Internet Authority G3"
}, {
"SubjectName": "*.google.com",
"ValidFrom": "2018-12-19 08:17:39",
"ValidTo": "2019-03-13 08:17:00",
"Issuer": "Google Internet Authority G3"
}, {
"SubjectName": "<span>www.google</span>.de",
"ValidFrom": "2018-12-19 08:16:00",
"ValidTo": "2019-03-13 08:16:00",
"Issuer": "Google Internet Authority G3"
}, {
"SubjectName": "*.g.doubleclick.net",
"ValidFrom": "2018-12-19 08:17:00",
"ValidTo": "2019-03-13 08:17:00",
"Issuer": "Google Internet Authority G3"
}, {
"SubjectName": "*.apis.google.com",
"ValidFrom": "2018-12-19 08:16:00",
"ValidTo": "2019-03-13 08:16:00",
"Issuer": "Google Internet Authority G3"
}],
"ASN": "AS15169"
},
"MaxMind": {
"Address": "8.8.8.8",
"ISP": "Google",
"UserType": "business",
"Organization": "Google LLC",
"ISO_Code": "US",
"Geo": {
"Location": "37.751, -97.822",
"Country": "United States",
"Continent": "North America",
"Accuracy": 1000
},
"ASN": 15169,
"RegisteredCountry": "United States"
}
}Nested Values
DT can access keys from nested dictionaries as well as dictionaries.
Using the above context data sample, you can access the following key values:
The "Continent" key value, located in the "Geo" dictionary, which is located in the "MaxMind" dictionary.
The "Hostname" key value, located in the "IP" dictionary.
The "FirstSeen" key value, located in the "RecordedFuture" dictionary, which is located in the "IP" dictionary.
Access the key values using the following DT statements:
Example | Result |
|---|---|
| North America |
| google-public-dns-a.google.com |
| 2010-04-27T12:46:51.000Z |
Dealing with arrays
Access array values as you access any other dictionary (or JSON) key in dot notation, with an index.
Note
Indices start with 0, not 1.
Using the above context data sample:
Under "URLScan", there is an array called "Certificates". If you want to access the "SubjectName" value of the first entry, use the following DT statement.
Example | Result |
|---|---|
| ["www.google.com", "*.google.com", "www.google.de", "*.g.doubleclick.net", "*.apis.google.com"] |
| www.google.com |
| ["www.google.com", "*.google.com", "www.google.de"] |
If you want to retrieve a range of results, you can use [0:9] where "0" is the beginning of the array and "9" is the 9th position in the array.
Selectors
DT also allows for conditions within the statement itself and uses JavaScript to select the context items. This is a way to bind results together by embedding a selection into the DT string.
Notice that val is used quite often in the DT string. This is a JavaScript method that exposes the value at the given context path.
The following are a few examples of selector methods:
Example | Result | Description |
|---|---|---|
|
| Returns only vendors that exactly match the urlscan.io description. |
|
| Returns any SubjectName that starts with "www". |
|
| Returns any SubjectName that contains doubleclick. |
|
| Returns any country that contains IE or ie or any mixed case. |
|
| Returns all ValidTo for certificates that have SubjectName ending with de. Notice that we tested a relative path to “SubjectName” (“de”) and returned a different path (“ValidTo”). |
|
| Returns all SubjectNames for certificates that have the same ValidTo time as the first certificate in the array. Note that the bind value (val1) does not start with ‘.’ and will DT the context from the top context. |
Selectors help avoid duplicate entries and can be used to add context to existing entries.
An example of how this can be used in your code:
ec = {
"Data": "www.demisto.com",
"Malicious": {
"Vendor": "Palo Alto Networks",
"Description": "This indicator found to be malicious by Palo Alto Networks"
}
}
demisto.results({
Type: entryTypes.note,
Contents: 'related',
ContentsFormat: formats.json,
HumanReadable: 'md',
EntryContext: {'URL(val.Data && val.Data == obj.Data)': ec
})The code snippet 'URL(val.Data && val.Data == obj.Data)' will look for entries in the context whose name found under "Data" are the same. If it finds a match, it will update the existing context, If it does not, it will create a new entry in the context because it views the entry as "unique" to the existing values.
Mutators
Since DT is JavaScript based, it can also mutate a result if your integration needs a different result format. A classic use case for this is joining a server address obtained from another integration to an endpoint found by your integration to create a URL which may be used by your integration at a later time.
Following are a few examples:
Example | Result | Description |
|---|---|---|
|
| Returns the timestamp where the SubjectName contains the word "doubleclick". Then it appends "Z" to the value. |
|
| Returns all the organizations but in lower case. |
|
| Returns all the vendors containing “Recorded” but in lower case. |
|
| Returns the concatenated ip and vendor for all DBotScores. |