Feed integrations allow fetching indicators from feeds, such as TAXII, AutoFocus, and Office 365.
An example feed integration can be seen here.
While feed integrations are developed the same as other integrations, they include several extra configuration parameters and APIs.
Naming convention
Feed integration names (id, name and display fields) should end with the word Feed. This consistent naming convention ensures that users can easily understand what the integration is used for.
Required parameters
Every feed integration should have the following parameters in the integration YAML file:
- display: Fetch indicators name: feed defaultvalue: true type: 8 required: false - display: Indicator Reputation name: feedReputation defaultvalue: feedInstanceReputationNotSet type: 18 required: false options: - None - Good - Suspicious - Bad additionalinfo: Indicators from this integration instance will be marked with this reputation. - display: Source Reliability name: feedReliability defaultvalue: F - Reliability cannot be judged type: 15 required: true options: - A - Completely reliable - B - Usually reliable - C - Fairly reliable - D - Not usually reliable - E - Unreliable - F - Reliability cannot be judged additionalinfo: Reliability of the source providing the intelligence data. - display: "" name: feedExpirationPolicy defaultvalue: indicatorType type: 17 required: false options: - never - interval - indicatorType - suddenDeath - display: "" name: feedExpirationInterval defaultvalue: "20160" type: 1 required: false - display: Feed Fetch Interval name: feedFetchInterval defaultvalue: "240" type: 19 required: false - display: Bypass exclusion list name: feedBypassExclusionList defaultvalue: "" type: 8 required: false additionalinfo: When selected, the exclusion list is ignored for indicators from this feed. This means that if an indicator from this feed is on the exclusion list, the indicator might still be added to the system.
The defaultvalue of the feedReputation, feedReliability, feedExpirationPolicy, and feedFetchInterval parameters should be set according to the qualities associated with the feed source for which you are developing a feed integration.
Incremental feeds
Incremental feeds pull only new or modified indicators that have been sent from the third party vendor. As the determination if the indicator is new or modified happens on the third-party vendor's side, and only indicators that are new or modified are sent to Cortex XSIAM, all indicators coming from these feeds are labeled new or modified.
Examples of incremental feeds usually include feeds that fetch based on a time range. For example, a daily feed which provides new indicators for the last day or a feed which is immutable and provides indicators from a search date onwards.
To indicate to Cortex XSIAM that a feed is incremental, add the configuration parameter: feedIncremental. If the user is not able to modify this setting, set the parameter to hidden with a defaultValue of true. For example:
- additionalinfo: Incremental feeds pull only new or modified indicators that have been sent from the integration. The determination if the indicator is new or modified happens on the third-party vendor's side, so only indicators that are new or modified are sent to Cortex XSIAM. Therefore, all indicators coming from these feeds are labeled new or modified.
defaultvalue: 'true'
display: Incremental feed
hidden: true
name: feedIncremental
required: false
type: 8If the feed supports both incremental and non-incremental modes, provide the configuration parameter as non-hidden. Thus, a user will be able to modify this settings as they see fit. In the feed code inspect the feedIncremental parameter to perform the proper fetch logic.
Code examples of incremental feeds:
Commands
Every feed integration has a minimum of three commands:
test-module- The command that is run when the Test button in the configuration panel of an integration is clicked.<product-prefix>-get-indicators- Where <product-prefix> is replaced by the name of the Product or Vendor source providing the feed. For example, if you were developing a feed integration for Microsoft Intune, this command might be called msintune-get-indicators. This command should fetch a limited number of indicators from the feed source and display them in the War Room.fetch-indicators- this command will initiate a request to the feed endpoint, format the data fetched from the endpoint to conform to Cortex XSIAM's expected input format, and create new indicators. If the integration instance is configured to fetch indicators, then this is the command that will be executed at the specified feed fetch Interval.
API command: demisto.createIndicators()
Use the demisto.createIndicators() function when the fetch-indicators command is executed. Here is an example from an existing feed integration:
def main():
params = demisto.params()
client = Client(params.get('insecure'),
params.get('proxy'))
command = demisto.command()
demisto.info(f'Command being called is {command}')
# Switch case
commands = {
'test-module': module_test_command,
'tor-get-indicators': get_indicators_command
}
try:
if demisto.command() == 'fetch-indicators':
indicators = fetch_indicators_command(client)
# we submit the indicators in batches
for b in batch(indicators, batch_size=2000):
demisto.createIndicators(b)
else:
readable_output, outputs, raw_response = commands[command](client, demisto.args())
return_outputs(readable_output, outputs, raw_response)
except Exception as e:
raise Exception(f'Error in {SOURCE_NAME} Integration [{e}]')The batch function is imported from CommonServerPython. We see that indicators are returned from calling fetch_indicators_command and are passed to demisto.createIndicators in batches.