Elasticsearch System Requirements - Administrator Guide - EoL - 6.5 - Cortex XSOAR - Cortex - Security Operations

The information in the following table is per Elasticsearch node, and assumes that the node is assigned all Elasticsearch node roles (for example, which data is written to disk and when).

Elasticsearch user permissions are available in the security guidelines.

Cortex XSOAR supports using Elasticsearch with all the major cloud service providers, AWS, OpenSearch, Azure, and Google Cloud Platform.

You can use Elasticsearch as a service provided by your cloud provider, or install Elasticsearch on a server in the cloud.

The hardware requirements for Elasticsearch in the cloud similar to those posted above. To achieve this with your cloud provider, Cortex XSOAR recommends you use the machines based on your intentions. For example:

You can configure your cloud environment to work with different regions provided that you can maintain the minimum latency requirements noted above.

It is recommended that you implement the following Elasticsearch configurations in Cortex XSOAR. The value of the shards and replica shards should match the sum total of Elasticsearch nodes that you have.

Set the number of shards for an index

This server configuration enables you to set the number of shards for a specific index upon creation, where <common-indicator> is the name of the index. The default is 1.

To improve the write-performance, you can increase the number of shards and decrease the number of replica shards.

elasticsearch.shards.<common-indicator>

Set the number of replica shards for an index

This server configuration enables you to set the number of replica shards for a specific index upon creation, where <common-indicator> is the name of the index. To increase search performance and data redundancy, you should set the value to the number of Elasticsearch nodes that you have. The default is 1.

elasticsearch.replicas.<common-indicator>

The following table compares the maximum total indicator capacity and disk usage for BoltDB and Elasticsearch. The maximum indicator capacity value was determined when testing the system.

We recommend using Elasticsearch if you plan to exceed at least one of the following maximum capacities for BoltDB.

The Cortex XSOAR indicators used to test the sizing requirements did not contain a significant number of additional fields nor custom fields. The maximum size of the indicators we tested had 20 additional or custom fields and a random string between 1-16 characters. Therefore, the indicators size tested were approximately 0.5KB. If you plan to have additional or custom fields for indicators, the maximum numbers should be reduced.

If performance is poor, or you know in advance that you will need more than the maximum number of indicators, you should consider scaling BoltDB or moving to Elasticsearch. If you are already in Elasticsearch, you can scale it as well. For both BoltDB and Elasticsearch, you can scale by either adding engines for one or more feed integrations or increasing the resources (CPU, RAM, Disk IOPS) of the Cortex XSOAR server. For Elasticsearch, you can also increase the Elasticsearch cluster size from 1 server to 2 or more servers.

The following table compares the disk usage for BoltDB and Elasticsearch.

The following table compares the number of indicators, time to ingestion, and disk usage for BoltDB and Elasticsearch.