Back up and restore a Cortex XSOAR elasticsearch deployment using snapshots.
Any Cortex XSOAR installation that uses an Elasticsearch database does not run automatic backups. Instead, you create and restore backups in Elasticsearch using snapshots.
Snapshots can include the entire database or specific indices. You can schedule snapshots to run automatically or take manual snapshots as needed. Snapshots usually take only a few minutes to complete and can be stored in a remote or local repository.
To create Elasticsearch snapshots, you will need a repository, preferably remote, to store the backup snapshots in a secure and available location for disaster recovery.
If you are using AWS Managed Elasticsearch, every Elasticsearch cluster is created with a default repository configured with a backend S3 bucket.
In an Elasticsearch environment, one or more nodes can fail and as a result one or more primary shards may become unavailable. When this happens, data may be unavailable and in some cases, depending on what was stored on the node, it may not be possible to access the Cortex XSOAR login page.
Depending on your Elasticsearch configuration, if any primary shards become inactive, Elasticsearch may try to automatically move the primary shards to any available node. To see whether Elasticsearch is attempting to move the primary shards automatically, use the Elasticsearch API to view all pending tasks:
GET /_cluster/pending_tasks. This method can take an extended period of time, and you might want to proceed with partial disaster recovery steps (restoring a snapshot) instead.
For disaster recovery, you can restore a snapshot of your entire active database or specific indices. If, for example, node 1 has failed and it contained the incidents index, you can restore only the incidents index to an already active node or to a new node. If all Elasticsearch nodes or an entire Elasticsearch cluster fail, you can immediately restore the latest snapshot on any Elasticsearch cluster. This process will restore all indices required for XSOAR to run.
Besides disaster recovery, snapshots can also be used to limit storage size. You can backup a specific index and then archive the data by deleting it from the database.
Elasticsearch security privileges must be configured to allow backups and restores. For example,
manage is required to create snapshots and to restore.
For comprehensive disaster recovery, we recommend scheduling automated backups, using your preferred enterprise backup solution, of all folders and files in
/var/lib/demisto/, with the exception of
/var/lib/demisto/temp. In addition, we recommend scheduling automated backups of the configuration file located at