Validate the Migration to Elasticsearch - Administrator Guide - EoL - 6.5 - Cortex XSOAR - Cortex - Security Operations

Cortex XSOAR Administrator Guide
Product
Cortex XSOAR
Version
6.5
Creation date
2022-09-28
Last date published
2024-07-16
Category
Administrator Guide
End of Life > EoL
Abstract

Verify that the Cortex XSOAR migration to Elasticsearch was successful.

After completing the migration process, you should verify that the migration completed successfully. Part of this validation should be done before you start the system, and part can only be done once you start the system.

  1. After completing the migration, navigate to the directory in which your logs are stored. By default, this is /var/log/demisto.

  2. Open the elastic.migration.log file.

  3. At the end of each object migration, there is a summary of the migration.

    Note

    To view the migration results in a user-friendly format, run the migration with previous-results flag.

    For example, we see here that 14725 objects out 14725 object in the main database for the tenant were migrated, meaning, everything was migrated successfully. info === file /var/lib/demisto/tenants/acc_<tenantName>/data/demisto.db migrated 14725 out of 14725 objects in 6.311s

    In contrast, if something was not migrated properly, you will see something similar to:info successfully migrated: insight: 3024 | 1 failed to migrate, took 1.884s

    In the event that you receive an error like this, contact your support representative.

  4. (Optional) - The migration tool, by default, skips objects larger than 100 MB. The migration summary includes information on any large objects that were not migrated. We recommend reviewing the specific objects and determining whether they are still required. If the large object is required, you can migrate skipped large objects by running the migration tool again, with the object-max-size flag set to a higher limit and by including the retry-large-objects flag. Note that the retry-large-objects flag migrates the entire bucket again, including items that may already have been migrated. If new data has been added in Elasticsearch since the earlier migration, this data will be overwritten.

    sudo ./elasticMigrator -config-path /usr/local/dev/copy_of_demisto.conf -db-path /usr/local/dev/lib_demisto_copy/data -object-max-size 200 -retry-large-objects

  5. (Optional) - If you are using Kibana, validate the migration in Kibana.

    Verify cluster status is green.

    Verify Cortex XSOAR indexes are green.

  6. Query the Elasticsearch API to verify incident ID offset, cluster health, number of shards, etc.

    Description

    Example

    Retrieves the Incident ID offset, which should match the latest Incident ID in your system prior to the migration. The ID is in the ‘offset’ key, and it is critical that this ID be equal to the ID of the last Incident in BoltDB. If data was not migrated in the correct order, the ID will not be equal. Do not restart the Cortex XSOAR server if it is not equal.

    curl -k -u username:password https://readyxsoarone:9200/dmst-common-incidentidtrack/_doc/thisistheid

    Get cluster health

    curl -k -u username:password https://readyxsoarone:9200/_cluster/health

    GET XSOAR indexes

    curl -k -u username:password https://readyxsoarone:9200/_cat/indices | grep dsmt-

    GET only the health, status, and index

    curl -k -u username:password https://readyxsoarone:9200/_cat/indices?h=health,status,index | grep dsmt-

  7. If no errors were found, start the Cortex XSOAR server.

  8. Perform a basic sanity check on your system. For example, ensure that your content and incidents were migrated.

  9. In the event that there are any errors when starting the server, or if you see that data was not migrated, contact your support representative.