Playbook Testing - Playbook Design Guide - 6.x - Cortex XSOAR - Cortex - Security Operations

Cortex XSOAR Playbook Design Guide

Product
Cortex XSOAR
Version
6.x
Creation date
2023-01-19
Last date published
2023-12-19
Category
Playbook Design Guide

After completing a playbook, the next step is to test the playbook on an incident to ensure the expected outcome. This can be done using a dev instance of a third-party integration to ingest incidents or by using the out of the box Sample Incident Generator.

Playbook debugger

Abstract

The Playbook Debugger enables you to troubleshoot playbooks and test multiple scenarios.

The Cortex XSOAR playbook debugger enables you to build and troubleshoot playbooks, by helping you find tasks that might fail and by testing different conditions, branches, and input and output options. Common use cases include:

  • Playbook development - test and improve playbooks as you build them.

  • Proof of concept - begin to create and test playbooks even before all integrations are in place, by manually providing inputs and outputs as needed.

  • Error troubleshooting - use the debugger to find and fix issues if a playbook stops on an error.

  • Explore Marketplace playbooks - install content packs and use the debugger to see whether the included playbooks are relevant for your use case.

Tip

The playbook debugger runs all commands as if it is in an incident. Make sure to skip automation tasks that make critical changes on third-party tools.

Building a playbook is an iterative process. The debugger provides a test environment where you can make changes to data and playbook logic and view the results in real time. You have the opportunity to see exactly what is written to context at each step and which indicators are extracted.

Debugger execution

The debugger runs the playbook with the permissions of the logged in user, not as dBot. When the user sets breakpoints, skips tasks, or overrides inputs or outputs, those changes only apply to the individual user’s session and do not permanently change the playbook. Using an existing incident as test data does not affect the original incident or change the original context data. When tasks run, however, they execute the same as they would without the debugger. For example, if you run the debugger and a task adds an item to a list, that item will be in the real list, accessible across Cortex XSOAR for all users with permission to view that list.

Test data

The debugger uses test data to execute the playbook, so you can see what your expected results would be. There are three options for test data.

  1. New Mock Incident - by default, the debugger runs using an empty mock incident. An empty mock incident is useful to test simple functionality, such as a playbook that does simple tasks such as parsing inputs.

  2. Playground - you can load the contents of the Playground as test data, enabling you to use uploaded files and custom context data for testing purposes.

  3. Existing Incident - you can select an existing Cortex XSOAR incident. For example, when debugging a phishing playbook, you might want to use an existing phishing incident that came from the mail listener integration. Using an existing incident in the debugger does not change the original incident.

    If you need to use event data from third-party software that is not yet set up as an integration, you can import a JSON file into Cortex XSOAR through the mapping feature and create an incident that can then be used as test data.Create an Incident

    You can use a file attachment for your test data by adding the file to an incident and selecting the incident or by uploading the file to the playground and using the playground as test data.

Inputs and outputs

The debugger enables you to temporarily override inputs and outputs for a playbook run and to view the results in real time. When you override an input or output in the debugger, the change is saved only in the debugger view and only for the user who made the change. If, after testing, you decide to keep the temporary changes you made, and apply them permanently to the playbook for all users, you need to cancel the override and edit the task. Tasks can be edited directly in the debugger or outside of the debugger using the standard playbook editing options.

Breakpoints

Breakpoints are used to pause playbook execution before a specific task. When the playbook is paused, the Debugger Panel displays the current state of context data, indicators, and task information.

At the breakpoint, you can override inputs and outputs to see how changes affect playbook execution.

In addition, conditional breakpoints set conditions for the playbook to proceed. The playbook only pauses if your condition is met, letting you manipulate data to see how different scenarios impact how the playbook runs. For example, you can set a conditional breakpoint to pause the playbook when a phishing incident targets a member of a VIP asset list. If there are no VIPs in this incident, the execution does not pause. If there is a VIP in the incident, you can check that the member was properly identified by the playbook task.

Skip tasks

For testing purposes, you might not want to close a port in a firewall, delete an email, or send a notification to a manager. For this purpose, you can skip a task. In other cases, you might skip a task where the integration has not yet been configured. By skipping a task and overriding the output, you can provide the data necessary to complete the playbook run. When you skip a conditional task, you can choose which branch runs after the skipped task, enabling you to test different outcomes for multiple branches.

Sample Incident Generator

The Sample Incident Generator enables testing different incident types. If you want to customize the generated incidents, you need to first duplicate the out of the box integration since out of the box content cannot be changed. You can then create custom incident JSON files to test multiple incident types.

sample-incident-generator-custom.png