Manage datasets in Notebooks - Create, edit, and delete datasets directly in Notebooks and use them in rules. - Administrator Guide - Cortex XSIAM - Cortex - Security Operations

Create, edit, and delete datasets directly in Notebooks and use them in rules.

You can create datasets in BigQuery through Notebooks using custom Cortex XSIAM APIs. You can then bring the insights and enriched data through machine learning into Cortex XSIAM to use them inside rules. For example, you can run a query in Cortex XSIAM that searches for an incident and correlates it to a sensitive users list you've created in Notebooks to trigger an alert.

To use the Cortex XSIAM APIs inside Notebooks, in Apps → Notebooks, import them from the Cortex SDK.

from cortex.dataset import define_dataset, create_dataset_from_dataframe, delete_dataset, get_created_datasets. 
from cortex.xql import start_query, get_query_results.

The created datasets are available for querying in the Query Builder and can be used when defining rules. You can view them under Dataset Management, and they can be selected for access when creating a user role. Creating and deleting datasets are recorded in the Management Audit Logs.

To change the schema of a dataset created using the Notebooks API, delete the dataset and create a new dataset with the updated schema.

You can use all the Google BigQuery functions to update the data in a dataset created using the Notebooks API.

The functions that are available for creating and editing datasets in Notebooks are listed below.

define_dataset(table_name: str, client: Optional[Client] = None)

create_dataset_from_dataframe(
    table_name: str,
    dataframe: DataFrame,
    schema: Optional[Sequence[Union[SchemaField, Mapping[str, Any]]]] = None,
    client: Optional[Client] = None,
    bq_client: Optional[BqClient] = None,
)

get_created_datasets(client: Optional[Client] = None)

delete_dataset(dataset_name: str, delete_underlying_bq_table: Optional[bool] = False, client: Optional[Client] = None)