pre-commit - Demisto SDK Guide - Cortex

Abstract

The pre-commit command runs checks and linters and generates a YAML file based on the content run.

This command enhances the content development experience by running a variety of checks and linters. It utilizes the pre-commit framework for managing and maintaining multi-language pre-commit hooks. A .pre-commit-config-template.yaml file is used to configure the hooks if they are found in the content repository. Otherwise, a default is used.

Since content items run in containers with different Python versions and dependencies, this command matches content items with similar configurations before passing the generated temporary .pre-commit-config.yaml file.

Note

An internet connection is required to use pre-commit.

Argument	Description
-i, --input, --files	The paths to run pre-commit on. May pass multiple paths.
--staged-only	Whether to run only on staged files.
--commited-only	Whether to run on committed files only.
-g, --git-diff	Whether to use Git to determine which files to run on.
--prev-ver	The previous version to compare against. If not provided, the previous version will be determined using Git.
-a, --all-files	Whether to run on all files.
--mode	Special mode to run the pre-commit with.
--skip	A list of pre-commit hooks to skip.
--validate/--no-validate	Whether to run demisto-sdk validate.
--format/--no-format	Whether to run demisto-sdk format.
--secrets/--no-secrets	Whether to run demisto-sdk secrets.
--show-diff-on-failure	Show diff on failure.
-dry-run	Whether to run the pre-commit hooks in dry-run mode, which only creates the configuration file.
--docker/--no-docker	Whether to run Docker based hooks.
--image-ref	The Docker image reference to run Docker hooks with. Overrides the Docker image from the YAML file or the native image configuration.
--docker-image	Override the `docker_image` property in the template file. This is a comma separated list of: `from-yml`, `native:dev`, `native:ga`, `native:candidate`.
--console-log-threshold	Minimum logging threshold for the console logger.
--file-log-threshold	Minimum logging threshold for the file logger.
--log-file-path	Path to save log files to.
--template-path	A custom path for the pre-defined pre-commit template. If not provided, the system uses the default template.

You can set different arguments for different modes. For example, you might want to exclude some rules from the nightly build. Any key can be set this way.

You can use any mode you want and have multiple modes for each hook as in the following example.

Example 3.

To set arguments for a mode:

  - id: sample-hook
    args:nightly: ['--exclude', '123']
    args:mode1: ['--test', '123']
    args: ['This is the default argument']
    otherkey:nightly: hello
    otherkey: world

To call the pre-commit command:

demisto-sdk pre-commit -a --mode nightly

Hooks can be set in the .pre-commit-config_template.yaml file. The syntax is similar to the official pre-commit hooks, but allows more keys to be set.

Skip key

To skip a certain hook from running, add a skip key to the hook configuration:

- id: sample-hook
  skip: true

The skip hook can be used together with mode, to skip a hook when running in a specific mode.

Needs key

The needs keys allows you to define dependencies between hooks. If a hook with needs is skipped, hooks that depend on it are also skipped. In this example, both hooks are skipped:

- id: sample-hook
  skip: true
- id: needs-example
  needs: ["sample-hook"]

Parallel key

The parallel key indicates whether a hook should run in parallel. By default, hooks such as mypy, Ruff, and Docker produce multiple hooks which run in parallel. Default is True. To avoid running a specific hook in parallel, you can set it to false. When setting the parallel of a hook to false, a split hook will run sequentially.

- id: sample-hook
  parallel: false

Ruff
Runs Ruff, the extremely fast Python linter, using the Python version used in the container image set for the content item.
This step uses the Ruff rules configured in the pyproject.toml at the root of the content repo. If pyproject.toml does not exist, it uses Ruff's default rules. The rules set under the official demisto/content repo pyproject.toml file were handpicked to ensure a high level of code quality, and prevent bugs often found in content.
This step may modify files using Ruff's --fix flag. When files are modified, the step fails. We recommend committing them into the repo.
Check JSON/YAML/AST(py)
Validates that the content of the files is formatted correctly.
Autopep8
A linter that automatically formats Python code to conform to the PEP 8 Style Guide.
Pycln
Pycln is a formatter for finding and removing unused import statements.

To run a command in a script's relevant container, you can set up a Docker hook.

A Docker hook must be under the local repo and it's id must end with in-docker.

Example

  - id: simple-in-docker
    name: simple-in-docker
    description: my simple description
    entry: simplelinter
    args: ['run-all-checks']

The entry key should be the command that should be run (eg. pylint, pytest).

env key

Some commands require environment variables to be set. To configure environment variables for docker hooks you can set the env key. An example with pytest:

  - id: simple-in-docker
    env:
      DEMISTO_SDK__PYTEST__TESTS_PATH: Tests/scripts

copy_files key

Some hooks require several files to exist in the same directory as the code file in order to run properly. To configure this, set the copy_files key as follows:

- id: simple-in-docker
  copy_files:
    - Tests/scripts/conftest.py

config_file_arg key

Often with commands we run in the Docker we have a configuration file specified per integration/script. To configure, you must set the config_file_arg key. The configuration file should be in the same directory as the code file.

Example with Ruff:

  - id: simple-in-docker
    config_file_arg:
      arg_name: '--config'
      file_name: 'ruff.toml'