Activate the Apache Kafka Collector - Administrator Guide - Cortex XDR - Cortex XSIAM - Cortex - Security Operations

In Cortex XDR, select Settings → Configurations → Data Broker → Broker VMs and locate your broker VM.

You can either right-click the broker VM and select Add App → Kafka Collector, or hover in the APPS column, and select Add → Kafka Collector.

Configure the Kafka Connection.

1. Specify the Bootstrap Server List—The <hostname/ip>:<port> of the bootstrap server (or servers). You can specify multiple servers, separated with a comma. For example, hostname1:9092,1.1.1.1:9092.

2. Select one of the Authentication Methods.

   • No Authentication—Default connection method for a new Apache Kafka setup, which doesn’t require authentication. With a standard Kafka setup, any user or application can write messages to any topic, as well as read data from any topic.

   • SSL Authentication—Authenticate your connection to Apache Kafka using an SSL certificate. Use this authentication method when the connection to the Apache Kafka server is a secure TCP, and upload the following.

     • Broker Certificate—Signed certificate used for the applet to authenticate to the Apache Kafka server.

     • Private Key—Private key for the applet used for decrypting the SSL messages coming from the Apache Kafka server.

     • (Optional)CA Certificate—CA certificate that was used to sign the server and private certificates. This CA certificate is also used to authenticate the Apache Kafka server identity.

   • SASL SSL (SCRAM-SHA-256)—Authenticate your connection to the Apache Kafka server with your Username, Password, and optionally your CA Certificate.

3. Test Connection to verify that you can connect to the Apache Kafka server. An error message is displayed for each server connection test that fails.

4. Configure the Topics Collection parameters.

   • Select the Topic Subscription Method for subscribing to Apache Kafka topics. Use List Topics to specify a list of topics. Use Regex Pattern Matching to specify a regular expression to search available topics.

   • Specify Topic(s) from the Apache Kafka server. For the List Topics subscription method, use a comma separated list of topics to subscribe to. For the Regex Pattern Matching subscription method, use a regular expression to match the Topic(s) to subscribe to.

   • (Optional) Specify a Consumer Group, a unique string or label that identifies the consumer group this log source belongs to. Each record that is published to an Apache Kafka topic is delivered to one consumer instance within each subscribing consumer group. Apache Kafka uses these labels to load balance the records over all consumer instances in a group. When specified, the Apache Kafka collector uses the given consumer group. When not specified, Cortex XDR assigns the Apache Kafka applet collector to a new automatically generated consumer group which is automatically generated for this log source with the name PAN-<Broker VM device name>-<topic name>.

   • Select the Log Format from the list as either RAW (default), JSON, CEF, LEEF, CISCO, or CORELIGHT. This setting defines the parser used to parse all the processed event types defined in the Topics field, regardless of the file names and extension. For example, if the Topics field is set to * and the Log Format is JSON, all files (even those named file.log) in the cluster are processed by the collector as JSON, and any entry that does not comply with the JSON format are dropped.

   • Specify the Vendor and Product which will be associated with each entry in the dataset. The vendor and product are used to define the name of your Cortex Query Language (XQL) dataset (<Vendor>_<Product>_raw).

     Note

     For CEF and LEEF logs, Cortex XDR takes the vendor and product names from the log itself, regardless of what you configure on this page.

5. (Optional)Add Topic to create another Topic Collection. Each topic can be added for a server only once.

6. (Optional) Other available options for Topic Collection.

   As needed, you can manage your Topic Collection settings. Here are the actions available to you.

   • Edit the Topics Collection details.

   • Disable/Enable a Topics Collection by hovering over the top area of the Topics Collection section, on the opposite side of the Topics Collection name, and selecting the applicable button.

   • Rename a Topics Collection by hovering over the top area of the Topics Collection section, on the opposite side of the Topics Collection name, and selecting the pen icon.

   • Delete a Topics Collection by hovering over the top area of the Topics Collection section, on the opposite side of the Topics Collection name, and selecting the delete icon.

(Optional)Add Connection to create another Apache Kafka Connection for collecting data.

(Optional) Other available options for Connections.

As needed, you can return to your Apache Kafka Collector settings to manage your connections.

Here are the actions available to you.

Activate the Apache Kafka Collector applet. Activate is enabled when all the mandatory fields are filled in.

After a successful activation, the APPS field displays Kafka with a green dot indicating a successful connection.

(Optional) To view metrics about the Apache Kafka Collector, in the Broker VMs page, hover over the Kafka connection displayed in the APPS field for your broker VM.

Cortex XDR displays Resources, including the amount of CPU, Memory, and Disk space the applet is using.

Manage the Kafka Collector.

After you activate the Apache Kafka Collector, you can make additional changes as needed. To modify a configuration, hover over the Kafka connection in the APPS column to display the Kafka Collector settings, and select the following.

Ensure that you Save your changes, which is enabled when all mandatory fields are filled in.

You can also Ingest Apache Kafka Events as Datasets.