Ingest Logs from Windows DHCP using Elasticsearch Filebeat - Administrator Guide - Cortex XDR - Cortex - Security Operations

Cortex XDR Pro Administrator Guide

Product
Cortex XDR
License
Pro
Creation date
2023-10-31
Last date published
2024-03-18
Category
Administrator Guide
Abstract

Learn how to configure Cortex XDR to receive Windows DHCP logs.

Note

Ingesting logs and data requires a Cortex XDR Pro per GB license.

You can configure Cortex XDR to receive Windows DHCP logs using Elasticsearch Filebeat with the following data collectors.

Ingest Windows DHCP Logs with an XDR Collector Profile
Abstract

Extend Cortex XDR visibility into logs from Windows DHCP using an XDR Collector Windows Filebeat profile.

Extend Cortex XDR visibility into logs from Windows DHCP using an XDR Collector Windows Filebeat profile.

Note

Ingesting logs and data requires a Cortex XDR Pro per GB license.

You can enrich network logs with Windows DHCP data when defining data collection in an XDR Collector Windows Filebeat profile. When you add a XDR Collector Windows Filebeat profile using the Elasticsearch Filebeat default configuration file called filebeat.yml, you can define whether the collected data undergoes follow-up processing in the backend for Windows DHCP data. Cortex XDR uses Windows DHCP logs to enrich your network logs with hostnames and MAC addresses that are searchable in XQL Search using the Windows DHCP Cortex Query Language (XQL) dataset (microsoft_dhcp_raw).XQL Search

While this enrichment is also available when configuring a Windows DHCP Collector for a cloud data collection integration, we recommend configuring Cortex XDR to receive Windows DHCP logs with an XDR Collector Windows Filebeat profile as it’s the ideal setup configuration.

Configure Cortex XDR to receive logs from Windows DHCP using an XDR Collector Windows Filebeat profile.

  1. Add an XDR Collector Profile for Windows.

    Follow the steps for creating a Windows Filebeat profile explained in this section, and configure the filebeat.yml file as explained in the following step.

  2. Configure the filebeat.yml file to collect Windows DHCP data.

    You can enrich network logs with Windows DHCP data when defining data collection by setting the vendor to “microsoft” and product to “dhcp” in the filebeat.yml file, which you can then query in the microsoft_dhcp_raw dataset. Here is an example of how to configure the filebeat.yml file to normalize Windows DHCP logs with an XDR Collector, which you can modify according to your system settings.

    Note

    To avoid formatting issues in filebeat.yml, we recommend that you validate the syntax of the YML.

    filebeat.inputs:
      - type: log
        enabled: true
        paths:
          - c:\Windows\System32\dhcp\DhcpSrvLog*.log
        processors:
      - add_host_metadata:
              when.not.contains.tags: forwarded
          - drop_event.when.not.regexp.message: "^[0-9]+,.*"
          - dissect:
              tokenizer:
    "%{id},%{date},%{time},%{description},%{ipAddress},%{hostName},%{macAddress},%{userName},%{transactionID},%{qResult},%{probationTime},%{correlationID},%{dhcid},%{vendorClassHex},%{vendorClassASCII},%{userClassHex},%{userClassASCII},%{relayAgentInformation},%{dnsRegError}"
          - drop_fields:
              fields: [ "message" ]
          - add_locale: ~
          - rename:
              fields:
                - from: "event.timezone"
                  to: "dissect.timezone"
              ignore_missing: true
              fail_on_error: false
          - add_fields:
              fields: 
                vendor: "microsoft"
                product: "dhcp"
Ingest Windows DHCP Logs with the Windows DHCP Collector
Abstract

Extend Cortex XDR visibility into logs from Windows DHCP using Elasticsearch Filebeat with the Windows DHCP data collector.

Extend Cortex XDR visibility into logs from Windows DHCP using Elasticsearch Filebeat with the Windows DHCP data collector.

Note

Ingesting logs and data requires a Cortex XDR Pro per GB license.

To receive Windows DHCP logs, you must configure data collection from Windows DHCP via Elasticsearch Filebeat. This is configured by setting up a Windows DHCP Collector in Cortex XDR and installing and configuring an Elasticsearch Filebeat agent on your Windows DHCP Server. Cortex XDR supports using Filebeat up to version 8.0.1 with the Windows DHCP Collector.

Certain settings in the Elasticsearch Filebeat default configuration file called filebeat.yml must be populated with values provided when you configure the Collection Integrations settings in Cortex XDR for the Windows DHCP Collector. To help you configure the filebeat.yml correctly, Cortex XDR provides an example file that you can download and customize. After you set up collection integration, Cortex XDR begins receiving new logs and data from the source.

Note

For more information on configuring the filebeat.yml file, see the Elastic Filebeat Documentation.

Windows DHCP logs are stored as CSV (comma-separated values) log files. The logs rotate by days (DhcpSrvLog-<day>.log), and each file contains two sections - Event ID Meaning and the events list.

As soon as Cortex XDR begins receiving logs, the app automatically creates a Windows DHCP XQL dataset (microsoft_dhcp_raw). Cortex XDR uses Windows DHCP logs to enrich your network logs with hostnames and MAC addresses that are searchable in XQL Search using the Windows DHCP Cortex Query Language (XQL) dataset.

Configure Cortex XDR to receive logs from Windows DHCP via Elasticsearch Filebeat with the Windows DHCP collector.

  1. Configure the Windows DHCP Collector in Cortex XDR.

    1. Select SettingsConfigurationsData CollectionCollection Integrations.

    2. In the Windows DHCP Collector configuration, click Add Instance to begin a new configuration.

    3. (Optional) Download example filebeat.yml file.

      To help you configure your filebeat.yml file correctly, Cortex XDR provides an example filebeat.yml file that you can download and customize.

    4. Specify a descriptive Name for your log collection configuration.

    5. Save & Generate Token. The token is displayed in a blue box, which is blurred out in the image below.

      Click the copy icon next to the key and record it somewhere safe. You will need to provide this key when you set the api_key value in the Elasticsearch Output section in the filebeat.yml file as explained in Step #2. If you forget to record the key and close the window you will need to generate a new key and repeat this process.

    6. Select Done to close the window.

    7. In the Integrations page for the Windows DHCP Collector that you created, select Copy api url and record it somewhere safe. You will need to provide this URL when you set the hosts value in the Elasticsearch Output section in the filebeat.yml file as explained in Step #2.

  2. Configure an Elasticsearch Filebeat agent on your Windows DHCP Server.

    1. Navigate to the Elasticsearch Filebeat installation directory, and open the filebeat.yml file to configure data collection with Cortex XDR. We recommend that you use the download example file provided by Cortex XDR.

    2. Update the following sections and tags in the filebeat.yml file. The example code below details the specific sections to make these changes in the file.

      Note

      To avoid formatting issues in your filebeat.yml, we recommend that you do not copy and paste the code syntax provided below into your file, and use the download example file to make your customizations.

      • Filebeat inputs—Define the paths to crawl and fetch. The code below provides an example of how to configure the Filebeat inputs section in the filebeat.yml file with these paths configured.

        # ============================== Filebeat inputs ===============================
        filebeat.inputs:
          # Each - is an input. Most options can be set at the input level, so
          # you can use different inputs for various configurations.
          # Below are the input specific configurations.
          - type: log  
            # Change to true to enable this input configuration.  
            enabled: true  
            # Paths that should be crawled and fetched. Glob based paths.  
            paths:       
              - c:\Windows\System32\dhcp\DhcpSrvLog*.log    
        
      • Elasticsearch Output—Set the hosts and api_key, where both of these values are obtained when you configured the Windows DHCP Collector in Cortex XDR as explained in Step #1. The code below provides an example of how to configure the Elasticsearch Output section in the filebeat.yml file and indicates which settings need to be obtained from Cortex XDR.

        # ---------------------------- Elasticsearch Output ----------------------------
        output.elasticsearch:  
          enabled: true  
          # Array of hosts to connect to.    
          hosts: ["OBTAIN THIS URL FROM CORTEX XDR"]  
          # Protocol - either `http` (default) or `https`.  
          protocol: "https"  
          compression_level: 5  
          # Authentication credentials - either API key or username/password. 
          api_key: "OBTAIN THIS KEY FROM CORTEX XDR"
      • Processors—Set the tokenizer and add a drop_event processor to drop all events that do not start with an event ID. The code below provides an example of how to configure the Processors section in the filebeat.yml file and indicates which settings need to be obtained from Cortex XDR.

        Note

        The tokenizer definition is dependent on the Windows server version that you are using as the log format differs.

        -For platforms earlier than Windows Server 2008, use "%{id},%{date},%{time},%{description},%{ipAddress},%{hostName},%{macAddress}"

        -For Windows Server 2008 and 2008 R2, use "%{id},%{date},%{time},%{description},%{ipAddress},%{hostName},%{macAddress},%{userName},%{transactionID},%{qResult},%{probationTime},%{correlationID}"

        For Windows Server 2012 and above, use "%{id},%{date},%{time},%{description},%{ipAddress},%{hostName},%{macAddress},%{userName},%{transactionID},%{qResult},%{probationTime},%{correlationID},%{dhcid},%{vendorClassHex},%{vendorClassASCII},%{userClassHex},%{userClassASCII},%{relayAgentInformation},%{dnsRegError}"

        # ================================= Processors =================================
        processors:  
          - add_host_metadata:      
            when.not.contains.tags: forwarded  
          - drop_event.when.not.regexp.message: "^[0-9]+,.*"  
          - dissect:       
            tokenizer: "%{id},%{date},%{time},%{description},%{ipAddress},%{hostName},%{macAddress},%{userName},%{transactionID},%{qResult},%{probationTime},%{correlationID},%{dhcid},%{vendorClassHex},%{vendorClassASCII},%{userClassHex},%{userClassASCII},%{relayAgentInformation},%{dnsRegError}"  
          - drop_fields:       
            fields: ["message"]  
          - add_locale: ~
          - rename:
              fields:
                - from: "event.timezone"
                  to: "dissect.timezone"
              ignore_missing: true
              fail_on_error: false
          - add_cloud_metadata: ~  
          - add_docker_metadata: ~  
          - add_kubernetes_metadata: ~
  3. Verify the status of the integration.

    Return to the Integrations page and view the statistics for the log collection configuration.

  4. After Cortex XDR begins receiving logs from Windows DHCP via Elasticsearch Filebeat, you can use the XQL Search to search for logs in the new dataset (microsoft_dhcp_raw).