dedup - Learn more about the Cortex Query Language dedup stage that removes duplicate occurrences of field values. - Administrator Guide - Cortex XDR - Cortex - Security Operations

dedup <field1>[,<field2>, ...] by asc | desc <field>

The dedup stage removes all records that contain duplicate values (or duplicate sets of values) from the result set. The record that is returned is identified by the by clause, which selects the record by either the first or last occurance of the field specified in this clause.

Return unique values for the actor_primary_username field. For any given field value, return the first chronologically occurring record.

dataset = xdr_data 
| fields actor_primary_username as apu 
| filter apu != null 
| dedup apu by asc _time

Return the last chronologically occurring record for any given actor_primary_username value.

dataset = xdr_data 
| fields actor_primary_username as apu 
| filter apu != null 
| dedup apu by desc _time

Return the first occurrence seen by for any given actor_primary_username. field value.

dataset = xdr_data 
| fields actor_primary_username as apu 
| filter apu != null 
| dedup apu by asc apu

Return unique groups of actor_primary_username and os_actor_primary_username field values. For each unique grouping, return the pair that first appears on a record with a non-NULL action_file_size field.

dataset = xdr_data 
| fields actor_primary_username as apu, 
         os_actor_primary_username as oapu, 
         action_file_size as afs 
| filter apu != null and afs != null 
| dedup apu, oapu by asc afs