Palo Alto Networks recognizes the immense responsibility entrusted to us in protecting our customers' critical environments. Our solutions operate at the core of their infrastructure, requiring seamless integration with minimal disruption. With years of experience deploying security updates and feature enhancements at scale, we have refined rigorous processes to ensure reliability, stability, and respect for our customers’ operations.
This document outlines the key safeguards and best practices we employ to balance the urgency of delivering timely updates with the uncompromising need for operational continuity.
Cortex Core Deployment Components: Product Updates and Content Updates
Product Updates
Product updates include new code delivered to customer environments. These updates include new capabilities, fixes to existing capabilities, and known issues.
Major product code deployments are offered in 3-month cycles, both in the cloud and the Cortex Agent. Minor product code deployments are released in a 6-8-week cycle.
All major and minor releases follow the same strict testing.
Product Quality Assurance
Internal testing includes operational, regression, and stress testing.
Product Deployment Phases
A five-layer ring deployment, with several weeks intervals between every layer.
Internal Deployments
Early Adopters
First Customer Ring (10%)
Second Customer Ring (45%)
Third Sensitive Customer Ring (100%)
*In minor releases - four-ring deployment plan, removing the second ring.
Product Deployment Protection Layers (Cortex Agent)
Beyond the layers mentioned above, the roll-out of an Agent version includes pre-built mechanisms that ensure a gradual and monitored release. With each deployment, customers can choose a different scoped setting for manual or fully automated agent upgrades (see agent updates guidelines).
For each automatic setting, regardless of scope, each deployment is capped within the first week of release to a limited number of upgrades, ensuring a smaller ring is in place before the upgrade is fully implemented.
Product Deployment Monitoring
Agent deployment is closely monitored to validate its success. Agent health signals are automatically validated alongside proactive hunting for abnormal behaviors. Dedicated teams ensure that our Cortex engineers investigate any signs of unexpected behavior and address them in a timely manner.
Product Rollback Scenarios
In rare cases of severe issues, Palo Alto Networks has multiple mechanisms in place to ensure impact reduction and remediation. These steps are optional and are applied according to the issue's severity:
Version removal
Version re-use prevention- This mechanism allows PANW to prevent new Agents from registering with a faulty version. It also applies to installers that are already in use.
In-Product Notifications - Detailing the impact and next steps.
Feature Introduction
In addition to product release deployment methods, sensitive features can be introduced gradually throughout multiple releases to ensure minimized operational impact. A gradual rollout could begin with a closed beta or feature flags, gradually introducing the release to the customer over time. These mechanisms allow Palo Alto Networks to ensure closed monitoring and contained impact models.
Content Updates
Content updates include a set of configurations deployed every week into the Cortex Agent. These configurations include:
Detection rules and logic
Capability settings (i.e., enablement, triggers, priority, etc.)
Operational settings - (i.e., disk quota allocation, etc.)
Compatibility settings (enabling and disabling capabilities based on local environments)
Content Quality Assurance
Content is subject to rigorous testing, with continuous automation run against anonymized data to detect false positives, performance issues, and regressions. All new content rules are introduced in ‘silent mode,’ allowing metric collection only. Silent rules are continuously monitored for quality and impact and are only included in a formal content release once they meet predefined thresholds and satisfy all validation KPIs.
Content Deployment Phases
A3-layer Ring Deployment
Internal Deployment
Staging Ring- 10% of endpoints
GA
*Urgent releases for emergency fixes or coverage for high-profile attacks may change the above process, with executive approval only.
* Silent Rules are continuously Deployed at all times
Content Deployment Configuration and Protection
Content updates are delivered automatically every week. Content deployment onto Agents is fully configurable.
Content Deployment options:
Staging Content - Configure a test environment to receive the staging content
Immediate - Configure updates immediately upon release by PANW
Delayed - Allow customers to define a delay period of up to 30 days post-release
Disabled
Content Deployment Monitoring
Content deployment is constantly monitored to track adoption and stability across Cortex Agents. Automatic validations are set against Agent metrics to ensure minimized impact. Dedicated teams within Palo Alto Networks ensure that content updates and system health are proactively monitored, engaging additional teams as needed.
Product Rollbackֿ / Remediation Scenarios
Remediation scenarios:
Content version removal - Blocks further use
Minor Releases - Allowing immediate rollout to agents configured to include them
Updated Version- With every release, a new version is created, ready to include additional fixes, and deployed within minutes.