Klaviyo Scales Massive Event Pipeline with Amazon MSK Express to Meet Record-Breaking Black Friday Demand

Klaviyo, the B2C customer relationship management (CRM) powerhouse, has successfully navigated the logistical gauntlet of the 2025 holiday shopping season by transitioning its mission-critical event streaming infrastructure to Amazon Managed Streaming for Apache Kafka (Amazon MSK) Express brokers. As an enterprise supporting over 200,000 global brands, Klaviyo manages an intake of approximately 3 billion events per day—a volume that spans everything from simple email engagement metrics to complex transactional purchase completions. The migration from a legacy RabbitMQ-based architecture to a modern, elastic Kafka-based backbone represents a significant pivot in how high-growth technology companies approach seasonal scalability and operational efficiency.
The Evolution of Event-Driven Architecture
For years, Klaviyo relied on RabbitMQ to manage the flow of customer data. However, as the platform’s user base expanded, the limitations of this legacy infrastructure became increasingly apparent. Managing steady-state traffic is one hurdle, but the unique requirements of the e-commerce sector—specifically the massive spikes associated with Black Friday and Cyber Monday (BFCM)—demanded a more resilient, self-healing, and elastic solution.
The engineering mandate was clear: the platform needed to sustain a baseline of 170,000 events per second while possessing the agility to absorb a 7x spike in traffic during peak retail hours. Traditional infrastructure often requires massive over-provisioning to survive such events, leading to idle resources and wasted capital. Klaviyo sought a architecture that could not only handle these bursts but also scale down immediately after the rush, minimizing operational overhead for its lean engineering team.
Chronology of the 2025 Scaling Strategy
The preparation for the 2025 holiday season began long before the first retail advertisements hit the airwaves. Klaviyo’s infrastructure team executed a multi-phased approach to ensure the event gateway could handle the anticipated volume:
- Steady-State Baseline (Q1–Q3 2025): Operating with a fleet of 12 brokers, the system maintained a consistent throughput of roughly 3 billion events daily, providing a stable foundation for testing the new MSK Express implementation.
- Early Capacity Expansion (October 2025): Recognizing the impending seasonal surge, the team expanded the fleet to 18 brokers. This preemptive move allowed for stress testing and validation of the new orchestration logic under elevated load.
- The BFCM Surge (Late October–November 2025): During the critical window of Black Friday and Cyber Monday, the cluster was scaled to 60 brokers. This quintupling of capacity was executed seamlessly, allowing the system to handle 22.7 billion messages over the period—a 25 percent increase over the previous year.
- Post-Season Normalization (December 2025): Immediately following the peak, the team scaled the infrastructure back down to 18 brokers, effectively rightsizing the cost structure and proving the agility of the managed service model.
Technical Innovations in Priority-Based Processing
One of the most significant architectural advancements implemented during this migration was the adoption of a unified, priority-based event processing model. In legacy systems, different event types often required isolated infrastructure, which is both costly and difficult to maintain. Klaviyo’s new design utilizes a gRPC-based publisher proxy that intelligently routes traffic into distinct priority tiers: p10, p20, and p30.

High-priority events, such as purchase confirmations and transactional flow triggers, are routed for immediate processing. Lower-priority data, such as batch-processed analytics or secondary user behavior logs, can sustain minor latency if the system encounters a bottleneck. By consolidating these disparate streams onto a single, unified processor set, Klaviyo reported a 30 percent reduction in operational costs. This consolidation improved resource utilization, ensuring that compute power is always directed where it is most needed during periods of extreme load.
The Role of Amazon MSK Express
The shift to Amazon MSK Express was not merely a change in messaging software; it was a fundamental shift in infrastructure management. The decision to adopt Express brokers was driven by three core technical advantages that directly addressed the team’s need for operational simplicity:
- Elimination of Storage Provisioning: Unlike traditional Kafka configurations that require manual planning for disk space, Express brokers abstract storage management entirely. By utilizing a pay-as-you-go model for data ingestion and storage, Klaviyo bypassed the need for tedious capacity planning, allowing the system to expand in response to actual traffic rather than speculative estimates.
- Intelligent Rebalancing: In standard Kafka environments, adding brokers to a cluster is a notoriously manual and high-risk task that requires re-distributing partition data across the new nodes. MSK Express automates this, with rebalancing speeds reported to be 180 times faster than standard brokers. This capability allowed Klaviyo to scale its fleet for the holiday season without requiring an engineer to monitor partition migration 24/7.
- High-Availability Enhancements: According to internal data provided by AWS, Express brokers offer a 90 percent faster recovery time from node failures compared to standard configurations. By automating the management of critical topic and cluster configurations, the service effectively offloads the "undifferentiated heavy lifting" of cluster maintenance, enabling Klaviyo’s engineers to focus on product features rather than uptime alerts.
Resilience and Data Integrity
In a system processing billions of events, data loss is not an option. Klaviyo’s new architecture features a multi-layered failure capture system. Should a publisher fail to reach the primary Kafka cluster, the system employs several fallback mechanisms, including secondary retries and persistent storage in Amazon S3 for later replay. During a recent three-month observation period, the system successfully recovered approximately 20,000 events without human intervention—a testament to the robustness of the "dual failure capture" design. Furthermore, the ability to retain seven days of event history ensures that even in the event of a downstream system crash, data can be replayed to maintain consistency.
Broader Implications for the CRM Industry
Klaviyo’s successful migration serves as a case study for the broader software-as-a-service (SaaS) industry. The trend toward managed streaming services signals a departure from self-managed, capital-intensive infrastructure toward lean, service-oriented architectures. By reducing the operational burden on internal teams, companies can effectively "buy" reliability through cloud-native services rather than building and maintaining it in-house.
The implications for the CRM and marketing automation sector are significant. As brands become increasingly reliant on real-time personalization, the ability to ingest and react to customer behavior instantaneously is a competitive necessity. Klaviyo’s infrastructure now supports not only the event pipeline but also its Change Data Capture (CDC) processes, with potential plans to migrate its data lake workloads to MSK in the future.
Conclusion
The successful scaling of the Klaviyo event pipeline during the 2025 holiday season highlights the maturity of managed streaming infrastructure. By leveraging Amazon MSK Express, Klaviyo successfully navigated a 25 percent year-over-year increase in event volume while simultaneously lowering its operational footprint. For high-growth organizations, the lesson is clear: when the stakes involve millions of customer interactions during the most critical days of the fiscal year, the ability to automate infrastructure elasticity is the defining characteristic of a resilient technical organization. As Klaviyo continues to expand its feature set, the scalability of this event-driven architecture will likely remain the bedrock of its platform’s reliability and performance.







