Delivery Hero Transforms Global Advertising Infrastructure with Real-Time Data Pipeline Migration to Amazon Managed Service for Apache Flink

Delivery Hero, the Berlin-based global leader in local delivery platforms, has successfully completed a comprehensive overhaul of its advertising measurement architecture, transitioning from a legacy batch-processing system to a real-time streaming pipeline powered by Amazon Managed Service for Apache Flink. This technological shift, executed in collaboration with a team of lead engineers including Kirill Tishenkov, Alexandru Pisarenco, Upendra Kambhampati, and Sabariesh Ganesan, marks a significant evolution in how the company manages the complex data flow generated by its operations in over 65 countries.

Operating at a scale that spans Asia, Europe, Latin America, the Middle East, and North Africa, Delivery Hero facilitates millions of daily orders for food, groceries, and essential goods. At the heart of this logistical network is a sophisticated advertising platform that connects over 1.5 million restaurant partners and local vendors with a massive, active consumer base. In 2025, this advertising arm generated approximately EUR 1.5 billion in revenue, a figure that underscores the commercial imperative of ensuring precise billing and high-fidelity data attribution.

The Technical Limitations of Legacy Batch Processing

Prior to the migration, Delivery Hero’s advertising measurement relied on a system that ingested impression, click, and order events from message queues, processed them via synchronous API calls, and finalized metrics in hourly batch cycles. While this architecture was sufficient during the early stages of the platform’s growth, it encountered critical failures as traffic volume soared to billions of events per day.

The legacy system suffered from five structural weaknesses that constrained the business. First, the lack of event-time semantics meant the system relied on processing time. This led to significant data skew when ingestion lagged or events arrived out of sequence, complicating the calculation of metrics like Return on Ad Spend (ROAS). Furthermore, the 61-minute average latency between an event occurring and its registration made real-time budget pacing virtually impossible.

Second, the reliance on synchronous API calls for data enrichment—attaching campaign metadata and product information to raw events—created a performance bottleneck. During traffic spikes, such as major sales events, these blocking calls exhausted connection pools, causing cascading failures across the billing and ad-serving layers.

Third, the database infrastructure was ill-suited for the workload. Originally designed for document storage, the NoSQL database struggled with the continuous deduplication and multi-day attribution lookups required for modern ad measurement. This resulted in significant read/write amplification, where storage costs scaled linearly with query latency, eventually leading to recurring production outages during peak demand.

Fourth, the system lacked a native mechanism for event reprocessing. Recovery from data corruption or logic errors required manual intervention, custom CLI scripts, and disparate tools for different consuming systems, creating a significant coordination burden across engineering teams. Finally, the "best-effort" nature of synchronous enrichment left substantial gaps in data quality, with millions of impressions arriving without reliable timestamps, which subsequently compromised the accuracy of machine learning models used for campaign ranking and conversion estimation.

The Path to Real-Time Streaming: Architecture and Implementation

To address these systemic issues, Delivery Hero’s engineering team defined a new, non-negotiable set of requirements: fault-tolerant processing to eliminate data loss, stateful stream processing capable of handling multi-day interaction history, and a fully managed infrastructure to reduce the operational burden on engineering staff.

The selection of Amazon Managed Service for Apache Flink was central to the solution. By leveraging Flink’s native event-time watermark model, the team gained the ability to accurately order late-arriving data. To solve the state management challenges that plagued the legacy system, the team utilized Flink’s RocksDB state backend, which allows for large keyed state storage on disk without placing excessive pressure on the Java Virtual Machine (JVM) heap.

The architecture was redesigned using a tiered approach. Amazon Kinesis Data Streams replaced the previous buffering mechanisms to serve as the upstream event bus, offering superior cost efficiency at scale. For enrichment, the team moved away from synchronous blocking calls, opting instead for Flink’s Async I/O API to query Amazon DynamoDB. This decoupling allowed the enrichment process to scale independently of the ingestion layer.

The production pipeline, now fully operational, follows a five-stage process: event ingestion and decryption, deduplication, enrichment from reference data, attribution, and finally, aggregation and emission. Granular events are persisted in Avro format on Amazon Simple Storage Service (S3), which also serves as the checkpoint store for Flink’s state. This setup ensures that in the event of a system failure, the pipeline can restore its state precisely from the last checkpoint, achieving effectively "exactly-once" billing, even across a distributed transport layer.

Quantifiable Performance Gains and Operational Impact

The transition to a real-time model has yielded measurable improvements across every key performance indicator. The most notable metric is the reduction in processing latency, which dropped from 61 minutes to just 1.2 seconds. This improvement enables the advertising platform to act on real-time signals, significantly enhancing the precision of budget pacing and campaign optimization.

Financial efficiency also improved dramatically. By optimizing the architecture and moving to a fully managed cloud service, Delivery Hero reduced its monthly operational costs by approximately 57 percent. This reduction was achieved alongside a marked increase in system reliability, as the new infrastructure eliminated the "fragility" of the previous synchronous chain.

Data quality metrics showed the most striking improvement. The rate of missing session data, which previously fluctuated between 30 and 40 percent, was reduced to 0 percent. Similarly, missing customer IDs fell from 5 percent to 0.8 percent, and missing impression timestamps—a major issue at 91 percent in the legacy system—dropped to a negligible 0.2 percent. These gains have provided the company’s machine learning models with a clean, comprehensive data stream, resulting in more accurate conversion-rate estimations and better-performing campaign ranking algorithms.

Industry Implications and Future Roadmap

The success of this migration offers a blueprint for other high-scale advertising platforms currently struggling with the limitations of batch-based processing. The integration of Amazon EventBridge Pipes for rapid deployment and the strategic use of S3 for durable, replayable event logs highlights a broader trend in the industry: the move toward "data as a first-class citizen," where historical logs are treated as an active asset rather than a secondary artifact.

According to the engineering team at Delivery Hero, this architecture is only the beginning. The platform is already planning to expand the real-time pipeline to accommodate additional ad formats and is exploring the integration of Flink-based aggregations directly into the ad-serving layer. Furthermore, the team intends to leverage this high-speed data foundation for advanced artificial intelligence initiatives, including live user personalization and grounded Large Language Model (LLM) recommendations—applications that were previously impractical under the limitations of a batch-oriented environment.

The shift at Delivery Hero serves as a compelling case study for companies navigating the complexities of digital advertising at a global scale. By reconciling the competing demands of billing accuracy, stateful attribution, and cost-effective scalability, the company has established a robust framework that is not only resilient to current traffic demands but is also prepared for the next generation of AI-driven consumer interactions. As digital marketplaces continue to mature, the transition from periodic reconciliation to continuous, real-time measurement is increasingly becoming the new industry standard.