How Sony LIV built real-time video streaming analytics with AWS | Amazon Web Services
The Shift from Batch to Real-Time: A Strategic Necessity
For years, streaming platforms relied on batch processing, where data logs from user sessions were collected over hours and then analyzed to provide business intelligence. While this approach was sufficient for historical trend analysis, it proved inadequate for the dynamic environment of live sports broadcasting. During a high-profile cricket match, for example, a technical failure at the source or a drop in video bitrate can affect millions of users simultaneously. In a batch-processing model, by the time the data is processed, the event may have already concluded, and the customer experience damaged.
Sony LIV identified three primary drivers for this architectural migration. First, the need for sub-second observability to detect playback errors and buffering issues instantly. Second, the requirement for real-time personalization to keep viewers engaged with relevant content suggestions. Finally, the necessity for cost-efficient scalability, as the platform needed to handle extreme traffic fluctuations without maintaining expensive, idle server capacity during off-peak hours.
The Architecture: Building for Sub-Second Precision
The technical transformation undertaken by the engineering team at Sony LIV involved a multi-layered approach to data ingestion and processing. The architecture, as detailed in recent technical disclosures, utilizes a bifurcated path to handle different types of data traffic.
At the ingestion layer, data from various endpoints—including Smart TVs, mobile applications, and web browsers—is routed through Application Load Balancers to an Amazon Elastic Kubernetes Service (EKS) cluster. This serves as the primary gateway for validation and initial preprocessing. Once validated, the data follows one of two distinct paths:
- The Real-Time Path: This path is reserved for mission-critical event telemetry, such as playback failures, quality-of-service (QoS) signals, and critical user interactions. By routing these events through Amazon Kinesis Data Streams, the platform achieves sub-second processing latency. These events are fed into ClickHouse, an open-source column-oriented database, which allows operations teams to query and visualize metrics as they happen.
- The Batch Path: For high-volume, non-urgent data, the architecture utilizes Amazon Data Firehose to stream events into an Amazon S3 data lake. This raw data is then processed via Amazon EMR using Apache Spark to perform heavy-duty deduplication, schema validation, and complex transformations. The output is stored as Apache Iceberg tables, ensuring ACID compliance and providing a structured, queryable foundation for historical analysis.
The integration of AWS Glue as a central data catalog provides a unified schema across these disparate storage layers. This creates a "single source of truth" that enables the organization to bridge the gap between real-time operational monitoring and long-term business intelligence.
Chronology of the Transformation
The transition to this modern architecture was not instantaneous. It followed a structured roadmap designed to minimize disruption to live broadcasts.
- Q1-Q2 2024: The engineering team began the initial audit of existing batch-processing bottlenecks, identifying the primary pain points in data ingestion and reporting delays.
- Q3 2024: Development of the real-time proof-of-concept began, focusing on Kinesis Data Streams for high-velocity telemetry and ClickHouse for low-latency analytics.
- Q4 2024: Implementation of the unified data layer using AWS Glue and Apache Iceberg, ensuring that the new system could integrate with existing legacy databases while providing a pathway to decommission older batch-only workflows.
- Q1-Q2 2025: Stress testing of the architecture during various live broadcasts, culminating in the full-scale deployment for the Asia Cup 2025, which saw unprecedented concurrent viewership numbers.
Supporting Data and Performance Metrics
The impact of this infrastructure overhaul is most visible during peak-load events. During the Asia Cup 2025, Sony LIV managed millions of concurrent sessions, a feat that would have likely overwhelmed their previous batch-dependent infrastructure. The new system allowed for real-time monitoring of Key Performance Indicators (KPIs) such as buffering rates, video start times, and peak concurrent viewership.
Before this implementation, analytics were often hours old, limiting the response time of the engineering and operations teams. Today, those same teams operate from a single unified dashboard, allowing for "near-instantaneous" intervention. By moving to a serverless-friendly, auto-scaling architecture, Sony LIV has also achieved a significant optimization in operational expenditure (OpEx). The system automatically adjusts its capacity based on demand, eliminating the need to provision for peak traffic at all times.
Official Perspectives and Collaboration
The collaboration between Sony LIV and AWS represents a growing trend in the media and entertainment sector, where cloud service providers are becoming integral to the product lifecycle of streaming applications. Mukund Acharya, a representative from the Sony LIV engineering team, emphasized the necessity of this shift in the face of India’s massive digital consumption growth.
From an industry perspective, this transition signifies a broader shift in the OTT landscape. As global competitors enter the Indian market, the ability to deliver a personalized, high-quality user experience is the primary competitive differentiator. Industry analysts note that real-time analytics is no longer a "nice-to-have" but a fundamental requirement for any platform aiming to achieve long-term subscriber retention.
Future Innovations: The Path Toward Hyper-Personalization
While the current architecture provides a robust foundation for real-time monitoring and historical analysis, Sony LIV is already looking toward the next phase of innovation. The current roadmap focuses heavily on the integration of advanced Machine Learning (ML) models.
By leveraging the data currently residing in the Apache Iceberg-based data lake, the platform intends to deploy models for real-time content recommendation, user churn prediction, and anomaly detection. These initiatives aim to move beyond simple data reporting to predictive intelligence, where the platform can anticipate user behavior and adjust content offerings before the user even initiates a search.
The shift toward unified customer profiles is another critical goal. By combining behavioral data, subscription history, and device metrics, Sony LIV aims to deliver a "hyper-personalized" experience that tailors the interface and content library to the individual preferences of each subscriber.
Broader Implications for the Streaming Industry
The success of Sony LIV’s migration offers a blueprint for other media organizations grappling with the complexities of digital scale. The key takeaway for the broader industry is the effectiveness of a hybrid architecture that balances the speed of real-time streaming with the reliability and cost-efficiency of batch processing.
By decoupling the data ingestion and processing layers from the storage layer, organizations can remain agile, swapping out components or scaling individual parts of the pipeline without needing a complete system overhaul. As the industry continues to mature, the focus will likely remain on reducing the "time-to-insight"—the duration between a user clicking a video and the platform understanding the performance and engagement metrics associated with that action.
In conclusion, the migration to AWS has not only solved the immediate challenges of latency and scalability for Sony LIV but has also provided a flexible, future-proof framework. As the company continues to refine its analytics stack, the ability to turn millions of raw events into actionable business insights will remain the engine driving its growth in one of the world’s most competitive streaming markets. The transition stands as a testament to the power of cloud-native architectures in meeting the demands of the modern, data-driven digital economy.
