Alight Solutions Achieves 55 Percent Cost Reduction and Enhanced Scalability Through Migration to Amazon OpenSearch Service

Alight Solutions, a global leader in cloud-based human capital technology and services, has successfully completed a massive migration of its logging infrastructure from a self-managed Elastic Stack to Amazon OpenSearch Service. This strategic move, conducted in partnership with Amazon Web Services (AWS), has resulted in an immediate 55% reduction in monthly infrastructure and licensing costs while eliminating approximately 2,000 hours of manual operational overhead annually. The transition addresses a critical bottleneck for Alight, whose technology stack generates more than 1 billion log records daily across a complex, containerized microservices architecture. At peak periods, such as annual health insurance enrollment windows, the system must process upwards of 100,000 records per second, a volume that had previously pushed their self-managed infrastructure to its breaking point.

The Operational Crisis: Scaling Challenges and Technical Debt

Since 2018, Alight had relied on a self-managed deployment of the Elastic Stack, comprising Elasticsearch, Logstash, and Kibana. While this environment initially met the company’s needs, the rapid growth of Alight’s flagship "Alight Worklife" platform created a compounding set of technical and financial challenges. By late 2024, the production environment had swelled to 15 Elasticsearch nodes supported by 168 TB of Amazon Elastic Block Store (EBS) storage.

The financial burden was significant, with Alight spending over $100,000 per month on infrastructure and Elastic Platinum licensing across its various environments. However, the cost was not merely monetary. The operational team found its entire budget and bandwidth consumed by routine maintenance, leaving no room for innovation or the adoption of advanced features included in their expensive licensing tier.

Furthermore, the aging Elasticsearch 7.x infrastructure was approaching its end-of-support (EoS) date. This looming deadline, combined with frequent "P1" (Priority 1) incidents caused by Logstash backpressure—which led to critical data loss and system instability—forced Alight’s leadership to seek a more sustainable, managed solution. The objective was clear: move away from the "undifferentiated heavy lifting" of managing clusters and toward a platform that offered automated scaling, integrated security, and predictable costs.

Evaluating the Path Forward: Why Amazon OpenSearch Service?

Alight’s evaluation process was rigorous, focusing on five primary factors: operational efficiency, cost-effectiveness, security integration, scalability, and performance. While the company considered several market alternatives, Amazon OpenSearch Service emerged as the preferred choice due to its status as a fully managed AWS service.

The decision was driven by the service’s ability to automate node replacement, patch software without downtime, and scale resources through simple API calls. Critically, the migration allowed Alight to move from a fixed, expensive licensing model to a pay-as-you-go structure with zero separate licensing fees. This shift alone promised a significant reduction in the Total Cost of Ownership (TCO).

Security also played a pivotal role in the selection. Alight required a solution that could integrate seamlessly with its existing identity management framework. Amazon OpenSearch Service’s support for AWS IAM Identity Center and System for Cross-domain Identity Management (SCIM) allowed Alight to maintain its "least privilege" security posture while providing single sign-on (SSO) access to its engineering teams.

A Seven-Month Chronology: The Migration Roadmap

The migration process was executed over a seven-month period, beginning in February 2025 and concluding in August of the same year. This deliberate timeline allowed the team to migrate five major applications, including the mission-critical Alight Worklife system, without disrupting services for millions of global users.

The journey began with a pilot phase where Alight enabled OpenSearch Service for two smaller, lower-risk applications. This allowed the engineering team to establish operational baselines and refine their Infrastructure as Code (IaC) templates using Terraform. By templatizing the onboarding process, Alight reduced the time required to integrate a new application from 80–120 hours down to a mere 4–8 hours.

A key component of the strategy was the "parallel run" methodology. For each application, logs were written to both the legacy Elasticsearch cluster and the new OpenSearch environment simultaneously. This approach served as a fail-safe, allowing the team to fine-tune ingestion pipelines and validate data integrity before a full cutover. To handle historical data, Alight migrated the most recent 30 days of live logs into OpenSearch, while archiving older records in Amazon S3 buckets. This ensured that while the primary "hot" data was readily available for analysis, older data could be re-ingested on demand, further optimizing storage costs.

Technical Architecture: Building for Billion-Record Scale

The new architecture leverages a cross-account model designed for isolation and security. Alight utilizes "Application Accounts" where workloads run on Amazon Elastic Container Service (ECS) and Amazon EC2, and a centralized "Log Account" that houses the Amazon OpenSearch Service domains.

To solve the historical issues with Logstash backpressure, Alight replaced the legacy ingestion layer with Amazon OpenSearch Ingestion (OSIS) and AWS FireLens. FireLens, utilizing a Fluent Bit sidecar, now collects logs from ECS containers. A critical innovation in this setup is the use of Amazon Elastic File System (EFS) to provide persistent filesystem buffering for the Fluent Bit sidecars. This ensures that if the ingestion pipeline experiences a transient failure, logs are buffered on a persistent disk rather than in volatile memory, preventing the data loss that had characterized previous P1 incidents.

For the production workload, Alight deployed a high-performance cluster configuration:

• Data Nodes: 18 im4gn.2xlarge.search instances, utilizing AWS Graviton2 processors for superior price-performance.
• Storage Tiers: A "Hot" tier providing 25 TB of high-speed storage and an "UltraWarm" tier providing 180 TB of cost-optimized storage.
• Capacity: The system manages between 100–105 TB of total data, including replicas.

The use of UltraWarm nodes is a cornerstone of Alight’s cost-saving strategy. UltraWarm uses Amazon S3 for backing storage, allowing Alight to retain massive amounts of data at a fraction of the cost of traditional EBS-backed data nodes while maintaining interactive query performance.

Data-Driven Results: Cost, Performance, and Operational Gains

The migration has delivered transformative results across every key performance indicator (KPI). The most immediate impact was financial; by decommissioning the self-managed infrastructure and eliminating Elastic Platinum licensing, Alight achieved a 55% reduction in monthly costs. As the remaining legacy clusters are phased out, the company expects total savings to reach 65%.

Operationally, the burden on Alight’s engineering staff has been virtually eliminated. The 2,000 hours per year previously spent on manual patching, node replacement, and cluster rebalancing—roughly equivalent to one full-time employee (FTE)—have been reclaimed. These resources are now redirected toward high-value activities, such as improving application performance and developing new features for the Alight Worklife platform.

From a reliability perspective, the new system has maintained a perfect record. Since the migration, Alight has reported zero P1 incidents related to the logging subsystem. The automated nature of the managed service means that security vulnerability patching, which previously required manual intervention and often took place during holidays or off-hours, is now handled automatically by AWS.

Analysis of Implications and Industry Context

Alight’s successful transition mirrors a broader trend in the enterprise technology sector: the move from "Day 2" operational management to cloud-native managed services. As microservices architectures become more complex, the volume of telemetry data (logs, metrics, and traces) grows exponentially. For many organizations, the cost and effort of maintaining the "observability stack" can begin to rival the cost of the applications themselves.

By adopting Amazon OpenSearch Service, Alight has effectively decoupled its ability to scale from its operational headcount. The use of Graviton-based instances and UltraWarm storage demonstrates a sophisticated approach to cloud economics, where performance is not sacrificed for cost savings. Furthermore, the integration of OSIS highlights the industry’s shift toward serverless ingestion, where the infrastructure automatically adjusts to handle bursts of data without manual tuning.

Mark Larson, Enterprise Architect at Alight, emphasized the strategic importance of this shift, noting that effective application logging is critical for analyzing system behaviors and troubleshooting. He characterized the ability to reconfigure and upgrade domains with zero downtime as a "game changer" for the organization.

Future Outlook: AI and Advanced Observability

With the foundational logging infrastructure now stabilized and optimized, Alight is looking toward the future. The company plans to leverage the advanced features of OpenSearch that were previously out of reach due to operational constraints.

Immediate priorities include the implementation of OpenSearch Anomaly Detection, which uses machine learning to identify unusual patterns in log data in real-time, allowing for proactive incident response. Additionally, Alight is exploring the integration of AI-powered log analytics to assist engineers in root-cause analysis. Other planned initiatives include expanding the use of OpenSearch Ingestion’s persistent buffering and implementing "Index State Management" (ISM) to further automate the lifecycle of data from hot to warm to cold storage.

Ultimately, Alight’s migration serves as a blueprint for other large-scale enterprises facing the dual pressures of rising data volumes and constrained operational budgets. By embracing managed services and cloud-native architectures, Alight has not only reduced its costs but has built a resilient, future-proof foundation for its global human capital services.