Machine Learning

How GS Caltex Streamlined MLOps and Slashed Deployment Times Using Amazon SageMaker

South Korean energy and chemical conglomerate GS Caltex has fundamentally transformed its machine learning operations (MLOps) by bridging the gap between no-code model development and enterprise-grade production workflows. Utilizing Amazon SageMaker Canvas and Amazon SageMaker Pipelines, the company has successfully slashed model deployment timelines from an entire week down to just a few hours. This architectural evolution highlights a broader industry shift toward collaborative, cross-functional artificial intelligence development, empowering domain experts—such as process engineers—to build predictive models without writing code, while allowing machine learning engineers to focus on scalable deployment, scheduled execution, and robust monitoring.

Background Context and Organizational Challenges

In heavy industries like energy refining and chemical manufacturing, operational efficiency is deeply tied to the ability to forecast complex physical and chemical processes accurately. Traditionally, deploying a machine learning model in these sectors required a cumbersome, manual handoff between domain specialists and technical developers. Process engineers possessed the vital, granular knowledge of variables, physical boundaries, and operational constraints, yet they frequently lacked the software engineering proficiency required to push models into production. Conversely, machine learning engineers understood cloud architecture and orchestration but spent valuable weeks translating experimental notebooks into production-ready code, configuring data pipelines, and setting up infrastructure.

Before adopting this streamlined MLOps framework, GS Caltex faced a persistent operational bottleneck. Model development, data preprocessing verification, and validation occurred in silos. Once a process engineer successfully built a model, machine learning teams had to manually reconstruct data pathways, recreate preprocessing steps, and manually configure deployment scripts. This process was not only prone to human error and discrepancies between training and inference environments, but it also severely limited the frequency at which models could be updated or integrated into daily decision-making processes. Recognizing the need for agility and structural efficiency, GS Caltex sought a unified architecture that could seamlessly reuse validated assets from visual development environments directly within automated enterprise pipelines.

The Technical Architecture: Bridging SageMaker Canvas and Pipelines

The solution implemented by GS Caltex leverages the strengths of two distinct AWS services: Amazon SageMaker Canvas, which allows users to build and evaluate models visually, and Amazon SageMaker Pipelines, which provides serverless workflow orchestration for end-to-end machine learning automation.

Under the new operational paradigm, the workflow is divided into two distinct yet interconnected paths: preparing a new model version for deployment and executing scheduled batch inferences. First, process engineers utilize SageMaker Canvas and Data Wrangler to visually prepare data, build predictive models, and validate results based on their profound domain expertise. Once validated, these models—along with their associated preprocessing flows (packaged as .flow files) and evaluation metrics—are registered within the Amazon SageMaker Model Registry.

Production MLOps at GS Caltex with Amazon SageMaker Canvas, part 1 | Amazon Web Services

Instead of forcing machine learning engineers to rewrite these assets from scratch, the architecture introduces an automated handoff mechanism. Machine learning engineers pull the preprocessing flows stored in Amazon Simple Storage Service (Amazon S3) and the model packages from the Model Registry, configuring them directly into a multi-step SageMaker Pipelines workflow.

This pipeline consists of five core sequential steps:

  1. PrepareInput: Automatically assesses incoming inference data and introduces placeholder target columns if schema discrepancies exist between training and inference datasets.
  2. PreprocessNewData: Executes the exact Data Wrangler .flow file created by the process engineer via Amazon SageMaker Processing, ensuring that data transformations remain identical to those validated during development.
  3. DropTargetColumn: Cleans the preprocessed data by removing unnecessary headers and placeholder columns, formatting the output strictly for the machine learning model.
  4. BatchPredict: Leverages SageMaker Batch Transform using an approved model version to generate predictions efficiently without necessitating a continuously running, cost-prohibitive real-time inference endpoint.
  5. AssembleResults: Merges the preprocessed feature data with the newly generated predictions by row position, producing a clean, unified result file stored securely in Amazon S3.

Operational Efficiency and Cost Optimization

One of the most profound impacts of this architectural transformation is the dramatic reduction in deployment latency. GS Caltex reports that machine learning engineers can now deploy fully validated models within a matter of hours, representing an 85% to 90% reduction compared to the previous week-long timeline. This rapid turnaround allows the company to iterate faster, adapt to shifting operational variables in real-time, and deploy updates with minimal friction.

Furthermore, the integration of Amazon EventBridge has enabled GS Caltex to automate batch inference executions precisely every 30 minutes. By opting for a scheduled batch transformation model rather than maintaining persistent real-time endpoints, the company has significantly lowered its cloud compute costs. Persistent endpoints require continuous provisioning and constant uptime, incurring ongoing expenses regardless of request volume. In contrast, SageMaker Pipelines provisions compute resources dynamically at the exact moment of execution and releases them immediately upon completion. Running 48 scheduled batch runs per day ensures high-frequency operational insights while consuming compute resources only when active processing is underway.

Cross-Functional Impact and Cultural Shift

Beyond the technical and financial metrics, the adoption of this MLOps framework has catalyzed a cultural shift within GS Caltex. Empowered by the intuitive interface of SageMaker Canvas, process engineers are no longer passive recipients of IT solutions; they are active creators of predictive tools. Witnessing their models successfully operating in production and driving business value has inspired domain experts to propose innovative machine learning applications across other areas of the refinery and chemical plants.

Meanwhile, machine learning engineers have been liberated from the tedious, repetitive cycle of manual integration work. Rather than spending days debugging pipeline configurations or reconciling training-versus-inference data schemas, engineering teams can now focus on high-value tasks such as pipeline monitoring, governance, anomaly detection, security compliance, and architectural optimization.

Production MLOps at GS Caltex with Amazon SageMaker Canvas, part 1 | Amazon Web Services

Fact-Based Analysis of Broader Industry Implications

The success of GS Caltex’s integration model offers a blueprint for heavy industries undergoing digital transformation. Historically, industrial sectors have struggled to bridge the "pilot purgatory"—the phase where promising artificial intelligence proofs-of-concept fail to scale into production environments due to cultural friction and technical complexity. By establishing a formalized handoff protocol between low-code/no-code development tools and serverless orchestration pipelines, organizations can democratize AI creation while maintaining rigorous enterprise governance.

The strategy of decoupling model registration from frequent batch inference runs also highlights a maturing approach to cloud economics. As enterprises face macroeconomic pressures to optimize cloud expenditures, architectures that favor serverless orchestration and ephemeral compute over perpetual resource allocation will likely become the gold standard for batch-oriented analytical workloads.

Future Outlook and Conclusion

GS Caltex’s implementation demonstrates that the democratization of artificial intelligence does not require compromising on enterprise reliability or scalability. By leveraging Amazon SageMaker Canvas for visual model development and Amazon SageMaker Pipelines for robust, scheduled execution, the South Korean energy leader has successfully minimized deployment times, reduced infrastructure costs, and fostered a collaborative ecosystem between domain specialists and core engineering teams.

As GS Caltex continues to refine its digital infrastructure—with subsequent initiatives exploring generative artificial intelligence tools like Amazon Bedrock to further streamline model migration—the company stands as a compelling case study in modern industrial MLOps maturity. Organizations looking to replicate this success are encouraged to adopt a phased rollout strategy: starting with a single validated model in a test environment, standardizing the handoff mechanisms, and progressively scaling automated workflows across the broader enterprise.

Related Articles

Back to top button
Whatvis
Privacy Overview

This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.