Midstream Natural Gas Operator Achieves Same-Day Operational Coordination Through Data Systems Consolidation, Eliminating 3-5 Day Delays
At a glance
A midstream energy company’s operational data was scattered across incompatible systems—WolfePak, Quorum TIPS, SAP Datasphere, and Excel—with manual consolidation taking 3-5 days, preventing schedulers and operations teams from coordinating daily natural gas gathering, transportation, and pipeline nominations.
Since March 2024, NMD has worked continuously with the client across five major phases to consolidate their fragmented data systems. After a 6-week Data Lakehouse foundation deployment immediately eliminated the 3-5 day delays, work systematically expanded to migrate WolfePak from deprecated SAP Advantage DB, enhance operational workflows through 13 iterative sprints, document and migrate 49 undocumented Quorum TIPS reports (ongoing), and integrate SAP Datasphere field operations (planned)—all while maintaining business continuity throughout.
The client achieved four outcomes: (1) same-day operational coordination, (2) unified scattered data enabling cross-functional visibility, (3) eliminated deprecated platform risk, and (4) built internal platform ownership—preparing the company for acquisition integration, scalable growth across new gathering systems and commodities, and real-time data coordination supporting operational excellence in midstream natural gas operations.
Industry
Use Case
- Workflow & Process Automation
- Inventory & Supply Chain Management
- Analytics & Business Intelligence
- Administrative Process Optimization
Solution implemented
- Data Lakehouse with Unity Catalog enterprise governance providing centralized data access and permissions management...enabling teams to access previously siloed data through unified governance framework
- Multi-source automated pipelines ingesting from WolfePak operational database, Quorum TIPS business intelligence, SAP Datasphere field operations, external natural gas trading feeds, and Excel reference files with Change Data Capture
- Marketing Book curated reports with custom calculations replacing exponentially growing Excel sheets; consolidating data from multiple sources into governed reports imported into Power BI for financial risk analysis and marketing data insights
- Comprehensive data quality dashboard in Power BI actively monitoring data quality metrics across all ingestion pipelines and data sources, providing proactive visibility into data integrity issues
- Custom scheduling coordination application replacing manual Excel consolidation workflows, enabling real-time data entry with validation rules and immediate availability for operational decisions
- Comprehensive validation framework reconciling source system data against consolidated lakehouse, ensuring natural gas volume measurements, contract allocations, and throughput metrics maintain integrity
- Infrastructure as Code deployment ensuring reproducible environments across development, test, and production for safely testing migrations of business-critical systems
- Structured DevOps practices including sprint-based delivery, UAT processes, and tiered SLA support for production systems supporting daily commodity scheduling
The value equation
- Enabled same-day operational coordination at scale, replacing 3-5 day manual consolidation delays with real-time visibility across gathering volumes, contract allocations, and field throughput—critical for daily pipeline nominations and commodity scheduling
- Unified scattered operational data previously siloed across WolfePak, Quorum TIPS, SAP Datasphere, and Excel, enabling cross-functional visibility between schedulers, operations teams, and analysts that was organizationally impossible
- Eliminated operational fragility from deprecated systems, proactively migrating off SAP Advantage DB before business disruption, documenting 49 undocumented Quorum TIPS reports, and replacing Visual Basic macros known by two people
- Built operational agility foundation supporting future growth including acquisition integration, new gathering systems, additional commodities, and business model expansion without platform redesign
- Custom scheduling coordination application replacing manual Excel consolidation workflows, enabling real-time data entry with validation rules and immediate availability for operational decisions
Company Snapshot
This midstream energy company manages natural gas gathering, processing, and transportation operations across multiple regions, handling complex daily scheduling of commodity movements, contract nominations, and field operations coordination requiring real-time data sharing between schedulers, operations teams, and business analysts for pipeline throughput optimization and contract compliance.
Location
United States
Customer Situation
The client operated daily natural gas gathering and transportation operations across fragmented legacy systems preventing the operational coordination their business required. Schedulers needed to coordinate commodity volume commitments with field operations actual throughput across multiple gathering systems and pipeline interconnects—coordination that determined contract compliance, capacity optimization, and throughput efficiency. But the data to enable it was scattered across incompatible systems with multi-day delays.
Operations teams manually consolidated volume data from schedulers across Excel spreadsheets for 3-5 days before information became available for decisions. During those days, field conditions changed, nomination deadlines passed, and contract commitments required adjustments—all while decision-makers lacked current visibility. For midstream natural gas operations where daily pipeline nominations and gathering system throughput require coordination between scheduled volumes and operational capacity, these delays directly impacted contract performance.
WolfePak operational data—gathering volumes, processing throughput, contract details—resided on SAP Advantage DB officially deprecated by the vendor, creating existential business continuity risk. Forty-nine critical business intelligence reports for allocation analysis and commodity optimization existed only in proprietary Quorum TIPS with no documentation of calculation logic. Field operations relied on Visual Basic macros known by two people. Excel spreadsheets served as the “integration layer,” with tribal knowledge about which version contained current contract terms or throughput capacity.
Schedulers couldn’t see current field operations throughput to validate volume commitments. Operations teams couldn’t verify that scheduled volumes matched gathering system capacity until after nominations were submitted. Business analysts had no unified view of system-wide utilization or contract performance because data lived in isolated silos.
In oil and gas midstream operations, daily commodity scheduling determines contract compliance and operational efficiency. Natural gas flows continuously; pipeline nominations occur daily with strict deadlines; gathering systems operate at capacity constraints. Any delay in data coordination means delayed decisions about throughput optimization and contract management. The client needed systematic consolidation of scattered operational data and real-time visibility enabling teams to coordinate daily commodity movements with confidence.
NMD Solution
NMD identified that the client’s core problem was operational coordination bottlenecks caused by data system fragmentation—schedulers, operations teams, and analysts couldn’t execute daily natural gas commodity movements with required speed and visibility because operational data was scattered across incompatible systems with manual consolidation delays.
Enabling this transformation required consolidating operational data from multiple source systems into unified governed infrastructure, creating automated data pipelines replacing manual workflows, and establishing shared real-time visibility into gathering volumes, contract commitments, and operational capacity.
What We Delivered
Over 20 months across five phases, NMD systematically consolidated the client’s fragmented operational data systems while maintaining business continuity.
- Phase 1 (6 weeks) deployed Data Lakehouse infrastructure with automated pipelines ingesting WolfePak operational data, natural gas trading feeds, and Excel reference files, plus a custom scheduling coordination application replacing 3-5 day manual Excel consolidation with same-day visibility.
- Phase 2 (2 weeks) migrated WolfePak from deprecated SAP Advantage DB to SQL Server before vendor sunset, preventing business disruption through comprehensive validation ensuring data integrity.
- Phase 3 (13 sprints) enhanced the scheduling application with non-standard contract management, bulk upload capabilities, custom station grouping, and refined workflows based on operational feedback.
- Phase 4 (ongoing) documented 49 previously undocumented Quorum TIPS allocation and optimization reports—capturing institutional knowledge existing nowhere else—and deployed automated pipelines loading this data into the consolidated lakehouse for unified analysis.
- Phase 5 (planned) will replace legacy AFO system built on Visual Basic macros and stored procedures with modern architecture ingesting SAP Datasphere field operations data, enabling field operations to be joined with scheduled volumes and contract allocations. Throughout all phases, knowledge transfer and DevOps practices ensured the client’s IT team independently owns and extends the platform.
Technical Outcomes
Within 20 months of NMD’s Workflow Automation and Data Integration solutions, the client achieved four strategic outcomes:
Same-Day Operational Coordination
Replaced 3-5 day manual Excel consolidation with same-day visibility enabling schedulers and operations teams to make daily commodity movement decisions based on current data rather than stale spreadsheets.
Unified Scattered Operational Data
Consolidated data previously siloed across WolfePak, Quorum TIPS, SAP Datasphere, and Excel, enabling schedulers to validate volume commitments against field throughput and operations teams to see scheduled nominations against actual gathering system performance.
Eliminated Business Continuity Risk
Successfully migrated off deprecated SAP Advantage DB before vendor sunset, documented 49 previously undocumented Quorum TIPS allocation reports capturing institutional knowledge, and planning replacement of Visual Basic macros known by two people.
Built Platform Extensibility
Created consolidated infrastructure supporting five major phases over 20 months without architectural limitations, proven capable of absorbing new gathering systems from acquisitions and new operational workflows without redesign
Related Case Studies
Vertically Integrated Utility Company
Migration of data systems and applications to AWS.
Ready to Transform Your Oil & Gas Business?
See how New Math Data can transform your Oil & Gas business with AWS-powered innovation.