A Journey Towards Building Real-Time Big Data Analytics Environment for Drilling Operations: Challenges and Lessons Learned

Recorded On: 01/14/2021

This webinar will cover the process of building a Real-Time Big Data Analytics environment and highlights key learning and challenges from this endeavor. The process of building such an environment is multifaceted. It involves building a Big Data solution architecture capable of handling the expected workload which includes servers, network, and software architecture. In addition, it requires establishing a data engineering pipeline that enables combining drilling data sets coming from different sources, reformatting and optimizing it to simplify and speed-up analytics. The data engineering pipeline is used to tackle different drilling data types which include a master structured database that describes the drilling planning, configurations, and operations, real-time drilling sensor data stored in WITSML semi-structured format (an E&P specific extension of XML), in addition to other unstructured content such as drilling reports. Data quality procedures were also added to the pipeline in order to identify and handle outliers in the data. In addition, data wrangling procedures were developed to ensure that the data is in the right shape and ready for analytics and machine learning. Moreover, this drilling analytics environment was built while maintaining all business security policies and fine-grained access control rules defined on the original data sources. The established Big Data Analytics environment results in a simpler and a shorter data science lifecycle and thus making it easy to combine, explore and deploy analytical models. This leads to more efficient business operations. Building a successful analytics environment requires much more than the technology piece. It involves many factors such as data ingestion, cleaning, wrangling while maintaining business security policies. This webinar provide a unique perspective on how to build such an environment for real-time drilling operations.

This webinar is categorized under the Drilling technical discipline.

All content contained within this webinar is copyrighted by Abdullah AlBar and its use and/or reproduction outside the portal requires express permission from Abdullah AlBar.

Abdullah AlBar

Content Management GroupLeader, Upstream Database Division

Abdullah AlBar has more than 17 years of experience in delivering IT solutions for the Oil & Gas industry. He holds a BS degree in Computer Science and a Master degree in Software Engineering. During his career he held different roles such as Petroleum Engineer System Analyst and Control System Engineer in Saudi Aramco. He also worked as Senior Consultant for I-Field solutions and Application development consultant for Microsoft. Abdullah is currently leading the Content Management group (CMG) in Upstream Database Division (UDSD) .

SPE Webinars are FREE to members courtesy of the



01/14/2021 at 10:00 AM (EST)   |  90 minutes
01/14/2021 at 10:00 AM (EST)   |  90 minutes
20 Questions
Live and Archive Viewing: 0.15 CEU/1.5 PDH credits and certificate available
Live and Archive Viewing: 0.15 CEU/1.5 PDH credits and certificate available