In today's U.S. shale fields, tiny sensors attached to production gear harvest data on everything from pumping pressure to the heat and rotational speed of drill bits boring into the rocky earth.
The sensors are leading Big Oil's mining of so-called big data, with some firms envisioning billions of dollars in savings over time by avoiding outages, managing supplies and identifying safety hazards.
The industry has long used sophisticated technologies to find oil and gas. But only recently have oil firms pooled data from across the company for wider operating efficiencies - one of many cost-cutting efforts spurred by the two-year downturn in
crude oil prices.
ConocoPhillips says that sensors scattered across its well fields helped it halve the time it once took to drill new wells in Eagle Ford shale basin of South Texas.
By comparing data from hundreds of sensors, its program automatically adjusts the weight placed on a drill bit and its speed, accelerating the extraction of oil, said Matt Fox,
ConocoPhillips (COP)' executive vice president for strategy, exploration and technology.
It is just one application, but if applied to the more than 3,000 wells ConocoPhillips hopes to drill in the Texas basin, those small sensors could lead to "billions and billions of dollars" in savings, Fox said in an interview.
"We started using data analytics in our Eagle Ford business," he said. "And everywhere we look there are applications for this."
The cost and complexity of such systems vary widely. Oil giants such as ConocoPhillips buy a mix of off-the-shelf and custom programs, along with data repositories. The Houston-based producer's employees use Tibco Software Inc's Spotfire data visualization package to analyze information from well sites.
Tibco declined to discuss its pricing.
Services firms including Schlumberger NV and General Electric Co oil and gas unit sell sensor-equipped gear, data repositories and software to improve producers' decision-making.
Back when oil traded at more than $100 a barrel - before the price crash in 2014 - data analysis was an "afterthought" for most oil firms, said Binu Mathew, who oversees digital products at GE Oil & Gas.
Now - with prices at about $43 a barrel after recovering from a low of about $26 in early 2016 - "the efficiency aspect is far, far more important," Mathew said.
FINDING HIDDEN VALUE
A survey by Ernst & Young last year examined 75 large oil and gas companies and found that 68 percent of them had invested more than $100 million each in data analytics during the past two years. Nearly three quarters of those firms planned to allocate between 6 and 10 percent of their capital budgets to
digital technology, the survey found.
Effectively mining large data sets could lead to supplanting workers with artificial intelligence and machine learning systems, according to firms selling and buying data-driven technology.
Simple sensors already increase safety and savings by eliminating the need to send workers to rigs or production facilities to gather data. Automating drilling decisions can produce more consistent results by cutting out human errors, said Duane Cuku, vice president of sales for rig technology at Precision Drilling Corp.
"The driller is now able to focus his attention on the well - and the performance and safety of his crews - as opposed to the manual manipulation of controls," Cuku said.
Occidental Petroleum Corp also uses an analytical tool to find the best design for hydraulic fracturing wells. A new version of the software analyzes data on well completions and geology to recommend whether injecting steam or water would produce more oil.
Abhishek Gaurav, a petroleum engineer for closely-held Texas Standard Oil, said he uses big-data analytics to help his company choose which properties to explore.
Using Spotfire, the same program utilized by Conoco, Standard applies a combination of data science and petroleum engineering to rank asking prices for land based on a variety of completion, production and geological variables - such as the amount of sand that likely would be required to complete a well in a given formation.
The technique, Gaurav said, has reduced the time needed for evaluating land parcels from weeks to hours - and resulted in better decisions.
"We found value in properties when many other teams did not," he said.
RECRUITING IN CALIFORNIA
Some of the information craved by oil firms isn't so easy to gather or analyze.
Surveys and maps that companies use to acquire acreage for drilling, for instance, are often not digitized. Older company data on wells may be unstructured or spread among suppliers using different storage formats, making integration and analysis a challenge.
General Electric and its oil-and-gas unit are moving aggressively into the business of digitizing industrial equipment for other firms, and have invested in large data processing centers for energy clients.
GE sees huge potential for market growth: A company study estimated that only 3 percent to 5 percent of oil and gas equipment is connected digitally, and less than one percent of the data collected gets used for decision-making, the study found.
Getting the industry more fully connected will take time.
"There is a huge amount of data prep, data sanitization and data extraction needed for big data to be totally disruptive," said Kate Richard, chief executive at private equity investor Warwick Energy.
She projects a major payoff from the technology is still five or ten years away.
Oklahoma City-based Warwick - which manages interests in thousands of wells across Oklahoma and Texas - is preparing for that payoff by hiring people from tech hubs in California, Richard said.
"They all have computer programming and data science backgrounds," she said.
Reporting by Swetha Gopinath and Liz Hampton