Big data: challenge and opportunity

Understanding how big data affects the oil and gas industry, and where companies are finding most success.

By Karen Field June 29, 2015

There is no better time for oil and gas companies to embrace the concept of big data analytics. But whether the industry, which some say is already spewing out data like oil gushing from a broken well, can realize the full benefit depends a lot on whether it can change. Steve Van Fleet, CEO of MAST (a subsidiary of Micromem), has been to quite a few customer meetings over the past few years. But there’s one in particular that he vividly remembers: "A woman stood up. She was furious, arms crossed, red faced," he said. "Then she launched into this big tirade about data: ‘We collect terabytes of data now, and we don’t do a damn thing with it. So don’t talk to me about data without telling me how it’s going to help me!’" That interruption stopped Van Fleet midway through a discussion about his company’s MEMS sensor technologies. The oil and gas industry may be lagging behind other industries, but with more and more big Data technologies and tools proven and available, and more successful use cases documented, companies have a huge opportunity to get improve the efficiency of their operations. Source: Datameer

MEMS is an acronym for micro-electromechanical systems. Developed out of its core expertise in magnetic sensing, these tiny biosensors are used in oil and gas applications, such as detecting wear contaminants in lubricating fluids, and identifying the location of hydraulic fractures caused by fracking. "At that moment, I realized our future oil and gas customers were not looking for a technology solution or a data collection solution," said Van Fleet, a former controls engineer who knows what it’s like to be on the customer side. "Maybe they didn’t actually articulate it that way, but what they were asking for was an end-to-end, big data solution that delivers actionable information and solves business problems."

The big data problem

How exactly is it that many in the oil and gas industry wound up trying for big data, but got a big problem with data instead? Mark Lochmann, a consultant who spent 40 years of his career in the oil and gas industry focusing on information technology solutions, has a theory: "The data explosion in oil and gas is happening because we are instrumenting more operations," he said. "Concurrently, we are able to capture and archive all that information pretty cheaply. Unfortunately, it is not growing in a very well-orchestrated fashion." That’s because for most companies, the manager of almost every asset makes his or her own decisions about what to do and how to do it.

"Sure, there’s a corporate structure that they generally adhere to," Lochmann said. "But because almost all of the work processes are manual, each engineer said, ‘Well, I have a problem, and I will solve it my way.’" Lochmann said that the one exception is the exploration space, which has made great strides with big data because it tends to be a corporate process. Some industry observers say that more oil and gas companies are now clamoring for end-to-end solutions-presumably to prevent collecting even more data they don’t know what to do with. An end-to-end solution, in the case of big data, is essentially a package of managed services: The deployment of sensors, data collection and storage, delivery of that data back to the customer in the required format, and just about everything in between-sold as a subscription service akin to a monthly phone bill. The level of interest in managed services today has surged since the days when oil was $110 per barrel.

"The majors and super-majors were totally against the idea," said Dave Lafferty, an oil and gas industry consultant. Formerly with the BP chief technology office, he worked on some of the earliest digital oilfield initiatives. "The tier three onshore oil and gas operators were the ones that first gravitated toward a services model because time-to-market was more critical to them than corporate ego, and they most definitely weren’t interested in building up massive IT departments." It’s been a different story since oil dropped to $50 per barrel. Now much of Lafferty’s consulting work with top tier companies centers on helping them to use big data more effectively.

The returns can be astounding. He points to one example involving a client that had been spending $100,000 annually to monitor just one oil well using a 1980s-era distributed control system. "The company was spending most of its time just manipulating the data," Lafferty said. "Its cost-per-well dropped after they switched to a subscription service." But those wins don’t always come easily. "There are literally hundreds of stories I could tell you about the millions of dollars spent by companies trying to get their heads around big data," said Lafferty.

"Often, it starts out as a simple request for corrosion or temperature measurement. Then, 2 years and a million dollars later they still haven’t gotten anywhere. That’s because they are so highly siloed. When they think they’ve solved this problem, they discover they have a whole new set of issues and need to involve other departments. It just grows and grows." Controls engineers, who have typically operated in a bubble, are beginning to grasp this problem first hand. Historically, they’ve been given a list of requirements and they build a system that meets those specific requirements, in many cases perhaps not even sharing that information with the controls engineers sitting right next to them.

"What we are finding now is that these guys must ensure that their designs are not just meeting the low-lying technical specifications, but that the designs are meeting the requirements across the organization," said Jerry Hines, North American oil and gas manager for NI’s Energy Segment. NI sells both hardware and software to the industry.

Hines said that controls engineers must take a higher-level view and focus on the business problem. "We’re seeing more engineers inviting IT and operations to have a seat at the table, making sure that that their needs are considered as well," he said. "And that’s a good thing."

Now or never for big data

The reality is that there has simply been no better time for big data. Most of the basic technology building blocks of big data are proven and available. Some technologies, such as drones, have been adapted from the military with no modifications required. And more technology vendors, such as large automation companies, and providers of drones and sensors are gravitating toward some kind of a managed services business model. "I would say that at this point in the 21st century, there is a technology out there to resolve any issue you could possibly encounter," said Mohamed Jradi, automation team lead at JMP Engineering, which provides integrated automation and engineering services for a variety of industries. Jradi oversees the company’s oil- and gas-related projects. That’s both good and bad, from his perspective. "As new technologies become available, people tend to lose sight of what it is they are really trying to achieve," Jradi said. "I think there’s a tendency for people to want to go with the latest thing. They read about cloud-based analytics and they think that’s the answer to their problems, when in reality they don’t even know what they don’t know or-worse yet, what they are trying to do, which makes it kind of impossible to write specifications." Before embarking on any new project, Jradi believes that every engineer must ask the question: "Do I understand the specific process that I am dealing with?" And be prepared to answer it. Swimming in data: While helping to select and implement a new technology for mobile operator rounds for refinery units in BP North America, analysts first studied the existing paper-based operations. Operators had spent years meticulously collecting data on every shift and placing the completed sheets in the “in box.” Where did all that data go? Into banker boxes without anyone ever looking at the data. Courtesy: Scientific Technical Services

Other industry observers agree that both legacy processes and thinking are impediments to big data success. "Companies have the data they need, they have the applications and technology that they need, so why is it they aren’t able to do more?" said Lochmann. He said that many companies have retained their manual engineering processes, rather than studying those processes to understand whether resources are being used to their full potential. "The sad truth is the engineering function has grown up as an autonomous island around each asset, and the attitude is one of: ‘Whatever is required, my operators and engineers will figure it out,’ and they have." Of course, one reason that companies cling to old methods is that it’s hard to change from manual to automated work processes, and there may be little impetus. "No question about it," said Lochmann. "In the early 1980s, we started to get these basic PCs, and we had role-playing games. The game placed you in the middle of a field and you had to explore the virtual space by telling the computer what you wanted it to do. It took people a long time to reduce their normal activities to a set of complete, granular commands that the game could process. Automated workflows require the same attention to detail." To counter that, Lafferty advised not biting off too much at the start of a big data project. "It may seem counterintuitive, but companies must decouple monitoring and control," he said. "Capture the monitoring prize first; there is a huge amount of health and safety gains to be made in monitoring applications. You can always move into control applications later."

Back to the future

Big data is evolving and will continue to evolve in the oil and gas industry. Which specific technologies and approaches will ultimately gain the most traction is still debatable, but Lochmann likes to point out that the industry has been here before. "In the mid-1980s, oil and gas exploration groups were going through the same transformation," Lochmann said. "At the time, the primary valuation of an oil company depended on the ability to find and replace reserves, and most of the exploration work at the time was done with maps and pencils, hanging sections up on the wall. Suddenly, we had computers, which replaced paper and gave you the computational power, which led to 3D imaging, which reduced the probability of dry holes." Because we now have shales with a large available reserve capacity, Lochmann said that finding and replacing reserves is still important but no longer the top priority it once was. "Oil and gas companies are going to be judged much more than they used to be on how efficiently they can take a reserve and turn it into cash they can use to fund exploration, drilling, and the other activities," he said. "And there’s the big opportunity for big data." Coming up: The latest advancements and applications of advanced sensors and data collection technologies for big data.

Six steps to big data success

  1. Make sure you understand the specific process that you are dealing with.
  2. Find the business drivers that will capture the attention of senior management.
  3. Decouple monitoring from control in big data applications.
  4. Focus on the business problem.
  5. Build a strong business case.
  6. Give other departments who have a stake in the project a seat at the table.

– A former mechanical design engineer, Karen Field has more than two decades of experience covering the electronics and automation industries, edited by Eric R. Eissler, Oil & Gas Engineering,