Data is in Our DNA
A couple of years ago, I managed a consulting project with a young analyst on our team. The project objective was to provide market due diligence for an oilfield water asset that our client wanted to acquire. We spent countless hours analyzing data to delineate the local market for produced water management, profile current and prospective customers, and give our assessment of growth opportunities for the business. It was a hurry-up project and you could hear the urgency, excitement, and nerves in our clients’ voices. “What are your blind spots?”, they asked. We detailed every potential deficiency in our analysis that we could think of, including the issues inherent in the data, which was notoriously challenging to work with. After the end of our last call to run through deliverables, my analyst colleague looked at me with wide eyes. “Woah,’ he said, “they’re about to decide whether or not to spend $100 million based on our analysis.” “Yep!” I replied, “This is why we spent so much time interrogating the data in so many different ways.” In that moment, he had his Eureka! moment. He understood that what had previously seemed a maddening degree of scrutiny we apply to our data was just the beginning of the work needed to make the data usable for decision making. At the core of this commitment to data quality is a basic, but profound concept: trust. Our clients trust us to provide the highest quality data and analysis possible and to be forthcoming about issues in the underlying data.
I’ve spent my entire career in companies that provide data used in high-impact business decision making like trading, infrastructure investment, or strategic planning. In my early career roles, a “data bust” inevitably meant an irate, 5 AM call from a trader whose positions were jeopardized. A few years ago, at B3, when we first began digitizing the very messy data that became the core of our OilfieldH2O offering, I got a similar call late at night. Often, the “data bust” is actually an issue with fundamentally mis-reported or poorly recorded data, but that doesn’t matter to a client who needs to put the data to work. When it happens, immediate action is required. That late night call resulted in my father, who was in from out of town, and I painstakingly hand-digitizing records. I think we wrapped up just before midnight, then called a member of our development team who processed all the data and promoted it for the client, who was also awake and still working. That episode marked an inflection point that changed the way we understand our data and the ongoing investment and attention needed to understand and improve their quality and utility.
When we did a post-mortem to figure out how to avoid a repeat of that night’s events and an erosion of trust in our data, it was clear that we needed to rethink our approach to data, almost starting from scratch. This set B3 on a new path – one focused on measurement, improvement, process management, and transparency. It forced us to define our data culture and ethos. We engaged industry experts and academics and delved into studies related to data quality, its measurement, and the myriad approaches to understanding it all.
B3 AND DATA
The first milestone in our journey was to create internal transparency and awareness about the latency, completeness, and status of key datasets. We built a series of dashboards that were displayed on monitors in the middle of the office that showed newly available records, records in processing, and still missing records. It was the first time that we could visualize how data became available from a source, went through our processes, and flowed into our data. Importantly, it meant we could communicate directly with customers about the completeness of data and help them understand how the data they consumed were created – in many cases, we sent them screen captures of the dashboard. For the first time, we could translate known issues with quality and completeness into communicated best practices for using our data. This transparency and attention to dataflows was just the beginning.
Data Quality and Security
The more we learned about the lifecycle of data the more the need for added sophistication in our processes became clear. We created a new role focused on internal software development for the sole purpose of understanding and improving data quality. With this role established, we engaged new third-party partners and focused heavily on process, even integrating the internal software role into our product development process whenever complex data issues were involved. We set testable standards for quality. For example, in our most challenging dataset, we wanted to be able to know that our client-facing data were 95% accurate, 95% of the time. This meant creating new internal pipelines for data, determining random sampling protocols for QA, and building new software platforms to make quality improvement scalable and detailed. It was a huge investment in time-consuming manual processes, supporting software, dashboards to drive transparency and accountability, and data science to develop automated processes to preemptively capture subpar data. For over a year, we invested more in these back-office capabilities than any client-facing products. That investment has paid off, enabling subsequent major upgrades to our data models and structure that have made our data and web applications more and more reliable, usable, insightful, and proprietary. However, trust in data requires more than basic transparency – it means enabling diverse methods of secure data collection, assurance of clearly articulated QA/QC protocols, strict access rules for B3 developers building services with the data, effective data entitlement to ensure only authorized users may access private data, and importantly, security. B3 has already begun to integrate increased security and protections into our data and technology.
THE FUTURE OF DATA IN OILFIELD WATER
While we were busy investing in our data quality, observable shifts emerged among our customers, particularly those leveraging our flagship offerings in the oil and gas industry. First, their use of the data became increasingly technical and specialized in nature and continues to move in this direction. Beyond our web application and workflow tools, they require data that they can leverage in large volumes, flowing into proprietary systems. The fast-growing number of professionals in the oil and gas industry who can write Python and run complex SQL procedures continues to drive the diverse and creative use of data. This has shifted the way that data are consumed from companies like B3. A key shift is that while a small number of people may collect and integrate the data from our API, diverse users from many parts of the organization leverage them in their specialized work, all drawing from an internal data hub. This means that B3 does not have direct contact with some of our users. Simplified data models, clear quality protocols, and documentation are therefore increasingly critical to ensuring customer success.
The second major shift that has emerged among users of B3 data is a stronger industry drive to collaborate. We have often characterized the issues facing the oilfield water management industry as a potential tragedy of the commons, where companies focus solely on their needs, ignoring collaborative opportunities to achieve superior outcomes or mitigate looming challenges. To gauge the industry’s sentiment relative to our own, we began surveying customers in 2019, asking them to rank industry collaboration among other pressing issues they faced. It routinely ranked in the bottom quartile of issues until last year, when it shot up to the top quartile. In today’s market, in which operators face pressure from ESG concerns, seismicity as a dynamic factor shaping water management, and a need to constantly improve processes and bottom line, collaboration has become an existential need for many operators. The implications for B3 are exciting – we now find ourselves partnering with customers to facilitate data sharing, benchmarking and indexation, create internal report cards, and provide external evaluation of operational data. As digital transformation and collaboration become more standard in water management, B3 can put its data models to work helping the industry quantify, compare, and improve key metrics that affect bottom line, asset development, and sustainability.
EMPOWERING SMART WATER MANAGEMENT
B3 Insight is building the definitive source for water data. We empower smart water management with data-driven intelligence for responsible and profitable decisions about water resources. Whatever your needs, B3 Insight’s OilfieldH2O can positively impact your water management and drilling completions – contact us at [email protected] or 720-664-8517 to learn more.
CEO and Co-Founder