DIGGS and Data

by Nicholas Miller, P.E.

In the geoscience profession, data is our daily life. We regularly evaluate existing data, search for more, and supplement it with new data. This data varies in size and has a multitude of meanings and interpretations. It drives decisions in design and construction, affecting the allotment of trillions of dollars a year. Yet despite its value and importance, much of that data ends up as a PDF stored in a folder somewhere, eventually archived in a place only IT knows about.

But what if we stored all our valuable geodata in a structured way? What could we do with that structured data? What if we could easily send and receive geodata?

Structured Data

The keyword with structured data is predictability, which allows you to have systems that directly deal with your data to always know where to look for what. You don't have to have an elaborate database or data lakehouse (though that is a worthy goal) to benefit from your data.

Fundamentally, structuring your data involves storing it in a predictable way so that you and the programs you utilize can immediately find what you're looking for. This can be done in any way that makes the most sense for you, as long as it is navigable and organized.

Here are some directions you can go for data storage methods:

• Consistent project file structures
   
• Online geotechnical software (Open Ground, BoreDM, TabLogs, RSLog, etc.)
   
• Converting all gINT files to a single format
   
• CSV files (can be done in Excel)

• SQL Database

Structured Data Uses

So, your data is structured. Now what? There are many things that can be done with structured data. Here are some examples to spark your imagination:

• Query for specific soil types: e.g., "clays within 5’ of proposed ground surface"
   
• Finding correlations more easily: filtering by depths, proximity, soil types, etc.
   
• Easy querying for old projects: query data instead of looking for it, e.g.: "Show borings performed near site in the last 20 years", "Where has previous testing been performed?" "How has the water table changed?" etc.)
   
• Graphics: more easily make tables and graphs that clearly communicate thought process
   
• Report writing: quickly filter for strata instead of manually finding and recording it; summarize lab testing at project; average index values in x geology strata; location of all organic layers; etc.)
   
• Project dashboards showing boring, laboratory, field investigation, CPT, etc.
   
• Prompt Engineering — feed data to LLM and ask it questions about the data: "Summarize field activities on x date." "Has y boring been drilled yet?" "Is z strata present in all borings?" etc.)
   
• LLM Training: train secured LLM on project data. Is x activity in the scope? What deadline is set for y submittal? Does our scope cover all the topics listed in the RFP? List specifications for z structure in planset; etc.)
   
• Future AI Training: train probabilistic models for reliability/risk integrated design; Bayesian Machine Learning; train system simulations that combine design recommendations and construction process; Reinforcement Learning image classifications from field inspections to classify soils, identify damages, identify problematic site conditions, etc.; Convolutional Neural Network; area evaluations using satellite and exploration data; Deep Learning

Data Transfer

Hopefully by now you have a fire in your belly to structure and use data, but let me throw one more thing at you: data transfer.  Often, the most difficult and time consuming part of using the technologies above is in cleaning and combining data from different sources: think about transcribing hand written field logs, translating pdfs of laboratory testing into gInt, or copying existing boring logs into your software of choice).

This is where DIGGS comes in: at its core, DIGGS is a standard way to talk about geo-data. If everyone talks about geo-data the same way, then data can easily move into your data structures and plug into your workflows, no matter from whom you're your receiving your data or to whom you're sending it.  

The nested structure of DIGGS files makes DIGGS very flexible, able to accommodate any purpose: field inspections, borings, CPT, lab testing, MWD.... you name it.

Learn More About DIGGS

Find out more about DIGGS on our mini-site. We also have presentations about DIGGS on our YouTube channel