ecuscino | Created: 10 Nov 2025 | Updated: 11 Nov 2025
Blog
DIGGS News
DIGGS
DIGGS Blog
Title graphic

DIGGS Data Processing Manager Usage Guide

Last Updated: October 2025 | Version 1.0.0

 

Managing geotechnical data shouldn't be complicated. Whether you're collecting field measurements, entering laboratory test results, or sharing data with clients, the DIGGS Data Processing Manager streamlines your workflow from start to finish.

In this comprehensive guide, we'll walk you through everything you need to know to get started with this powerful geotechnical data processing tool.

What Is DIGGS Data Processing Manager?

DIGGS Data Processing Manager is a specialized application designed for geotechnical engineers who need to manage their field and laboratory data efficiently. The tool acts as a bridge between familiar Excel spreadsheets, robust SQLite databases, and industry-standard DIGGS XML files.

Think of it as your data translation hub—converting your information into whatever format you need, whenever you need it.

Generate Excel Templates - Create perfectly structured blank Excel files ready for your geotechnical data entry. No more formatting from scratch or worrying about proper column headers.

Convert Excel to SQLite - Transform your completed Excel spreadsheets into efficient database format, making your data queryable and more robust.

Export to DIGGS XML - Create DIGGS 2.6 compliant XML files that meet industry standards for sharing and archival purposes.

Import DIGGS XML - Receive and process DIGGS XML files from other organizations, converting them into usable database format.

Visualize Your Data - View your geotechnical information through charts, graphs, and tables for quality control and presentation preparation.

Who Should Use This Tool?

This application is perfect for:

  • Geotechnical engineers collecting field data and managing multiple project sites
  • Laboratory technicians entering test results and maintaining quality records
  • Project managers organizing geotechnical information across teams
  • Anyone needing to share or manipulate data in DIGGS format for compliance or collaboration

Getting Started: Installation Made Simple

Instructions apply to the GitHub repo found at https://github.com/DIGGSml/DIGGS_SQL/tree/main

Three Installation Options

Option A: Using the Executable (Recommended)

The simplest approach for most users:

  1. Download the DIGGS_Processor_Manager.exe file
  2. Double-click to run—no installation necessary
  3. If Windows displays a security warning, click "More info" then "Run anyway"
  4. The application opens and you're ready to go

Option B: Using Python (For Advanced Users)

If you prefer working with Python:

  1. Ensure Python 3.8 or higher is installed on your system
  2. Navigate to the executable folder
  3. Run: python diggs_processor_gui_final.py

Option C: Desktop Shortcuts (Windows)

For convenient access:

  1. Navigate to the executable folder
  2. Double-click install.bat
  3. Desktop and Start Menu shortcuts are automatically created
  4. Launch from the shortcut whenever needed

First Launch: Familiarizing Yourself with the Interface

When you open DIGGS Data Processing Manager for the first time, you'll see five tabs at the top:

  • Excel Templates - Generate blank or sample templates
  • Excel to SQLite - Convert spreadsheets to databases
  • SQLite to DIGGS - Export databases to XML format
  • DIGGS to SQLite - Import XML files to databases
  • About - Version and project information

Take a moment to click through each tab. The interface is intuitive, with clear labels and helpful prompts throughout.

Step-by-Step Workflows

Let's explore the three most common workflows you'll use with this application.

Workflow 1: Starting a New Geotechnical Project

This is your beginning-to-end process for fresh projects.

Step 1: Generate a Blank Template

Start by creating your data entry file:

  1. Click the Excel Templates tab
  2. Select "Blank Template" from the dropdown menu
  3. Click Browse to choose your save location
  4. Click Generate Template
  5. Wait for the success confirmation
  6. Open your newly created Excel file

Step 2: Enter Your Data

Now comes the data entry:

  1. Fill in the Project sheet with project details
  2. Complete the HoleInfo sheet with borehole locations and depths
  3. Enter sample information in the Samples sheet
  4. Add test results (SPT, Atterberg Limits, etc.) in their respective sheets
  5. Save your work regularly

Step 3: Convert to Database

Transform your spreadsheet into a robust database:

  1. Return to DIGGS Data Processing Manager
  2. Click the Excel to SQLite tab
  3. Browse to select your completed Excel file
  4. The output filename is generated automatically
  5. Click Convert to Database
  6. Confirm the success message

Step 4: Export to DIGGS XML

Create your shareable, industry-standard file:

  1. Click the SQLite to DIGGS tab
  2. Browse and select your database file from Step 3
  3. Output filename auto-generates
  4. Click Convert to DIGGS XML
  5. Wait for confirmation
  6. Your DIGGS XML file is ready to distribute

Workflow 2: Processing Received DIGGS Data

When a colleague or client sends you DIGGS data, here's how to work with it.

Step 1: Import the DIGGS XML File

  1. Navigate to the DIGGS to SQLite tab
  2. Click Browse next to Input XML File
  3. Select the received DIGGS XML file
  4. Output database name generates automatically
  5. Click Import to Database
  6. Review the import summary showing record counts

Step 2: Access Your Imported Data

Once imported, note where the database file is saved. You can now:

  • Export it back to DIGGS XML if modifications are needed
  • Generate Excel templates to view or edit the data manually
  • Use database tools to query the information directly

Workflow 3: Quality Control Check

This workflow validates your data integrity through a round-trip test.

Step 1: Convert Your Excel Data

Follow Workflow 1, Steps 1-3 to transform your Excel data into database format.

Step 2: Export to DIGGS

Complete Workflow 1, Step 4 to create your DIGGS XML file.

Step 3: Re-import for Validation

Now for the validation step:

  1. Use Workflow 2, Step 1 to import the DIGGS file you just created
  2. This confirms your data is fully DIGGS-compliant
  3. Carefully review the import summary

Step 4: Compare and Verify

Check that imported record counts match your original data. Any discrepancies indicate potential issues with your Excel formatting or data relationships.

Troubleshooting Common Issues

File Won't Open

Problem: Error message when trying to open files

Solution:

  • Verify the file exists at the specified location
  • Ensure the file extension is correct (.xlsx, .db, or .xml)
  • Close the file if it's open in another application
  • Check that you have read permissions for the file

Excel Conversion Fails

Problem: Errors appear during Excel to SQLite conversion

Solution:

  • Confirm your Excel file was generated by this application's template function
  • Check that required sheets (Project, HoleInfo) are present
  • Verify column names haven't been changed or deleted
  • Look for blank rows in the middle of your data
  • Compare with a sample template to verify structure

DIGGS XML Export Creates Tiny Files

Problem: The exported XML file is unexpectedly small or empty

Solution:

  • Verify your database contains data (file size should exceed 10 KB)
  • Ensure Project and HoleInfo tables have records
  • Check that foreign key relationships are correct (samples reference valid holes)
  • Try importing your database back to verify its structure

Zero Records Imported

Problem: DIGGS import shows no records were imported

Solution:

  • Open the XML file in a text editor to verify it's actually a DIGGS file
  • Check that the XML isn't corrupted (should be readable text)
  • Ensure the file follows DIGGS 2.6 structure
  • Try opening the XML in a web browser to check for syntax errors

Application Becomes Unresponsive

Problem: Progress bar freezes or the application stops responding

Solution:

  • Wait at least 2-3 minutes—large files take time to process
  • Check Windows Task Manager to see if the process is still active
  • If truly frozen, close and restart the application
  • Process a smaller subset of data first to test
  • Verify you have at least 100 MB of free disk space

Generated Files in Unexpected Location

Problem: Can't find your processed files

Solution:

  • Check the output path displayed in the interface before clicking process
  • Use the Browse button to explicitly set your desired location
  • Search Desktop, Documents, and Downloads folders
  • Search your computer for .db or .xml files modified today

Data Missing After Conversion

Problem: Some data doesn't appear in the converted file

Solution:

  • Verify all data sheets reference valid hole IDs from HoleInfo
  • Confirm sample IDs match between Samples sheet and test result sheets
  • Check for Excel formula cells instead of actual values
  • Look for data in hidden columns or rows
  • Review for special characters that may cause parsing issues

Permission Denied Errors

Problem: "Permission denied" or "Access denied" messages

Solution:

  • Close the input file if open in Excel or another program
  • Check that the output location isn't write-protected
  • Run the application as Administrator (right-click → Run as Administrator)
  • Choose a different output folder like Desktop or Documents

Visualization Not Working

Problem: Charts and graphs won't display

Solution:

  • The matplotlib library may not be included in the .exe distribution
  • Try using the Python version: python diggs_processor_gui_final.py
  • Install matplotlib: pip install matplotlib
  • Alternatively, open the .db file in DB Browser for SQLite to visualize data

Getting Help and Support

Project Documentation: The README.md file in your installation folder contains technical details and updates.

Database Schema: Visual database structure available at https://dbdiagram.io/d/DIGGS-SQL-Structure-668dcbd19939893dae7ebb48

DIGGS Standard: Official DIGGS specification at http://www.diggsml.org

Source Code: Full source code repository at https://github.com/geotechnick/DIGGS_SQL

Reporting Issues

If you encounter a bug or unexpected behavior:

  1. Note the Exact Error Message: Take a screenshot or write down the complete text
  2. Document Your Steps: List exactly what you did leading up to the problem
  3. Check Error Logs: Look in the error_logs folder if it exists
  4. Reproduce with Sample Data: Try to recreate the issue using the sample template
  5. Report on GitHub: Submit detailed issue reports at https://github.com/geotechnick/DIGGS_SQL/issues

Getting Support

For questions or assistance:

  1. Consult This Guide: Most common questions are answered here
  2. Review the FAQ: Many quick solutions are in the FAQ section
  3. Read the README: Technical details in the README.md file
  4. Search Existing Issues: Someone may have already solved your problem on GitHub
  5. Post a New Issue: Provide detailed information when creating new issue reports
  6. Connect on LinkedIn: Contact the developer at https://www.linkedin.com/in/geotech-nick/

Helpful Information for Support Requests

When asking for help, include these details:

  • Your Goal: What were you trying to accomplish?
  • Steps Taken: What actions did you perform?
  • Expected Result: What should have happened?
  • Actual Result: What actually happened?
  • Error Messages: Exact text of any errors (screenshots help)
  • System Information: Windows version you're using
  • File Details: Size of input files, approximate number of records
  • Reproducibility: Can you make the problem happen again?
 

Conclusion

The DIGGS Data Processing Manager bridges the gap between everyday Excel workflows and industry-standard geotechnical data formats. Whether you're managing a small site investigation or coordinating data from multiple projects, this tool streamlines your processes and ensures data compatibility across organizations.

Start with the simple workflows, experiment with the sample templates, and gradually incorporate the advanced features as your comfort level grows. The investment in learning this tool will pay dividends in time saved and errors prevented throughout your geotechnical projects.

Ready to get started? Download the application from the GitHub repository and begin transforming your geotechnical data management today.

Have questions or suggestions? The geotechnical community welcomes your feedback. Connect on GitHub or LinkedIn to share your experiences and help improve this tool for everyone.