Automated Data Extraction from BOL Documents

Stacks of paperwork to process

Our client was the CEO of a 3PL Provider that served a major food distributor in the mid-west.

The CEO operated a large warehouse that took in tens of thousands of shipments per month. Each shipment generated a Bill of Lading document (or BOL) that confirmed successful delivery.

They were looking for a way to automate the data extraction and entry of tens of thousands of Bill of Lading (BOL) Documents per month.

Before

Automating the processing of these documents was critical for the warehouse for 3 main reasons:

1. Manual, error-prone process

It was someone's full-time job to sort through the tens of thousands of BOL documents every month.

It was error-prone, tedious, yet critical for the warehouse as the BOL documents were the source of truth for what was coming in. 

A person working on this full-time could process at most 100 BOL documents per hour. Given the high volume of shipments, this would take 100 hours for a full-time employee to process, almost 2.5 weeks per month.

2. Lack of data

Without processing these BOL documents, the CEO wouldn't have any data on what was coming in and out of the warehouse. Particularly since BOL documents were issued upon delivery, it contained the source of truth information for all shipments. 

Processing these BOL documents was critical to make data-driven decisions to improve relationships with their partners and operations in the warehouse.

3. Invoice reconciliation was impossible

Searching through these documents was also impossible beforehand, as you had to manually go through all the documents to find the corresponding BOL documents per order.

This made invoice reconciliation difficult, and papers were prone to getting lost.

---

As such, our client was seeking a software solution to automate the data extraction and entry of these BOL documents. After aborting an engagement with another software agency, they turned to All-In Consulting for a solution.

Our Approach

We spent two weeks on a discovery phase followed by six weeks on implementation. Discovery was focused on three meetings:

  • business requirements gathering
  • product flow mockups
  • engineering architecture

During discovery, we came to three main conclusions:

The need to account for edge cases from the beginning

Tooling for logistics is different than regular product building in that warehouse operations tend to have many edge cases and exceptions.

Resolving these exceptions forms a large portion of the workflow in any tooling, so we needed to account for these exceptions from the beginning rather than treating it as an after thought.

Performance of Data Extraction API's Varies Wildly

During this time, we investigated the performance of several different data extraction API's, and we discovered that the performance varies wildly between them.

Open source API's were able to extract data around ~50% of the time, while proprietary API's extracted the data successfully around 90% of the time.

This investigation helped save us weeks of time during implementation, as we knew which API's to integrate with from the beginning.

Lack of Standardization Among BOL Documents

Different distributors had different formats for their BOL documents. Certain fields were spelled differently: for example a "Purchase Order Number" could be spelled as "PO Number, "P.O. Number", "PO #" and so on. 

Furthermore the data could be in different positions. For example the actual PO number could be above, below, left or right of the term we were searching for.

Our approach was to make a list of every different way a field could appear in the documents, along with every potential placement of that field.

For example, for PO numbers, we discovered 27 different combinations of spelling and placements of this data in BOL documents, and we ran the tool through every case at the end to ensure the tool would work for as many different combinations as possible.

Results

We built a tool that automated the processing of their BOL documents for them in 8 weeks.

The tool reduced manual extraction work by 91% and was able to process 1,800 BOL documents per hour.

For context, a human worker doing this would process at most 100 documents per hour at the fastest, with significant room for error.

This project gave them the ability to quickly search through BOL documents for invoice and shipment reconciliation, which was not possible before. 

This automation project brought immediate returns for the client. They began using the tool immediately after implementation, and continue to use it every day to this day.

They have not needed to come back to us for further maintenance or bug fixes after deployment.

The client was extremely satisfied with the engagement, and extended our engagement to continue rolling this tool out to their other warehouses.

They were also so satisfied that they brought us on for other automation projects in their business.

Summary of Results:

  • 257 BOL test documents
  • 91% extraction rate
  • Graceful exception resolution
  • Invoice reconciliation possible
  • 6 week time to value
  • Engagement extended for further automation and optimizations at their warehouse

---

If you have automation needs at your warehouse, contact michael@allinengineeringconsulting.com or reach out to us at https://www.allinengineeringconsulting.com/contact for a free 30-minute intro call.