Turning Hard-Earned Expertise into an AI-Driven Build and Audit Pipeline

Problem

Anybody who has built infrastructure to conduct common freight functions like rating, pickups, tracking, etc. knows that each service for each carrier has different specifications that can change without warning.

Context

Leading connectivity providers during the cloud era spent hundreds of millions of dollars, hired and managed international teams of engineers, and understood that this connectivity layer is the unseen layer that underpins all transactions between the multi-billion dollar TMS industry and freight carriers.

Thesis

Claude is very good at coding. Combining the right skills and context should give Claude the ability to create and audit carrier connections at scale. If you can lower the burden to create the connectivity layer, you can create other products (for humans and AIs) on top of it without ever contracting 3rd party freight infrastructure providers.

The broader thesis, and this project is just one example of it, is that you can create a framework that teaches hard-earned human expertise to an AI, keep adding to that expertise together, and build at speeds and accuracy that weren't possible before.

Path

With or without Claude, the first part of this project would be to create a core app that contains the internal data structures that would be the internal standard and reference point for transacting in different required structures. You can receive or send out different structures once you've normalized them to a predictable standard. Creating the normalization layers is where the fun starts.

Auditing the Docs

Sometimes carrier API documentation is incomplete, outdated, or just erroneous. Discovering this manually pre-AI was an arduous task.

It was clear that this was a case for using agentic coding tools, Claude Code in this case, to design a system that checks the docs against scripts that provided real API feedback, allowing problems to surface that aren't immediately visible. A few examples:

• A field name in the documentation didn't match the field the live API actually returned. The docs had a typo and the API was correct. Any integration built from the documentation alone would have failed silently on that field.
• One carrier's documentation was wrong about date formats, field types, and service level codes, not in one place but across several. Each was the kind of error you only find by making a real request and reading the rejection.
• Another carrier rates over REST but books over SOAP and XML. No single service's documentation tells you that a carrier changes protocol partway through the shipment lifecycle.

Teaching the Builder

When I had to write these manually, it was very helpful to have a set of instructions and "gotchas" I'd created for myself based on the docs before actually building the API service connection to the internal standard. I took 10 years of that experience, and iterated many many times in order to teach Claude how I used to create my instructions and common "gotchas". This led to the creation of the "Builder" agent, written as a Claude Code skill and run after the docs audit was complete. Each iteration left more information to analyze about what Claude might have misinterpreted or omitted, and the recipe was refined accordingly. Eventually, Claude and I got to a point where the builder agent produces high quality outputs in the first round, and it makes it easier for the subsequent step to take place.

The Auditor

I often tell people if they don't trust AI yet to let it audit their work and suggest improvements. Anybody who has done this exercise will see the power in having an agent like Claude find issues before they hit production. This auditor step might be the most important one when it comes to putting this in front of real people moving real money. The auditor usually surfaces business ambiguities as well as technical ambiguities, and this is where past experience working in freight tech becomes valuable. This context helps keep the project going and leaves flexibility in cases where there will almost certainly be real world ambiguity that needs to be worked out on a case by case basis.

Where Humans Decide

At this point it was clear the machinery to audit docs, create code, and audit created code was working and improving. The issues surfaced in the audit required human decision-making in order to either update doc audit instructions, or make changes to the API connection that was built. I don't foresee that piece being totally automated yet, but the process of writing the code based on the human decision is still a dramatic efficiency improvement over the old way of manually doing this.

What Twenty Connections Is Worth

It's still early to know the exact time figure for how long it will take to output each new carrier service. The important point for now is that it gets faster each time. It's also critical to note that this project doesn't need many thousands of carrier API connections to prove its value. Yes, the incumbent infrastructure providers have those numbers and it's impressive, but the average firm moving LTL needs around 20 connections. If 20 is the number to achieve utility, then all of the sudden moving 1 connection from weeks to hours or days has very significant impact.

Bottlenecks

The main time drain is actually getting the carrier credentials and being allowed access to the APIs. Once that happens, Claude needs to know where to look for the docs, and assembling them from sometimes disparate sources can be time consuming as well. I'm still working on AI assisted methods to address the second issue (e.g. browser agents exporting JSON from specs across multiple tabs).

Limitations

I can only teach Claude what I've seen and experienced, therefore my teaching is limited by experience. However, since Claude is taking notes and learning with each pass, new use cases will allow the AI to learn beyond what I've experienced, and adapt to real-world use cases as data points.

What's Next

We've now started testing the early framework for teaching Claude my experience based expertise in creating freight carrier connectivity infrastructure. The only way to continue improving it is to continue to iterate with more test cases. Since the AI is making notes for itself with each pass, and since my experience is influencing those notes, we are creating a similar human experiential learning pattern for the AI.

Since Claude isn't the only AI, teams working with other agents could adopt this pattern and take the learning I'm doing with Claude as a starting point.