top document problems I see
Here's the most common questions I get about documents and OCR.
What model do I choose?
Free resources: How to Choose OCR Model and VLM for hard OCR Docs. These talks give a framework and tradeoffs that let you evaluate models as they change.
How do I stay up to date?
AI and document news sharing is a huge stream of gold, crap, and slop. Staying on top of it requires an immense amount of time, which is why I have started an opinionated news feed of document news (it's free).
I spend many hours each day to read, try things for myself, and decide what I think is high signal using an old school classification technique called human-as-a-judge.
How do I pick a vendor?
Evaluating the tradeoffs is a whole project to compare costs, privacy policies, latency, consistency/dedicated compute etc. It's really hard to give general advice because the requirements for vendors for each company is so different.
I can help provide the individual guidance in my paid community or in my upcoming course. Students in the course will also get $700 in credits from the top OCR vendors to evaluate with (course is only $500). Credit amount will probably go up, but that's what's confirmed by OCR vendors so far.
What about observability for OCR?
Once you pick a model you often want to continuously improve, and this is where observability is critical.
- How do you find and fix errors in the product or workflow and incorporate new techniques or models as they come out?
- How do you know if the PDFs from today have new types of issues than the ones from a year ago?
- How do you know if it's worth it to add a pipeline step or change a model?
If observability is done right, this process becomes faster over time. If not, it's becomes more difficult and more complex each time.
I do not think any of the eval specific vendors do a good job here for document workflows. My recommendation is to use your database of choice, I often use postgres. Fundamentally you need to track the inputs (pdfs), outputs (model response), and any arguments passed to functions to create them (model name, prompt, params) and keep those tied together.
I recommend having this schema yourself for most applications, because for most applications this schema will be much lesson complicated that any of the other database stuff (like user data, permissions, and payment setup).
I find that eval vendors overcomplicate it, makes is more confusing, and do not help you follow good practices. I hope that one day there will be a vendor that does this that I can truly vouch for.
I just have a few questions over time, can I ask you questions for quick answers?
Yes, just respond to any of my newsletter emails and ask questions. I read every response and answer as best as I can (time permitting).
Priority for Q&A and research time goes to community members though, so if you have several questions, need quick answers, or ones that require a lot of nuance you're best bet is the paid community for unlimited aysnc questions of the community forum + weekly live Q&A calls.
Where can I go to get all the information needed to build document processing for an important product or workflow?
My blog and Joe Barrow's blog has a ton of free information and we're always working on adding more. And let's be honest, all information is available online for free in multiple places if you want to search for it.
For a faster, curated path and help applying that information to real use-cases faster, take our full-day intensive course. There's no deep secrets that do not exist anywhere else on the internet, but it is all the information you need delivered in 1 day without all the noise. And with 30 days of Q&A support from people who have worked through all the painful parts already that will help you apply it to your product or workflow faster.
Here’s a link to the curriculum to learn more about the course, or just jump to the stripe payment page if you know you need this.
Some other common questions regarding the course:
- Can I submit my own PDF? Yes, we want you to. Tell us what your product needs from it, and we may use it in a live walkthrough. There will be lots of time for Q&A during the course and in the following support period as well.
- Do I need to code? No. The lectures, examples, and Q&A do not require you to write or run code. The notebooks and code walkthroughs are optional: use them yourself, share them with an engineer, or give them to a coding agent when you’re ready to implement.
- What happens if I cannot attend the full day? Every session is recorded and the recordings are available after the workshop. You can return to any session you miss. You’ll also receive the notebooks and 30 days of follow-up support and Q&A so you don't miss out on questions just because you were't live.
Go deeper
- PDF to Production: Build reliable products and workflows that use PDFs and documents in a full-day workshop with 30 days of support and 700+ of OCR credits
