Use cases
AI employee for document review
Tobiloba Odejinmi · 12 Mar 2026 · 6 min · 1,089 words
Direct answer
A document-review employee pulls fields from the pile, flags gaps, and leaves the odd cases for a person. Good looks like a structured record you can check, not a summary. Escalate money, legal interpretation, and anything the schema cannot hold. Do not start with unsupervised sign-off. At SmartComply, review work fell by about half because the loop was tight. Measure review time and miss rate.
- The job is the first pass on a pile of similar documents.
- Structured output you can check beats a fluent summary.
- A person still owns the miss and any legal call.
- Measure hours on the pile and misses you can explain.
What is document review as a job?
It is the same document, again. A policy. A statement. An onboarding pack. Someone reads it, types fields into a form, and does it tomorrow. That is not a strategy problem. That is a process you can write down. The employee does the first pass. A person does the rest.
If the documents are all different, you do not have a review job. You have a research job. Do not start there. Start with a pile that shares a shape: invoices, KYC packs, clinic agreements, the same vendor form. Volume plus sameness is what makes this worth building.
I learned this at SmartComply. The pile was the product. Once we treated it that way, the work got lighter. Not because the model was special. Because we stopped asking it to be a lawyer and started asking it to fill a form.
What does a good first pass look like?
A record that matches a schema. Every field is a value, missing, or a quote. A reviewer can scan it in a minute and click through to the page. If they have to read the whole PDF to trust the extract, you did not save time.
We used structured output so the system had to return a shape we could check. That is how you stop a model from inventing a field. Fluency is a liability here. I would rather have an ugly JSON that is true than a paragraph that sounds sure.
Good also means the weird document is easy to spot. Low confidence, failed schema, unknown layout. Those should jump the queue to a person. Burying them in a “done” list is how you get an incident.
- Schema first, prose second
- Missing is a valid value
- Quote or page reference on risky fields
- Unknown layouts go to a person
What must escalate?
Legal meaning. Money. Medical conclusions. Any document the schema rejects. Any time two fields disagree. The employee can show the conflict. A person decides which one is true.
Escalation is a state handoff. The reviewer gets the file, the extract, the failed checks, and the reason. Not a Slack dump of PDFs. If the reviewer starts from zero, you automated nothing.
Hard gates stay hard. I will not let a first pass sign a compliance position. I will let it fill the form that a reviewer then signs. That difference is the whole design.
What should you not automate first?
Do not start with unsupervised approval. Do not start with a mixed pile of every document the company has ever seen. Do not start with “ask the bot anything about this folder”. That last one is a demo. Demos do not shrink review time.
Skip the clause-rewrite fantasy. Changing a contract is legal work. Extracting the renewal date is review work. People blur those because both involve PDFs. Keep them apart.
Do not start by turning off the old checklist. Run both until the miss rate is something you can say out loud. Then move the first pass. Keep the person.
How do you measure review without hiding misses?
Review time on the pile. Accept rate without edits. Miss rate on a set you already labelled. Time to explain a miss. At SmartComply, review work fell by about half. That number only mattered because we could still see the misses.
If review time falls and miss rate is unknown, stop. You cannot manage a hidden error. Sample. Label. Write down why. If you cannot explain a miss, you are not ready to go live on that document type.
Incident response should get faster, not slower. People should have time to look at the odd case. If they are still buried, you added a tool and kept the pile. That is a failed install.
Why does structured output matter more than the model?
Because the model will change and the check should not. A schema is a contract with yourself. It says what “done” means. When a new model arrives, you run the same checks. When a new document type arrives, you add fields on purpose, not by accident.
I like TypeScript and I like checks. That is not a personality quirk. It is how you keep a system honest when the input is a PDF that was scanned twice. The model proposes. The schema disposes.
If a vendor will only give you a chat window, you are buying a summary. Summaries feel like work. They are not a record. Buyers who later look at your process will want the record. I learned that selling companies. Fancy slides do not survive that meeting.
Questions people ask
What does the employee do with a document?
It returns a shape you defined: fields, missing fields, and quotes from the page. It does not “understand the contract” in a vague sense. If a field is not there, it says so. That is the product.
Did SmartComply remove reviewers?
No. We took the first pass off them. They still own the cases that need judgment. Manual review work fell by about half. Incident response got faster because people were not buried in the pile.
What should never auto-approve?
Money movement, legal interpretations, medical conclusions, and any document type you have not sampled. Auto-extract is fine. Auto-approve is how you get a silent miss.
Why not just use a chatbot on the PDF?
Because you cannot check a chat. You can check a schema. Zod is unglamorous. It is also how you stop a model from inventing a clause. If you need a narrative, generate it from the fields after they pass a check.
How do we measure it?
Minutes per document for the first pass. Share that a reviewer accepts without edits. Miss rate on a labelled set. Time to find a field when something goes wrong. “Documents processed” is volume. Volume without a miss rate is how you fool yourself.
Written by
Tobiloba Odejinmi
Head of Engineering at 10mg Health. I have run engineering at Zeeh Africa and sold Insurpass and Shopl. I still write the code. If you have one process that still runs on people copying things, we can look at it in thirty minutes.
