AI audit trail for an IME practice: what to log and keep
Short version: keep your own AI use log, 1 row per AI run per case, with who ran it, what went in, what came out, who checked it and what was corrected. The vendor's system logs do not replace it, because they cannot show what the physician checked. Keep the log with the case file and keep policy records for at least 6 years.
Limits: This site gives templates and a process, not legal advice. We read the rules it cites on 2026-11-12; your state, your requesters' contracts and your counsel decide what your practice must do.
2 kinds of audit trail, and why you need your own
Most AI tools keep system logs: sign-ins, uploads, exports. Those logs help with security, and the HIPAA security rule asks for audit controls that record and examine activity in systems holding electronic PHI (45 CFR 164.312(b)). The rule allows those controls to be procedural, not only software.
A system log cannot answer the questions an IME practice gets asked. It does not know which facts the physician opened, which dates were wrong, or whether the AI text reached the report. Only your practice knows that, so only your log can show it. The FSMB's April 2024 policy says there must be documentation of how AI output was used. This log is that documentation.
The 14 fields
| # | Field | What goes in it | Why it is there |
|---|---|---|---|
| 1 | Case number | Your internal case ID, not the examinee's name | Links the row to the file without copying PHI into the log |
| 2 | Run date | Date of the AI run | Sequence of events if challenged |
| 3 | Run type | Initial, addendum, re-run after correction | Addenda get the same scrutiny as first reports |
| 4 | Tool and plan | Product, plan or edition, as on the approved tools list | Shows the run used an approved, BAA-covered plan |
| 5 | Run by | Initials and role | Who touched the file |
| 6 | Task | OCR and sort, chronology, file questions, report draft, quality check | Matches the task to what the policy approves |
| 7 | Input | Which documents went in, with page count | Minimum necessary; matches the cover letter |
| 8 | Output | What came out: chronology, answer, draft section, flag list | What the physician then had in front of them |
| 9 | Notice given | Date sent; "read at check-in"; or "objected, no AI" | Proves the examinee was told |
| 10 | Checked by | Physician initials | The person who verified, not the person who ran it |
| 11 | Facts verified | Number of cited facts opened against the source page | Shows verification happened, and how much |
| 12 | Corrections | Each error found, with the page: "PT note dated 2019, page shows 2021, p. 877" | Feeds the monthly review and the closed-file test |
| 13 | Reached report | Yes, no, or "edited by physician" | Answers "did AI text go into the signed report?" |
| 14 | Row closed | Office manager initials and date | The release gate |
Copy the header
Paste into a spreadsheet, your case system's notes field, or a shared document stored with the case file.
A synthetic case log
Synthetic example, no real patient, case or tool. Case RV-0412 is a made-up lumbar workers' comp file of 1,312 pages. The tool is shown as "approved record tool" on purpose.
| Date | Run type and task | By | Input, output | Checked, corrections | Reached report |
|---|---|---|---|---|---|
| 2026-10-02 | Notice sent, Version 1 and 2 | AK, intake | Appointment letter by mail | Not applicable | Not applicable |
| 2026-10-03 | Initial; OCR, sort, duplicates, chronology | JL, records | 1,312 pages in; chronology of 214 entries, 96 duplicate pages flagged, 3 possible missing records | Page count matches the cover letter. 1 missing operative note requested from the requester | No |
| 2026-10-06 | Initial; file questions | Dr. M | "First mention of radicular symptoms?" Answer cites p. 212 | Dr. M opened p. 212 and searched "numbness"; found an earlier mention on p. 188. Correction logged | Edited by physician |
| 2026-10-06 | Verification of chronology | Dr. M | 41 facts the opinion relies on | 41 opened. 2 corrections: PT note year 2019 should be 2021 (p. 877); left knee should be right knee (p. 1,104) | Not applicable |
| 2026-10-09 | Exam; check-in script read | AK, intake | Examinee asked who sees the records; scripted answer given | Not applicable | Not applicable |
| 2026-10-10 | Initial; report draft from dictated notes | Dr. M | History, exam and record sections drafted; opinion sections left blank by policy | Dr. M edited every sentence, wrote all opinions, added the methods sentence | Edited by physician |
| 2026-10-11 | Row closed; report released | RS, office | Signed report | All rows complete | Not applicable |
2 things to notice. The file question that came back with p. 212 was right as far as it went, and still missed an earlier mention; the 2nd search is what caught it. And the 2 corrections are the kind a closed-file test is meant to surface before any live file.
What the log answers when someone asks
| Question you may get | From | Fields that answer it |
|---|---|---|
| "Did you personally review the records you relied on?" | Opposing counsel at deposition | 10, 11, 12 |
| "Who prepared the record summary?" | Opposing counsel; a California 4628 challenge | 5, 6, 10 |
| "Was any of the report written by software?" | Opposing counsel; requester | 6, 13 |
| "Which vendors touched our claimant files, under what agreement?" | Requester audit or questionnaire | 4, and the approved tools list |
| "Was the examinee told?" | Board complaint; requester | 9 |
| "What changed between the report and the addendum?" | Any party | 3, 7, 12 |
Where to keep it, and for how long
- With the case file. The log rows belong to the case and travel with it. Use the case number, not names, so the log can be reviewed without opening PHI.
- At least 6 years for the policy layer. 45 CFR 164.316(b)(2) keeps required security documentation for 6 years from creation or from the date last in effect, whichever is later. The approved tools list, training records and policy versions fall under it.
- Longer if your state says so. Many states set their own retention periods for medical and medical-legal records. Keep the case log as long as you keep the case file.
- Not inside the AI tool only. If you leave the vendor, its logs may leave with it. Export or keep your own copy.
The monthly review
45 CFR 164.308(a)(1)(ii)(D) asks for regular review of activity records such as audit logs. For the AI use log, a monthly 30-minute read by the owner covers 5 checks:
- Open rows: any report released with a row not closed.
- Unapproved tools: any tool or plan in field 4 that is not on the list.
- Thin verification: rows where field 11 is blank or very low for a large file.
- Correction patterns: the same error type twice (handwritten dates, laterality, medication doses) means a new rule in section 6 of the policy.
- Objections: every "objected, no AI" case shows no AI run afterwards.
Write the date and the findings at the bottom of the month's log. The review is itself a record the policy says to keep. The roles that write each field are on the staff workflow page.