ray
173 lines
12 KiB
Markdown
173 lines
12 KiB
Markdown
---
|
|
name: ocr-text-parsing
|
|
description: "Parse semi-structured OCR text into typed records using regex — no AI/API needed. Boundary detection, field extraction, OCR error tolerance."
|
|
version: 1.1.0
|
|
author: Hermes Agent
|
|
license: MIT
|
|
platforms: [linux, macos, windows, browser]
|
|
metadata:
|
|
hermes:
|
|
tags: [OCR, Parsing, Regex, Data-Extraction, Browser, CPU]
|
|
---
|
|
|
|
# OCR Text → Structured Records (Rule-Based)
|
|
|
|
When OCR output contains multiple records (appointments, contacts, invoices) and you need to parse them into typed fields without calling an AI API. Runs in-browser or server-side, instant, zero cost.
|
|
|
|
## When to Use
|
|
|
|
- Screenshot → OCR → structured form fields
|
|
- Multiple records in one image (table, list, stacked cards)
|
|
- User says "no API", "use CPU", "make it efficient", "parse locally"
|
|
- OCR quality is imperfect (Tesseract artifacts, garbled characters)
|
|
|
|
## Core Algorithm
|
|
|
|
**Strip noise → Extract fields → Split → Assign** (strict order):
|
|
|
|
1. **Strip advisor/tech noise FIRST** — before any field extraction. Remove known advisor names and OCR corruption variants (see Pitfall #10). This prevents advisor text from contaminating customer name or service.
|
|
2. **Extract structured fields in order** — Time → Duration → Phone → Email → VIN → Date. Use targeted regex and remove each match from the text. Order matters: email before VIN prevents `garyb9623@gmail.com` digits from being mistaken for VIN characters. Phone before email prevents phone digits from matching email fragments.
|
|
3. **Post-extraction cleanup** — strip residual label tokens (`RO`, `VIN`, `os`, `b=`). These are common OCR artifacts that survive field extraction.
|
|
4. **Extract OP code + service** — if bracketed codes like `[REP]`, `[23-046:6MX00]` exist, extract them and capture the service text that follows on the same line. Service regex MUST stop at newline (`[^\n\[]*`, not `[^\[]*`) to prevent vehicle info on the next line from being absorbed.
|
|
5. **Extract vehicle** — year + make + model (`(\d{4})\s+([A-Za-z]+)\s+([A-Za-z0-9\-]+)`). Fallback: year + make only.
|
|
6. **Name from remainder** — first remaining part after stripping leading non-letter characters.
|
|
7. **Recurse** — find next phone/VIN in leftover text, walk back to nearest multi-space gap, split, re-enter step 1.
|
|
|
|
## Key Patterns
|
|
|
|
### Phone extraction (forgiving formats)
|
|
```js
|
|
var phoneRe = /\(?\d{3}\)?[\s.\-]*\d{3}[\s.\-]*\d{4}/;
|
|
```
|
|
|
|
### VIN extraction (OCR-tolerant)
|
|
```js
|
|
// Accept O/I/Q (common OCR errors for 0/1) — normalize after match
|
|
var vinRe = /\b[A-HJ-NPR-Z0-9OIQ]{17}\b/i;
|
|
function fixVin(v) { return v.toUpperCase().replace(/O/g,'0').replace(/I/g,'1').replace(/Q/g,'0'); }
|
|
```
|
|
|
|
### Email extraction (extract before VIN to prevent false matches)
|
|
```js
|
|
var emailRe = /\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b/;
|
|
```
|
|
Extract email IMMEDIATELY after phone, before VIN. Emails like `garyb9623@gmail.com` contain 4-digit numbers that can false-match phone/date/VIN patterns. Removing the email from the text first prevents these collisions.
|
|
|
|
### Date extraction (multiple formats → YYYY-MM-DD)
|
|
Handle: `2024-06-15`, `06/20/2024`, `6/20/2024`, `Jan 15, 2024`, `January 15 2024`. Month-name lookup with 3-letter prefixes.
|
|
|
|
### Time extraction (12h + 24h)
|
|
Handle: `9:00 AM`, `1:30PM`, `14:00`, `8am`. Convert to 24h `HH:MM`.
|
|
|
|
### Multi-record boundary detection
|
|
- **Table format**: Detect when 3+ lines have consistent word counts. Skip the header line if it contains `name|phone|date|time|service|vehicle|vin`.
|
|
- **Row-anchor split (preferred)**: When each record starts with a predictable pattern like `9:00AM 0.5 hrs`, use it as the primary block delimiter instead of blank lines. Find all anchor positions with `RegExp.exec()` in a loop, split the text at each anchor index. This survives OCR horizontal-scan order where blank lines may be absent. Falls back to blank-line splitting if fewer than 2 anchors found.
|
|
```js
|
|
var rowAnchorRe = /(\d{1,2}):(\d{2})\s*(AM|PM)\s+(\d+(?:\.\d+)?)\s*hrs?/gi;
|
|
var anchors = [];
|
|
while ((am = rowAnchorRe.exec(t)) !== null) anchors.push({ idx: am.index, len: am[0].length });
|
|
if (anchors.length >= 2) { /* split at anchor boundaries */ }
|
|
else { /* fallback: blank-line split */ }
|
|
```
|
|
- **Free-form**: After extracting one record's fields, find the next phone/VIN. Walk back to the last `\s{2,}` gap (or fallback: last 1-2 capitalized words). Split there — everything before belongs to record N, everything after is record N+1.
|
|
- **Trailing whitespace trap**: Phones are often preceded by a space that lands in `before` text. Use `\s*$` in the walk-back regex to tolerate it.
|
|
|
|
### Critical pitfalls
|
|
|
|
1. **Collapse whitespace AFTER splitting, never before.** If you `/\s+/ → ' '` first, multi-space gaps vanish and you can't detect column boundaries.
|
|
2. **Remove fields with `' '` replacement, not `''`.** Empty-string removal concatenates adjacent words and loses boundaries.
|
|
3. **VIN regex must accept O/I/Q** — Tesseract turns `0`→`O` and `1`→`I` frequently. Normalize after matching.
|
|
4. **Don't use `{0,3}` greedy in walk-back** — `{0,1}` keeps just the customer name with the next record. Larger ranges eat into the previous record's service/notes fields.
|
|
5. **Strip all-caps header lines and separator lines** (`------`) before parsing blocks.
|
|
6. **⚠️ Blank-line filter destroys block separators.** A filter like `t.split('\n').filter(l => l.trim().length > 0)` removes blank lines, destroying the block separators needed for `t.split(/\n\s*\n/)`. Use `.map()` instead — return `''` for blank lines, strip only unwanted content.
|
|
7. **⚠️ Header detection false-positives.** The regex `/\b(name|phone|date|time|vehicle|vin)\b/i` matches "VIN" inside a data line like "2023 Honda RIDGELINE VIN VIN: SEPYK3F70PB011523", causing the parser to strip the entire appointment. Fix: require 3+ header-word matches AND no phone/VIN pattern in the line before stripping.
|
|
8. **⚠️ Trailing-space breaks gap-walk regex.** After phone extraction, `before` text often ends with a trailing space. The regex `/\s{2,}(?=\S+(?:\s+\S+){0,1}$)/` fails because `\S+$` can't match. Fix: add `\s*$` tolerance: `/\s{2,}(?=\S+(?:\s+\S+){0,1}\s*$)/`
|
|
9. **⚠️ Vehicle detection needs year+make combination.** A part like "Tailgate wiring harness recall. 15,000 2023 Honda RIDGELINE" contains both vehicle and service. Basic detection picks the whole part. Fix: prefer parts with BOTH a year pattern AND a known make word.
|
|
10. **⚠️ Advisor+RO prefix contamination.** Shop management screenshots often include advisor names and RO codes: `Rrahman [REP] Service description`. Strip with `^([A-Z][a-z]+)\s+(\[[^\]]+\])\s*(.*)` and capture the RO code in notes. **Strip advisor noise BEFORE field extraction** — if done after, advisor names can bleed into customer name or phone extraction.
|
|
- **OCR corruption variants**: Tesseract frequently garbles advisor names. Strip these known corruptions: `Rrahman`, `Rrshman`, `Grajqevci`, `Grokevel`, and line-noise patterns like `eE 3 zoom`, `Eee om 3 gem`. Use a single `replace()` with alternation: `/\b(?:Rrahman|Rrshman|Grajqevci|Grokevel)\b/gi`.
|
|
11. **⚠️ Service regex must stop at newline.** A regex like `\[REP\]\s*([^\[]*)` captures everything until another bracket — including vehicle info on the next line (e.g., "2023 Honda RIDGELINE"). Fix: use `[^\n\[]*` to stop at newlines. Service descriptions and vehicles are always on separate lines in table-formatted screenshots.
|
|
12. **⚠️ Newlines must be preserved in block text.** When passing blocks to `parseOne()`, do NOT `.replace(/\n/g, ' ')`. Newlines are the primary separator between service line and vehicle line. Collapsing them into spaces merges vehicle info into service text. Split parts on BOTH `\s{2,}` and `\n`: `/ \s{2,}|\n/`.
|
|
13. **⚠️ Junk token accumulation.** After field extraction, residual labels like `RO`, `VIN`, `os`, `b=` remain in the text. Strip them in a dedicated cleanup step before part splitting: `b.replace(/\bVIN\b/gi, ' ').replace(/\bRO\b/g, ' ').replace(/\b(?:os|b=)\b/gi, ' ')`. Without this, `RO` becomes a prefix on customer names and `VIN` contaminates vehicle or service fields.
|
|
14. **⚠️ Regex whack-a-mole ceiling.** When you've added more than 15 regex patterns and are still hitting edge cases every session, the format is too variable for rule-based parsing. At that point, switch to a local small LLM (1-3B params, Q4 quant): Qwen2.5-1.5B or Llama-3.2-3B on llama.cpp. The LLM replaces only the `parseWithRules()` step — Tesseract.js still handles OCR in-browser. Setup: `llama-server -hf bartowski/Qwen2.5-1.5B-Instruct-GGUF:Q4_K_M -ngl 99` on GPU.
|
|
|
|
### Cross-block customer name carryover
|
|
|
|
When OCR places a customer name at the end of one block but the record's data starts the next block (common in stacked card layouts):
|
|
|
|
```js
|
|
// After assigning service text, check for trailing all-caps name
|
|
if (appt.serviceType){
|
|
var tn = appt.serviceType.match(/\s+([A-Z]{2,}(?:\s+[A-Z]{2,}){1,2})\s*$/);
|
|
if (tn && tn[1].length > 3){
|
|
appt.serviceType = appt.serviceType.substring(0, appt.serviceType.length - tn[0].length).trim();
|
|
nextName = tn[1]; // carry to next block
|
|
}
|
|
}
|
|
```
|
|
|
|
The `parseOne` function should return `nextName` so it can be passed as `carryName` to the next block's parse call.
|
|
|
|
### Advisor + RO code stripping
|
|
|
|
```js
|
|
var roCode = null;
|
|
parts = parts.map(function(p){
|
|
var m = p.match(/^([A-Z][a-z]+)\s+(\[[^\]]+\])\s*(.*)/);
|
|
if (m){ roCode = roCode || m[2]; return m[3] || ''; }
|
|
return p;
|
|
});
|
|
// Later: if (roCode) appt.notes = 'RO: ' + roCode;
|
|
```
|
|
|
|
## When NOT to Use
|
|
|
|
- OCR quality is so poor that fields are unreadable → tell the user to get a better screenshot
|
|
- Data is truly unstructured prose (narrative text) → use an LLM
|
|
- Only one record in the screenshot → simpler single-field extraction suffices
|
|
- **Regex fatigue**: You've added 10+ patterns and still hit edge cases every session. The format is too variable — switch to a local small LLM (see Pitfall #14)
|
|
|
|
## Switching to Local LLM
|
|
|
|
When regex hits the ceiling, replace only the `parseWithRules()` step with a local model. Tesseract.js still runs in-browser; the browser sends OCR text to a local `llama-server` endpoint.
|
|
|
|
**Recommended models for text parsing (not image analysis):**
|
|
|
|
| Model | Size (Q4_K_M) | VRAM needed |
|
|
|-------|--------------|-------------|
|
|
| Qwen2.5-1.5B-Instruct | ~1 GB | ~1.5 GB |
|
|
| Llama-3.2-3B-Instruct | ~2 GB | ~2.5 GB |
|
|
| Gemma-2-2B-Instruct | ~1.5 GB | ~2 GB |
|
|
|
|
**Quick setup (llama.cpp):**
|
|
```bash
|
|
# Install
|
|
brew install llama.cpp # or git clone + cmake
|
|
|
|
# Launch server with GPU offload
|
|
llama-server -hf bartowski/Qwen2.5-1.5B-Instruct-GGUF:Q4_K_M -ngl 99
|
|
|
|
# Test
|
|
curl http://localhost:8080/v1/chat/completions \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"messages":[{"role":"user","content":"Extract as JSON: names, phones, VINs from this OCR text: ..."}]}'
|
|
```
|
|
|
|
**Prompt template for appointment parsing:**
|
|
```
|
|
Extract appointment fields from this OCR text as JSON array.
|
|
Fields: customerName, customerPhone, customerEmail, vin, vehicleInfo, serviceType, notes.
|
|
OCR text:
|
|
<raw text here>
|
|
Return only valid JSON, no explanation.
|
|
```
|
|
|
|
Keep the review modal — the LLM replaces parsing, not the whole workflow.
|
|
|
|
## Related
|
|
|
|
- `ocr-and-documents` — for extracting text from PDFs/scans (pymupdf, marker-pdf)
|
|
- `references/appointment-parser.js` — full production parser from the ShopProQuote project
|
|
- `references/parser-patterns.md` — compact extraction regex catalog with verified test cases
|
|
- `references/regex-catalog.md` — complete regex catalog with OCR corruption variants
|
|
- `llama-cpp` — local LLM setup when regex hits the ceiling (see Pitfall #14 and "Switching to Local LLM")
|