initial commit

2026-07-12 10:17:17 -04:00
commit dab5a4ebc6
1424 changed files with 330463 additions and 0 deletions
@@ -0,0 +1,93 @@
+---
+name: ocr-data-extraction
+description: "Extract structured data (names, phones, VINs, dates) from OCR'd text using rule-based regex parsing — no AI/API needed."
+version: 1.0.0
+author: Hermes Agent
+license: MIT
+metadata:
+  hermes:
+    tags: [OCR, parsing, regex, data-extraction, screenshots, browser, Tesseract]
+    related_skills: [ocr-and-documents]
+---
+
+# OCR → Structured Data Extraction (Rule-Based)
+
+Extract structured fields from OCR'd text using pure regex — no API keys, no network calls, no AI. Runs entirely on CPU, either in-browser (Tesseract.js CDN) or server-side (Tesseract CLI).
+
+**When to use this approach vs AI/API:**
+- Data follows predictable patterns (phones, VINs, dates, names)
+- Latency matters (instant vs 1-3s API call)
+- Cost matters (free vs per-call billing)
+- Privacy matters (data stays local)
+- User explicitly prefers CPU-based — **honor this signal immediately**
+
+## Quick Start (Browser)
+
+```html
+<script src="https://cdn.jsdelivr.net/npm/tesseract.js@5/dist/tesseract.min.js"></script>
+```
+
+Then call `Tesseract.recognize(file, 'eng', { logger })` to get text, then run the parser.
+
+## Parser Architecture
+
+The parser uses **layered extraction** in this order:
+
+### 1. Block Splitting
+Split OCR text on blank lines first. If that fails, detect table-like structures (consistent word counts across lines).
+
+### 2. Structured Field Extraction (per block)
+Extract and REMOVE these from the text first (order matters — early extraction simplifies later parsing):
+
+```
+Phone:  /\(?\d{3}\)?[\s.\-]*\d{3}[\s.\-]*\d{4}/
+VIN:    /\b[A-HJ-NPR-Z0-9OIQ]{17}\b/i   ← lenient: accepts O→0, I→1, Q→0
+Date:   YYYY-MM-DD, MM/DD/YYYY, "Jan 15, 2024"
+Time:   9:00 AM, 1:30PM, 14:00, 8am
+Duration: /\b(\d+)\s*(?:min|minutes?|hrs?|hours?)\b/i
+```
+
+**OCR VIN tolerance**: Tesseract commonly confuses `0→O`, `1→I`. Accept O/I/Q in the regex, then normalize:
+```js
+function fixVin(v) { return v.toUpperCase().replace(/O/g,'0').replace(/I/g,'1').replace(/Q/g,'0'); }
+```
+
+### 3. Remaining Text → Name / Vehicle / Service
+After removing structured fields, split remaining text on multi-spaces (preserves column structure). Then:
+- **Name**: first part, or consecutive capitalized words without digits
+- **Vehicle**: part containing year pattern (`19xx`/`20xx`) or known makes (`ford|toyota|honda|bmw|...`)
+- **Service**: everything else
+
+### 4. Advisor + RO Code Stripping
+If the source has `AdvisorName [RO_CODE]` prefixes, strip them:
+```js
+var m = part.match(/^([A-Z][a-z]+)\s+(\[[^\]]+\])\s+(.*)/);
+if (m) { roCode = m[2]; return m[3]; }  // return service, save RO as note
+```
+
+### 5. Multi-Appointment Boundary Detection
+When multiple appointments appear without blank-line separation, detect boundaries by:
+- Find the next phone/VIN in the remaining text
+- **Walk back** to the nearest multi-space gap or capitalized name before it
+- Split there — text before goes to current appointment, text after recurses
+- Regex for gap walk-back: `/\s{2,}(?=\S+(?:\s+\S+){0,1}\s*$)/`
+- Fallback (no gap): detect last 1-2 capitalized words before phone: `/([A-Z][A-Za-z]+(?:\s+\S+){0,1})\s*$/`
+
+### 6. Header/Noise Stripping
+- Separator lines: `/^[\-–=_*#]{3,}$/` → remove
+- Short all-caps headers: `< 25 chars, all uppercase letters/spaces` → remove
+- Table header row: contains `name|phone|date|time|service|vehicle|vin` → skip first line
+- Known header words: `schedule|appointments|roster|calendar|upcoming` → filter from parts
+
+## Pitfalls
+
+- **Don't collapse whitespace before splitting**: Split on `\s{2,}` FIRST, then clean each part. If you collapse to single spaces first, multi-space splitting silently breaks.
+- **Trailing space before boundary breaks regex**: Phone/VIN markers are often preceded by a space in the OCR. Trim or use `\s*$` in lookahead regexes.
+- **"O" in VIN ≠ letter O**: Always use lenient VIN regex and normalize. OCR will turn zeros into O's.
+- **Advisor names vs customer names**: In shop management systems, "Word [CODE]" is advisor+RO, not customer. Strip it.
+- **Block recursion can re-include old text**: When recursing, pass only the text AFTER the boundary split point, not the full remaining `b` variable.
+- **Table detection false positives**: Only use line-by-line table parsing when lines have consistent word counts (≤3 word difference from average). Exclude header line before checking consistency.
+
+## Reference File
+
+See `references/parser-patterns.md` for the full extraction regex catalog and test cases.