hermes-config/skills/productivity/ocr-text-parsing/SKILL.md

name, description, version, author, license, platforms, metadata

name	description	version	author	license	platforms	metadata
ocr-text-parsing	Parse semi-structured OCR text into typed records using regex — no AI/API needed. Boundary detection, field extraction, OCR error tolerance.	1.1.0	Hermes Agent	MIT	linux macos windows browser	hermes tags OCR Parsing Regex Data-Extraction Browser CPU

OCR Text → Structured Records (Rule-Based)

When OCR output contains multiple records (appointments, contacts, invoices) and you need to parse them into typed fields without calling an AI API. Runs in-browser or server-side, instant, zero cost.

When to Use

• Screenshot → OCR → structured form fields
• Multiple records in one image (table, list, stacked cards)
• User says "no API", "use CPU", "make it efficient", "parse locally"
• OCR quality is imperfect (Tesseract artifacts, garbled characters)

Core Algorithm

Strip noise → Extract fields → Split → Assign (strict order):

1. Strip advisor/tech noise FIRST — before any field extraction. Remove known advisor names and OCR corruption variants (see Pitfall #10). This prevents advisor text from contaminating customer name or service.
2. Extract structured fields in order — Time → Duration → Phone → Email → VIN → Date. Use targeted regex and remove each match from the text. Order matters: email before VIN prevents garyb9623@gmail.com digits from being mistaken for VIN characters. Phone before email prevents phone digits from matching email fragments.
3. Post-extraction cleanup — strip residual label tokens (RO, VIN, os, b=). These are common OCR artifacts that survive field extraction.
4. Extract OP code + service — if bracketed codes like [REP], [23-046:6MX00] exist, extract them and capture the service text that follows on the same line. Service regex MUST stop at newline ([^\n\[]*, not [^\[]*) to prevent vehicle info on the next line from being absorbed.
5. Extract vehicle — year + make + model ((\d{4})\s+([A-Za-z]+)\s+([A-Za-z0-9\-]+)). Fallback: year + make only.
6. Name from remainder — first remaining part after stripping leading non-letter characters.
7. Recurse — find next phone/VIN in leftover text, walk back to nearest multi-space gap, split, re-enter step 1.

Key Patterns

Phone extraction (forgiving formats)

var phoneRe = /\(?\d{3}\)?[\s.\-]*\d{3}[\s.\-]*\d{4}/;

VIN extraction (OCR-tolerant)

// Accept O/I/Q (common OCR errors for 0/1) — normalize after match
var vinRe = /\b[A-HJ-NPR-Z0-9OIQ]{17}\b/i;
function fixVin(v) { return v.toUpperCase().replace(/O/g,'0').replace(/I/g,'1').replace(/Q/g,'0'); }

Email extraction (extract before VIN to prevent false matches)

var emailRe = /\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b/;

Extract email IMMEDIATELY after phone, before VIN. Emails like garyb9623@gmail.com contain 4-digit numbers that can false-match phone/date/VIN patterns. Removing the email from the text first prevents these collisions.

Date extraction (multiple formats → YYYY-MM-DD)

Handle: 2024-06-15, 06/20/2024, 6/20/2024, Jan 15, 2024, January 15 2024. Month-name lookup with 3-letter prefixes.

Time extraction (12h + 24h)

Handle: 9:00 AM, 1:30PM, 14:00, 8am. Convert to 24h HH:MM.

Multi-record boundary detection

• Table format: Detect when 3+ lines have consistent word counts. Skip the header line if it contains name|phone|date|time|service|vehicle|vin.
• Row-anchor split (preferred): When each record starts with a predictable pattern like 9:00AM 0.5 hrs, use it as the primary block delimiter instead of blank lines. Find all anchor positions with RegExp.exec() in a loop, split the text at each anchor index. This survives OCR horizontal-scan order where blank lines may be absent. Falls back to blank-line splitting if fewer than 2 anchors found.

  var rowAnchorRe = /(\d{1,2}):(\d{2})\s*(AM|PM)\s+(\d+(?:\.\d+)?)\s*hrs?/gi;
  var anchors = [];
  while ((am = rowAnchorRe.exec(t)) !== null) anchors.push({ idx: am.index, len: am[0].length });
  if (anchors.length >= 2) { /* split at anchor boundaries */ }
  else { /* fallback: blank-line split */ }
  

• Free-form: After extracting one record's fields, find the next phone/VIN. Walk back to the last \s{2,} gap (or fallback: last 1-2 capitalized words). Split there — everything before belongs to record N, everything after is record N+1.
• Trailing whitespace trap: Phones are often preceded by a space that lands in before text. Use \s*$ in the walk-back regex to tolerate it.

Critical pitfalls

1. Collapse whitespace AFTER splitting, never before. If you /\s+/ → ' ' first, multi-space gaps vanish and you can't detect column boundaries.
2. Remove fields with ' ' replacement, not ''. Empty-string removal concatenates adjacent words and loses boundaries.
3. VIN regex must accept O/I/Q — Tesseract turns 0→O and 1→I frequently. Normalize after matching.
4. Don't use {0,3} greedy in walk-back — {0,1} keeps just the customer name with the next record. Larger ranges eat into the previous record's service/notes fields.
5. Strip all-caps header lines and separator lines (------) before parsing blocks.
6. ⚠️ Blank-line filter destroys block separators. A filter like t.split('\n').filter(l => l.trim().length > 0) removes blank lines, destroying the block separators needed for t.split(/\n\s*\n/). Use .map() instead — return '' for blank lines, strip only unwanted content.
7. ⚠️ Header detection false-positives. The regex /\b(name|phone|date|time|vehicle|vin)\b/i matches "VIN" inside a data line like "2023 Honda RIDGELINE VIN VIN: SEPYK3F70PB011523", causing the parser to strip the entire appointment. Fix: require 3+ header-word matches AND no phone/VIN pattern in the line before stripping.
8. ⚠️ Trailing-space breaks gap-walk regex. After phone extraction, before text often ends with a trailing space. The regex /\s{2,}(?=\S+(?:\s+\S+){0,1}$)/ fails because \S+$ can't match. Fix: add \s*$ tolerance: /\s{2,}(?=\S+(?:\s+\S+){0,1}\s*$)/
9. ⚠️ Vehicle detection needs year+make combination. A part like "Tailgate wiring harness recall. 15,000 2023 Honda RIDGELINE" contains both vehicle and service. Basic detection picks the whole part. Fix: prefer parts with BOTH a year pattern AND a known make word.
10. ⚠️ Advisor+RO prefix contamination. Shop management screenshots often include advisor names and RO codes: Rrahman [REP] Service description. Strip with ^([A-Z][a-z]+)\s+(\[[^\]]+\])\s*(.*) and capture the RO code in notes. Strip advisor noise BEFORE field extraction — if done after, advisor names can bleed into customer name or phone extraction.
    • OCR corruption variants: Tesseract frequently garbles advisor names. Strip these known corruptions: Rrahman, Rrshman, Grajqevci, Grokevel, and line-noise patterns like eE 3 zoom, Eee om 3 gem. Use a single replace() with alternation: /\b(?:Rrahman|Rrshman|Grajqevci|Grokevel)\b/gi.
11. ⚠️ Service regex must stop at newline. A regex like \[REP\]\s*([^\[]*) captures everything until another bracket — including vehicle info on the next line (e.g., "2023 Honda RIDGELINE"). Fix: use [^\n\[]* to stop at newlines. Service descriptions and vehicles are always on separate lines in table-formatted screenshots.
12. ⚠️ Newlines must be preserved in block text. When passing blocks to parseOne(), do NOT .replace(/\n/g, ' '). Newlines are the primary separator between service line and vehicle line. Collapsing them into spaces merges vehicle info into service text. Split parts on BOTH \s{2,} and \n: / \s{2,}|\n/.
13. ⚠️ Junk token accumulation. After field extraction, residual labels like RO, VIN, os, b= remain in the text. Strip them in a dedicated cleanup step before part splitting: b.replace(/\bVIN\b/gi, ' ').replace(/\bRO\b/g, ' ').replace(/\b(?:os|b=)\b/gi, ' '). Without this, RO becomes a prefix on customer names and VIN contaminates vehicle or service fields.
14. ⚠️ Regex whack-a-mole ceiling. When you've added more than 15 regex patterns and are still hitting edge cases every session, the format is too variable for rule-based parsing. At that point, switch to a local small LLM (1-3B params, Q4 quant): Qwen2.5-1.5B or Llama-3.2-3B on llama.cpp. The LLM replaces only the parseWithRules() step — Tesseract.js still handles OCR in-browser. Setup: llama-server -hf bartowski/Qwen2.5-1.5B-Instruct-GGUF:Q4_K_M -ngl 99 on GPU.

Cross-block customer name carryover

When OCR places a customer name at the end of one block but the record's data starts the next block (common in stacked card layouts):

// After assigning service text, check for trailing all-caps name
if (appt.serviceType){
    var tn = appt.serviceType.match(/\s+([A-Z]{2,}(?:\s+[A-Z]{2,}){1,2})\s*$/);
    if (tn && tn[1].length > 3){
        appt.serviceType = appt.serviceType.substring(0, appt.serviceType.length - tn[0].length).trim();
        nextName = tn[1]; // carry to next block
    }
}

The parseOne function should return nextName so it can be passed as carryName to the next block's parse call.

Advisor + RO code stripping

var roCode = null;
parts = parts.map(function(p){
    var m = p.match(/^([A-Z][a-z]+)\s+(\[[^\]]+\])\s*(.*)/);
    if (m){ roCode = roCode || m[2]; return m[3] || ''; }
    return p;
});
// Later: if (roCode) appt.notes = 'RO: ' + roCode;

When NOT to Use

• OCR quality is so poor that fields are unreadable → tell the user to get a better screenshot
• Data is truly unstructured prose (narrative text) → use an LLM
• Only one record in the screenshot → simpler single-field extraction suffices
• Regex fatigue: You've added 10+ patterns and still hit edge cases every session. The format is too variable — switch to a local small LLM (see Pitfall #14)

Switching to Local LLM

When regex hits the ceiling, replace only the parseWithRules() step with a local model. Tesseract.js still runs in-browser; the browser sends OCR text to a local llama-server endpoint.

Recommended models for text parsing (not image analysis):

Model	Size (Q4_K_M)	VRAM needed
Qwen2.5-1.5B-Instruct	~1 GB	~1.5 GB
Llama-3.2-3B-Instruct	~2 GB	~2.5 GB
Gemma-2-2B-Instruct	~1.5 GB	~2 GB

Quick setup (llama.cpp):

# Install
brew install llama.cpp  # or git clone + cmake

# Launch server with GPU offload
llama-server -hf bartowski/Qwen2.5-1.5B-Instruct-GGUF:Q4_K_M -ngl 99

# Test
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Extract as JSON: names, phones, VINs from this OCR text: ..."}]}'

Prompt template for appointment parsing:

Extract appointment fields from this OCR text as JSON array.
Fields: customerName, customerPhone, customerEmail, vin, vehicleInfo, serviceType, notes.
OCR text:
<raw text here>
Return only valid JSON, no explanation.

Keep the review modal — the LLM replaces parsing, not the whole workflow.

Related

• ocr-and-documents — for extracting text from PDFs/scans (pymupdf, marker-pdf)
• references/appointment-parser.js — full production parser from the ShopProQuote project
• references/parser-patterns.md — compact extraction regex catalog with verified test cases
• references/regex-catalog.md — complete regex catalog with OCR corruption variants
• llama-cpp — local LLM setup when regex hits the ceiling (see Pitfall #14 and "Switching to Local LLM")