ray
105 lines
5.2 KiB
Markdown
105 lines
5.2 KiB
Markdown
# Rule-Based OCR Parser Architecture
|
|
|
|
Reference for the `parseWithRules()` parser in `appointments.html` (ShopProQuote). Extracts structured appointment data from Tesseract.js OCR output — no AI/API, pure JavaScript regex, runs entirely in the browser.
|
|
|
|
## Architecture Overview
|
|
|
|
```
|
|
Raw OCR text
|
|
→ Phase 1: Normalize & Clean (strip headers, separators, metadata)
|
|
→ Phase 2: Block Splitting (row-anchor primary, blank-line fallback, table fallback)
|
|
→ Phase 3: parseOne() recursive extraction per block
|
|
→ Output: array of {customerName, customerPhone, customerEmail, vin, vehicleInfo, serviceType, notes}
|
|
```
|
|
|
|
## Phase 2 — Row-Anchor Splitting (Primary)
|
|
|
|
Instead of splitting on `\n\n` (fragile — Tesseract's horizontal table scan often produces text with inconsistent blank lines), split on **Time + Duration patterns**:
|
|
|
|
```javascript
|
|
var rowAnchorRe = /(\d{1,2}):(\d{2})\s*(AM|PM)\s+(\d+(?:\.\d+)?)\s*hrs?/gi;
|
|
```
|
|
|
|
Find all anchor positions, slice text at boundaries. Each chunk starts with a time+duration — one appointment per chunk.
|
|
|
|
Fallback: if < 2 anchors found, revert to `\n\n` blank-line splitting.
|
|
|
|
## Phase 3 — parseOne() Consume-and-Destroy Order
|
|
|
|
**Critical: extraction order is strict.** Each field is matched and removed from the text before the next field is searched — prevents false matches.
|
|
|
|
```
|
|
1a. Time (anchor) — extract first, remove
|
|
1b. Duration — minutes/hours, remove
|
|
1c. Phone — 10-digit, any format, remove
|
|
1d. Email — extract before VIN (prevents email digits matching VIN/date)
|
|
1e. VIN — 17-char boundary, OCR-tolerant (O→0, I→1, Q→0), remove
|
|
1f. Post-VIN cleanup — strip residual "VIN" label, standalone "RO" token
|
|
1g. Advisor sanitize — hard-strip "Rrahman Grajqevci" and variants
|
|
1h. Date — 3 formats: YYYY-MM-DD, M/D/YYYY, "Mon DD, YYYY"
|
|
```
|
|
|
|
After structured extraction, remaining text parts are split on `\s{2,}` OR `\n` (preserves multi-line OCR structure — do NOT collapse newlines to spaces before splitting).
|
|
|
|
### Advisors / RO Codes
|
|
|
|
Pattern: `AdvisorName [RO_CODE]` is stripped from parts. The `[CODE]` is captured into `notes` as `RO: [CODE]`.
|
|
|
|
### Cross-Block Name Carry
|
|
|
|
Trailing ALL-CAPS names at the bottom of a block belong to the NEXT block. Parser detects via `serviceType.match(/\s+([A-Z]{2,}(?:\s+[A-Z]{2,}){1,2})\s*$/)` and passes as `carryName` to the next `parseOne()` call.
|
|
|
|
### Vehicle Extraction
|
|
|
|
Primary: find part matching **year + make** (e.g., "2023 Honda")
|
|
Fallback: find part matching year OR make alone
|
|
Model capture: if next part is ALL-CAPS alphanumeric and not a service keyword, append to vehicle
|
|
|
|
### Header Metadata Stripping
|
|
|
|
Before block splitting, lines matching schedule headers are removed:
|
|
- Day-of-week + date: `Wednesday Jun 18, 2025`
|
|
- Single-word labels: `Weekly`, `Advisor`, `Refresh`, `Daily`
|
|
|
|
## Key Regex Patterns
|
|
|
|
| Field | Pattern | Notes |
|
|
|-------|---------|-------|
|
|
| Phone | `\(?\d{3}\)?[\s.\-]*\d{3}[\s.\-]*\d{4}` | Any format |
|
|
| VIN | `\b[A-HJ-NPR-Z0-9OIQ]{17}\b/i` | OCR-tolerant, word-bounded |
|
|
| Email | `\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b` | Extracted pre-VIN |
|
|
| Date | `(\d{4})[\/\-.](\d{1,2})[\/\-.](\d{1,2})` | ISO format primary |
|
|
| Time | `\b(\d{1,2}):(\d{2})\s*(AM|PM)?\b` | With/without meridian |
|
|
| Duration | `\b(\d+(?:\.\d+)?)\s*(?:min|minutes?|hrs?|hours?)\b` | Decimal hours OK |
|
|
| Row Anchor | `(\d{1,2}):(\d{2})\s*(AM|PM)\s+(\d+(?:\.\d+)?)\s*hrs?` | Global flag, used for block splitting |
|
|
| Makes | 25+ automotive brands | Used in vehicle detection |
|
|
| Year | `\b(19|20)\d{2}\b` | Model years 1900-2099 |
|
|
|
|
## Pitfalls
|
|
|
|
- **Double-escape in regex strings:** When editing regex in HTML patches, `\\d` in the file is two characters. Using `\\\\d` produces four. Verify with `grep -n '\\\\\\\\d'` after editing.
|
|
- **Don't collapse newlines in block processing:** The block-iteration step must NOT do `.replace(/\n/g, ' ')`. Parts splitting uses `\n` as a delimiter; collapsing them first destroys multi-line structure.
|
|
- **"RO" token contamination:** The text "RO Gary Bowers" has standalone "RO" before the name. Strip `\bRO\b` after VIN extraction, before advisor stripping.
|
|
- **"VIN" label residue:** After VIN extraction, the word "VIN" remains. Strip `\bVIN\b` in post-VIN cleanup.
|
|
- **Email false positives:** Email addresses contain digits (e.g., `garyb9623@gmail.com`). Extract email BEFORE VIN to prevent the digits matching VIN patterns.
|
|
- **Table fallback only fires when blank-line split produces 1 block.** The row-anchor path runs first and can produce multiple blocks — table fallback is a separate code path for the `blocks.length === 1` case.
|
|
|
|
## Test Suite
|
|
|
|
7 test cases in the parser test harness cover:
|
|
1. Two appointments with blank-line separation
|
|
2. Shop header + separator lines
|
|
3. Single appointment
|
|
4. Table format (uniform columns)
|
|
5. Real 3-appointment screenshot with blank lines + email + advisor
|
|
6. OCR horizontal-scan format with NO blank lines (row-anchor test)
|
|
7. Email/code false-positive guard
|
|
|
|
Run tests via:
|
|
```bash
|
|
cd /mnt/seagate8tb/Websites/ShopProQuote
|
|
sed -n '1147,1363p' appointments.html > /tmp/parser_fn.js
|
|
echo 'module.exports = parseWithRules;' >> /tmp/parser_fn.js
|
|
node -e "var p = require('/tmp/parser_fn.js'); /* ... test cases ... */"
|
|
```
|