ray
121 lines
5.2 KiB
Markdown
121 lines
5.2 KiB
Markdown
# Structured Extraction from OCR/Messy Text
|
|
|
|
Pattern: use a small local LLM (1-3B params) to extract structured JSON from messy, semi-structured text — OCR output, logs, emails, forms.
|
|
|
|
## Model recommendation
|
|
|
|
**Qwen2.5-1.5B-Instruct Q4_K_M** (~1 GB) is excellent for this task class. It follows structured output instructions well at this size, runs at ~30 tok/s on CPU-only (4 threads), and loads in under 500ms. The 32K context window is overkill for extraction tasks; `-c 4096` is sufficient.
|
|
|
|
Other viable options at similar size: Llama-3.2-3B, Gemma-2-2B.
|
|
|
|
## Prompt template
|
|
|
|
The key insight: small models need explicit rules about what NOT to include, not just what TO include. The system prompt must call out specific contamination patterns.
|
|
|
|
```
|
|
Extract appointment details from OCR text. Return ONLY valid JSON, no markdown.
|
|
|
|
RULES:
|
|
- customerName: person name before phone number. Strip "RO" prefix. NEVER include advisor names.
|
|
- customerPhone: 10 digits only, no dashes
|
|
- customerEmail: actual email with @. If text after phone is advisor+opcode, leave empty.
|
|
- vin: 17-char uppercase VIN
|
|
- vehicleInfo: year + make + model
|
|
- serviceType: work description after opcode
|
|
- opCode: only bracketed code like [REP] — no advisor names
|
|
- appointmentTime: 24h format (e.g. "09:00")
|
|
- duration: integer minutes
|
|
|
|
JSON: {"appointments":[{"customerName":"","customerPhone":"","customerEmail":"","vin":"","vehicleInfo":"","serviceType":"","opCode":"","appointmentTime":"","duration":0}]}
|
|
```
|
|
|
|
**Critical rules that fixed real failures**:
|
|
- `NEVER include advisor names` — without this, the model puts "Rrahman Grajqevci [REP]" in the opCode field
|
|
- `Strip "RO" prefix` — OCR text often has "RO Gary Bowers"; the model needs explicit instruction to drop RO
|
|
- `If text after phone is advisor+opcode, leave empty` — prevents email field from catching advisor name + bracket pattern
|
|
|
|
## Temperature
|
|
|
|
Always use `temperature: 0` for extraction tasks. Any non-zero temperature introduces field hallucination risk.
|
|
|
|
## Markdown fence stripping
|
|
|
|
Small models reliably wrap JSON output in ``` fences even when told not to. Always strip client-side:
|
|
|
|
```javascript
|
|
raw = raw.replace(/^```(?:json)?\s*\n?/i, '').replace(/\n?```\s*$/i, '');
|
|
```
|
|
|
|
## Field normalization
|
|
|
|
Post-extraction normalization avoids subtle bugs:
|
|
|
|
```javascript
|
|
appointments = parsed.appointments.map(a => ({
|
|
customerName: a.customerName || '',
|
|
customerPhone: (a.customerPhone || '').replace(/\D/g, ''), // digits only
|
|
customerEmail: a.customerEmail || '',
|
|
vin: (a.vin || '').toUpperCase(),
|
|
vehicleInfo: a.vehicleInfo || '',
|
|
serviceType: a.serviceType || '',
|
|
appointmentTime: a.appointmentTime || '',
|
|
duration: parseInt(a.duration) || 60,
|
|
notes: a.opCode ? 'RO: ' + a.opCode : ''
|
|
}));
|
|
```
|
|
|
|
## OCR Preprocessing (Tesseract time-digit failures)
|
|
|
|
Small digits in narrow table columns (e.g., appointment times like 9:00, 12:00, 3:00) are frequently misread by Tesseract as identical values. Upscaling + sharpening before OCR significantly improves digit recognition.
|
|
|
|
### Browser-side Canvas preprocessing
|
|
|
|
Before passing the image to Tesseract.js, preprocess on a canvas:
|
|
|
|
```javascript
|
|
const preprocessed = await new Promise((resolve, reject) => {
|
|
const img = new Image();
|
|
img.onload = () => {
|
|
const canvas = document.createElement('canvas');
|
|
const ctx = canvas.getContext('2d');
|
|
// Upscale 2x — critical for small text
|
|
canvas.width = img.width * 2;
|
|
canvas.height = img.height * 2;
|
|
// Boost contrast and brightness
|
|
ctx.filter = 'contrast(1.2) brightness(1.1)';
|
|
ctx.drawImage(img, 0, 0, canvas.width, canvas.height);
|
|
// Unsharp mask: overlay semi-transparent shifted copies
|
|
ctx.filter = 'none';
|
|
ctx.globalAlpha = 0.3;
|
|
ctx.drawImage(canvas, -1, 0, canvas.width, canvas.height);
|
|
ctx.drawImage(canvas, 1, 0, canvas.width, canvas.height);
|
|
ctx.drawImage(canvas, 0, -1, canvas.width, canvas.height);
|
|
ctx.drawImage(canvas, 0, 1, canvas.width, canvas.height);
|
|
ctx.globalAlpha = 1.0;
|
|
canvas.toBlob(blob => blob ? resolve(blob) : reject(new Error('toBlob failed')), 'image/png');
|
|
};
|
|
img.onerror = () => reject(new Error('Image load failed'));
|
|
img.src = URL.createObjectURL(file);
|
|
});
|
|
|
|
// Then pass to Tesseract
|
|
const { data: { text } } = await Tesseract.recognize(preprocessed, 'eng', { ... });
|
|
```
|
|
|
|
**Why this works**: Tesseract's default DPI assumption is 70. Doubling the pixel dimensions effectively doubles the perceived DPI. The unsharp mask enhances digit edges that Tesseract's LSTM models rely on for character discrimination.
|
|
|
|
**Limitation**: If preprocessing alone doesn't fix time-digit recognition (narrow columns with very small fonts), the LLM cannot compensate — it only sees what Tesseract outputs. In that case, consider cropping the time column and OCR-ing it separately at 4x upscale.
|
|
|
|
## Fallback
|
|
|
|
Always have a deterministic fallback (regex parser, manual input) for when the LLM server is unreachable. The fetch should be wrapped in try/catch:
|
|
|
|
```javascript
|
|
try {
|
|
appointments = await llmParse(text);
|
|
} catch (err) {
|
|
console.warn('LLM unavailable, falling back:', err.message);
|
|
appointments = ruleBasedParse(text);
|
|
}
|
|
```
|