Structured Extraction from OCR/Messy Text

Pattern: use a small local LLM (1-3B params) to extract structured JSON from messy, semi-structured text — OCR output, logs, emails, forms.

Model recommendation

Qwen2.5-1.5B-Instruct Q4_K_M (~1 GB) is excellent for this task class. It follows structured output instructions well at this size, runs at ~30 tok/s on CPU-only (4 threads), and loads in under 500ms. The 32K context window is overkill for extraction tasks; -c 4096 is sufficient.

Other viable options at similar size: Llama-3.2-3B, Gemma-2-2B.

Prompt template

The key insight: small models need explicit rules about what NOT to include, not just what TO include. The system prompt must call out specific contamination patterns.

Extract appointment details from OCR text. Return ONLY valid JSON, no markdown.

RULES:
- customerName: person name before phone number. Strip "RO" prefix. NEVER include advisor names.
- customerPhone: 10 digits only, no dashes
- customerEmail: actual email with @. If text after phone is advisor+opcode, leave empty.
- vin: 17-char uppercase VIN
- vehicleInfo: year + make + model
- serviceType: work description after opcode
- opCode: only bracketed code like [REP] — no advisor names
- appointmentTime: 24h format (e.g. "09:00")
- duration: integer minutes

JSON: {"appointments":[{"customerName":"","customerPhone":"","customerEmail":"","vin":"","vehicleInfo":"","serviceType":"","opCode":"","appointmentTime":"","duration":0}]}

Critical rules that fixed real failures:

NEVER include advisor names — without this, the model puts "Rrahman Grajqevci [REP]" in the opCode field
Strip "RO" prefix — OCR text often has "RO Gary Bowers"; the model needs explicit instruction to drop RO
If text after phone is advisor+opcode, leave empty — prevents email field from catching advisor name + bracket pattern

Temperature

Always use temperature: 0 for extraction tasks. Any non-zero temperature introduces field hallucination risk.

Markdown fence stripping

Small models reliably wrap JSON output in ``` fences even when told not to. Always strip client-side:

raw = raw.replace(/^```(?:json)?\s*\n?/i, '').replace(/\n?```\s*$/i, '');

Field normalization

Post-extraction normalization avoids subtle bugs:

appointments = parsed.appointments.map(a => ({
    customerName: a.customerName || '',
    customerPhone: (a.customerPhone || '').replace(/\D/g, ''),  // digits only
    customerEmail: a.customerEmail || '',
    vin: (a.vin || '').toUpperCase(),
    vehicleInfo: a.vehicleInfo || '',
    serviceType: a.serviceType || '',
    appointmentTime: a.appointmentTime || '',
    duration: parseInt(a.duration) || 60,
    notes: a.opCode ? 'RO: ' + a.opCode : ''
}));

OCR Preprocessing (Tesseract time-digit failures)

Small digits in narrow table columns (e.g., appointment times like 9:00, 12:00, 3:00) are frequently misread by Tesseract as identical values. Upscaling + sharpening before OCR significantly improves digit recognition.

Browser-side Canvas preprocessing

Before passing the image to Tesseract.js, preprocess on a canvas:

const preprocessed = await new Promise((resolve, reject) => {
    const img = new Image();
    img.onload = () => {
        const canvas = document.createElement('canvas');
        const ctx = canvas.getContext('2d');
        // Upscale 2x — critical for small text
        canvas.width = img.width * 2;
        canvas.height = img.height * 2;
        // Boost contrast and brightness
        ctx.filter = 'contrast(1.2) brightness(1.1)';
        ctx.drawImage(img, 0, 0, canvas.width, canvas.height);
        // Unsharp mask: overlay semi-transparent shifted copies
        ctx.filter = 'none';
        ctx.globalAlpha = 0.3;
        ctx.drawImage(canvas, -1, 0, canvas.width, canvas.height);
        ctx.drawImage(canvas, 1, 0, canvas.width, canvas.height);
        ctx.drawImage(canvas, 0, -1, canvas.width, canvas.height);
        ctx.drawImage(canvas, 0, 1, canvas.width, canvas.height);
        ctx.globalAlpha = 1.0;
        canvas.toBlob(blob => blob ? resolve(blob) : reject(new Error('toBlob failed')), 'image/png');
    };
    img.onerror = () => reject(new Error('Image load failed'));
    img.src = URL.createObjectURL(file);
});

// Then pass to Tesseract
const { data: { text } } = await Tesseract.recognize(preprocessed, 'eng', { ... });

Why this works: Tesseract's default DPI assumption is 70. Doubling the pixel dimensions effectively doubles the perceived DPI. The unsharp mask enhances digit edges that Tesseract's LSTM models rely on for character discrimination.

Limitation: If preprocessing alone doesn't fix time-digit recognition (narrow columns with very small fonts), the LLM cannot compensate — it only sees what Tesseract outputs. In that case, consider cropping the time column and OCR-ing it separately at 4x upscale.

Fallback

Always have a deterministic fallback (regex parser, manual input) for when the LLM server is unreachable. The fetch should be wrapped in try/catch:

try {
    appointments = await llmParse(text);
} catch (err) {
    console.warn('LLM unavailable, falling back:', err.message);
    appointments = ruleBasedParse(text);
}

5.2 KiB Raw Blame History