ray
53 lines
2.9 KiB
Markdown
53 lines
2.9 KiB
Markdown
# Recipe Scraper Site Compatibility
|
|
|
|
Mealie's built-in recipe scraper extracts structured data from recipe websites. Compatibility varies — some sites block automated access (Cloudflare, aggressive bot detection), others work flawlessly.
|
|
|
|
## Reliable (high success rate)
|
|
|
|
| Site | Notes |
|
|
|---|---|
|
|
| **BBC Good Food** (bbcgoodfood.com) | Excellent. Fast parsing, full ingredient + step extraction. |
|
|
| **RecipeTin Eats** (recipetineats.com) | Very reliable. Good image extraction. |
|
|
| **Budget Bytes** (budgetbytes.com) | Works ~70% of time. Rate-limit sensitive — add 1-2s delay between imports. |
|
|
| **Love and Lemons** (loveandlemons.com) | Works well. Vegan/vegetarian focus. |
|
|
|
|
## Unreliable / Blocked (low success rate)
|
|
|
|
| Site | Issue |
|
|
|---|---|
|
|
| **AllRecipes** (allrecipes.com) | Aggressive bot detection. HTTP 400 on most attempts. |
|
|
| **Simply Recipes** (simplyrecipes.com) | Blocks automated scrapers. |
|
|
| **Damn Delicious** (damndelicious.net) | Cloudflare or similar protection. Fails consistently. |
|
|
| **Gimme Some Oven** (gimmesomeoven.com) | Same blocking issue as above. |
|
|
|
|
## Curated import strategy (near-100% success)
|
|
|
|
When batch-importing by cuisine or theme, search **only** BBC Good Food and RecipeTin Eats for individual recipe URLs. Restricting to these two sites yields near-100% scraper success, compared to ~50-60% when mixing sources.
|
|
|
|
**Workflow for cuisine batch imports:**
|
|
1. Search `site:bbcgoodfood.com <cuisine> recipe` and `site:recipetineats.com <cuisine> recipe`
|
|
2. Pick individual recipe URLs (not category/list pages — URLs must end in a recipe slug like `/easy-chicken-fajitas`)
|
|
3. Run the batch import script from `references/batch-import.md` with 2-second delays
|
|
4. Expect 90-100% success rate
|
|
|
|
**Real-world example:** A 20-recipe batch (10 Mexican, 10 Mediterranean) from BBC Good Food + RecipeTin Eats imported with 20/20 success (100%). The same approach with mixed sources typically yields 10-12 successes.
|
|
|
|
## Workarounds for blocked sites
|
|
|
|
1. **Manual entry:** Copy-paste ingredients and steps into Mealie's manual recipe form.
|
|
2. **Screenshot + OCR:** Not implemented yet, but Mealie supports OCR recipe creation (requires OpenAI API key or local LLM).
|
|
3. **Alternative source:** Many popular recipes exist on multiple sites — search BBC Good Food or RecipeTin Eats for a similar version.
|
|
4. **Import from JSON:** If you have recipes in JSON-LD or schema.org format, use Mealie's JSON import endpoint.
|
|
|
|
## Testing a new site
|
|
|
|
```bash
|
|
# Quick test — if this returns 400, the site is likely blocked
|
|
curl -s -X POST "http://127.0.0.1:9925/api/recipes/create/url" \
|
|
-H "Authorization: Bearer *** \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"url": "https://example.com/recipe/test"}'
|
|
```
|
|
|
|
A successful import returns 200/201 with recipe JSON. 400 usually means the scraper couldn't extract data from the page. 500 means the site blocked the request entirely.
|