# Reddit Scraping for Briefings

Reddit aggressively blocks programmatic access. Here's the definitive landscape as of June 2026.

## What Works

| Method | Subreddit | Status |
|--------|-----------|--------|
| `.rss` (Atom XML) | `r/wallstreetbets` | ✅ **HTTP 200** — valid XML with titles |
| `.rss` (Atom XML) | All other subs | ❌ HTTP 403 |

**That's it.** Only WSB's RSS feed is accessible without authentication.

## What Does NOT Work

| Method | Subreddit | Error |
|--------|-----------|-------|
| Firecrawl SDK | Any Reddit URL | `403: "We do not support this site"` |
| `.json` API | Any sub | Returns HTML "whoa there, pardner" |
| `.rss` | `r/stocks` | `HTTP 403` |
| `.rss` | `r/StockMarket` | `HTTP 403` |
| `.rss` | `r/pennystocks` | `HTTP 403` |
| `old.reddit.com` | Any sub | Same blocks as `www.reddit.com` |

## WSB RSS Pattern

```bash
curl -sL -H "User-Agent: Mozilla/5.0 (X11; Linux x86_64)" \
  "https://www.reddit.com/r/wallstreetbets/.rss" -o /tmp/wsb.xml
```

Parse the Atom XML for `<title>` elements. Extract ticker mentions (`$TICKER` or ASYMBOL patterns) and sentiment from titles.

## If You Need More Reddit Data

- **Reddit's official API** requires OAuth and app registration at reddit.com/prefs/apps. Even then, rate limits are strict for free-tier apps.
- **Pushshift** (historical Reddit data) has been unreliable since 2023 API changes.
- **Paid alternatives**: SocialGrep, TrackReddit, etc. — none tested from cron.

## Recommendation

For stock briefing cron jobs: use WSB `.rss` as the sole Reddit source. Do not attempt to scrape other subreddits — they will fail every time. If WSB chatter isn't enough, supplement with StockTwits trending or financial news sentiment instead.