Define a local batch rather than a live subscription
Reddit Stream API is an independent historical archive, not official Reddit or a live event service. It provides paged GET responses, without SSE, WebSocket subscriptions, Reddit OAuth, posting, voting or private messages. A recurring job polls the archive again; it does not subscribe to incoming activity or establish how quickly current Reddit content would become available.
Start with https://api.redditstreamapi.com/v1/posts?subreddit=datasets&limit=3&page=2, the homepage example. It requests at most three posts from datasets on page two, not two pages at once. Keep the key on your server and use X-API-Key: <key> or Authorization: Bearer <key>. Consult the documentation for the route contract and the key page for account setup before payment.
For a date-bounded task, set from and to as YYYY-MM-DD UTC dates filtering created_utc. The complete final day is included. Define any next date interval deliberately so that a shared inclusive endpoint does not create an accidental overlap in your own plan. Even carefully chosen adjacent intervals cannot prove complete archive coverage, absence of duplicates or stable results when rerun.
Persist the received page before moving the checkpoint
Record filters, limit, page and the actual response in your own storage before marking that page processed. Validate data and the fields required by your destination. This makes your local transformation inspectable without another remote read. A checkpoint records your job state only; it is not an archive-issued cursor, a snapshot identifier or proof that all earlier records have been retrieved.
Content requests return up to 100 records and page ranges from 1 to 100 across separate calls. Choose a limit appropriate to your processing needs rather than treating 100 as a guaranteed throughput setting. The last returned record cannot serve as a chronological watermark: _doc is internal order, not creation time, relevance or score. min_score filters a threshold without changing that order.
Use local identifiers to investigate duplicate records when available, while preserving missing fields and unresolved parent_id or link_id references. Grouping flat records locally does not produce a guaranteed complete discussion. Reprocessing a saved response does not call the API. A new remote request may return different data, and even local deduplication cannot retroactively prevent the credit charged for a successful repeated retrieval.
Stop and retry according to observed outcomes
Read total together with relation. eq is an exact query total, gte a lower bound and unknown unreliable or null. An empty deeper page does not establish a zero total. Neither exact counts, page traversal nor smaller date windows prove an exhaustive download. Preserve has_more and window_exhausted and stop at the supported page boundary; do not invent a cursor or bypass page 100 by making more identical calls.
Handle HTTP 400 by correcting parameters, 401 by checking domain and key, 402 by pausing work and checking the account balance, and 429 with delayed bounded retries. Public /v1/status is free and keyless, but only confirms an HTTP response and static scope. It does not establish search health, archive freshness or whether an earlier uncertain retrieval consumed credit.
Each successful request, including empty or repeated responses, costs one credit. Invalid requests do not debit and upstream failures refund reservations. A client timeout may follow a charged success, and your own storage failure does not cancel it. Read pricing before scheduling repeated windows: $1 per 1000 successes, a $50 paid minimum and conditional 100 trial credits. Password-confirmed key regeneration immediately invalidates the old key, so update your workers deliberately rather than expecting overlapping credentials.
