Honeyguide

The scrapers existed only as Apify Store listings, which are hard to search, group by platform, or find through a search engine. Maintaining a separate site by hand for close to a hundred actors would drift out of date within weeks as prices, inputs and usage numbers change. The site had to be zero-maintenance: anything published on Apify appears automatically, anything private never can.

A Next.js App Router site reads two public Apify endpoints: the store listing for names, pricing and usage, and each actor's default build for its input schema and output fields. Responses are cached for an hour, so pages stay current with no API token and no manual edits. A scraper's platform is inferred from the first word of its Apify name.

Cold builds request every scraper at once, so the data layer runs through a small concurrency gate (two requests in flight) with up to ten jittered retries. That change stopped builds failing on Apify's per-second rate limit.

The site, each platform hub and each scraper page gets its own generated social card, structured data, sitemap entry and FAQ. Hand-written MDX files in src/content can layer a guide or extra FAQs onto any page without replacing the generated parts.

The content engine is a Python pipeline. It syncs fact sheets from the same Apify API, writes a post with an LLM agent, generates images, then runs deterministic gates: SEO and AI-citation scores, a fact check against the sheets, internal link checks and a compile of the MDX with the site's own compiler. A post can only state facts present in the sheets, and publishing means opening a pull request; merging it makes the post live.