Actor Creator Agent
Agent pipeline that turns an API manifest into a published Apify actor
- Pipeline skills
- 26
- Actor builds generated
- 70+
- Unit tests
- 210+
- Human gates
- 1 required
- Static checks before review
- 5
- Model routing
- Per skill
Actor Creator Agent takes a third-party API described by an MCP manifest and produces a production-ready Apify actor: code, input schema, README, actor manifest, store metadata, and a live build on Apify. A two-stage pipeline of 26 skills ingests the API, pauses for a human review of input fields, then generates, tests, reviews, pushes, and verifies the actor. Every skill is checkpointed, and a run can be resumed or restarted from any step.
The problem
Wrapping an external API as a marketplace actor is repetitive and demands knowledge of two platforms at once: the upstream API's parameters and auth, and Apify's actor layout, input schema rules, build system, and store conventions. Doing it by hand for dozens of APIs does not scale, and a naive single-prompt generator produces code that fails to build, leaks provider details into user-facing fields, or ships an input schema that disagrees with what the code actually reads. The hard part is not generating code once; it is getting to a verified, published actor with bounded retries and a clear record of what happened.
How it is built
Work is split into two stages. Alpha ingests the manifest with deterministic parsing (no model calls), structures endpoints into an OpenAPI spec, catalogs input fields, pauses at a human review gate where the operator edits labels and descriptions (including by Excel export and re-import), and generates test cases. Beta runs 21 skills from requirements through code generation, hardening, live testing, schema, README and manifest generation, extended tests, code review, UX review, push and build, log analysis, a run gate, README enrichment, store metadata, and a final push.
Each skill is a row in the database: a versioned Jinja2 prompt template, a model assignment, and retry and timeout settings. Prompts and models are editable from the UI without a deploy, and every save creates a version that can be restored. Skills that need custom logic are Python plugins discovered by a registry; everything else runs through a generic executor.
The orchestrator loop checkpoints after every completed skill, compresses the context snapshot, and supports resume, restart from a step, and skill-requested rewinds (for example a UX review sending the pipeline back to the schema step) with a per-target cycle cap so loops always terminate. Deterministic checks run before any model review: AST parsing, async usage outside the SDK, hardcoded key patterns, banned provider strings, and a cross-check that schema keys match the keys the generated code reads.
Runs execute in Celery workers and stream progress to the browser over a WebSocket. Secrets are Fernet-encrypted in the database and injected into actors as environment variables, never written into generated code.
What it taught me
- Context from twenty-plus skills overflows any model window; old checkpoints are summarized once and the summary stored, so trimming is cheap and repeatable.
- Doing deterministic validation and auto-fixing before the model sees an error leaves the model only the repairs code cannot make, which cut retries and made failures legible.
- Review loops that can rewind the pipeline need a cycle counter per restart target, or a disagreement between two reviewers never terminates.
Architecture
A Next.js console talks to a FastAPI backend over REST and a WebSocket. Pipeline runs are queued through Redis to Celery workers, where the orchestrator loads skills from PostgreSQL, renders prompts, calls the assigned model or a tool, checkpoints the result, and emits live events. Tools run generated code against the upstream API locally and push finished actors to Apify, then poll builds and run logs.
Layer 1
Creator consoleClient
Next.js; runs, skills, settings
→ FastAPI backend (REST + WS)
Layer 2
FastAPI backendService
REST plus WebSocket stream
→ Redis (enqueue run)
Layer 3
RedisQueue
Celery broker
→ Celery worker (dispatch)
Layer 4
Celery workerWorker
runs the orchestrator loop
→ Skill orchestrator (execute skills)
Layer 5
Skill orchestratorService
checkpoints, retries, rewinds
→ Model providers (render, call) → PostgreSQL (checkpoint) → Upstream API (live test) → Apify platform (push and build) → FastAPI backend (live events)
Layer 6
Model providersModel
per-skill prompt and model
PostgreSQLStore
skills, runs, checkpoints, artifacts
Upstream APIExternal
live tests of generated code
Apify platformExternal
push, build, run, logs
Stack
Backend
- Python 3.13
- FastAPI
- Celery
- SQLAlchemy (async)
- Alembic
- Jinja2
Data
- PostgreSQL
- Redis
Frontend
- Next.js 14
- React 18
- Tailwind CSS
- SWR
- WebSocket run stream
AI
- OpenAI and Anthropic APIs
- per-skill model assignment
- versioned prompt templates
Infrastructure
- Apify API (push, build, run)
- local subprocess test runner