Agents & MCP

The justcrawl MCP server gives any LLM-powered agent (Claude Desktop, Cursor, Codex CLI) the ability to submit scrape jobs, list workflows, run saved BI queries, and search docs — all from inside the chat.

Coming soon

The @justcrawl/mcp-server npm package ships with Phase 3 of the agent rollout. The config snippets below are the final published shape — npx -y @justcrawl/mcp-server returns an npm 404 until then. Paste them now if you want your host config ready for the publish; restart the host after the package goes live.

Which host are you on?

Each MCP host has a different config file and process model. Pick yours:

Claude Desktop Edit claude_desktop_config.json, restart, done. Most popular path.

Cursor Drop into ~/.cursor/mcp.json (global) or .cursor/mcp.json (per project).

Codex CLI Add an [mcp_servers.justcrawl] block to ~/.codex/config.toml.

What you can do once it's wired

The server exposes these tools to the agent:

| Tool | What it does | |------|-------------| | jc_jobs_submit | Submit a single scrape job for a URL | | jc_jobs_list | Paginated job list, filter by status or tag | | jc_jobs_get | Fetch a job + its result (auto-resolves the presigned URL when done) | | jc_jobs_cancel | Cancel a queued or running job | | jc_workflows_list / jc_workflows_get | Browse the workflows in your org | | jc_providers_list | List providers and per-account connection status | | jc_urls_list | List URLs in your library, filter by tag or search | | jc_bi_list_saved_queries | List BI saved queries you've created in the dashboard | | jc_bi_run_saved | Run a saved query with parameters (raw SQL is NOT exposed by design) | | jc_schedules_list / jc_schedules_trigger | List schedules and trigger a one-off run | | jc_docs_search / jc_docs_get | Search and read justcrawl.io docs |

No raw SQL

The MCP server intentionally rejects any sql field in tool arguments. Saved queries and parameter binding only. Raw SQL ad-hoc execution lives on the dashboard. This is a deliberate defense against prompt-injection-driven data exfiltration.

What you'll need before you start

1. A justcrawl.io API key. Settings → API Keys in the dashboard. Copy the token (it starts with sr_live_); it's only shown once.
2. Node.js 20+ on the machine running the agent — the MCP server runs as a local stdio process via npx.
3. The host you're configuring (Claude Desktop / Cursor / Codex CLI) installed and running.

Next

Pick a host card above. Every page is one config snippet, one restart, and a quick sanity check.