Skip to content
justcrawl.io
Search
Ctrl
K
Cancel
Select theme
Dark
Light
Auto
Menu
Getting Started
What is justcrawl.io?
Quickstart
API Quickstart
SDKs
Core Concepts
Workflows
Providers
Service node settings
URLs
Schedules
Jobs
Job lifecycle
Roles, permissions & API keys
Extraction & schemas
BI / SQL console
Smart Workflows
Smart Workflows
Strategies
Optimization Suggestions
Integrations & More
Integrations
Webhook delivery
Idempotency
Rate limits & quotas
Analytics
Billing & Plans
Agents
Overview
Claude Desktop
Cursor
Codex CLI
Errors
Errors
already_exists
auth_code_consumed
auth_code_expired
auth_forbidden
auth_invalid
auth_missing
auth_org_mismatch
BI error codes
conflict
internal_error
invalid_input
invalid_url
invalid_workflow
job_terminal
node_type_unknown
not_found
org_not_found
payment_required
plan_required
provider_unavailable
quota_exceeded
service_unavailable
workflow_disabled
API Playground
Try It Now
API Reference
Overview
Authentication
Overview
Register a new account
Log in with email and password
Refresh access token using a refresh token
Invalidate a refresh token
Request a password reset link
Reset password using a reset token
Verify email address using a verification token
Resend email verification token (requires authentication)
Get the authenticated user
Begin a CLI device-authorization flow
Look up a user_code's metadata for the authorize page
Approve a pending CLI device code from the browser
Poll for the result of a CLI device-authorization flow
Jobs
Overview
List jobs
Submit a scrape job
Batch poll job statuses
Dashboard-widget aggregated counters with prior-window deltas
Get job detail
Get scrape result metadata + presigned blob URL
Workflows
Overview
List workflows
Create workflow
Create default smart workflow (no benchmark)
Get workflow
Update workflow
Delete workflow
Clone as custom
Validate DAG without saving
Publish workflow
Unpublish workflow
Set workflow route
Get a specific workflow version
Smart Workflows
Overview
Get smart workflow config
Update auto/suggest mode
List suggestions
Apply suggestion
Dismiss suggestion
List all pending suggestions for org
URLs
Overview
List URLs
Create single URL
Batch create URLs
Delete URLs in bulk
Get URL detail
Delete URL
Update a URL
Enable or disable a URL
List scrape jobs for a URL
List extraction history for a URL
Schedules
Overview
List schedules
Create schedule
List schedule runs
Get schedule
Update schedule
Delete schedule
Enable/disable schedule
Trigger schedule now
Providers
Overview
List providers and their capabilities
Integrations
Overview
List input queue configs
Create input queue config
Update input config
Delete input config
Toggle input config
Test input connection (coming soon)
List output configs
Create output config
Get the system-managed internal output config
Update output config
Delete output config
Enable or disable an output config
Test output connection (coming soon)
Get the org's blob-storage config
Switch storage between platform and customer S3 bucket
Webhooks
Overview
Ingest URLs via webhook
Analytics
Overview
Get overview metrics
Get timeseries data
Provider comparison
Domain breakdown
Domain x provider matrix
Plans
Overview
Get plan status
Credit transaction history
Check for a pending recharge request
Submit recharge request
Benchmarks
Overview
Start benchmark
Get latest benchmark
Get benchmark status
Cancel benchmark
Get benchmark results
Agents
Overview
Read an agent conversation
Send a message or click an agent card action
Read an agent conversation's durable cards
Extraction
Overview
List extraction results
Get extraction result for a job
List extraction schemas
Get latest schema for a domain + page type
Trigger extraction schema discovery for an already-scraped job
List org custom attributes for a domain
Create or update org custom attributes
Remove org custom attributes
Test XPath attributes against stored HTML
Re-extract historical results with the current schema
List extraction history for one URL item
Get one extraction result by its own ID
Get access to an extraction's raw HTML
BI
Overview
List queryable tables
Describe one table
List submitted queries
Submit a query
Get query status
Get result manifest
Fetch a page of results
Cancel a running query
List saved queries
Create a saved query
Get a saved query
Delete a saved query
Update a saved query
Run a saved query
Convert a structured saved query to editable PostgreSQL
Request a result export
Get export status and download link
Account
Delete your account (GDPR right to erasure)
Export all personal data (GDPR right to data portability)
Audit
List audit logs for the organization
Select theme
Dark
Light
Auto
Overview
Extraction
Section titled “Extraction”
Structured data extraction results, schemas, and custom attributes
Operations
Section titled “Operations”
GET
/api/v1/extraction/results
GET
/api/v1/extraction/results/{jobId}
GET
/api/v1/extraction/schemas
GET
/api/v1/extraction/schemas/{domain}/{pageType}
POST
/api/v1/extraction/discover
GET
/api/v1/extraction/attributes/{domain}
PUT
/api/v1/extraction/attributes/{domain}
DELETE
/api/v1/extraction/attributes/{domain}
POST
/api/v1/extraction/test-xpath
POST
/api/v1/extraction/backfill
GET
/api/v1/extraction/urls/{urlItemId}/extractions
GET
/api/v1/extraction/results/by-id/{id}
GET
/api/v1/extraction/results/{id}/raw