Trigger extraction schema discovery for an already-scraped job

POST

/api/v1/extraction/discover

curl --request POST \
  --url https://api.justcrawl.io/api/v1/extraction/discover \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{ "jobId": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0", "pageType": "product", "attributes": [ "example" ] }'

• Production

Re-drives extraction against a job’s cached HTML — the provider is never re-hit. When no extraction schema covers the requested attributes, this also triggers LLM XPath discovery for the job’s domain and page type.

Discovery is shared across orgs and costs LLM budget, so it runs at most once per domain/page-type: a schema that already covers every requested attribute is reused as-is, and a schema missing some of them is topped up rather than rediscovered from scratch.

Returns 202 — the work happens asynchronously. Poll GET /api/v1/extraction/schemas/{domain}/{pageType} for the schema and GET /api/v1/extraction/results/{jobId} for this job’s extracted values.

Authorizations

• bearerAuth

Request Body^required

Media typeapplication/json

object

jobId

required

A completed job owned by the caller’s org, whose body blob is still retained.

string format: uuid

pageType

string

default: product

Allowed values: product product_list serp article job_posting

attributes

Attribute names to ensure the schema covers. Omit for the page type’s full default attribute set. Names must match ^[a-z][a-z0-9_]*$.

Array<string>

<= 50 items

Responses

202

Discovery request accepted and queued.

Media typeapplication/json

object

domain

The canonical (www-stripped) domain the schema is keyed on.

string

pageType

string

Example^generated

{
  "domain": "example",
  "pageType": "example"
}

400

Missing jobId, a pageType outside the enum, or an attributes list that is too long or contains a malformed name.

401

Missing or invalid authentication token

Media typeapplication/json

object

error

string

Example

{
  "error": "Missing or invalid authentication token"
}

403

Insufficient permissions for this operation

Media typeapplication/json

object

error

string

Example

{
  "error": "No organization. Complete onboarding first."
}

404

No such job in the caller’s org.

409

The job is still pending, running or waiting_retry, so it has not produced a body yet. Retryable — wait for the job to finish and call again.

410

The job’s cached body is gone (never stored, or past its retention window), so there is nothing to re-extract.

500

Unexpected server error. Logs and PostHog $exception capture

Media typeapplication/json

object

error

string

Example

{
  "error": "Something went wrong"
}