Skip to content
justcrawl.io

Test XPath attributes against stored HTML

POST
/api/v1/extraction/test-xpath
 
curl --request POST \
  --url https://dashboard.justcrawl.io/api/v1/extraction/test-xpath \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{ "domain": "example", "pageType": "example", "attributes": [ { "name": "example", "xpaths": [ "example" ], "type": "text", "description": "example", "antiPatterns": [ "example" ], "semanticType": "brand_name", "validation": { "minLength": 1, "maxLength": 1, "min": 1, "max": 1, "pattern": "example" } } ], "jobId": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0" }'

Synchronously re-runs extraction with the supplied attribute set against an existing cached HTML blob — the dashboard’s “Try this XPath before saving” surface. The blob is sourced from either the explicit jobId (when supplied) or the most recent job with a stored body for the domain. No new scrape is dispatched.

Capped at 5MB of HTML per call; larger blobs return 413 rather than time out. The extractor is run with a one-shot schema (id: 'test', version: 1) so this never mutates the org’s saved attributes — use PUT /attributes/{domain} for that. Requires workflows:write.

Authorizations

Section titled “Authorizations ”

Request Body required

Section titled “Request Body required ”
Media type application/json
object
domain
required
string
pageType
required
string
attributes
required
Array<object>
>= 1 items
object
name
required
string
xpaths
required
Array<string>
type
required
string
Allowed values: text number url boolean text[] url[]
description
string
nullable
antiPatterns
Array<string>
nullable
semanticType
string
nullable
Allowed values: brand_name review_count product_id product_image star_rating
validation
object
minLength
integer
maxLength
integer
min
number
max
number
pattern
string
jobId

Optional. If omitted, uses the most recent job with stored HTML for the domain.

string format: uuid

Responses

Section titled “ Responses ”

200

Section titled “200 ”

Extracted values, quality scores, and the source job’s URL.

Media type application/json
object
values
object
key
additional properties
any
qualityScore
object
completeness
number
validation
number
composite
number
jobId
string
url
string
Example generated
{
  "values": {},
  "qualityScore": {
    "completeness": 1,
    "validation": 1,
    "composite": 1
  },
  "jobId": "example",
  "url": "example"
}

400

Section titled “400 ”

Validation failed — request body or query is missing/invalid

Media type application/json
object
error
string
Example 
{
  "error": "Missing or invalid \"url\" field"
}

401

Section titled “401 ”

Missing or invalid authentication token

Media type application/json
object
error
string
Example 
{
  "error": "Missing or invalid authentication token"
}

403

Section titled “403 ”

Insufficient permissions for this operation

Media type application/json
object
error
string
Example 
{
  "error": "No organization. Complete onboarding first."
}

404

Section titled “404 ”

Resource not found

Media type application/json
object
error
string
Example 
{
  "error": "Workflow not found"
}

413

Section titled “413 ”

HTML blob exceeds the 5MB sync limit. Narrow the test set or split the page.

500

Section titled “500 ”

Unexpected server error. Logs and PostHog $exception capture

Media type application/json
object
error
string
Example 
{
  "error": "Something went wrong"
}