Service node settings

A service node in a workflow calls one provider with one set of options. The properties panel surfaces 13 settings grouped into four disclosure sections: Basic, Output formats, Identity, and Advanced. Vendor-specific fields render below the shared sections.

This page is the reference for every option. If a setting isn’t supported by your selected provider, the workflow editor hides it. The capability matrix in Providers shows which providers expose which options.

Field shape

All settings flow through node.config.scrapeOptions and are spread into the provider adapter’s ScrapeRequest. The TypeScript shape is:

interface ScrapeRequest {
  url: string;
  // --- Basic --------------------------------------------------------------
  country?: string;                 // ISO-2 lowercase: 'us', 'gb', 'de'
  renderJs?: boolean;
  waitForSelector?: string;         // CSS selector
  waitForSelectorTimeout?: number;  // milliseconds
  // --- Output formats -----------------------------------------------------
  parse?: boolean;
  markdown?: boolean;
  screenshot?: boolean;
  // --- Identity -----------------------------------------------------------
  customHeaders?: Record<string, string>;
  customCookies?: Record<string, string>;
  sessionId?: string;
  device?: 'desktop' | 'mobile' | 'tablet';
  // --- HTTP control -------------------------------------------------------
  httpMethod?: 'GET' | 'POST';
  httpBody?: string;
  // --- Vendor-specific (passthrough) --------------------------------------
  [key: string]: unknown;
}

The adapter is responsible for converting these typed fields to vendor-specific request bodies. Every adapter applies the same field semantics; only the wire format differs.

---

Basic (always visible)

These four settings are visible by default in the Scrape Options panel for every supported provider.

Country

Geo-target the proxy. Lowercase ISO-2 country code. The workflow editor surfaces the most common 14 (US, UK, DE, FR, JP, IN, BR, AU, CA, ES, IT, MX, KR, NL); for any other code, use Advanced (JSON): {"country": "se"}.

{ "country": "us" }

Adapters convert to vendor-native format:

• Bright Data: passed as-is (lowercase).
• Oxylabs / Decodo: converted to full country name ("us" → "United States").
• Nimble Way / Zyte: uppercased ("us" → "US").

JS Rendering

Toggle. Enable headless browser rendering. Required for waitForSelector, screenshots, and most modern dynamic sites.

{ "renderJs": true }

Bright Data: always-on. Web Unlocker can’t disable JS — the toggle is shown disabled-checked.

Wait for Selector

CSS selector to wait for in the rendered DOM before returning the body. Requires renderJs: true (except on Bright Data, where rendering is implicit).

{ "waitForSelector": ".product-title", "waitForSelectorTimeout": 15000 }

Selector Timeout (ms)

Timeout for waitForSelector in milliseconds. Default 10000 (10s). Adapters convert to vendor-native units (Zyte uses seconds; Oxylabs/Decodo/Nimble use milliseconds).

---

Output formats (collapsed disclosure)

Three toggles that change the response body type. Only one effective format applies per request — if multiple are set, the adapter picks per its own precedence rules. The response includes contentType so downstream validators know what to expect.

Parse (vendor-side structured)

Toggle. Request the vendor’s parsed JSON output instead of raw HTML.

{ "parse": true }

• Oxylabs: parse: true returns Oxylabs’s structured JSON (e.g., Amazon product fields). Body is JSON.stringify(parsed_data), contentType: application/json.
• Decodo: same shape — parse: true → parsed_data with contentType: application/json.
• Zyte: auto-extraction (product / article / jobPosting) is TODO 3; the flags work via Advanced (JSON) but the contentType pipeline isn’t wired yet.
• Bright Data / Nimble Way: not supported.

Screenshot (PNG)

Toggle. Returns a base64-encoded PNG of the rendered page.

{ "screenshot": true }

• Body is a base64 PNG string. contentType: image/png.
• Implies rendering on every supported provider.
• Zyte: screenshot is its own browser-mode trigger (doesn’t require renderJs). Adds a per-request cost.
• Oxylabs / Decodo: screenshot wins over markdown and renderJs if multiple are set.
• Bright Data: screenshot wins over markdown if both are set.

Markdown output

Toggle. Returns markdown-converted body text instead of HTML.

{ "markdown": true }

• Body is plain markdown. contentType: text/markdown.
• Decodo: auto-enables headless rendering when set.
• Oxylabs / Zyte: not supported.

---

Identity (collapsed disclosure)

Headers, cookies, sessions, and device emulation — anything that affects how the proxy request looks to the target site.

Custom Headers

Key/value editor in the UI. Stored as a flat Record<string, string>.

{
  "customHeaders": {
    "User-Agent": "MyScraper/1.0",
    "Accept-Language": "en-US,en;q=0.9"
  }
}

• Bright Data: merged into request body headers. Requires “Custom Headers & Cookies” enabled on the zone.
• Oxylabs: passed via context: [{key: "headers", value: {...}}].
• Nimble Way: body.headers (object).
• Zyte: branched by mode — HTTP mode uses customHttpRequestHeaders (array, full passthrough); browser mode uses requestHeaders (object, only Referer honored).
• Decodo: body.headers object.

Custom Cookies

Key/value editor in the UI. Stored as a flat Record<string, string> of cookie name → value.

{
  "customCookies": {
    "session_token": "abc123",
    "preferred_lang": "en"
  }
}

Each cookie’s domain defaults to the request URL’s hostname when the adapter assembles the wire payload (eng-review A2). Override per-cookie via Advanced (JSON) if you need a different domain (e.g., .example.com for subdomain coverage).

• Bright Data: not supported (Web Unlocker has no first-class cookie passthrough).
• Oxylabs / Decodo: flat record passed via vendor-native shape.
• Nimble Way: array of {key, value, domain} with auto-default domain.
• Zyte: array of {name, value, domain} with auto-default domain.

Session ID

Free-form text. Sticky-session ID for providers that support session continuity (same IP across multiple requests).

{ "sessionId": "user-checkout-flow-1" }

• Oxylabs: passed via context: [{key: "session_id", value: ...}].
• Zyte: passed as session.id. Zyte requires v4 UUIDs — the adapter SHA-1-hashes any free-form string to a deterministic v4 UUID, so the same sessionId always produces the same Zyte session.
• Decodo: session_id.
• Bright Data / Nimble Way: not supported.

Device

Select. desktop / mobile / tablet. Adapters map to vendor-specific user-agent / driver / device-type fields.

{ "device": "mobile" }

• Oxylabs: user_agent_type (tablet → mobile).
• Nimble Way: drives driver selection — mobile / tablet → vx10-pro with rendering enabled. desktop alone is a no-op (combine with renderJs or waitForSelector).
• Zyte: HTTP mode only (browser mode silently drops). Tablet → mobile.
• Decodo: device_type passed as-is.
• Bright Data: not supported.

---

Advanced (collapsed disclosure)

A free-form JSON textarea per provider. Power-user passthrough — anything you put in is shallow-merged into the final wire request last, so it can override anything the typed surface set.

{
  "advancedOptions": "{\"viewport\": {\"width\": 1280}, \"sessionContext\": [{\"name\": \"loggedIn\"}]}"
}

The textarea accepts a JSON object. Validation happens on blur and on save:

• Invalid JSON → inline error, save blocked.
• Non-object root (array, string, number) → rejected.
• Keys __proto__, constructor, prototype at the root → rejected (prototype-pollution defense).

Use cases by provider:

• Bright Data: any field the typed surface doesn’t expose (data_format overrides, custom dataset filters).
• Oxylabs: parsing_instructions (custom parser), geo_location overrides, extra context entries.
• Nimble Way: parsing (CSS-selector schema), network_capture, locale, sub-country state/city, custom viewport.
• Zyte: viewport, actions (full sequences beyond waitForSelector), sessionContext, auto-extract flags (product: true, article: true, jobPosting: true — TODO 3).
• Decodo: pages, start_page, vendor flags not exposed in typed UI.

Override semantics. If you set country: "us" in the typed UI and {"country": "uk"} in Advanced (JSON), the JSON wins. Useful for testing; risky for production. Workflow validation does not flag conflicts.

---

Vendor-specific fields

Below the shared groups, each provider renders its own fields. These don’t have shared capability flags because they don’t generalize across vendors:

Provider	Field	Notes
Bright Data	Zone (select)	Required. Web Unlocker zone from your account. See Bright Data > Required configuration.
Bright Data	Response Format (json / raw)	Optional. Default json.
Oxylabs	Source (select, 80+)	Optional override of auto-detection. Empty = auto.
Nimble Way	Driver (select)	Optional override of inferred stealth profile.
Zyte	IP Type (datacenter / residential)	Optional. Residential requires KYC + per-request surcharge.
Decodo	Target (select, 40+)	Optional override of auto-detection.
Decodo	Proxy Pool (standard / premium)	Optional.
Decodo	Page From / Page Count	Only meaningful for paginated targets (google_search, amazon_search).

---

API submission

When submitting a job via API, pass scrapeOptions in the request body. The fields match the ScrapeRequest shape above.

curl -X POST https://api.justcrawl.io/api/v1/jobs \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://www.amazon.com/dp/B08Y72CH1F",
    "scrapeOptions": {
      "country": "us",
      "parse": true,
      "device": "desktop",
      "customHeaders": {
        "Accept-Language": "en-US"
      }
    }
  }'

The workflow router resolves which provider to use (based on domain routing + smart strategy), and the chosen provider’s adapter consumes the shared fields above plus any vendor-specific extras you pass through.

For the full API request/response schemas, see the API Reference — ScrapeOptions is documented under the POST /api/v1/jobs endpoint.

---

Response content type

Service-node responses include a contentType discriminator that drives downstream validation:

contentType	Meaning	Validation behavior
text/html (default)	Rendered HTML body	validateHtml runs (size, error patterns, selectors).
application/json	Parsed structured data (parse mode)	HTML validator skipped.
image/png	Base64-encoded screenshot	HTML validator skipped.
text/markdown	Markdown-converted text	HTML validator skipped.

This means turning on Parse, Screenshot, or Markdown automatically bypasses any HTML output-validation rules you’ve configured on the node — the validator knows the body isn’t HTML.

API scrapers: set an output format or switch validation off

Output validation runs on every service node by default, but it does not force a rung to return HTML unless you ask it to. A body that is not markup — JSON, plain text — passes the HTML checks rather than failing them, so an API scraper works without special configuration. Set validateOutput: true on the node to make HTML mandatory, in which case a non-markup body is rejected as not HTML and the job is refunded rather than delivered.

Two things are always rejected, whatever the node asks for: an empty body, and a non-markup body too short to be a payload — a bare Internal Server Error arriving with a 200 is a failed delivery, not a delivered page. Structured JSON of any size is fine. Setting one of the output formats above is still the clearest way to declare intent, and it skips the validator entirely. See Billing & Plans for what a refunded crawl means on your balance.