POST
/
scrape

The scrape endpoint allows you to extract data from a single URL at a time. You can choose between two scraping modes:

  1. Basic Scraping: Extracts data from the provided URL without AI assistance.
  2. AI-Enhanced Scraping: Uses AI to process the scraped content with either:
    • A custom prompt to guide the extraction
    • A prompt combined with a JSON schema for structured and consistent output

​
Use Cases

  • Extract specific content from web pages
  • Transform unstructured web content into structured data
  • Ensure consistent data format using JSON schema validation

Authorizations

​
Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json
​
url
string
required
​
ai_prompt
string | null
​
javascript_scenario
object[] | null
​
json_schema
object | null

Optional schema definition for structured data extraction. Format should follow OpenAI's function calling schema format (https://platform.openai.com/docs/guides/structured-outputs).

Example types:

  • string: "type": "string"
  • integer: "type": "integer"
  • number: "type": "number"
  • boolean: "type": "boolean"
  • array: "type": "array", "items": {"type": "string"}
  • object: "type": "object", "properties": {...}

Response

200 - application/json
​
job_id
string
required

The identifier for the scraping job

IntroductionScrape With Login