GET
/
list_scrapes
Get Scrapes Metadata
curl --request GET \
  --url https://api.datafuel.dev/list_scrapes \
  --header 'Authorization: Bearer <token>'
[
  {
    "job_id": "f47ac10b-58cc-4372-a567-0e02b2c3d479",
    "job_status": "finished",
    "scrape_id": "550e8400-e29b-41d4-a716-446655440000",
    "scrape_status": "success",
    "scrape_timestamp": "2024-03-15T10:30:00Z",
    "scrape_url": "https://example.com",
    "signed_url": "https://docyjgyvimrauivbukcp.supabase.co/storage/v1/object/sign/scrapes-data/..."
  }
]

Authorizations

​
Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Query Parameters

​
job_id
string | null
​
markdown
boolean
default:false
​
ai_response
boolean
default:false
​
url
string | null

Filter by URL

Response

Successful Response

​
job_id
string
required

The unique identifier for the scraping job that can contain multiple scrapes

Example:

"f47ac10b-58cc-4372-a567-0e02b2c3d479"

​
scrape_status
enum<string>
required
Available options:
pending,
failed,
success
​
scrape_url
string
required

The URL that was scraped

Example:

"https://example.com"

​
scrape_timestamp
string
required

The timestamp of when the scrape was created

Example:

"2024-03-15T10:30:00Z"

​
job_status
enum<string>
required
Available options:
pending,
finished
​
scrape_id
string | null

The unique identifier for the each URL scrape

Example:

"550e8400-e29b-41d4-a716-446655440000"

​
signed_url
string | null

Pre-signed URL containing the scraping results in JSON format with structure: { "ai_response": "AI-generated analysis or structured data", "markdown": "Content formatted in markdown", "html": "Raw HTML content from the scraped page" }

Example:

"https://docyjgyvimrauivbukcp.supabase.co/storage/v1/object/sign/scrapes-data/..."

​
ai_response
string | null

AI-generated analysis or structured data

​
markdown
string | null

Content formatted in markdown