Endpoints
Scrape With Login
Scrapes web content from authenticated pages by handling login processes before accessing the target URL. Supports authentication form-based login
POST
/
scrape_with_login
curl --request POST \
--url https://api.datafuel.dev/scrape_with_login \
--header 'Authorization: Bearer <token>' \
--header 'Content-Type: application/json' \
--data '{
"url": "<string>",
"ai_prompt": "<string>",
"json_schema": {
"description": "Schema for capturing product information",
"name": "Product Schema",
"schema": {
"properties": {
"product_url": {
"description": "The URL of the specific product",
"type": "string"
},
"product_name": {
"description": "The name of the specific product",
"type": "string"
},
"price": {
"description": "The price of the product",
"type": "number"
},
"product_images": {
"description": "List of product image URLs",
"items": {
"properties": {
"url": {
"description": "URL of the product image",
"type": "string"
}
},
"required": [
"url"
],
"type": "object"
},
"type": "array"
}
},
"required": [
"product_url",
"product_name",
"price",
"product_images"
],
"type": "object"
}
},
"javascript_scenario": [
{}
],
"username": "<string>",
"password": "<string>",
"login_url": "<string>"
}'
{
"job_id": "f47ac10b-58cc-4372-a567-0e02b2c3d479"
}
Authorizations
Bearer authentication header of the form Bearer <token>
, where <token>
is your auth token.
Body
application/json
Response
200
application/json
Successful Response
The response is of type object
.
curl --request POST \
--url https://api.datafuel.dev/scrape_with_login \
--header 'Authorization: Bearer <token>' \
--header 'Content-Type: application/json' \
--data '{
"url": "<string>",
"ai_prompt": "<string>",
"json_schema": {
"description": "Schema for capturing product information",
"name": "Product Schema",
"schema": {
"properties": {
"product_url": {
"description": "The URL of the specific product",
"type": "string"
},
"product_name": {
"description": "The name of the specific product",
"type": "string"
},
"price": {
"description": "The price of the product",
"type": "number"
},
"product_images": {
"description": "List of product image URLs",
"items": {
"properties": {
"url": {
"description": "URL of the product image",
"type": "string"
}
},
"required": [
"url"
],
"type": "object"
},
"type": "array"
}
},
"required": [
"product_url",
"product_name",
"price",
"product_images"
],
"type": "object"
}
},
"javascript_scenario": [
{}
],
"username": "<string>",
"password": "<string>",
"login_url": "<string>"
}'
{
"job_id": "f47ac10b-58cc-4372-a567-0e02b2c3d479"
}
Assistant
Responses are generated using AI and may contain mistakes.