# Scrappey Documentation
> Full documentation for the Scrappey web data API. Source: https://docs.scrappey.com/docs
---
# Welcome to Scrappey
Source: https://docs.scrappey.com/docs/getting-started/welcome
> Scrappey.com is a web data and browser-automation API. It manages browser sessions, proxy configuration, and headless browsing, returning HTML, headers, cookies, screenshots, and structured data so you can focus on building your application.
**Disclaimer:** Use Scrappey responsibly. Automate only sites and accounts you own or are authorized to access, and respect each website's terms of service and applicable laws. Scrappey.com is not responsible for misuse.
**Scrappey.com** runs your requests through managed, headless browser sessions and returns the rendered result over a simple HTTP API. It handles the operational work of launching browsers, managing cookies and sessions, and routing through configurable proxies, so you can build reliable data pipelines without maintaining browser infrastructure yourself.
## Your First Request
Every request is a `POST` to the API endpoint with your API key in the query string. The simplest call fetches a page through a managed browser session:
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com"
}'
```
The rendered HTML, cookies, headers, and status code are returned in the `solution` object. See [request.get](/docs/request-get) for the full list of options.
## Why Choose Scrappey.com?
**Managed Browser API:** Scrappey.com runs each request through a managed, headless browser session with configurable proxies, returning fully rendered HTML and structured data. It supports sites with modern browser requirements — JavaScript rendering, dynamic content, and interactive page flows — so your pipelines stay reliable.
**Headless Browsing:** Access pages that require authentication or element interaction. Scrappey.com lets you script interactive browser workflows — navigation, clicks, typing, waits, and form submission — through the `browserActions` API, so multi-step flows such as login or paginated content are straightforward to automate.
**Flexible Pricing:** Transparent, flexible plans for individuals, startups, and enterprises. Concurrent requests, premium proxies, JavaScript rendering, and more — billed by usage, cancel anytime.
**Stay Updated:** Check out our [Scrappey Blog](https://wiki.scrappey.com/blog) for the latest articles about web scraping, guides, tutorials, and industry insights.
## Report Abuse
### Our Position and Report Abuse
At Scrappey.com, we are committed to providing services that prioritize the well-being and interests of our users. We strictly prohibit any activities that infringe upon the rights of others, such as cybercrime, cracking, and fraudulent actions. Our primary goal is to assist start-up companies by offering powerful AI algorithms for SEO, AD verification, benign crawlers, and other business growth scenarios. By leveraging our services, we aim to help these companies save their budget and achieve their objectives.
If you believe that someone has unlawfully used our services to infringe upon your interests, we urge you to reach out to us immediately. Once we confirm the validity of the case, we will promptly terminate the services provided to the offender and provide you with any available information to aid in recovering your losses. Our team will do everything possible to assist you throughout this process.
We have a strong aversion to cybercrime, and we stand firmly against it. If you represent an anti-bot company and have noticed fraudulent activities associated with one of our users, we take this matter seriously. We encourage you to contact us promptly with your concerns. Should we verify the legitimacy of your claim, the offending business will be swiftly removed from our system and barred from any further interactions.
### Contact Us
If you need to report abuse or have any inquiries, please don't hesitate to reach out to us through the following channels:
**Email:** [scrappeycom@gmail.com](mailto:scrappeycom@gmail.com)
We appreciate your cooperation in maintaining a safe and ethical environment for all users of our services.
---
# Getting Started
Source: https://docs.scrappey.com/docs/getting-started/introduction
> Scrappey.com is a browser-based access proxy. It runs requests through managed browser sessions and returns rendered HTML and cookies, so you can work with the pages your application needs — including sites that require a full browser environment.
## What is Scrappey
Scrappey.com makes it straightforward to collect web data through managed browser sessions. It acts as a proxy server: you send a request, and it returns the rendered HTML and the cookies set during the session. Those cookies can be reused with other HTTP clients to maintain session continuity with the same site. You can also script multi-step browser interactions through the `browserActions` API. It's well suited to workflows that demand high reliability and repeatability — QA, uptime and content monitoring, indexing your own content, ad verification, SEO analytics, and internal automation. Use it only where permitted and in compliance with each site's terms and applicable laws.
## Features
Scrappey is a browser-automation engine that provides complete, consistent browser environments, helping you reliably access sites that require a real browser to render. Here's how it works:
Scrappey runs a full browser environment with realistic, configurable settings:
1. It manages browser configuration — device type, browser, language, time zone, screen size, and related properties — so pages render as they would in a standard desktop or mobile browser.
2. It supports realistic interaction primitives such as mouse movement, scrolling, and typing, so interactive pages and dynamically loaded content render and behave correctly.
Scrappey provides a standards-compliant browser environment:
1. It runs your automation as a standard browser session, executing JavaScript and loading assets like a normal browser.
2. It gives you control over browser configuration and privacy-related settings (WebGL, WebRTC, canvas, fonts, and more).
3. It normalizes technical signals so requests behave like an ordinary browser, which improves rendering consistency across sites.
Scrappey aligns the browser environment with your selected proxy region:
1. It sets a consistent language, locale, and time zone to match the proxy.
2. It matches browser settings to your configured proxy region, so regional testing and localized content render correctly.
Additional capabilities:
1. Generate consistent, unique device profiles that persist across requests (see Profiles)
2. Run full browser environments on headless servers
3. Control WebGL, image loading, WebRTC, and other browser-level settings
In short, Scrappey helps you collect data, run QA and monitoring against your own or authorized sites, and automate browser tasks — with sessions that behave like an ordinary browser environment. Use it only where permitted and in line with applicable terms and laws.
## Setup and Installation
1. Sign up at [Scrappey](https://www.scrappey.com).
2. Obtain your unique API key from your Scrappey dashboard.
3. Integrate Scrappey into your projects via simple HTTP requests or SDKs provided.
## Sending Requests
Sending requests through Scrappey.com is a straightforward process. You simply need to send a request to an endpoint, providing the URL, headers (optional), cookies (optional), and proxy (optional). Once you initiate the request, Scrappey handles browser execution, session state, proxy routing, and response collection.
To authenticate your requests, you'll need an API key, which you can obtain by visiting the [Dashboard](https://app.scrappey.com/#/). This API key ensures that your requests are properly authorized and processed.
Not sure what to do? Check out our [Request Builder](https://app.scrappey.com/#/builder).
## Example Request
```javascript
// Example endpoint
const endpoint = "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY";
// Available request types: "request.get", "request.post", "request.put", "request.delete", "request.patch", "request.publish"
const requestData = {
cmd: "request.get",
url: "https://example.com"
};
```
## Example CURL
```bash
# Or CURL example
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"cmd":"request.get","url":"https://example.com"}'
```
---
# Profiles
Source: https://docs.scrappey.com/docs/api-reference/profiles
> Attach a `profileId` to any request to maintain a consistent browser configuration, cookies, proxy, and session data across all future requests using that profile.
## Overview
Persistent profiles solve a common challenge: maintaining a stable, reproducible browser environment across requests. Each profile stores:
- **Browser configuration** - Canvas, WebGL, fonts, screen size, user agent, and 50+ other browser properties
- **Cookies** - All cookies are automatically saved and restored
- **Session data** - localStorage and other session information
- **Proxy & IP** - The proxy and IP address used with this profile (automatically remembered)
Once created, a profile's configuration never changes. This gives you a stable, reproducible browser environment every time you use that profile — useful for consistent QA runs, monitoring, and session continuity.
## Parameters
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `profileId` | string | Yes | Unique identifier for the profile. Use any string (alphanumeric, hyphens, underscores). |
| `proxy` | string | No | Your proxy URL. If provided, it is saved to the profile and used for all future requests. |
| `forceNewProxy` | boolean | No | When `true`, ignores the stored proxy and assigns a fresh one. The new proxy replaces the old one. |
## Request
Add `profileId` to your request to use a persistent profile:
```json
{
"cmd": "request.get",
"url": "https://example.com",
"profileId": "my-profile-123"
}
```
If the profile does not exist it is automatically created with a unique browser configuration. All subsequent requests with the same `profileId` will reuse that exact configuration.
### First Request (Profile Creation)
```json
{
"cmd": "request.get",
"url": "https://browserleaks.com/canvas",
"profileId": "account-store-001"
}
```
- A new browser configuration is generated and permanently saved
- A proxy is automatically assigned and saved to the profile
- The browser launches with this configuration and proxy
- Any cookies set by the website are saved when the session ends
### Subsequent Requests (Profile Reuse)
```json
{
"cmd": "request.get",
"url": "https://example.com/account",
"profileId": "account-store-001"
}
```
- The saved configuration is loaded (100% identical to first request)
- The saved proxy is automatically used (same IP address)
- Saved cookies are injected into the browser
- The site sees the same consistent browser environment as before
## Use Cases
### Account Management
Maintain separate, isolated browser environments for different accounts:
```json
{"profileId": "email-account-1", "url": "https://example.com"}
{"profileId": "email-account-2", "url": "https://example.com"}
```
### Session Persistence
Stay logged in across requests without re-authenticating:
```json
{"profileId": "my-store", "url": "https://mystore.example.com/admin"}
{"profileId": "my-store", "url": "https://mystore.example.com/admin/orders"}
```
### Consistent Environment
Present a stable, returning-visitor environment for sites that expect session continuity:
```json
{"profileId": "research-browser", "url": "https://site-with-fingerprinting.com"}
```
## Best Practices
### Profile Naming
Use descriptive, consistent naming:
```
account-{platform}-{identifier} → account-store-john
session-{purpose}-{id} → session-scraping-batch42
```
### One Profile Per Identity
Each unique identity should have its own profile.
### Profile Isolation
Two users with the same `profileId` get completely separate profiles and data.
## Configuration Consistency
Every property of the browser environment remains identical across sessions, which makes runs reproducible:
| Component | Consistency |
|-----------|-------------|
| Canvas fingerprint | 100% identical |
| WebGL renderer/vendor | 100% identical |
| Audio fingerprint | 100% identical |
| Font list | 100% identical |
| Screen dimensions | 100% identical |
| User agent | 100% identical |
| Platform/OS | 100% identical |
| Timezone | 100% identical |
| Language | 100% identical |
## Proxy Persistence
Profiles automatically remember the proxy used during their session, so a profile maintains a consistent IP location across requests. This provides session continuity — the same browser environment and the same regional proxy every time.
### With Your Own Proxy
```json
{
"cmd": "request.get",
"url": "https://example.com",
"profileId": "my-profile",
"proxy": "http://user:pass@proxy.example.com:8000"
}
```
- Your proxy is saved to the profile
- All future requests automatically use this proxy
- If the proxy stops working, you'll get an error asking you to provide a new one
### Without a Proxy (Automatic Assignment)
```json
{
"cmd": "request.get",
"url": "https://example.com",
"profileId": "my-profile"
}
```
- A proxy is automatically assigned and saved to the profile
- All future requests use the same proxy (same IP)
- If the proxy stops working, a new one is automatically assigned
### Subsequent Requests
Once a profile has a proxy saved, just use the profile — no proxy needed:
```json
{
"cmd": "request.get",
"url": "https://example.com",
"profileId": "my-profile"
}
```
The saved proxy is automatically used.
### Proxy Use Cases
#### Consistent IP + Environment
```json
{"profileId": "account-1", "url": "https://example.com"}
{"profileId": "account-1", "url": "https://example.com"}
```
#### Stable Sessions
```json
{"profileId": "email-main", "url": "https://example.com"}
```
#### Using Your Own Proxy
```json
{"profileId": "scraper-1", "proxy": "http://user:pass@your-proxy.com:8000", "url": "https://target.com"}
{"profileId": "scraper-1", "url": "https://target.com/page1"}
{"profileId": "scraper-1", "url": "https://target.com/page2"}
```
#### Update the Proxy
```json
{
"profileId": "my-profile",
"proxy": "http://new-proxy:8000",
"url": "https://example.com"
}
```
#### Force a New Proxy
```json
{
"profileId": "my-profile",
"forceNewProxy": true,
"url": "https://example.com"
}
```
### Why Use Proxy Persistence?
| Benefit | Description |
|---------|-------------|
| IP Consistency | Maintains the same browser environment and regional IP across requests |
| Simpler Requests | Proxy is automatically handled — no need to manage it yourself |
| Stable Sessions | Avoids session resets caused by IP changes mid-workflow |
| Automatic Recovery | If an internal proxy fails, a new one is automatically assigned |
## Limitations
- Profiles are permanent once created — the browser configuration cannot be changed
- Maximum profile storage depends on your plan
- Very large cookie stores (>10MB) may impact performance
## FAQ
**Q: What happens if I don't specify a profileId?**
A: A temporary browser with a randomly generated configuration is used. Nothing is persisted.
**Q: Can I delete a profile?**
A: Yes, contact support or use the profile management API.
**Q: How long are profiles stored?**
A: Indefinitely, unless you delete them.
**Q: Do profiles work with proxies?**
A: Yes. Proxies are automatically saved and reused with profiles. The browser configuration stays the same regardless of which proxy you use.
**Q: Will a site recognize the same profile across requests?**
A: Yes — a profile provides a consistent browser environment, so the site sees a stable returning visitor rather than a brand-new environment on every request. This is the intended behavior for maintaining sessions.
**Q: What happens if my proxy stops working?**
A: If you provided your own proxy, you'll get an error asking you to provide a new one. If you're using an automatically assigned proxy, a new one is automatically assigned.
**Q: Can I use a profile without any proxy?**
A: Profiles always use a proxy to ensure consistent IP addresses. If you don't provide one, one is automatically assigned.
**Q: How do I change the proxy for an existing profile?**
A: Use `forceNewProxy: true` to assign a new proxy, or provide your own `proxy` URL. The new proxy replaces the stored one.
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Stateful web scraping](https://scrappey.com/qa/web-scraping-apis/what-is-stateful-web-scraping) — reusing state across requests
- [Browser fingerprinting](https://scrappey.com/qa/web-scraping-apis/what-is-browser-fingerprinting) — the client signature a profile pins
- [Browser environment consistency](https://scrappey.com/qa/anti-bot/what-is-browser-fingerprinting-evasion) — why stable profiles matter for repeatable browser workflows
---
# Making Requests
Source: https://docs.scrappey.com/docs/api-reference/requests
> Send a POST request to the Scrappey API with a URL, optional headers, cookies, and proxy, and the platform manages browser execution, rendering, session state, and response collection automatically.
## Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | ----------- |
| `cmd` | string | Yes | The command to execute, e.g. `request.get` or `request.post`. |
| `url` | string | Yes | The target URL to request. |
| `headers` | object | No | Custom request headers to send. |
| `cookies` | array | No | Cookies to include with the request. |
| `proxy` | string | No | Proxy to route the request through. |
## Request
Authenticate by including your API key in the query string. Send the command and target URL as a JSON body.
```javascript
const response = await fetch('https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
cmd: 'request.get',
url: 'https://example.com'
})
});
const data = await response.json();
console.log(data.solution.response);
```
## Response
A successful response returns `solution.verified: true` along with the page content, cookies, headers, and IP info.
```json
{
"solution": {
"verified": true,
"currentUrl": "https://httpbin.rs/get",
"statusCode": 200,
"userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:135.0) Gecko/20100101 Firefox/135.0",
"innerText": "{\"body_string\":\"\",\"headers\":{\"accept\":\"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\",\"accept-encoding\":\"gzip, br\",\"accept-language\":\"en-US,en;q=0.5\",\"cache-control\":\"no-cache\",\"cdn-loop\":\"cloudflare; loops=1\",\"cf-connecting-ip\":\"161.0.248.139\",\"cf-ipcountry\":\"TT\",\"cf-ray\":\"947fb8313e8fd6db-IAD\",\"cf-visitor\":\"{\\\"scheme\\\":\\\"https\\\"}\",\"host\":\"httpbin.rs\",\"pragma\":\"no-cache\",\"priority\":\"u=0, i\",\"sec-fetch-dest\":\"document\",\"sec-fetch-mode\":\"navigate\",\"sec-fetch-site\":\"none\",\"sec-fetch-user\":\"?1\",\"upgrade-insecure-requests\":\"1\",\"user-agent\":\"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:135.0) Gecko/20100101 Firefox/135.0\",\"x-forwarded-for\":\"161.0.248.139, 172.71.194.68\",\"x-forwarded-host\":\"httpbin.rs\",\"x-forwarded-port\":\"80\",\"x-forwarded-proto\":\"https\",\"x-forwarded-server\":\"ab65c85a31ae\",\"x-is-trusted\":\"yes\",\"x-real-ip\":\"161.0.248.139\"},\"json\":null,\"method\":\"GET\",\"origin\":\"161.0.248.139\",\"query\":null,\"uri\":\"/get\"}",
"cookies": [],
"cookieString": "",
"response": "
",
"responseHeaders": {
"date": "Fri, 30 May 2025 16:48:31 GMT",
"content-type": "application/json; charset=utf-8",
"cf-ray": "947fb8313e8fd6db-IAD",
"server": "cloudflare",
"content-encoding": "zstd",
"cf-cache-status": "DYNAMIC",
"report-to": "{\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report\\/v4?s=vjiHLteAY5BudLpu8FBSR7H1zrEVyMaB3p05NXTLkkFdkMwe2nW7pH2mmR2H1YZX6SnueOYkFzZ%2F6vNoSp9vHy5WiHmX%2FnYJ2SJ%2B3I7eebUZYEefUHJTlkRsbjMD\"}], \"group\":\"cf-nel\", \"max_age\":604800}",
"nel": "{\"success_fraction\":0, \"report_to\":\"cf-nel\", \"max_age\":604800}",
"alt-svc": "h3=\":443\"; ma=86400",
"server-timing": "cfCacheStatus;desc=\"DYNAMIC\", cfL4;desc=\"?proto=TCP&rtt=87643&min_rtt=82738&rtt_var=21002&sent=7&recv=9&lost=0&retrans=0&sent_bytes=3978&recv_bytes=2399&delivery_rate=52499&cwnd=238&unsent_bytes=0&cid=76cb71c0b81a2239&ts=388&x=0\"",
"x-firefox-spdy": "h2"
},
"requestHeaders": {
"host": "httpbin.rs",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:135.0) Gecko/20100101 Firefox/135.0",
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"accept-language": "en-US,en;q=0.5",
"accept-encoding": "gzip, deflate, br, zstd",
"upgrade-insecure-requests": "1",
"sec-fetch-dest": "document",
"sec-fetch-mode": "navigate",
"sec-fetch-site": "none",
"sec-fetch-user": "?1",
"connection": "keep-alive",
"priority": "u=0, i",
"pragma": "no-cache",
"cache-control": "no-cache"
},
"ipInfo": {
"status": "success",
"country": "Trinidad and Tobago",
"countryCode": "TT",
"region": "CHA",
"regionName": "Chaguanas",
"city": "Chaguanas",
"zip": "",
"lat": 10.5167,
"lon": -61.4167,
"timezone": "America/Port_of_Spain",
"isp": "Columbus Communications Trinidad Limited.",
"org": "Columbus Communications Trinidad Limited",
"as": "AS27665 Columbus Communications Trinidad Limited.",
"mobile": false,
"proxy": false,
"hosting": false,
"query": "161.0.248.139"
},
"method": "GET",
"type": "browser"
},
"timeElapsed": 10269,
"data": "success",
"session": "06c9de79-a61b-4eae-9121-11814440cc07"
}
```
## Error Response
An error returns a status code 200 with `solution.verified` set to `false`. This request is not charged. The response always includes `"data": "error"` and an `"error"` field explaining the failure reason.
```json
{
"solution": {
"verified": false
},
"timeElapsed": 4405,
"data": "error",
"error": "Datadome verification could not be completed for this request. Retry with different proxy settings.",
"info": "https://wiki.scrappey.com/getting-started#db336d2e7b3e42429966b01e2efb7d18",
"session": "4f8904bb-aac3-40d1-939e-55a00d3acafd"
}
```
---
# Screenshot
Source: https://docs.scrappey.com/docs/api-reference/screenshot
> Capture a full-page browser screenshot and receive it as a base64-encoded JPEG in the response, or have it uploaded to a public URL.
## Parameters
| Parameter | Type | Required | Description |
| ------------------ | ------- | -------- | ---------------------------------------------------------------------------- |
| `screenshot` | boolean | Yes | Enables full-page screenshot capture and returns base64 in the response. |
| `screenshotUpload` | boolean | No | Also uploads the screenshot to DigitalOcean Spaces and returns a public URL. |
| `screenshotWidth` | number | No | Width of the viewport before capture (in pixels). Default: 1280. |
| `screenshotHeight` | number | No | Height of the viewport before capture (in pixels). Default: 1024. |
| `fullPage` | boolean | No | Captures the full scrollable page height. Default: false. |
| `abortOnDetection` | array | No | List of URL patterns to block before page load (e.g. cookie-banner scripts). |
| `abortOnPostRequest` | boolean | No | When true, only POST requests matching `abortOnDetection` are blocked. |
Notes:
* Full-page capture is always used regardless of the resolution you set.
* Screenshots are produced as JPEG with quality 80.
## Request
Full-page base64 only:
```json
{
"url": "https://example.com",
"screenshot": true,
"screenshotWidth": 1440,
"screenshotHeight": 4000
}
```
Full-page base64 + upload to Spaces:
```json
{
"url": "https://example.com",
"screenshot": true,
"screenshotUpload": true,
"screenshotWidth": 1920,
"screenshotHeight": 1080
}
```
Blocking cookie banners for a clean screenshot:
```json
{
"url": "https://example.com",
"screenshot": true,
"abortOnDetection": [
"cookiebot.com",
"onetrust.com",
"quantcast.com",
"trustarc.com",
"/consent",
"gdpr-consent",
"cookie-law-info"
]
}
```
If any network request URL contains one of the specified values, it will be aborted before loading.
## Response
| Field | Type | Description |
| ------------------------ | ------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `solution.screenshot` | string | Base64-encoded JPEG of the full page. |
| `solution.screenshotUrl` | string | Public URL in Spaces, present only when `screenshotUpload: true`. Filenames include a prefix with dimensions: `screenshots/{width}x{height}_{random}.jpg`. |
| `abortOnDetectionResponse` | array | List of blocked requests, each with URL, headers, and payload. Present when `abortOnDetection` is used. |
Response snippet when `screenshotUpload: true`:
```json
{
"solution": {
"screenshot": "",
"screenshotUrl": "https:///screenshots/1920x1080_abCDeFgHiJKLmNopQrSt.jpg"
}
}
```
## Blocking Cookie Banners
Many websites show cookie or GDPR consent banners that can cover content and appear in screenshots. Use `abortOnDetection` to block the network requests these banners depend on before the page loads.
The browser intercepts all network requests while loading the page. If a request URL matches a pattern you specify, that request is blocked. Cookie banners often fail to load when their consent or tracking scripts are blocked, preventing the banner from ever appearing.
Common providers whose URLs you may want to block:
* Cookiebot
* OneTrust
* Quantcast
* TrustArc
* Cookie Law Info
* Custom GDPR or consent endpoints
Their request URLs often contain keywords like: `cookie`, `consent`, `gdpr`, `privacy`.
By default, all matching requests are blocked. Set `abortOnPostRequest: true` to restrict blocking to POST requests only.
After the page loads, blocked requests are available in `abortOnDetectionResponse`. Each entry includes the request URL, headers, and payload (if present) — useful for debugging or fine-tuning which patterns you block.
## Tips
* Base64 payloads can be large. If response size is a concern, prefer `screenshotUpload: true`.
* Choose `screenshotWidth` / `screenshotHeight` to influence layout before capture.
* JPEG is used to optimize size.
* Blocking cookie banners improves screenshot consistency across sites.
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Headless browser](https://scrappey.com/qa/web-scraping-apis/what-is-a-headless-browser) — what renders the page you capture
- [Headless browsers](https://scrappey.com/qa/web-automation/headless-browsers-guide) — driving a real browser to scrape
---
# request.post
Source: https://docs.scrappey.com/docs/api-reference/requests/request-post
> Sends a POST request using a real browser session, with all the power of request.get plus support for sending request bodies.
## Parameters
This endpoint accepts all the same parameters as [`request.get`](/docs/request-get), with one key additional parameter:
| Parameter | Type | Required | Description |
| ---------- | ------ | -------- | ----------- |
| `postData` | string | Yes | The body of the POST request. Must be a string in `application/x-www-form-urlencoded` format. Example: `a=b&c=d`. To send JSON data, set the appropriate content type using `customHeaders`: `"customHeaders": { "content-type": "application/json" }` |
### Shared Parameters
Refer to [`request.get`](/docs/request-get) for detailed descriptions of shared parameters like:
* `session`
* `cookies` / `cookiejar`
* `proxy` / `proxyCountry`
* `customHeaders`
* `includeImages`
* `includeLinks`
* `requestType`
* `localStorage`
## Request
### Send a Form-Encoded Body
The default. Pass `postData` as an `application/x-www-form-urlencoded` string:
```json
{
"cmd": "request.post",
"url": "https://httpbin.rs/post",
"postData": "a=b"
}
```
### Send a JSON Body
To send JSON, pass `postData` as an object and set the `content-type` header:
```json
{
"cmd": "request.post",
"url": "https://httpbin.org/post",
"postData": {
"a": "b"
},
"customHeaders": {
"content-type": "application/json"
}
}
```
## Response
```json
{
"solution": {
"verified": true,
"currentUrl": "https://httpbin.rs/post",
"statusCode": 200,
"userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:135.0) Gecko/20100101 Firefox/135.0",
"innerText": "{\"body_string\":\"a=b\",\"headers\":{ ... },\"json\":null,\"method\":\"POST\",\"origin\":\"223.181.110.217\",\"query\":null,\"uri\":\"/post\"}",
"cookies": [],
"cookieString": "",
"response": "...
...
",
"responseHeaders": {
"date": "Fri, 30 May 2025 16:58:07 GMT",
"content-type": "application/json; charset=utf-8",
"cf-ray": "947fc6437bdfe237-MRS",
"server": "cloudflare",
"content-encoding": "zstd",
"access-control-allow-origin": "https://httpbin.rs",
"vary": "Origin",
"cf-cache-status": "DYNAMIC",
"report-to": "{\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report\\/v4?s=eUhoz6vNC8yrH4zwf%2FPbaolJQUZ11kO%2BbGKyYYhw1AMo6wJBn%2BobEe%2Fy5KD6Zbz7cn%2B18Yjflsajy40MsUbANQ7AqtDdlfGgkmbwHRUGcK1WpRIw3LueBTl7zEy%2F\"}], \"group\":\"cf-nel\", \"max_age\":604800}",
"nel": "{\"success_fraction\":0, \"report_to\":\"cf-nel\", \"max_age\":604800}",
"alt-svc": "h3=\":443\"; ma=86400",
"server-timing": "cfCacheStatus;desc=\"DYNAMIC\", cfL4;desc=\"?proto=TCP&rtt=195244&min_rtt=136910&rtt_var=69448&sent=29&recv=20&lost=0&retrans=0&sent_bytes=11642&recv_bytes=3211&delivery_rate=60733&cwnd=256&unsent_bytes=0&cid=a544805ada7c1328&ts=4901&x=0\"",
"x-firefox-spdy": "h2"
},
"requestHeaders": {
"host": "httpbin.rs",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:135.0) Gecko/20100101 Firefox/135.0",
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"accept-language": "en-US,en;q=0.5",
"accept-encoding": "gzip, deflate, br, zstd",
"content-type": "application/x-www-form-urlencoded",
"content-length": "3",
"referer": "https://httpbin.rs/",
"origin": "https://httpbin.rs",
"upgrade-insecure-requests": "1",
"sec-fetch-dest": "document",
"sec-fetch-mode": "navigate",
"sec-fetch-site": "same-origin",
"sec-fetch-user": "?1",
"connection": "keep-alive"
},
"requestBody": "a=b",
"method": "POST",
"type": "browser"
},
"timeElapsed": 46276,
"data": "success",
"session": "67142d3a-40dd-4eda-8619-f95603f3bed7"
}
```
---
# request.get
Source: https://docs.scrappey.com/docs/api-reference/requests/request-get
> Make a GET request through a real browser session with full support for cookies, proxies, local storage, and more.
**Session Timeout:** A session will automatically expire **200 seconds** after the last request.
## Parameters
| Parameter | Type | Required | Description |
| --------------- | ------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `url` | string | Yes | The target URL to fetch. |
| `session` | string | No | Use an existing browser session. If not provided, a temporary instance is created and destroyed after the request. |
| `cookiejar` | array | No | Accepts an array of cookie objects to be set before visiting the page. Example: `[{ "name": "cookie1", "value": "value1", "domain": "domain.com", "path": "/" }]` |
| `cookies` | string | No | Raw cookie string to be injected before page load. Example: `_ga=GA1.1.143964650.1682153807; _fbp=fb.1.1711313596391.220282865` |
| `proxy` | string | No | Use a custom proxy. Supports `http://`, `socks4://`, or `socks5://`. Authentication is supported. Example: `http://username:password@127.0.0.1:8888`. Ignored if `session` is set — use session-specific proxy in `sessions.create` instead. |
| `proxyCountry` | string | No | Select a specific country when using the built-in rotating proxies. Example: `"UnitedStates"` |
| `customHeaders` | object | No | Override default browser headers. Example: `{ "auth": "value" }` |
| `includeImages` | boolean | No | Set to `true` to return a list of all image URLs found on the page. |
| `includeLinks` | boolean | No | Set to `true` to return a list of all hyperlinks found on the page. |
| `requestType` | string | No | Choose between `"browser"` (default full browser) and `"request"` (faster, no rendering). Automatically sets appropriate headers unless `customHeaders` is used. |
| `localStorage` | object | No | Set key-value pairs in the browser's localStorage before loading the page. Example: `{ "ac": "dsjfh84hfdsfk", "onboarding": "false" }` |
### Important Note
If you're using a **Cloudflare clearance cookie**, make sure the **User-Agent** used in your headers **exactly matches** the one used during clearance. Mismatched headers will trigger a challenge page.
## Advanced Options
These optional parameters control the output format, browser configuration, and session behavior. They work on `request.get`, `request.post`, and the other request commands.
| Parameter | Type | Required | Description |
| ------------------------ | ---------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `markdown` | boolean | No | Return the page converted to Markdown in `solution.markdown`. Respects `includeImages` / `includeLinks`. Combine with `filter: ["markdown"]` to return only the Markdown. |
| `pdf` | boolean | No | Render the loaded page to a PDF. The PDF is returned base64-encoded in the response. |
| `pageUrl` | string | No | Page-first fetch: load this URL first to establish a real session and cookies, then fetch the target `url` from within that browser context. Useful when an API endpoint depends on cookies or state established by a normal page visit. |
| `build` | string \| array | No | Select the browser build: `ff144`, `ff146`, `cl146`, `ff147`, `ff148`, `ff149`, or `ff150`. Pass an array (e.g. `["ff148", "ff150"]`) to select from available browser builds for compatibility testing. Defaults to the newest build. |
| `forceUniqueFingerprint` | boolean | No | Use a fresh browser configuration for this request. Useful for compatibility testing and isolated runs. Uses more resources. |
| `webrtcIpv4` | string | No | IPv4 address to align WebRTC network settings with when `forceUniqueFingerprint` is enabled. Defaults to the proxy's IPv4. |
| `webrtcIpv6` | string | No | IPv6 address to align WebRTC network settings with when `forceUniqueFingerprint` is enabled. Defaults to an IPv6 derived from the proxy, if available. |
| `captchaMethod` | string | No | Interactive challenge handling mode when `automaticallySolveCaptchas` is enabled. `"token"` (default) uses the integrated service; `"click"` handles the step interactively within the browser. |
| `overwriteLocale` | string | No | Force `navigator.language`, `navigator.languages`, and the `Accept-Language` header to this locale (e.g. `"en-US"`) instead of auto-detecting it from the proxy IP's country. |
### Return the page as Markdown
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"markdown": true,
"filter": ["markdown"]
}'
```
### Return the page as a PDF
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"pdf": true
}'
```
### Page-first fetch (session warm-up)
Load a normal page to acquire cookies/session, then fetch a protected API endpoint from inside the same browser:
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com/api/data",
"pageUrl": "https://example.com"
}'
```
### Pin or randomize the browser build
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"build": ["ff148", "ff149", "ff150"]
}'
```
### Force a unique fingerprint
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"forceUniqueFingerprint": true
}'
```
### Force a specific locale
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"proxyCountry": "Germany",
"overwriteLocale": "en-US"
}'
```
## Request
```json
{
"cmd": "request.get",
"url": "https://httpbin.org/get"
}
```
## Response
```json
{
"solution": {
"verified": true,
"currentUrl": "https://httpbin.rs/get",
"statusCode": 200,
"userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:135.0) Gecko/20100101 Firefox/135.0",
"innerText": "{\"body_string\":\"\",\"headers\":{\"accept\":\"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\",\"accept-encoding\":\"gzip, br\",\"accept-language\":\"en-US,en;q=0.5\",\"cache-control\":\"no-cache\",\"cdn-loop\":\"cloudflare; loops=1\",\"cf-connecting-ip\":\"161.0.248.139\",\"cf-ipcountry\":\"TT\",\"cf-ray\":\"947fb8313e8fd6db-IAD\",\"cf-visitor\":\"{\\\"scheme\\\":\\\"https\\\"}\",\"host\":\"httpbin.rs\",\"pragma\":\"no-cache\",\"priority\":\"u=0, i\",\"sec-fetch-dest\":\"document\",\"sec-fetch-mode\":\"navigate\",\"sec-fetch-site\":\"none\",\"sec-fetch-user\":\"?1\",\"upgrade-insecure-requests\":\"1\",\"user-agent\":\"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:135.0) Gecko/20100101 Firefox/135.0\",\"x-forwarded-for\":\"161.0.248.139, 172.71.194.68\",\"x-forwarded-host\":\"httpbin.rs\",\"x-forwarded-port\":\"80\",\"x-forwarded-proto\":\"https\",\"x-forwarded-server\":\"ab65c85a31ae\",\"x-is-trusted\":\"yes\",\"x-real-ip\":\"161.0.248.139\"},\"json\":null,\"method\":\"GET\",\"origin\":\"161.0.248.139\",\"query\":null,\"uri\":\"/get\"}",
"cookies": [],
"cookieString": "",
"response": "
",
"responseHeaders": {
"date": "Fri, 30 May 2025 16:48:31 GMT",
"content-type": "application/json; charset=utf-8",
"cf-ray": "947fb8313e8fd6db-IAD",
"server": "cloudflare",
"content-encoding": "zstd",
"cf-cache-status": "DYNAMIC",
"report-to": "{\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report\\/v4?s=vjiHLteAY5BudLpu8FBSR7H1zrEVyMaB3p05NXTLkkFdkMwe2nW7pH2mmR2H1YZX6SnueOYkFzZ%2F6vNoSp9vHy5WiHmX%2FnYJ2SJ%2B3I7eebUZYEefUHJTlkRsbjMD\"}], \"group\":\"cf-nel\", \"max_age\":604800}",
"nel": "{\"success_fraction\":0, \"report_to\":\"cf-nel\", \"max_age\":604800}",
"alt-svc": "h3=\":443\"; ma=86400",
"server-timing": "cfCacheStatus;desc=\"DYNAMIC\", cfL4;desc=\"?proto=TCP&rtt=87643&min_rtt=82738&rtt_var=21002&sent=7&recv=9&lost=0&retrans=0&sent_bytes=3978&recv_bytes=2399&delivery_rate=52499&cwnd=238&unsent_bytes=0&cid=76cb71c0b81a2239&ts=388&x=0\"",
"x-firefox-spdy": "h2"
},
"requestHeaders": {
"host": "httpbin.rs",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:135.0) Gecko/20100101 Firefox/135.0",
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"accept-language": "en-US,en;q=0.5",
"accept-encoding": "gzip, deflate, br, zstd",
"upgrade-insecure-requests": "1",
"sec-fetch-dest": "document",
"sec-fetch-mode": "navigate",
"sec-fetch-site": "none",
"sec-fetch-user": "?1",
"connection": "keep-alive",
"priority": "u=0, i",
"pragma": "no-cache",
"cache-control": "no-cache"
},
"ipInfo": {
"status": "success",
"country": "Trinidad and Tobago",
"countryCode": "TT",
"region": "CHA",
"regionName": "Chaguanas",
"city": "Chaguanas",
"zip": "",
"lat": 10.5167,
"lon": -61.4167,
"timezone": "America/Port_of_Spain",
"isp": "Columbus Communications Trinidad Limited.",
"org": "Columbus Communications Trinidad Limited",
"as": "AS27665 Columbus Communications Trinidad Limited.",
"mobile": false,
"proxy": false,
"hosting": false,
"query": "161.0.248.139"
},
"method": "GET",
"type": "browser"
},
"timeElapsed": 10269,
"data": "success",
"session": "06c9de79-a61b-4eae-9121-11814440cc07"
}
```
---
# sessions.destroy
Source: https://docs.scrappey.com/docs/api-reference/sessions/sessions-destroy
> Cleanly shuts down a browser session and frees up system resources.
## Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | ------------------------------------------ |
| `session` | string | Yes | The ID of the session you want to destroy. |
## Request
```json
{
"cmd": "sessions.destroy",
"session": "my-session-id"
}
```
## Response
```json
{
"solution": {},
"data": "success",
"timeElapsed": 14,
"session": "my-session-id"
}
```
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Stateful web scraping](https://scrappey.com/qa/web-scraping-apis/what-is-stateful-web-scraping) — reusing cookies and state across requests
---
# sessions.create
Source: https://docs.scrappey.com/docs/api-reference/sessions/sessions-create
> Start a persistent browser session that retains cookies, speeds up requests, and reduces repeated setup work across multi-step browser workflows.
A session remains active for **200 seconds** after the last request. After that, it is automatically destroyed.
## Parameters
| Parameter | Type | Required | Description |
| --- | --- | --- | --- |
| `session` | string | No | Custom session ID. If not set, a random UUID is generated. |
| `proxy` | string | No | Use a custom proxy. Supports `http://`, `socks4://`, or `socks5://` with authentication. Example: `"http://username:password@127.0.0.1:8888"`. Default: built-in rotating proxies from random countries. |
| `whitelistedDomains` | array | No | Allow requests to specified external domains. Example: `["probot.io", "google.com"]`. Useful for loading assets from third-party domains. |
| `datacenter` | boolean | No | Set to `true` to use a datacenter IP instead of a residential IP. Datacenter IPs offer higher speed and stability, but lower anonymity. |
| `browser` | array | No | Define the browser type and version range. Options: `chrome`, `firefox`, `safari`. Example: `[{ "name": "chrome", "minVersion": 116, "maxVersion": 117 }]`. |
| `operatingSystem` | array | No | Specify OS options. Options: `windows`, `macos`, `linux`, `android`, `ios`. Example: `["windows", "linux"]`. |
| `device` | array | No | Choose device type. Options: `desktop`, `mobile`. Example: `["desktop", "mobile"]`. |
## Why Use sessions.create
- **Speed:** Reuses the same browser instance across multiple requests.
- **Persistence:** Retains cookies and session data until explicitly destroyed.
- **Convenience:** Reuses cookies and browser state across requests.
## Request
```json
{
"cmd": "sessions.create",
"session": "my-session-id"
}
```
## Response
```json
{
"solution": {
"verified": true
},
"data": "success",
"session": "my-session-id",
"fingerprint": {},
"context": {
"_type": "BrowserContext",
"_guid": "browser-context@2356ec33a1fa91f08e3b3da7b55b13f6"
}
}
```
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Stateful web scraping](https://scrappey.com/qa/web-scraping-apis/what-is-stateful-web-scraping) — reusing cookies and state across requests
- [Session cookie](https://scrappey.com/qa/anti-bot/what-is-a-session-cookie) — the cookie that ties a session together
---
# proxyCountry
Source: https://docs.scrappey.com/docs/api-reference/requests/request-get/proxy-country
> Use any of the following values to specify a country for Scrappey's rotating proxies.
## Example
Pass `proxyCountry` alongside any request to route it through an IP in that country. Values are case-sensitive and must match the table below exactly (e.g. `UnitedStates`, not `united states`).
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"proxyCountry": "UnitedStates"
}'
```
## Supported Countries
| Country |
|---------|
| Afghanistan |
| Albania |
| Algeria |
| Argentina |
| Armenia |
| Aruba |
| Australia |
| Austria |
| Azerbaijan |
| Bahamas |
| Bahrain |
| Bangladesh |
| Belarus |
| Belgium |
| BosniaandHerzegovina |
| Brazil |
| BritishVirginIslands |
| Brunei |
| Bulgaria |
| Cambodia |
| Cameroon |
| Canada |
| Chile |
| China |
| Colombia |
| CostaRica |
| Croatia |
| Cuba |
| Cyprus |
| Czechia |
| Denmark |
| DominicanRepublic |
| Ecuador |
| Egypt |
| ElSalvador |
| Estonia |
| Ethiopia |
| Finland |
| France |
| Georgia |
| Germany |
| Ghana |
| Greece |
| Guatemala |
| Guyana |
| HashemiteKingdomofJordan |
| HongKong |
| Hungary |
| India |
| Indonesia |
| Iran |
| Iraq |
| Ireland |
| Israel |
| Italy |
| Jamaica |
| Japan |
| Kazakhstan |
| Kenya |
| Kosovo |
| Kuwait |
| Latvia |
| Liechtenstein |
| Luxembourg |
| Macedonia |
| Madagascar |
| Malaysia |
| Mauritius |
| Mexico |
| Mongolia |
| Montenegro |
| Morocco |
| Mozambique |
| Myanmar |
| Nepal |
| Netherlands |
| NewZealand |
| Nigeria |
| Norway |
| Oman |
| Pakistan |
| Palestine |
| Panama |
| PapuaNewGuinea |
| Paraguay |
| Peru |
| Philippines |
| Poland |
| Portugal |
| PuertoRico |
| Qatar |
| RepublicofLithuania |
| RepublicofMoldova |
| Romania |
| Russia |
| SaudiArabia |
| Senegal |
| Serbia |
| Seychelles |
| Singapore |
| Slovakia |
| Slovenia |
| Somalia |
| SouthAfrica |
| SouthKorea |
| Spain |
| SriLanka |
| Sudan |
| Suriname |
| Sweden |
| Switzerland |
| Syria |
| Taiwan |
| Tajikistan |
| Thailand |
| TrinidadandTobago |
| Tunisia |
| Turkey |
| Uganda |
| Ukraine |
| UnitedArabEmirates |
| UnitedKingdom |
| UnitedStates |
| Uzbekistan |
| Venezuela |
| Vietnam |
| Zambia |
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Residential proxy](https://scrappey.com/qa/proxies/what-is-a-residential-proxy) — the IP type used for geo-targeting
- [Residential vs datacenter](https://scrappey.com/qa/web-automation/proxy-types-comparison) — which proxy type to choose
- [Proxy web scraping](https://scrappey.com/qa/proxies/what-is-proxy-web-scraping) — how proxies route your requests
---
# request.[others]
Source: https://docs.scrappey.com/docs/api-reference/requests/request-others
> These commands let you send PUT, DELETE, PATCH, and PUBLISH HTTP requests through a real browser session, sharing the same structure as request.post.
## Parameters
These commands share the exact same structure and options as [`request.post`](#requestpost), including:
* All shared parameters from [`request.get`](#requestget)
* Support for `postData` (required)
* Support for `customHeaders`, including setting the appropriate `Content-Type`
| Parameter | Type | Required | Description |
| ---------- | ------ | -------- | ----------- |
| `postData` | string | Yes | The request body to send. Use string format (`application/x-www-form-urlencoded`) or set the content type to `application/json` via `customHeaders`. |
## Example
### request.put
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.put",
"url": "https://example.com/api/resource/1",
"postData": "field=value"
}'
```
### request.delete
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.delete",
"url": "https://example.com/api/resource/1"
}'
```
### request.patch (JSON body)
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.patch",
"url": "https://example.com/api/resource/1",
"postData": { "field": "new_value" },
"customHeaders": { "content-type": "application/json" }
}'
```
## Notes
For all other options, refer to:
* [`request.get`](#requestget) for general parameters
* [`request.post`](#requestpost) for body-related behavior
---
# Remaining balance
Source: https://docs.scrappey.com/docs/api-reference/remaining-balance
> Check how many requests remain in your Scrappey account balance with a single GET request.
## Request
```bash
GET https://publisher.scrappey.com/api/v1/balance?key=YOUR_API_KEY
```
Replace `YOUR_API_KEY` with your actual API key, which can be found in your [Scrappey Dashboard](https://publisher.scrappey.com).
## Response
```json
{
"balance": 1547
}
```
The value represents the number of **requests** you have left in your current balance.
---
# Session Management
Source: https://docs.scrappey.com/docs/api-reference/sessions
> Create and destroy persistent browser sessions that retain cookies and browser state, reducing repeated setup and overhead for subsequent requests.
## sessions.create
This launches a new browser instance that retains cookies until you destroy it with `sessions.destroy`. Use sessions when you want to reuse cookies and browser state across requests and skip the overhead of spinning up a new browser each time.
### Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | --------------------------------------------------- |
| `cmd` | string | Yes | Must be `sessions.create`. |
| `session` | string | Yes | A unique identifier for the browser session to create. |
### Request
```json
{
"cmd": "sessions.create",
"session": "my-session-id"
}
```
## sessions.destroy
This properly shuts down a browser instance. When you no longer need a session, close it to free resources.
### Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | ----------------------------------------------------- |
| `cmd` | string | Yes | Must be `sessions.destroy`. |
| `session` | string | Yes | The identifier of the browser session to destroy. |
### Request
```json
{
"cmd": "sessions.destroy",
"session": "my-session-id"
}
```
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Stateful web scraping](https://scrappey.com/qa/web-scraping-apis/what-is-stateful-web-scraping) — reusing cookies and state across requests
- [Session cookie](https://scrappey.com/qa/anti-bot/what-is-a-session-cookie) — the cookie that ties a session together
---
# Concurrency limits
Source: https://docs.scrappey.com/docs/api-reference/concurrency-limits
> Scrappey gives new accounts up to 200 concurrent threads by default and scales automatically up to 1000, with higher limits available on request.
## How Many Concurrent Threads Can I Use?
Scrappey allows all **new registered users** to start with **up to 200 concurrent threads** by default.
This is more than enough for the majority of use cases and ensures smooth performance and fair resource distribution.
If your project requires more, Scrappey makes it easy to scale.
## Scaling Beyond 200 Threads
We automatically allow **increases up to 1000 concurrent threads** without manual review, provided:
- Your **overall success rate is above 50%**.
- You are following our general API guidelines.
- You are not abusing endpoints.
## Special Cases / Higher Limits
If you require **more than 1000 concurrent threads**, this is no problem — simply reach out to support with a short explanation of your use case.
We generally approve higher limits quickly for:
- High-throughput data pipelines
- Large-scale monitoring, indexing, and QA workloads
- Long-running automation workflows that require sustained concurrency
There is **no hard upper limit** as long as your use case fits within reasonable resource bounds and success rates remain healthy.
## Quick Summary
| User Type | Threads Allowed | Requirements |
|-----------------|-----------------|------------------------|
| New Account | Up to 200 | None |
| Active Account | Up to 1000 | > 50% success rate |
| Special Request | > 1000 | Case-by-case approval |
## Tips for Scaling Smoothly
- **Monitor your success rate** regularly in the dashboard.
- Increase threads gradually to avoid sudden overload.
- If you encounter issues, check error codes in the docs first.
## Example
Concurrency is not a request parameter — it's simply how many requests you have **in flight at the same time**. Each simultaneous request consumes one thread. Fire requests in parallel from your own code and Scrappey processes them concurrently up to your account limit:
```bash
# Fire 5 requests in parallel — each one uses one concurrent thread
for url in \
"https://example.com/page1" \
"https://example.com/page2" \
"https://example.com/page3" \
"https://example.com/page4" \
"https://example.com/page5"
do
curl -s -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d "{\"cmd\": \"request.get\", \"url\": \"$url\"}" &
done
wait
```
If you exceed your allotted threads, additional requests queue until a thread frees up. Keep your concurrent count at or below your limit to avoid delays.
---
# Wrappers
Source: https://docs.scrappey.com/docs/guides/wrappers
> Use the official Python or Node.js wrapper to interact with the Scrappey API without writing raw HTTP calls.
Scrappey offers official wrappers for both **Python** and **Node.js**, enabling developers to interact seamlessly with the Scrappey API. These wrappers simplify session management, request execution, browser workflows, and response parsing. ([GitHub](https://github.com/pim97))
## Python Wrapper
* **GitHub**: [pim97/scrappey-wrapper-python](https://github.com/pim97/scrappey-wrapper-python)
* **PyPI**: [scrappeycom](https://pypi.org/project/scrappeycom/)
### Installation
```bash
pip install scrappeycom
```
### Usage Example
```python
from scrappeycom.scrappey import Scrappey
import uuid
api_key = 'YOUR_API_KEY'
scrappey = Scrappey(api_key)
session_data = {
'session': str(uuid.uuid4())
}
# Create a session
session = scrappey.create_session(session_data)
print('Session created:', session['session'])
# Make a GET request
response = scrappey.get({
'session': session['session'],
'url': 'https://httpbin.org/get'
})
print('GET Response:', response)
# Destroy the session
scrappey.destroy_session(session_data)
```
### Features
* Manages proxy configuration and response parsing
* Simplifies session creation and destruction
* Supports GET and POST requests with custom headers
* Provides error handling for common request issues
## Node.js Wrapper
* **GitHub**: [DemonMartin/scrappey-wrapper](https://github.com/DemonMartin/scrappey-wrapper)
* **NPM**: [scrappey](https://www.npmjs.com/package/scrappey)
### Installation
```bash
npm install scrappey
```
### Usage Example
```javascript
const Scrappey = require('scrappey');
const apiKey = 'YOUR_API_KEY';
const scrappey = new Scrappey(apiKey);
(async () => {
// Create a session
const sessionData = await scrappey.createSession();
const session = sessionData.session;
console.log('Session created:', session);
// Make a GET request
const response = await scrappey.getRequest({
url: 'https://httpbin.org/get',
session
});
console.log('GET Response:', response);
// Destroy the session
await scrappey.destroySession(session);
})();
```
### Features
* Manages proxy configuration and response parsing
* Manages sessions efficiently
* Supports both form data and JSON in POST requests
* Allows custom headers for requests
## Notes
For more detailed information and advanced usage, refer to the respective GitHub repositories:
* [scrappey-wrapper-python](https://github.com/pim97/scrappey-wrapper-python)
* [scrappey-wrapper](https://github.com/DemonMartin/scrappey-wrapper)
These wrappers are designed to streamline your web scraping tasks using Scrappey's powerful API.
---
# Wait For Load State
Source: https://docs.scrappey.com/docs/guides/browser-actions/wait-for-load-state
> Pause the browser's execution until a specific load state is reached, ensuring actions and data extraction occur only after the page is ready.
## Command
`"wait_for_load_state"`
## Parameters
| Parameter | Type | Required | Description |
| ------------------ | ------ | -------- | ------------------------------------------------------------------------------ |
| `waitForLoadState` | string | Yes | The load state to wait for. Options: `domcontentloaded`, `networkidle`, `load` |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "wait_for_load_state",
"waitForLoadState": "networkidle"
}
]
}'
```
## Notes
| State | Description |
| ------------------ | -------------------------------------------------------------- |
| `domcontentloaded` | Fired when the DOM is fully loaded (before styles/images/etc). |
| `networkidle` | Waits until there are no network requests for 500 ms. |
| `load` | Waits for the full load event including images and resources. |
---
# Direct Request
Source: https://docs.scrappey.com/docs/guides/browser-actions/direct-http-request
> A lightweight HTTP GET request with browser-compatible TLS settings — useful when full rendering is unnecessary.
## Command
`"cmd": "request.get"`, `"requestType": "request"`
## Cost
**0.2 credit(s)** per successful request.
## Description
This is a raw HTTP `GET` request that **does not launch a browser**. It uses a TLS fingerprint to imitate a real browser. It's useful for fast and cost-effective scraping when browser-based rendering is overkill.
### Pros
* Very **fast** (avg. ~5 seconds)
* Very **cheap** (0.2 credits per request)
* Low **resource usage**
* Low **proxy consumption**
* Good for **simple websites** and **API endpoints**
### Cons
* Cannot interact with the browser
* Cannot handle **advanced anti-bot protection**
* Not suitable for **JavaScript-heavy** or **SPA** sites
* Some sites may detect the request as bot traffic
## When to Use
Use `request.get` with `"requestType": "request"` when:
* The website doesn't require JavaScript rendering
* Speed and cost are more important than complexity
* You don't need to interact with the page (no clicking, scrolling, etc.)
* You're accessing simple data, APIs, or static HTML pages
For more complex sites that use JavaScript or heavy anti-bot techniques, consider switching to a full browser-based `request.get` without `"requestType": "request"`.
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"requestType": "request",
"url": "https://example.com"
}'
```
```json
{
"cmd": "request.get",
"url": "https://httpbin.rs/get",
"requestType": "request"
}
```
## Response
```json
{
"solution": {
"verified": true,
"statusCode": 200,
"userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0",
"cookies": [],
"response": "{\"body_string\":\"\",\"headers\":{\"accept\":\"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\",\"accept-encoding\":\"gzip, br\",\"accept-language\":\"en-US\",\"cdn-loop\":\"cloudflare; loops=1\",\"cf-connecting-ip\":\"119.18.77.197\",\"cf-ipcountry\":\"KR\",\"cf-ray\":\"947fd38f3f33aa5f-ICN\",\"cf-visitor\":\"{\\\"scheme\\\":\\\"https\\\"}\",\"dnt\":\"1\",\"host\":\"httpbin.rs\",\"priority\":\"u=0, i\",\"sec-fetch-dest\":\"document\",\"sec-fetch-mode\":\"navigate\",\"sec-fetch-site\":\"same-site\",\"sec-fetch-user\":\"?1\",\"upgrade-insecure-requests\":\"1\",\"user-agent\":\"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0\",\"x-forwarded-for\":\"119.18.77.197, 172.71.110.86\",\"x-forwarded-host\":\"httpbin.rs\",\"x-forwarded-port\":\"80\",\"x-forwarded-proto\":\"https\",\"x-forwarded-server\":\"ab65c85a31ae\",\"x-is-trusted\":\"yes\",\"x-real-ip\":\"119.18.77.197\"},\"json\":null,\"method\":\"GET\",\"origin\":\"119.18.77.197\",\"query\":null,\"uri\":\"/get\"}",
"responseHeaders": {
"alt-svc": "h3=\":443\"; ma=86400",
"cf-cache-status": "DYNAMIC",
"cf-ray": "947fd38f3f33aa5f-ICN",
"content-encoding": "zstd",
"content-type": "application/json; charset=utf-8",
"date": "Fri, 30 May 2025 17:07:12 GMT",
"nel": "{\"success_fraction\":0,\"report_to\":\"cf-nel\",\"max_age\":604800}",
"report-to": "{\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report\\/v4?s=HlXrquVBA%2BxOKRRHKkaWp%2FycU5igGEA2wtg5FIADowjVuesxwuAvZeZP%2FLANqbbGdvFOdkeImdOJsUO%2FS4LRLdVhgaz22cfqa4a0519XznkoSFmBbMdAebMvtkBk\"}],\"group\":\"cf-nel\",\"max_age\":604800}",
"server": "cloudflare",
"server-timing": "cfCacheStatus;desc=\"DYNAMIC\", cfL4;desc=\"?proto=TCP&rtt=21231&min_rtt=14258&rtt_var=14940&sent=7&recv=8&lost=0&retrans=0&sent_bytes=3903&recv_bytes=2091&delivery_rate=307195&cwnd=236&unsent_bytes=0&cid=e52ebc3ecd344e1c&ts=823&x=0\""
},
"requestHeaders": {
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"accept-encoding": "gzip, deflate, br, zstd",
"accept-language": "en-US",
"dnt": "1",
"sec-fetch-dest": "document",
"sec-fetch-mode": "navigate",
"sec-fetch-site": "same-site",
"sec-fetch-user": "?1",
"te": "trailers",
"upgrade-insecure-requests": "1",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0"
},
"method": "GET",
"type": "request"
},
"timeElapsed": 5493,
"data": "success",
"session": "ba4ff7a7-2422-41b9-99fa-dd11a5928ad4"
}
```
---
# Browser request
Source: https://docs.scrappey.com/docs/guides/browser-actions/browser-request
> Execute a fully rendered browser session to handle anti-bot mechanisms and retrieve dynamic or JavaScript-heavy content.
## Command
`"cmd": "request.get"`
## Parameters
| Parameter | Type | Required | Description |
| ---------- | ------ | -------- | -------------------------------------------- |
| `cmd` | string | Yes | Must be `"request.get"` for browser GET requests. |
| `url` | string | Yes | The URL to request. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com"
}'
```
## Notes
**Cost:** 1 credit per successful request.
**Pros**
* Handles bot protections like Cloudflare, CAPTCHA, and JS challenges
* Retrieves content that normal `GET` cannot access
* Supports browser actions (e.g., clicking, scrolling, waiting)
* Enables scraping from single-page applications (SPAs)
**Cons**
* Slower execution (avg ~30 seconds)
* Higher cost than simple `GET` requests
* More resource-intensive
* Relies on proxy usage and higher infrastructure demand
**Also Works With**
* `"cmd": "request.post"` for browser-based POST requests
* Compatible with `browserActions`, `autoparse`, proxies, and custom headers
**Use Cases**
* Scraping JavaScript-heavy websites
* Interacting with form submissions or dynamic elements
* Accessing sites when standard scraping is blocked
This is the most powerful and flexible way to extract data through Scrappey, ideal for advanced scraping workflows.
---
# Json Post Request
Source: https://docs.scrappey.com/docs/guides/browser-actions/json-post-request
> Send structured JSON payloads using a `POST` request through a real browser session for interacting with modern APIs expecting JSON-formatted input.
## Command
`"cmd": "request.post"`
## Parameters
| Parameter | Type | Required | Description |
| --------------- | ------ | -------- | ----------------------------------------------------------- |
| `url` | string | Yes | The endpoint to send the POST request to. |
| `postData` | object | Yes | JSON object to be sent in the body of the request. |
| `customHeaders` | object | Yes | Set `content-type` to `application/json` for JSON requests. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.post",
"url": "https://httpbin.org/post",
"customHeaders": {
"content-type": "application/json"
},
"postData": {
"a": "b"
}
}'
```
## Notes
* Ensure that `content-type` is explicitly set to `application/json`.
* The `postData` value should be a valid JSON object.
* You can also include additional headers like authorization tokens or user-agents under `customHeaders`.
### Use Cases
* Authenticating to RESTful APIs
* Submitting form data in JSON format
* Posting content to webhooks or services like Zapier, Slack, etc.
---
# Post Request
Source: https://docs.scrappey.com/docs/guides/browser-actions/post-request
> Send a POST request with form-encoded or JSON body to a target URL, useful for submitting forms, logging in, or triggering APIs.
## Command
`"cmd": "request.post"`
## Parameters
| Parameter | Type | Required | Description |
| --------------- | ------ | -------- | ------------------------------------------------------------------- |
| `url` | string | Yes | The URL to send the POST request to. |
| `postData` | string | Yes | Data to send. Must be `application/x-www-form-urlencoded`. |
| `customHeaders` | object | No | Set content type to `application/json` to send raw JSON. |
## Example
### Form Data
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.post",
"url": "https://httpbin.rs/post",
"postData": "a=b"
}'
```
### JSON Data
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.post",
"url": "https://httpbin.rs/post",
"postData": "{\"username\": \"john\", \"password\": \"1234\"}",
"customHeaders": {
"content-type": "application/json"
}
}'
```
## Notes
* `postData` must be a string.
* If using `application/json`, the JSON must be stringified.
* You can combine this with other browser actions like cookies, headers, or click automation.
Use `request.post` when interacting with login forms, submitting data, or calling APIs that require POST payloads.
---
# Retry
Source: https://docs.scrappey.com/docs/guides/browser-actions/retry
> Automatically retry a failed request up to 5 times to improve success rates against timeouts, captchas, or network instability.
## Command
`"retries"`
**Purpose**: Retries the entire request if it fails.
**Limits**:
- Maximum retries: `5`
- Max total execution time: `240` seconds (4 minutes)
## Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | --------------------------------------------- |
| `retries` | number | No | Number of automatic retries (1–5 recommended) |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"retries": 3
}'
```
## Notes
- Retries are only performed if the request fails (e.g., timeout, blocked, crash).
- Each retry counts against your total scrape usage.
- If the combined retries exceed 240 seconds, the request will be aborted.
Use automatic retries for reliability in unstable environments or when targeting sites with aggressive anti-bot systems.
---
# Add Referer
Source: https://docs.scrappey.com/docs/guides/browser-actions/add-referer
> Simulate natural browser navigation by visiting a page first or injecting a static Referer header before making a POST request.
## Command
`"request.post"` — use `getUrl` or `customHeaders` to supply the referer.
## Parameters
| Parameter | Type | Required | Description |
| --- | --- | --- | --- |
| `cmd` | string | Yes | Should be `"request.post"` |
| `url` | string | Yes | Target POST endpoint |
| `postData` | string | Yes | Data to send in the POST body (URL-encoded or JSON as required) |
| `getUrl` | string | No | Visit this URL first; it becomes the referer (most natural approach) |
| `customHeaders` | object | No | Add `"Referer"` directly via headers (when no anti-bot checks are involved) |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.post",
"url": "https://lift-api.vfsglobal.com/user/login",
"getUrl": "https://vfsglobal.com",
"postData": "a=b&b=c"
}'
```
## Notes
- `getUrl` is preferred when facing anti-bot protection, as it simulates a real navigation path.
- `customHeaders` is simpler and faster if no referer validation is enforced.
- This approach increases request authenticity and helps avoid blocks from systems that expect a valid navigation path before a POST operation.
---
# Record Video
Source: https://docs.scrappey.com/docs/guides/browser-actions/record-video
> Enable session video recording to capture and replay browser behavior for debugging and traceability.
## Command
`"video"` — set the `video` parameter to `true` on any request to record the entire browser session. The response returns a link to download or view the recorded video.
## Parameters
| Parameter | Type | Required | Description |
| --------- | ------- | -------- | -------------------------------------------- |
| `video` | boolean | Yes | Set to `true` to record the browser session. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"video": true
}'
```
## Notes
Use video recording to:
- Debug failed scraping sessions visually.
- Review step-by-step browser automation.
- Present recorded demonstrations of scraping flows.
- Confirm interaction success or failures post-request.
This feature is highly valuable for debugging JavaScript-heavy sites, login automation, complex interactions, or when you want traceability over what the browser executed.
---
# Mouse Movements
Source: https://docs.scrappey.com/docs/guides/browser-actions/mouse-movements
> Simulate realistic mouse movement patterns to better emulate real user behavior on sites that analyze behavioral signals.
## Command
`"mouseMovements"`
## Parameters
| Parameter | Type | Required | Description |
| ---------------- | ------- | -------- | ------------------------------------------------- |
| `mouseMovements` | boolean | Yes | Set to `true` to enable realistic mouse movement. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"mouseMovements": true
}'
```
## Notes
Enable this feature when:
- Navigating sites with bot protection based on behavioral analysis.
- Clicking buttons, filling out forms, or interacting with content.
- Needing more human-like scraping sessions.
This is ideal for interacting naturally with advanced bot-detection systems like those used on login forms, checkout pages, or gated content behind interaction-based verification. The feature uses actual recorded patterns instead of fully random coordinates.
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Behavioural detection](https://scrappey.com/qa/anti-bot/what-is-behavioural-detection) — why human-like movement matters
- [Playwright](https://scrappey.com/qa/web-scraping-apis/what-is-playwright) — the automation engine behind browser actions
---
# Mobile Proxy
Source: https://docs.scrappey.com/docs/guides/browser-actions/mobile-proxy
> Route your requests through real mobile carrier IPs to help satisfy strict anti-bot measures on websites that heavily track IP reputation and device behavior.
## Command
`"mobileProxy"`
## Parameters
| Parameter | Type | Required | Description |
| -------------- | ------- | -------- | ------------------------------------------------------------------- |
| `mobileProxy` | boolean | Yes | Set to `true` to use a mobile proxy. |
| `proxyCountry` | string | No | Country to route the mobile proxy through (e.g., `"UnitedStates"`). |
> **Tip**: View all available countries in the [Scrappey Builder](https://app.scrappey.com/#/builder) under **Advanced Options → Custom Proxy Country**.
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"mobileProxy": true,
"proxyCountry": "UnitedStates"
}'
```
## Notes
Best suited for:
* Accessing websites with aggressive anti-bot protection (e.g., ticketing, sneaker sites, streaming).
* Avoiding rate limits and CAPTCHAs.
* Mimicking real user traffic from mobile networks.
Use `mobileProxy` when authenticity and access success are more important than speed or cost. Cost is 7 credits per request.
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [What is a mobile proxy?](https://scrappey.com/qa/proxies/what-is-a-mobile-proxy) — what it is and when to use it
- [Residential proxy](https://scrappey.com/qa/proxies/what-is-a-residential-proxy) — the closest alternative
- [Proxy web scraping](https://scrappey.com/qa/proxies/what-is-proxy-web-scraping) — how proxies route your requests
- [ISP proxy](https://scrappey.com/qa/proxies/what-is-an-isp-proxy) — another high-trust option
---
# Datacenter Proxy
Source: https://docs.scrappey.com/docs/guides/browser-actions/datacenter-proxy
> Route the request through a datacenter proxy for speed-critical scraping where anti-bot detection is not a concern.
## Command
`"datacenter"` — Set the `datacenter` parameter to `true` on any request to route it through a datacenter proxy. Cost: 1 credit per request. Optionally choose a specific country via `proxyCountry`.
## Parameters
| Parameter | Type | Required | Description |
| -------------- | ------- | -------- | ------------------------------------------------------------ |
| `datacenter` | boolean | Yes | Set to `true` to enable datacenter proxy. |
| `proxyCountry` | string | No | Country to route the proxy through (e.g., `"UnitedStates"`). |
> **Note**: Full country list available at [Scrappey Builder](https://app.scrappey.com/#/builder) under Advanced Options → Custom Proxy Country.
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"datacenter": true,
"proxyCountry": "UnitedStates"
}'
```
## Notes
Best suited for:
* High-volume scraping with low latency requirements.
* Situations where IP reputation is not a major concern.
* Reducing cost while maintaining speed.
Use `datacenter` proxies when you need performance and cost-efficiency over stealth.
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [What is a datacenter proxy?](https://scrappey.com/qa/proxies/what-is-a-datacenter-proxy) — what it is and its trade-offs
- [Residential vs datacenter](https://scrappey.com/qa/web-automation/proxy-types-comparison) — which proxy type to choose
- [Proxy web scraping](https://scrappey.com/qa/proxies/what-is-proxy-web-scraping) — how proxies route your requests
- [Rotating proxy](https://scrappey.com/qa/proxies/what-is-a-rotating-proxy) — cycling IPs across requests
---
# Premium Proxy
Source: https://docs.scrappey.com/docs/guides/browser-actions/premium-proxy
> Route requests through real residential IPs to handle region restrictions and anti-bot systems, at a cost of 4 credits per request.
## Command
`"premiumProxy"`
## Parameters
| Parameter | Type | Required | Description |
| -------------- | ------- | -------- | ------------------------------------------------------------- |
| `premiumProxy` | boolean | Yes | Set to `true` to enable premium residential proxy. |
| `proxyCountry` | string | No | Country to route the proxy through (e.g., `"UnitedStates"`). |
> **Note**: Full country list available at [Scrappey Builder](https://app.scrappey.com/#/builder) under Advanced Options → Custom Proxy Country.
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"premiumProxy": true,
"proxyCountry": "UnitedStates"
}'
```
## Notes
Best suited for:
* Geo-targeted scraping (e.g., accessing US-specific content).
* Avoiding detection by using real residential IPs.
* Gaining access to harder-to-reach or bot-protected websites.
Use `premiumProxy` to significantly increase your success rate on highly protected sites by blending in as regular residential traffic.
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [What is a residential proxy?](https://scrappey.com/qa/proxies/what-is-a-residential-proxy) — what powers the premium pool
- [Residential vs datacenter](https://scrappey.com/qa/web-automation/proxy-types-comparison) — which proxy type to choose
- [Rotating proxy](https://scrappey.com/qa/proxies/what-is-a-rotating-proxy) — cycling IPs across requests
- [Proxy web scraping](https://scrappey.com/qa/proxies/what-is-proxy-web-scraping) — how proxies route your requests
---
# Wait For Url Load
Source: https://docs.scrappey.com/docs/guides/browser-actions/wait-for-url-load
> Pause browser execution until a specific URL substring is loaded, ensuring the page transition has completed before continuing with other browser actions.
## Command
`"waitForUrl"`
Uses `.includes()` to check if the current URL contains the given substring. This does not return the response body — it only ensures the page has transitioned.
## Parameters
| Parameter | Type | Required | Description |
| ------------ | ------ | -------- | ------------------------------------------------------------------------- |
| `url` | string | Yes | The starting URL to navigate to. |
| `waitForUrl` | string | Yes | Substring to wait for in the destination URL before continuing execution. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://www.mcmaster.com/products/screws/socket-head-screws~/alloy-steel-socket-head-screws-8/",
"waitForUrl": "/Content/Explanations.aspx?"
}'
```
## Notes
Best suited for:
* Websites that redirect or dynamically change the URL after certain actions.
* Ensuring the next page has loaded before executing further browser actions.
* Waiting for final navigation steps before data scraping or input.
Using `waitForUrl` guarantees your scraping workflow proceeds only after the target page has been fully navigated to.
---
# Intercept XHR Request
Source: https://docs.scrappey.com/docs/guides/browser-actions/intercept-xhr
> Use `interceptFetchRequest` to capture and return the responses of specific `fetch()` or XHR requests made during a page load.
## Command
`"interceptFetchRequest"` — monitor network activity and return specific XHR/fetch responses. URL matching uses `.includes()`; if more than one request matches the string, all are returned as an array.
## Parameters
| Parameter | Type | Required | Description |
| ----------------------- | ------ | -------- | ---------------------------------------------------------------------- |
| `interceptFetchRequest` | array | Yes | List of strings to match against outgoing request URLs |
| `url` | string | Yes | URL to be loaded and monitored |
| `browserActions` | array | No | Browser interactions (e.g., captcha solving) to perform during request |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://ahrefs.com/website-authority-checker/?input=ahrefs.com",
"interceptFetchRequest": [
"https://ahrefs.com/v4/stGetFreeWebsiteOverview"
],
"browserActions": [
{
"type": "solve_captcha",
"captcha": "turnstile"
}
]
}'
```
You will receive an array of JSON responses from all fetch/XHR requests that matched the string(s) provided.
## Notes
Great for:
- Sites that dynamically load content via JavaScript
- Skipping complex DOM parsing by getting direct API responses
- Interacting with protected or hidden endpoints
Leverage `interceptFetchRequest` to unlock rich backend data directly from dynamic web apps.
---
# Deepseek Autoparse
Source: https://docs.scrappey.com/docs/guides/browser-actions/deepseek-autoparse
> Use the DeepSeek large language model to convert dynamic web content into structured JSON in a single request.
## Command
`"Deepseek AI Autoparse"` — costs **2 credits** per successful request.
- **Model**: `deepseek`
- **Autoparse**: Set `"autoparse": true`
- **Optional**: Supply `"structure"` and `"cssSelector"` to guide parsing.
## Parameters
| Parameter | Type | Required | Description |
| ------------- | ------- | -------- | --------------------------------------------------------------------------- |
| `model` | string | Yes | Must be `"deepseek"` |
| `autoparse` | boolean | Yes | Enable autoparsing |
| `cssSelector` | string | No | Specific region of the website to parse |
| `structure` | object | No | Structure of the desired output. If omitted, model infers one automatically |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://www.imdb.com/title/tt0087363/?ref_=ls_t_10",
"filter": [
"autoparse"
],
"model": "deepseek",
"autoparse": true,
"cssSelector": "section[class='ipc-page-background ipc-page-background--base sc-afa4bed1-0 iMxoKo']",
"structure": {
"movie_info": {
"title": "str",
"language": "str"
},
"duration": "str",
"release_date": "str",
"rating": {
"average_rating": "float",
"total_votes": "int"
},
"certification": "str",
"cast": [
{
"actor_name": "str",
"role": "str"
}
],
"crew": [
{
"crew_name": "str",
"role": "str"
}
]
}
}'
```
Example response:
```json
{
"solution": {
"verified": true,
"type": "browser",
"autoparse": {
"movie_info": {
"title": "Gremlins",
"language": "English"
},
"duration": "1h 46m",
"release_date": "June 8, 1984",
"rating": {
"average_rating": 7.3,
"total_votes": 259000
},
"certification": "PG",
"cast": [
{
"actor_name": "Zach Galligan",
"role": "Billy"
},
{
"actor_name": "Phoebe Cates",
"role": "Kate"
}
],
"crew": [
{
"crew_name": "Joe Dante",
"role": "Director"
},
{
"crew_name": "Chris Columbus",
"role": "Writer"
}
]
}
},
"timeElapsed": 41380,
"data": "success",
"session": "b0380e15-3381-413c-8cad-daefe574afb1"
}
```
## Notes
DeepSeek uses a Mixture of Experts (MoE) architecture for fast inference. It is open-source, multilingual, and performs well on coding, math, and structured parsing tasks. DeepSeek-V3 access is available for free.
---
# Autoparse
Source: https://docs.scrappey.com/docs/guides/browser-actions/autoparse
> Automatically parse structured data from a web page using AI, with no need for exact selectors or knowledge of the HTML structure.
## Command
Enable autoparsing by setting `"autoparse": true` and providing a `structure` field that defines the JSON format you want the response to follow.
## Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | ----------- |
| `autoparse` | boolean | Yes | Set to `true` to enable AI-based autoparsing of the response. |
| `structure` | object | Yes | JSON object describing the data shape you want the AI to extract. |
## Example
```json
"structure": {
"movie_info": {
"title": "str",
"language": "str"
},
"duration": "str",
"release_date": "str",
"rating": {
"average_rating": "float",
"total_votes": "int"
},
"certification": "str",
"cast": [
{
"actor_name": "str",
"role": "str"
}
],
"crew": [
{
"crew_name": "str",
"role": "str"
}
]
}
```
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://movies.com/",
"autoparse": true,
"structure": {
"movie_info": {
"title": "str",
"language": "str"
},
"duration": "str",
"release_date": "str",
"rating": {
"average_rating": "float",
"total_votes": "int"
},
"certification": "str",
"cast": [
{
"actor_name": "str",
"role": "str"
}
],
"crew": [
{
"crew_name": "str",
"role": "str"
}
]
}
}'
```
## Notes
- Costs **6 credits** per successful request.
- Parses HTML into JSON automatically using AI.
- You define the desired data structure — the AI fills it in.
- Ideal for dynamic pages or unknown HTML structures.
---
# Wait For Function
Source: https://docs.scrappey.com/docs/guides/browser-actions/wait-for-function
> Waits until a custom JavaScript expression evaluates to `true` inside the browser, useful for dynamic content or specific page states.
## Command
`"wait_for_function"`
## Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | ------------------------------------------------------------------- |
| `code` | string | Yes | JavaScript code to evaluate. Must return a truthy value to proceed. |
| `timeout` | number | No | Max time (ms) to wait for the condition (default: 60000 ms). |
## Example
#### Wait for element to appear
```javascript
document.querySelector('.dynamic-content') !== null
```
#### Wait for text and status change
```javascript
const element = document.querySelector('.status');
return element &&
element.textContent.includes('Ready') &&
!element.classList.contains('loading')
```
#### Example Request (cURL)
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "wait_for_function",
"code": "document.querySelector('.dynamic-content') !== null",
"timeout": 30000
},
{
"type": "wait_for_function",
"code": "const el = document.querySelector('.status'); return el && el.textContent.includes('Ready') && !el.classList.contains('loading');"
}
]
}'
```
## Notes
- The condition is evaluated in the **browser context**.
- It **polls every 200ms** until the condition is met.
- **Throws an error** if the timeout is reached.
- Any **JavaScript expression that evaluates to true/false** can be used.
---
# Dropdown Action
Source: https://docs.scrappey.com/docs/guides/browser-actions/dropdown-action
> Select an option from a dropdown menu on a webpage by targeting it with a CSS selector and specifying the desired option via index or value.
## Command
`"dropdown"`
## Parameters
| Parameter | Type | Required | Description |
| ----------------- | ------- | -------- | ------------------------------------------------------------------- |
| `cssSelector` | string | Yes | CSS selector or XPath for the dropdown element. |
| `index` | number | No | Index of the option to select (starting from 0). Either `index` or `value` must be provided. |
| `value` | string | No | Value of the option to select (matches HTML `value` attribute). Either `index` or `value` must be provided. |
| `wait` | number | No | Time in seconds to wait after the action. |
| `waitForSelector` | string | No | Wait for a specific element after selection. |
| `timeout` | number | No | Max time in milliseconds to wait for the dropdown (default: 60000). |
| `ignoreErrors` | boolean | No | Continue even if the action fails. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "dropdown",
"cssSelector": "#exampleDropdown",
"index": 1,
"wait": 2
},
{
"type": "dropdown",
"cssSelector": "#anotherDropdown",
"value": "option2",
"waitForSelector": "#resultElement"
}
]
}'
```
## Notes
Either `index` or `value` must be provided. Do not use both at the same time.
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Playwright](https://scrappey.com/qa/web-scraping-apis/what-is-playwright) — the automation engine behind browser actions
- [Dynamic content scraping](https://scrappey.com/qa/web-automation/dynamic-content-scraping) — working with interactive pages
---
# Click Action
Source: https://docs.scrappey.com/docs/guides/browser-actions/click-action
> Simulate a click on a specific element on a webpage using a CSS selector or XPath within a browser session.
## Command
`"click"`
## Parameters
| Parameter | Type | Required | Description |
| ----------------- | ------- | -------- | -------------------------------------------------------- |
| `cssSelector` | string | Yes | CSS selector or XPath of the element to click. |
| `wait` | number | No | Wait for X seconds after clicking. |
| `waitForSelector` | string | No | Wait for a specific element to appear after the click. |
| `when` | string | No | Lifecycle phase to run the action: `beforelaunch`, `beforeload`, `afterload`, or `after_captcha`. Default is `afterload`. |
| `ignoreErrors` | boolean | No | Set to `true` to continue even if an error occurs. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "click",
"cssSelector": "#exampleButton"
}
]
}'
```
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Playwright](https://scrappey.com/qa/web-scraping-apis/what-is-playwright) — the automation engine behind browser actions
- [Headless browsers](https://scrappey.com/qa/web-automation/headless-browsers-guide) — driving a real browser to scrape
---
# Type Actions
Source: https://docs.scrappey.com/docs/guides/browser-actions/type-action
> Use this action to type a specific text into a selected input field on a webpage.
## Command
`"type"`
## Parameters
| Parameter | Type | Required | Description |
| -------------- | ------- | -------- | ------------------------------------------------------------ |
| `cssSelector` | string | Yes | CSS selector of the input field where text should be typed. |
| `text` | string | Yes | The string to be typed. |
| `wait` | number | No | Wait for X seconds after typing the text. |
| `when` | string | No | Lifecycle phase to run the action: `beforelaunch`, `beforeload`, `afterload`, or `after_captcha`. Default is `afterload`. |
| `ignoreErrors` | boolean | No | Set to `true` to continue even if an error occurs. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "type",
"cssSelector": "#searchInput",
"text": "Example search"
}
]
}'
```
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Playwright](https://scrappey.com/qa/web-scraping-apis/what-is-playwright) — the automation engine behind browser actions
- [Dynamic content scraping](https://scrappey.com/qa/web-automation/dynamic-content-scraping) — working with interactive pages
---
# Discord Login
Source: https://docs.scrappey.com/docs/guides/browser-actions/discord-login
> Automatically authenticate with Discord using a valid authorization token by injecting it into the browser session.
## Command
`"discord_login"`
This action logs into Discord by injecting an authorization token into the browser session. Use it to skip the login page and access authenticated areas directly.
## Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | ------------------------------------------------------------------------------------ |
| `token` | string | Yes | Your Discord authorization token. Required for login. |
| `wait` | number | No | Time in seconds to wait after logging in. Helpful for letting Discord fully load. |
| `when` | string | No | Run the action `beforeload` or `afterload`. Default is `afterload`. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://discord.com",
"browserActions": [
{
"type": "discord_login",
"token": "YOUR_DISCORD_AUTH_TOKEN",
"wait": 5
}
]
}'
```
## Notes
Using Discord tokens for automation is against Discord's Terms of Service. Only use this feature on accounts you own and control, and for ethical automation purposes.
Use case ideas:
* Access a Discord server's dashboard to scrape settings or user data.
* Automatically navigate to specific pages inside Discord after login.
* Pre-authenticate before submitting bot commands or forms.
Tips:
* Always wait a few seconds (`wait: 5`) to allow the Discord client to fully render after login.
* Combine with `goto`, `click`, or `execute_js` to interact with Discord's interface post-login.
---
# Go To
Source: https://docs.scrappey.com/docs/guides/browser-actions/go-to
> Navigate to a different page or route during your scraping flow — essential for multi-page interactions and workflows.
## Command
`"goto"`
This browser action navigates the browser to a new URL, just like a user would by clicking a link or entering a URL in the address bar.
## Parameters
| Parameter | Type | Required | Description |
| -------------- | ------- | -------- | ------------------------------------------------------------------------------------------------------------------------------ |
| `url` | string | Yes | The target URL to visit. If the domain is different from the base request, make sure it is added to your `whitelistedDomains`. |
| `when` | string | No | Lifecycle phase to run the action: `beforelaunch`, `beforeload`, `afterload`, or `after_captcha`. Default is `afterload`. |
| `ignoreErrors` | boolean | No | Set to `true` to continue execution even if the navigation fails. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "goto",
"url": "https://example.com/page"
}
]
}'
```
## Notes
Use case ideas:
* Navigate to a form or detail page before scraping fields.
* Handle multi-step flows by jumping between URLs.
* Move to a CAPTCHA or verification page before solving.
Tips:
* Always ensure the target domain is whitelisted in your request settings (`whitelistedDomains`).
* Use with `wait_for_selector` or `solve_captcha` to ensure the next step happens after the page is fully loaded.
* Combine with `if` or `while` for conditional navigation logic.
---
# Wait For Selector
Source: https://docs.scrappey.com/docs/guides/browser-actions/wait-for-selector
> Pause execution until a specific element appears in the DOM — ideal for waiting on AJAX content, modal windows, or any dynamically loaded section.
## Command
`"wait_for_selector"`
This action waits until a particular CSS selector appears on the page. It's useful for synchronization and ensuring elements are present before interacting with them.
## Parameters
| Parameter | Type | Required | Description |
| -------------- | ------- | -------- | ----------------------------------------------------------------------------------------------- |
| `cssSelector` | string | Yes | The CSS selector to wait for. Example: `[class='col']` |
| `timeout` | number | No | Time in milliseconds to wait before timing out. Default is system-defined. Example: `30000` |
| `when` | string | No | When to run the check: `"beforeload"` or `"afterload"`. Default is `"afterload"`. |
| `ignoreErrors` | boolean | No | Set to `true` to continue even if the selector is not found. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "wait_for_selector",
"cssSelector": "[class='\''example'\'']",
"timeout": 30000
}
]
}'
```
## Notes
Use Case Ideas:
* Wait for content to load after an AJAX call before scraping.
* Detect when modals or confirmation boxes appear before clicking.
* Ensure dynamic tables or product listings are fully rendered.
Tips:
* Use `wait_for_selector` before actions like `click`, `type`, or `solve_captcha`.
* Combine with `ignoreErrors: true` if the element may not always appear, but scraping should continue.
---
# Wait
Source: https://docs.scrappey.com/docs/guides/browser-actions/wait
> Pause the execution of browser actions for a specified amount of time — useful for waiting for dynamic content, animations, or cooldowns.
## Command
`"wait"`
This action delays further execution for a defined number of seconds, giving the page time to load or content to render.
## Parameters
| Parameter | Type | Required | Description |
| -------------- | ------- | -------- | ------------------------------------------------------------------------------------ |
| `wait` | number | Yes | Number of seconds to pause execution. |
| `when` | string | No | Define whether the wait happens `beforeload` or `afterload`. Default is `afterload`. |
| `ignoreErrors` | boolean | No | Set to `true` to suppress any error if the wait fails and continue execution. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "wait",
"wait": 5
}
]
}'
```
## Notes
* Give the page time to fully load before interacting with elements.
* Wait for popups or animations to finish before scraping.
* Introduce realistic pauses between actions to simulate human behavior.
* Combine with `click`, `scroll`, or `solve_captcha` to ensure smooth flow.
* Use `when: "beforeload"` if you need to delay before the page begins loading.
---
# Execute Javascript
Source: https://docs.scrappey.com/docs/guides/browser-actions/execute-javascript
> Run custom JavaScript code directly within the page context during a scraping session.
## Command
`"execute_js"`
This action injects and runs your JavaScript code inside the target page, just like a script executed from the browser's console.
## Parameters
| Parameter | Type | Required | Description |
| ----------------- | ------- | -------- | ------------------------------------------------------------------------------------------------- |
| `code` | string | Yes | The JavaScript code to execute. This must be a valid JavaScript string. |
| `dontReturnValue` | boolean | No | Set to `true` if the script has no return value. By default the evaluated result is captured. |
## Return Values
The result of each `execute_js` action is pushed onto the `javascriptReturn` array (in execution order). You can reference a previous result inside a later action using the `{javascriptReturn[n]}` placeholder, where `n` is the zero-based index:
```json
{
"browserActions": [
{
"type": "execute_js",
"code": "document.querySelector('.token').dataset.value"
},
{
"type": "type",
"cssSelector": "#input",
"text": "{javascriptReturn[0]}"
}
]
}
```
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "execute_js",
"code": "console.log('\''Hello, world!'\'');"
}
]
}'
```
## Notes
Use Case Ideas:
* Extract values or text content from the DOM.
* Trigger JavaScript events like clicks or input changes.
* Interact with pages that use frameworks like React or Vue by invoking component methods.
* Modify the DOM before parsing the response.
Tips:
* Avoid long-running loops or infinite scripts — they may cause timeouts.
* Escape special characters properly when sending code inside JSON.
* You can combine this with `if`, `scroll`, `click`, or `type` for dynamic control flows.
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Dynamic content scraping](https://scrappey.com/qa/web-automation/dynamic-content-scraping) — running JS to reach content
- [Headless browser](https://scrappey.com/qa/web-scraping-apis/what-is-a-headless-browser) — the engine that runs your JS
---
# Keyboard
Source: https://docs.scrappey.com/docs/guides/browser-actions/keyboard
> Simulate specific keyboard actions during your scraping session, useful for navigation, form control, or clearing fields.
## Command
`"keyboard"`
This browser action triggers a specific key press event, such as `Enter`, `Tab`, or `ArrowDown`, optionally on a selected element.
## Parameters
| Parameter | Type | Required | Description |
| ------------- | ------ | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| `value` | string | Yes | The keyboard action to simulate. Must be one of: `tab`, `enter`, `space`, `arrowdown`, `arrowup`, `arrowleft`, `arrowright`, `backspace`, `clear`. |
| `cssSelector` | string | No | Required only when using `clear`. Specifies which element the action applies to. |
## Supported `value` Options
| Action | Description |
| ------------ | ----------------------------------------------------------- |
| `tab` | Simulates pressing the Tab key |
| `enter` | Simulates pressing Enter |
| `space` | Simulates pressing Spacebar |
| `arrowdown` | Scrolls or moves focus down |
| `arrowup` | Scrolls or moves focus up |
| `arrowleft` | Moves cursor/focus left |
| `arrowright` | Moves cursor/focus right |
| `backspace` | Deletes one character |
| `clear` | Clears the value of an input field (requires `cssSelector`) |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "keyboard",
"value": "enter"
}
]
}'
```
## Notes
* Submit a form using `enter` after typing into an input field.
* Navigate a dropdown or form with `tab` and `arrow` keys.
* Clear a field before typing a new value using `clear`.
* Combine with `type`, `click`, or `wait` for more natural form interactions.
* Make sure the `cssSelector` targets an interactive or input element when using `clear`.
---
# Scroll
Source: https://docs.scrappey.com/docs/guides/browser-actions/scroll
> Perform a scrolling action either to a specific element or to the bottom of the page — useful for loading dynamic content or triggering lazy-loaded elements.
## Command
`"scroll"`
Use this command to simulate user-like scrolling behavior during scraping or automation.
## Parameters
| Parameter | Type | Required | Description |
| ------------- | ------ | -------- | ------------------------------------------------------------------------------------------------------------ |
| `cssSelector` | string | No | The CSS selector of the element to scroll to. If omitted, the page will scroll to the bottom by default. |
| `repeat` | number | No | Number of times to repeat the scroll. This enables infinite scrolling behavior. |
| `delayMs` | number | No | Delay between each scroll in milliseconds — useful for waiting on content to load between scrolls. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "scroll",
"cssSelector": "#bottomElement",
"repeat": 3,
"delayMs": 1000
}
]
}'
```
## Notes
Use Case Ideas:
* Load more search results on websites with infinite scroll.
* Trigger lazy-loaded images or content blocks.
* Scroll to a form or button before interacting with it (e.g., clicking or typing).
Tips:
* Combine with `waitForSelector` after scroll to ensure the element has loaded before scraping.
* Use high `repeat` counts + delay for deep content loading.
* Can be combined with `if` or `while` browser actions for intelligent scrolling.
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Scraping infinite-scroll pages](https://scrappey.com/qa/web-scraping-apis/how-to-scrape-infinite-scroll-pages) — loading content as you scroll
- [Dynamic content scraping](https://scrappey.com/qa/web-automation/dynamic-content-scraping) — working with interactive pages
---
# While Loop
Source: https://docs.scrappey.com/docs/guides/browser-actions/while-loop
> Repeat a set of browser actions until a specific condition is no longer true.
## Command
`"while"`
Use this command to perform looping logic within the browser context. It's useful for tasks like handling CAPTCHAs, paginated scraping, or waiting for content to disappear/appear.
## Parameters
| Parameter | Type | Required | Description |
| ------------- | ------- | -------- | ------------------------------------------------------------------------------------------------------------- |
| `condition` | string | Yes | A JavaScript expression that returns `true` or `false`. The loop continues as long as it evaluates to `true`. |
| `then` | array | Yes | Array of browser actions to execute on each iteration while the condition is true. |
| `maxAttempts` | number | No | Max number of loop iterations to prevent infinite loops. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "while",
"condition": "document.querySelector('body').innerText.toLowerCase().includes('terms')",
"then": [
{
"type": "goto",
"url": "https://example.com/alternate-page"
},
{
"type": "solve_captcha",
"captcha": "turnstile"
}
]
}
]
}'
```
## Notes
* You can combine this with other browser actions like `click`, `type`, `goto`, `wait`, or even another `if` or `while` block.
* Always consider setting a `maxAttempts` to avoid infinite loops and excessive resource usage.
* Use Case Ideas: looping through paginated content until no "Next" button appears, retrying CAPTCHA solving until successful, or waiting for an element to disappear before proceeding.
---
# Conditional
Source: https://docs.scrappey.com/docs/guides/browser-actions/conditional
> Execute different browser actions depending on the content of the page, using JavaScript expressions evaluated in the browser context.
## Command
`"if"`
Use conditional logic to make your scraping smarter — only proceed with certain actions if specific content is detected.
## Parameters
| Parameter | Type | Required | Description |
| ----------- | ------ | -------- | -------------------------------------------------------------------------------------------------- |
| `condition` | string | Yes | A JavaScript expression that evaluates to `true` or `false`. This is executed in the page context. |
| `then` | array | Yes | An array of browser actions to execute if the condition returns `true`. |
| `or` | array | No | An array of browser actions to execute if the condition returns `false`. |
You can nest `if` blocks inside each other for complex decision-making.
## Example
In this example, we visit a page and:
* If the body text contains the word "terms", we navigate to a different URL.
* Otherwise, we click a button.
```json
{
"cmd": "request.get",
"url": "https://www.arena-top100.com/index.php?a=in&u=sergey1234",
"browserActions": [
{
"type": "if",
"condition": "document.querySelector('body').innerText.toLowerCase().includes('terms')",
"then": [
{
"type": "if",
"condition": "document.querySelector('body').innerText.toLowerCase().includes('terms')",
"then": [
{
"type": "goto",
"url": "https://www.arena-top100.com/index.php?a=in&u=test1"
}
]
}
],
"or": [
{
"type": "click",
"cssSelector": "#continueButton"
}
]
}
]
}
```
Test it with cURL:
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://www.arena-top100.com/index.php?a=in&u=sergey1234",
"browserActions": [ ... ]
}'
```
## Notes
The `if` conditional action works across all Scrappey clients, including Python, JavaScript (Node.js), PHP, Java, Go, Ruby, C#, and cURL.
Conditional logic is well-suited for handling unexpected redirects, skipping cookie/consent modals, navigating multi-step flows, and avoiding unnecessary browser actions.
---
# Error Codes
Source: https://docs.scrappey.com/docs/guides/error-codes
> Reference table of all Scrappey error codes, their error messages, and what to do when you encounter them.
## Example
When a request fails, the response has `"data": "error"` and an `error` field containing the message. Match the message against the table below to find the corresponding `CODE-XXXX`, which is the stable identifier to handle in your code. Public messages are normalized for clarity; raw, vendor-specific diagnostics are available in the debug view of your Scrappey dashboard.
```json
{
"solution": {},
"timeElapsed": 1234,
"data": "error",
"session": "your-session-id",
"error": "Cloudflare blocked"
}
```
## Error Code Reference
| **Code** | **Error Message** | **Description** |
| ------------ | --------------------------------------------------------------------- | ------------------------------------------------------------------ |
| `CODE-0001` | Server is overloaded | All server capacity is used, please try again. |
| `CODE-0002` | Cloudflare blocked | Cloudflare blocked the request, preventing access. |
| `CODE-0003` | Cloudflare too many attempts, try again | Too many attempts from your IP; try again later. |
| `CODE-0004` | Invalid cmd command | The provided command is invalid. |
| `CODE-0005` | Tunnel connection failed | Tunnel connection could not be established. |
| `CODE-0006` | ERR\_HTTP\_RESPONSE\_CODE\_FAILURE | Failure in HTTP response. |
| `CODE-0007` | Could not click turnstile button | Clicking the CAPTCHA button failed. |
| `CODE-0007` | Proxy error - ERR\_TUNNEL\_CONNECTION\_FAILED or ERR\_EMPTY\_RESPONSE | Proxy error. Try a different proxy. |
| `CODE-0008` | Ticketmaster blocked | Ticketmaster blocked the request. |
| `CODE-0009` | Error from ChatGPT, try again | ChatGPT API returned an error. |
| `CODE-0010` | Blocked proxy on Datadome | Proxy blocked by Datadome. |
| `CODE-0011` | Datadome verification could not be completed | A Datadome verification step could not be completed. |
| `CODE-0012` | Could not parse datadome cookie | Issue parsing Datadome cookie. |
| `CODE-0013` | Datadome cookie processing error | A Datadome cookie could not be processed. |
| `CODE-0014` | Could not load Datadome | Datadome could not be loaded. |
| `CODE-0015` | Socks4 With Authentication not Supported | Socks4 with authentication is not supported. |
| `CODE-0016` | Socks5 With Authentication not Supported | Socks5 with authentication is not supported. |
| `CODE-0017` | Cloudflare verification temporarily unavailable, try again later | Temporary incompatibility with an updated Cloudflare release. |
| `CODE-0018` | Too high error rate for this URL | Temporary ban due to high error rate. Contact support. |
| `CODE-0019` | The proxy server is refusing connections | Check proxy settings. Proxy is refusing connections. |
| `CODE-0020` | Could not find intercept request | System could not find the intercept request. |
| `CODE-0021` | Unknown error occurred with request | An unknown request error occurred. |
| `CODE-0022` | Captcha type solve\_captcha is not found | The specified captcha type was not found. |
| `CODE-0023` | Turnstile solve\_captcha was not found | Turnstile CAPTCHA handler not found. |
| `CODE-0024` | Proxy timeout - proxy too slow | Proxy took too long to respond. |
| `CODE-0025` | NS\_ERROR\_NET\_TIMEOUT - proxy too slow | Browser-level timeout due to slow proxy. |
| `CODE-0026` | Internal browser error | Internal browser failure. |
| `CODE-0027` | No elements found for this CSS selector | The specified CSS selector did not match any elements. |
| `CODE-0028` | PerimeterX verification could not be completed | A PerimeterX verification step could not be completed. |
| `CODE-0029` | Too many sessions open | Maximum sessions exceeded. Sessions auto-expire after 240 seconds. |
| `CODE-0030` | Browser name must be: firefox, chrome or safari | Only these browsers are supported. |
| `CODE-0031` | Request error, please try again | General request error. Retry. |
| `CODE-0032` | Turnstile verification could not be completed | A Turnstile verification step could not be completed. Try a different proxy. |
| `CODE-0033` | MtCaptcha verification could not be completed | An MtCaptcha verification step could not be completed. |
| `CODE-0034` | Datadome verification could not be completed after multiple attempts | Retry with different proxy settings. |
| `CODE-0035` | Could not load geetest | Geetest CAPTCHA failed to load. |
| `CODE-0036` | Keyboard action value not found | The specified keyboard action is invalid. |
| `CODE-0037` | Datadome was blocked | Datadome blocked the request. Try again with a different proxy. |
| `CODE-10000` | Unknown error - has to be specified | An unspecified error occurred. Needs review. |
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [403 Forbidden](https://scrappey.com/qa/http-errors/what-is-a-403-error) — the classic "blocked by anti-bot" code
- [429 Too Many Requests](https://scrappey.com/qa/http-errors/what-is-a-429-error) — rate-limit responses
- [503 Service Unavailable](https://scrappey.com/qa/http-errors/what-is-a-503-error) — overload / challenge responses
---
# Browser Actions
Source: https://docs.scrappey.com/docs/guides/browser-actions
> Automate interactions with web pages by specifying a sequence of typed actions in the `browserActions` array of your request payload.
## Command
Browser actions are passed as the `browserActions` array on any `request.get` or `request.post` call.
## Parameters
All actions share a common set of top-level properties.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `type` | string | Yes | The action type (see Action Types below). |
| `when` | string | No | Lifecycle phase to run this action. Defaults to `"afterload"`. |
| `ignoreErrors` | boolean | No | If `true`, errors in this action will not stop execution. |
| `timeout` | number | No | Timeout in milliseconds. Default `60000`. |
## Execution Lifecycle
Browser actions run at specific points during the request lifecycle. Use the `when` property to control when each action executes. If `when` is omitted it defaults to `"afterload"`.
### Lifecycle Phases (in order)
```
1. beforelaunch — Before browser navigates to the URL
2. beforeload — After browser is ready, before the page loads
3. afterload — After the page has loaded (default)
4. [antibot detection & captcha solving happens here]
5. after_captcha — After all antibot/captcha solving is complete
```
### Phase Details
| Phase | When it runs | Use case |
| -------------- | ------------ | -------- |
| `beforelaunch` | Before any navigation. No page context yet. | Setting up browser state, pre-flight actions |
| `beforeload` | After cookies are added, before GET/POST navigation | Pre-navigation setup, adding localStorage |
| `afterload` | After page navigation completes, before antibot detection | Main page interactions — filling forms, clicking buttons, solving captchas, extracting data |
| `after_captcha` | After all antibot detection (Cloudflare, DataDome, etc.) and captcha solving (auto + CaptchaListener) complete | Post-solve actions — clicking buttons that appear after a captcha, submitting forms that were blocked |
Each action in the `browserActions` array is tagged with a `when` phase. The engine loops through the array multiple times — once per phase — and only runs actions whose `when` matches the current phase. Actions within the same phase execute in order.
When actions are nested inside `if.then`, `if.or`, or `while.then`, the `when` property of nested actions is ignored — they run immediately when the parent action executes. Only top-level actions in the `browserActions` array are filtered by `when`.
## Action Types
### `click` - Click an Element
Clicks on an element using CSS selector.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `cssSelector` | string | Yes | CSS selector of element to click. |
| `wait` | number | No | Wait time in seconds after clicking. |
| `waitForSelector` | string | No | Wait for this selector to appear after click. |
| `direct` | boolean | No | Use direct click instead of cursor simulation. |
**Example:**
```json
{
"type": "click",
"cssSelector": "#submit-button",
"wait": 1,
"waitForSelector": ".success-message"
}
```
### `type` - Type Text into Input
Types text into an input field with human-like delays.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `cssSelector` | string | Yes | CSS selector of input element. |
| `text` | string | Yes | Text to type. |
| `wait` | number | No | Wait time in seconds after typing. |
| `direct` | boolean | No | Use direct typing instead of cursor simulation. |
**Example:**
```json
{
"type": "type",
"cssSelector": "#username",
"text": "myusername"
}
```
### `goto` - Navigate to URL
Navigates the browser to a new URL.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `url` | string | Yes | URL to navigate to. |
| `wait` | number | No | Wait time in seconds after navigation. |
**Example:**
```json
{
"type": "goto",
"url": "https://example.com/page2"
}
```
### `wait` - Wait for Duration
Pauses execution for a specified time.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `wait` | number | Yes | Wait time in seconds. |
**Example:**
```json
{
"type": "wait",
"wait": 2
}
```
### `wait_for_selector` - Wait for Element
Waits for an element to appear in the DOM.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `cssSelector` | string | Yes | CSS selector to wait for. |
| `timeout` | number | No | Timeout in ms. Default `60000`. |
**Example:**
```json
{
"type": "wait_for_selector",
"cssSelector": ".loaded-content",
"timeout": 30000
}
```
### `wait_for_function` - Wait for JavaScript Condition
Waits until a JavaScript expression returns truthy.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `code` | string | Yes | JavaScript code that returns truthy when done. |
| `timeout` | number | No | Timeout in ms. Default `60000`. |
**Example:**
```json
{
"type": "wait_for_function",
"code": "window.dataLoaded === true",
"timeout": 30000
}
```
### `wait_for_load_state` - Wait for Page State
Waits for a specific page load state.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `waitForLoadState` | string | Yes | `"domcontentloaded"`, `"networkidle"`, or `"load"`. |
**Example:**
```json
{
"type": "wait_for_load_state",
"waitForLoadState": "networkidle"
}
```
### `wait_for_cookie` - Wait for Cookie
Waits for a specific cookie to be set.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `cookieName` | string | Yes | Name of the cookie to wait for. |
| `cookieValue` | string | No | Expected value of the cookie. |
| `cookieDomain` | string | No | Domain the cookie should be set on. |
| `pollIntervalMs` | number | No | Poll interval. Default `200` ms. |
| `timeout` | number | No | Timeout in ms. Default `60000`. |
**Example:**
```json
{
"type": "wait_for_cookie",
"cookieName": "session_id",
"timeout": 30000
}
```
### `execute_js` - Execute JavaScript
Executes JavaScript code on the page. Results are stored in the `javascriptReturn` array.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `code` | string | Yes | JavaScript code to execute. |
| `dontReturnValue` | boolean | No | If `true`, the return value is not captured. |
**Example:**
```json
{
"type": "execute_js",
"code": "document.querySelector('.price').innerText"
}
```
Access results in subsequent actions via `{javascriptReturn[0]}`.
### `scroll` - Scroll Page/Element
Scrolls to an element or the bottom of the page.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `cssSelector` | string | No | Element to scroll to. If omitted, scrolls to bottom. |
| `repeat` | number | No | Number of times to repeat the scroll. |
| `delayMs` | number | No | Delay between scrolls. Default `100` ms. |
**Example:**
```json
{
"type": "scroll",
"cssSelector": "#footer",
"repeat": 3,
"delayMs": 500
}
```
### `hover` - Hover Over Element
Hovers the mouse over an element.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `cssSelector` | string | Yes | CSS selector of element to hover. |
| `timeout` | number | No | Timeout in ms. |
**Example:**
```json
{
"type": "hover",
"cssSelector": ".dropdown-trigger"
}
```
### `keyboard` - Press Keyboard Key
Simulates keyboard key presses.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `value` | string | Yes | Key to press (see supported values below). |
| `cssSelector` | string | No | Element to focus first (required for `clear`). |
| `wait` | number | No | Wait time in seconds after pressing. |
| `waitForSelector` | string | No | Wait for selector after pressing. |
**Supported values:** `tab`, `enter`, `space`, `arrowdown`, `arrowup`, `arrowleft`, `arrowright`, `backspace`, `clear`
**Example:**
```json
{
"type": "keyboard",
"value": "enter"
}
```
### `dropdown` - Select Dropdown Option
Selects an option from a dropdown/select element.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `cssSelector` | string | Yes | CSS selector of select element. |
| `index` | number | No | Option index to select. |
| `value` | string | No | Option value to select. |
| `wait` | number | No | Wait time in seconds after selection. |
| `waitForSelector` | string | No | Wait for selector after selection. |
Either `index` or `value` is required.
**Example:**
```json
{
"type": "dropdown",
"cssSelector": "#country-select",
"value": "US"
}
```
### `switch_iframe` - Switch to iFrame
Switches context to an iframe for subsequent actions.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `cssSelector` | string | Yes | CSS selector of the iframe. |
**Example:**
```json
{
"type": "switch_iframe",
"cssSelector": "#payment-iframe"
}
```
### `set_viewport` - Set Browser Viewport
Changes the browser viewport size.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `width` | number | No | Viewport width. Default `1280`. |
| `height` | number | No | Viewport height. Default `1024`. |
| `wait` | number | No | Wait time in seconds after setting. |
**Example:**
```json
{
"type": "set_viewport",
"width": 1920,
"height": 1080
}
```
### `if` - Conditional Execution
Executes actions conditionally based on a JavaScript condition.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `condition` | string | Yes | JavaScript condition to evaluate. |
| `then` | array | No | Actions to run if condition is true. |
| `or` | array | No | Actions to run if condition is false. |
**Example:**
```json
{
"type": "if",
"condition": "document.querySelector('.captcha') !== null",
"then": [
{ "type": "solve_captcha", "captcha": "turnstile" }
],
"or": [
{ "type": "click", "cssSelector": "#continue" }
]
}
```
### `while` - Loop Execution
Loops actions while a condition is true.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `condition` | string | Yes | JavaScript condition to evaluate. |
| `then` | array | Yes | Actions to run each iteration. |
| `maxAttempts` | number | No | Maximum iterations (prevents infinite loops). |
**Example:**
```json
{
"type": "while",
"condition": "document.querySelector('.load-more') !== null",
"then": [
{ "type": "click", "cssSelector": ".load-more" },
{ "type": "wait", "wait": 1 }
],
"maxAttempts": 10
}
```
### `solve_captcha` - Solve Captcha
Solves various captcha types.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `captcha` | string | Yes | Captcha type (see list below). |
| `captchaData` | object | No | Additional captcha configuration. |
| `websiteUrl` | string | No | Website URL (for some captcha types). |
| `websiteKey` | string | No | Site key (for some captcha types). |
| `cssSelector` | string | No | Captcha element selector. |
| `inputSelector` | string | No | Input field for captcha answer. |
| `clickSelector` | string | No | Submit button to click after solving. |
| `iframeSelector` | string | No | iFrame containing the captcha. |
| `coreName` | string | No | Core name for custom captcha. |
**Captcha Types:**
- `turnstile` - Cloudflare Turnstile
- `recaptcha` - Google reCAPTCHA v2
- `recaptchav2` - reCAPTCHA v2 with sitekey
- `recaptchav3` - reCAPTCHA v3
- `hcaptcha` - hCaptcha
- `hcaptcha_inside` - hCaptcha with sitekey
- `hcaptcha_enterprise_inside` - hCaptcha Enterprise
- `funcaptcha` - FunCaptcha/Arkose Labs
- `perimeterx` - PerimeterX
- `mtcaptcha` - MTCaptcha
- `mtcaptchaisolated` - MTCaptcha isolated
- `v4guard` - v4Guard captcha
- `custom` - Custom image captcha
- `fingerprintjscom` - FingerprintJS
- `fingerprintjs_curseforge` - FingerprintJS CurseForge
**captchaData options:**
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `sitekey` | string | No | Captcha site key. |
| `action` | string | No | reCAPTCHA action. |
| `pageAction` | string | No | reCAPTCHA v3 page action. |
| `invisible` | boolean | No | Invisible captcha flag. |
| `base64Image` | string | No | Base64 image for custom captcha. |
| `cssSelector` | string | No | Turnstile container selector. |
| `reset` | boolean | No | Reset captcha state before solving. |
| `fast` | boolean | No | Use fast solving mode. |
**Example:**
```json
{
"type": "solve_captcha",
"captcha": "turnstile",
"captchaData": {
"sitekey": "0x4AAAAAAA...",
"cssSelector": ".cf-turnstile"
}
}
```
### `discord_login` - Discord Token Login
Logs into Discord using a token.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `token` | string | Yes | Discord auth token. |
| `direct` | boolean | No | Skip navigation to the login page. |
| `wait` | number | No | Wait time in ms after login. |
| `timeout` | number | No | Timeout for login. Default `60000`. |
**Example:**
```json
{
"type": "discord_login",
"token": "your_discord_token"
}
```
### `remove_iframes` - Remove All iFrames
Removes all iframes from the page (experimental). No additional properties required.
**Example:**
```json
{
"type": "remove_iframes"
}
```
### `click_and_hold` - Click and Hold
*Note: This action is not yet implemented.*
## Example
### Using JavaScript Return Values
Results from `execute_js` are stored in the `javascriptReturn` array and can be referenced in subsequent actions:
```json
{
"browserActions": [
{
"type": "execute_js",
"code": "document.querySelector('.token').dataset.value"
},
{
"type": "type",
"cssSelector": "#input",
"text": "{javascriptReturn[0]}"
}
]
}
```
### Full Login Flow
```json
{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "wait_for_selector",
"cssSelector": "#login-form",
"when": "afterload"
},
{
"type": "type",
"cssSelector": "#username",
"text": "myuser"
},
{
"type": "type",
"cssSelector": "#password",
"text": "mypassword"
},
{
"type": "solve_captcha",
"captcha": "turnstile"
},
{
"type": "click",
"cssSelector": "#submit",
"waitForSelector": ".dashboard"
},
{
"type": "execute_js",
"code": "document.querySelector('.user-data').innerText"
}
]
}
```
## Notes
### FastPeopleSearch bot-check example
```json
{
"browserActions": [
{
"type": "if",
"condition": "document.querySelector('#bot-check')",
"when": "after_captcha",
"then": [
{
"type": "click",
"cssSelector": "button[class='btn btn-success']"
}
]
}
]
}
```
This action waits for all antibot solving to finish (Cloudflare challenge, captcha polling, etc.), then checks if the page shows a "strange activity from your IP address" bot-check message, and if found clicks the green success button to proceed. Without `"when": "after_captcha"`, this would run during `afterload` — before Cloudflare is solved — when the bot-check element does not exist yet.
### XtremeTop100 vote submission example
```json
{
"browserActions": [
{ "type": "solve_captcha", "captcha": "turnstile" },
{ "type": "click", "cssSelector": "[name='ticki']", "when": "after_captcha" },
{ "type": "wait", "wait": 3, "when": "after_captcha" }
]
}
```
The turnstile captcha is solved during `afterload` (default). The vote button click and wait only run `after_captcha`, ensuring the captcha token is ready before form submission.
[All the browser actions](https://www.scrappey.com/scrapers/BrowserActions)
---