# Scrappey Documentation
> Full documentation for the Scrappey web data API. Source: https://docs.scrappey.com/docs
---
# Welcome to Scrappey
Source: https://docs.scrappey.com/docs/getting-started/welcome
> Scrappey.com is an authorized browser-workflow API. It manages browser sessions, proxy configuration, and browser-based rendering, returning HTML, headers, cookies, screenshots, and structured data so you can focus on building your application.
**Scrappey.com** runs your requests through managed, headless browser sessions and returns the rendered result over a simple HTTP API. It handles the operational work of launching browsers, managing cookies and [sessions](/docs/sessions-create), and routing through configurable [proxies](/docs/premium-proxy), so you can build reliable data pipelines without maintaining browser infrastructure yourself.
## Your First Request
Every request is a `POST` to the API endpoint with your API key in the query string. The simplest call fetches a page through a managed browser session:
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com"
}'
```
The rendered HTML, cookies, headers, and status code are returned in the `solution` object. See [request.get](/docs/request-get) for the full list of options.
## Why Choose Scrappey.com?
**Managed Browser API:** Scrappey.com runs each request through a managed, headless browser session with configurable proxies, returning fully rendered HTML and structured data. It supports sites with modern browser requirements — JavaScript rendering, dynamic content, and interactive page flows — so your pipelines stay reliable.
**Headless Browsing:** Access pages that require authentication or element interaction. Scrappey.com lets you script interactive browser workflows — navigation, clicks, typing, waits, and form submission — through the [`browserActions` API](/docs/browser-actions), so multi-step flows such as login or paginated content are straightforward to automate.
**Flexible Pricing:** Transparent, flexible plans for individuals, startups, and enterprises. Concurrent requests, premium proxies, JavaScript rendering, and more — billed by usage, cancel anytime.
**Stay Updated:** Check out our [Scrappey Blog](https://wiki.scrappey.com/blog) for the latest articles about web scraping, guides, tutorials, and industry insights.
## Report Abuse
### Our Position and Report Abuse
At Scrappey.com, we are committed to providing services that prioritize the well-being and interests of our users. We strictly prohibit any activities that infringe upon the rights of others, such as cybercrime, cracking, and fraudulent actions. Our primary goal is to assist start-up companies by offering powerful AI algorithms for SEO, AD verification, benign crawlers, and other business growth scenarios. By leveraging our services, we aim to help these companies save their budget and achieve their objectives.
If you believe that someone has unlawfully used our services to infringe upon your interests, we urge you to reach out to us immediately. Once we confirm the validity of the case, we will promptly terminate the services provided to the offender and provide you with any available information to aid in recovering your losses. Our team will do everything possible to assist you throughout this process.
We have a strong aversion to cybercrime, and we stand firmly against it. If you represent an anti-bot company and have noticed fraudulent activities associated with one of our users, we take this matter seriously. We encourage you to contact us promptly with your concerns. Should we verify the legitimacy of your claim, the offending business will be swiftly removed from our system and barred from any further interactions.
### Contact Us
If you need to report abuse or have any inquiries, please don't hesitate to reach out to us through the following channels:
**Email:** [scrappeycom@gmail.com](mailto:scrappeycom@gmail.com)
We appreciate your cooperation in maintaining a safe and ethical environment for all users of our services.
---
# Getting Started
Source: https://docs.scrappey.com/docs/getting-started/introduction
> Scrappey.com is an authorized browser-workflow proxy. It runs requests through managed browser sessions and returns rendered HTML and cookies, so you can work with the pages your application needs — including dynamic websites that require JavaScript rendering.
## What is Scrappey
Scrappey.com makes it straightforward to collect web data through managed browser sessions. It acts as a proxy server: you send a request, and it returns the rendered HTML and the cookies set during the session. Those cookies can be reused with other HTTP clients to maintain session continuity with the same site. You can also script multi-step browser interactions through the [`browserActions` API](/docs/browser-actions). It's well suited to workflows that demand high reliability and repeatability — QA, uptime and content monitoring, indexing your own content, ad verification, SEO analytics, and internal automation. Use it only where permitted and in compliance with each site's terms and applicable laws.
## Features
Scrappey is a browser-automation engine that provides complete, consistent browser environments, helping you reliably access sites that require a real browser to render. Here's how it works:
Scrappey runs a full browser environment with realistic, configurable settings:
1. It manages browser configuration — device type, browser, language, time zone, screen size, and related properties — so pages render as they would in a standard desktop or mobile browser.
2. It supports realistic interaction primitives such as mouse movement, scrolling, and typing, so interactive pages and dynamically loaded content render and behave correctly.
Scrappey provides a standards-compliant browser environment:
1. It runs your automation as a standard browser session, executing JavaScript and loading assets like a normal browser.
2. It gives you control over browser configuration and privacy-related settings (WebGL, WebRTC, canvas, fonts, and more).
3. It keeps browser settings internally consistent, which improves rendering consistency across sites.
Scrappey aligns the browser environment with your selected proxy region:
1. It sets a consistent language, locale, and time zone to match the proxy.
2. It matches browser settings to your configured proxy region, so regional testing and localized content render correctly.
Additional capabilities:
1. Generate consistent, unique device profiles that persist across requests (see [Profiles](/docs/profiles))
2. Run full browser environments on headless servers
3. Control WebGL, image loading, WebRTC, and other browser-level settings
In short, Scrappey helps you collect data, run QA and monitoring against your own or authorized sites, and automate browser tasks — with sessions that behave like an ordinary browser environment. Use it only where permitted and in line with applicable terms and laws.
## Setup and Installation
1. Sign up at [Scrappey](https://www.scrappey.com).
2. Obtain your unique API key from your Scrappey dashboard.
3. Integrate Scrappey into your projects via simple HTTP requests or SDKs provided.
## Sending Requests
Sending requests through Scrappey.com is a straightforward process. You simply need to send a request to an endpoint, providing the URL, headers (optional), cookies (optional), and proxy (optional). Once you initiate the request, Scrappey handles browser execution, [session state](/docs/sessions-create), [proxy routing](/docs/premium-proxy), and response collection.
To authenticate your requests, you'll need an API key, which you can obtain by visiting the [Dashboard](https://app.scrappey.com/#/). This API key ensures that your requests are properly authorized and processed.
Not sure what to do? Check out our [Request Builder](https://app.scrappey.com/#/builder).
## Example Request
```javascript
// Example endpoint
const endpoint = "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY";
// Available request types: "request.get", "request.post", "request.put", "request.delete", "request.patch", "request.publish"
const requestData = {
cmd: "request.get",
url: "https://example.com"
};
```
## Example CURL
```bash
# Or CURL example
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"cmd":"request.get","url":"https://example.com"}'
```
---
# Profiles
Source: https://docs.scrappey.com/docs/api-reference/profiles
> Attach a `profileId` to any request to maintain a consistent browser configuration, cookies, proxy, and session data across all future requests using that profile.
## Overview
Persistent profiles solve a common challenge: maintaining a stable, reproducible browser environment across requests. Each profile stores:
- **Browser configuration** - Canvas, WebGL, fonts, screen size, user agent, and 50+ other browser properties
- **Cookies** - All cookies are automatically saved and restored
- **Session data** - localStorage and other session information
- **Proxy & IP** - The proxy and IP address used with this profile (automatically remembered)
Once created, a profile's configuration never changes. This gives you a stable, reproducible browser environment every time you use that profile — useful for consistent QA runs, monitoring, and session continuity. Unlike a [`sessions.create`](/docs/sessions-create) session, which expires after ~200 seconds, a profile persists indefinitely.
## Parameters
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `profileId` | string | Yes | Unique identifier for the profile. Use any string (alphanumeric, hyphens, underscores). |
| `proxy` | string | No | Your proxy URL. If provided, it is saved to the profile and used for all future requests. |
| `forceNewProxy` | boolean | No | When `true`, ignores the stored proxy and assigns a fresh one. The new proxy replaces the old one. |
## Request
Add `profileId` to your request to use a persistent profile:
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"profileId": "my-profile-123"
}'
```
If the profile does not exist it is automatically created with a unique browser configuration. All subsequent requests with the same `profileId` will reuse that exact configuration.
### First Request (Profile Creation)
```json
{
"cmd": "request.get",
"url": "https://browserleaks.com/canvas",
"profileId": "account-store-001"
}
```
- A new browser configuration is generated and permanently saved
- A proxy is automatically assigned and saved to the profile
- The browser launches with this configuration and proxy
- Any cookies set by the website are saved when the session ends
### Subsequent Requests (Profile Reuse)
```json
{
"cmd": "request.get",
"url": "https://example.com/account",
"profileId": "account-store-001"
}
```
- The saved configuration is loaded (100% identical to first request)
- The saved proxy is automatically used (same IP address)
- Saved cookies are injected into the browser
- The site sees the same consistent browser environment as before
## Use Cases
### Account Management
Maintain separate, isolated browser environments for different accounts:
```json
{
"cmd": "request.get",
"url": "https://example.com",
"profileId": "email-account-1"
}
```
Give each account its own `profileId` (e.g. `email-account-2`) so their configurations, cookies, and proxies stay fully isolated.
### Session Persistence
Stay logged in across requests without re-authenticating:
```json
{
"cmd": "request.get",
"url": "https://mystore.example.com/admin",
"profileId": "my-store"
}
```
Reuse the same `profileId` on follow-up requests (e.g. `https://mystore.example.com/admin/orders`) to keep the session alive.
### Consistent Environment
Present a stable, returning-visitor environment for sites that expect session continuity:
```json
{
"cmd": "request.get",
"url": "https://site-with-fingerprinting.com",
"profileId": "research-browser"
}
```
## Best Practices
### Profile Naming
Use descriptive, consistent naming:
```
account-{platform}-{identifier} → account-store-john
session-{purpose}-{id} → session-scraping-batch42
```
### One Profile Per Identity
Each unique identity should have its own profile.
### Profile Isolation
Two users with the same `profileId` get completely separate profiles and data.
## Configuration Consistency
Every property of the browser environment remains identical across sessions, which makes runs reproducible:
| Component | Consistency |
|-----------|-------------|
| Canvas fingerprint | 100% identical |
| WebGL renderer/vendor | 100% identical |
| Audio fingerprint | 100% identical |
| Font list | 100% identical |
| Screen dimensions | 100% identical |
| User agent | 100% identical |
| Platform/OS | 100% identical |
| Timezone | 100% identical |
| Language | 100% identical |
## Proxy Persistence
Profiles automatically remember the proxy used during their session, so a profile maintains a consistent IP location across requests. This provides session continuity — the same browser environment and the same regional proxy every time.
### With Your Own Proxy
```json
{
"cmd": "request.get",
"url": "https://example.com",
"profileId": "my-profile",
"proxy": "http://user:pass@proxy.example.com:8000"
}
```
- Your proxy is saved to the profile
- All future requests automatically use this proxy
- If the proxy stops working, you'll get an error asking you to provide a new one
### Without a Proxy (Automatic Assignment)
```json
{
"cmd": "request.get",
"url": "https://example.com",
"profileId": "my-profile"
}
```
- A proxy is automatically assigned and saved to the profile
- All future requests use the same proxy (same IP)
- If the proxy stops working, a new one is automatically assigned
### Subsequent Requests
Once a profile has a proxy saved, just use the profile — no proxy needed:
```json
{
"cmd": "request.get",
"url": "https://example.com",
"profileId": "my-profile"
}
```
The saved proxy is automatically used.
### Proxy Use Cases
#### Consistent IP + Environment
```json
{
"cmd": "request.get",
"url": "https://example.com",
"profileId": "account-1"
}
```
Repeat with the same `profileId` to reuse the same IP and browser environment on every request.
#### Stable Sessions
```json
{
"cmd": "request.get",
"url": "https://example.com",
"profileId": "email-main"
}
```
#### Using Your Own Proxy
```json
{
"cmd": "request.get",
"url": "https://target.com",
"profileId": "scraper-1",
"proxy": "http://user:pass@your-proxy.com:8000"
}
```
Later requests with the same `profileId` (e.g. `https://target.com/page1`) reuse the saved proxy automatically — no need to send it again.
#### Update the Proxy
```json
{
"cmd": "request.get",
"url": "https://example.com",
"profileId": "my-profile",
"proxy": "http://new-proxy:8000"
}
```
#### Force a New Proxy
```json
{
"cmd": "request.get",
"url": "https://example.com",
"profileId": "my-profile",
"forceNewProxy": true
}
```
### Why Use Proxy Persistence?
| Benefit | Description |
|---------|-------------|
| IP Consistency | Maintains the same browser environment and regional IP across requests |
| Simpler Requests | Proxy is automatically handled — no need to manage it yourself |
| Stable Sessions | Avoids session resets caused by IP changes mid-workflow |
| Automatic Recovery | If an internal proxy fails, a new one is automatically assigned |
## Limitations
- Profiles are permanent once created — the browser configuration cannot be changed
- Maximum profile storage depends on your plan
- Very large cookie stores (>10MB) may impact performance
## FAQ
**Q: What happens if I don't specify a profileId?**
A: A temporary browser with a randomly generated configuration is used. Nothing is persisted.
**Q: Can I delete a profile?**
A: Yes, contact support or use the profile management API.
**Q: How long are profiles stored?**
A: Indefinitely, unless you delete them.
**Q: Do profiles work with proxies?**
A: Yes. Proxies are automatically saved and reused with profiles. The browser configuration stays the same regardless of which proxy you use.
**Q: Will a site recognize the same profile across requests?**
A: Yes — a profile provides a consistent browser environment, so the site sees a stable returning visitor rather than a brand-new environment on every request. This is the intended behavior for maintaining sessions.
**Q: What happens if my proxy stops working?**
A: If you provided your own proxy, you'll get an error asking you to provide a new one. If you're using an automatically assigned proxy, a new one is automatically assigned.
**Q: Can I use a profile without any proxy?**
A: Profiles always use a proxy to ensure consistent IP addresses. If you don't provide one, one is automatically assigned.
**Q: How do I change the proxy for an existing profile?**
A: Use `forceNewProxy: true` to assign a new proxy, or provide your own `proxy` URL. The new proxy replaces the stored one.
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Stateful web scraping](https://scrappey.com/qa/web-scraping-apis/what-is-stateful-web-scraping) — reusing state across requests
- [Browser fingerprinting](https://scrappey.com/qa/web-scraping-apis/what-is-browser-fingerprinting) — the client signature a profile pins
- [Browser environment consistency](https://scrappey.com/qa/anti-bot/what-is-browser-fingerprinting-evasion) — why stable profiles matter for repeatable browser workflows
---
# Making Requests
Source: https://docs.scrappey.com/docs/api-reference/requests
> Send a POST request to the Scrappey API with a URL, optional headers, cookies, and proxy, and the platform manages browser execution, rendering, session state, and response collection automatically.
## Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | ----------- |
| `cmd` | string | Yes | The command to execute, e.g. [`request.get`](/docs/request-get), [`request.post`](/docs/request-post), or one of the [other request commands](/docs/request-others). |
| `url` | string | Yes | The target URL to request. |
| `headers` | object | No | Custom request headers to send. |
| `cookies` | array | No | Cookies to include with the request. |
| `proxy` | string | No | [Proxy](/docs/premium-proxy) to route the request through. |
## Request
Authenticate by including your API key in the query string. Send the command and target URL as a JSON body.
```javascript
const response = await fetch('https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
cmd: 'request.get',
url: 'https://example.com'
})
});
const data = await response.json();
console.log(data.solution.response);
```
## Response
A successful response returns `solution.verified: true` along with the page content, cookies, headers, and IP info.
```json
{
"solution": {
"verified": true,
"currentUrl": "https://httpbin.rs/get",
"statusCode": 200,
"userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:135.0) Gecko/20100101 Firefox/135.0",
"innerText": "{\"body_string\":\"\",\"headers\":{\"accept\":\"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\",\"accept-encoding\":\"gzip, br\",\"accept-language\":\"en-US,en;q=0.5\",\"cache-control\":\"no-cache\",\"cdn-loop\":\"cloudflare; loops=1\",\"cf-connecting-ip\":\"161.0.248.139\",\"cf-ipcountry\":\"TT\",\"cf-ray\":\"947fb8313e8fd6db-IAD\",\"cf-visitor\":\"{\\\"scheme\\\":\\\"https\\\"}\",\"host\":\"httpbin.rs\",\"pragma\":\"no-cache\",\"priority\":\"u=0, i\",\"sec-fetch-dest\":\"document\",\"sec-fetch-mode\":\"navigate\",\"sec-fetch-site\":\"none\",\"sec-fetch-user\":\"?1\",\"upgrade-insecure-requests\":\"1\",\"user-agent\":\"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:135.0) Gecko/20100101 Firefox/135.0\",\"x-forwarded-for\":\"161.0.248.139, 172.71.194.68\",\"x-forwarded-host\":\"httpbin.rs\",\"x-forwarded-port\":\"80\",\"x-forwarded-proto\":\"https\",\"x-forwarded-server\":\"ab65c85a31ae\",\"x-is-trusted\":\"yes\",\"x-real-ip\":\"161.0.248.139\"},\"json\":null,\"method\":\"GET\",\"origin\":\"161.0.248.139\",\"query\":null,\"uri\":\"/get\"}",
"cookies": [],
"cookieString": "",
"response": "
",
"responseHeaders": {
"date": "Fri, 30 May 2025 16:48:31 GMT",
"content-type": "application/json; charset=utf-8",
"cf-ray": "947fb8313e8fd6db-IAD",
"server": "cloudflare",
"content-encoding": "zstd",
"cf-cache-status": "DYNAMIC",
"report-to": "{\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report\\/v4?s=vjiHLteAY5BudLpu8FBSR7H1zrEVyMaB3p05NXTLkkFdkMwe2nW7pH2mmR2H1YZX6SnueOYkFzZ%2F6vNoSp9vHy5WiHmX%2FnYJ2SJ%2B3I7eebUZYEefUHJTlkRsbjMD\"}], \"group\":\"cf-nel\", \"max_age\":604800}",
"nel": "{\"success_fraction\":0, \"report_to\":\"cf-nel\", \"max_age\":604800}",
"alt-svc": "h3=\":443\"; ma=86400",
"server-timing": "cfCacheStatus;desc=\"DYNAMIC\", cfL4;desc=\"?proto=TCP&rtt=87643&min_rtt=82738&rtt_var=21002&sent=7&recv=9&lost=0&retrans=0&sent_bytes=3978&recv_bytes=2399&delivery_rate=52499&cwnd=238&unsent_bytes=0&cid=76cb71c0b81a2239&ts=388&x=0\"",
"x-firefox-spdy": "h2"
},
"requestHeaders": {
"host": "httpbin.rs",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:135.0) Gecko/20100101 Firefox/135.0",
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"accept-language": "en-US,en;q=0.5",
"accept-encoding": "gzip, deflate, br, zstd",
"upgrade-insecure-requests": "1",
"sec-fetch-dest": "document",
"sec-fetch-mode": "navigate",
"sec-fetch-site": "none",
"sec-fetch-user": "?1",
"connection": "keep-alive",
"priority": "u=0, i",
"pragma": "no-cache",
"cache-control": "no-cache"
},
"ipInfo": {
"status": "success",
"country": "Trinidad and Tobago",
"countryCode": "TT",
"region": "CHA",
"regionName": "Chaguanas",
"city": "Chaguanas",
"zip": "",
"lat": 10.5167,
"lon": -61.4167,
"timezone": "America/Port_of_Spain",
"isp": "Columbus Communications Trinidad Limited.",
"org": "Columbus Communications Trinidad Limited",
"as": "AS27665 Columbus Communications Trinidad Limited.",
"mobile": false,
"proxy": false,
"hosting": false,
"query": "161.0.248.139"
},
"method": "GET",
"type": "browser"
},
"timeElapsed": 10269,
"data": "success",
"session": "06c9de79-a61b-4eae-9121-11814440cc07"
}
```
## Error Response
An error returns a status code 200 with `solution.verified` set to `false`. This request is not charged. The response always includes `"data": "error"` and an `"error"` field explaining the failure reason.
```json
{
"solution": {
"verified": false
},
"timeElapsed": 4405,
"data": "error",
"error": "Verification provider rejected session. Review proxy configuration and authorization for the target site.",
"info": "https://wiki.scrappey.com/getting-started#db336d2e7b3e42429966b01e2efb7d18",
"session": "4f8904bb-aac3-40d1-939e-55a00d3acafd"
}
```
---
# Screenshot
Source: https://docs.scrappey.com/docs/api-reference/screenshot
> Capture a full-page browser screenshot and receive it as a base64-encoded JPEG in the response, or have it uploaded to a public URL.
## Parameters
| Parameter | Type | Required | Description |
| ------------------ | ------- | -------- | ---------------------------------------------------------------------------- |
| `screenshot` | boolean | Yes | Enables full-page screenshot capture and returns base64 in the response. |
| `screenshotUpload` | boolean | No | Also uploads the screenshot to DigitalOcean Spaces and returns a public URL. |
| `screenshotWidth` | number | No | Width of the viewport before capture (in pixels). Default: 1280. |
| `screenshotHeight` | number | No | Height of the viewport before capture (in pixels). Default: 1024. |
| `fullPage` | boolean | No | Captures the full scrollable page height. Default: false. |
| `abortOnDetection` | array | No | List of URL patterns to block before page load (e.g. cookie-banner scripts). |
| `abortOnPostRequest` | boolean | No | When true, only POST requests matching `abortOnDetection` are blocked. |
Notes:
* Full-page capture is always used regardless of the resolution you set.
* Screenshots are produced as JPEG with quality 80.
## Request
Full-page base64 only:
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"screenshot": true,
"screenshotWidth": 1440,
"screenshotHeight": 4000
}'
```
Full-page base64 + upload to Spaces:
```json
{
"cmd": "request.get",
"url": "https://example.com",
"screenshot": true,
"screenshotUpload": true,
"screenshotWidth": 1920,
"screenshotHeight": 1080
}
```
Blocking cookie banners for a clean screenshot:
```json
{
"cmd": "request.get",
"url": "https://example.com",
"screenshot": true,
"abortOnDetection": [
"cookiebot.com",
"onetrust.com",
"quantcast.com",
"trustarc.com",
"/consent",
"gdpr-consent",
"cookie-law-info"
]
}
```
If any network request URL contains one of the specified values, it will be aborted before loading.
## Response
| Field | Type | Description |
| ------------------------ | ------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `solution.screenshot` | string | Base64-encoded JPEG of the full page. |
| `solution.screenshotUrl` | string | Public URL in Spaces, present only when `screenshotUpload: true`. Filenames include a prefix with dimensions: `screenshots/{width}x{height}_{random}.jpg`. |
| `abortOnDetectionResponse` | array | List of blocked requests, each with URL, headers, and payload. Present when `abortOnDetection` is used. |
Response snippet when `screenshotUpload: true`:
```json
{
"solution": {
"screenshot": "",
"screenshotUrl": "https:///screenshots/1920x1080_abCDeFgHiJKLmNopQrSt.jpg"
}
}
```
## Blocking Cookie Banners
Many websites show cookie or GDPR consent banners that can cover content and appear in screenshots. Use `abortOnDetection` to block the network requests these banners depend on before the page loads.
The browser intercepts all network requests while loading the page. If a request URL matches a pattern you specify, that request is blocked. Cookie banners often fail to load when their consent or tracking scripts are blocked, preventing the banner from ever appearing.
Common providers whose URLs you may want to block:
* Cookiebot
* OneTrust
* Quantcast
* TrustArc
* Cookie Law Info
* Custom GDPR or consent endpoints
Their request URLs often contain keywords like: `cookie`, `consent`, `gdpr`, `privacy`.
By default, all matching requests are blocked. Set `abortOnPostRequest: true` to restrict blocking to POST requests only.
After the page loads, blocked requests are available in `abortOnDetectionResponse`. Each entry includes the request URL, headers, and payload (if present) — useful for debugging or fine-tuning which patterns you block.
## Tips
* Base64 payloads can be large. If response size is a concern, prefer `screenshotUpload: true`.
* Choose `screenshotWidth` / `screenshotHeight` to influence layout before capture.
* JPEG is used to optimize size.
* Blocking cookie banners improves screenshot consistency across sites.
* For a full session recording instead of a single still, use [Record Video](/docs/record-video).
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Headless browser](https://scrappey.com/qa/web-scraping-apis/what-is-a-headless-browser) — what renders the page you capture
- [Headless browsers](https://scrappey.com/qa/web-automation/headless-browsers-guide) — driving a real browser to scrape
---
# request.post
Source: https://docs.scrappey.com/docs/api-reference/requests/request-post
> Sends a POST request using a real browser session, with all the power of request.get plus support for sending request bodies.
## Parameters
This endpoint accepts all the same parameters as [`request.get`](/docs/request-get), with one key additional parameter:
| Parameter | Type | Required | Description |
| ---------- | ------ | -------- | ----------- |
| `postData` | string | Yes | The body of the POST request. Must be a string in `application/x-www-form-urlencoded` format. Example: `a=b&c=d`. To send JSON data, set the appropriate content type using `customHeaders`: `"customHeaders": { "content-type": "application/json" }` |
### Shared Parameters
Refer to [`request.get`](/docs/request-get) for detailed descriptions of shared parameters like:
* `session`
* `cookies` / `cookiejar`
* `proxy` / `proxyCountry`
* `customHeaders`
* `includeImages`
* `includeLinks`
* `requestType`
* `localStorage`
## Request
### Send a Form-Encoded Body
The default. Pass `postData` as an `application/x-www-form-urlencoded` string:
```json
{
"cmd": "request.post",
"url": "https://httpbin.rs/post",
"postData": "a=b"
}
```
### Send a JSON Body
To send JSON, pass `postData` as an object and set the `content-type` header (see the [JSON POST request guide](/docs/json-post-request) for a worked example):
```json
{
"cmd": "request.post",
"url": "https://httpbin.org/post",
"postData": {
"a": "b"
},
"customHeaders": {
"content-type": "application/json"
}
}
```
## Response
```json
{
"solution": {
"verified": true,
"currentUrl": "https://httpbin.rs/post",
"statusCode": 200,
"userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:135.0) Gecko/20100101 Firefox/135.0",
"innerText": "{\"body_string\":\"a=b\",\"headers\":{ ... },\"json\":null,\"method\":\"POST\",\"origin\":\"223.181.110.217\",\"query\":null,\"uri\":\"/post\"}",
"cookies": [],
"cookieString": "",
"response": "...
...
",
"responseHeaders": {
"date": "Fri, 30 May 2025 16:58:07 GMT",
"content-type": "application/json; charset=utf-8",
"cf-ray": "947fc6437bdfe237-MRS",
"server": "cloudflare",
"content-encoding": "zstd",
"access-control-allow-origin": "https://httpbin.rs",
"vary": "Origin",
"cf-cache-status": "DYNAMIC",
"report-to": "{\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report\\/v4?s=eUhoz6vNC8yrH4zwf%2FPbaolJQUZ11kO%2BbGKyYYhw1AMo6wJBn%2BobEe%2Fy5KD6Zbz7cn%2B18Yjflsajy40MsUbANQ7AqtDdlfGgkmbwHRUGcK1WpRIw3LueBTl7zEy%2F\"}], \"group\":\"cf-nel\", \"max_age\":604800}",
"nel": "{\"success_fraction\":0, \"report_to\":\"cf-nel\", \"max_age\":604800}",
"alt-svc": "h3=\":443\"; ma=86400",
"server-timing": "cfCacheStatus;desc=\"DYNAMIC\", cfL4;desc=\"?proto=TCP&rtt=195244&min_rtt=136910&rtt_var=69448&sent=29&recv=20&lost=0&retrans=0&sent_bytes=11642&recv_bytes=3211&delivery_rate=60733&cwnd=256&unsent_bytes=0&cid=a544805ada7c1328&ts=4901&x=0\"",
"x-firefox-spdy": "h2"
},
"requestHeaders": {
"host": "httpbin.rs",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:135.0) Gecko/20100101 Firefox/135.0",
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"accept-language": "en-US,en;q=0.5",
"accept-encoding": "gzip, deflate, br, zstd",
"content-type": "application/x-www-form-urlencoded",
"content-length": "3",
"referer": "https://httpbin.rs/",
"origin": "https://httpbin.rs",
"upgrade-insecure-requests": "1",
"sec-fetch-dest": "document",
"sec-fetch-mode": "navigate",
"sec-fetch-site": "same-origin",
"sec-fetch-user": "?1",
"connection": "keep-alive"
},
"requestBody": "a=b",
"method": "POST",
"type": "browser"
},
"timeElapsed": 46276,
"data": "success",
"session": "67142d3a-40dd-4eda-8619-f95603f3bed7"
}
```
---
# request.get
Source: https://docs.scrappey.com/docs/api-reference/requests/request-get
> Make a GET request through a real browser session with full support for cookies, proxies, local storage, and more.
**Session Timeout:** A session will automatically expire **200 seconds** after the last request.
## Parameters
| Parameter | Type | Required | Description |
| --------------- | ------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `url` | string | Yes | The target URL to fetch. |
| `session` | string | No | Use an existing browser session. If not provided, a temporary instance is created and destroyed after the request. |
| `cookiejar` | array | No | Accepts an array of cookie objects to be set before visiting the page. Example: `[{ "name": "cookie1", "value": "value1", "domain": "domain.com", "path": "/" }]` |
| `cookies` | string | No | Raw cookie string to be injected before page load. Example: `_ga=GA1.1.143964650.1682153807; _fbp=fb.1.1711313596391.220282865` |
| `proxy` | string | No | Use a custom proxy. Supports `http://`, `socks4://`, or `socks5://`. Authentication is supported. Example: `http://username:password@127.0.0.1:8888`. Ignored if `session` is set — use session-specific proxy in [`sessions.create`](/docs/sessions-create) instead. |
| `proxyCountry` | string | No | Select a specific country when using the built-in rotating proxies. Example: `"UnitedStates"` |
| `customHeaders` | object | No | Override default browser headers. Example: `{ "auth": "value" }` |
| `includeImages` | boolean | No | Set to `true` to return a list of all image URLs found on the page. |
| `includeLinks` | boolean | No | Set to `true` to return a list of all hyperlinks found on the page. |
| `requestType` | string | No | Choose between `"browser"` (default full browser) and `"request"` (faster, no rendering). Automatically sets appropriate headers unless `customHeaders` is used. |
| `localStorage` | object | No | Set key-value pairs in the browser's localStorage before loading the page. Example: `{ "ac": "dsjfh84hfdsfk", "onboarding": "false" }` |
### Important Note
If you're using a **verification clearance cookie**, make sure the **User-Agent** used in your headers **exactly matches** the one used during clearance. Mismatched headers will trigger a verification page.
## Advanced Options
These optional parameters control the output format, browser configuration, and session behavior. They work on `request.get`, [`request.post`](/docs/request-post), and the [other request commands](/docs/request-others).
| Parameter | Type | Required | Description |
| ------------------------ | ---------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `markdown` | boolean | No | Return the page converted to Markdown in `solution.markdown`. Respects `includeImages` / `includeLinks`. Combine with `filter: ["markdown"]` to return only the Markdown. |
| `pdf` | boolean | No | Render the loaded page to a PDF. The PDF is returned base64-encoded in the response. |
| `pageUrl` | string | No | Page-first fetch: load this URL first to establish a real session and cookies, then fetch the target `url` from within that browser context. Useful when an API endpoint depends on cookies or state established by a normal page visit. |
| `build` | string \| array | No | Select the browser build: `ff144`, `ff146`, `cl146`, `ff147`, `ff148`, `ff149`, or `ff150`. Pass an array (e.g. `["ff148", "ff150"]`) to select from available browser builds for compatibility testing. Defaults to the newest build. |
| `forceUniqueFingerprint` | boolean | No | Use a fresh browser configuration for this request. Useful for compatibility testing and isolated runs. Uses more resources. |
| `webrtcIpv4` | string | No | IPv4 address to align WebRTC network settings with when `forceUniqueFingerprint` is enabled. Defaults to the proxy's IPv4. |
| `webrtcIpv6` | string | No | IPv6 address to align WebRTC network settings with when `forceUniqueFingerprint` is enabled. Defaults to an IPv6 derived from the proxy, if available. |
| `captchaMethod` | string | No | Interactive challenge handling mode when `automaticallySolveCaptchas` is enabled. `"token"` (default) uses the integrated service; `"click"` handles the step interactively within the browser. |
| `overwriteLocale` | string | No | Force `navigator.language`, `navigator.languages`, and the `Accept-Language` header to this locale (e.g. `"en-US"`) instead of auto-detecting it from the proxy IP's country. |
### Return the page as Markdown
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"markdown": true,
"filter": ["markdown"]
}'
```
### Return the page as a PDF
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"pdf": true
}'
```
### Page-first fetch (session warm-up)
Load a normal page to acquire cookies/session, then fetch a protected API endpoint from inside the same browser:
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com/api/data",
"pageUrl": "https://example.com"
}'
```
### Pin or randomize the browser build
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"build": ["ff148", "ff149", "ff150"]
}'
```
### Force a unique fingerprint
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"forceUniqueFingerprint": true
}'
```
### Force a specific locale
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"proxyCountry": "Germany",
"overwriteLocale": "en-US"
}'
```
## Request
```json
{
"cmd": "request.get",
"url": "https://httpbin.org/get"
}
```
## Response
```json
{
"solution": {
"verified": true,
"currentUrl": "https://httpbin.rs/get",
"statusCode": 200,
"userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:135.0) Gecko/20100101 Firefox/135.0",
"innerText": "{\"body_string\":\"\",\"headers\":{\"accept\":\"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\",\"accept-encoding\":\"gzip, br\",\"accept-language\":\"en-US,en;q=0.5\",\"cache-control\":\"no-cache\",\"cdn-loop\":\"cloudflare; loops=1\",\"cf-connecting-ip\":\"161.0.248.139\",\"cf-ipcountry\":\"TT\",\"cf-ray\":\"947fb8313e8fd6db-IAD\",\"cf-visitor\":\"{\\\"scheme\\\":\\\"https\\\"}\",\"host\":\"httpbin.rs\",\"pragma\":\"no-cache\",\"priority\":\"u=0, i\",\"sec-fetch-dest\":\"document\",\"sec-fetch-mode\":\"navigate\",\"sec-fetch-site\":\"none\",\"sec-fetch-user\":\"?1\",\"upgrade-insecure-requests\":\"1\",\"user-agent\":\"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:135.0) Gecko/20100101 Firefox/135.0\",\"x-forwarded-for\":\"161.0.248.139, 172.71.194.68\",\"x-forwarded-host\":\"httpbin.rs\",\"x-forwarded-port\":\"80\",\"x-forwarded-proto\":\"https\",\"x-forwarded-server\":\"ab65c85a31ae\",\"x-is-trusted\":\"yes\",\"x-real-ip\":\"161.0.248.139\"},\"json\":null,\"method\":\"GET\",\"origin\":\"161.0.248.139\",\"query\":null,\"uri\":\"/get\"}",
"cookies": [],
"cookieString": "",
"response": "
",
"responseHeaders": {
"date": "Fri, 30 May 2025 16:48:31 GMT",
"content-type": "application/json; charset=utf-8",
"cf-ray": "947fb8313e8fd6db-IAD",
"server": "cloudflare",
"content-encoding": "zstd",
"cf-cache-status": "DYNAMIC",
"report-to": "{\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report\\/v4?s=vjiHLteAY5BudLpu8FBSR7H1zrEVyMaB3p05NXTLkkFdkMwe2nW7pH2mmR2H1YZX6SnueOYkFzZ%2F6vNoSp9vHy5WiHmX%2FnYJ2SJ%2B3I7eebUZYEefUHJTlkRsbjMD\"}], \"group\":\"cf-nel\", \"max_age\":604800}",
"nel": "{\"success_fraction\":0, \"report_to\":\"cf-nel\", \"max_age\":604800}",
"alt-svc": "h3=\":443\"; ma=86400",
"server-timing": "cfCacheStatus;desc=\"DYNAMIC\", cfL4;desc=\"?proto=TCP&rtt=87643&min_rtt=82738&rtt_var=21002&sent=7&recv=9&lost=0&retrans=0&sent_bytes=3978&recv_bytes=2399&delivery_rate=52499&cwnd=238&unsent_bytes=0&cid=76cb71c0b81a2239&ts=388&x=0\"",
"x-firefox-spdy": "h2"
},
"requestHeaders": {
"host": "httpbin.rs",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:135.0) Gecko/20100101 Firefox/135.0",
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"accept-language": "en-US,en;q=0.5",
"accept-encoding": "gzip, deflate, br, zstd",
"upgrade-insecure-requests": "1",
"sec-fetch-dest": "document",
"sec-fetch-mode": "navigate",
"sec-fetch-site": "none",
"sec-fetch-user": "?1",
"connection": "keep-alive",
"priority": "u=0, i",
"pragma": "no-cache",
"cache-control": "no-cache"
},
"ipInfo": {
"status": "success",
"country": "Trinidad and Tobago",
"countryCode": "TT",
"region": "CHA",
"regionName": "Chaguanas",
"city": "Chaguanas",
"zip": "",
"lat": 10.5167,
"lon": -61.4167,
"timezone": "America/Port_of_Spain",
"isp": "Columbus Communications Trinidad Limited.",
"org": "Columbus Communications Trinidad Limited",
"as": "AS27665 Columbus Communications Trinidad Limited.",
"mobile": false,
"proxy": false,
"hosting": false,
"query": "161.0.248.139"
},
"method": "GET",
"type": "browser"
},
"timeElapsed": 10269,
"data": "success",
"session": "06c9de79-a61b-4eae-9121-11814440cc07"
}
```
---
# sessions.destroy
Source: https://docs.scrappey.com/docs/api-reference/sessions/sessions-destroy
> Cleanly shuts down a browser session and frees up system resources.
Destroy the session you started with [`sessions.create`](/docs/sessions-create) as soon as you're done, instead of waiting for the 200-second idle timeout to reclaim it.
## Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | ------------------------------------------ |
| `session` | string | Yes | The ID of the session you want to destroy. |
## Request
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "sessions.destroy",
"session": "my-session-id"
}'
```
## Response
```json
{
"solution": {},
"data": "success",
"timeElapsed": 14,
"session": "my-session-id"
}
```
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Stateful web scraping](https://scrappey.com/qa/web-scraping-apis/what-is-stateful-web-scraping) — reusing cookies and state across requests
---
# sessions.create
Source: https://docs.scrappey.com/docs/api-reference/sessions/sessions-create
> Start a persistent browser session that retains cookies, speeds up requests, and reduces repeated setup work across multi-step browser workflows.
A session remains active for **200 seconds** after the last request. After that, it is automatically destroyed — or end it early with [`sessions.destroy`](/docs/sessions-destroy). For state that must persist beyond a single session, use a [persistent profile](/docs/profiles) instead.
## Parameters
| Parameter | Type | Required | Description |
| --- | --- | --- | --- |
| `session` | string | No | Custom session ID. If not set, a random UUID is generated. |
| `proxy` | string | No | Use a custom proxy. Supports `http://`, `socks4://`, or `socks5://` with authentication. Example: `"http://username:password@127.0.0.1:8888"`. Default: built-in rotating proxies from random countries — pin one with [`proxyCountry`](/docs/proxy-country). |
| `whitelistedDomains` | array | No | Allow requests to specified external domains. Example: `["probot.io", "google.com"]`. Useful for loading assets from third-party domains. |
| `datacenter` | boolean | No | Set to `true` to use a [datacenter IP](/docs/datacenter-proxy) instead of a residential IP. Datacenter IPs offer higher speed and stability, but lower anonymity. |
| `browser` | array | No | Define the browser type and version range. Options: `chrome`, `firefox`, `safari`. Example: `[{ "name": "chrome", "minVersion": 116, "maxVersion": 117 }]`. |
| `operatingSystem` | array | No | Specify OS options. Options: `windows`, `macos`, `linux`, `android`, `ios`. Example: `["windows", "linux"]`. |
| `device` | array | No | Choose device type. Options: `desktop`, `mobile`. Example: `["desktop", "mobile"]`. |
## Why Use sessions.create
- **Speed:** Reuses the same browser instance across multiple requests.
- **Persistence:** Retains cookies and session data until explicitly destroyed.
- **Convenience:** Reuses cookies and browser state across requests.
## Request
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "sessions.create",
"session": "my-session-id"
}'
```
## Response
```json
{
"solution": {
"verified": true
},
"data": "success",
"session": "my-session-id",
"fingerprint": {},
"context": {
"_type": "BrowserContext",
"_guid": "browser-context@2356ec33a1fa91f08e3b3da7b55b13f6"
}
}
```
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Stateful web scraping](https://scrappey.com/qa/web-scraping-apis/what-is-stateful-web-scraping) — reusing cookies and state across requests
- [Session cookie](https://scrappey.com/qa/anti-bot/what-is-a-session-cookie) — the cookie that ties a session together
---
# proxyCountry
Source: https://docs.scrappey.com/docs/api-reference/requests/request-get/proxy-country
> Use any of the following values to specify a country for Scrappey's rotating proxies.
## Example
Pass `proxyCountry` alongside any request to route it through an IP in that country. It works with [premium](/docs/premium-proxy), [datacenter](/docs/datacenter-proxy), and [mobile](/docs/mobile-proxy) proxies. Values are case-sensitive and must match the table below exactly (e.g. `UnitedStates`, not `united states`).
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"proxyCountry": "UnitedStates"
}'
```
## Supported Countries
| Country |
|---------|
| Afghanistan |
| Albania |
| Algeria |
| Argentina |
| Armenia |
| Aruba |
| Australia |
| Austria |
| Azerbaijan |
| Bahamas |
| Bahrain |
| Bangladesh |
| Belarus |
| Belgium |
| BosniaandHerzegovina |
| Brazil |
| BritishVirginIslands |
| Brunei |
| Bulgaria |
| Cambodia |
| Cameroon |
| Canada |
| Chile |
| China |
| Colombia |
| CostaRica |
| Croatia |
| Cuba |
| Cyprus |
| Czechia |
| Denmark |
| DominicanRepublic |
| Ecuador |
| Egypt |
| ElSalvador |
| Estonia |
| Ethiopia |
| Finland |
| France |
| Georgia |
| Germany |
| Ghana |
| Greece |
| Guatemala |
| Guyana |
| HashemiteKingdomofJordan |
| HongKong |
| Hungary |
| India |
| Indonesia |
| Iran |
| Iraq |
| Ireland |
| Israel |
| Italy |
| Jamaica |
| Japan |
| Kazakhstan |
| Kenya |
| Kosovo |
| Kuwait |
| Latvia |
| Liechtenstein |
| Luxembourg |
| Macedonia |
| Madagascar |
| Malaysia |
| Mauritius |
| Mexico |
| Mongolia |
| Montenegro |
| Morocco |
| Mozambique |
| Myanmar |
| Nepal |
| Netherlands |
| NewZealand |
| Nigeria |
| Norway |
| Oman |
| Pakistan |
| Palestine |
| Panama |
| PapuaNewGuinea |
| Paraguay |
| Peru |
| Philippines |
| Poland |
| Portugal |
| PuertoRico |
| Qatar |
| RepublicofLithuania |
| RepublicofMoldova |
| Romania |
| Russia |
| SaudiArabia |
| Senegal |
| Serbia |
| Seychelles |
| Singapore |
| Slovakia |
| Slovenia |
| Somalia |
| SouthAfrica |
| SouthKorea |
| Spain |
| SriLanka |
| Sudan |
| Suriname |
| Sweden |
| Switzerland |
| Syria |
| Taiwan |
| Tajikistan |
| Thailand |
| TrinidadandTobago |
| Tunisia |
| Turkey |
| Uganda |
| Ukraine |
| UnitedArabEmirates |
| UnitedKingdom |
| UnitedStates |
| Uzbekistan |
| Venezuela |
| Vietnam |
| Zambia |
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Residential proxy](https://scrappey.com/qa/proxies/what-is-a-residential-proxy) — the IP type used for geo-targeting
- [Residential vs datacenter](https://scrappey.com/qa/web-automation/proxy-types-comparison) — which proxy type to choose
- [Proxy web scraping](https://scrappey.com/qa/proxies/what-is-proxy-web-scraping) — how proxies route your requests
---
# request.[others]
Source: https://docs.scrappey.com/docs/api-reference/requests/request-others
> These commands let you send PUT, DELETE, PATCH, and PUBLISH HTTP requests through a real browser session, sharing the same structure as request.post.
## Parameters
These commands share the exact same structure and options as [`request.post`](/docs/request-post), including:
* All shared parameters from [`request.get`](/docs/request-get)
* Support for `postData` (required)
* Support for `customHeaders`, including setting the appropriate `Content-Type`
| Parameter | Type | Required | Description |
| ---------- | ------ | -------- | ----------- |
| `postData` | string | Yes | The request body to send. Use string format (`application/x-www-form-urlencoded`) or set the content type to `application/json` via `customHeaders`. |
## Example
### request.put
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.put",
"url": "https://example.com/api/resource/1",
"postData": "field=value"
}'
```
### request.delete
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.delete",
"url": "https://example.com/api/resource/1"
}'
```
### request.patch (JSON body)
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.patch",
"url": "https://example.com/api/resource/1",
"postData": { "field": "new_value" },
"customHeaders": { "content-type": "application/json" }
}'
```
## Notes
For all other options, refer to:
* [`request.get`](/docs/request-get) for general parameters
* [`request.post`](/docs/request-post) for body-related behavior
---
# Remaining balance
Source: https://docs.scrappey.com/docs/api-reference/remaining-balance
> Check how many requests remain in your Scrappey account balance with a single GET request.
## Request
```bash
GET https://publisher.scrappey.com/api/v1/balance?key=YOUR_API_KEY
```
Replace `YOUR_API_KEY` with your actual API key, which can be found in your [Scrappey Dashboard](https://publisher.scrappey.com).
## Response
```json
{
"balance": 1547
}
```
The value represents the number of **requests** you have left in your current balance. Each successful [`request.get`](/docs/request-get) or other command draws it down; see [Concurrency limits](/docs/concurrency-limits) for how many you can run in parallel.
---
# Session Management
Source: https://docs.scrappey.com/docs/api-reference/sessions
> Create and destroy persistent browser sessions that retain cookies and browser state, reducing repeated setup and overhead for subsequent requests.
## sessions.create
This launches a new browser instance that retains cookies until you destroy it with [`sessions.destroy`](/docs/sessions-destroy). Use sessions when you want to reuse cookies and browser state across requests and skip the overhead of spinning up a new browser each time. For the full parameter list (proxy, browser, device, OS), see [`sessions.create`](/docs/sessions-create); for state that persists across many requests, see [Profiles](/docs/profiles).
### Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | --------------------------------------------------- |
| `cmd` | string | Yes | Must be `sessions.create`. |
| `session` | string | Yes | A unique identifier for the browser session to create. |
### Request
```json
{
"cmd": "sessions.create",
"session": "my-session-id"
}
```
## sessions.destroy
This properly shuts down a browser instance. When you no longer need a session, close it to free resources.
### Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | ----------------------------------------------------- |
| `cmd` | string | Yes | Must be `sessions.destroy`. |
| `session` | string | Yes | The identifier of the browser session to destroy. |
### Request
```json
{
"cmd": "sessions.destroy",
"session": "my-session-id"
}
```
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Stateful web scraping](https://scrappey.com/qa/web-scraping-apis/what-is-stateful-web-scraping) — reusing cookies and state across requests
- [Session cookie](https://scrappey.com/qa/anti-bot/what-is-a-session-cookie) — the cookie that ties a session together
---
# Concurrency limits
Source: https://docs.scrappey.com/docs/api-reference/concurrency-limits
> Scrappey gives new accounts up to 200 concurrent threads by default and scales automatically up to 1000, with higher limits available on request.
## How Many Concurrent Threads Can I Use?
Scrappey allows all **new registered users** to start with **up to 200 concurrent threads** by default.
This is more than enough for the majority of use cases and ensures smooth performance and fair resource distribution.
If your project requires more, Scrappey makes it easy to scale.
## Scaling Beyond 200 Threads
We automatically allow **increases up to 1000 concurrent threads** without manual review, provided:
- Your **overall success rate is above 50%**.
- You are following our general API guidelines.
- You are not abusing endpoints.
## Special Cases / Higher Limits
If you require **more than 1000 concurrent threads**, this is no problem — simply reach out to support with a short explanation of your use case.
We generally approve higher limits quickly for:
- High-throughput data pipelines
- Large-scale monitoring, indexing, and QA workloads
- Long-running automation workflows that require sustained concurrency
There is **no hard upper limit** as long as your use case fits within reasonable resource bounds and success rates remain healthy.
## Quick Summary
| User Type | Threads Allowed | Requirements |
|-----------------|-----------------|------------------------|
| New Account | Up to 200 | None |
| Active Account | Up to 1000 | > 50% success rate |
| Special Request | > 1000 | Case-by-case approval |
## Tips for Scaling Smoothly
- **Monitor your success rate** regularly in the dashboard.
- Increase threads gradually to avoid sudden overload.
- If you encounter issues, check [error codes](/docs/error-codes) in the docs first.
## Example
Concurrency is not a request parameter — it's simply how many requests you have **in flight at the same time**. Each simultaneous request consumes one thread. Fire requests in parallel from your own code and Scrappey processes them concurrently up to your account limit:
```bash
# Fire 5 requests in parallel — each one uses one concurrent thread
for url in \
"https://example.com/page1" \
"https://example.com/page2" \
"https://example.com/page3" \
"https://example.com/page4" \
"https://example.com/page5"
do
curl -s -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d "{\"cmd\": \"request.get\", \"url\": \"$url\"}" &
done
wait
```
If you exceed your allotted threads, additional requests queue until a thread frees up. Keep your concurrent count at or below your limit to avoid delays.
---
# Wrappers
Source: https://docs.scrappey.com/docs/guides/wrappers
> Use the official Python or Node.js wrapper to interact with the Scrappey API without writing raw HTTP calls.
Scrappey offers official wrappers for both **Python** and **Node.js**, enabling developers to interact seamlessly with the Scrappey API. These wrappers simplify [session management](/docs/sessions-create), [request execution](/docs/request-get), browser workflows, and response parsing. ([GitHub](https://github.com/pim97))
## Python Wrapper
* **GitHub**: [pim97/scrappey-wrapper-python](https://github.com/pim97/scrappey-wrapper-python)
* **PyPI**: [scrappeycom](https://pypi.org/project/scrappeycom/)
### Installation
```bash
pip install scrappeycom
```
### Usage Example
```python
from scrappeycom.scrappey import Scrappey
import uuid
api_key = 'YOUR_API_KEY'
scrappey = Scrappey(api_key)
session_data = {
'session': str(uuid.uuid4())
}
# Create a session
session = scrappey.create_session(session_data)
print('Session created:', session['session'])
# Make a GET request
response = scrappey.get({
'session': session['session'],
'url': 'https://httpbin.org/get'
})
print('GET Response:', response)
# Destroy the session
scrappey.destroy_session(session_data)
```
### Features
* Manages proxy configuration and response parsing
* Simplifies session creation and destruction
* Supports GET and POST requests with custom headers
* Provides error handling for common request issues
## Node.js Wrapper
* **GitHub**: [DemonMartin/scrappey-wrapper](https://github.com/DemonMartin/scrappey-wrapper)
* **NPM**: [scrappey](https://www.npmjs.com/package/scrappey)
### Installation
```bash
npm install scrappey
```
### Usage Example
```javascript
const Scrappey = require('scrappey');
const apiKey = 'YOUR_API_KEY';
const scrappey = new Scrappey(apiKey);
(async () => {
// Create a session
const sessionData = await scrappey.createSession();
const session = sessionData.session;
console.log('Session created:', session);
// Make a GET request
const response = await scrappey.getRequest({
url: 'https://httpbin.org/get',
session
});
console.log('GET Response:', response);
// Destroy the session
await scrappey.destroySession(session);
})();
```
### Features
* Manages proxy configuration and response parsing
* Manages sessions efficiently
* Supports both form data and JSON in POST requests
* Allows custom headers for requests
## Notes
For more detailed information and advanced usage, refer to the respective GitHub repositories:
* [scrappey-wrapper-python](https://github.com/pim97/scrappey-wrapper-python)
* [scrappey-wrapper](https://github.com/DemonMartin/scrappey-wrapper)
These wrappers are designed to streamline your authorized browser workflows using Scrappey's API.
---
# Wait For Load State
Source: https://docs.scrappey.com/docs/guides/browser-actions/wait-for-load-state
> Pause the browser's execution until a specific load state is reached, ensuring actions and data extraction occur only after the page is ready.
## Command
`"wait_for_load_state"`
One of the [browser actions](/docs/browser-actions) — use [`wait_for_selector`](/docs/wait-for-selector) to wait for a specific element, or [`wait_for_function`](/docs/wait-for-function) to wait on a JavaScript condition.
## Parameters
| Parameter | Type | Required | Description |
| ------------------ | ------ | -------- | ------------------------------------------------------------------------------ |
| `waitForLoadState` | string | Yes | The load state to wait for. Options: `domcontentloaded`, `networkidle`, `load` |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "wait_for_load_state",
"waitForLoadState": "networkidle"
}
]
}'
```
## Notes
| State | Description |
| ------------------ | -------------------------------------------------------------- |
| `domcontentloaded` | Fired when the DOM is fully loaded (before styles/images/etc). |
| `networkidle` | Waits until there are no network requests for 500 ms. |
| `load` | Waits for the full load event including images and resources. |
---
# Direct Request
Source: https://docs.scrappey.com/docs/guides/browser-actions/direct-http-request
> A lightweight HTTP GET request with consistent browser-compatible TLS settings — useful when full rendering is unnecessary.
## Command
`"cmd": "request.get"`, `"requestType": "request"`
## Cost
**0.2 credit(s)** per successful request.
## Description
This is a raw HTTP `GET` request that **does not launch a browser**. It uses a consistent browser-compatible TLS configuration. It's useful for fast and cost-effective requests when browser-based rendering is unnecessary.
### Pros
* Very **fast** (avg. ~5 seconds)
* Very **cheap** (0.2 credits per request)
* Low **resource usage**
* Low **proxy consumption**
* Good for **simple websites** and **API endpoints**
### Cons
* Cannot interact with the browser
* Cannot handle **browser verification flows**
* Not suitable for **JavaScript-heavy** or **SPA** sites
* Not suitable for sites that require full browser rendering
## When to Use
Use `request.get` with `"requestType": "request"` when:
* The website doesn't require JavaScript rendering
* Speed and cost are more important than complexity
* You don't need to interact with the page (no clicking, scrolling, etc.)
* You're accessing simple data, APIs, or static HTML pages
For dynamic websites that require JavaScript rendering or browser verification flows, consider switching to a full [browser-based `request.get`](/docs/browser-request) without `"requestType": "request"`.
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"requestType": "request",
"url": "https://example.com"
}'
```
```json
{
"cmd": "request.get",
"url": "https://httpbin.rs/get",
"requestType": "request"
}
```
## Response
```json
{
"solution": {
"verified": true,
"statusCode": 200,
"userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0",
"cookies": [],
"response": "{\"body_string\":\"\",\"headers\":{\"accept\":\"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\",\"accept-encoding\":\"gzip, br\",\"accept-language\":\"en-US\",\"cdn-loop\":\"cloudflare; loops=1\",\"cf-connecting-ip\":\"119.18.77.197\",\"cf-ipcountry\":\"KR\",\"cf-ray\":\"947fd38f3f33aa5f-ICN\",\"cf-visitor\":\"{\\\"scheme\\\":\\\"https\\\"}\",\"dnt\":\"1\",\"host\":\"httpbin.rs\",\"priority\":\"u=0, i\",\"sec-fetch-dest\":\"document\",\"sec-fetch-mode\":\"navigate\",\"sec-fetch-site\":\"same-site\",\"sec-fetch-user\":\"?1\",\"upgrade-insecure-requests\":\"1\",\"user-agent\":\"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0\",\"x-forwarded-for\":\"119.18.77.197, 172.71.110.86\",\"x-forwarded-host\":\"httpbin.rs\",\"x-forwarded-port\":\"80\",\"x-forwarded-proto\":\"https\",\"x-forwarded-server\":\"ab65c85a31ae\",\"x-is-trusted\":\"yes\",\"x-real-ip\":\"119.18.77.197\"},\"json\":null,\"method\":\"GET\",\"origin\":\"119.18.77.197\",\"query\":null,\"uri\":\"/get\"}",
"responseHeaders": {
"alt-svc": "h3=\":443\"; ma=86400",
"cf-cache-status": "DYNAMIC",
"cf-ray": "947fd38f3f33aa5f-ICN",
"content-encoding": "zstd",
"content-type": "application/json; charset=utf-8",
"date": "Fri, 30 May 2025 17:07:12 GMT",
"nel": "{\"success_fraction\":0,\"report_to\":\"cf-nel\",\"max_age\":604800}",
"report-to": "{\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report\\/v4?s=HlXrquVBA%2BxOKRRHKkaWp%2FycU5igGEA2wtg5FIADowjVuesxwuAvZeZP%2FLANqbbGdvFOdkeImdOJsUO%2FS4LRLdVhgaz22cfqa4a0519XznkoSFmBbMdAebMvtkBk\"}],\"group\":\"cf-nel\",\"max_age\":604800}",
"server": "cloudflare",
"server-timing": "cfCacheStatus;desc=\"DYNAMIC\", cfL4;desc=\"?proto=TCP&rtt=21231&min_rtt=14258&rtt_var=14940&sent=7&recv=8&lost=0&retrans=0&sent_bytes=3903&recv_bytes=2091&delivery_rate=307195&cwnd=236&unsent_bytes=0&cid=e52ebc3ecd344e1c&ts=823&x=0\""
},
"requestHeaders": {
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"accept-encoding": "gzip, deflate, br, zstd",
"accept-language": "en-US",
"dnt": "1",
"sec-fetch-dest": "document",
"sec-fetch-mode": "navigate",
"sec-fetch-site": "same-site",
"sec-fetch-user": "?1",
"te": "trailers",
"upgrade-insecure-requests": "1",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0"
},
"method": "GET",
"type": "request"
},
"timeElapsed": 5493,
"data": "success",
"session": "ba4ff7a7-2422-41b9-99fa-dd11a5928ad4"
}
```
---
# Browser request
Source: https://docs.scrappey.com/docs/guides/browser-actions/browser-request
> Execute a fully rendered browser session to render dynamic or JavaScript-heavy content with session reliability for authorized workflows.
## Command
`"cmd": "request.get"`
## Parameters
| Parameter | Type | Required | Description |
| ---------- | ------ | -------- | -------------------------------------------- |
| `cmd` | string | Yes | Must be `"request.get"` for browser GET requests. |
| `url` | string | Yes | The URL to request. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com"
}'
```
## Notes
**Cost:** 1 credit per successful request.
**Pros**
* Handles browser verification flows and JavaScript challenges within authorized workflows
* Renders dynamic content that a normal `GET` cannot produce
* Supports [browser actions](/docs/browser-actions) (e.g., clicking, scrolling, waiting)
* Works with single-page applications (SPAs)
**Cons**
* Slower execution (avg ~30 seconds)
* Higher cost than simple `GET` requests
* More resource-intensive
* Relies on proxy usage and higher infrastructure demand
**Also Works With**
* [`"cmd": "request.post"`](/docs/request-post) for browser-based POST requests
* Compatible with `browserActions`, [`autoparse`](/docs/autoparse), proxies, and custom headers
**Use Cases**
* Rendering JavaScript-heavy websites that require a full browser
* Interacting with form submissions or dynamic elements
* Dynamic websites that require JavaScript rendering or browser verification flows
This is the most flexible way to render content through Scrappey, ideal for authorized browser workflows. Respect third-party terms, robots.txt where applicable, rate limits, and applicable law.
---
# Json Post Request
Source: https://docs.scrappey.com/docs/guides/browser-actions/json-post-request
> Send structured JSON payloads using a [`POST` request](/docs/request-post) through a real browser session for interacting with modern APIs expecting JSON-formatted input.
## Command
`"cmd": "request.post"`
## Parameters
| Parameter | Type | Required | Description |
| --------------- | ------ | -------- | ----------------------------------------------------------- |
| `url` | string | Yes | The endpoint to send the POST request to. |
| `postData` | object | Yes | JSON object to be sent in the body of the request. |
| `customHeaders` | object | Yes | Set `content-type` to `application/json` for JSON requests. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.post",
"url": "https://httpbin.org/post",
"customHeaders": {
"content-type": "application/json"
},
"postData": {
"a": "b"
}
}'
```
## Notes
* Ensure that `content-type` is explicitly set to `application/json`.
* The `postData` value should be a valid JSON object.
* You can also include additional headers like authorization tokens or user-agents under `customHeaders`.
* For a form-encoded body instead of JSON, see the standard [POST request](/docs/post-request) guide.
### Use Cases
* Authenticating to RESTful APIs
* Submitting form data in JSON format
* Posting content to webhooks or services like Zapier, Slack, etc.
---
# Post Request
Source: https://docs.scrappey.com/docs/guides/browser-actions/post-request
> Send a POST request with form-encoded or JSON body to a target URL, useful for submitting forms, logging in, or triggering APIs.
## Command
`"cmd": "request.post"`
## Parameters
| Parameter | Type | Required | Description |
| --------------- | ------ | -------- | ------------------------------------------------------------------- |
| `url` | string | Yes | The URL to send the POST request to. |
| `postData` | string | Yes | Data to send. Must be `application/x-www-form-urlencoded`. |
| `customHeaders` | object | No | Set content type to `application/json` to send raw JSON. |
## Example
### Form Data
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.post",
"url": "https://httpbin.rs/post",
"postData": "a=b"
}'
```
### JSON Data
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.post",
"url": "https://httpbin.rs/post",
"postData": "{\"username\": \"john\", \"password\": \"1234\"}",
"customHeaders": {
"content-type": "application/json"
}
}'
```
## Notes
* `postData` must be a string.
* If using `application/json`, the JSON must be stringified.
* You can combine this with other [browser actions](/docs/browser-actions) like cookies, headers, or [click automation](/docs/click-action).
* To send a structured JSON body instead of form data, see the [JSON POST request](/docs/json-post-request) guide.
Use [`request.post`](/docs/request-post) when interacting with login forms, submitting data, or calling APIs that require POST payloads.
---
# Retry
Source: https://docs.scrappey.com/docs/guides/browser-actions/retry
> Automatically retry a failed request up to 5 times to improve session reliability against timeouts, verification flows, or network instability.
## Command
`"retries"`
**Purpose**: Retries the entire request if it fails.
**Limits**:
- Maximum retries: `5`
- Max total execution time: `240` seconds (4 minutes)
## Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | --------------------------------------------- |
| `retries` | number | No | Number of automatic retries (1–5 recommended) |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"retries": 3
}'
```
## Notes
- Retries are only performed if the request fails (e.g., timeout, blocked, crash).
- Each retry counts against your total scrape usage.
- If the combined retries exceed 240 seconds, the request will be aborted.
Use automatic retries for session reliability in unstable environments or on sites with browser verification flows, within authorized workflows. Pair it with [persistent sessions](/docs/sessions-create) and a [premium proxy](/docs/premium-proxy) for more resilient runs.
---
# Add Referer
Source: https://docs.scrappey.com/docs/guides/browser-actions/add-referer
> Set the navigation context by visiting a page first or adding a static Referer header before making a POST request.
## Command
[`"request.post"`](/docs/request-post) — use `getUrl` or `customHeaders` to supply the referer.
## Parameters
| Parameter | Type | Required | Description |
| --- | --- | --- | --- |
| `cmd` | string | Yes | Should be `"request.post"` |
| `url` | string | Yes | Target POST endpoint |
| `postData` | string | Yes | Data to send in the POST body (URL-encoded or JSON as required) |
| `getUrl` | string | No | Visit this URL first; it becomes the referer (most natural approach) |
| `customHeaders` | object | No | Add `"Referer"` directly via headers (when no referer validation is involved) |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.post",
"url": "https://lift-api.vfsglobal.com/user/login",
"getUrl": "https://vfsglobal.com",
"postData": "a=b&b=c"
}'
```
## Notes
- `getUrl` is preferred for authorized browser workflows, as it preserves the expected browser navigation context.
- `customHeaders` is simpler and faster if no referer validation is enforced.
- This approach preserves the expected browser navigation context for systems that expect a valid navigation path before a POST operation.
---
# Record Video
Source: https://docs.scrappey.com/docs/guides/browser-actions/record-video
> Enable session video recording to capture and replay browser behavior for debugging and traceability.
## Command
`"video"` — set the `video` parameter to `true` on any request to record the entire browser session. The response returns a link to download or view the recorded video.
## Parameters
| Parameter | Type | Required | Description |
| --------- | ------- | -------- | -------------------------------------------- |
| `video` | boolean | Yes | Set to `true` to record the browser session. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"video": true
}'
```
## Notes
Use video recording to:
- Debug failed scraping sessions visually.
- Review step-by-step browser automation.
- Present recorded demonstrations of scraping flows.
- Confirm interaction success or failures post-request.
This feature is highly valuable for debugging JavaScript-heavy sites, login automation, complex interactions, or when you want traceability over what the browser executed. For a single still frame instead of a full recording, use a [screenshot](/docs/screenshot).
---
# Mouse Movements
Source: https://docs.scrappey.com/docs/guides/browser-actions/mouse-movements
> Perform pointer movement actions during authorized browser workflows.
## Command
`"mouseMovements"`
## Parameters
| Parameter | Type | Required | Description |
| ---------------- | ------- | -------- | ------------------------------------------------- |
| `mouseMovements` | boolean | Yes | Set to `true` to enable pointer movement actions. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"mouseMovements": true
}'
```
## Notes
Enable this feature when:
- Performing authorized browser workflows that benefit from pointer movement actions.
- Clicking buttons, filling out forms, or interacting with content.
- Preserving expected browser navigation context during interaction.
This is useful for authorized browser workflows on login forms, checkout pages, or content behind interaction-based verification flows. The feature uses actual recorded patterns instead of fully random coordinates. Combine it with [click](/docs/click-action) and [scroll](/docs/scroll) [browser actions](/docs/browser-actions) for a complete interaction.
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Behavioural detection](https://scrappey.com/qa/anti-bot/what-is-behavioural-detection) — why pointer movement actions matter
- [Playwright](https://scrappey.com/qa/web-scraping-apis/what-is-playwright) — the automation engine behind browser actions
---
# Mobile Proxy
Source: https://docs.scrappey.com/docs/guides/browser-actions/mobile-proxy
> Route your requests through real mobile carrier IPs to preserve expected browser navigation context on websites that heavily track IP reputation and device behavior.
## Command
`"mobileProxy"`
## Parameters
| Parameter | Type | Required | Description |
| -------------- | ------- | -------- | ------------------------------------------------------------------- |
| `mobileProxy` | boolean | Yes | Set to `true` to use a mobile proxy. |
| `proxyCountry` | string | No | Country to route the mobile proxy through (e.g., `"UnitedStates"`). |
> **Tip**: View all available countries in the [Scrappey Builder](https://app.scrappey.com/#/builder) under **Advanced Options → Custom Proxy Country**.
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"mobileProxy": true,
"proxyCountry": "UnitedStates"
}'
```
## Notes
Use a mobile proxy for:
* Geo-accurate testing of your own properties and permitted region-specific content.
* Working within rate limits and verification flows.
* Routing authorized traffic through real mobile networks.
Use `mobileProxy` when authenticity and access success are more important than speed or cost. Cost is 7 credits per request. For lower-cost alternatives, consider a [premium residential proxy](/docs/premium-proxy) or a [datacenter proxy](/docs/datacenter-proxy); target a specific country with [`proxyCountry`](/docs/proxy-country).
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [What is a mobile proxy?](https://scrappey.com/qa/proxies/what-is-a-mobile-proxy) — what it is and when to use it
- [Residential proxy](https://scrappey.com/qa/proxies/what-is-a-residential-proxy) — the closest alternative
- [Proxy web scraping](https://scrappey.com/qa/proxies/what-is-proxy-web-scraping) — how proxies route your requests
- [ISP proxy](https://scrappey.com/qa/proxies/what-is-an-isp-proxy) — another high-trust option
---
# Datacenter Proxy
Source: https://docs.scrappey.com/docs/guides/browser-actions/datacenter-proxy
> Route the request through a datacenter proxy for speed-critical authorized workflows where residential IP reputation is not required.
## Command
`"datacenter"` — Set the `datacenter` parameter to `true` on any request to route it through a datacenter proxy. Cost: 1 credit per request. Optionally choose a specific country via [`proxyCountry`](/docs/proxy-country).
## Parameters
| Parameter | Type | Required | Description |
| -------------- | ------- | -------- | ------------------------------------------------------------ |
| `datacenter` | boolean | Yes | Set to `true` to enable datacenter proxy. |
| `proxyCountry` | string | No | Country to route the proxy through (e.g., `"UnitedStates"`). |
> **Note**: Full country list available at [Scrappey Builder](https://app.scrappey.com/#/builder) under Advanced Options → Custom Proxy Country.
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"datacenter": true,
"proxyCountry": "UnitedStates"
}'
```
## Notes
Best suited for:
* High-volume scraping with low latency requirements.
* Situations where IP reputation is not a major concern.
* Reducing cost while maintaining speed.
Use `datacenter` proxies when you need performance and cost-efficiency over maximum IP trust. For higher-trust residential IPs, use a [premium proxy](/docs/premium-proxy) instead, or a [mobile proxy](/docs/mobile-proxy) for the strictest sites.
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [What is a datacenter proxy?](https://scrappey.com/qa/proxies/what-is-a-datacenter-proxy) — what it is and its trade-offs
- [Residential vs datacenter](https://scrappey.com/qa/web-automation/proxy-types-comparison) — which proxy type to choose
- [Proxy web scraping](https://scrappey.com/qa/proxies/what-is-proxy-web-scraping) — how proxies route your requests
- [Rotating proxy](https://scrappey.com/qa/proxies/what-is-a-rotating-proxy) — cycling IPs across requests
---
# Premium Proxy
Source: https://docs.scrappey.com/docs/guides/browser-actions/premium-proxy
> Route requests through real residential IPs to handle region restrictions and verification flows during authorized browser workflows. Premium residential proxies are used by default, at a cost of 1 credit per request.
## Command
`"premiumProxy"`
## Parameters
| Parameter | Type | Required | Description |
| -------------- | ------- | -------- | ------------------------------------------------------------- |
| `premiumProxy` | boolean | No | Premium residential proxy. Used by default — set `datacenter: true` to route through a [datacenter IP](/docs/datacenter-proxy) instead. |
| `proxyCountry` | string | No | Country to route the proxy through (e.g., `"UnitedStates"`). |
> **Note**: Full country list available at [Scrappey Builder](https://app.scrappey.com/#/builder) under Advanced Options → Custom Proxy Country.
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"premiumProxy": true,
"proxyCountry": "UnitedStates"
}'
```
## Notes
Best suited for:
* Geo-targeted scraping (e.g., accessing US-specific content).
* Using real residential IPs for higher trust in authorized workflows.
* Reaching sites that present strict browser verification, on content you are permitted to access.
Use `premiumProxy` to improve session reliability on sites with strict browser verification by using regular residential traffic. Target a specific country with [`proxyCountry`](/docs/proxy-country), or switch to a [datacenter](/docs/datacenter-proxy) or [mobile](/docs/mobile-proxy) proxy for different speed/trust trade-offs.
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [What is a residential proxy?](https://scrappey.com/qa/proxies/what-is-a-residential-proxy) — what powers the premium pool
- [Residential vs datacenter](https://scrappey.com/qa/web-automation/proxy-types-comparison) — which proxy type to choose
- [Rotating proxy](https://scrappey.com/qa/proxies/what-is-a-rotating-proxy) — cycling IPs across requests
- [Proxy web scraping](https://scrappey.com/qa/proxies/what-is-proxy-web-scraping) — how proxies route your requests
---
# Wait For Url Load
Source: https://docs.scrappey.com/docs/guides/browser-actions/wait-for-url-load
> Pause browser execution until a specific URL substring is loaded, ensuring the page transition has completed before continuing with other browser actions.
## Command
`"waitForUrl"`
Uses `.includes()` to check if the current URL contains the given substring. This does not return the response body — it only ensures the page has transitioned.
## Parameters
| Parameter | Type | Required | Description |
| ------------ | ------ | -------- | ------------------------------------------------------------------------- |
| `url` | string | Yes | The starting URL to navigate to. |
| `waitForUrl` | string | Yes | Substring to wait for in the destination URL before continuing execution. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://www.mcmaster.com/products/screws/socket-head-screws~/alloy-steel-socket-head-screws-8/",
"waitForUrl": "/Content/Explanations.aspx?"
}'
```
## Notes
Best suited for:
* Websites that redirect or dynamically change the URL after certain actions.
* Ensuring the next page has loaded before executing further browser actions.
* Waiting for final navigation steps before data scraping or input.
Using `waitForUrl` guarantees your scraping workflow proceeds only after the target page has been fully navigated to. Pair it with [`goto`](/docs/go-to) for multi-page flows, or [`wait_for_selector`](/docs/wait-for-selector) to wait on a specific element.
---
# Intercept XHR Request
Source: https://docs.scrappey.com/docs/guides/browser-actions/intercept-xhr
> Use `interceptFetchRequest` to inspect permitted network responses generated by an authorized browser session by capturing specific `fetch()` or XHR requests during a page load.
## Command
`"interceptFetchRequest"` — monitor network activity and return specific XHR/fetch responses. URL matching uses `.includes()`; if more than one request matches the string, all are returned as an array.
## Parameters
| Parameter | Type | Required | Description |
| ----------------------- | ------ | -------- | ---------------------------------------------------------------------- |
| `interceptFetchRequest` | array | Yes | List of strings to match against outgoing request URLs |
| `url` | string | Yes | URL to be loaded and monitored |
| `browserActions` | array | No | [Browser actions](/docs/browser-actions) (e.g., verification flow handling) to perform during request |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com/dashboard?input=example.com",
"interceptFetchRequest": [
"https://example.com/api/v4/overview"
],
"browserActions": [
{
"type": "solve_captcha",
"captcha": "custom"
}
]
}'
```
You will receive an array of JSON responses from all fetch/XHR requests that matched the string(s) provided.
## Notes
Great for:
- Sites that dynamically load content via JavaScript
- Skipping complex DOM parsing by reading documented or permitted application endpoints
- Working with documented or permitted application endpoints
Use `interceptFetchRequest` to inspect permitted network responses generated by authorized browser sessions. Pair it with [`execute_js`](/docs/execute-javascript) to post-process the captured data.
---
# Deepseek Autoparse
Source: https://docs.scrappey.com/docs/guides/browser-actions/deepseek-autoparse
> Use the DeepSeek large language model to convert dynamic web content into structured JSON in a single request.
## Command
`"Deepseek AI Autoparse"` — costs **2 credits** per successful request.
- **Model**: `deepseek`
- **Autoparse**: Set `"autoparse": true`
- **Optional**: Supply `"structure"` and `"cssSelector"` to guide parsing.
## Parameters
| Parameter | Type | Required | Description |
| ------------- | ------- | -------- | --------------------------------------------------------------------------- |
| `model` | string | Yes | Must be `"deepseek"` |
| `autoparse` | boolean | Yes | Enable autoparsing |
| `cssSelector` | string | No | Specific region of the website to parse |
| `structure` | object | No | Structure of the desired output. If omitted, model infers one automatically |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://www.imdb.com/title/tt0087363/?ref_=ls_t_10",
"filter": [
"autoparse"
],
"model": "deepseek",
"autoparse": true,
"cssSelector": "section[class='ipc-page-background ipc-page-background--base sc-afa4bed1-0 iMxoKo']",
"structure": {
"movie_info": {
"title": "str",
"language": "str"
},
"duration": "str",
"release_date": "str",
"rating": {
"average_rating": "float",
"total_votes": "int"
},
"certification": "str",
"cast": [
{
"actor_name": "str",
"role": "str"
}
],
"crew": [
{
"crew_name": "str",
"role": "str"
}
]
}
}'
```
Example response:
```json
{
"solution": {
"verified": true,
"type": "browser",
"autoparse": {
"movie_info": {
"title": "Gremlins",
"language": "English"
},
"duration": "1h 46m",
"release_date": "June 8, 1984",
"rating": {
"average_rating": 7.3,
"total_votes": 259000
},
"certification": "PG",
"cast": [
{
"actor_name": "Zach Galligan",
"role": "Billy"
},
{
"actor_name": "Phoebe Cates",
"role": "Kate"
}
],
"crew": [
{
"crew_name": "Joe Dante",
"role": "Director"
},
{
"crew_name": "Chris Columbus",
"role": "Writer"
}
]
}
},
"timeElapsed": 41380,
"data": "success",
"session": "b0380e15-3381-413c-8cad-daefe574afb1"
}
```
## Notes
DeepSeek uses a Mixture of Experts (MoE) architecture for fast inference. It is open-source, multilingual, and performs well on coding, math, and structured parsing tasks. DeepSeek-V3 access is available for free.
This is an alternative model for the standard [AI Autoparse](/docs/autoparse) feature — both turn a page into structured JSON without selectors.
---
# Autoparse
Source: https://docs.scrappey.com/docs/guides/browser-actions/autoparse
> Automatically parse structured data from a web page using AI, with no need for exact selectors or knowledge of the HTML structure.
## Command
Enable autoparsing by setting `"autoparse": true` and providing a `structure` field that defines the JSON format you want the response to follow.
## Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | ----------- |
| `autoparse` | boolean | Yes | Set to `true` to enable AI-based autoparsing of the response. |
| `structure` | object | Yes | JSON object describing the data shape you want the AI to extract. |
## Example
```json
"structure": {
"movie_info": {
"title": "str",
"language": "str"
},
"duration": "str",
"release_date": "str",
"rating": {
"average_rating": "float",
"total_votes": "int"
},
"certification": "str",
"cast": [
{
"actor_name": "str",
"role": "str"
}
],
"crew": [
{
"crew_name": "str",
"role": "str"
}
]
}
```
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://movies.com/",
"autoparse": true,
"structure": {
"movie_info": {
"title": "str",
"language": "str"
},
"duration": "str",
"release_date": "str",
"rating": {
"average_rating": "float",
"total_votes": "int"
},
"certification": "str",
"cast": [
{
"actor_name": "str",
"role": "str"
}
],
"crew": [
{
"crew_name": "str",
"role": "str"
}
]
}
}'
```
## Notes
- Costs **6 credits** per successful request.
- Parses HTML into JSON automatically using AI.
- You define the desired data structure — the AI fills it in.
- Ideal for dynamic pages or unknown HTML structures.
- For a cheaper LLM-based alternative, see [DeepSeek Autoparse](/docs/deepseek-autoparse).
---
# Wait For Function
Source: https://docs.scrappey.com/docs/guides/browser-actions/wait-for-function
> Waits until a custom JavaScript expression evaluates to `true` inside the browser, useful for dynamic content or specific page states.
## Command
`"wait_for_function"`
One of the [browser actions](/docs/browser-actions) — use [`wait_for_selector`](/docs/wait-for-selector) to wait on the DOM, or [`wait_for_load_state`](/docs/wait-for-load-state) to wait on the page load lifecycle instead of a JavaScript expression.
## Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | ------------------------------------------------------------------- |
| `code` | string | Yes | JavaScript code to evaluate. Must return a truthy value to proceed. |
| `timeout` | number | No | Max time (ms) to wait for the condition (default: 60000 ms). |
## Example
#### Wait for element to appear
```javascript
document.querySelector('.dynamic-content') !== null
```
#### Wait for text and status change
```javascript
const element = document.querySelector('.status');
return element &&
element.textContent.includes('Ready') &&
!element.classList.contains('loading')
```
#### Example Request (cURL)
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "wait_for_function",
"code": "document.querySelector('.dynamic-content') !== null",
"timeout": 30000
},
{
"type": "wait_for_function",
"code": "const el = document.querySelector('.status'); return el && el.textContent.includes('Ready') && !el.classList.contains('loading');"
}
]
}'
```
## Notes
- The condition is evaluated in the **browser context**.
- It **polls every 200ms** until the condition is met.
- **Throws an error** if the timeout is reached.
- Any **JavaScript expression that evaluates to true/false** can be used.
---
# Dropdown Action
Source: https://docs.scrappey.com/docs/guides/browser-actions/dropdown-action
> Select an option from a dropdown menu on a webpage by targeting it with a CSS selector and specifying the desired option via index or value.
## Command
`"dropdown"`
One of the [browser actions](/docs/browser-actions) — combine it with [click](/docs/click-action) and [type](/docs/type-action) actions.
## Parameters
| Parameter | Type | Required | Description |
| ----------------- | ------- | -------- | ------------------------------------------------------------------- |
| `cssSelector` | string | Yes | CSS selector or XPath for the dropdown element. |
| `index` | number | No | Index of the option to select (starting from 0). Either `index` or `value` must be provided. |
| `value` | string | No | Value of the option to select (matches HTML `value` attribute). Either `index` or `value` must be provided. |
| `wait` | number | No | Time in seconds to wait after the action. |
| `waitForSelector` | string | No | Wait for a specific element after selection. |
| `timeout` | number | No | Max time in milliseconds to wait for the dropdown (default: 60000). |
| `ignoreErrors` | boolean | No | Continue even if the action fails. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "dropdown",
"cssSelector": "#exampleDropdown",
"index": 1,
"wait": 2
},
{
"type": "dropdown",
"cssSelector": "#anotherDropdown",
"value": "option2",
"waitForSelector": "#resultElement"
}
]
}'
```
## Notes
Either `index` or `value` must be provided. Do not use both at the same time.
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Playwright](https://scrappey.com/qa/web-scraping-apis/what-is-playwright) — the automation engine behind browser actions
- [Dynamic content scraping](https://scrappey.com/qa/web-automation/dynamic-content-scraping) — working with interactive pages
---
# Click Action
Source: https://docs.scrappey.com/docs/guides/browser-actions/click-action
> Simulate a click on a specific element on a webpage using a CSS selector or XPath within a browser session.
## Command
`"click"`
One of the [browser actions](/docs/browser-actions) — combine it with [type](/docs/type-action), [dropdown](/docs/dropdown-action), and [keyboard](/docs/keyboard) actions to drive a full interaction.
## Parameters
| Parameter | Type | Required | Description |
| ----------------- | ------- | -------- | -------------------------------------------------------- |
| `cssSelector` | string | Yes | CSS selector or XPath of the element to click. |
| `wait` | number | No | Wait for X seconds after clicking. |
| `waitForSelector` | string | No | Wait for a specific element to appear after the click. |
| `when` | string | No | Lifecycle phase to run the action: `beforelaunch`, `beforeload`, or `afterload`. Default is `afterload`. |
| `ignoreErrors` | boolean | No | Set to `true` to continue even if an error occurs. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "click",
"cssSelector": "#exampleButton"
}
]
}'
```
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Playwright](https://scrappey.com/qa/web-scraping-apis/what-is-playwright) — the automation engine behind browser actions
- [Headless browsers](https://scrappey.com/qa/web-automation/headless-browsers-guide) — driving a real browser to scrape
---
# Type Actions
Source: https://docs.scrappey.com/docs/guides/browser-actions/type-action
> Use this action to type a specific text into a selected input field on a webpage.
## Command
`"type"`
One of the [browser actions](/docs/browser-actions) — pair it with [click](/docs/click-action) and [keyboard](/docs/keyboard) actions to fill and submit forms.
## Parameters
| Parameter | Type | Required | Description |
| -------------- | ------- | -------- | ------------------------------------------------------------ |
| `cssSelector` | string | Yes | CSS selector of the input field where text should be typed. |
| `text` | string | Yes | The string to be typed. |
| `wait` | number | No | Wait for X seconds after typing the text. |
| `when` | string | No | Lifecycle phase to run the action: `beforelaunch`, `beforeload`, or `afterload`. Default is `afterload`. |
| `ignoreErrors` | boolean | No | Set to `true` to continue even if an error occurs. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "type",
"cssSelector": "#searchInput",
"text": "Example search"
}
]
}'
```
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Playwright](https://scrappey.com/qa/web-scraping-apis/what-is-playwright) — the automation engine behind browser actions
- [Dynamic content scraping](https://scrappey.com/qa/web-automation/dynamic-content-scraping) — working with interactive pages
---
# Go To
Source: https://docs.scrappey.com/docs/guides/browser-actions/go-to
> Navigate to a different page or route during your scraping flow — essential for multi-page interactions and workflows.
## Command
`"goto"`
This [browser action](/docs/browser-actions) navigates the browser to a new URL, just like a user would by clicking a link or entering a URL in the address bar.
## Parameters
| Parameter | Type | Required | Description |
| -------------- | ------- | -------- | ------------------------------------------------------------------------------------------------------------------------------ |
| `url` | string | Yes | The target URL to visit. If the domain is different from the base request, make sure it is added to your `whitelistedDomains`. |
| `when` | string | No | Lifecycle phase to run the action: `beforelaunch`, `beforeload`, or `afterload`. Default is `afterload`. |
| `ignoreErrors` | boolean | No | Set to `true` to continue execution even if the navigation fails. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "goto",
"url": "https://example.com/page"
}
]
}'
```
## Notes
Use case ideas:
* Navigate to a form or detail page before scraping fields.
* Handle multi-step flows by jumping between URLs.
* Move to a CAPTCHA or verification page before handling it.
Tips:
* Always ensure the target domain is whitelisted in your request settings (`whitelistedDomains`).
* Use with [`wait_for_selector`](/docs/wait-for-selector) or `solve_captcha` to ensure the next step happens after the page is fully loaded.
* Combine with [`if`](/docs/conditional) or [`while`](/docs/while-loop) for conditional navigation logic.
---
# Wait For Selector
Source: https://docs.scrappey.com/docs/guides/browser-actions/wait-for-selector
> Pause execution until a specific element appears in the DOM — ideal for waiting on AJAX content, modal windows, or any dynamically loaded section.
## Command
`"wait_for_selector"`
This action waits until a particular CSS selector appears on the page. It's useful for synchronization and ensuring elements are present before interacting with them.
## Parameters
| Parameter | Type | Required | Description |
| -------------- | ------- | -------- | ----------------------------------------------------------------------------------------------- |
| `cssSelector` | string | Yes | The CSS selector to wait for. Example: `[class='col']` |
| `timeout` | number | No | Time in milliseconds to wait before timing out. Default is system-defined. Example: `30000` |
| `when` | string | No | When to run the check: `"beforeload"` or `"afterload"`. Default is `"afterload"`. |
| `ignoreErrors` | boolean | No | Set to `true` to continue even if the selector is not found. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "wait_for_selector",
"cssSelector": "[class='\''example'\'']",
"timeout": 30000
}
]
}'
```
## Notes
Use Case Ideas:
* Wait for content to load after an AJAX call before scraping.
* Detect when modals or confirmation boxes appear before clicking.
* Ensure dynamic tables or product listings are fully rendered.
Tips:
* Use `wait_for_selector` before actions like [`click`](/docs/click-action), [`type`](/docs/type-action), or `solve_captcha`.
* Combine with `ignoreErrors: true` if the element may not always appear, but scraping should continue.
---
# Wait
Source: https://docs.scrappey.com/docs/guides/browser-actions/wait
> Pause the execution of browser actions for a specified amount of time — useful for waiting for dynamic content, animations, or cooldowns.
## Command
`"wait"`
This action delays further execution for a defined number of seconds, giving the page time to load or content to render.
## Parameters
| Parameter | Type | Required | Description |
| -------------- | ------- | -------- | ------------------------------------------------------------------------------------ |
| `wait` | number | Yes | Number of seconds to pause execution. |
| `when` | string | No | Define whether the wait happens `beforeload` or `afterload`. Default is `afterload`. |
| `ignoreErrors` | boolean | No | Set to `true` to suppress any error if the wait fails and continue execution. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "wait",
"wait": 5
}
]
}'
```
## Notes
* Give the page time to fully load before interacting with elements.
* Wait for popups or animations to finish before scraping.
* Introduce pauses between actions to allow page state to settle.
* Combine with [`click`](/docs/click-action), [`scroll`](/docs/scroll), or `solve_captcha` to ensure smooth flow.
* Use `when: "beforeload"` if you need to delay before the page begins loading.
---
# Execute Javascript
Source: https://docs.scrappey.com/docs/guides/browser-actions/execute-javascript
> Run custom JavaScript code directly within the page context during a scraping session.
## Command
`"execute_js"`
This action injects and runs your JavaScript code inside the target page, just like a script executed from the browser's console.
## Parameters
| Parameter | Type | Required | Description |
| ----------------- | ------- | -------- | ------------------------------------------------------------------------------------------------- |
| `code` | string | Yes | The JavaScript code to execute. This must be a valid JavaScript string. |
| `dontReturnValue` | boolean | No | Set to `true` if the script has no return value. By default the evaluated result is captured. |
## Return Values
The result of each `execute_js` action is pushed onto the `javascriptReturn` array (in execution order). You can reference a previous result inside a later action using the `{javascriptReturn[n]}` placeholder, where `n` is the zero-based index:
```json
{
"browserActions": [
{
"type": "execute_js",
"code": "document.querySelector('.token').dataset.value"
},
{
"type": "type",
"cssSelector": "#input",
"text": "{javascriptReturn[0]}"
}
]
}
```
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "execute_js",
"code": "console.log('\''Hello, world!'\'');"
}
]
}'
```
## Notes
Use Case Ideas:
* Extract values or text content from the DOM.
* Trigger JavaScript events like clicks or input changes.
* Interact with pages that use frameworks like React or Vue by invoking component methods.
* Modify the DOM before parsing the response.
Tips:
* Avoid long-running loops or infinite scripts — they may cause timeouts.
* Escape special characters properly when sending code inside JSON.
* You can combine this with [`if`](/docs/conditional), [`scroll`](/docs/scroll), [`click`](/docs/click-action), or [`type`](/docs/type-action) for dynamic control flows.
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Dynamic content scraping](https://scrappey.com/qa/web-automation/dynamic-content-scraping) — running JS to reach content
- [Headless browser](https://scrappey.com/qa/web-scraping-apis/what-is-a-headless-browser) — the engine that runs your JS
---
# Keyboard
Source: https://docs.scrappey.com/docs/guides/browser-actions/keyboard
> Simulate specific keyboard actions during your scraping session, useful for navigation, form control, or clearing fields.
## Command
`"keyboard"`
This [browser action](/docs/browser-actions) triggers a specific key press event, such as `Enter`, `Tab`, or `ArrowDown`, optionally on a selected element.
## Parameters
| Parameter | Type | Required | Description |
| ------------- | ------ | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| `value` | string | Yes | The keyboard action to simulate. Must be one of: `tab`, `enter`, `space`, `arrowdown`, `arrowup`, `arrowleft`, `arrowright`, `backspace`, `clear`. |
| `cssSelector` | string | No | Required only when using `clear`. Specifies which element the action applies to. |
## Supported `value` Options
| Action | Description |
| ------------ | ----------------------------------------------------------- |
| `tab` | Simulates pressing the Tab key |
| `enter` | Simulates pressing Enter |
| `space` | Simulates pressing Spacebar |
| `arrowdown` | Scrolls or moves focus down |
| `arrowup` | Scrolls or moves focus up |
| `arrowleft` | Moves cursor/focus left |
| `arrowright` | Moves cursor/focus right |
| `backspace` | Deletes one character |
| `clear` | Clears the value of an input field (requires `cssSelector`) |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "keyboard",
"value": "enter"
}
]
}'
```
## Notes
* Submit a form using `enter` after typing into an input field.
* Navigate a dropdown or form with `tab` and `arrow` keys.
* Clear a field before typing a new value using `clear`.
* Combine with [`type`](/docs/type-action), [`click`](/docs/click-action), or [`wait`](/docs/wait) to sequence form interactions.
* Make sure the `cssSelector` targets an interactive or input element when using `clear`.
---
# Scroll
Source: https://docs.scrappey.com/docs/guides/browser-actions/scroll
> Perform a scrolling action either to a specific element or to the bottom of the page — useful for loading dynamic content or triggering lazy-loaded elements.
## Command
`"scroll"`
Use this command to simulate user-like scrolling behavior during scraping or automation.
## Parameters
| Parameter | Type | Required | Description |
| ------------- | ------ | -------- | ------------------------------------------------------------------------------------------------------------ |
| `cssSelector` | string | No | The CSS selector of the element to scroll to. If omitted, the page will scroll to the bottom by default. |
| `repeat` | number | No | Number of times to repeat the scroll. This enables infinite scrolling behavior. |
| `delayMs` | number | No | Delay between each scroll in milliseconds — useful for waiting on content to load between scrolls. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "scroll",
"cssSelector": "#bottomElement",
"repeat": 3,
"delayMs": 1000
}
]
}'
```
## Notes
Use Case Ideas:
* Load more search results on websites with infinite scroll.
* Trigger lazy-loaded images or content blocks.
* Scroll to a form or button before interacting with it (e.g., [clicking](/docs/click-action) or [typing](/docs/type-action)).
Tips:
* Combine with `waitForSelector` after scroll to ensure the element has loaded before scraping.
* Use high `repeat` counts + delay for deep content loading.
* Can be combined with [`if`](/docs/conditional) or [`while`](/docs/while-loop) [browser actions](/docs/browser-actions) for intelligent scrolling.
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [Scraping infinite-scroll pages](https://scrappey.com/qa/web-scraping-apis/how-to-scrape-infinite-scroll-pages) — loading content as you scroll
- [Dynamic content scraping](https://scrappey.com/qa/web-automation/dynamic-content-scraping) — working with interactive pages
---
# While Loop
Source: https://docs.scrappey.com/docs/guides/browser-actions/while-loop
> Repeat a set of browser actions until a specific condition is no longer true.
## Command
`"while"`
Use this command to perform looping logic within the browser context. It's useful for tasks like handling CAPTCHAs, paginated scraping, or waiting for content to disappear/appear. It's the looping counterpart to the [`if`](/docs/conditional) conditional, and one of the [browser actions](/docs/browser-actions).
## Parameters
| Parameter | Type | Required | Description |
| ------------- | ------- | -------- | ------------------------------------------------------------------------------------------------------------- |
| `condition` | string | Yes | A JavaScript expression that returns `true` or `false`. The loop continues as long as it evaluates to `true`. |
| `then` | array | Yes | Array of browser actions to execute on each iteration while the condition is true. |
| `maxAttempts` | number | No | Max number of loop iterations to prevent infinite loops. |
## Example
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "while",
"condition": "document.querySelector('body').innerText.toLowerCase().includes('terms')",
"then": [
{
"type": "goto",
"url": "https://example.com/alternate-page"
},
{
"type": "solve_captcha",
"captcha": "custom"
}
]
}
]
}'
```
## Notes
* You can combine this with other browser actions like [`click`](/docs/click-action), [`type`](/docs/type-action), [`goto`](/docs/go-to), [`wait`](/docs/wait), or even another [`if`](/docs/conditional) or `while` block.
* Always consider setting a `maxAttempts` to avoid infinite loops and excessive resource usage.
* Use Case Ideas: looping through paginated content until no "Next" button appears, retrying CAPTCHA verification handling until successful, or waiting for an element to disappear before proceeding.
---
# Conditional
Source: https://docs.scrappey.com/docs/guides/browser-actions/conditional
> Execute different browser actions depending on the content of the page, using JavaScript expressions evaluated in the browser context.
## Command
`"if"`
Use conditional logic to make your scraping smarter — only proceed with certain actions if specific content is detected. It's the branching counterpart to the [`while`](/docs/while-loop) loop, and one of the [browser actions](/docs/browser-actions).
## Parameters
| Parameter | Type | Required | Description |
| ----------- | ------ | -------- | -------------------------------------------------------------------------------------------------- |
| `condition` | string | Yes | A JavaScript expression that evaluates to `true` or `false`. This is executed in the page context. |
| `then` | array | Yes | An array of browser actions to execute if the condition returns `true`. |
| `or` | array | No | An array of browser actions to execute if the condition returns `false`. |
You can nest `if` blocks inside each other for complex decision-making.
## Example
In this example, we visit a page and:
* If the body text contains the word "terms", we navigate to a different URL.
* Otherwise, we click a button.
```json
{
"cmd": "request.get",
"url": "https://www.arena-top100.com/index.php?a=in&u=sergey1234",
"browserActions": [
{
"type": "if",
"condition": "document.querySelector('body').innerText.toLowerCase().includes('terms')",
"then": [
{
"type": "if",
"condition": "document.querySelector('body').innerText.toLowerCase().includes('terms')",
"then": [
{
"type": "goto",
"url": "https://www.arena-top100.com/index.php?a=in&u=test1"
}
]
}
],
"or": [
{
"type": "click",
"cssSelector": "#continueButton"
}
]
}
]
}
```
Test it with cURL:
```bash
curl -X POST "https://publisher.scrappey.com/api/v1?key=YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://www.arena-top100.com/index.php?a=in&u=sergey1234",
"browserActions": [ ... ]
}'
```
## Notes
The `if` conditional action works across all Scrappey clients, including Python, JavaScript (Node.js), PHP, Java, Go, Ruby, C#, and cURL.
Conditional logic is well-suited for handling unexpected redirects, skipping cookie/consent modals, navigating multi-step flows, and avoiding unnecessary browser actions.
---
# Error Codes
Source: https://docs.scrappey.com/docs/guides/error-codes
> Reference table of all Scrappey error codes, their error messages, and what to do when you encounter them.
## Example
When a request fails, the response has `"data": "error"` and an `error` field containing the message. Handle the stable `CODE-XXXX` identifier in your code — it is the canonical key. The public `error` messages below are normalized categories; the raw, provider-specific diagnostics (including which verification provider was involved) are available in the debug view of your Scrappey dashboard. Many errors are transient — an automatic [retry](/docs/retry) or a different [proxy](/docs/premium-proxy) often resolves them.
```json
{
"solution": {},
"timeElapsed": 1234,
"data": "error",
"session": "your-session-id",
"error": "Browser verification failed"
}
```
## Error Code Reference
| **Code** | **Error Message** | **Description** |
| ------------ | --------------------------------------------------------------------- | ------------------------------------------------------------------ |
| `CODE-0001` | Server is overloaded | All server capacity is used, please try again. |
| `CODE-0002` | Browser verification failed | Browser verification could not be completed, preventing access. |
| `CODE-0003` | Too many verification attempts, try again | Too many attempts from your IP; try again later. |
| `CODE-0004` | Invalid cmd command | The provided command is invalid. |
| `CODE-0005` | Tunnel connection failed | Tunnel connection could not be established. |
| `CODE-0006` | ERR\_HTTP\_RESPONSE\_CODE\_FAILURE | Failure in HTTP response. |
| `CODE-0007` | Could not complete interactive verification | Clicking the verification widget failed. |
| `CODE-0007` | Proxy error - ERR\_TUNNEL\_CONNECTION\_FAILED or ERR\_EMPTY\_RESPONSE | Proxy error. Review proxy configuration for the target site. |
| `CODE-0008` | Target site rejected request | The target site rejected the request. |
| `CODE-0009` | Error from ChatGPT, try again | ChatGPT API returned an error. |
| `CODE-0010` | Proxy rejected by verification provider | Proxy rejected by the verification provider. Review proxy configuration and authorization for the target site. |
| `CODE-0011` | Verification provider rejected session | A verification step could not be completed. |
| `CODE-0012` | Could not parse verification cookie | Issue parsing a verification cookie. |
| `CODE-0013` | Verification cookie processing error | A verification cookie could not be processed. |
| `CODE-0014` | Could not load verification provider | The verification provider could not be loaded. |
| `CODE-0015` | Socks4 With Authentication not Supported | Socks4 with authentication is not supported. |
| `CODE-0016` | Socks5 With Authentication not Supported | Socks5 with authentication is not supported. |
| `CODE-0017` | Browser verification temporarily unavailable, try again later | Temporary incompatibility with an updated verification release. |
| `CODE-0018` | Too high error rate for this URL | Temporary ban due to high error rate. Contact support. |
| `CODE-0019` | The proxy server is refusing connections | Check proxy settings. Proxy is refusing connections. |
| `CODE-0020` | Could not find intercept request | System could not find the intercept request. |
| `CODE-0021` | Unknown error occurred with request | An unknown request error occurred. |
| `CODE-0022` | Captcha type solve\_captcha is not found | The specified captcha type was not found. |
| `CODE-0023` | Interactive verification handler not found | The interactive verification handler was not found. |
| `CODE-0024` | Proxy timeout - proxy too slow | Proxy took too long to respond. |
| `CODE-0025` | NS\_ERROR\_NET\_TIMEOUT - proxy too slow | Browser-level timeout due to slow proxy. |
| `CODE-0026` | Internal browser error | Internal browser failure. |
| `CODE-0027` | No elements found for this CSS selector | The specified CSS selector did not match any elements. |
| `CODE-0028` | Verification provider rejected session | A verification step could not be completed. |
| `CODE-0029` | Too many sessions open | Maximum sessions exceeded. [Sessions](/docs/sessions-create) auto-expire after 240 seconds. |
| `CODE-0030` | Browser name must be: firefox, chrome or safari | Only these browsers are supported. |
| `CODE-0031` | Request error, please try again | General request error. Retry. |
| `CODE-0032` | Interactive verification could not be completed | An interactive verification step could not be completed. Review proxy configuration and authorization for the target site. |
| `CODE-0033` | Managed verification could not be completed | A managed verification step could not be completed. |
| `CODE-0034` | Verification could not be completed after multiple attempts | Review proxy configuration and authorization for the target site. |
| `CODE-0035` | Could not load verification widget | The verification widget failed to load. |
| `CODE-0036` | Keyboard action value not found | The specified keyboard action is invalid. |
| `CODE-0037` | Verification provider rejected session | The verification provider rejected the request. Review proxy configuration and authorization for the target site. |
| `CODE-10000` | Unknown error - has to be specified | An unspecified error occurred. Needs review. |
## Related concepts
Go deeper in the [Scrappey knowledge base](https://scrappey.com/qa):
- [403 Forbidden](https://scrappey.com/qa/http-errors/what-is-a-403-error) — the classic "blocked by anti-bot" code
- [429 Too Many Requests](https://scrappey.com/qa/http-errors/what-is-a-429-error) — rate-limit responses
- [503 Service Unavailable](https://scrappey.com/qa/http-errors/what-is-a-503-error) — overload / challenge responses
---
# Browser Actions
Source: https://docs.scrappey.com/docs/guides/browser-actions
> Automate interactions with web pages by specifying a sequence of typed actions in the `browserActions` array of your request payload.
## Command
Browser actions are passed as the `browserActions` array on any `request.get` or `request.post` call.
## Parameters
All actions share a common set of top-level properties.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `type` | string | Yes | The action type (see Action Types below). |
| `when` | string | No | Lifecycle phase to run this action. Defaults to `"afterload"`. |
| `ignoreErrors` | boolean | No | If `true`, errors in this action will not stop execution. |
| `timeout` | number | No | Timeout in milliseconds. Default `60000`. |
## Execution Lifecycle
Browser actions run at specific points during the request lifecycle. Use the `when` property to control when each action executes. If `when` is omitted it defaults to `"afterload"`.
### Lifecycle Phases (in order)
```
1. beforelaunch — Before browser navigates to the URL
2. beforeload — After browser is ready, before the page loads
3. afterload — After the page has loaded (default)
```
### Phase Details
| Phase | When it runs | Use case |
| -------------- | ------------ | -------- |
| `beforelaunch` | Before any navigation. No page context yet. | Setting up browser state, pre-flight actions |
| `beforeload` | After cookies are added, before GET/POST navigation | Pre-navigation setup, adding localStorage |
| `afterload` | After page navigation completes | Main page interactions — filling forms, clicking buttons, extracting data |
Additional lifecycle phases used by authorized verification workflows are documented on the login-gated [Verification Flow Handling](/docs/solve-captcha) guide — sign in with your Scrappey account to view them.
Each action in the `browserActions` array is tagged with a `when` phase. The engine loops through the array multiple times — once per phase — and only runs actions whose `when` matches the current phase. Actions within the same phase execute in order.
When actions are nested inside `if.then`, `if.or`, or `while.then`, the `when` property of nested actions is ignored — they run immediately when the parent action executes. Only top-level actions in the `browserActions` array are filtered by `when`.
## Action Types
### `click` - Click an Element
Clicks on an element using CSS selector. See the dedicated [Click Action](/docs/click-action) guide.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `cssSelector` | string | Yes | CSS selector of element to click. |
| `wait` | number | No | Wait time in seconds after clicking. |
| `waitForSelector` | string | No | Wait for this selector to appear after click. |
| `direct` | boolean | No | Use direct click instead of cursor simulation. |
**Example:**
```json
{
"type": "click",
"cssSelector": "#submit-button",
"wait": 1,
"waitForSelector": ".success-message"
}
```
### `type` - Type Text into Input
Types text into an input field with configurable delays between keystrokes. See the dedicated [Type Action](/docs/type-action) guide.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `cssSelector` | string | Yes | CSS selector of input element. |
| `text` | string | Yes | Text to type. |
| `wait` | number | No | Wait time in seconds after typing. |
| `direct` | boolean | No | Use direct typing instead of cursor simulation. |
**Example:**
```json
{
"type": "type",
"cssSelector": "#username",
"text": "myusername"
}
```
### `goto` - Navigate to URL
Navigates the browser to a new URL. See the dedicated [Go To](/docs/go-to) guide.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `url` | string | Yes | URL to navigate to. |
| `wait` | number | No | Wait time in seconds after navigation. |
**Example:**
```json
{
"type": "goto",
"url": "https://example.com/page2"
}
```
### `wait` - Wait for Duration
Pauses execution for a specified time. See the dedicated [Wait](/docs/wait) guide.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `wait` | number | Yes | Wait time in seconds. |
**Example:**
```json
{
"type": "wait",
"wait": 2
}
```
### `wait_for_selector` - Wait for Element
Waits for an element to appear in the DOM. See the dedicated [Wait For Selector](/docs/wait-for-selector) guide.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `cssSelector` | string | Yes | CSS selector to wait for. |
| `timeout` | number | No | Timeout in ms. Default `60000`. |
**Example:**
```json
{
"type": "wait_for_selector",
"cssSelector": ".loaded-content",
"timeout": 30000
}
```
### `wait_for_function` - Wait for JavaScript Condition
Waits until a JavaScript expression returns truthy. See the dedicated [Wait For Function](/docs/wait-for-function) guide.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `code` | string | Yes | JavaScript code that returns truthy when done. |
| `timeout` | number | No | Timeout in ms. Default `60000`. |
**Example:**
```json
{
"type": "wait_for_function",
"code": "window.dataLoaded === true",
"timeout": 30000
}
```
### `wait_for_load_state` - Wait for Page State
Waits for a specific page load state. See the dedicated [Wait For Load State](/docs/wait-for-load-state) guide.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `waitForLoadState` | string | Yes | `"domcontentloaded"`, `"networkidle"`, or `"load"`. |
**Example:**
```json
{
"type": "wait_for_load_state",
"waitForLoadState": "networkidle"
}
```
### `wait_for_cookie` - Wait for Cookie
Waits for a specific cookie to be set.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `cookieName` | string | Yes | Name of the cookie to wait for. |
| `cookieValue` | string | No | Expected value of the cookie. |
| `cookieDomain` | string | No | Domain the cookie should be set on. |
| `pollIntervalMs` | number | No | Poll interval. Default `200` ms. |
| `timeout` | number | No | Timeout in ms. Default `60000`. |
**Example:**
```json
{
"type": "wait_for_cookie",
"cookieName": "session_id",
"timeout": 30000
}
```
### `execute_js` - Execute JavaScript
Executes JavaScript code on the page. Results are stored in the `javascriptReturn` array. See the dedicated [Execute Javascript](/docs/execute-javascript) guide.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `code` | string | Yes | JavaScript code to execute. |
| `dontReturnValue` | boolean | No | If `true`, the return value is not captured. |
**Example:**
```json
{
"type": "execute_js",
"code": "document.querySelector('.price').innerText"
}
```
Access results in subsequent actions via `{javascriptReturn[0]}`.
### `scroll` - Scroll Page/Element
Scrolls to an element or the bottom of the page. See the dedicated [Scroll](/docs/scroll) guide.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `cssSelector` | string | No | Element to scroll to. If omitted, scrolls to bottom. |
| `repeat` | number | No | Number of times to repeat the scroll. |
| `delayMs` | number | No | Delay between scrolls. Default `100` ms. |
**Example:**
```json
{
"type": "scroll",
"cssSelector": "#footer",
"repeat": 3,
"delayMs": 500
}
```
### `hover` - Hover Over Element
Hovers the mouse over an element.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `cssSelector` | string | Yes | CSS selector of element to hover. |
| `timeout` | number | No | Timeout in ms. |
**Example:**
```json
{
"type": "hover",
"cssSelector": ".dropdown-trigger"
}
```
### `keyboard` - Press Keyboard Key
Simulates keyboard key presses. See the dedicated [Keyboard](/docs/keyboard) guide.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `value` | string | Yes | Key to press (see supported values below). |
| `cssSelector` | string | No | Element to focus first (required for `clear`). |
| `wait` | number | No | Wait time in seconds after pressing. |
| `waitForSelector` | string | No | Wait for selector after pressing. |
**Supported values:** `tab`, `enter`, `space`, `arrowdown`, `arrowup`, `arrowleft`, `arrowright`, `backspace`, `clear`
**Example:**
```json
{
"type": "keyboard",
"value": "enter"
}
```
### `dropdown` - Select Dropdown Option
Selects an option from a dropdown/select element. See the dedicated [Dropdown Action](/docs/dropdown-action) guide.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `cssSelector` | string | Yes | CSS selector of select element. |
| `index` | number | No | Option index to select. |
| `value` | string | No | Option value to select. |
| `wait` | number | No | Wait time in seconds after selection. |
| `waitForSelector` | string | No | Wait for selector after selection. |
Either `index` or `value` is required.
**Example:**
```json
{
"type": "dropdown",
"cssSelector": "#country-select",
"value": "US"
}
```
### `switch_iframe` - Switch to iFrame
Switches context to an iframe for subsequent actions.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `cssSelector` | string | Yes | CSS selector of the iframe. |
**Example:**
```json
{
"type": "switch_iframe",
"cssSelector": "#payment-iframe"
}
```
### `set_viewport` - Set Browser Viewport
Changes the browser viewport size.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `width` | number | No | Viewport width. Default `1280`. |
| `height` | number | No | Viewport height. Default `1024`. |
| `wait` | number | No | Wait time in seconds after setting. |
**Example:**
```json
{
"type": "set_viewport",
"width": 1920,
"height": 1080
}
```
### `if` - Conditional Execution
Executes actions conditionally based on a JavaScript condition. See the dedicated [Conditional](/docs/conditional) guide.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `condition` | string | Yes | JavaScript condition to evaluate. |
| `then` | array | No | Actions to run if condition is true. |
| `or` | array | No | Actions to run if condition is false. |
**Example:**
```json
{
"type": "if",
"condition": "document.querySelector('.captcha') !== null",
"then": [
{ "type": "solve_captcha", "captcha": "custom" }
],
"or": [
{ "type": "click", "cssSelector": "#continue" }
]
}
```
### `while` - Loop Execution
Loops actions while a condition is true. See the dedicated [While Loop](/docs/while-loop) guide.
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------- |
| `condition` | string | Yes | JavaScript condition to evaluate. |
| `then` | array | Yes | Actions to run each iteration. |
| `maxAttempts` | number | No | Maximum iterations (prevents infinite loops). |
**Example:**
```json
{
"type": "while",
"condition": "document.querySelector('.load-more') !== null",
"then": [
{ "type": "click", "cssSelector": ".load-more" },
{ "type": "wait", "wait": 1 }
],
"maxAttempts": 10
}
```
### `solve_captcha` - Verification Flow Handling
Handles interactive verification steps that can appear during an authorized browser workflow, so a permitted session can continue. The action takes a `captcha` type and an optional `captchaData` configuration object.
The supported verification types and the full `captchaData` parameter reference are documented on the login-gated [Verification Flow Handling](/docs/solve-captcha) guide — sign in with your Scrappey account to view them. Only use verification handling within authorized workflows where you have a lawful basis to proceed.
**Example:**
```json
{
"type": "solve_captcha",
"captcha": "custom"
}
```
### Authenticated login
Some workflows need to start from an already-authenticated session. Token-based login actions are documented on the login-gated [Authenticated Login Flow](/docs/authenticated-login-flow) guide; sign in with your Scrappey account to view the parameters and example. Only use them to automate accounts you own or are authorized to access.
### `remove_iframes` - Remove All iFrames
Removes all iframes from the page (experimental). No additional properties required.
**Example:**
```json
{
"type": "remove_iframes"
}
```
### `click_and_hold` - Click and Hold
*Note: This action is not yet implemented.*
## Example
### Using JavaScript Return Values
Results from `execute_js` are stored in the `javascriptReturn` array and can be referenced in subsequent actions:
```json
{
"browserActions": [
{
"type": "execute_js",
"code": "document.querySelector('.token').dataset.value"
},
{
"type": "type",
"cssSelector": "#input",
"text": "{javascriptReturn[0]}"
}
]
}
```
### Full Login Flow
```json
{
"cmd": "request.get",
"url": "https://example.com",
"browserActions": [
{
"type": "wait_for_selector",
"cssSelector": "#login-form",
"when": "afterload"
},
{
"type": "type",
"cssSelector": "#username",
"text": "myuser"
},
{
"type": "type",
"cssSelector": "#password",
"text": "mypassword"
},
{
"type": "solve_captcha",
"captcha": "custom"
},
{
"type": "click",
"cssSelector": "#submit",
"waitForSelector": ".dashboard"
},
{
"type": "execute_js",
"code": "document.querySelector('.user-data').innerText"
}
]
}
```
## Notes
### Sequencing actions after a verification step
Actions that must run only after an authorized verification step completes use a dedicated lifecycle phase. That phase, and a worked example of submitting a form once a verification token is ready, are documented on the login-gated [Verification Flow Handling](/docs/solve-captcha) guide — sign in with your Scrappey account to view them.
[All the browser actions](https://www.scrappey.com/scrapers/BrowserActions)
---