Auto-Detect Protection
Analyze any URL to detect its bot-protection and CAPTCHA provider, recommended proxy country, and a ready-to-use Scrappey request body.
Send a URL and get back the detected protection provider, the recommended proxy country, and a ready-to-use Scrappey request body — so you can configure each scrape automatically instead of guessing.
What it does
Point this endpoint at any URL and it inspects the live response, identifies which bot-protection or CAPTCHA provider the site uses, infers the best proxy country from the site's location, and returns a complete Scrappey request body you can send straight to the request.get API. It's the programmatic equivalent of the Playground's "auto" setup: detect the target, pick the best method, and run.
This is most useful when you scrape many unpredictable domains (user-imported listings, mixed portals, unknown sites) and don't know the right settings up front.
Request
The url value must be URL-encoded. No API key is required for the detection call itself.
| Parameter | Type | Required | Description |
|---|---|---|---|
url |
string | Yes | Full URL to analyze. Must be URL-encoded and a valid HTTP or HTTPS URL. |
Response
The scrappeySettings.apiRequestBody field is abbreviated above — see Use the result for the complete body it produces.
Use the result
Take scrappeySettings.apiRequestBody from the response and POST it directly to the Scrappey API — it already includes the right requestType, proxyCountry, and session settings for the detected target. The body below is the kind of request apiRequestBody produces:
The recommended flow is detect → take apiRequestBody → send to Scrappey. For repeated scrapes of the same site, cache the detection result per domain so you only call detect once per new domain.
Response fields
Top-level
| Parameter | Type | Required | Description |
|---|---|---|---|
success |
boolean | Yes | Whether detection completed successfully. |
url |
string | No | The URL that was analyzed. |
domain |
string | No | Extracted domain name. |
statusCode |
number | No | HTTP status code returned by the target site. |
timestamp |
string | No | ISO 8601 timestamp of the detection. |
`detection`
| Parameter | Type | Required | Description |
|---|---|---|---|
providers |
string[] | No | All detected provider IDs. |
primaryProvider |
string | No | Provider with the highest confidence, or "none". |
confidence |
object | No | Map of provider ID to confidence score (0-100). |
challenged |
boolean | No | Whether the site returned a challenge or block page (403, 503, CAPTCHA). |
details |
array | No | Individual pattern matches, each with provider, type, pattern, description, and confidence. |
`country`
| Parameter | Type | Required | Description |
|---|---|---|---|
code |
string | No | ISO 3166-1 alpha-2 country code (e.g. US, GR). |
name |
string | No | Human-readable country name. |
proxyCountry |
string | No | Value to pass as Scrappey's proxyCountry (e.g. UnitedStates, Greece). |
detectedVia |
string | No | How the country was inferred: geoip, language, tld, or none. |
`recommendation`
| Parameter | Type | Required | Description |
|---|---|---|---|
explanation |
string | No | Human-readable summary of the detection and recommendation. |
tips |
string[] | No | Suggestions for handling the detected protection. |
proxyCountry |
string | No | Recommended proxy country. |
`scrappeySettings`
| Parameter | Type | Required | Description |
|---|---|---|---|
recommended |
object | No | Full recommended settings with explanatory notes. |
apiRequestBody |
object | No | Minimal request body, ready to POST to the Scrappey API. |
How detection works
- The endpoint makes an HTTP request to the target with a realistic browser User-Agent.
- It collects the response headers, cookies, and HTML content.
- It runs detector rule sets against that data, matching known patterns in cookies, headers, content, and script URLs.
- Each match has a confidence score; the provider's overall confidence is the highest match plus a boost for multiple matches, capped at 100.
- Country is inferred via IP geolocation first, falling back to the page's
langattribute or the domain's country TLD.
Integration examples
JavaScript / Node.js
Python
cURL
Error responses
A failed detection returns success: false with an error message. Common cases:
| Case | error message |
|---|---|
Missing url |
Missing required query parameter: url |
| Invalid URL | Invalid URL. Must be a valid HTTP or HTTPS URL. |
| Target unreachable | Failed to reach the target URL. The site may be down or blocking requests. |
| Server error | Detection failed: <details> |
Related concepts
proxyCountry— the country values returned incountry.proxyCountry.request.get— the API command you send the recommended body to.- Premium proxy — the residential proxy type recommended for protected sites.