SDKs for Python and Node.js

The official SDKs wrap the ScrapeIt REST API for the endpoints you call with an API key: the Web Scraping API (sync and async), Google SERP, Google Maps, Amazon, the Website Crawler, jobs and usage. Request parameters are typed from the OpenAPI document, so your editor completes every scrape option; authentication, retries and error codes are handled for you.

	Node.js / TypeScript	Python
Package	scrapeit-sdk on npm	scrapeit-sdk on PyPI (import scrapeit)
Runtime	Node.js 18+, ESM and CommonJS, no dependencies	Python 3.11+, httpx, sync and async clients
Install	npm install scrapeit-sdk	pip install scrapeit-sdk

Set up the client

Create an API key in Workspace → API Keys (Authentication & API keys) and either pass it to the client or export it as SCRAPEIT_API_KEY:

cURL

export SCRAPEIT_API_KEY="YOUR_SCRAPEIT_API_KEY"

Node.js

import { Scrapeit } from 'scrapeit-sdk';

// Reads SCRAPEIT_API_KEY when apiKey is omitted.
const client = new Scrapeit({ apiKey: 'YOUR_SCRAPEIT_API_KEY' });

Python

from scrapeit import Scrapeit

# Reads SCRAPEIT_API_KEY when the key is omitted.
client = Scrapeit("YOUR_SCRAPEIT_API_KEY")

Options: baseUrl / base_url (default https://api.scrapeit.io), maxRetries / max_retries (default 2), timeoutMs / timeout (per attempt, default 200 seconds - above the 180-second ceiling of a synchronous scrape). Node.js also accepts a custom fetch, Python a ready httpx.Client.

Scrape a page

scrape() calls POST /api/v1/scrape/web and accepts every parameter of that endpoint:

Node.js

const page = await client.scrape({
  url: 'https://example.com',
  jsRendering: true,
  outputFormat: ['html', 'text'],
});

console.log(page.data); // the { data, meta } envelope, exactly as the API returns it

Python

page = client.scrape(url="https://example.com", jsRendering=True, outputFormat=["html", "text"])

print(page["data"])  # the {"data", "meta"} envelope, exactly as the API returns it

Run a scrape in the background

scrapeAsync() / scrape_async() start the request and return its job at once; waitForAsyncScrape() / wait_for_async_scrape() poll it until it is finished, failed, stopped or skipped. Pass webhookUrl to be notified instead of polling (Webhooks).

Node.js

const job = await client.scrapeAsync({ url: 'https://example.com', jsRendering: true });
const done = await client.waitForAsyncScrape(job.data.id, { pollIntervalMs: 2000 });

console.log(done.data.status);

Python

job = client.scrape_async(url="https://example.com", jsRendering=True)
done = client.wait_for_async_scrape(job["data"]["id"], poll_interval=2)

print(done["data"]["status"])

The async Python client has the same methods - await each of them:

import asyncio
from scrapeit import AsyncScrapeit

async def main() -> None:
    async with AsyncScrapeit() as client:
        job = await client.scrape_async(url="https://example.com")
        done = await client.wait_for_async_scrape(job["data"]["id"])
        print(done["data"]["status"])

asyncio.run(main())

Product APIs

Node.js

const serp = await client.googleSearch({ q: 'coffee near me', gl: 'us', hl: 'en' });
const places = await client.googleMaps.search({ q: 'coffee', location: 'Austin, TX' });
const place = await client.googleMaps.place({ place_id: 'ChIJ...' });
const products = await client.amazon.search({ q: 'usb-c charger', domain: 'amazon.com' });
const product = await client.amazon.product({ asin: 'B0EXAMPLE1' });

Python

serp = client.google_search(q="coffee near me", gl="us", hl="en")
places = client.google_maps.search(q="coffee", location="Austin, TX")
place = client.google_maps.place(place_id="ChIJ...")
products = client.amazon.search(q="usb-c charger", domain="amazon.com")
product = client.amazon.product(asin="B0EXAMPLE1")

Parameters and response fields of each product: Google SERP API, Google Maps API, Amazon Scraper API.

Crawl a website

Node.js

const crawl = await client.crawler.start({ urls: ['https://example.com'], maxDepth: 2, limit: 50 });
await client.crawler.wait(crawl.data.id);

const pages = await client.crawler.results(crawl.data.id, { page: 1, per_page: 50 });

Python

crawl = client.crawler.start(urls=["https://example.com"], maxDepth=2, limit=50)
client.crawler.wait(crawl["data"]["id"])

pages = client.crawler.results(crawl["data"]["id"], page=1, per_page=50)

See Website Crawler API for the crawl options.

Jobs, usage and other endpoints

jobs.get(jobId) returns the sanitized diagnostics of any job (Getting job results), and usage() the credits, concurrency and usage of the key's organization. Every other endpoint that accepts an API key (Zillow, Allegro, Apartments.com, Booking.com, Yelp, Walmart, Amazon Best Sellers, Redfin, Airbnb, Tripadvisor, Yellow Pages, their no-code bulk jobs, batch scraping) is one request() call away, with the same authentication, retries and errors:

Node.js

const usage = await client.usage();
const listing = await client.request('get', '/api/v1/scrape/zillow/listing', {
  query: { city: 'Chicago', state: 'IL', home_types: ['houses', 'condos'] },
});

Python

usage = client.usage()
listing = client.request(
    "GET",
    "/api/v1/scrape/zillow/listing",
    params={"city": "Chicago", "state": "IL", "home_types": ["houses", "condos"]},
)

Errors and retries

A failed call raises ScrapeitError with the stable code from the API { error } envelope, the HTTP status, details and the correlationId / correlation_id to quote in support requests. Branch on code, not on the message:

Node.js

import { ScrapeitError } from 'scrapeit-sdk';

try {
  await client.scrape({ url: 'https://example.com' });
} catch (error) {
  if (error instanceof ScrapeitError && error.code === 'INSUFFICIENT_CREDITS') {
    // top up or upgrade the plan
  }
  throw error;
}

Python

from scrapeit import ScrapeitError

try:
    client.scrape(url="https://example.com")
except ScrapeitError as error:
    if error.code == "INSUFFICIENT_CREDITS":
        ...  # top up or upgrade the plan
    raise

The SDKs retry automatically on 429 (rate limit or a busy concurrency slot - rejected before any work), 503 and 500, waiting for Retry-After when the API sends it and backing off exponentially otherwise. 502 (for example TARGET_BLOCKED), 504 (SCRAPE_TIMEOUT), network failures and client timeouts are not retried: resending is your decision. The SDK's own codes are MISSING_API_KEY, NETWORK_ERROR, REQUEST_TIMEOUT, INVALID_RESPONSE and WAIT_TIMEOUT.

Types

Request types are generated from the OpenAPI document and shipped with the packages (paths, components and operations in Node.js, scrapeit.models TypedDicts in Python). Responses are returned as the { data, meta } envelope as-is; in TypeScript data is unknown - narrow it, or describe the fields you read in your own type.