Skip to content

Caching ​

The shared cache lets a repeated scrape skip fetching and rendering entirely: the response comes back faster and costs about half the normal credits. Only successful HTTP 200 responses are ever stored.

Enabling the cache ​

json
{
  "url": "https://example.com/pricing",
  "cache": {
    "enabled": true,
    "strategy": "prefer_cache",
    "maxAgeSeconds": 3600,
    "shareScope": "organization"
  }
}
Field (cache.*)TypeExampleDescription
enabledbooleantrueTurn cache lookup/write on or off for this request.
strategystringprefer_cacheprefer_cache, fresh or stale_while_revalidate (below).
maxAgeSecondsnumber3600Oldest acceptable cache entry age; older entries are treated as a miss.
shareScopestringorganizationWho may reuse the entry: private, organization or global.

Strategies ​

strategyBehaviour
prefer_cache (default)Serve a fresh-enough cached entry if one exists; otherwise scrape and store.
freshBypass the cache, always scrape, then update the entry.
stale_while_revalidateAccepted, but behaves like prefer_cache: an expired entry is never served stale.

Share scope ​

shareScopeWho can reuse the entry
privateOnly your own identical requests.
organizationAnyone in your workspace.
globalAny ScrapeIt tenant (only anonymous, non-personalized pages should use this).

Reading the cache decision ​

Every response reports what the cache did in request_metadata.cache_status:

json
{ "request_metadata": { "cache_status": "hit" } }

hit means the page came from cache; miss (or bypass) means it was scraped fresh.

What a cache hit costs ​

A cache hit is charged at half price, rounded up: ceil(credits / 2). So a 4-credit fetch is 2 on a hit, a 30-credit render is 15 and a 15-credit residential fetch is 8 (ceil of 7.5). You still pay a little because the platform served and metered a result - but never the full render/proxy cost. The "normal cost" is what the cached request was actually charged: a hit on a page that a proxyType: "auto" request got from a datacenter exit costs half the datacenter price, not half the dearest tier the balance check reserved.

Normal costCache-hit cost
4 (fetch)2
30 (render)15
15 (residential fetch)8
50 (residential render)25

Good candidates for caching ​

  • Reference pages that change slowly (pricing, docs, catalogs) with a generous maxAgeSeconds.
  • High-fan-out workloads where many jobs hit the same URLs - set shareScope: organization.
  • Development and testing, so you are not re-charged full price while iterating.

Avoid caching personalized or rapidly-changing pages, or use strategy: fresh for them.

ScrapeIt Web Scraping API documentation. Managed web scraping services: www.scrapeit.io.