answers.delivery

How-to · 3 min read

How to scrape a page that loads its content with JavaScript

Your scraper fetches a product or docs page and gets back an empty shell, because the prices, specs or text only appear after JavaScript runs in a browser. Running headless Chrome yourself fixes it, but now you are babysitting browser processes, timeouts and memory for every job. Paying a scraping service for full browser rendering on every request works too, until the bill arrives for pages that never needed a browser in the first place. You want the real content, at the lowest cost each page allows.

Published
17 Nov 2026
Delivery
No. 12
Topic
Developer
Evidence
9 claims quoted from spicrawl.com

01
Why it's hard

There is no way to tell from a URL whether the page needs JavaScript, and many sites mix plain pages with heavy ones. So most scripts pick one approach for everything: a plain HTTP fetch that misses dynamic content, or a browser for every page, which is slow and costly. Some sites also block plain requests or hide content behind a login, which adds sessions and retries to the mix. Then the waiting itself is tricky, because grabbing the page too early returns half-loaded markup.

This part describes the usual way. It makes no claims about Spicrawl, so nothing is quoted here.

02
How Spicrawl does it

Spicrawl is a web scraping API that has three tiers per request: a plain fetch, JavaScript rendering, or a full browser, at 1, 3 or 8 credits. With mode set to auto, it starts with the cheap plain fetch and moves up only if the page needs it or the site blocks it, and you pay for the tier that actually returned the page. You can tell it to wait for a CSS selector, like a product card, before reading, and get the result as Markdown, JSON or HTML. Failed requests and bot challenges cost nothing.

1, 3 or 8 credits by tier. mode: auto starts cheap and climbs only when needed.Quoted from spicrawl.com
Starts with a plain request and moves up to a browser only if the site blocks it. Blocked requests cost nothing.Quoted from spicrawl.com
Logins stay open with sessions, and browser actions can click, type and scroll first.Quoted from spicrawl.com

03
Step by step

  1. 1Sign up for a free API key during the beta.
  2. 2Send a POST to the scrape endpoint with the URL and mode set to auto.
  3. 3Add wait_for with a selector that only exists once the content has loaded.
  4. 4Pick the response format your next step needs, such as JSON for an app or Markdown for a prompt.
  5. 5Set max_cost so a request that would cost more is refused before it runs.
  6. 6Use a session if the page sits behind a login, and browser actions if you need to click or scroll first.

04
Tips

Pick a wait_for selector that belongs to the data, not the layout. A header shows up early, while a price element or a table row only appears once the real content is in. If the selector never appears, you learn that the page changed, instead of quietly storing an empty result.

General advice. Not a claim about the product.

Watch the response headers on your first runs. They show what each request cost and how many credits are left, which tells you how many of your pages really need a browser.

X-Credits-Charged shows what a request cost; X-Credits-Remaining shows what is left this month.Quoted from spicrawl.com

Do not count on the cache to save money. It makes repeat requests faster, but a cache hit costs the same as the fetch that stored it. Use it to speed up repeated runs, and budget credits as if every request were fresh.

The cache makes repeats faster, not cheaper.Quoted from spicrawl.com

Know what is not there yet. Managed proxies, stealth mode and a remote browser are listed as coming soon, so for tough sites today you bring your own proxy.

Coming soon: managed proxies, stealth mode, a remote browser and AI extraction from a plain-language prompt.Quoted from spicrawl.com

05
Questions

How many credits does a page cost?

It depends on the tier that served it. A plain fetch is 1 credit, JavaScript rendering is 3 and a full browser is 8.

A plain fetch is 1 credit. JavaScript rendering is 3. Full-browser rendering is 8.Quoted from spicrawl.com

Do failed requests use credits?

No. Errors, bot challenges and timeouts cost 0 credits.

Any error, including bot challenges and timeoutsQuoted from spicrawl.com

How much can I scrape for free?

1,000 credits a month during the beta, with no credit card. That is about 333 JavaScript pages or 125 browser pages.

Every account gets 1,000 credits a month and you do not need a credit card.Quoted from spicrawl.com

More from Spicrawl

All 2 answers