Quick Start
This guide on how to use ScrapingBee covers every feature:
- Getting Started — Create your account and get your api key
- How to Use AI Web Scraping — pull complex structured information from web pages using plain English
- How to Use Google Search API — scrape Google search results without managing proxy ips yourself
- How to Use Google Shopping Scraper — collect product listings and the extracted price for any search term
- How to Use JavaScript Web Scraping — handle javascript rendering on web pages built with React or Vue
- How to Use ChatGPT Scraper — capture responses and page source from dynamic AI chat pages
- How to Use Google News Scraper API — pull headlines and full articles through simple api requests
- How to Use Fast Data Scraping — run fast data scraping while ScrapingBee rotates proxies for you
- How to Use Programmatic Screenshot API — capture a full page screenshot of any url programmatically
- How to Use Scrape Google Jobs — gather job listings as complex structured information
Time needed: 5 minutes per feature
Also in this guide: Pro Tips | Common Mistakes | Troubleshooting | Pricing | Alternatives
Why Trust This Guide
I have used ScrapingBee for over a year and tested every feature here. This tutorial comes from real hands-on scraping, not vendor screenshots.
ScrapingBee is a cloud based scraping tool that handles the hard parts of web scraping.
It manages headless browsers and rotates proxies for you.
This guide shows you how to use every major feature step by step.
With screenshots, a simple example, and pro tips throughout.
ScrapingBee Tutorial
This complete ScrapingBee tutorial walks you through every feature, from your first api request to advanced javascript scenario tricks.
ScrapingBee
Scrape any website without managing proxy ips or headless browsers. ScrapingBee handles javascript rendering through one simple rest api. Get 1000 free api calls to start — no credit card needed.
Getting Started with ScrapingBee
Before using any feature, complete this one-time setup.
It takes about three minutes.
Here is a quick look at the workflow:
Now let us walk through each step.
Step 1: Create Your Account
Go to the ScrapingBee website and click “Sign Up”.
Enter promo code ADNAN through the referral link for extra api credit.
✓ Checkpoint: Your dashboard shows your unique api key.
Step 2: Set Up Your Code
In Python, run pip install scrapingbee or just import requests.
The requests library lets you call the api from any programming language.
✓ Checkpoint: Your environment is ready to send api requests.
Step 3: Make Your First Call
Send a request to the https://app.scrapingbee.com/api/v1/ endpoint with your api key and url.
This simple example lets you scrape web pages with a few lines of complete code.
✅ Done: You are ready to use any feature below.
How to Use ScrapingBee AI Web Scraping
AI Web Scraping lets you pull complex structured information from web pages using plain English.
Here is how to use it step by step.
Step 1: Get Your API Key
Sign up and copy your api key from the dashboard.
Step 2: Describe the Data
Add the target url and write a plain-English query.
Here is what this looks like:
✓ Checkpoint: A clean JSON object appears with your fields.
Step 3: Set Extraction Rules
Adjust your extraction rules to capture each field exactly.
✅ Result: Structured JSON without writing css selectors by hand.
💡 Pro Tip: Pair AI extraction with css selectors when you need precise control.
How to Use ScrapingBee Google Search API
Google Search API lets you scrape Google search results without managing proxy ips yourself.
Here is how to use it step by step.
Step 1: Choose the Endpoint
Point your request at the Google Search API.
Step 2: Set the Query
Add your keyword and a random country for local results.
Here is what this looks like:
✓ Checkpoint: Ranked results return as structured data.
Step 3: Read the JSON
Parse the response with the requests library in Python.
✅ Result: Search data through one simple rest api call.
💡 Pro Tip: Use country_code for premium proxy geolocation on every query.
How to Use ScrapingBee Google Shopping Scraper
Google Shopping Scraper lets you collect product listings and the extracted price for any search term.
Here is how to use it step by step.
Step 1: Send a Shopping Request
Call the shopping endpoint with your product query.
Step 2: Parse the Response
Map each listing to title, link, and extracted price.
Here is what this looks like:
✓ Checkpoint: Product rows return as clean JSON.
Step 3: Export the Results
Save the data to CSV for further analysis.
✅ Result: Live pricing data ready for any spreadsheet.
💡 Pro Tip: Schedule runs to track the extracted price over time.
How to Use ScrapingBee JavaScript Web Scraping
JavaScript Web Scraping lets you handle javascript rendering on web pages built with React or Vue.
Here is how to use it step by step.
Step 1: Enable render_js
Set render_js=true so ScrapingBee uses headless browsers.
Step 2: Wait for an Element
Tell it to wait for a particular element to load.
Here is what this looks like:
✓ Checkpoint: The full page renders before extraction.
Step 3: Capture the HTML
Return the original html or raw page source after rendering.
✅ Result: Fully rendered web pages without web drivers to manage.
💡 Pro Tip: Add a javascript scenario for clicking, scrolling, or form filling.
How to Use ScrapingBee ChatGPT Scraper
ChatGPT Scraper lets you capture responses and page source from dynamic AI chat pages.
Here is how to use it step by step.
Step 1: Target the Page
Send the page url with render_js switched on.
Step 2: Render and Extract
Let headless scraping load the reply, then grab the text.
Here is what this looks like:
✓ Checkpoint: The rendered answer returns in the page source.
Step 3: Save the Output
Store the parsed html for later processing.
✅ Result: Clean text pulled from a fully dynamic page.
💡 Pro Tip: Chain a javascript scenario to walk through multi-step prompts.
How to Use ScrapingBee Google News Scraper API
Google News Scraper API lets you pull headlines and full articles through simple api requests.
Here is how to use it step by step.
Step 1: Set the News Endpoint
Aim your request at the Google News API.
Step 2: Add Filters
Filter by topic, language, or random country.
Here is what this looks like:
✓ Checkpoint: Matching headlines return as structured data.
Step 3: Collect Articles
Loop through results to gather each story.
✅ Result: Fresh news data straight from the scrapingbee api.
💡 Pro Tip: Rotate the country code to compare regional coverage.
How to Use ScrapingBee Fast Data Scraping
Fast Data Scraping lets you run fast data scraping while ScrapingBee rotates proxies for you.
Here is how to use it step by step.
Step 1: Build Your Request
Assemble the url, api key, and parameters once.
Step 2: Run Concurrent Calls
Fire many api requests at the same time.
Here is what this looks like:
✓ Checkpoint: Responses stream back in parallel.
Step 3: Check Status Codes
Confirm a 200 http code on every status code returned.
✅ Result: High-volume scraping handled on remote machines.
💡 Pro Tip: ThreadPoolExecutor speeds up scraping web pages dramatically.
How to Use ScrapingBee Programmatic Screenshot API
Programmatic Screenshot API lets you capture a full page screenshot of any url programmatically.
Here is how to use it step by step.
Step 1: Add the Screenshot Flag
Append screenshot=true to your api request.
Step 2: Choose Full Page
Set full_page=true to capture the entire layout.
Here is what this looks like:
✓ Checkpoint: A full page image renders edge to edge.
Step 3: Download the Image
Save the returned file to disk.
✅ Result: Pixel-perfect captures generated on demand.
💡 Pro Tip: The full_page parameter ensures the whole page renders first.
How to Use ScrapingBee Scrape Google Jobs
Scrape Google Jobs lets you gather job listings as complex structured information.
Here is how to use it step by step.
Step 1: Call the Jobs Endpoint
Send your role and location to the jobs API.
Step 2: Filter Listings
Narrow results by title, company, or city.
Here is what this looks like:
✓ Checkpoint: Job rows return as clean JSON.
Step 3: Structure the Data
Parse html fields into a tidy table.
✅ Result: Job data ready to parse and store.
💡 Pro Tip: Pair with css selectors to pull custom fields.
ScrapingBee Pro Tips and Shortcuts
After testing ScrapingBee for over a year, here are my best tips for scraping websites.
Key API Parameters
| Parameter | What It Does |
|---|---|
| render_js | Renders javascript in headless browsers |
| premium_proxy | Switches to residential proxy ips |
| country_code | Sets premium proxy geolocation by random country |
| block_ads | Lets you block ads to speed up requests |
Hidden Features Most People Miss
- session_id: Routes api requests through the same ip address for five minutes, so you keep the same http code and session.
- return_page_source: Returns the original html and full page source before any javascript runs.
- Custom headers and cookies: Send your own custom headers and custom cookies, or fall back to the default headers ScrapingBee generates.
- Header control: Swap an existing header for a new header, display headers generated automatically, then keep only particular headers and drop the other headers.
ScrapingBee Common Mistakes to Avoid
Mistake #1: Ignoring robots.txt before you scrape
❌ Wrong: Firing requests at any site you want to scrape without checking the rules first.
✅ Right: Check the robots.txt file and terms of service before scraping web pages.
Mistake #2: Forgetting to enable JavaScript rendering
❌ Wrong: Expecting dynamic content but getting an empty page source every time.
✅ Right: Set render_js=true so headless browsers load the full page first.
Mistake #3: Sending only the default headers
❌ Wrong: Relying on default headers alone, which some sites flag and block.
✅ Right: Send your own custom headers and a real user agent parameter when you want to scrape.
ScrapingBee Troubleshooting
Problem: Empty or partial page source
Cause: The javascript on the page never rendered, so the html came back blank.
Fix: Set render_js=true and wait for a particular element to load the full page.
Problem: Requests keep getting blocked
Cause: The default datacenter proxy ips were flagged by the target site.
Fix: Enable premium proxies to rotate in a new ip address from a random country.
Problem: You see a 401 or 403 http code
Cause: Your api key is wrong, or you are out of api credit.
Fix: Confirm the api key, check your status code, and top up credits if needed.
📌 Note: If none of these fix your issue, contact ScrapingBee support.
What is ScrapingBee?
ScrapingBee is a cloud based scraping tool that handles headless browsers and rotates proxies for you.
Think of it as a hosted browser you control through a rest api.
Watch this quick overview:
It includes these key features:
- AI Web Scraping: Lets you pull complex structured information from web pages using plain English.
- Google Search API: Lets you scrape Google search results without managing proxy ips yourself.
- Google Shopping Scraper: Lets you collect product listings and the extracted price for any search term.
- JavaScript Web Scraping: Lets you handle javascript rendering on web pages built with React or Vue.
- ChatGPT Scraper: Lets you capture responses and page source from dynamic AI chat pages.
- Google News Scraper API: Lets you pull headlines and full articles through simple api requests.
- Fast Data Scraping: Lets you run fast data scraping while ScrapingBee rotates proxies for you.
- Programmatic Screenshot API: Lets you capture a full page screenshot of any url programmatically.
- Scrape Google Jobs: Lets you gather job listings as complex structured information.
For a full review, see our ScrapingBee review.
ScrapingBee Pricing
Here is what ScrapingBee costs in 2026:
| Plan | Price | Best For |
|---|---|---|
| Freelance | $49/month | Solo developers and small projects |
| Startup | $99/month | Growing teams scaling api requests |
| Business | $249/month | High-volume web scraping workloads |
| Business+ | $599/month | Enterprise teams on remote machines |
Free trial: Yes — 1000 free api calls, no credit card required.
Money-back guarantee: Cancel anytime; billing is monthly with no lock-in.
💰 Best Value: Startup — the best balance of api credit and price for most teams.
ScrapingBee vs Alternatives
How does ScrapingBee compare? Here is the competitive landscape:
Watch this comparison:
| Tool | Best For | Price | Rating |
|---|---|---|---|
| ScrapingBee | JavaScript rendering via one rest api | $49/mo | ⭐ 4.5 |
| Apify | Ready-made scraper actors | Varies | ⭐ — |
| ScrapeGraphAI | AI-driven extraction rules | Varies | ⭐ — |
| Bright Data | Enterprise proxy ips | Varies | ⭐ — |
| Browse AI | No-code point and click | Varies | ⭐ — |
| Oxylabs | Premium enterprise proxies | Varies | ⭐ — |
| ThorData | Budget proxy network | Varies | ⭐ — |
| Zyte | Developer scraping api | Varies | ⭐ — |
| Webscraper.io | Browser extension capture | Varies | ⭐ — |
| Helium Scraper | Desktop site mapping | Varies | ⭐ — |
| Firecrawl | LLM-ready markdown | Varies | ⭐ — |
Quick picks:
- Best overall: ScrapingBee — one rest api for rendering, search, and screenshots.
- Best budget: ThorData — cheap proxy ips for simple scraping purposes.
- Best for beginners: Browse AI — no code and no programming language needed.
- Best for enterprise: Bright Data — built for huge, compliant scraping operations.
🎯 ScrapingBee Alternatives
Looking for ScrapingBee alternatives? Here are the top options:
- 🚀 Apify: A full scraping platform with ready-made actors and a generous free tier for testing ideas fast.
- 🧠 ScrapeGraphAI: An AI-first scraper that turns plain prompts into extraction rules for complex structured information.
- 🏢 Bright Data: Enterprise-grade proxy ips and datasets built for very large scraping operations and strict compliance.
- 👶 Browse AI: A no-code tool that records your clicks to build scrapers without writing any code.
- 🔒 Oxylabs: A premium proxy and scraper API provider with strong reliability for demanding enterprise workloads.
- 💰 ThorData: A budget-friendly proxy network covering residential and datacenter proxy ips for everyday scraping jobs.
- 🔧 Zyte: A developer-focused scraping API, formerly Scrapinghub, with smart bans handling and clean page source.
- 🎨 Webscraper.io: A browser-extension scraper that builds sitemaps visually inside Chrome for point-and-click data capture.
- 💼 Helium Scraper: A desktop scraping app for Windows that maps complex sites without any cloud dependency.
- ⚡ Firecrawl: An AI crawler that converts whole sites into clean markdown for feeding large language models.
For the full list, see our ScrapingBee alternatives guide.
⚔️ ScrapingBee Compared
Here is how ScrapingBee stacks up against each competitor:
- ScrapingBee vs Apify: Apify offers a marketplace of scrapers; ScrapingBee wins on simple rest api access and faster setup.
- ScrapingBee vs ScrapeGraphAI: ScrapeGraphAI leans fully on AI; ScrapingBee gives you both AI extraction and raw css selectors.
- ScrapingBee vs Bright Data: Bright Data scales to enterprise; ScrapingBee is cheaper and simpler for small to mid teams.
- ScrapingBee vs Browse AI: Browse AI suits non-coders; ScrapingBee fits developers who want a rest api and full control.
- ScrapingBee vs Oxylabs: Oxylabs targets large enterprises; ScrapingBee offers lower entry pricing and an easier learning curve.
- ScrapingBee vs ThorData: ThorData competes on proxy price; ScrapingBee bundles rendering, screenshots, and AI into one api.
- ScrapingBee vs Zyte: Zyte has deep developer roots; ScrapingBee is friendlier for quick projects and small teams.
- ScrapingBee vs Webscraper.io: Webscraper.io runs in the browser; ScrapingBee handles javascript rendering server-side at real scale.
- ScrapingBee vs Helium Scraper: Helium Scraper is desktop-only; ScrapingBee runs in the cloud so you skip web drivers entirely.
- ScrapingBee vs Firecrawl: Firecrawl targets LLM pipelines; ScrapingBee covers broader scraping with screenshots and search APIs.
Start Using ScrapingBee Now
You learned how to use every major ScrapingBee feature:
- ✅ AI Web Scraping
- ✅ Google Search API
- ✅ Google Shopping Scraper
- ✅ JavaScript Web Scraping
- ✅ ChatGPT Scraper
- ✅ Google News Scraper API
- ✅ Fast Data Scraping
- ✅ Programmatic Screenshot API
- ✅ Scrape Google Jobs
Next step: Pick one feature and try it now.
Most people start with AI Web Scraping.
A popular first project is creating an OLX scraper to pull listings.
During development ScrapingBee runs the browser on remote machines for you.
Explore further features once your first OLX scraper works.
It takes less than five minutes to scrape your first page.
Frequently Asked Questions
How does ScrapingBee work?
ScrapingBee is a cloud based scraping tool. You send a request with your api key and a url, and it returns the page using headless browsers while it rotates proxies for you.
How much does ScrapingBee cost?
Plans start at $49/month for Freelance, then $99 Startup, $249 Business, and $599 Business+. A free trial gives you 1000 api calls to test scraping purposes.
Is web scraping illegal?
Scraping public web pages is generally legal. Always check a site’s robots.txt and terms first, avoid personal data, and respect rate limits to stay on the safe side.
Does ScrapingBee support JavaScript rendering?
Yes. Set render_js=true and ScrapingBee loads web pages in headless browsers. This javascript rendering works on sites built with React or Vue out of the box.
Which programming language does ScrapingBee support?
Any. ScrapingBee is a rest api, so you call it from Python, Node, PHP, or any language. In Python just import requests and send your api requests.
English 
Spanish 
Chinese 
German 
French 
Arabic 
Russian 
Portuguese 
Japanese 
Korean 
Italian 
Turkish 
Dutch 
Polish 
Indonesian 
Thai 
Vietnamese 