Amazon product pages carry more scrapable fields than just price — title, price, star rating, review count, ASIN, images, availability, and Buy Box seller are all present in the page's own HTML, no login required.
Amazon's robots.txt and Conditions of Use restrict automated access, and this session's own fetch tooling was refused when it tried to load an amazon.com URL directly — treat any scraping project as needing a compliance review, not just a technical one.
Amazon's official Product Advertising API (PA-API 5.0) is deprecated. Calls now return an HTTP 403 telling callers to migrate to the new Creators API, which requires active Amazon Associates enrollment plus at least 10 qualifying sales in the past 30 days — a bar most non-affiliate teams can't clear.
That gap is exactly why a managed crawling API is the practical middle path between the now-gated official API and building an anti-bot stack from scratch — Nstdata Crawl handles JavaScript rendering and proxy routing so the tutorial code below only has to handle parsing.
A single product page and a search-results page use different selector patterns. Product pages key off IDs like #productTitle; search-results pages key off the data-asin attribute on each [data-component-type="s-search-result"] block.
The same scrape can become a price tracker with about ten extra lines of code — the Bonus Tip below turns the one-off request into a scheduled loop, and Nstdata Proxy Manager is the natural next step once that loop runs across dozens of SKUs or marketplaces at once.
What Data You Can Actually Scrape From Amazon
Amazon product and search-results pages expose product title, current price, star rating, review count, ASIN, primary image URL, stock/availability status, and — on the product page — the seller currently winning the Buy Box, all inside HTML that renders without logging in. Customer reviews carry additional fields (review text, star rating, "verified purchase" flag) but also a reviewer display name, which counts as personal data and deserves the same minimization treatment as any other personal identifier collected from a public page.
Teams typically pull this data for one of four reasons: competitive price monitoring across a catalog, market research on category rankings and review sentiment, MAP (minimum advertised price) compliance checks for a brand's own listings, and lead or assortment research before entering a new marketplace. All four are variations on the same underlying job — fetch a page, extract a stable set of fields, repeat on a schedule — which is the pattern this tutorial builds.
Take a Quick Look
Amazon renders prices and availability with JavaScript and blocks most unrotated IPs within a handful of requests — Nstdata Crawl handles the browser rendering and proxy routing in one API call, so the code below only has to parse the page it gets back.
Legality depends on what's collected and how, not on whether scraping Amazon is possible at all. Amazon's robots.txt disallows automated crawling of most paths — this article's own research tooling was refused when it tried to fetch an amazon.com URL directly, which is a useful illustration of how broadly that restriction is applied — and Amazon's Conditions of Use separately prohibit data-mining tools without permission. Neither of those facts settles the legal question by itself: in hiQ Labs v. LinkedIn, the Ninth Circuit found that scraping publicly accessible profile data likely didn't violate the Computer Fraud and Abuse Act, and in Van Buren v. United States, the Supreme Court narrowed the CFAA to unauthorized access rather than improper use of otherwise-accessible data. Craigslist v. 3Taps points the other way: continuing to scrape after an explicit block or cease-and-desist notice substantially raises legal exposure.
In practice, lower-risk practice looks like collecting publicly visible listings, prices, and availability at a reasonable request rate, focusing on marketplace metadata rather than reviewer personal information, and preferring an official API when a project qualifies for one. Higher-risk practice looks like scraping behind a login, harvesting personal or payment data, bypassing CAPTCHA or other explicit technical blocks, and continuing after Amazon has already throttled or blocked the source. This tutorial stays in the first category: publicly visible catalog data, no authentication bypass, and no guidance on defeating CAPTCHA challenges.
Official API vs. DIY Scraper vs. Managed Crawling API
Three paths reach the same product data, and the choice mostly comes down to eligibility and how much anti-bot infrastructure a team wants to own. Amazon's own Product Advertising API (PA-API 5.0) was the lowest-risk source for years, but Amazon has deprecated it — a call to the old endpoint now returns an HTTP 403 instructing developers to migrate to the new Creators API, which requires active enrollment in the Amazon Associates program and at least 10 qualifying sales in the past 30 days. That eligibility bar rules out most teams doing price monitoring, market research, or MAP compliance who aren't already running an active affiliate storefront. A fully DIY scraper (requests or Playwright plus BeautifulSoup) is the second path, but it means building and maintaining JavaScript rendering, proxy rotation, and retry logic in-house before writing a single line of extraction code, and Amazon changes its markup often enough that the maintenance cost compounds. Nstdata Crawl is the third path, and the rest of this tutorial builds on it.
Nstdata Crawl is an API that turns a URL into Markdown, cleaned HTML, or raw page data through a single request, with JavaScript rendering in a real browser and Nstdata's own proxy pool handling the network layer underneath — the two pieces a DIY Amazon scraper otherwise has to assemble itself. It fits this job specifically because Amazon's product and search pages depend on client-side rendering for price and availability, and because a scraper hitting Amazon from a single static IP gets rate-limited or blocked within a small number of requests. The API bills per successful fetch rather than per attempt, so a page that returns a 403 or 404 still counts as billable (the request itself succeeded) but a retry that never reaches Amazon at all does not.
JavaScript rendering included — product and search pages that depend on client-side scripts for price and stock status render fully before the API returns a result, instead of returning a half-loaded shell.
Proxy routing handled automatically — requests route through Nstdata's residential or datacenter pool without a separate proxy vendor or IP-rotation script to maintain.
Markdown or HTML output — the tutorial below parses returned HTML directly, but Markdown output works just as well for teams feeding results into an LLM-based summarizer instead of a fixed parser.
Prerequisites
Before writing any code, get an API key from the Nstdata dashboard and install the Python packages this tutorial uses:
pip install requests beautifulsoup4
requests calls the Nstdata Crawl API itself; beautifulsoup4 parses the HTML that comes back. No Amazon account, login, or browser installation is required on your end — Nstdata Crawl runs the browser side.
Step 1: Scrape a Single Amazon Product Page
Send the product URL to Nstdata Crawl's scrape endpoint, requesting both markdown and html output. Appending ?async=true makes the call synchronous — it waits for the page to finish rendering and returns the result directly, rather than handing back a task ID to poll:
import requests
API_KEY ="YOUR_API_KEY"PRODUCT_URL ="https://www.amazon.com/dp/B0BSHF7WHW"# replace with a real ASIN URLresponse = requests.post("https://api.nstdata.io/api/v1/crawl/scrape?async=true", headers={"x-api-key": API_KEY,"Content-Type":"application/json"}, json={"url": PRODUCT_URL,"formats":["html"],"onlyMainContent":False,"timeout":30000,},)result = response.json()ifnot result.get("success"):raise RuntimeError(f"Scrape failed: {result.get('errorMessage')}")page_html = result["data"]["html"]
Checking result["success"] matters here — an HTTP 200 only confirms Nstdata Crawl received the request, not that Amazon returned a usable product page. onlyMainContent is set to False because the price and Buy Box widgets on an Amazon product page sit outside what a generic "main content" heuristic would keep.
Step 2: Parse Title, Price, Rating, and ASIN
Amazon's product page keys its core fields off stable element IDs rather than generated CSS class names, which is what makes them worth targeting directly instead of scraping visible text:
from bs4 import BeautifulSoup
import re
soup = BeautifulSoup(page_html,"lxml")title_el = soup.select_one("#productTitle")title = title_el.get_text(strip=True)if title_el elseNoneprice_el = soup.select_one("#corePrice_feature_div span.a-offscreen")price = price_el.get_text(strip=True)if price_el elseNonerating_el = soup.select_one("#acrPopover")rating = rating_el.get("title","").replace(" out of 5 stars","")if rating_el elseNoneasin_match = re.search(r"/dp/([A-Z0-9]{10})", page_html)# ASIN lives in the canonical URL, not a dedicated elementasin = asin_match.group(1)if asin_match elseNoneproduct ={"asin": asin,"title": title,"price": price,"rating": rating}print(product)
Each select_one call is guarded with a fallback to None rather than letting a missing element raise an exception — Amazon runs different page layouts for A/B tests, so a selector that matches most listings will occasionally miss one. Building each field defensively, the way BeautifulSoup and lxml are typically used together for HTML parsing, keeps one missing widget from crashing an entire batch job.
Step 3: Scrape an Amazon Search Results Page for Multiple ASINs
A search-results page returns many products in one request, which is more efficient than scraping product pages one at a time when the goal is a category snapshot rather than deep detail on a single item. The request to Nstdata Crawl looks the same as Step 1, just pointed at a search URL:
Each result card carries its ASIN in a data-asin attribute, which is a far more stable anchor than any visual layout — filtering out cards with no data-asin also quietly drops sponsored placements and layout widgets that aren't real product results.
Sample Output Schema
Whether the source was a single product page or a search-results page, normalizing to the same schema keeps downstream storage and comparison simple:
Storing scraped_at alongside the rest of the record is what makes the next section possible — without a timestamp, there's no way to tell whether a price changed or the scrape just ran at a different moment.
Bonus Tip: Turn This Into an Always-On Price Tracker
A single scrape answers "what does this cost right now"; a scheduled loop answers "did this change" — which is the more useful question for MAP compliance or competitive monitoring. Wrapping Step 1's request in a scheduler and diffing against the last stored price gets there with a small addition:
Swap the print for a Slack webhook or email call and this becomes an actual alerting pipeline rather than a console log. Once that loop runs against dozens of ASINs across multiple marketplaces or regional domains rather than one, the operational bottleneck usually shifts from parsing logic to proxy and credential management — that's the point at which centralizing routing rules, per-project pool allocation, and monitoring through Nstdata Proxy Manager is worth adding on top of the scraping logic itself, rather than hand-rolling proxy assignment per script.
Observations and Limits
Amazon's markup changes often enough that any selector list, including the ones above, should be treated as current-as-of-today rather than permanent — defensive None-checks on every field are what keep a layout tweak from crashing a batch job instead of just returning one incomplete record. Rate limits are real even with proxy rotation handled: spacing requests out and keeping concurrency modest reduces the chance of tripping Amazon's bot-detection thresholds, and no proxy layer makes an unbounded, high-concurrency crawl of Amazon a good idea. Review data carries reviewer display names, which is personal data even when publicly visible — minimize what's stored and for how long if reviews are part of a scrape. Finally, none of the code above works against pages that require solving a CAPTCHA challenge or logging in; both are explicit signals to stop rather than problems to route around.
Conclusion
Scraping Amazon product data is less about finding a clever bypass and more about picking the right layer to solve the JavaScript-rendering and IP-reputation problems that a static requests.get() call can't handle alone. With Amazon's official Product Advertising API now gated behind an Associates-plus-sales-volume requirement most teams won't meet, a managed crawling API that handles rendering and proxy routing — paired with the same defensive parsing patterns shown above — is the practical middle ground between an over-restricted official API and a self-maintained anti-bot stack.
Scraping publicly visible listing data (price, title, rating) carries lower legal risk than scraping behind a login or continuing after an explicit block, based on case law like hiQ Labs v. LinkedIn and Craigslist v. 3Taps, but Amazon's own robots.txt and Conditions of Use restrict automated access — reviewing both before starting a project matters more than any single technical trick.
Q: Can I use Amazon's official API instead of scraping?
Amazon's Product Advertising API (PA-API 5.0) is deprecated and now returns an HTTP 403 directing callers to the Creators API, which requires active Amazon Associates enrollment plus at least 10 qualifying sales in the past 30 days — a bar that excludes most teams doing price monitoring or market research rather than running an affiliate storefront.
Q: Why does a static requests.get() call fail on Amazon product pages?
Price and availability widgets on Amazon product pages depend on client-side JavaScript to finish rendering, so a plain HTTP client without a JavaScript-capable renderer sitting behind it often receives an incomplete page or gets blocked outright after a handful of requests from the same IP.
Q: How do I find a product's ASIN without opening the page?
On a search-results page, each result block carries its ASIN directly in a data-asin attribute ([data-component-type="s-search-result"]), which avoids needing to open every product page just to collect its identifier.
Q: Do I need a proxy to scrape Amazon at any real scale?
Yes for anything beyond a handful of manual requests — Amazon rate-limits and blocks based on IP reputation and request patterns, so a single static IP degrades quickly, while a rotating residential or datacenter pool keeps a monitoring job running without constant manual unblocking.
Q: What should I do if I hit a CAPTCHA?
Stop and back off rather than trying to solve or route around it — a CAPTCHA is an explicit signal from Amazon that the current request pattern looks automated, and continuing anyway is squarely in the higher-risk category described above.
Q: Can I scrape Amazon customer reviews?
Review text and star ratings are visible on the page the same way price and title are, but reviews also include a reviewer display name, which is personal data — minimizing what's stored, for how long, and for what purpose is worth deciding before collecting review data rather than after.
Lena Zhou
Sep. 15th 2026
110M+ real IPs with 99.9% access success
Blazing-fast average response ~0.5s for high-concurrency tasks
From only $0.1/GB
Get immediate access to premium residential, datacenter, IPv6 and ISP proxy pools.
