What is Amazon URL Parameters Construction
Amazon URL parameter construction refers to dynamically building complete page access links programmatically according to Amazon’s official URL structure rules. This technique is crucial in e-commerce data scraping, allowing developers to directly generate precise URLs for target pages without manually searching or navigating on Amazon’s website, thereby achieving automated data extraction.
A standard Amazon URL consists of three core components: base domain (such as https://www.amazon.com), page path (such as /s for search pages, /dp/ for product detail pages), and query parameters (starting with ?, containing multiple key=value pairs connected by &). Query parameters are the core of URL construction, controlling the specific content displayed on pages, including search keywords, category filtering, price ranges, sorting methods, pagination positions, etc.
Mastering URL parameter construction techniques can significantly improve data scraping efficiency and precision. For example, when using Pangolin Scrape API for data collection, you don’t need to visit Amazon’s website first, enter keywords, or set filter conditions. Instead, you directly pass the constructed URL as a parameter to the API to obtain target page data. This approach is not only faster but can also batch process thousands of different scraping requirements, achieving true automation and scalability.
Why URL Parameter Construction is Essential
In e-commerce data scraping scenarios, URL parameter construction is a fundamental skill for achieving precise and efficient data acquisition. Its core value is reflected in the following aspects:
1. Automated Batch Scraping
Suppose you need to scrape product data for 100 different keywords across 10 different price ranges. Manual operation would require 1,000 searches and filters on Amazon’s website. Through URL parameter construction, you only need to write a loop program to automatically generate 1,000 different URLs, then batch send them to the Scrape API to complete the collection.
2. Precise Control of Scraping Scope
URL parameters allow you to precisely specify the data range for collection. For example, you can obtain price information for specific regions through zip code parameters, filter products in specific price ranges through price parameters, and get results sorted by sales or ratings through sorting parameters. This precise control is difficult to achieve with manual operations.
3. Improved API Call Efficiency
When using Pangolin Scrape API, correct URL parameter construction ensures that accurate target data is obtained with a single request, avoiding repeated requests due to parameter errors, thus saving API call counts and costs. Meanwhile, standardized URL structures also help improve API response speed and stability.
4. Support for Complex Business Scenarios
In actual business, multiple filter conditions often need to be combined. For example, scraping products “priced between $50-200, rated 4 stars and above, with Prime badge, sorted by sales volume.” Such complex requirements can only be achieved through correct URL parameter combinations; manual operation is both inefficient and error-prone.
Amazon URL Parameter Official Rules Summary
Amazon’s URL parameter system is vast and complex. Here are the most commonly used core parameters and their usage rules:
| Parameter Name | Parameter Type | Description | Example Values |
|---|---|---|---|
k | Search Control | Search keyword, supports multi-word combinations | wireless+headphones bluetooth+speaker |
i | Search Control | Category ID, limits search scope to specific category | electronics aps (all departments) |
rh | Filter Conditions | Composite filter parameter, can include brand, rating, etc. | p_72:1249150011 (4+ stars) p_89:Sony (brand) |
low-price | Filter Conditions | Minimum price (most categories use cents) | 5000 (represents $50.00) |
high-price | Filter Conditions | Maximum price (most categories use cents) | 20000 (represents $200.00) |
s | Sorting Method | Result sorting rules | relevanceblender (relevance) price-asc-rank (price ascending) review-rank (review count) date-desc-rank (newest) |
page | Pagination Control | Page number (usually max 20 pages) | 1, 2, 3…20 |
ref | Tracking Identifier | Source tracking, simulates real user behavior | sr_pg_1 (search result page 1) nb_sb_noss (navigation bar search) |
qid | Tracking Identifier | Query timestamp | 1702284567 (Unix timestamp) |
node | Category Navigation | Category node ID, used for bestsellers and category browsing | 172282 (Electronics category) 283155 (Books category) |
field-keywords | Search Control | Keyword (legacy parameter, still used on some pages) | laptop |
bbn | Category Navigation | Browse Bin Number, category browsing number | 172282 |
ie | Encoding Setting | Character encoding | UTF8 |
tag | Affiliate Identifier | Amazon Associates affiliate ID | youraffid-20 |
Important Notes:
- Price parameters in most categories use cents as the unit, i.e., $50 should be written as 5000
- Spaces in parameter values need to be encoded as
+or%20 - Special characters (such as &, =, ?) need URL encoding
- While parameter order theoretically doesn’t affect results, it’s recommended to organize them in the order: k→i→filters→sorting→pagination
- Different sites (.com, .co.uk, .co.jp, etc.) may have different category IDs and certain parameter values
Practical Cases: URL Construction Application Scenarios
Case 1: Getting Keyword Search Results Page
Requirement: Scrape search results for keyword “wireless headphones” in Electronics category
Construction Logic:
Base URL: https://www.amazon.com/s Parameter combination: - k=wireless+headphones (keyword) - i=electronics (category) - ref=nb_sb_noss (source identifier) Complete URL: https://www.amazon.com/s?k=wireless+headphones&i=electronics&ref=nb_sb_noss
Using Pangolin Scrape API:
POST https://api.pangolinfo.com/scrape
{
"url": "https://www.amazon.com/s?k=wireless+headphones&i=electronics&ref=nb_sb_noss",
"type": "search",
"format": "json"
}
Case 2: Getting Products in Specific Price Range
Requirement: Scrape Bluetooth speakers priced between $50-$200, sorted by price ascending
Construction Logic:
Base URL: https://www.amazon.com/s Parameter combination: - k=bluetooth+speaker - i=electronics - low-price=5000 ($50 converted to cents) - high-price=20000 ($200 converted to cents) - s=price-asc-rank (price ascending) - ref=sr_st_price-asc-rank Complete URL: https://www.amazon.com/s?k=bluetooth+speaker&i=electronics&low-price=5000&high-price=20000&s=price-asc-rank&ref=sr_st_price-asc-rank
Case 3: Getting Best Sellers List Page
Requirement: Scrape Electronics category Best Sellers page 1
Construction Logic:
Base URL: https://www.amazon.com/gp/bestsellers Parameter combination: - node=172282 (Electronics category node ID) - ref=zg_bs_nav_electronics_0 Complete URL: https://www.amazon.com/gp/bestsellers/electronics/ref=zg_bs_nav_electronics_0
Note: Bestseller URL structure differs from search pages; category name is directly included in the path
Case 4: Getting Multiple Pages of Search Results
Requirement: Scrape first 5 pages of search results for keyword “laptop”
Construction Logic (Python Example):
base_url = "https://www.amazon.com/s"
keyword = "laptop"
category = "computers"
urls = []
for page in range(1, 6):
params = {
'k': keyword,
'i': category,
'page': page,
'ref': f'sr_pg_{page}'
}
url = f"{base_url}?{'&'.join([f'{k}={v}' for k, v in params.items()])}"
urls.append(url)
# Batch send to Pangolin API
for url in urls:
response = scrape_api.get(url)
Case 5: Combining Complex Filter Conditions
Requirement: Scrape laptops with 4+ stars, Prime badge, priced $100-$500
Construction Logic:
Base URL: https://www.amazon.com/s Parameter combination: - k=laptop - i=computers - rh=p_72:1249150011,p_85:2470955011 (4+ stars + Prime) - low-price=10000 - high-price=50000 - ref=sr_nr_p_72_1 Complete URL: https://www.amazon.com/s?k=laptop&i=computers&rh=p_72:1249150011,p_85:2470955011&low-price=10000&high-price=50000&ref=sr_nr_p_72_1
rh Parameter Explanation:
p_72:1249150011– 4 stars and above ratingp_85:2470955011– Prime products- Multiple conditions connected by commas
- Specific encoding values need to be obtained by analyzing Amazon pages
Python Implementation: URL Construction Utility Class
To improve URL construction efficiency and accuracy, it’s recommended to encapsulate a dedicated utility class. Here’s a complete Python implementation example:
Basic URL Builder
from urllib.parse import urlencode, quote_plus
from typing import Dict, Optional, List
class AmazonURLBuilder:
"""Amazon URL Builder"""
# Base URLs for different marketplaces
BASE_URLS = {
'us': 'https://www.amazon.com',
'uk': 'https://www.amazon.co.uk',
'jp': 'https://www.amazon.co.jp',
'de': 'https://www.amazon.de',
'ca': 'https://www.amazon.ca'
}
# Common sorting options
SORT_OPTIONS = {
'relevance': 'relevanceblender',
'price_asc': 'price-asc-rank',
'price_desc': 'price-desc-rank',
'review': 'review-rank',
'newest': 'date-desc-rank'
}
def __init__(self, marketplace: str = 'us'):
"""
Initialize URL builder
Args:
marketplace: Marketplace code (us, uk, jp, de, ca)
"""
self.base_url = self.BASE_URLS.get(marketplace, self.BASE_URLS['us'])
self.marketplace = marketplace
def build_search_url(
self,
keyword: str,
category: Optional[str] = None,
min_price: Optional[float] = None,
max_price: Optional[float] = None,
sort_by: str = 'relevance',
page: int = 1,
prime_only: bool = False,
min_rating: Optional[int] = None
) -> str:
"""
Build search page URL
Args:
keyword: Search keyword
category: Category ID (e.g., 'electronics', 'books')
min_price: Minimum price (USD)
max_price: Maximum price (USD)
sort_by: Sorting method
page: Page number
prime_only: Prime products only
min_rating: Minimum rating (1-5)
Returns:
Complete search URL
"""
params = {'k': keyword}
# Category
if category:
params['i'] = category
# Price range (convert to cents)
if min_price is not None:
params['low-price'] = int(min_price * 100)
if max_price is not None:
params['high-price'] = int(max_price * 100)
# Sorting
params['s'] = self.SORT_OPTIONS.get(sort_by, sort_by)
# Pagination
params['page'] = page
# ref parameter (simulate real user)
params['ref'] = f'sr_pg_{page}'
# Composite filter conditions
rh_parts = []
if prime_only:
rh_parts.append('p_85:2470955011')
if min_rating:
rating_codes = {
4: 'p_72:1249150011', # 4+ stars
3: 'p_72:2661618011', # 3+ stars
}
if min_rating in rating_codes:
rh_parts.append(rating_codes[min_rating])
if rh_parts:
params['rh'] = ','.join(rh_parts)
# Build complete URL
query_string = urlencode(params, quote_via=quote_plus)
return f"{self.base_url}/s?{query_string}"
def build_bestseller_url(
self,
category: str,
page: int = 1
) -> str:
"""
Build Best Sellers list URL
Args:
category: Category name (e.g., 'electronics', 'books')
page: Page number
Returns:
Bestseller URL
"""
if page == 1:
return f"{self.base_url}/gp/bestsellers/{category}"
else:
return f"{self.base_url}/gp/bestsellers/{category}/ref=zg_bs_pg_{page}?ie=UTF8&pg={page}"
def build_product_url(self, asin: str) -> str:
"""
Build product detail page URL
Args:
asin: Product ASIN code
Returns:
Product detail page URL
"""
return f"{self.base_url}/dp/{asin}"
# Usage examples
builder = AmazonURLBuilder(marketplace='us')
# Example 1: Basic search
url1 = builder.build_search_url(
keyword='wireless headphones',
category='electronics'
)
print("Basic search:", url1)
# Example 2: With price filtering and sorting
url2 = builder.build_search_url(
keyword='laptop',
category='computers',
min_price=500,
max_price=1500,
sort_by='price_asc',
page=1
)
print("Price filtering:", url2)
# Example 3: Prime products + high rating
url3 = builder.build_search_url(
keyword='bluetooth speaker',
prime_only=True,
min_rating=4,
sort_by='review'
)
print("Prime + high rating:", url3)
# Example 4: Bestseller URL
url4 = builder.build_bestseller_url(category='electronics', page=2)
print("Bestseller page:", url4)
Integration with Pangolin Scrape API
After constructing URLs, you can directly integrate with Pangolin Scrape API to achieve automated data collection. Here’s a complete integration example:
import requests
from typing import Dict, List
class PangolinScraper:
"""Pangolin API Integration Class"""
def __init__(self, api_key: str):
"""
Initialize Pangolin scraper
Args:
api_key: Pangolin API key
"""
self.api_key = api_key
self.api_url = 'https://api.pangolinfo.com/scrape'
self.url_builder = AmazonURLBuilder()
def scrape_search(
self,
keyword: str,
**kwargs
) -> Dict:
"""
Scrape search results
Args:
keyword: Search keyword
**kwargs: Other URL parameters
Returns:
Parsed data
"""
# Build URL
url = self.url_builder.build_search_url(keyword, **kwargs)
# Call Pangolin API
payload = {
'api_key': self.api_key,
'url': url,
'type': 'search',
'format': 'json'
}
response = requests.post(self.api_url, json=payload)
response.raise_for_status()
return response.json()
def scrape_bestsellers(
self,
category: str,
max_pages: int = 5
) -> List[Dict]:
"""
Scrape Best Sellers list
Args:
category: Category
max_pages: Maximum pages
Returns:
List of data from all pages
"""
all_data = []
for page in range(1, max_pages + 1):
url = self.url_builder.build_bestseller_url(category, page)
payload = {
'api_key': self.api_key,
'url': url,
'type': 'bestsellers',
'format': 'json'
}
response = requests.post(self.api_url, json=payload)
if response.status_code == 200:
data = response.json()
all_data.append(data)
else:
print(f"Page {page} scraping failed")
break
return all_data
# Usage examples
scraper = PangolinScraper(api_key='your_api_key_here')
# Example 1: Scrape search results
search_data = scraper.scrape_search(
keyword='wireless mouse',
category='electronics',
min_price=10,
max_price=50,
prime_only=True
)
print(f"Found {len(search_data.get('products', []))} products")
# Example 2: Scrape bestsellers
bestseller_data = scraper.scrape_bestsellers(
category='electronics',
max_pages=3
)
print(f"Scraped {len(bestseller_data)} pages of bestseller data")
Professional Tools: The Dimensional Reduction Advantage
Although mastering URL parameter construction techniques can solve most data scraping needs, in actual business scenarios, manually handling these complex technical details often consumes substantial development and maintenance costs. This is precisely the value of professional data scraping API services—they encapsulate complex URL construction logic, anti-scraping countermeasures, data parsing, and other aspects, allowing developers to focus on business logic rather than underlying technical details.
Take Pangolin Scrape API as an example; it provides highly abstracted interfaces where developers only need to specify business parameters like keywords, categories, zip codes, and price ranges, and the API automatically handles URL construction, request sending, data parsing, and all other technical aspects. More importantly, Pangolin has unique technical advantages in ad placement scraping, achieving 98% capture rates, which ordinary crawlers struggle to match. For scenarios requiring precise tracking of competitor advertising strategies, this high capture rate means data completeness and analysis conclusion reliability.
Additionally, Pangolin supports specified zip code scraping, hourly batch updates, and rich data field extraction, including product descriptions, Customer Says, and other deep information. The realization of these capabilities relies on deep understanding and continuous optimization of Amazon’s URL parameter system. For small and medium-sized sellers or tool companies without professional crawler teams, using mature API services like Pangolin can achieve in days what would take months to develop, while avoiding issues faced by self-built crawlers like IP bans, data instability, and high maintenance costs.
If your team lacks API integration capabilities or only needs small-scale data monitoring, consider using the AMZ Data Tracker plugin. This is a zero-code configuration solution supporting minute-level scheduled scraping, anomaly alerts, and precise ASIN full data extraction. Through a visual interface, you can easily set up scraping tasks without writing a single line of code, achieving real-time tracking of competitor new products, price changes, and sales fluctuations. This tool is particularly suitable for operations personnel, lowering the technical barrier to data scraping and enabling more people to enjoy the benefits of data-driven decision-making.
Conclusion: From Technical Details to Strategic Advantages
Amazon URL parameter construction seems like a purely technical issue but actually relates to data scraping quality, efficiency, and cost. Mastering core URL construction rules, understanding different parameters’ action mechanisms and combination techniques, can establish significant competitive advantages in the data scraping field. Whether it’s precise zip code positioning, flexible price filtering, efficient category traversal, or stable ad placement extraction, all depend on fine control of URL parameters.
However, technology is just a means; the ultimate goal is serving business objectives. When choosing between building your own crawler or using professional API services, you need to comprehensively consider team technical capabilities, time costs, data quality requirements, and other factors. For sellers and tool companies of certain scale needing personalized data analysis and hoping to escape homogeneous competition, investing resources to deeply research URL parameter construction technology is worthwhile; for teams pursuing rapid launch and stable reliability, choosing mature API services like Pangolin might be a wiser decision.
The essence of data scraping is competition in information acquisition capabilities. On Amazon, a platform with highly asymmetric information, whoever can obtain data faster, more accurately, and more comprehensively gains the upper hand in critical aspects like product selection, pricing, and ad placement. URL parameter construction technology is precisely the indispensable foundational capability in this competition. I hope this article’s in-depth analysis and practical code help you avoid detours on the data scraping journey, faster achieving the closed loop from data to insights, from insights to action.