Amazon is one of the world’s largest e-commerce platforms, boasting a vast array of product information and user reviews. This makes it a valuable data source for e-commerce operators, market analysts, product developers, and more. However, Amazon’s page structure is complex, data distribution is uneven, and collecting data can be challenging. If traditional web scraping techniques are used, various problems and difficulties may arise, such as:

• Slow page loading requiring wait times for dynamically rendered content.
• Page content changing based on factors like user geographical location, browser settings, login status, etc., necessitating the simulation of different environments and parameters.
• Anti-scraping mechanisms on the page, such as captchas, IP restrictions, and request rate limitations, requiring circumvention or resolution.
• Inconsistent page data formats, requiring parsing and extraction for different page types and content.

To address these challenges, Pangolin Scrape API can be employed. It is an API specifically designed for scraping Amazon pages, enabling quick, simple, and efficient data retrieval without the need for complex web scraping code. With just a simple request, you can asynchronously receive the collected data. The advantages of using Pangolin Scrape API include:

• Fast speed with no need to wait for page loading; data is returned directly.
• Stability and reliability, eliminating concerns about anti-scraping mechanisms and ensuring data integrity and accuracy.
• Flexibility and convenience, requiring no installation of software or libraries; just one HTTP request is needed.
• Rich data support for various Amazon page types, such as search results pages, product detail pages, review pages, etc., with structured data for easy post-processing and analysis.

In this tutorial, we will guide you on how to use Pangolin Scrape API to collect Amazon data, including the following steps:

1. Register and obtain a token.
2. Write request parameters.
3. Send requests.
4. Deploy the receiving service.
5. Process the data.

Before you begin, ensure you have the following:

• A Pangolin account for token acquisition and task management.
• A service address for receiving data, which can be your own server, cloud service, or a third-party webhook service.
• A tool for sending requests, such as your preferred programming language or framework, or tools like Postman.
• A tool for processing data, such as Excel, a database, visualization tools, etc., based on your needs and scenarios.

If you have these prerequisites ready, let’s get started!

Scrape API Usage Guide Pangolin Scrape API is designed for scraping Amazon e-commerce pages. It can asynchronously return page data based on a specified URL and zip code. Here’s what you need to do:

1. Register and obtain a token: Register an account on Pangolin’s official website and obtain a token for identity and permission verification.
2. Write request parameters: Construct a JSON-formatted request parameter with the following fields:
   • url: The URL of the Amazon page you want to scrape, e.g., https://www.amazon.com/s?k=baby.
   • callbackUrl: The service address for receiving data; Pangolin will push data to this address via HTTP after scraping is complete.
   • bizContext: An optional field to specify the Amazon zip code for consistent consumer-related page data, e.g., {“zipcode”:”90001″}.
3. Send requests: Use the HTTP POST method to send the request parameters to Pangolin’s API address, e.g., http://...*/api/task/receive/v1?token=xxx, where xxx is your token.
4. Receive the response: You will receive a JSON-formatted response with the following fields:
   • code: System status code, where 0 indicates success, and others indicate failure.
   • message: System status information, where “ok” indicates success, and others indicate the reason for failure.
   • data: An object containing the following fields:
     • taskId: Spider task ID to identify your scraping task; Pangolin includes this ID when pushing data.
5. Deploy the receiving service: Deploy a simple HTTP service to receive data pushed by Pangolin. You can refer to the Java Springboot version of the receiving service code at the end of the document or implement a similar function using other languages and frameworks.
6. Process the data: Your receiving service will receive JSON-formatted data with the following fields:
   • taskId: Spider task ID, consistent with the data field in the response you received earlier.
   • data: An object containing the collected page data, with specific fields and structure depending on the page’s content and type.

These are the basic steps for collecting Amazon e-commerce page data using Pangolin Scrape API. You can modify and optimize your request parameters and receiving service based on your needs and scenarios. Refer to Pangolin’s official documentation and examples for more details and functionality. We hope this guide is helpful to you.