Scrape Spreadsheet
The Scrape Spreadsheet lets you upload a list of product page URLs. merchi.ai visits each page, extracts images, descriptions, and documents, then generates enriched product content using your schema and writing knowledge.
This is useful when your supplier has a product catalogue online and you want to import a specific set of products without visiting each page manually.
This is an Enterprise feature. You must be on an Enterprise plan to access it.
Preparing your spreadsheet
File formats
- CSV (.csv)
- Excel (.xls, .xlsx)
Required columns
| Column | Description |
|---|---|
product_id | Your unique identifier for the product (e.g. SKU or internal code) |
url | The full URL of the product page to scrape |
Optional columns
You can include any additional columns to provide context alongside scraped data. The AI will use both the scraped content and your spreadsheet data together.
| Column | Description |
|---|---|
title | Existing product title |
description | Existing description |
category | Product category |
brand | Brand name |
price | Product price |
Any column you add is passed to the AI as context — the more relevant information you include, the more accurate the generated output.
Example
product_id,url,brand
SKU-001,https://supplier.com/products/widget-pro,Acme
SKU-002,https://supplier.com/products/widget-lite,Acme
How scraping works
When a scrape job runs, merchi.ai:
- Visits the product URL using a cascading fetch strategy — direct request first, headless browser if the page is dynamic, Bright Data proxies if the site blocks automated access
- Extracts images, product data, and any linked PDF documents
- Passes everything to the AI model alongside your schema and writing knowledge
The scraper identifies itself using the user agent merchi-ai-bot/1.0. If a supplier is blocking scrape jobs, ask their team to whitelist this user agent. See URL Scraping for details on robots.txt compliance and how to get suppliers to whitelist access.
PDF documents
If your account has PDF storage configured in Assets, any PDF documents found on the product page (data sheets, manuals, certificates) will be downloaded and stored. If a PDF type is also set to Use in AI, its content is passed directly to the AI model during content generation.
Running the import
- Go to Add Products and select Scrape Spreadsheet
- Upload your CSV or Excel file
- Verify the column mappings — merchi.ai will try to detect
product_idandurlautomatically - Select your AI model
- Click Start Import
Processing happens in the background. You can track progress on the Runs page. Each product URL is scraped and processed as a separate task — if one fails the others continue.
Credits
Each product URL consumes 1 credit when processed. Credits are checked before processing begins. If your account does not have enough credits, the import will not start.
Troubleshooting
Pages returning no content
The site may be blocking automated access. Ask the supplier to whitelist merchi-ai-bot/1.0. Some sites also require JavaScript rendering — the headless browser fallback handles most cases.
PDFs not storing Check your Assets settings and verify the document type is enabled for storage.
Wrong images extracted The AI selects images based on your scrape prompt configuration. If the wrong images are being pulled, review your prompt in Settings.
Import failing for some rows The Runs page shows the status of each individual scrape task. Open a failed run to see the error detail.