Setup & Installation
Or with the ClawHub CLI, for registry-managed skill folders outside a full OpenClaw workspace:
What This Skill Does
Turns a scraping brief or target URL into a complete client-ready deliverable. Analyzes the target site, generates a Python scraper with rate limiting and resume support, cleans the output data, and packages everything with documentation.
Covers the full pipeline from site analysis to client handoff, including data cleaning and a quality report, rather than just generating a bare scraper script.
When to use it
- Extracting product catalogs from e-commerce sites
- Aggregating real estate listings across multiple pages
- Collecting job postings from a job board
- Archiving news articles from a publication
- Building a business directory dataset from a listings site
Example Workflow
Here's how your AI assistant might use this skill in practice.
User asks: Scrape all products from example-store.com — need name, price, description, images. CSV output.
- 1Fetches the target URL to determine if it is server-rendered or JavaScript-rendered
- 2Identifies pagination pattern, data structure, and visible anti-scraping measures
- 3Generates scrape.py with rate limiting, user-agent rotation, retry logic, and resume capability
- 4Runs the scraper, removes duplicates, and normalizes field values
- 5Packages data.csv, data.json, a data quality report, and a README into delivery/
2,487 product records in delivery/data.csv with a data quality report showing field completeness and any cleaned values