Setup & Installation
openclaw skills install @mariusfit/smart-web-scraper
Or with the ClawHub CLI, for registry-managed skill folders outside a full OpenClaw workspace:
npx clawhub install smart-web-scraper
What This Skill Does
Extracts structured data from web pages using CSS selectors or automatic detection of tables and lists. Outputs results as JSON, CSV, text, or Markdown. Supports multi-page crawling with pagination following.
Auto-installs its own dependencies via uv, so there is no manual setup before scraping a page.
When to use it
- Pulling product names and prices from an e-commerce listing
- Extracting all external links from a documentation site
- Scraping a pricing table into a spreadsheet-ready CSV
- Collecting article headlines across multiple paginated pages
- Grabbing contact details from a directory page
Example Workflow
Here's how your AI assistant might use this skill in practice.
INPUT
User asks: extract all products from https://shop.example.com as JSON
AGENT
- 1Run the extract command with the shop URL and a CSS selector targeting product elements
- 2Specify JSON output format with the -f json flag
- 3Save the output to a local file using -o products.json
- 4Review the resulting JSON array for product text and tag data
OUTPUT
A products.json file containing a JSON array with each product's text, tag, and class attributes