Setup & Installation
Or with the ClawHub CLI, for registry-managed skill folders outside a full OpenClaw workspace:
Version History
Security: removed default API key values, added explicit env validation
What This Skill Does
Fetches full web page content using a local Crawl4AI instance with JavaScript rendering. Offers two endpoints: a proxy for clean markdown output and a direct endpoint for structured data including links, HTML, and media. No usage limits since it runs locally.
Runs on a local instance with no rate limits or API costs, and reliably handles dynamic JavaScript content that simpler fetch-based scrapers miss.
When to use it
- Scraping JavaScript-rendered single-page applications
- Extracting article text from JS-heavy or dynamic news sites
- Collecting product data from e-commerce pages
- Archiving web pages with full link and media metadata
- Pulling content from documentation portals that require JS execution
Example Workflow
Here's how your AI assistant might use this skill in practice.
User asks: Get the full content from https://example.com/docs/getting-started
- 1Identify the target URL from the user request
- 2Call crawl4ai.js with the URL via the proxy endpoint for clean output
- 3Receive the page rendered as structured markdown with metadata
- 4Return the extracted content to the user
Full page content as clean markdown, including any dynamically loaded text
Requirements
Accounts, API keys, or tools you or your AI assistant may need to set up while using this skill.