Setup & Installation
Or with the ClawHub CLI, for registry-managed skill folders outside a full OpenClaw workspace:
What This Skill Does
A macOS tool that runs OCR on screen content using Apple's built-in Vision framework. It extracts text and [X, Y] coordinates locally, sending only the recognized text to the AI rather than full screenshots. Supports mixed Chinese and English recognition.
Running OCR locally via the Vision framework avoids sending screenshots to an AI API, which eliminates token costs and keeps screen content private.
When to use it
- Reading text from a non-standard desktop app that has no API
- Monitoring a balance or progress indicator on screen without manual checking
- Clicking a button in a GUI app by locating its label text
- Automating repetitive UI interactions across any macOS application
- Extracting text from a screen region that can't be copy-pasted
Example Workflow
Here's how your AI assistant might use this skill in practice.
User asks: help me click the Confirm button in this app
- 1Runs vision_ocr.swift to scan the current screen
- 2Receives recognized text with [X, Y] coordinates for each element
- 3Locates the text matching 'Confirm' and retrieves its coordinates
- 4Passes coordinates to click.swift to perform a physical mouse click
The Confirm button is clicked at the detected screen coordinates
Requirements
Accounts, API keys, or tools you or your AI assistant may need to set up while using this skill.