LaunchKit · 2026
Back to Skills

mlx-local-inference

Full local AI inference stack on Apple Silicon Macs via MLX.

1
329 downloads
by @bendusy

Setup & Installation

openclaw skills install @bendusy/mlx-local-inference

Or with the ClawHub CLI, for registry-managed skill folders outside a full OpenClaw workspace:

npx clawhub install mlx-local-inference

Version History

v2.2.0Feb 25, 2026 - Version note by skill developer:

One-line agent install, keep-alive/on-demand loading, multi-language mix support, model upgrade guide

What This Skill Does

Runs a full AI inference stack locally on Apple Silicon Macs using the MLX framework. Covers LLM chat (Qwen3-14B, Gemma3-12B), speech-to-text via Whisper and Qwen3-ASR, text embeddings, OCR with PaddleOCR, and text-to-speech. All services expose OpenAI-compatible APIs on localhost.

Eliminates cloud API costs and keeps all data local while exposing the same OpenAI-compatible API format that most AI tooling already supports.

When to use it

  • Transcribing Cantonese audio without uploading to cloud services
  • Running privacy-sensitive LLM queries entirely on-device
  • Extracting text from scanned documents or images via OCR
  • Building local semantic search with on-device text embeddings
  • Auto-transcribing a folder of meeting recordings in the background

Example Workflow

Here's how your AI assistant might use this skill in practice.

INPUT

User asks: Transcribe and clean up the Cantonese meeting recording at ~/Downloads/meeting.m4a

AGENT
  1. 1Copy meeting.m4a into ~/transcribe/ to trigger the daemon
  2. 2Daemon detects the file within 15 seconds and sends it to Qwen3-ASR on port 8788
  3. 3ASR output is saved as meeting_raw.md with the raw transcript
  4. 4Qwen3-14B reads the raw transcript, corrects homophones, adds punctuation, and preserves Cantonese characters
  5. 5Corrected file is moved to ~/transcribe/done/meeting_corrected.md
OUTPUT

meeting_corrected.md in ~/transcribe/done/ with a clean, punctuated Cantonese transcript

Requirements

Accounts, API keys, or tools you or your AI assistant may need to set up while using this skill.

Apple Silicon Mac (M1, M2, M3, or M4)Python packages: mlx, mlx-lm, mlx-audio, mlx-vlm