Setup & Installation
Or with the ClawHub CLI, for registry-managed skill folders outside a full OpenClaw workspace:
What This Skill Does
Python SDK for evaluating generative AI applications using Azure OpenAI. Supports quality metrics (groundedness, relevance, coherence), NLP-based scoring (F1, BLEU, ROUGE), and safety evaluations (violence, hate, self-harm). Results can be logged to Azure AI Foundry for tracking across runs.
Combines quality, NLP, and safety evaluators in one SDK with direct Azure AI Foundry integration, eliminating the need to wire together separate scoring libraries.
When to use it
- Scoring RAG pipeline responses for groundedness against source documents
- Running safety checks on chatbot outputs before production deployment
- Batch evaluating a dataset of query/response pairs with multiple metrics
- Logging evaluation runs to Azure AI Foundry for regression tracking
- Building custom domain-specific evaluators for specialized content
Example Workflow
Here's how your AI assistant might use this skill in practice.
User asks: evaluate my RAG app responses for groundedness and safety
- 1Load model config from AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_KEY, and AZURE_OPENAI_DEPLOYMENT env vars
- 2Initialize GroundednessEvaluator with model config and ContentSafetyEvaluator with Azure AI project scope
- 3Call evaluate() with test_data.jsonl, mapping query/context/response columns to both evaluators
- 4Print result metrics and studio_url for Foundry dashboard link
Groundedness and safety scores per row plus aggregate metrics, with a link to view results in Azure AI Foundry
Requirements
Accounts, API keys, or tools you or your AI assistant may need to set up while using this skill.