azure-ai-evaluation-py
Azure AI Evaluation SDK for Python.
Setup & Installation
Install command
clawhub install thegovind/azure-ai-evaluation-pyIf the CLI is not installed:
Install command
npx clawhub@latest install thegovind/azure-ai-evaluation-pyOr install with OpenClaw CLI:
Install command
openclaw skills install thegovind/azure-ai-evaluation-pyor paste the repo link into your assistant's chat
Install command
https://github.com/openclaw/skills/tree/main/skills/thegovind/azure-ai-evaluation-pyWhat This Skill Does
Python SDK for evaluating generative AI applications using Azure OpenAI. Supports quality metrics (groundedness, relevance, coherence), NLP-based scoring (F1, BLEU, ROUGE), and safety evaluations (violence, hate, self-harm). Results can be logged to Azure AI Foundry for tracking across runs.
Combines quality, NLP, and safety evaluators in one SDK with direct Azure AI Foundry integration, eliminating the need to wire together separate scoring libraries.
When to Use It
- Scoring RAG pipeline responses for groundedness against source documents
- Running safety checks on chatbot outputs before production deployment
- Batch evaluating a dataset of query/response pairs with multiple metrics
- Logging evaluation runs to Azure AI Foundry for regression tracking
- Building custom domain-specific evaluators for specialized content
Example Workflow
Here's how your AI assistant might use this skill in practice.
User asks: Scoring RAG pipeline responses for groundedness against source documents
- 1Scoring RAG pipeline responses for groundedness against source documents
- 2Running safety checks on chatbot outputs before production deployment
- 3Batch evaluating a dataset of query/response pairs with multiple metrics
- 4Logging evaluation runs to Azure AI Foundry for regression tracking
- 5Building custom domain-specific evaluators for specialized content
Azure AI Evaluation SDK for Python.
Security Audits
These signals reflect official OpenClaw status values. A Suspicious status means the skill should be used with extra caution.