DeepEval
Open source LLM evaluation with pytest style tests and a red teaming module.
About DeepEval
DeepEval is a simple-to-use, open-source LLM evaluation framework for evaluating large-language model systems. It is similar to Pytest but specialized for unit testing LLM apps. The project is licensed under Apache-2.0.
Product teams manage datasets, evaluate AI applications before launch, and monitor live traces with online evals and signals in production. Platform teams define one organization-wide quality standard and enforce it through governance and native red teaming.
Drafted from github.com · not yet confirmed by Confident AI
How it works
DeepEval lets you write unit tests for LLM outputs with metrics like hallucination, answer relevancy, bias, toxicity and custom LLM judged criteria, run in pytest, and it includes a red teaming module that generates adversarial attacks across vulnerability categories. Confident AI sells a hosted dashboard on top.
From the project
Reviews
Be the first to review it
AnonymousKnow a team weighing DeepEval? Send them this page.