Use Case in Software Testing: How to Turn One Into Test Cases
Every path through a use case becomes at least one test case. A worked bank-transfer example shows the test table, and ISTQB's 2023 and 2025 syllabus changes explain…
AI and Machine Learning
Testing Services
Vervali tests AI-enabled products end to end: model behaviour, prompts and retrieval, guardrails, APIs and integrations, latency, fallback paths, human hand-off and regression after a model or prompt changes. Need statistical evidence for accuracy, calibration and review-board approval? See our AI model validation and testing services.
150+ clients
450+ projects
275+ engineers
15+ years
ISO/IEC 17025:2017Accredited testing laboratory
CMMI Maturity Level 3The process is written down and repeats
ISO 9001:2015Quality management
ISO/IEC 27001Information security
What AI and Machine Learning Testing Covers
The model can score correctly while the product still fails. We test the complete decision path, from data and model output through integrations, guardrails, user experience and recovery.
Data and Model Behaviour
Inputs, outputs, confidence and malformed cases across the paths the product actually uses. For statistical approval evidence, see our AI model validation and testing services.
User Journeys and Decision Logic
The model result reaches the right workflow, role and customer message. We test approvals, rejections, escalation and the human-review path, not only the score returned by an endpoint.
LLM and RAG Behaviour
Retrieval, grounding, hallucination, refusal, safety and prompt regression against a versioned set drawn from real product traffic.
Integrations and Fallback
Model APIs, vector stores, data services and third-party dependencies under timeout, partial response and unavailable-model conditions. The fallback must preserve the journey.
Performance and Resilience
Latency, concurrency, rate limits and cost-sensitive paths under expected load. A correct response that arrives after the customer leaves is still a product failure.
Regression and Monitoring
Stored cases run after model, prompt, retrieval or integration changes. Production thresholds and ownership are defined so silent behaviour changes become visible.
A model that scores well can still fail the customer.
Free Model ReviewTesting LLM and Generative Workflows
A versioned evaluation set tests retrieval, generation, safety and prompt changes in the workflow customers actually use, not on a public leaderboard.
An Evaluation Set From Your Traffic
Prompts, expected behaviour, and the cases that already failed in production. A public leaderboard does not prove your product. The stored set is what makes the next model or prompt change visible. We write the rubric with your owner before the first run.
Hallucination, Grounding and Safety
Does the answer stay inside the sources you named. Does it refuse what it should refuse. A fluent wrong answer is still a fail. Safety and refusal are tested alongside accuracy, not left as a slide of policy language.
Prompt Regression
The same stored cases run after a prompt, model version or retrieval change. The diff shows which tasks improved, regressed or changed unexpectedly.
Bias, Fairness and Human Review
Fairness is tested inside the customer journey, where a biased decision actually reaches a person.
Group and Threshold Performance
Group and subgroup performance, more than one fairness measure because a single metric can hide a slice, and threshold effects that appear only when you pick an operating point.
Explainable Decisions and Human Review
Whether a reviewer can see why the model decided, and whether the human-review and escalation paths actually catch the cases they were designed for.
Fairness Trade-Offs
When the fair choice and the accurate choice differ, we report the trade-off and you decide the risk appetite. Detailed measures run through our AI bias and explainability testing.
Production Monitoring and Drift
Baseline distributions, output rates and product outcomes become production controls with named thresholds, owners and actions.
Data Drift Testing
Inputs and outputs that no longer match the validated baseline, with a named threshold, an owner and the action an alert triggers, set up through our data drift testing.
Monitoring and Retraining
Live accuracy, product outcomes and the point where retraining is warranted rather than reflexive, run for deployed models through our AI model monitoring and retraining service.
How We Work With Product and ML Teams
We map the user journey with product, define model evidence with ML, and return defects and failed evaluation cases in forms each team can act on.
Endpoint or Scored File
Most vendor and in-house models can be tested as a black box. If a check requires weights or training code, that dependency is named before the quote.
Held-Out Data You Control
Your data, your groups, your intended use. A vendor score on their own set is not the engagement. We do not copy production customer records into a test environment without an agreed mask or synthetic set.
Findings a Scientist Can Act On
The slice that failed, the examples, the trade-off when fairness and accuracy differ, and what was not tested. A report only a tool vendor can read is a report that changes nothing.
Proof From AI Builds
AI products built and tested by Vervali across identity, sales and operations, measured by the business task each system had to improve.
Financial services · KYC deepfake detection
Manual review was the bottleneck on identity. A detection model for know-your-customer cut that load: 70% less manual review at 75% detection accuracy. Accuracy, the review load, and whether the model could be defended were measured together.
70% less manual review75% detection accuracyAI calling engine · lead qualification
An AI calling engine, built and validated by Vervali. After: 35% more qualified leads and a 28% shorter sales cycle. The checks were whether the engine actually qualified the lead, not a leaderboard score.
35% more qualified leads28% shorter sales cycleOperations · real-time SOP image detection
A real-time image detection system for standard operating procedure compliance, built and validated by Vervali. The check was whether the system caught the SOP miss on the floor, not a demo reel of clean frames.
Frequently Asked Questions
Vervali checks the model against held-out data for accuracy and calibration, then bias and fairness across groups it should not treat differently, drift since training, whether a decision can be explained, robustness to malformed and adversarial input, and data leakage that would make a sandbox score look better than production. Findings come with the evidence a model review board will ask for. A free review of one model, for one hour, is the first look.
Yes. Most vendor and open-source models are validated as a black box: an endpoint or a scored file, a held-out set you control, the intended use, and the decision the model supports. Training code and weights are not required in most cases. We report how the model behaves on your data and your groups, not how the vendor scored it on theirs. Book a call if the vendor already shipped the model.
No. Supervised classifiers and rankers, unsupervised systems, and generative and large language model systems are in scope. The checks change with the type: labelled accuracy and calibration for a classifier, retrieval and generation evals for an LLM. If a model type is outside what we can evidence, we say so in the design before we quote. A free review of one model is the first look.
Held-out accuracy, precision, recall and calibration, so a 0.9 score means something. Error analysis by slice, not only a global number. Cross-validation where the data volume allows it. Confidence intervals on the metrics a review board will read. The training set is not the proof. If the sandbox score and the held-out score disagree, the held-out score wins. Book a call once the held-out set is reachable.
We build an evaluation set from your own traffic, not a public leaderboard. Hallucination and grounding checks, safety and refusal behaviour, and regression on prompts so a prompt or model change cannot silently alter output. Retrieval quality is tested separately from generation, because most failures are retrieval failures wearing a generation costume. The stored set is what makes the next change visible.
Group and subgroup performance, more than one fairness measure because a single metric can hide a slice, and threshold effects that appear only when you pick an operating point. When the fair choice and the accurate choice differ, we report the trade-off, because Vervali does not decide your risk appetite. Detailed fairness measures are part of our AI bias and explainability testing service.
Validation is a point in time. Drift is the world moving after that point: inputs that no longer match training, or a relationship between inputs and outcomes that has changed. We say which inputs and outputs to monitor, what threshold should fire, what an alert should trigger, and when retraining is warranted rather than reflexive. Continuous monitoring runs through our data drift testing and AI model monitoring and retraining services.
Judge a vendor on whether they test on held-out data you control, whether bias and LLM output are in the same engagement as accuracy, whether findings are evidence a review board can read, and whether they are independent of the team that trained the model. Tool vendors sell software. Vervali is a testing laboratory: ISO/IEC 17025 accredited, working 9am to 1pm Eastern with delivery from India, and we will also build the model when that is the job.
Work is scoped to the product, model, integrations and checks you need, then quoted before we start. Typical work is $25 to $49 per hour. A free review of one model, for one hour, is the first look. A one-off test cycle is a fixed price. Ongoing monitoring is billed separately. Extra workflows, evaluation sets and LLM red-teaming are named in the estimate.
Email Us
Call Us
