Evaluating What You Built: Does It Actually Work?

Evaluation Is What Makes a Prototype Trustworthy

Most AI tools built by non-engineers fail not because they were built wrong, but because they were never properly evaluated. The builder tested a few examples, it worked, and they deployed it. Then edge cases appeared, failure modes surfaced, and trust eroded.

Systematic evaluation - done before and after deployment - is what prevents this.

Before You Deploy: Define Success

The most important evaluation step happens before you test anything. Write down in one sentence what a successful output looks like.

"The tool correctly classifies customer emails as Billing, Technical, Feature Request, or Other."

Now make it measurable: "The tool achieves at least 85% correct classification on a representative sample of 50 emails."

This simple act of quantification transforms evaluation from a vibe check to a measurement.

Building Your Test Set

A test set is a collection of inputs where you know the correct output.

For a classifier: 50 examples you have manually labeled. For a summarizer: 20 documents where you have noted the key points. For a question-answering system: 30 questions whose correct answers you can verify.

Source your test set from real usage, not constructed examples. Real inputs reveal failure modes that constructed examples miss.

Measuring Performance

pythonCopy
def evaluate_classifier(test_cases: list[dict]) -> dict:
    """
    test_cases: list of {"input": "...", "expected": "..."}
    """
    correct = 0
    failures = []

    for case in test_cases:
        predicted = classify_text(case["input"])  # Your classifier function
        if predicted == case["expected"]:
            correct += 1
        else:
            failures.append({
                "input": case["input"][:100],
                "expected": case["expected"],
                "predicted": predicted
            })

    accuracy = correct / len(test_cases)
    return {"accuracy": accuracy, "failures": failures}

results = evaluate_classifier(test_cases)
print(f"Accuracy: {results['accuracy']:.1%}")
print(f"\nFailure examples:")
for f in results["failures"][:5]:
    print(f"  Expected: {f['expected']}, Got: {f['predicted']}")
    print(f"  Input: {f['input']}")

Classifying Failure Modes

After running your test set, look at the failures and classify them. Common patterns:

• Ambiguous inputs: The input is genuinely hard to classify. Your categories may need refinement.
• Edge cases the prompt does not handle: The prompt needs a constraint or example for this type.
• Systematic errors on a category: One category is consistently misclassified - possibly undertrained due to few examples or a confusable boundary.
• Out-of-scope inputs: Inputs that do not fit any category. You need a "none of the above" handling path.

Each failure mode suggests a specific fix. This is the improvement loop.

After Deployment: Monitoring and Feedback

Once deployed, collect feedback signals:

• User corrections (if users can flag wrong outputs)
• Periodic spot-checks (sample 20 outputs per week and evaluate manually)
• Downstream metrics (if your tool routes emails, track how often the routing was overridden)

These signals tell you when the tool is drifting and when to retrain or re-prompt.

The Evaluation Report

Document your evaluation results in a simple report:

• What the tool does
• Success criteria you set
• Test set size and composition
• Accuracy and primary failure modes
• What you would improve next

This report is the artifact that proves your tool was built responsibly. It is also the basis for communicating honestly about the tool's limitations to users.

Phase 4 Gate

Before moving to Phase 5, you should have:

• A working AI tool you built (from Module 16, 17, or 18)
• A test set of at least 20 real inputs
• A measured accuracy or success rate
• A classified list of failure modes
• A written evaluation report

This is the deliverable of Phase 4. It proves you can build and evaluate, not just build.