Real-World Evidence

Case Studies

What separated the wins from the failures — across banks, FinTech firms, and universities.

JPMorgan Chase✔ Success

COIN — Contract Intelligence

ML reviews loan agreements in seconds vs. 360,000 lawyer-hours annually.

Lesson: Start with a bounded, high-volume task. Redeploy people — don't just cut them.

Goldman Sachs / Apple Card✕ Caution

Credit Limit Algorithm

Algorithm gave women systematically lower credit limits. NY DFS investigation followed.

Lesson: Deploy without bias testing at your regulatory and reputational peril.

Upstart Lending✔ Success

AI Credit Scoring Beyond FICO

27% more approvals at equivalent or lower default rates vs. traditional models.

Lesson: AI can expand access — not just cut costs — when designed intentionally.

Zillow iBuying✕ Caution

AI Home Price Prediction

Model failed in post-COVID volatility. $500M loss. Division shut down entirely.

Lesson: AI trained on historical data fails in regime shifts. Always keep human override.

Georgia Tech — Jill Watson✔ Success

AI Teaching Assistant

AI answered student questions for months before students realized it wasn't human.

Lesson: AI can scale personalized learning — transparency after pilot is essential.

Proctorio — Exam Surveillance✕ Caution

AI Remote Proctoring

High false positive rates disproportionately affected students of color. Lawsuits and bans followed.

Lesson: Validate for bias rigorously before any high-stakes deployment.

Arizona State University✔ Success

Adaptive Learning Platform

AI adapted curriculum pace per student. Measurable STEM completion improvement.

Lesson: Faculty co-design is essential for sustainable academic AI adoption.

UK A-Level Algorithm 2020✕ Caution

AI Grade Prediction (COVID)

Systematically downgraded students from lower-income schools. Policy reversed within days.

Lesson: Fairness audits are non-negotiable when outcomes affect life trajectories.