evidoria

← Back to browse

Good practice Imported

Benefit Navigator — Imagine LA and Nava's AI Chatbot Improves Caseworker Accuracy by 40% in a Los Angeles County Pilot

United States of America · Los Angeles · See the United States of America profile · See the Los Angeles profile

Evidence: Randomised controlled trial Top 12% 80/100 · Ask Evidence Copilot about this practice

A generative-AI chatbot built by Nava and nonprofit Imagine LA for Los Angeles County benefits caseworkers lifted response accuracy by 40% in a rigorously evaluated pilot, after an earlier rollout helped 500+ case managers secure a further $10,869 per household on average.

+40%
Improvement in caseworker response accuracy
$10,869
Additional benefits secured per household (earlier tool) (Sep 2023-Nov 2024)
65%
Caseworkers who actually used the chatbot
11
Net Promoter Score
Benefit Navigator — Imagine LA and Nava's AI Chatbot Improves Caseworker Accuracy by 40% in a Los Angeles County Pilot

Details

Maturity
Scaling
Promoter
Imagine LA & Nava PBC (with Los Angeles County social-service agencies)
Period
2023-2026
Keywords
social services, benefits administration, nonprofit technology, caseworker tools

Context

Los Angeles County benefits caseworkers navigate complex, college-level policy manuals to answer client questions; an earlier Benefit Navigator tool (Sep 2023-Nov 2024) had already helped 500+ case managers secure additional benefits for over 10,000 beneficiaries.

Activities

Imagine LA and Nava built a generative-AI chatbot drawing on curated government and program documents, returning cited, plain-language and Spanish-language summaries. Nava evaluated it with a randomised controlled trial (125 caseworkers) and a 14-week quasi-experimental pilot (61 caseworkers, six organisations).

Results

The evaluation found the chatbot improved caseworker accuracy by an average of 40%, with the largest gains on the hardest questions, and lowered reading level from college level to 10th-12th grade. The earlier tool secured an average of $10,869 in additional benefits per household.

Conclusions

Results were mixed on adoption: only 65% of caseworkers with access used the tool, usage declined over the pilot, and Net Promoter Score was a modest 11; effects on caseworker burden were inconclusive due to small sample size.

Implementation
Indicative cost
Medium (€50k–€500k)
Time to results
Medium (1–3 years)
Staffing & skills
Imagine LA (nonprofit), Nava PBC (technology partner), six partner social-service organisations' caseworkers

Conditions for success

  • curated, cited government/program document base
  • plain-language and Spanish-language rewrites

Common failure modes

  • low sustained usage (65% adoption, declining over pilot)
  • modest Net Promoter Score
  • inconclusive effect on caseworker burden

Commonly funded by

National / regional programmes Philanthropic / foundation funding

Indicative funding routes for practices of this type — always check each programme's current calls and eligibility rules.

Do you run this practice? Claim it — verified implementers get a public contact pathway and can propose corrections.

Data sources

Where this practice's information was retrieved from, and when.

Attachments

Similar practices you may find useful