The bait and switch behind AI risk prediction tools

Toronto recently used an AI tool to predict when a public beach will be safe. It went horribly awry. The developer claimed the tool achieved over 90% accuracy in predicting when beaches would be safe to swim in. But the tool did much worse: on a majority of the days when the water was in fact unsafe, beaches remained open based on the tool’s assessments. It was less accurate than the previous method of simply testing the water for bacteria each day.

over a year ago

Remove from reading list Add to reading list [alt+a] Read now [→]

Improve your reading experience

Logged in users get linked directly to articles resulting in a better reading experience. Please login for free, it takes less than 1 minute.

More from AI Snake Oil

Could AI slow science?

Confronting the production-progress paradox

2 weeks ago • 21 votes

AGI is not a milestone

There is no capability threshold that will lead to sudden impacts

3 months ago • 18 votes

AI as Normal Technology

A new paper that we will expand into our next book

3 months ago • 46 votes

Is AI progress slowing down?

Making sense of recent technology trends and claims

7 months ago • 93 votes

We Looked at 78 Election Deepfakes. Political Misinformation is not an AI Problem.

Technology Isn’t the Problem—or the Solution.

7 months ago • 88 votes

More in AI

ML for SWEs #61: The secret behind how Google came back

Curated SWE AI Content, Jobs, and Resources 7-29-2025

2 days ago • 3 votes

Ads are inevitable in AI, and that's okay

Convergent evolution in LLMs will get us there

3 days ago • 7 votes

The Bitter Lesson versus The Garbage Can

Does process matter? We are about to find out.

3 days ago • 15 votes

The US Should Run Faster on AI Instead of Trying to Trip Up China

Export Controls and Trump's New AI Action Plan

5 days ago • 9 votes

AI Roundup 128: AI Action Plan

July 25, 2025.

6 days ago • 10 votes

New here?