A Farmer Followed AI Pesticide Advice and Reportedly Lost 25 Acres of Crops

Hey ChatGPT, what pesticide should I use?

We all know AI can hallucinate and give bad advice. Heck, sometimes it gives terrible advice. But it sounds plausible, and it’s often useful, so millions of people have come to rely on it as a primary source of information.

Sometimes, that can backfire spectacularly.

Take, for instance, the case of farmer Wu. The 67-year-old farmer from Anhui Province in China watched sesame seedlings across 10 hectares (or 25 acres) die after he sprayed a weed-and-pest treatment recommended by an AI tool.

Pesticide Roulette

Wu had reportedly been consulting an AI application about farming for roughly a year. He initially distrusted it, but apparently became more confident after receiving advice he considered useful.

If you’ve used AI chatbots yourself, that may sound oddly familiar.

Eventually, the farmer asked for help controlling both weeds and insects in his sesame field. The chatbot eventually recommended a combination that included two herbicides, haloxyfop-P-methyl and fomesafen, alongside insecticides.

Haloxyfop-P-methyl is a selective herbicide designed mainly to kill grasses. Because sesame is a broadleaf crop, using such a grass-selective herbicide can make agricultural sense under the right conditions.

Fomesafen is trickier. It’s a broadleaf herbicide widely used in crops such as soybeans. In China, registered fomesafen products come with tightly specified instructions covering crop type, application rate, weed growth stage and spray conditions. Product labels also warn against allowing the herbicide to drift onto sensitive crops or applying too much.

×

Get smarter every day...

Stay ahead with ZME Science and subscribe.

Daily Newsletter
The science you need to know, every weekday.

Weekly Newsletter
A week in science, all in one place. Sends every Sunday.
No spam, ever. Unsubscribe anytime. Review our Privacy Policy.

Thank you! One more thing...

Please check your inbox and confirm your subscription.

Wu says he mixed the recommended products and treated the entire field.

Within a day, the sesame began to wilt and die.

Hey AI, What Happpened?

Wu went back to the chatbot and asked what had happened.

According to screenshots and Wu’s account, the AI went back to the already classical “Oh, that may have been a mistake.”

The same system that had recommended the treatment was now explaining why it may have been a mistake.

But this is a good example of why trusting AI can be so risky. This isn’t even a hallucination, per se, because the herbicides themselves are legitimate. Combinations containing haloxyfop and fomesafen are even commercially registered for certain crops, including soybean fields.

The problem is that pesticide recommendations are intensely context-dependent.

A treatment that works on soybeans can damage another crop. A dose that is safe at one growth stage may be dangerous at another. Tank-mixing products can alter their effects. Weather, formulation, spray volume and crop variety can all matter.

The Problem With Being Right Most of the Time

We don’t have a chemical analysis or any scientific report for this case. We’re taking Wu’s account at face value and can’t verify it. But the episode illustrates a subtler kind of AI failure, and one that seems to be increasingly common.

RelatedPosts

Genetically engineered crops reach 11.5% of the total arable land
Agriculture is one of today’s biggest emitters. But it could go carbon negative by 2050
AI Is Changing Education — But Are We Keeping Up?
AI Voice Inspired by a Reality TV Star Calls 3,000 Pubs to Find the Cheapest Pint of Guinness in Ireland

It boils down to one of the harder questions surrounding AI chatbots: what happens when the system is good enough to earn trust, but not good enough to always be right?

We know AI can give good advice, and we know it can make mistakes. It tends to do well in universal questions, and not as well in contextual questions. Agriculture is especially unforgiving as questions are rarely as universal as they sound.

“When should I plant?”

“What should I spray?”

“How much fertilizer should I use?”

These are heavily contextual. They depend on the local climate, soil, time of year, and so much more. A language model can know a great deal about pesticides and still miss the one fact that matters most in a particular field. Wu’s case greatly illustrates that.

This can be even more brutal in human health.

The right treatment can depend on a person’s age, weight, other medications, allergies, medical history, pregnancy status, kidney or liver function, and the exact diagnosis. An AI system may know a great deal about a drug or disease yet miss the one detail that makes otherwise sensible advice dangerous for a particular patient.

AI systems increasingly offer advice in fields where being usually right (or almost right) isn’t good enough.

And once people have seen the systems be right often enough, the “AI can make mistakes” warning at the bottom of the screen may not matter much at all.