Cornell robotics and artificial intelligence save grapes



FINGER LAKES – A radical collaboration between a Cornell biologist and engineer drives efforts to protect grape crops. The technology they developed, which uses robotics and artificial intelligence (AI) to identify grape plants infected with a devastating fungus, will soon be available to nationwide researchers working on a variety of plant and animal studies.

The biologist Lance Cadle-Davidson, Ph.D. ’03, an associate professor in the School of Integrative Plant Science (SIPS), is working on developing grape varieties that are more resistant to powdery mildew, but his laboratory’s research has been constrained by the need to manually examine thousands of grape leaf samples for evidence of the Infection.

Powdery mildew, a fungus that affects many plants such as grapes and table grapes, leaves sickly white spores on leaves and fruit and costs wine growers around the world billions of dollars in lost fruit and fungicide costs each year.

Cadle-Davidson is also a research plant pathologist with the US Department of Agriculture’s Agricultural Research Service (USDA-ARS). He works in the Grape Genetics Research Unit in Geneva, and his team developed VitisGen2 as part of the USDA-ARS-funded grape breeding project and in collaboration with the Research Center for Light and Health. This partnership led to the development of a robotic camera called “BlackBird”. However, the extraction of relevant biological information from these images was still an urgent need.

Enter the engineer and computer scientist: Yu Jiang, Assistant Research Professor in the SIPS Horticulture Section at Cornell AgriTech. Jiang’s research focuses on systems engineering, data analysis, and artificial intelligence. The BlackBird robot can collect information on a scale of 1.2 micrometers per pixel – that’s equivalent to a normal optical microscope. For every 1-centimeter leaf sample examined, the robot delivers 8,000 by 5,000 pixels of information.

Anna Underhill, a technician at the USDA-ARS Grape Genetics Research Unit in Geneva, New York, calibrates a leaf disk shell in the BlackBird imaging robot.  This system can image samples from 351 grapevines in two hours.

Extracting useful information from such a large, high-resolution image was Jiang’s challenge, and his team used AI to solve it. Using breakthroughs in deep neural networks developed for computer vision tasks such as face recognition, Jiang applied this knowledge to analyzing microscopic images of grape leaves. In addition, Jiang and his team implemented the visualization of the network inference processes, which help biologists better understand the analysis process and build trust in AI models.

The Cadle-Davidson team works together to test and validate what the robots see so that Jiang’s team can teach them how to recognize biological characteristics more effectively. The results are amazing, said Cadle-Davison. For research experiments that previously took his entire laboratory team six months, the BlackBird robots now only need one day.

“It revolutionized our science,” said Cadle-Davidson. “And we find that Yu’s AI tools can actually explain the genetics of these grapes better than we can sit in front of a microscope for months and do backbreaking work.”

In July alone, the cooperation won a prize and two new grants. On July 1, the team received a $ 100,000 grant from USDA-ARS to distribute BlackBird to ARS field offices working on other plants doing the same type of high-throughput phenotyping work.

“We hope to find collaborative laboratories that can join us in using this tool,” said Jiang. “We see potential applications for this research in plant studies, animal experiments or medical purposes.”

On July 12, the team’s article on their project received the Best Paper Award for Information Technology, Sensors and Control Systems at the American Society of Agricultural and Biological Engineers’ annual international meeting in 2021. And on July 27, they received a two-year grant from the Cornell Institute for Digital Agriculture Research Innovation Fund of $ 150,000 to begin upgrading the BlackBird robot to go beyond the red, green, and blue color spectrum in See infrared.

Plant diseases such as powdery mildew can appear in the infrared before they are visible to the naked eye; If researchers can develop tools to help farmers identify disease early, farmers could target fungicide sprays before the infection spreads, which means fewer fungicides and fewer crop failures. They are also working on integrating AI more effectively with scientists in data analysis.

“This work is greatly accelerating the pace of breeding and genetics work on grapes,” said Donnell Brown, president of the National Grape Research Alliance. “When we invest in research in the industry, we usually do so knowing that we may never see the results of our investments in our lifetime – it really is a faith-based investment in future generations of breeders. But now this technology is really shortening that time frame to the benefit of producers and consumers. “



Leave A Reply