Antarctica’s emperor penguin population has suffered such severe climate-related breeding issues that it’s at risk of disappearing by the year 2100, according to a 2019 study by the British Antarctic Survey. In search of a solution, an Intel-led group of tech companies developed a computer vision solution to help ecologists count the remaining penguins faster and more accurately than before.
The coalition consists of Intel’s AI Builders — the company’s curated ecosystem of software vendors — along with Microsoft’s AI for Earth initiative and data science consultancy Gramener. The consultancy sourced a corpus containing photos of Antarctica’s penguin colonies from Oxford University’s Penguin Watch Project, which over the past decade has sourced millions of time-lapse images from camera traps in over 40 locations and recruited online volunteers to annotate them. It then fed the data through a convolutional neural network — a type of AI model most commonly applied to analyzing visual imagery — that preserved spatial information while localizing penguin counts and estimating overall tallies.
The AI model was cascaded and two-stage in design. The first classified images into broad image categories, while the second estimated the density by taking inputs from earlier stages to generate maps. By approximating the number of penguins in clusters of different sizes, this density-based approach managed to handle most of the imaging challenges.
Intel Xeon Scalable processors were used to train the model within a virtual environment on Microsoft’s Azure cloud platform, which was then benchmarked using Intel’s Optimization for PyTorch toolkit. From this benchmarking exercise and other open source tools and libraries, Gramener says it was able to reduce training time from days to hours.
It was quite the feat, all told. Some images exhibited perspective distortion, where penguin faces in the foreground appeared larger than those in the background, and occlusion, where penguins in the background were hidden from view by objects or other penguins in the foreground. There was variability with respect to image quality and shooting angles, owing to unpredictable weather conditions, and there were often differences within the same picture, with some portions containing unusually high clusters of penguins.
Above: The AI system picks out penguins in a crowd.
“We faced multiple challenges. Almost 70% of images were unusable, as either the lighting wasn’t good or it was foggy, and penguins weren’t visible,” said lead data scientist Soumya Ranjan Mohanty. “These are easy for the human eye to perceive but difficult for an algorithm.”
It isn’t Hyderabad-based Gamener’s first AI project of an ecological nature. The consultancy partnered with Microsoft to build a model capable of classifying thousands of different flora and fauna species, using open source data from iNaturalist. It also developed a two-step solution that weeds out unusable images from camera traps and identifies those that contain animals, which became Microsoft’s Camera Trap API, as well as a system that provides high-resolution land cover information. Separately, Gamener worked with the Nisqually River Foundation to pick out fish species by analyzing video footage from an underwater camera that’s activated when fish pass certain infrared sensors. And in Kenya it worked with local researchers to build a model that could distinguish elephants from other livestock in aerial photographs.
Of course, Gamener is far from the first to apply AI to these kinds of problems. In December 2019, Google partnered with Conservation International and other organizations to process one of the world’s largest and most diverse databases of photographs taken from motion-activated cameras. DeepMind last year detailed ecological research its science team is conducting to develop AI systems that will help study the behavior of animal species in Tanzania’s Serengeti National Park. Microsoft recently highlighted a Santa Cruz, California-based startup called Conservation Metrics that’s leveraging machine learning to track African elephants, akin to a separate effort by an independent team of researchers to develop an algorithm trained on Snapshot Serengeti that can identify, describe, and count wildlife with 96.6% accuracy. And Intel’s own TrailGuard AI system prevents poaching by detecting motion with cameras using an on-device AI algorithm.
