You might have heard about AI that teaches four-legged robots to walk and autonomous systems that generate photorealistic images of butterflies, but what about models that forecast the shot location of tennis balls? In a newly published preprint paper on Arxiv.org (“Memory Augmented Deep Generative models for Forecasting the Next Shot Location in Tennis“), researchers from Queensland University of Technology describe an AI system that’s not only capable of anticipating a tennis opponent’s actions, but doing so with “player-level” behavioral patterns.

“Inspired by recent neuroscience discoveries we incorporate neural memory modules to model the episodic and semantic memory components of a tennis player,” the researchers wrote.

Tracking a tennis ball is no easy feat at the professional level, given that they whiz by at speeds exceeding 130 miles per hour. Some studies suggest that expert players are more adept generally at detecting events in advance, in fact, and have better knowledge of situational probabilities.

The researchers used those biological insights to design what they dubbed Memory-augmented Semi Supervised Generative Adversarial Network (MSS-GAN), based on a generative adversarial network (GAN) — a two-part neural network consisting of generators that produce samples and discriminators that attempt to distinguish between the generated samples and real-world samples — whose memory networks loosely mimic those in the brain. A novel structure stored episodic memories (long-term memories that involve conscious recollection of previous experiences), and a framework that propagated knowledge from the episodic memories to another structure responsible for storing semantic memory (long-term memory involving the capacity to recall words, concepts, or numbers).

Here’s how it worked in practice: A perception component processed input data to obtain embeddings, or mathematical representation, that represented shots taken by tennis players. Combined with other embeddings from the episodic memory and semantic memory, they were used to generate next shot predictions, which a GAN framework validated for accuracy.

To train the AI system, the researchers fed it ball behavior — including trajectory, speed, angle, and player foot movements — from 8,780 shots taken by the top three players (Rafael Nadal, Roger Federer, and Novak Djokovic) at the 2012 Australian Open Men’s singles. In tests, it managed to predict shots from Nadal, Federer, and Djokovic to within 0.87 meters, 0.79 meters, and 1.14 meters, respectively. But perhaps more impressively, the researchers observed that, even with reduced training data, the AI system’s performance wasn’t significantly impacted, indicating its ability to infer different player styles.

“We demonstrate that the proposed framework can be utilized not only for high-performance coaching, and designing intelligent camera systems for automatic broadcasting, where the system anticipate the next shot and shot type to better capture the player behavior; but also for better understanding of player strategies, strengths, and weaknesses,” the researchers wrote. “The proposed model learns these attributes automatically via modeling the player knowledge and experiences through neural memory networks and outperforms the state-of-the-art baselines. [E]xamples [observed during tests] demonstrate that the proposed … model is capable of capturing match context and the player tactical elements which are essential when anticipating player behavior.”