Pokémon GO Players Unknowingly Helped Build a 30 Billion AR Image Map of the World

In 2016, Niantic launched Pokémon GO, an augmented reality (AR) game that took the world by storm. Now, 10 years later, spin-off company Niantic Spatial is using all the visual data gathered by players to build a massive geospatial model that can power robots and AI.

Niantic Spatial is an AI company spun off of Niantic when it was sold to Scopely last May. Scopely is owned by the Savvy Games Group, which is part of Saudi Arabia’s Public Investment Fund. Niantic Spatial, however, is owned by the original Niantic Investors, with some investment from Scopely.

As explained by the MIT Technology Review, Pokemon GO (along with Pikmin Bloom and Monster Hunter Now) had access to players’ smartphone cameras during gameplay. That access was not going to be ignored, and the games pulled in an astronomical amount of visual data.

All three games were hugely popular, but GO, in particular, was a massive success.

“Five hundred million people installed that app in 60 days,” Brian McClendon, CTO at Niantic Spatial, tells MIT. It continued to have a large user base for a decade, each one of those players continuing to grab visual data while chasing Pokémon around the planet. The result is what it calls its Large Geospatial Model, or LGM, which is at the core of its Visual Positioning System, or VPS, that is based on the 30 billion images captured by Niantic Games users over 10 years.

Niantic Spatial launched using that trove of what it characterizes as “crowdsourced data” to build its LGM. Because it has access to those 30 billion AR images of urban landmarks that have been geotagged with extremely accurate location markers, thanks to the hundreds of millions of Pokémon GO players, the company can pinpoint a user’s exact location on a map to within a few centimeters based on landmarks and buildings within a field of view.

“Our technology is based on a third-generation digital map that captures the content of the world at a level of fidelity never before achieved, enabling both people and machines to understand it in new exciting ways. This is part of the connective tissue that will enable AI to meaningfully understand and interact with the physical world,” Niantic Spatial says.

This data is already being used in a collaboration with Coco Robotics, a startup that builds delivery robots.

“The promise of last-mile robotics is immense, but the reality of navigating chaotic city streets is one of the hardest engineering challenges,” John Hanke, CEO of Niantic Spatial, says. “We are thrilled to be working with Coco Robotics as our first robotics partner and deploying spatial intelligence to help solve these challenges head-on.”

Coco cannot rely on GPS in cities because radio signals are untrustworthy, as they can bounce off buildings and interfere with each other.

“It gives us reliable access to localization services that further improve robot navigation,” Zach Rash, Co-Founder and CEO of Coco Robotics, says of his company’s partnership with Niantic Spatial.

“It turns out that getting Pikachu to realistically run around and getting Coco’s robot to safely and accurately move through the world is actually the same problem,” he explains to MIT.

“We know where you’re standing within several centimeters of accuracy and, most importantly, where you’re looking,” Brian McClendon, CTO at Niantic Spatial, adds. “We had a million-plus locations around the world where we can locate you precisely.”

Niantic Spatial’s technology is impressive, but it wouldn’t have been possible without Pokémon. The more sinister aspect of this story is that Niantic was capturing this data without its users necessarily knowing it was happening, getting it for free, and now it’s being used to drive robotics and, theoretically, for anything else, as long as a company is willing to pay Niantic for it.

Image credits: Header Photo via The Pokémon Company