AI makes AR make sense

The fusion of AI and AR empowers us to unlock the full potential of our cognitive abilities, enabling us to solve complex problems, make informed decisions, and share them efficiently.

I want to delve into how we can use brain science to think about how artificial intelligence (AI) and augmented reality (AR) converge, creating a powerful synergy that holds immense potential for transforming the way we perceive and interact with ideas.

First, let me describe what I mean about the ability of AR to interact with ideas. At Flow, I’ve been defining and building a platform that shows data visualization in 3D in XR headsets. It’s similar to the many SciFi movies that explain movie-world context with an interactive AR visualization. Also like Iron Man, where Tony Stark creates a new element using augmented reality - maybe that’s a new definition of Future of Work! I’ll use “AR” to mean this “informational AR” in this post. Take a look to get the idea:

Let's explore the profound reasons why AI is so vital to (informational) AR, and vice versa, why AR is equally vital to AI, because either technology on its own doesn’t make sense when it comes to presenting information.

AR needs AI

AR plus AI can be a powerful catalyst to bridge the gap between our minds and the digital realm. At the core of this vital connection lies the human mind. Our brains are remarkable information processors, capable of making sense of complex data and creating meaningful connections. However, traditional interfaces often fall short in effectively tapping into the brain's innate processing power. PowerPoint is a good example of this, where each slide disappears from our visual view without showing its relationships to past or future slides.

Our brain has to work extra hard to make these connections because the brain’s hippocampus can’t create spatial connection to disappearing non-things.

But in AR, we can see these relationships with words and connection lines and animations, maintaining connections maybe just out of our field of view, but available with a turn of the head.

AR still has challenges with a limited field of view and non-retina resolution, making complex interfaces non-viable. And even in the future when these are solved, we expect the UI floating in our space to be simple or it clutters too much of our real world. Buttons in space are not going to do it. But now, the combination of AI and AR enables us to interact with digital content in a natural and seamless manner using voice commands that have become newly useful because of large language models. A voice interface, especially one with a broad set of commands for manipulating data, has been the missing piece for a massively usable application. AR without AI has always been handicapped… but not anymore. And in fact, as SciFi movies have shown us, visualizing our data and information in 3D in our world should be providing us with the “cool” feeling of mastery and control, and these combined technologies do that.

AI also needs AR

But the connection goes even deeper because AR is equally vital to AI, as it provides the missing piece of the puzzle when it comes to data visualization and comprehension. While AI algorithms excel at extracting insights from complex datasets, the challenge lies in effectively presenting this information to the human mind. This is where AR's visualization capabilities come into play, transforming abstract data into tangible and visually stimulating representations. Chatbots in particular generate reams of text, which we humans are comfortable with and enjoy. But when it comes to making relationships in our mind, really employing our visual cortex, humans have an amazing tool to build 3D mental models. How we each translate words into these models is unique for each of us, and also a lot of work for the brain. But with AR, now we can leverage all the brain processing power to make the models more detailed and more concrete. The hippocampus can use it’s spatial awareness to help you remember better because more mental relationships mean better long-term memory.

And here’s the real kicker: these mental models need to be shared with your audience or team, which is so much easier when seeing the model than describing the model to colleagues with a bunch of words. What is that worth to you?

AI’s reams of words don’t do that.

Conclusions

The implications of this vital connection between AI and AR are profound. From education to healthcare, manufacturing to marketing, the integration of these technologies revolutionizes the way we work, learn, and interact with information. Students can engage in immersive and interactive learning experiences, doctors can visualize medical data with unprecedented clarity, and businesses can gain deeper insights into their operations, leading to more informed decision-making.

At Flow, we are at the forefront of this transformative convergence. We’re already sharing our prototypes, and yes, everyone says it feels like Jarvis from Iron Man.

Please share your thoughts and ideas. We believe in the power of collaboration and the importance of collective intelligence. How can we make this practical for you?