Brown University researchers open source AR mobile hand-object interaction library

As the metaverse buzz continues to heat up, discussions about the technologies that will power it are also becoming more mainstream. In a previous article published by VentureBeat, Gary Grossman, senior vice president of technology practice at Nobleman and Global Lead of the Edelman AI Center of Excellence, said AI, AR, VR, 5G and blockchain could come together to power the metaverse. However, other experts like Louis Rosenberg, CEO and chief scientist at Unanimous AI, believe that augmented reality (AR) — not virtual reality (VR) — will be at the heart of the metaverse.

A new chapter in the AR conversation is smartphone AR, which allows everyday users to interact with the virtual environment. Many companies in the space are focusing on AR as a tool to improve the effectiveness and efficiency of the virtual sales processes. According to Gartner, current trends show: 75% of capital-intensive industries will use AR as a key component for cost reduction and avoidance among frontline workers.

Enterprise-focused AR and VR solutions are proof of AR’s ever-growing power, with Google’s powered Project Tango and Apple’s IR sensors showing the potential of AR to turn simple devices into genius supercomputers that leverage the use of being able to understand, predict and anticipate space . AR’s promises are exciting, but the cost of devices and the expertise it requires hinders its development and implementation at scale.

While AR seems to be penetrating the enterprise, Jeff Huang, an academic researcher and professor of computer science at Brown University, told VentureBeat that the enterprise is an escape route for failed consumer AR. Huang said certain Chinese manufacturers will launch wireless AR glasses in 2024 – 2026 that will be more accessible than any enterprise-grade AR by big tech.

Huang and other staff on Brown HCI .’s Portalble concept believe AR on smartphones will become ubiquitous and want to help the race to a digital-first future by open-sourcing the project that the research team has been building for years. The open-source mobile AR hand-object interaction library currently runs on phones, with no additional hardware required, but Huang said there will eventually be a more premium version with a Leap Motion. controller attachment†

As an open-source documentation project, Portalble allows developers to use the codes in their projects. Without the necessary gadgets that usually come with using AR, Portalble allows a normal Android phone to become a centerfold for AR action.

Time is ticking for a convergence of worlds

AR allows users to interact with an object in the magnified space with their body parts. This creates a distinction from the metaverse, where the whole experience takes place in a space outside the physical world as we know it. Perhaps the AR experience reveals the need for a convergence of worlds – a shift from the norm – but merging technologies and bringing devices together to actualize the experience is often problematic.

Aleissia Laidacker, global director of creative technology at The millwrote in a Forbes article that due to cost, there is still a long way to go before people can fully exploit the potential of AR. But Huang seems to think the time for consumer AR is closer than we think, noting that 2025 – 2026 will be the year when consumer AR becomes impressive for everyday use. However, he noted that we won’t see AR everywhere until 2029. He cited the issues of technological advancement and the need to build and customize devices for an overall seamless experience as part of the hurdles that must be overcome before AR can become commonplace. Not only that, he also noted that light field cameras will become part of AR glasses for better tracking in the near future.

This timing is in line with experts’ predictions for the metaverse blooming. Gartner predicts the metaverse will not be fully mature before 2030 and advocates immediate testing of devices and technologies essential to the metaverse. AR is one of those technologies. So is a Forrester article identifies a mix of VR, mixed reality (MR), and especially AR as the keys to unlocking the best of the metaverse.

With the time ticking for the convergence of the various technologies into the building block of a bold new world, Huang is convinced that the scope of the work done on Portalble is enough to fit it into the metaverse discussion. Although he notes that current battery, overheating and field of view (FOV) challenges need to be solved before that can be updated.

AR for everyone

Portalble AR can turn any Android phone into an AR portal, allowing users to transcend the real world and move virtual objects with their hand. Various actions can be performed, such as sketching, manipulating and throwing objects, as demonstrated on Portalble’s repository page† The difference between what Portalble offers and what is currently popular in the AR space is that Portalble allows for direct interactions with the objects, rather than the usual hands-in-the-air gesture.

With the open-source nature of Portalble’s AR infrastructure, security can become a concern for developers and researchers. Open source is a double-edged sword: it is the backbone of digital innovation, but it also poses unique threats. However, Huang defended the decision to make Portalble open source, arguing that it is best to leave the project this way, in the spirit of academic research and honest belief in the untapped potential of every other researcher in the space. He hopes that one day other researchers can build on the work already done.

Huang said that while there are other players in the space such as Manomotion, he claims that Portalble is the best technology for working by hand alone. He said Portalble has a key advantage over other AR projects, which is that it allows developers to leverage Portalble’s framework and incorporate it into their games and apps. Huang’s goal is to make AR available everywhere, knowing that others are building for other devices – which is why AI for mobile phones has been a big focus for the company. Developers interested in Portalble’s AR solutions can check out the GitHub . see warehouse†