Development of Apple Vision Pro app

Since the COVID-19 pandemic, the boundary between the real and virtual worlds has rapidly blurred, and the use of e-commerce has quickly become prevalent in society. However, opportunities for communication with friends, family, and retail staff have decreased, highlighting the concern of reduced experiential value from shopping itself.

To address these challenges, TOPPAN has developed ‘Metapa,’ a metaverse shopping mall that offers virtual stores while also linking with physical stores to enable an integrated experience. Takram has been involved from the concept planning stage, supporting research and development.

In February 2024, the Apple Vision Pro was launched in the United States, garnering significant attention. This wearable device is noted for its spatial computing capabilities, seamlessly integrating digital content into the physical space.

Takram supported the planning and development of ‘Metapa for Vision’ in conjunction with the domestic release of Apple Vision Pro. From an early stage, we conducted research and prototyped using simulators and actual devices, in pursuit of a new shopping experience unique to Apple Vision Pro.

The Apple Vision Pro can simulate physical space and ambient light, presenting virtual objects as if they exist in the real world. ‘Metapa for Vision’ leverages this feature, allowing users to grab virtual products and spaces from miniature stores into their own rooms, enabling a multi-sensory experience of ‘feel and enjoy.’ Items brought into the user’s room are illuminated with ambient light to blend into the real space and can physically interact with the user’s room.”

Unlike conventional Head Mounted Displays (HMDs) that offer a fully immersive experience, the Vision Pro is a multitasking device that allows users to simultaneously experience the real world while opening multiple apps. In ‘Metapa for Vision,’ we focused on the multitasking nature of the device by incorporating an auditory ‘multitasking experience.’ As users explore the miniature world, they can hear sounds from the radio and the chatter of NPCs (Non-Player Characters), creating an experience where users naturally ‘learn and enjoy’ information about products and stores.

To leverage the multitasking capability for an optimal ‘multitasking experience,’ we believed it was important to design the flow of the app experience to start with real-world interactions and gradually lead users to immersive key moments, rather than immersing them from the outset.

In ‘Metapa for Vision,’ the experience begins with the user ‘learning and enjoying’ in a miniature world that appears in their room. As users gradually immerse themselves, the user experience leads them to key moments where they ‘feel and enjoy.’ Once these key moments have concluded, users can leave comments or emotes to share their impressions and ‘connect and enjoy’ with other users. By designing the app to align with the unique experience curve of spatial computing, we explored new possibilities for the shopping experience.

Maintaining the concept of ‘high-tech, high-touch’ that Metapa has upheld from the beginning, an important point in this development was how to utilize spatial computing to create an experience with human-touch. Instead of searching for organized products like you would on e-commerce, we aimed to let users naturally encounter interesting products and stores while ‘enjoying’ the spatial computing experience.

In addition to products, the stores feature gachapon machines that users can spin and play with, as well as retail staff that can be picked up and moved around. The stores are bustling with avatars freely walking around, and you can’t help but listen in on the spontaneous chatter that arises. Furthermore, the audio from the stereos is enhanced with low-pass filters and noise processing to create a more authentic radio effect. These dynamic and playful elements are designed to spark enjoyment and curiosity.

Experiences in the metaverse, VR, and spatial computing often share the common challenge of users feeling isolated in virtual spaces. In response, Metapa adds various dynamic and playful elements to the experience, providing opportunities for users to enjoy themselves through serendipitous discoveries and encounters with NPCs. This approach creates diverse moments for enjoyment and fosters connections between users and brands.