Zurich-based startup Mictic has launched an experimental wearable that converts gestures and movements into music tones and sound effects using the mobile app we designed and developed. The Mictic product generated a lot of interest when it was launched in 2021 and was featured on BBC, Mashable's Future/Blink and Cheddar TV, among others.

Audio augmented reality

Mictic is an app to make the hearts of air guitarists beat faster. Working in conjunction with the connected wristbands, it interprets the user’s movements as sounds. While the app plays a key role in this process, it is designed to function completely in the background as the user ‘plays’. Once the app and wristbands are connected, the user can make music with Mictic even while the smartphone is locked, allowing them to focus fully on the movement and the resulting sounds.

The app shown here is just the beginning. Currently, there are plans for new functions to enable multiple Mictic owners to connect directly to the metaverse. There is potential for several more applications based on the present technology - for example, games or rehabilitation. The opportunities are vast. For now, though, they remain visions of the future.

The features

Once the wristband is assembled and connected to the iPhone, users can choose from different experiences in the app:

Instruments
Soundscapes
Songs
Games

Each experience requires the user to create different movement patterns and gestures. Some of these can be picked up intuitively by the user (e.g. the cello experience), as they simulate the physical actions involved in playing the instrument in question. There are also more complex experiences such as ‘soundscapes’ (e.g. hip-hop), for which the user must first understand and learn the movement patterns and gestures involved. The app provides tutorial videos for each type of experience to enable the user to learn quickly by imitation.

For the most part, the instrument sounds are generated by interpreting the rotation acceleration of the sensors wherever they happen to be located in space. By contrast, in the soundscape experiences, virtual zones are placed around the user and a variety of sound effects are triggered in each zone. Additional sounds can be activated by integrating defined gestures (e.g. pulling both hands down sharply).

The individual experiences come with their own corresponding detail settings. For example, the guitar experience can be set for a right or left-handed player, or additional playing options can be switched on or off (e.g. full drum set vs. snare + bass drum for the drum option).

Design and identity

We were lucky enough not only to support Mictic with the design and development of the app, but to help shape the entire corporate identity. The software was adopted as a starting point for this process. The result was a dark or ‘inverted’ appearance characterized by pink and blue accents.

The logo animation with the color highlights, which serves as the app intro.

As well as injecting playfulness into the app’s graphic design, the dual-color accents serve a practical function. Upon connecting the two wristbands to the app, the user is prompted to move their right hand. Once the app has detected which wristband is on the right wrist, the color of the two dots on the UI changes accordingly:

When the wristbands are connected to the app, the two colored dots controlled by the wristbands move around in the background of the main menu. This serves as playful feedback to the user that the wristbands are connected and ready to use.

The accents function as visual feedback as the user connects the wristbands to the app.

In the future, this motion-responsive visual feedback will also be deployed to guide users through the individual experiences and illustrate the movement patterns and virtual zones. This feature is currently under development.

The app uses the ‘GT Maru’ font from Swiss type foundry Grilli Type throughout. The rounded typeface is the perfect complement to the icons, which were custom-designed for each experience and are animated in eye-catching fashion to signify newly unlocked experiences.

Credits

Our services: Concept, UX research, ideation, design, iOS software development, testing
Audio engine: Robert Thomas
Sensor development: Arendi AG
Customer: Mictic AG
Platforms: iOS