AI Translation Glasses for Travel: How Do They Compare to Phone Apps? 

Key takeaways

• AI glasses can bring live translation directly into the ear of the user, removing the need to rely entirely on text in a smartphone app
• Users are able to register non-verbal cues more easily when using smart AI glasses, increasing comprehension
• Smart glasses enable users to ask AI assistants to quickly translate documents or menus from foreign languages
• The global real-time translation smart glasses market is set to grow to $7bn by 2032
• AI glasses often look just like normal spectacles, with a range of styles available

How do AI glasses help with translation?

Smart AI glasses use artificial intelligence, either on device or in the cloud, to live-translate conversations from a different language. 

AI glasses are also able to translate documents, menus and signs in foreign languages - giving the wearer access to information in dozens of dialects without needing to get distracted by looking at a smartphone.

While translation apps, such as Google Translate, are able to translate 249 languages, using an app requires the user to look up and down. This is a problem when speaking to another person as it breaks eye contact, creates social awkwardness and slows down the conversation.

When using AI translation in smart glasses, the physical limitations are overcome. By delivering translated words in close-to real time, the non-verbal conversation cues can also be picked up with ease.

This is key for understanding the nuances of human interaction - after all, in some circumstances communication can be mostly non-verbal. According to the University of Texas, you can derive 55% of communication through body language, 38% through tone of voice, and only 7% through spoken words, even if the person is saying something contradictory.

In a separate experiment, studies of thousands of TED Talks showed that 1,600 participants rated speakers who employed hand gestures as ‘more clear, competent and persuasive’, meaning it’s critical to be able to see the full range of motion, rather than looking down at a phone screen, from the person you’re speaking with.

Of course, there are advantages to apps - many allow you to download a huge amount of languages for use offline, where most AI glasses need a live connection.

This is to have the translation performed in the cloud, which can add a small amount of delay to the conversation. 

What can work best is a fusion of the AI glasses and an app. For instance, if two speakers are both wearing a pair of smart glasses, they both will be able to communicate in real time without needing to read an app.

But if smart glasses wearers need to speak to someone who’s not wearing AI-powered spectacles, then they can use an app to display the words in a different language to the other speaker.

This means the glasses wearer won’t need to look at the screen themselves - simply listening while the information is passed to the in-device speakers.

Do I need a display to get live translation?<H2>Do I need a display to get live translation?

No, you don’t. There are two main types of AI glasses, and they present live translation in different ways:

• Audio-first AI glasses don’t have a screen, with translation handled either on-device or through an app and then relayed back through the speakers 
• Visual-first AI glasses project information into the eyeline of the wearer allowing them to read translations rather than hearing them spoken 

Audio first

With audio-first glasses, the information is (unsurprisingly) passed directly into the ears of the user through the open-ear speakers in the arms of the glasses.

Listening, rather than reading, a conversation, allows the wearer to be more present in the moment, as it reduces the load on a single sense according to Wicken’s Multiple Resource Theory.

Audio-first translation is handled either on AI glasses themselves, or via the smartphone, where the data is sent to the app, passed to the cloud for translation, then streamed back into the ears of the listener.

Audio-first translation is found in the HTC VIVE Eagle, Solos AirGo 3 and the Ray-Ban Meta Gen 2 series. These glasses are lighter to wear than those with a display, and look more similar to conventional glasses.

These use advanced noise-cancelling algorithms to ensure translated conversations are kept private, using smart arrays of microphones to reduce sound leakage and picking the voice of the speaker with more precision.

These glasses can handle two-way conversations either by translating the audio into the user’s ear, and using a smartphone to play out the correct translated response or, if both people in the conversation are wearing AI glasses, they can simply speak and hear the words translated into their native tongue.

Audio-first glasses are usually lighter than visual-first, and thus are more comfortable to wear for extended periods of time, so users can continue conversations throughout the day. 

Visual first

Visual-first uses a small heads-up display to show information on a small display in one (or both) lenses. Glasses using this technology include Even Realities G1, RayNeo X3, Meta Ray-Ban Display, and XRAI AR2.

These glasses also use microphone arrays embedded within the frames to listen to the foreign language, but, instead of playing the words back into the ears of the listener, they’re displayed as captions floating in front of their face.

This is done using small projectors that bounce light onto the lenses and into the retinas of the wearer.

There are potential challenges to this technology. In sunny climates there might be an issue with visibility - if you’re using a heads-up display projecting information onto the lens, in bright sunlight this can be hard to make out if the smart glasses don’t have a powerful enough projector.

Choosing the right AI glasses when abroad or speaking to people in other languages depends on personal preference - audio-first glasses allow you to stay more present in the conversation without the distraction of reading, but visual-first glasses allow you to absorb the information at your own pace.

How can I use AI glasses to travel hands-free?

When translating text and speech abroad, you’ll need a pair of AI glasses with an accompanying app and, usually, a connection to the internet as well.

When equipped with these elements, you’ll be able to quickly interact with locals wherever you are in the world.

Menus and signs

AI glasses enable you to read signs and menus when travelling abroad, even if you don’t speak the language.

To do this, your AI smart glasses need to have a camera embedded, in order to take a picture of the text you want to translate. From here, the image is sent to the smartphone, where OCR (optical character recognition) is used to identify the characters.

This is then sent via the cloud for further processing, where the text is translated into the language you speak. The words are then converted back into speech and passed through the open-ear speakers, translating the text to the user.

Alternatively, if the glasses use displays in the lenses, the text is translated in real time in front of them using the same OCR technology.

How to translate menus, signs and documents in another country<H3> How to translate menus, signs and documents in another country

1. When you see an image of text you don’t understand, simply say the wake word of your smart specs to activate AI (or tap the dedicated button), and ask it what the text that you’re looking at says.
2. The AI glasses will take a picture of what’s in your vision, interpret what’s being looked at, and auto translate it when it detects a different language.
3. If the glasses are audio-only, the information will be passed through to the speakers, and read out to the wearer within seconds, allowing them to understand and interpret text even in another language.
4. If the smart glasses have a visual display, the information will be shown in the user’s eyeline.

This works in a variety of languages, depending on the glasses in question. For instance, HTC VIVE Eagle allows users to translate 96 different languages, such as French, German, Finnish and Chinese, whereas others, such as Meta Ray-Ban or Even Realities allow you to translate up to around 35.

When taking pictures of foreign text, these images can be saved to your glasses. Make sure that your AI glasses give you the choice of whether to sync these to your smartphone app, rather than auto downloading, as this saves storage space and maintains privacy in case the documents in question are sensitive.

Live conversations 

If you’re visiting another country and want to use AI glasses to converse with a native speaker - perhaps in a meeting or making small talk in a cafe - there are a couple of ways you can do it.

Method 1: both wearing AI glasses

1. Both press the AI button your smart glasses or say the wake word, and command the glasses to start live translation
2. Begin speaking at an even pace
3. Each pair of glasses will translate the text back to the listener, so wait a short time for the other person to respond.

Method 2: Only one person wearing AI glasses 

1. If the other person isn’t wearing smart spectacles, open the AI glasses’ app and enable live translation
2. Speak your command into the phone - ideally, letting the other person know they can respond in their native tongue
3. Let the app translate, and let the other person read the text
4. Continue the conversation, with their words translated into your ear, while they’re able to read your responses on the phone screen in their native tongue.

In this method, the wearer does not need to look at their smartphone during the conversation, allowing them to see body language and other nuances and gain a better understanding of the conversation. 

For both of these methods, any delays when using the AI translation tools are usually because of slower connections, so always make sure you’ve got the fastest data speeds on offer, as research shows any delay of longer than 5 seconds can result in errors in comprehension.

5G is a good, low latency choice (meaning there’s very little delay between sending and receiving the information) and helps enable more fluid conversations.

When is it better to use a phone app for translation abroad?

• AI glasses work best when translating conversations when the other person is also wearing smart spectacles for the same reason, leading to fluid interactions across many languages.
• They’re also perfect for translating signs, menus or any literature in another dialect, as they follow your line of sight and don’t require dragging out a phone.
• Phone apps are best for translation when not all speakers have AI glasses, allowing the wearer to get responses translated, while other participants can read responses in their native language on a smartphone screen

Smart glasses will generally be the better option for translation, as they can follow your gaze and keep you present in the conversation. However, there are times when you might need to default to a translation app.

Noisy settings 

If you’re trying to have a conversation in a busy airport, cafe or train station, AI specs with embedded microphones will capture a large amount of the speech, but some smart glasses will struggle with too much ambient noise.

To reduce this problem, always look for AI glasses with at least four multi-directional microphones as, combined with proprietary AI platforms, this will allow the smart specs to help distinguish a speaker’s voice from background sounds. . VIVE Eagle has five on-board microphones, to optimize sounds by using clever noise cancelling algorithms.

However, if the distance from the speaker is more than 3-4 feet away with high levels of ambient noise, translated conversations through glasses can be disrupted.

This is less of an issue for smartphone apps, so the best AI glasses will have a smartphone companion app that can be used even without the smart spectacles on.

Bright sunlight

If you’re using smart glasses with a heads up display, you may find using them in hot, sunny countries can be tricky, especially if the display is under 1000 nits of brightness. 

If this is the case, using an app can be simpler and more straightforward, rather than not being able to read what’s on the screen in front of you.

(Nits, or cd/m², are units of brightness measurement - for indoor conditions, 500-1000 nits is enough to see crisp images. However, in direct sunlight, the brightness is hundreds of times stronger, so you’ll need a display with thousands of nits to compensate and read text updates clearly).

Some models of display-enabled AI glasses, such as the RayNeo X3 will have over 3,000 nits of brightness (and can go even higher if needed, up to 6,000 nits) but these can retail at more than $1,000 and weigh at least 70g to compensate for the added hardware.