Can Your Smart Glasses Control Your Phone? Android + Gemini Integration Explained
Key takeaways
- Frequent smartphone pickup has been shown to have negative mental health benefits, where AI glasses help offset this need
- Using HTC VIVE Eagle gives you access to Google Gemini, allowing you to control your phone using your voice
- Google can be used to control your basic phone functionality, such as setting alarms, timers or making calls
- However, Gemini can be accessed to perform web searches, interact with apps and provide richer, deeper information
- You’ll need to do a little set up at the start, but this will pay off in terms of giving you more complex functionality
What does Google in AI glasses mean?
The idea of constantly relying on our smartphone is one that many of us want to get away from. A recent study of smartphone usage showed that Americans pick up their phones 186 times a day, and that’s causing an issue as too frequent pickups caused mental overload.
In fact a study from Aalto University found people using their smartphone frequently before 12 PM were far more likely to suffer from mental overload - so it would make sense to just use a smartphone less, right?
However, there’s a reason that many of us do so - it’s to work, stay in contact with loved ones or just do simple things like setting an alarm or turning on the torch.
In that moment of pickup, distractions can occur, even with the greatest willpower in the world.
However, combining the use of your phone with smart glasses can allow you to get the best of both worlds: the ability to have access to vast amounts of information and control your phone at the same time, without needing to constantly fish around for it in your pocket.
HTC VIVE Eagle is unique in that it’s a pair of AI Glasses that offers connection to its own LLM, VIVE AI, for quick, local queries - but also to Google’s Gemini, allowing you to control your phone with just your voice.
This results in fewer moments where you need to unlock your handset, which can have significant mental health benefits.
A two-week study blocked internet access from smartphones (but not other options, such as desktop computers) and found that 91% of participants had improved mental health, better well-being or a better ability to maintain attention.
So while it’s important to maintain connection to the world, being able to remove the need to look at phone screen regularly can really have beneficial effects.
Why HTC VIVE Eagle uses Gemini
If you’re using an Android phone, then you won’t be able to control your phone with smart glasses unless it’s connected to Gemini. You were previously able to use Google Assistant to control your smartphone, but this was retired on 4 September to support the rollout of Google Gemini.
HTC has access to both OpenAI GPT and Gemini onboard VIVE Eagle (and sends de-identified data so neither platform can easily build a profile of you, preserving your privacy with AI glasses).
But given there are two third-party assistants embedded within the VIVE Eagle, why can’t both be used to control your phone using your voice?
Well, most AI assistants on smartphones are sandboxed, meaning they’re locked into a certain amount of actions they can take. This is to prevent them accessing sensitive parts of the phone and performing unintentional actions.
However, as Google has developed Gemini itself, it’s integrated within the Android operating system, giving users far more access to key controls. This is possible as Google has tested and approved the functionality - and adds more features regularly.
This means Gemini has wider access to your smartphone meaning, when combined with VIVE Eagle, AI glasses wearers can access key Android apps using just their voice
How can smart AI glasses control Android?
There are two ways you can use Google’s Gemini with HTC VIVE Eagle, depending on what you want to achieve:
- Gemini to control your phone
As you can access Google’s AI assistant directly from your glasses, you can use it to connect to your phone’s utilities.
This means you can use it to do simple tasks, like setting a timer, creating an alarm or turning the flashlight on (which is great if you can’t find your phone in a dark room).
You can also get access to your phone’s calling and messaging functions, using the interwoven capabilities to access core communication abilities.
2. Accessing the power of Google using your voice
With the power of Gemini in your smart AI glasses, HTC VIVE Eagle has the capability to answer many questions about the latest news, information from the web or interacting with apps from the Google Suite.
This is where the power of Google in smart AI glasses really comes to the fore - you’re suddenly able to ask ‘What new emails have I got today?’, ask which events are coming up in the next few hours, or summarize documents in your Google Drive if needed.
Of course, these actions aren’t meant to replace the smartphone, but as we talked about above, it’s designed to stop you reaching for your handset and be deluged with notifications and distractions.
Once you’ve set the correct permissions, you gain control of a large number of functions on your phone through Gemini.
However, some things will be guarded for security, such as things that require biometric security (i.e. banking apps), any secured sections of the phone and multi-step queries, to avoid confusion when not looking at your device.
What can and can’t I do with Google on VIVE Eagle AI glasses?
Allowed Safeguarded
Calendar and email management Biometric access (banking apps or fingerprint / face locking)
Phone function control (toggling Wi-Fi, Bluetooth etc) Outside of signal - Gemini won’t function without a connection
Dictating messages on text or WhatsApp Enterprise containers - if your phone has a secured section, this can’t be accessed
Image context - using camera to understand images Multi-turn prompts - only singular prompts to keep functionality simple
There’s a whole world of things you can do with Gemini on your phone - Google has a full list to help you navigate the options.
How to control your Android phone using VIVE Eagle
Controlling your Android phone with AI glasses takes a few moments to set up, but it will allow you instant access to a range of Google apps and functions.
- Set your assistant
Once you’ve paired your glasses to your phone, head into the Android system settings and define your digital assistant of choice (Settings>Apps>Default Apps>Digital Assistant App). Ensure this is set to Google
(Note - if you’re running a Samsung phone, you might need to head into the Bixby app and disable it so it doesn’t confuse the interactions. Settings>Apps>Bixby>Disable (at the bottom of the screen)).
2. Connect the apps
Open the Gemini app on your phone, hit the account profile icon in the top right corner, and hit ‘Connected Apps’.
Here you can activate whichever functionality you want - users will want to turn on ‘Device Assistance’ if they want to be able to control their phone from HTC VIVE Eagle.
You can also access Spotify for music control, Google Workspace to interact with Google Docs, Gmail and Calendar, as well as Phone and Text capabilities.
3. Lock screen
Head back into the Gemini settings menu and find ‘Gemini on lock screen’ - if you want to be able to access Gemini without unlocking your phone, this is important as it won’t wake the phone and will let you interact hands free.
4. Speed up access
You can customize the AI button on VIVE Eagle to activate Gemini with a quick double press, shortcutting the time it takes to get access to Google.
Open the Settings menu in the VIVE Connect app, and head to Setting the AI button behavior panel.
Choose to enable single- or double-press to start your phone’s assistant (in this case, it will be Gemini) and you’ll have control of your phone with just a tap or two.