Unlisted voice commands on Google Glass

Here at 3spin  – my current employer –  we lately experimented with unlisted voice commands on Google Glass. During the process we discovered some pitfalls you might encounter as well.

Why unlisted voice commands?

On Google Glass there exist two types of voice commands. As Glass is mostly a hand free device you can define a voice trigger command – most likely a short sentence – to start your application without any physical interaction. The other type of voice command is called contextual menu command and can be used to navigate the menu structure of your application by voice only.

A list of both voice trigger commands and contextual menu commands is build in in every Glass platform release. Google claims these commands are especially tuned and therefore will be recognized particularly well.

“Glass voice commands are carefully tuned and designed for the best accuracy and recognition. This means they take quite a bit of time for us to model and develop.”

https://developers.google.com/glass/distribute/voice-checklist

When developing Glassware you may need other voice commands than the build in ones. For example there are contextual menu commands for “play video” and “pause video” but none for “stop video” at the moment. In this case you could submit the missing command to Google for approval, hoping it will pass and be part of the next platform release.

In the meantime it is possible to use the desired command as an unlisted voice command. Any string can be used as an unlisted command as long as the DEVELOPMENT permission is requested in the AndroidManifest.xml file of your application.

Why not use unlisted voice commands all the time?

Fixed, build in lists of voice commands do seem rather inflexible at first. Application developers will be highly dependent on Googles approval process and the XE versions currently installed on their audiences devices. However, there are several disadvantages to be aware of when using unlisted voice commands.

Worse recognition quality

We have not tested this in detail but it seems very likely that unlisted voice commands will have worse recognition rates than the build in, fine-tuned ones. Especially longer and more complex unlisted commands are only recognized with some difficulties.

No MyGlass approval

This is a no-go for most applications. The DEVELOPMENT permission required for unlisted voice commands prevents the application from being approved for MyGlass  – Google Glass’ own app store. So you either have to wait until your unlisted voice command gets approved and part of the next platform version or you don’t care about market access at all. The latter may be the case for internal business applications that are distributed through other channels than MyGlass.

Needs network access

This took us a while to figure out. Unlisted voice commands need network access at least one time before they can be used. Our theory is that these commands need some kind of pattern definitions that are downloaded from Google servers. This happens right after you have installed the application with unlisted voice commands. Glass will contact a Google server, download the definitions and cache them on the device. After this process the unlisted voice commands will work fine, even with all network connections turned off.

When using a Glass not connected to a Wi-Fi network or paired with a phone that is connected to a Wi-Fi network your unlisted voice commands will just not be recognized and you might be wondering why. We encountered the problem that our Glass was paired with a phone that in turn was connected to the cellular phone network. So Glass had network access but nevertheless it refused to download the definitions. When connecting the phone to our Wi-Fi network the unlisted voice commands started to work almost instantly.

Roundup

Unlisted voice commands are a great tool for prototyping and development as long as you are aware of their disadvantages. Also they might be useful for business applications in controlled environments. For production ready consumer applications however you should submit missing voice commands as soon as possible for approval.