AiboHack

AIBO Voice Recognition Technical Info

OLDER WEBPAGE (not being kept current) - a technical discussion of voice recognition and phonemes

Files

A voice dictionary is stored in 3 files:

Phonemes

Each language has its own HMMDATA.BIN file. These represent the phonemes of the language. They are called "PLU"s (Phone-Like-Units or Phoneme-Like-Units) since they aren't always strictly phenemes. There are many versions of each PLU.
Currently there are four different HMMDATA.BIN files:

Phases, Commands

There is an important difference between a phrase (DicID) and voice command (VCmd#?). Each voice command has one or more phrases. For example "Good Boy" and "Good Girl" are both interpreted as VCmd#9. Also, there may be more than one way to pronounce a word or phrase (for example 'g uh_' and 'g uh_ d' for the word "good"). Each different pronunciation has its own phrase dictionary entry.

Which phrases are recognized is determined by the phrase dictionary (DICDATA.BIN). How the phrases are turned into semantic voice commands is determined by the ommand dictionary (SEMATA.BIN).

Even though all English versions of AiboWare share the same HMMDATA.BIN file, they may have different SEMDATA.BIN and DICDATA.BIN files. Currently there are three different command sets: AiboLife (which is the same as Hello AIBO), Party Mascot and Messenger (which is the same as Navigator).

How it all works

At the heart of AIBO's voice recognition is a Hidden Markov Model (HMM) voice recognition engine. The voice recognition is "continuous", which means AIBO is listening to almost everything you say, and trying to recognize the words and phrases it knows (in the phrase dictionary).

Sound comes in the ear microphones, is digitized, (using LPC I believe) and then Vector Quantized. I don't have many details about the Vector Quantization approach used, so details in this area are vague.

Each of the phrases in the phrase dictionary (DICDATA.BIN) is tested to compare how closely it matches the sound heard (the number of phrases is limited by the CPU speed). The best matched phrase is then turned into a semantic voice command (by looking in SEMDATA.BIN).

After that, the semantic voice command is sent to the behavior logic (as in the case of AiboLife) to trigger some specific reaction to hearing the command. In the case of RCode Plus, the value is stored in "AP_Voice_Cmd".

// add picture here

Using AiboScope Live!

If you have your AIBO attached to your PC via the WiFi LAN, you can use AiboScope LIVE! to monitor the voice commands heard by your AIBO. In addition to the voice command detected (VCmd#, the semantic voice command), the "dicid" from the phrase dictionary is also displayed for the top 10 matches. Be sure to pick the proper language vocabulary from the Aibo Scope menu (English AiboLife is the default).

RCode Plus

If you have gotten this far in the technical explanation, hopefully you are already programming AIBO using Master Studio and RCode Plus. Using RCode Plus, you can do some more advanced things.
After using the "AP_Voice_Cmd" variable to isolate a voice command you are insteresting in, you can use the "AP_Voice_DicID" variable to further distinguish which phrase was said.
The default RCode Plus voice command set is the AiboLife set (in English, Japanese or French). If you are unhappy with the default (AiboLife) vocabulary, you can use the Party Mascot or Messenger voice commands instead. Email me for details.
For more advanced applications, the DICDATA.BIN file can be customized to your own set of voice commands. I am still working on this, so email me for details.

Recorded Name / Name Registration

When AIBO records his name with "Name Registration", the same phoneme dictionary and phoneme phrase analysis is used. This means recorded names are language specific (what works on a Japanese stick will may not work on an English stick and vice-versa).
The file ARRDATA.BIN on the memory stick saves the recorded name in a format similar to the unchanging DICDATA.BIN file.
The ERS-210 Browser (version 1.1 or later) will dump the phoneme's of your AIBO's recorded name.