LU4R app for speech recognition

Credits

LU4R is an Android app to interface Google Speech Recognition with MARRtino robot. It has been developed by Andrea Vanzo within the LU4R project. Wilson Villa contributed with several improvements.

This version of the app has been adapted for the use with MARRtino robot.

Minimum requirement: Android – 5.0 Lollipop

Download

lu4r-android-app.apk

Installation

Copy the apk file on your smartphone and run it. Note: you may need to enable an option to install third-party software.

Network connection

MARRtino software (either on real robot or simulated one) must be connected to the same network of the Android app, which must be connected to Internet.

Possible options:

1. Use an Android device connected with MARRtino through wireless and with Internet through its data connection

2. Use an external router with Internet access and connect MARRtino and Android device to such a router

Note: To connect a VM with the LU4R app, the VM and the Android device must be on the same network. First make sure that the two machines are connected to the same WLAN (e.g., using a common router or with the real machine connected directly to the Android device acting as a router). Then in the VM you must select the 'Bridged network' ('Scheda con bridge') mode in the Virtual Network Settings of your machine. This can be done even when the VM is on, by selecting on the top menu Macchina -> Impostazioni -> Rete -> Connesso a "scheda con bridge" (or using the network icon in the bottom bar of VirtualBox window, disconnect the network card, modify the configuration selecting bridged network, and reconnecting the network card). To check the IP address received by the VM, issue the command ifconfig on a terminal of the VM. The IP mentioned here is the one that must be configured in the settings of the LU4R app.

Another option is to activate a router on your Android device and connect the laptop to the WLAN generated by this router.

Bringup Audio server

Web start (MARRtino version 2.x)

Use web Bringup panel to start Audio service

Manual start

1. Make sure you have the update version of marrtino_apps

cd src/marrtino_apps

git pull

2. Run MARRtino ASR server

cd src/marrtino_apps/audio

python audio_server.py

3. Run the robot (real or simulated in the VM)

cd src/marrtino_apps/stage

roslaunch simrobot.launch

Use Android app

1. Run LU4R app on your Android device, select the 'Settings' menu and 'Speech-To-Text', choose your Speech language and select connection options (suggested: all disabled = white/left dot)

2. Select 'Connection' in the 'Settings' menu and enter IP address and Port of the MARRtino robot or Virtual Machine (Port 9002 is used by default by MARRtino ASR server).

3. Select 'Speech Interface' from the main menu and connect to the robot (use connection symbol in the top bar).

4. Push and keep pressed the microphone button while you say something, the text should appear in the top of the message list and if connection is enabled this text is sent to the audio server on the robot.

Note: if Continuous ASR is enabled in the Speech-To-Text options, then you can just press the mic button once to activate ASR that will listen to sentences until you press it again to deactivate ASR.

Test

On your VM/Raspberry

Run the audio_command.py example to drive the robot with spoken commands (this example is in Italian and recognize commands like 'avanti', 'indietro', 'destra', 'sinistra', 'esci').

cd src/marrtino_apps/program

python audio_command.py

In general, you can use the asr() function (defined in robot_cmd_ros.py) in a program to get the string sent by the app and process it accordingly.