wiki:SEMAINE-1.0-speech2face

SEMAINE-1.0 speech to ECA dialog system

This page describes the full system demonstrator, combining speech input with Embodied Conversational Agent (ECA) output.

Pre-condition for all demonstrator configurations

Start an ActiveMQ server.

If you intend to run the demonstrator distributed across several machines, read Running a distributed system.

Text input, ECA output

In this configuration, the user the user types his/her input via a GUI window, and the system response is produced by a speaking ECA.

This system configuration pre-supposes installation of SEMAINE-1.0-java and SEMAINE-1.0-windows.

The system can be started as follows.

1. Start the Java component

  • on Windows:
    SEMAINE-1.0\bin\semaine-speech2face.bat 
    

2. Start the Windows components

The following two Windows components need to be started. Either call the .bat files from a command prompt or double-click them. Note: if you are Running a distributed system, you need to use a command prompt to set the environment variable CMS_URL.

Run

    SEMAINE-1.0\bin\semaine-behavior-realizer.bat

to start the behavior realizer; run

    SEMAINE-1.0\bin\semaine-greta-player.bat

to start the Greta player.

Speech input, ECA output

In this configuration, the user speaks to the system through a microphone, and the system response is produced by a speaking ECA.

This system configuration pre-supposes installation of SEMAINE-1.0-java, SEMAINE-1.0-linux and SEMAINE-1.0-windows. It must necessarily be run as a distributed system, because it involves both Linux and Windows native components.

The system can be started as follows.

1. Start the Java component

  • on Linux/Mac/Unix:
    SEMAINE-1.0/bin/semaine-speech2face.sh
    
  • on Windows:
    SEMAINE-1.0\bin\semaine-speech2face.bat 
    

2. Start the Windows components

The following two Windows components need to be started. Either call the .bat files from a command prompt or double-click them. Note: if you are Running a distributed system, you need to use a command prompt to set the environment variable CMS_URL.

Run

    SEMAINE-1.0\bin\semaine-behavior-realizer.bat

to start the behavior realizer; run

    SEMAINE-1.0\bin\semaine-greta-player.bat

to start the Greta player.

3. Start the Linux components

Make sure you have compiled the linux code.

The minimal Linux component to start is the SMILE component, doing feature extraction, voice activity detection, and emotion/interest recognition. Start it as:

SEMAINE-1.0/bin/run_components/start_component_tum.smile

In addition, it is advisable to start the Automatic Speech Recognition (ASR) component, so that the system has a chance to understand what the user is saying. (Note that the quality of ASR output at this stage is extremely limited due to very preliminary training data.)

SEMAINE-1.0/bin/run_components/start_component_tum.asr

Testing microphone level

It is essential for the proper functioning of the SMILE component that the microphone recording level is set to a reasonable value. To test that, watch the shell from which you started the SMILE component. When you start talking, the SMILE component should output a message saying "detected turn start", and when you stop talking, it should output a message saying "detected turn end". If there are no "turn start" messages, increase the recording volume; if there are no "turn end" messages, decrease the recording volume.

Last modified 8 years ago Last modified on 01/05/09 15:31:44