wiki:SEMAINE-2.0

Version 3 (modified by masc01, 10 years ago) (diff)

--

SEMAINE-2.0: The first full implementation of an autonomous Sensitive Artificial Listener system

Released 22 December 2009.

The aim of the SEMAINE project is to build a Sensitive Artificial Listener (SAL) – a multimodal dialogue system with the social interaction skills needed for a sustained conversation with a human user. SEMAINE-2.0 is the first full implementation of a SAL.

Installation

As a pre-condition for installing and running the SEMAINE-2.0 system, make sure that you have installed the Required Software.

The open source parts of the system can be downloaded from http://sourceforge.net/projects/semaine/. It comes in three parts.

  • SEMAINE-2.0-windows (~100 MB) includes binary versions of the Greta agent components, the Opensmile speech analysis components, and the message-oriented middleware ActiveMQ.
  • SEMAINE-2.0-java (~650 MB) includes the System manager component, the dialogue components, and the speech synthesizer MARY TTS. The file is so large because the speech synthesizer comes with four high-quality TTS voices which need a lot of space.

To run the SEMAINE-2.0 system, both the windows and the java package are required. They can run together on a fast machine (tested on a laptop with a 2.53 GHz Core2Duo CPU with 4 GB RAM), or you can set up SEMAINE-2.0 as a distributed system.

  • SEMAINE-2.0-source is an optional package which can be used to compile the system for Linux or Mac OS X, or to rebuild the windows components from source.

The Video analysis components are distributed as closed-source freeware from http://www.doc.ic.ac.uk/~maja/ (search for "SEMAINE Visual Components" on that page). Watch out for the Camera driver requirements if you are using a Firewire camera. If installed in the default location, the SEMAINE-2.0-windows start.bat script will notice that the video analysis components are installed and will try to run them. Since they are computationally heavy, you may need an additional computer to run them.

The SEMAINE-2.0 system will work without the video analysis components, but will then not be able to pick up the same amount of information from the user.

Developer documentation

With some effort you can build the components from source.

Detailed documentation of the SEMAINE API is available in a number of documents:

Background Information

Detailed information on the system and the underlying software architecture can be found in the public project deliverable report D1b: First integrated system.