SEMAINE-1.0: The first public SEMAINE demonstrator system
Released 23 December 2008.
The aim of the SEMAINE project is to build a Sensitive Artificial Listener (SAL) – a multimodal dialogue system with the social interaction skills needed for a sustained conversation with a human user. SEMAINE-1.0 is the first public demonstrator of the SEMAINE project, resulting from development in the first project year.
As a snapshot of “work in progress”, the demonstrator aims to illustrate the following:
- the SEMAINE project has proposed a system architecture for realising multimodal analysis and generation in a user-and-ECA dialogue situation;
- a SEMAINE API has been created to allow processing components to communicate with each others according to the architecture, using appropriate representation formats at interfaces between components;
- an initial set of components exist to partially realise the core functions of a SAL system.
The system is not yet intended to function as a fully operational, real-time SAL system; its main aim is to give an “early view” to technologically interested experts, indicating the direction in which the work is going. Individuals and research teams interested in cooperation around the SEMAINE platform are explicitly invited to contact the project team.
Installation and Use
As a pre-condition for installing and running the SEMAINE-1.0 system, you need to make sure that you have installed the Required Software.
The system comes in three parts.
- SEMAINE-1.0-java. The core part is written in Java. It is needed for all configurations of the system.
- SEMAINE-1.0-linux. Speech input components are native C++ components running on Linux and Mac OS X. You can run a speech-to-speech dialogue system with a preview of backchanneling capabilities on a single Linux/Mac OS X computer.
- SEMAINE-1.0-windows. Visual output using the Embodied Conversational Agent (ECA) Greta is realised using native C++ components running on Windows. You can run a system with text input and audio-visual output on a single Windows computer.
The full system with speech input and audio-visual output requires at least two machines: one running Linux/Mac OS X, and one running Windows.
Developer documentation
Detailed documentation of the SEMAINE API is available in section 3 of the deliverable report D1b: First integrated system.
For the SEMAINE-1.0 release, Javadoc is available.
Background Information
Detailed information on the system and the underlying software architecture can be found in the public project deliverable report D1b: First integrated system.
