Building emotion-oriented systems with the SEMAINE API

The SEMAINE API is the component integration middleware created for the SEMAINE project, serving as the integration layer for system components in SEMAINE.

The SEMAINE API uses a message-oriented middleware (MOM) for all communication in the system. As a result, all communication is asynchronous, which decouples the various parts of the system. The actual processing is done in “components”, which communicate with one another over “Topics” below the named Topic*. Each component has its own “meta messenger”, which interfaces between the component and a central system manager. When a component is started, its meta messenger registers with the system manager over a special meta communication channel, the Topicsemaine.meta. At registration time, the meta messenger describes the component in terms of the data Topics that it sends data to and that it receives data from; if the component is an input or output component (in the sense of the user interface), that status is communicated as well. The system manager is keeping track of the components that have been registered, and checks at regular intervals whether all components are still alive by sending a “ping”. In reply to such a ping, each meta messenger confirms the respective component's status and sends debug information such as the average time spent processing incoming requests. The system manager keeps track of the information about registered components, and sends global meta messages informing all components that the overall system is ready or, if a component has an error or is stalled, that the system is not ready. Also, the system manager resets a global timer to zero when the system becomes ready. All components use this global time via their meta messenger, and thus can meaningfully communicate about timing of user and system events even across different computers with potentially unsynchronised hardware clocks.

A centralised logging functionality uses the Topics below semaine.log.*. By convention, messages are sent to semaine.log.<component>.<severity>, e.g. the Topic semaine.log.UtteranceInterpreter.debug would be used for debug messages of component UtteranceInterpreter. The severities used are “debug”, “info”, “warn” and “error”. Through this design, it is possible for a log reader to subscribe, e.g., to all types of messages from one component, or to all messages from all components that have at least severity “info”, etc. Furthermore, a configurable message logger can optionally be used to log certain messages in order to follow and trace them. Notably, it is possible to read log messages in one central place, independently of the computer, operating system or programming language used by any given component.

The following figure illustrates this system architecture. Components communicate with each other via Topics in the hierarchy (indicated by black arrows). Meta information is passed between each component's meta messenger and the system manager via the semaine.meta Topic (grey arrows). Optionally, components can write log messages, and a message logger can log the content messages being sent; a configurable log reader can receive and display a configurable subset of the log messages (dashed grey arrows).

Optionally, a system monitor GUI visualises the information collected by the system manager as a message flow graph. Input components are placed at the bottom left, output components at the bottom right, and the other components are sorted to the extent possible based on the data input/output relationships, along a half-circle from left to right. Component B comes later in the graph than component A if A's output is an input to B or if there is a sequence of components that can process A's output into B's input. This criterion is overly simplistic for complex architectures, especially with circular message flows, but is sufficient for simple quasi-linear message flow graphs. If a new component is added, the organisation of the flow graph is recomputed. This way, it is possible to visualise message flows without having to pre-specify the layout.

The following pages present three emotion-oriented example systems, in order to corroborate the claim that the SEMAINE API is easy to use for building new emotion-oriented systems out of new and/or existing components. Source code is provided in order to allow the reader to follow in detail the steps needed for using the SEMAINE API. The code is written in Java, and can be obtained from the SEMAINE sourceforge page [57]⁠. The SEMAINE API parts of the code would look very similar in C++.

The source code for these examples is included in the SVN repository and in the source release package of SEMAINE-3.0.


This page is based on the following paper (an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited).

Schröder, M. (2010). The SEMAINE API: Towards a standards-based framework for building emotion-oriented systems. Advances in Human-Machine Interaction, Volume 2010, Article ID 319406. doi:10.1155/2010/319406.

It is one advantage of open access publishing that the information in the journal paper can be developed and updated on this wiki page.

Last modified 12 years ago Last modified on 10/18/10 14:10:54

Attachments (3)

Download all attachments as: .zip