wiki:EmotionMirror

Emotion mirror

Back to Building emotion-oriented systems

The Emotion mirror is a variant of the Hello system. Instead of analysing text and deducing emotions from keywords, it uses the openSMILE speech feature extraction and emotion detection for interpreting the user's emotion. The output is rendered using the same EmoticonOutput component from the Hello system.

Only one new component is needed to build this system. EmotionExtractor has an emotion Sender (l. 2 and l. 7) just like the HelloAnalyser had, but uses an EMMA Receiver (l. 6) to read from the topic that the Emotion detection component from the SEMAINE system publishes to (see ComponentArchitecture). Upon reception of an EMMA message, the method react() is called (l. 10). As the only receiver registered by the component is an EMMA receiver, the message can be directly cast into an EMMA message (l. 11) which allows for comfortable access to the document structure to extract emotion markup (l. 12-13). Where emotion markup is present, it is inserted into a standalone EmotionML document (l. 16-18) and sent to the output Topic (l. 19).

 1 public class EmotionExtractor extends Component {
 2   private XMLSender emotionSender =
         new XMLSender("semaine.data.hello.emotion", "EmotionML", getName());
 3 
 4   public EmotionExtractor() throws JMSException {
 5     super("EmotionExtractor");
 6     receivers.add(
             new EmmaReceiver("semaine.data.state.user.emma.emotion.voice"));
 7     senders.add(emotionSender);
 8   }
 9 
10   @Override protected void react(SEMAINEMessage m) throws JMSException {
11     SEMAINEEmmaMessage emmaMessage = (SEMAINEEmmaMessage) m;
12     Element interpretation = emmaMessage.getTopLevelInterpretation();
13     List<Element> emotionElements =
           emmaMessage.getEmotionElements(interpretation);
14     if (emotionElements.size() > 0) {
15       Element emotion = emotionElements.get(0);
16       Document emotionML = XMLTool.newDocument(EmotionML.ROOT_ELEMENT,
                                                  EmotionML.namespaceURI);
17       emotionML.adoptNode(emotion);
18       emotionML.getDocumentElement().appendChild(emotion);
19       emotionSender.sendXML(emotionML, meta.getTime());
20     }
21   }
22 }

The config file contains only the components SystemManager, EmotionExtractor and EmoticonOutput. As the SMILE component is written in C++, it needs to be started as a separate process as documented in the SEMAINE wiki documentation SEMAINE-3.0. The resulting message flow graph is shown in the following figure.

Back to Building emotion-oriented systems

Last modified 6 years ago Last modified on 03/24/11 09:40:43

Attachments (2)

Download all attachments as: .zip