Changes between Initial Version and Version 1 of PlayerProtocol


Ignore:
Timestamp:
10/18/10 15:37:00 (9 years ago)
Author:
masc01
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • PlayerProtocol

    v1 v1  
     1[[PageOutline]] 
     2 
     3== Protocol for the Player in SEMAINE == 
     4Any player component in SEMAINE must follow the following protocol so that it supports ahead-of-time preparation of possible utterances. The player must keep a collection of "Animations" which can be played by a "playCommand". 
     5 
     6This protocol is currently implemented by two players: The audio-visual Windows native Player­Ogre using the Greta agent, and the speech-only player in Java class [source:trunk/java/src/eu/semaine/components/mary/QueuingAudioPlayer.java QueuingAudioPlayer]. 
     7 
     8=== Data flow === 
     9Low-level player data is sent to the player via the Topics 
     10 
     11{{{ 
     12    semaine.data.synthesis.lowlevel.* 
     13}}} 
     14currently 
     15 
     16{{{ 
     17    semaine.data.synthesis.lowlevel.audio 
     18    semaine.data.synthesis.lowlevel.video.FAP 
     19    semaine.data.synthesis.lowlevel.video.BAP 
     20}}} 
     21Incoming messages have the following properties: 
     22 
     23 * a message type specific to the payload format         (currently: !BytesMessage for audio, !TextMessage for FAP and BAP). 
     24 
     25 * a data type         (obtained by message.getDatatype()) identifying the type of message         (current values are "AUDIO", "FAP" and "BAP"). 
     26 
     27 * a content ID and a content creation time         (obtained by message.getContentID() and         message.getContentCreationTime()) which are used to assemble an         Animation, to match data and command messages, and to identify the         content item in callback and log messages. 
     28 
     29The idea is that a unit of player data (an "Animation") is assembled in the player from the individual data items that are coming in (currently, AUDIO, FAP and BAP). Certain data types are optional (currently: AUDIO). A message can either contain the complete data of the given type (currently the case for AUDIO) or it can contain a chunk of data (currently the case for FAP and BAP). A chunk contains information about its position in the Animation; it can be dynamically added even if the Animation is already playing. 
     30 
     31=== Command messages === 
     32There are two types of command messages: messages with data typesdataInfoandplayCommand. 
     33 
     34 * Data info commands: For a given content ID,         define the data types that must be present in the Animation:         HASAUDIO, HASFAP and HASBAP. Each can be 0 for "not needed"         or 1 for "needed". 
     35 
     36 * Player commands:         For a given content ID, define the playback conditions. This         includes the following aspects: 
     37 
     38 * STARTAT: when to                 start the playback of the Animation (in milliseconds from the                 moment when the Animation becomes ready); 
     39 
     40 * PRIORITY: the                 priority of the Animation in case of competing Animations; 
     41 
     42 * LIFETIME: the lifetime of the Animation,                 counting from the moment when the animation becomes ready. When the                 lifetime is exceeded and the animation has not started playing, it                 will be marked as "dead" and removed. 
     43 
     44Commands are sent to topic 
     45 
     46{{{ 
     47    semaine.data.synthesis.lowlevel.command 
     48}}} 
     49and have the data type `dataInfo`for data info commands and `playCommand` for player start trigger commands. 
     50 
     51For every content ID, a playCommand is required in order to play that animation. Without a matching playCommand, an animation will never be played. 
     52 
     53A command has the following format: 
     54 
     55 * its content ID is         '''identical''' to the content ID of the Animation for         which it defines playback conditions; 
     56 
     57 * message format is '''!TextMessage''';         the text consists of space-separated key-value pairs, one pair per         line, where the keys are strings and the values are floating point         numbers. 
     58 
     59The following features are used: 
     60 
     61 * for playCommand: 
     62 
     63 * STARTAT (0 means                 start at the moment when all required parts are present, a positive                 number means milliseconds after that condition is met) 
     64 
     65 * LIFETIME (in                 milliseconds from the moment the animation is triggered; -1 means                 it will never expire) 
     66 
     67 * PRIORITY (a value between 0 and 1, where 0                 is the lowest and 1 the highest possible priority) 
     68 
     69 * for dataInfo: 
     70 
     71 * HASAUDIO (a                 binary feature, 0 means the Animation does not have audio, 1 means                 the Animation has audio data) 
     72 
     73 * HASFAP (a binary                 feature, 0 means the Animation does not have FAP data, 1 means the                 Animation has FAP data) 
     74 
     75 * HASBAP (a binary feature, 0 means the                 Animation does not have BAP data, 1 means the Animation has BAP                 data) 
     76 
     77Every player command must contain all features of its respective type. 
     78 
     79=== Callback messages === 
     80Event-based callback messages are sent when certain conditions are met for a given Animation. The messages go to Topic 
     81 
     82{{{ 
     83    semaine.callback.output.player 
     84}}} 
     85and have the following format: 
     86 
     87{{{ 
     88<callback xmlns="http://www.semaine-project.eu/semaineml"> 
     89   <event id="CONTENT_ID" time="META_TIME" type="EVENT_TYPE"/> 
     90</callback> 
     91}}} 
     92where content ID and meta time are like before, and type is one of the following: 
     93 
     94 * `ready` means the Animation has received all required data, so it is ready         for playing back. This event is triggered independently of the         question whether a command has been received or not. 
     95 
     96 * `deleted` means the Animation was removed before it started playing, e.g.         because it has exceeded its lifetime in the output queue. 
     97 
     98 * `start` means the Animation has started playing. 
     99 
     100 * `stopped` means the Animation was stopped while playing but before it was         finished, e.g. because a request to change character was received. 
     101 
     102 * `end` means the Animation has         finished playing. 
     103 
     104=== Error conditions === 
     105The content ID must be unique for the lifetime of a system. This leads to the following error conditions. 
     106 
     107It is an error condition... 
     108 
     109 * if a data chunk is         received for an Animation that has already been discarded (because         it finished playing, or exceeded its lifetime in the queue); 
     110 
     111 * if data is         received for a data type that does not form chunks; 
     112 
     113 * if a playCommand is received for a content ID         that has been started already, or that is already discarded; 
     114 
     115An error condition should be reported as a WARN log message, and otherwise ignored. 
     116 
     117It is '''not''' an error condition... 
     118 
     119 * if a second playCommand is received after an         animation has become ready but before it started playing. In this         case, the new priority etc. overwrites the previous values.