wiki:FeatureVectors

Feature vectors

Feature vectors can be represented in an ad hoc format. In text form, the feature vectors consist of straightforward key-value pairs – one feature per line, values preceding features.

0.000860535 rmsEnergy
12.6699 logEnergy
-2.59005e-05 rmsEnergy-De
-0.0809427 logEnergy-De
...

As feature vectors may be sent very frequently (e.g., every 10 ms in the SEMAINE system), compact representation is a relevant issue. For this reason, a binary representation of feature vectors is also available. In binary form, the feature names are omitted, and only feature values are being communicated. The first four bytes represent an integer containing the number of features in the vector; the remaining bytes contain the float values one after the other.

Details

In all Topic names, we ommit the prefix “semaine.data.”.

Type of informationTopicSending frequencyList of feature names + value range
frontal face detectionanalysis.features.video.facedetectioncameraFrameRate* xPositionTopLeft [0,xCameraResolution] (top left corner of the bounding box of the face detected)
* yPositionTopLeft [0, yCameraResolution] (top left corner of the bounding box of the face detected)
* width [0,xCameraResolution] (width of the bounding box of the face detected)
* height [0,yCameraResolution] (height of the bounding box of the face detected)
* (all 0 if no face detected)
head motionanalysis.features.video.2dheadmotioncameraFrameRate* motionDirection [-π, π] (angle of the motion)
* motionMagnitudeNormalised [0, large number] (pixels per frame)
* motionX [-large number, large number] (pixels per frame)
* motionY [-large number, large number] (pixels per frame)
facial pointsanalysis.features.video.facialpointscameraFrameRatexRightOuterEyeCorner
yRightOuterEyeCorner
xLeftOuterEyeCorner
yLeftOuterEyeCorner
xRightInnerEyeCorner
yRightInnerEyeCorner
xLeftInnerEyeCorner
yLeftInnerEyeCorner
xRightInnerBrowCorner
yRightInnerBrowCorner
xLeftInnerBrowCorner
yLeftInnerBrowCorner
xRightOuterBrowCorner
yRightOuterBrowCorner
xLeftOuterBrowCorner
yLeftOuterBrowCorner
xRightUpperEyelid
yRightUpperEyelid
xLeftUpperEyelid
yLeftUpperEyelid
xRightLowerEyelid
yRightLowerEyelid
xLeftLowerEyelid
yLeftLowerEyelid
xRightNostril
yRightNostril
xLeftNostril
yLeftNostril
xRightMouthCorner
yRightMouthCorner
xLeftMouthCorner
yLeftMouthCorner
xUpperLip
yUpperLip
xLowerLip
yLowerLip
xChin
yChin
xNose
yNose
xRightPupil
yRightPupil
xLeftPupil
yLeftPupil
speech featuresanalysis.features.voicemicrophoneFrameRate* F0frequency [0,600] (fundamental frequency in Hz)
* voiceProb [0,1] (probability that the current frame is harmonic)
* RMSenergy [0, 1] (energy of the signal frame)
* LOGenergy [-100,0] (energy of the signal frame, in dB)
* (more upon request...!)
Last modified 7 years ago Last modified on 12/14/10 19:11:09