User avatar
Leslie
Lead Software Architect
Posts: 352
Joined: Fri Sep 14, 2012 12:20 pm
Contact: Website

Creating 3D Avatar

Sun Sep 28, 2014 10:22 pm

Creating a 3D Avatar is slightly more complicated when compared to a 2D Avatar. The new Graphic Engine uses DirectX 11 and above and loads the Files stored in .XNB Format. You shouldn''t be worried about file types because you can anytime send us ( avatar[@]syn.co.in ) your 3D models ( preferably in .FBX ) as an attachment and we will send back .XNB format files by converting them for you. If you plan to do so just add all your files including the 3D Models, 2D Textures and Xml Configuration inside a single directory and archive it. Upload it to any uploading portal and send us the link with your Full Name as the Subject of your Email. Avatars sent to avatar[@]syn.co.in ( without square brackets ) will not be used by Syn Network.

Example: The free 3D Avatar Lilith by Stefan Habicht followed the above procedure. The developer wanted the 3D Avatar to be available freely so we added it to Syn Store. If at any point you feel lost we recommend you to use Syn Forum. Coming back to our Tutorial... To create a 3D Avatar you''ll need a 3D Modelling software like 3ds Max, Maya or any freely available 3D Modelling software that can save files in .FBX format.

Before you begin we recommend you to Download 3D Avatar Demo files to get a better understanding of what we might be learning in this tutorial. So lets get started. Whenever the Speech Synthesizer is speaking a text/word the visemes in the current word is used to determine which 3D Model will be loaded. If you are creating a realistic 3D Model with proper lip sync we recommend you support all the 21 Phonemes used by our Graphics Engine. Now, Lets say the Avatar is going to speak the word "BAN" the viseme IDs that may be used are ( 21 , 1 and 19 ). That means to properly speak the word "BAN" its important that you have 3 3D Models that map to the respective Viseme IDs. The 3 3D Models for the Viseme IDs are shown below

Dave ID 21
Image

Dave ID 1
Image

Dave ID 19
Image

The Graphic Engine will blend the 3D models between those IDs and create a proper lip sync animation. Once you have created your 3D Models supporting all the Phonemes. You will have to create a configuration file in Xml to map your 3D Models to the right Viseme IDs. Sample Configuration File

XML Code

<main> 
<expression id="0" model="Head_Base"/> 
<expression id="1" model="Head_AEI"/> 
<expression id="2" model="Head_AEI"/>
<expression id="3" model="Head_O"/> 
<expression id="4" model="Head_AEI"/>
<expression id="5" model="Head_AEI"/>
<expression id="6" model="Head_FV"/> 
<expression id="7" model="Head_U"/> 
<expression id="8" model="Head_O"/>
<expression id="9" model="Head_O"/>
<expression id="10" model="Head_O"/> 
<expression id="11" model="Head_AEI"/> 
<expression id="12" model="Head_AEI"/> 
<expression id="13" model="Head_AEI"/> 
<expression id="14" model="Head_TLN"/>
<expression id="15" model="Head_AEI"/> 
<expression id="16" model="Head_TLN"/>
<expression id="17" model="Head_AEI"/>
<expression id="18" model="Head_FV"/>
<expression id="19" model="Head_TLN"/>
<expression id="20" model="Head_AEI"/>
<expression id="21" model="Head_BMP"/> 
<expression id="default" model="Head_Smile_Slight"/>
<expression id="plain" model="Head_Base"/>
<expression id="blink" model="Head_Blink"/>
<expression id="disgust" model="Head_Disgust"/>
<expression id="frown" model="Head_Frown"/>
<expression id="jaw.opening" model="Head_Jaw_Opening"/>
<expression id="kiss" model="Head_Kiss"/> 
<expression id="sad" model="Head_Sad"/> 
<expression id="smile" model="Head_Smile_Slight"/> 
<expression id="squint" model="Head_Squint"/> 
</main>

In the above configuration file the first 21 expression tags map your 3D model ( without .xnb extension ) to their respective Viseme IDs. The id attribute denotes the Viseme or Emotional Expression ID and the model attribute is the name of the 3D Model file ( in the same directory ) without its extension. After that the rest of the expression tags map your 3D models ( without .xnb extension ) to their respective "emotional expression" IDs. Do not get confused here. If the ID attribute is a Number then its assumed that its a Viseme ID if the ID in the expression Tag is a text value then its assumed that the ID is referring to some emotional expression and is not used by the Speech Synthesizer. Syn Virtual Assistant and Syn Engine by default support 21 different Expressions as mentioned here. The best part of declaring an emotional expression is that it helps developers and the community to add expressions when some texts are spoken. For example "I am happy" may involve the expression "smile". If your Avatar doesn''t have expressions then the Engine will ignore emotional expressions within generated responses. As for now Bots like ( AIML , Pandorabots , Verbot and Swatch ) do not interact with Emotional Expressions on the other hand responses saved in the Settings files of Plugins do interact and invoke Emotional Expressions. If the emotional expression say "smile" exists in a response then after the sentence "I am happy" is spoken the Avatar may use the Expression Smile.

Dave ID Smile
Image

Configuration File as an Avatar[/b] Saving an Avatar is easy. Syn Virtual Assistant by default will look for 3D Avatars in "C:\\Program Files (x86)\\Syn Virtual Assistant\\Syn\\Avatars\\3D\\" directory.

So the name of an Avatar is taken from the name of the Folder. For example, you wish to name your Avatar Sam then you should save your Avatar files in the C:\\Program Files (x86)\\Syn Virtual Assistant\\Syn\\Avatars\\3D\\Sam\\ folder.

The folder should contain all the 3D Model files and textures including the Xml Configuration file. Note that the user may change the Avatar name using the Settings Panel but this feature does not affect the Folder Name. Once done you will need to restart your Virtual Assistant to check if the Avatar has been detected. All 3D Models of the selected Avatar will be loaded into memory for faster interaction thereby reducing the time required to blend between different 3D Models

Return to “Developer Tutorials”

Who is online

Users browsing this forum: No registered users and 1 guest