-1

What I want is that on a-frame, when I talk, my 3D model avatar is also being made talking.

Following this guide, https://aframe.io/docs/1.1.0/introduction/models.html#animating-models, I created 3D model avatar with this resource, https://sketchfab.com/3d-models/bake-talking3-e715ab67be934a108d0a952d90c07210

But this gltf 3D model is talking all the time. I need interactive 3D model talking whenever I talk.

Let's assume my voice detection is already implemented.

Can anyone answer this, please?

webtech
  • 3
  • 1
  • So you need to Play and pause the animation, or Play one cycle when the voice is detected – Piotr Adam Milewski Dec 23 '20 at 12:11
  • Hi, Piotr Adam Milewski. During the voice detection, the animation should be played and when the voice is stopped, the animation should be stopped and in the next voice detection, the animation should be played again. Can you give me any kind of example or can we animate specific part like mouse movement effect so people look like the 3D human modal talking. – webtech Dec 24 '20 at 01:36

1 Answers1

0

The animation-mixer component has two methods that should help

  • playAction() which will play the
  • stopAction() which will stop the

Let's assume my voice detection is already implemented.

Then Your code could look like this:

const modelEntity = document.querySelector("[gltf-model]")
const animationComponent = modelEntity.components["animation-mixer"]
mySpeechRecognition.onspeechstart = function() {
   animationComponent.playAction();
}

mySpeechRecognition.onspeechend= function() {
   animationComponent.stopAction();
}

Something like what I did in this glitch. Green plays, red stops. Click on the fish to check out the source.

Piotr Adam Milewski
  • 14,150
  • 3
  • 21
  • 42
  • Thanks for your reply., Piotr. This is very helpful. By the way, this gltf 3D model is used as an avatar as below screenshot. In this case, can we control the animation play/stop action too? prnt.sc/w9tie8 – webtech Dec 25 '20 at 16:16
  • It should work in any case provided you call the methods (play/stop) on the correct entity. Did you try it out and had any problems? – Piotr Adam Milewski Dec 25 '20 at 19:40
  • Yes, I confirmed your solution worked fine. If user talks, let's assume that I get the speaker's user id from back end, then can I control the specific avatar's gltf animation? – webtech Dec 25 '20 at 20:14
  • I think there are multiple ways to do this. Upon connection you create the avatar - you can keep a map `users[id] = element` to keep track who is speaking, or you can store it in the avatar entity `el.userdata.uuid = uuid` and then search for the entity with the specific `uuid`. If the solution is working feel free to accept the anwser :) – Piotr Adam Milewski Dec 25 '20 at 20:22
  • Yes, I will. One question again. Can I set the uuid to the player avatar? uuid is from back end api. Please check this screenshot. https://prnt.sc/w9xulu I am using networked-aframe for building player avatar. – webtech Dec 25 '20 at 20:36
  • How do you want to pass the `startvoice` / `stopvoice` to others? The `networked` component has a `networkId` property - isn't this the network UUID? I have more experience with raw websockets and socket.io than `aframe-networked` I'm afraid. – Piotr Adam Milewski Dec 25 '20 at 21:39
  • Let's assume that from back end api, the id of the user who is currently speaking is being received. And when page is loading, from api, the current user id will be also received. With those, how to get the right avatar networked component to play/stop the gltf 3D animation. – webtech Dec 26 '20 at 19:56
  • From what i gather from [here](https://github.com/networked-aframe/networked-aframe#events) You can get the network id, and entity reference from any connected client. So You can keep track which network ID belongs to which avatar. – Piotr Adam Milewski Dec 26 '20 at 20:23
  • Yes, I can get the clientId. And I can get networkId too, but at the moment, it is empty. how to know which network ID belongs to which avatar? In avatar, can I set custom network ID with the user id from api? – webtech Dec 26 '20 at 21:29
  • @webtech check out [this thread](https://stackoverflow.com/questions/65465301/a-frame-networked-how-to-match-an-avatar-with-a-broadcasting-client/65465302#65465302) and let me know there if You still have trouble connecting the IDs to the avatars – Piotr Adam Milewski Dec 27 '20 at 11:36
  • It was very helpful, by the way, is there any simple way to implement this within NAF.schemas.add? – webtech Dec 28 '20 at 05:29