How to simulate a person in pepper emulator?

Question

I'm trying to make pepper robot to listen to basic voice commands. The trouble is that the language is not supported and I have to use google speech to text for this instead of robot's default recognition libraries. It would be more convenient to test this with emulator rather than installing each time on the real robot. But I can't find any information about how to simulate a person appearing in front of robot in the emulator. If that's impossible, maybe there are some workarounds?

override fun onRobotFocusGained(qiContext: QiContext) {
    this.qiContext = qiContext
    Utils.defaultHolderBuilder(qiContext).build().async()?.release()

    qiContext.humanAwareness?.addOnHumansAroundChangedListener { humansAround ->
        if (humansAround.isNotEmpty()) {
            listenToHuman(qiContext, humansAround)
        }
    }
}

Perhaps I can do with just calling that function, supplied to addOnUpdatedListener, but how should I call it? Maybe simulate some test broadcast from within a program? The listenToHuman function:

private fun listenToHuman(qiContext: QiContext, humansAround: MutableList<Human>) {
    val actuation: Actuation = qiContext.actuation
    val robotFrame: Frame = actuation.robotFrame()

    val closestHuman = ...get closest human

    closestHuman?.headFrame?.addOnUpdatedListener {
        val distance: Double = ...computeDistance

        if (availableForListening) {
            availableForListening = false

            Qi.onUiThread {
                mLastResultTextView.text = "Listening"
                mSpeechRecognizer.startListening(speechIntent)
            }
        }
    }
}

score 1 · Accepted Answer · answered Jul 21 '21 at 10:03

There is no tool around the Pepper emulator to simulate humans. However, your test can be automated and the classes from the Qi SDK can be mocked.

Solution 1

In this public sample, a mock LookAt action is used to test whether the code behaves properly upon success or failure. It uses Mockk to do so.

With this in mind, you can as well mock the HumanAwareness class, provide a mock Human when getHumansAround is called, and trigger the call of the listener of the humans around property. This is quite some work, especially when you know that this is Java a wrapping for a simpler object system, libQi's.

Solution 2

So I wrote a helper class to create Qi Objects directly in Kotlin (sorry Java), and I use it to create mocks more easily:

abstract class FakeHumanAwareness: QiObjectImpl() {

    val humansAround = Property<List<Human>>(listOf())
    val recommendedHumanToApproach = Property(Human::class.java)
    val recommendedHumanToEngage = Property(Human::class.java)

    abstract fun makeEngageHuman(robotContext: RobotContext, human: Human): EngageHuman

    override fun advertise(objectBuilder: DynamicObjectBuilder): DynamicObjectBuilder {
        objectBuilder.advertiseProperty(this, FakeHumanAwareness::humansAround)
        objectBuilder.advertiseProperty(this, FakeHumanAwareness::recommendedHumanToApproach)
        objectBuilder.advertiseProperty(this, FakeHumanAwareness::recommendedHumanToEngage)
        objectBuilder.advertiseMethod(this, FakeHumanAwareness::makeEngageHuman)
        return objectBuilder
    }
}

Then I can mock that, and convert it into a HumanAwareness object:

val fakeHumanAwareness: FakeHumanAwareness = spyk()
val humanAwareness: HumanAwareness = qiObjectCast(fakeHumanAwareness)

You can do the same with Human (FakeHuman class not included here), mock it and set HumanAwareness.humansAround directly:

val fakeHuman: FakeHuman = spyk()
val human: Human = qiObjectCast(fakeHuman)
fakeHumanAwareness.humansAround.setValue(listOf(human))

When the value is set, the listeners are automatically called, and getHumansAround will return that value.

How to simulate a person in pepper emulator?

1 Answers1

Solution 1

Solution 2