2

I've been trying to make Agora work properly in Unreal engine 5 for two days no with no luck at all. it packages correctly but after opening camera it crashes. I have found the function and the line causing the crash and I've tried every solution that I can think of and it still crashes. here is the crash message.

LogOutputDevice: Error: Ensure condition failed: oldValue == newValue [File:D:\build\++UE5\Sync\Engine\Source\Runtime\Core\Private\HAL\ThreadingBase.cpp] [Line: 300] 
oldValue(1) newValue(0) If this check fails make sure that there is a FTaskTagScope(ETaskTag::EParallelRenderingThread) as deep as possible on the current callstack, you can see the current value in ActiveNamedThreads(1), GRenderingThread(d606da40), GIsRenderingThreadSuspended(0)
LogStats:             FDebug::EnsureFailed -  0.000 s

The Crash Log:

Unhandled Exception: EXCEPTION_ACCESS_VIOLATION reading address 0x000002428c15b020

I've used the template project with the Agora Blueprint plugin. the project worked perfectly on 4.26. I converted it to UE5 and only disabled Android and IOS from target platforms.

Here is my code:

VideoFrameObserver.cpp

bool VideoFrameObserver::onCaptureVideoFrame( VideoFrame& videoFrame )
{
   const std::lock_guard<std::mutex> lock( localFrameMutex );
    
   if( !localFrame.texture ||
       localFrame.texture->GetSizeX() != videoFrame.width ||
       localFrame.texture->GetSizeY() != videoFrame.height )
   {
      localFrame.region.reset( new FUpdateTextureRegion2D( 0, 0, 0, 0, videoFrame.width, videoFrame.height ) );
      localFrame.texture = UTexture2D::CreateTransient( videoFrame.width, videoFrame.height, PF_R8G8B8A8 );
        

        //Causes the crash on Unreal Engine 5
        localFrame.texture->UpdateResource();
   }

   if (!ensure(localFrame.texture != nullptr))
   {
       return false;
   }  
   
   localFrame.texture->UpdateTextureRegions( 0, 1, localFrame.region.get(), videoFrame.yStride, ( uint32 ) argbPixSize, static_cast< uint8_t* >( videoFrame.yBuffer ) );

   localFrame.fresh = true;

   return true;
}


void VideoFrameObserver::OnTick( float DeltaTime )
{
   localFrameMutex.lock();

   if( localFrame.fresh )
   {
      if( m_agora )
         m_agora->OnLocalFrameReceivedDelegate.Broadcast( localFrame.texture );
      localFrame.fresh = false;
   }

   localFrameMutex.unlock();

   remoteFrameMutex.lock();
   std::for_each( remoteUsersFrames.begin(), remoteUsersFrames.end(),
                  [this]( auto& frame )
   {
      if( frame.second.fresh )
      {
         if( m_agora )
            m_agora->OnRemoteFrameReceivedDelegate.Broadcast( frame.first, frame.second.texture );
         frame.second.fresh = false;
      }
   } );
   remoteFrameMutex.unlock();
}

The line that causes the crash:

localFrame.texture->UpdateResource()

Here is the call stack: The Call Stack

After enabling debugging for engine code it throws exception here.

ThreadingBase.cpp

The function in engine code that throw exception

another thing that if I open the editor in debug mode through visual studio when I hit play and open camera it throws exception but if I press continue it works greatly and the camera plays properly !! which is very weird. the problem can be solved with try catch but it's disabled and discouraged in Unreal Engine.

The template Project: https://github.com/AgoraIO-Community/Agora-Unreal-SDK-Blueprint

The Agora Plugin: https://github.com/AgoraIO-Community/Agora-Unreal-SDK-Blueprint/releases

yousef Badr
  • 21
  • 3

1 Answers1

2

I did run into almost precisely this exact bug with Agora and UE5 and while I can't be 100% certain that this will fix it for you I was no longer able to reproduce the crash.

The first issue is that (according to the Unreal UE5 Engine source code for Texture.cpp) any texture copy operations must NOT occur on the render thread. To meet the apparent requirement I wrapped the entire block of code in a GameThread lambda - see:

AsyncTask(ENamedThreads::GameThread, [=](){
    // your routine
})

There is a second concern however. Agora has no responsibility to hold onto the frame for any duration longer than this call. Your call to UpdateTextureRegions() may fail because videoFrame.yBuffer may become invalidated. To circumvent this I manually used memcpy() to copy the raw buffer to transient memory - and then copy that copy at my leisure (asynchronously inside the asynchronous lambda).

int yStride = videoFrame.yStride;
uint8* yBuffer2 = new uint8[ videoFrame.height * videoFrame.yStride ];
memcpy(yBuffer2,videoFrame.yBuffer,videoFrame.height * videoFrame.yStride);
AsyncTask(ENamedThreads::GameThread, [=](){
    // ... insert other code here
    // copy buffer at your leisure
    localFrame.texture->UpdateTextureRegions(
        0, 1,
        localFrame.region.get(),
        yStride,
        ( uint32 ) argbPixSize,
        static_cast< uint8_t* >( yBuffer2 )
     );
     // liberate the buffer
     delete[] yBuffer2;
 })

Remember to use delete[] to delete your buffer. Hope this helps.

Later Edit: I've revised my approach somewhat and I have a more complete example here: https://gist.github.com/anselm/2c786cc7f395c5c819197d511726c27a

Anselm Hook
  • 31
  • 3