Open GL performance on the Raspberry PI 4B and another SBC

Question

Good day!

I got a project from a firm and now I need to develop an arcade cabinet with Raspberry Pi 3. It has a joystick and some external devices, which are controlled via GPIO. I have already organized the communication between the RPI 4B 8Gb and the joystick and I can already receive signals from the coin acceptor. The next trouble is performance. I create the videogame using Java and append the internal processing libraries (www.processing.org) to the project for the graphic output and GPIO interaction.

The processing-core library has two abilities to render graphics: software rendering (using parameter JAVA2D in the size() function) and hardware accelerated (using parameter P2D or P3D in the same function). The parameters P2D and P3D enable the OpenGL support and the performance of all my games on the normal desktop PC or android device is higher as the performance in the JAVA2D mode. But not on my single board computers.

On the Raspberry PI 4B 8Gb (Raspberry Pi OS 64 bit desktop) I got: 11 FPS in JAVA2D mode and 6 FPS in P2D mode. On my Khadas Vim3 pro (Ubuntu 20.04 64 bit) I got 14 FPS in JAVA2D mode and 8 FPS in P2D mode.

I don’t understand what is the trouble with the performance on the SBC? My game is pretty simple: The screen size is: 1920x1080. All the game graphic is packed in a single PNG file. I draw a images for background, than for 3 airplaines, than for bullets and than a full screen explosion. All the graphic is drawn without scaling using function:

image(source, posX, poxY, width, height, leftPixelInSource, upperPixelInSource, rightPixelInSource, lowerPixelInSource);

I need to get at least 20 FPS to complete the project.

I use Intellij IDEA (with sudo parameter) and I see in the logcat the next 3 lines when I use OpenGL (P2D parameter in the size()-function):

error: XDG_RUNTIME_DIR not set in the environment.
error: XDG_RUNTIME_DIR not set in the environment.
error: XDG_RUNTIME_DIR not set in the environment.

, but the game runs. When I close the game I receive the next messages in the logcat.

X11Util.Display: Shutdown (JVM shutdown: true, open (no close attempt): 2/2, reusable (open, marked uncloseable): 0, pending (open in creation order): 2)
X11Util: Open X11 Display Connections: 2
X11Util: Open[0]: NamedX11Display[:0, 0x7f1c07cc90, refCount 1, unCloseable false]
X11Util: Open[1]: NamedX11Display[:0, 0x7f1c0a2150, refCount 1, unCloseable false]

In Java mode I haven’t any errors. I tried to launch without the sudo parameter and with the enabled P2D parameter. I have a new error and can not launch the game. The log cat shows:

Resource compiler: Error: Couldn't copy [/home/mgdsstudio/GD/Workspace/Airfight/Libs/META-INF/MANIFEST.MF] to [/home/mgdsstudio/GD/Workspace/Airfight/out/production/desktop/META-INF/MANIFEST.MF]
java.io.IOException: Couldn't copy [/home/mgdsstudio/GD/Workspace/Airfight/Libs/META-INF/MANIFEST.MF] to [/home/mgdsstudio/GD/Workspace/Airfight/out/production/desktop/META-INF/MANIFEST.MF]
    at com.intellij.openapi.util.io.FileUtil.performCopy(FileUtil.java:419)
    at com.intellij.openapi.util.io.FileUtil.copyContent(FileUtil.java:408)
    at org.jetbrains.jps.incremental.FSOperations.copy(FSOperations.java:472)
    at org.jetbrains.jps.incremental.resources.ResourcesBuilder.copyResource(ResourcesBuilder.java:123)
    at org.jetbrains.jps.incremental.resources.ResourcesBuilder.lambda$build$0(ResourcesBuilder.java:67)
    at org.jetbrains.jps.incremental.fs.BuildFSState.processFilesToRecompile(BuildFSState.java:374)
    at org.jetbrains.jps.incremental.IncProjectBuilder$6.processDirtyFiles(IncProjectBuilder.java:1261)
    at org.jetbrains.jps.incremental.resources.ResourcesBuilder.build(ResourcesBuilder.java:56)
    at org.jetbrains.jps.incremental.resources.ResourcesBuilder.build(ResourcesBuilder.java:27)
    at org.jetbrains.jps.incremental.IncProjectBuilder.buildTarget(IncProjectBuilder.java:1266)
    at org.jetbrains.jps.incremental.IncProjectBuilder.runBuildersForChunk(IncProjectBuilder.java:1166)
    at org.jetbrains.jps.incremental.IncProjectBuilder.buildTargetsChunk(IncProjectBuilder.java:1303)
    at org.jetbrains.jps.incremental.IncProjectBuilder.buildChunkIfAffected(IncProjectBuilder.java:1118)
    at org.jetbrains.jps.incremental.IncProjectBuilder.buildChunks(IncProjectBuilder.java:889)
    at org.jetbrains.jps.incremental.IncProjectBuilder.runBuild(IncProjectBuilder.java:471)
    at org.jetbrains.jps.incremental.IncProjectBuilder.build(IncProjectBuilder.java:197)
    at org.jetbrains.jps.cmdline.BuildRunner.runBuild(BuildRunner.java:150)
    at org.jetbrains.jps.cmdline.BuildSession.runBuild(BuildSession.java:371)
    at org.jetbrains.jps.cmdline.BuildSession.run(BuildSession.java:193)
    at org.jetbrains.jps.cmdline.BuildMain$MyMessageHandler.lambda$channelRead0$0(BuildMain.java:211)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.nio.file.AccessDeniedException: /home/mgdsstudio/GD/Workspace/Airfight/out/production/desktop/META-INF/MANIFEST.MF
    at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
    at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
    at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
    at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219)
    at java.base/java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:478)
    at java.base/java.nio.file.Files.newOutputStream(Files.java:220)
    at com.intellij.openapi.util.io.FileUtil.openOutputStream(FileUtil.java:452)
    at com.intellij.openapi.util.io.FileUtil.performCopy(FileUtil.java:415)
    ... 22 more

I read some info about default OpenGL driver troubles on RPI. But I don’t understand what I must do to increase the FPS or maybe repair the OpenGL? Are there differences in the Raspberry Pi OS 32 bit between desktop and no desktop distribution? Maybe I need to try another OS distribution to get better performance?

I have made my sketch simpler. It has now only one function in the game loop:

@
Override
    public void settings(){
        size(1920, 1080, P2D);        
    }

    @Override
    public void setup(){}



    @Override
    public void draw(){
        System.out.println("FPS: " + frameRate);        
    }

And I got the FPS not more than 13 frames per second. The game loop does nothing - only console output. When I launch glxgears programm after I have installed mesa-utils with the command:

sudo apt-get install mesa-utils

I get 75 FPS. I don’t understand why the simplest processing sketch with only console output and the not updateable screen content runs so slow

Isn't a problem that a raspberry pi doesn't come with a GPU to run the game's graphics? I remember a few years ago when I tried to use pygame (v1.0, which used the SDL 1.0), the graphics calculations were done directly on the CPU. This is an explicit performance problem, as the CPU needs to compute all the graphics at each frame. The problem grows, the larger your screen size is (1920x1080 is big). So, depending on how fast the CPU is, you're bound to have low FPS no matter what you do (as this is more of a hardware problem). Or, you could try decreasing the screen size. — Carl HR, Mar 05 '23 at 14:15
Carl HR, I have made a new test and added this info to the post. The clear test without any graphic output - not updateable window 1920x1080 with one console output per frame. And I get 13 FPS. Standart test glxgears shows 75 fps. Maybe trouble is not in the RPI? I can not change the resolution: this is a point in the specification of the customer. — Alexander Gorodilov, Mar 05 '23 at 15:23
_"I don’t understand why the simplest processing sketch with only console output and the not updateable screen content runs so slow"_ When you try to print anything into a console, it's expected that the print function call slows down the application. That's why that you can never know the true FPS you're getting within your application, as any technique that you may try will slow it down. Now, to get such a low frame rate, how are you testing the application? The raspberry is taking care of an entire OS and multiple applications at the same time? Or just the game (no OS)? — Carl HR, Mar 05 '23 at 15:50
Given the possiblity that the raspberry pi is being slowed down, due to have to process the entire screen, and other applications at the same time (ie: game + console + os interface), try to redirect the standard output to a file inside your project. This way, the console won't output anything when a print occurs, which won't force any screen updates. Later on, when the application is closed, you can open the file and check the output (frame rate). It should be bigger. If it's not, then I don't know what it is causing it to be low. — Carl HR, Mar 05 '23 at 16:54
@AlexanderGorodilov Unfortunately I don't have access to a RPI 4 to test, but I think you're on the right track with the P2D blank sketch you did. In my experience Processing/Java runs fairly slow on RPI and in the past (using RPI 1, 2) resources were very limited. IIRC, by default, only a fraction of the system RAM is shared for video. Try running `sudo raspi-config` and enable a lot more video memory, reboot and run your test again. Try running a few tests from `Processing > Examples > Demos > Graphics` and also `Performance`. — George Profenza, Mar 05 '23 at 16:55
Additionally, try your blank test with smaller resolutions (e.g. 720p, 360p) and hopefully there's a lower resolution that behaves well. If that's the case you can try using a `PGraphics` buffer at half res (720p) for example to do most of the work, then render it at the end of `draw()` as `1080p`. Sure, it won't be as crisp, but hopefully still looking ok/acceptable but moving faster. If you're not constrained to Processing Java you can look at pygame (if that applies) or even openFrameworks (c++). OpenFrameworks is inspired Processing and has many similarities. setup/compilation will be slow — George Profenza, Mar 05 '23 at 16:57
Carl HR, the output source is not the main reason. I made an another experiment: I created on my Khadas Vim3 pro a clear LibGDX project, that draws a single picture on the screen in the main game loop and print the frame rate in the console. And I got stable 60 FPS. I think, that the trouble is in the Processing for the single boards computers. The internal libraries are not adjusted for using OpenGl on the ARM architecture. — Alexander Gorodilov, Mar 05 '23 at 20:44
George Profenza, I can not even launch any P2D examples from the Processing IDE. It doesn't want to create the OpenGl window. I have install Raspbian 32 bit but the situation is the same. Maybe the customer accepts the low resolution, but I don't know. I know only Java, but wanted to learn C++. But right now I can not even understand how to assembly libraries and append they to a clear C++ project. I wanted try to use processing for C++ (an open source unofficial port to Cpp), but couldn't assembly. — Alexander Gorodilov, Mar 05 '23 at 20:55
@George Profenza, If I can not find a solution how to use all OpenGl power from Processing, I think I will use LibGDX. I have tested it today and didn't get any performance troubles. It uses the same programming language and I have three books about this framework. The game is not very difficult and I think I can port my game in LibGDX in 4-6 hours. I also found a Java library (I think It is Pi4J) , that can be used for controlling of the GPIO. I don't want but I think it is the shortest way to complete the project. — Alexander Gorodilov, Mar 05 '23 at 21:10
@AlexanderGorodilov Pi4J can indeed help you to control the game with a joystick (see article on https://blogs.oracle.com/javamagazine/post/look-out-duke-part-2-control-a-java-game-with-a-raspberry-pi-and-a-joystick), but also contains documentation on how to run your application in desktop only mode. This is for JavaFX application, but may also be helpful for our use case: https://pi4j.com/getting-started/fxgl/kiosk-mode/ — Frank, Mar 06 '23 at 07:00
@Frank, I have wrote at the begin of the topic that "I have already organized the communication between the RPI 4B 8Gb and the joystick". I have used a Java-port of evdev library. But you have presented me a good idea. Processing has a JavaFX library. I will try to create not the OpenGL or Java window, but JavaFX window using FX2D as the third parameter in the size() function. I'll try it on my single board computers. I hope it is more powerfull mode for RPI — Alexander Gorodilov, Mar 06 '23 at 10:48
@Frank, JavaFX (FX2D as the third parameter in the size() function), increases the game performance. It is better than the default renderer. Thank Frank! But not enough. It will be better if the P2D works correct. — Alexander Gorodilov, Mar 08 '23 at 11:06
If your post needs _this_ many comments in order for someone to sus out the particulars, it's well past time to [edit] your post and move all the information you added in comments [into your post instead](/help/how-to-ask). Also, Processing has a [user forum](https://discourse.processing.org/) exactly for questions like this, so while posting here is of course fine, did you _also_ post there? Because SO is your last resort, not your first: first ask the people who are best suited to answer your question. And only if that yields no result does posting to SO make sense =) — Mike 'Pomax' Kamermans, Mar 10 '23 at 18:00

score 0 · Answer 1 · answered Mar 19 '23 at 07:43

Well, I have transfered my game into LibGDX. It uses also Java as the programming language and Open GL ES for the rendering. I have got 50-60 FPS. The answer is pretty simple: Don't use Processing for creation of applications with the Open GL context running on ARM Single board computers with Linux as the operating system.

Open GL performance on the Raspberry PI 4B and another SBC

1 Answers1