UPDATE + SOLUTION ===============================
Sorry to be posting the solution here instead of in a comment, but something about my work's filtering doesn't allow the comment functionality to work for me.
I ended up using the -b 0.0.0.0 property in jboss to bind to all addresses, so I could try accessing machine A's server with machine B as the client, and vice-versa. I found that it always failed to load when running on machine B, whether or not I was connecting from A or B.
I started wireshark on a windows machine on the same network, and observed the TCP connection that was loading the webpage. I saw that the request for the .swf in the cases where it failed had a content length of 2 million or so, and when I right clicked the wireshark logs and selected "view conversation" or something like that, the size of the total conversation to get the .swf file was only 130,000. Looking at about:cache, that was about equal to what it ended up caching before saying "Done" on the page.
I ended up finding that there is a bug with the useSendFile property. (http://community.jboss.org/thread/148651?tstart=0). This causes it to only send part of the file if you are running low on kernel memory. Using useSendFile="false" in our server.xml has seemed to resolve the problem.
==================================================
Original Problem
I have a JBoss (5.1.0.GA) application server. I am using GraniteDS to connect between the application server and the client. The client side is flash-based.
Granite DS requires the use of the APR library (apache native library), so I am loading it. I see in the JBoss logs that it says it loaded the apache native library just fine (version 1.18, though I've also tried 1.20).
The issue is that when I have it so the APR library loads successfully, then the Flash side of the application does not usually load. I'll have to hit refresh a bunch of times and eventually it will usually load, but normally I'll see either a black webpage that says "Done" or the loading progress bar never move. Only by repeatedly hitting refresh will the page load. It will load eventually by hitting refresh enough, but it is not consistent and this obviously will not work for our clients who have to clear their browser's cache every time.
This problem only exists on Solaris, our application works fine on Windows. We've tried multiple patch-levels of solaris, and have verified with the "ldd" command that the library that needs to be loaded has all its dependencies there.
We've verified it isn't our swf file's size by testing: 1) Our regular SWF (1660 kb). 2) A random large-ish SWF (950 kb). 3) A small SWF with one label component that says "Test" (277 kb).
All 3 were unable to load when JBoss was also loading the native library, and loaded just fine without it. We need the native library to load successfully for Granite to connect between the client and server though, so not loading it isn't an option (unless there's some way to use the NIO connector with JBoss, but it appears unsupported).... if there is a way to use the NIO connector then we shouldn't need the APR library.
Has anybody run into this before? Anybody have any ideas or recommendations?