The root cause in your case is a bug in Jaybird 3.0.4 to 3.0.7 (and 4.0.0-beta-1) with Firebird 3 (and 4) when wire encryption is enabled in firebird.conf (setting WireCrypt = Enabled).
I had previously only seen this problem with Jaybird 4 when the tests of Jaybird were run in a specific order, and as that version is not yet released, I hadn't given it priority to find the root cause.
With your help (thanks again!) I was able to identify the problem. This bug is now fixed in Jaybird 3.0.8, which is available from Firebird: JDBC Driver.
Explanation of the bug
Certain types of buffers (eg column data) in the Firebird wire protocol are padded with bytes to multiples of four. The implementation in Jaybird relied on InputStream.skip(long) to skip this padding.
Specifically, it did:
public int skipFully(int n) throws IOException {
int total = 0;
int cur;
while (total < n && (cur = (int) in.skip(n - total)) > 0) {
total += cur;
}
return total;
}
This worked fine without wire encryption, because the combination of the skip implementations in BufferedInputStream and SocketInputStream would at least skip 1 byte for each call to skip unless the socket was closed.
When CipherInputStream was added, this expectation no longer held: if the BufferedInputStream has no bytes in its buffer, it will call skip(long) on the CipherInputStream, and if that is at the end of its buffer, it will skip 0 bytes (which is allowed for skip).
As a result, Jaybird could skip 1, 2 or 3 bytes too few, which caused subsequent reads to read wrong data. Eventually this would either cause a buffer size to be read incorrectly, causing the read to block waiting for more data, or it would result in Jaybird reading a wrong operation code, which will cause an exception with a message starting with "Unsupported or unexpected operation code".
Workarounds or solutions for other problems with similar behaviour
As a workaround to hanging connections, you can use the connection property soTimeout to set the socket read timeout (see Jaybird Manual, appendix A.2 Other properties).
The problem could also be that Firebird is 'just' waiting indefinitely on a lock. To rule out Firebird waiting on a lock, you could also change the transaction wait mode or timeout. The default for Jaybird is to use an indefinite wait.
For example, to change the wait mode of read committed to no-wait, you can set connection property TRANSACTION_READ_COMMITTED with value read_committed,rec_version,write,nowait.
To use wait, but with a timeout, you can use value read_committed,rec_version,write,wait,lock_timeout=5. This will apply a 5 second lock timeout.
If your problem is due to Firebird waiting on a lock held by another transaction, then this should result in an exception with the text containing lock conflict on no wait transaction (for the nowait option) or lock time-out on wait transaction (for the lock_timeout option).
See also Jaybird Manual, appendix A.3 Transaction isolation levels and Jaybird Manual, section 6.4. Transaction Isolation Levels (although I now see that I will need to add more information in there).
