I'm currently working in an environment where we used to have multiple messaging engines using a mixture of file stores and database backed message stores. That design was originally done by IBM consultants. However, the messaging engines configured with file stores caused lots of issues and we finally came to the conclusion that they lack the reliability required for mission critical production usage. We finally eliminated all file stores (except for a few non business critical use cases) and changed to database backed message stores.
The following statement from IBM Level 3 support (in response to a PMR about a file store unexpectedly reported as full) may shed some light on the problem you are encountering:
I suspect that the observed problem is related to filestore fragmentation for which there is no defragment process, only limitation. The pattern of messaging in the test described is exactly the pattern that can induce the filestore to become full and not recover despite
PM10591.
Essentially the problem is due to the requirement that the entire size of the log file needs to be able to fit inside the store file contiguously. Under normal circumstances this contiguous free space (which is never used but required to be there) is at the end of the store file, eg:
0...message data...400Mb....free contiguous space...500Mb
When the store file is full it means there is not enough contigious free space in the store file to fit the log file. Imagine you have filled the file from 0 to 400Mb with message data, and then consumed message data from 0 to 200 Mb (draining the MDB input queue, but the system exception destination remains full). Because the store file is essentially a filesystem itself, and the node and directory meta data stored within the file at variable locations, the 0 to 200Mb store area is not a contiguous free space just because you have consumed the message data. Rather at random intervals there is a small amount of meta data. So in total there is 300Mb free space, but not a single contiguous 100Mb piece where the log can go. This is the problem for which there is currently no defragmenting solution.
Usually consuming the message data from all the queues will free the vital few bytes to recreate the free contiguous free space at the end of the store file. i.e. consuming the message data on the exception destination (which is more useful that consuming the message data from the first queue to fill)
The full solution to this problem is to not encounter store full. The only way to know what is the maximum required is to fill all queues to their configured limit with the maximum possible size of the messages and see what size store file is required. Or you can configure the file store to have unlimited size.
In v7 there are changes to attempt to resist fragmentation, but again no defragmenter. This is because online defragmentation would be very expensive, and the true solution to this problem is to ensure that enough space is available to contain all the message data expected in the worst case scenario.
What this means in practice is that there are particular usage patterns where the permanent store requires significantly more space than what one would expect based on the number and sizes of the messages actually stored. Obviously this makes planning the storage requirements of a file store basically impossible.
