0

I have quite a complex setup:

  • an Informix (11.5) legacy database which contains some of the business data,
  • a Postgres (9.3) database which connects via foreign data wrapper (FDW) to the Informix instance, and
  • a Grails application (2.4.4) which should run against Postgres.

We use the searchable-plugin for full text indexing.

The setup of this worked quite well. But when I start the indexing process, the Informix instance dies after some minutes. We are indexing 100.000+ rows, but the crash happens quite randomly: sometimes after 2 minutes, sometimes after 10.

There is no obvious error message in the online.log (log file from Informix). While indexing got this message (dozens of times):

14:53:15  Dynamically allocated new virtual shared memory segment (size 8192KB)
14:53:15  Memory sizes:resident:111492 KB, virtual:1089424 KB, no SHMTOTAL limit

Then the application crashes and Postgres informs the Grails application, that the network connection is broken or (we had different error messages) or the connection is refused.

In the IntelliJ console I can see, that the indexing process fires a lot of SQL-statements to the FDW-tables.

I guess this is quite a rare setup, but I want to ask if anybody has experience with FDW and Grails/Searchable and can point me to what might be the problem.

The FDW-tables are working fine in our application and in SQL Maestro.

Or could be just a basic configuration problem with Informix?

Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278
Klaus
  • 1
  • The continually allocating dynamic memory messages are worrying. Do the values keep increasing? When you're indexing, where is the indexing occurring — in the Grails, or are you using Informix's system (BTS, built on CLucene)? Does the whole Informix server crash, or just the particular parts that the Grails-via-Postgres code is using? Are you sure there's nothing about an AF or similar in the log file? Any indication that Informix knows it is going down? Are you on Linux? If so, could the OOM-Killer be striking your Informix because of the memory allocation issue? – Jonathan Leffler Aug 21 '15 at 20:17
  • Yes, the values are increasing. The indexing is occuring in Grails and saved in the file system of another server. The informix server crashes completly and loses the connection to the raw devices where the tablespaces are located. No, informix is not aware and the server processes are still running, but you can´t login or get data ... Yes, Linux Debian. And it´s small VM with 2GB RAM. But I checked memory in top and I saw the decreasing memory. Unfortunaley the server is maintained by a contractor and I can´t change the configuration easily. – Klaus Aug 22 '15 at 12:30
  • Solution: I changed the indexAll command to a loop and added a small delay (sleep (2)) to the loop and everything works fine. There are no log entries regarding the shared-memory anymore. I still wonder how this could happen, but right now I have a solution. – Klaus Aug 24 '15 at 14:13
  • Congratulations on getting a solution/workaround. I'm not sure what's going on either, but there's a lot of software in the middle compounding the difficulty in diagnosing what's going on. Please note that Informix 11.50 is getting towards ancient. If it is not already out of support, it soon will be. You should be aiming to upgrade to 12.10 (rather than 11.70) if you are going to continue using Informix. – Jonathan Leffler Aug 24 '15 at 14:15

0 Answers0