1

Here is what I did:

  1. Ran a q process with a limiting vmem argument (say in a 100GB system, running vmem of 50GB)
  2. Logged a unix top command

After the entire process was completed, I was trying to analyse the memory usage. I saw that the process %age memory usage crossed 90% mark. I believed that vmem restricts the memory consumption. But it seems that my process used more than 90GB memory at times.

How can this be explained? Am I missing something?

Donald_W
  • 1,773
  • 21
  • 35
Pratyush
  • 71
  • 1
  • 10
  • Can you give a bit more info maybe? What exactly were your cmd-line args when starting q? Were you monitoring the "VIRT" field of the top command? – terrylynch Nov 04 '14 at 13:43
  • i had passed -vmem x in the cmd-line arg for starting q process. i am monitoring %MEM field of the TOP command – Pratyush Nov 07 '14 at 06:36
  • I'm not sure if that's a valid cmd line arg. You need to use "-w". For example: "q -w 50000" Starts a q process with a 50gb memory limit. See http://code.kx.com/wiki/Reference/Cmdline – terrylynch Nov 07 '14 at 11:30
  • I see. I think I mixed concepts. I had earlier understood that -w limits the memory for execution of a query from external source. Arggh. This makes more sense now. Thanks. – Pratyush Nov 10 '14 at 07:22
  • if you can post this as an answer, i would like to mark it as accepted answer. – Pratyush Nov 10 '14 at 07:26

1 Answers1

1

As #terrylynch said, command line parameter "-w" is used to set memory cap in kdb/q process. I checked and found that "-vmem" is not a kdb command line parameter.

vmem, in this context, is used in unix to manage memory of processes.

Pratyush
  • 71
  • 1
  • 10
  • 1
    Good! As for "limiting memory for execution of a query from an external source" - there is no in-built way to control this but generally you can use gateway processes to be a middle-man between the external source and the database (rdb or hdb). This way if an inefficient query hits the memory limit it will not bring down the rdb/hdb, it will only kill the gateway process. – terrylynch Nov 10 '14 at 11:48
  • thanks very much!!i was able to analyse my process and am working on making it more efficient – Pratyush Nov 10 '14 at 13:01