9

In my code I am using an external C library and the library calls madvise with MADV_SEQUENTIAL option which takes too long to finish. In my opinion only calling madvise with MADV_SEQUENTIAL is enough for our job. My first question is, why multiple madvise system calls are made, is there a logic in calling madvise with different options sequentially? My second question is, do you have any idea why madvise with MADV_SEQUENTIAL takes too long, sometimes about 1-2 minutes?

[root@mymachine ~]# strace -ttT my_compiled_code
...
13:11:35.358982 open("/some/big/file", O_RDONLY) = 8 <0.000010>
13:11:35.359060 fstat64(8, {st_mode=S_IFREG|0644, st_size=953360384, ...}) = 0 <0.000006>
13:11:35.359155 mmap2(NULL, 1073741824, PROT_READ, MAP_SHARED, 8, 0) = 0x7755e000 <0.000007>
13:11:35.359223 madvise(0x7755e000, 1073741824, MADV_NORMAL) = 0 <0.000006>
13:11:35.359266 madvise(0x7755e000, 1073741824, MADV_RANDOM) = 0 <0.000006>
13:11:35.359886 madvise(0x7755e000, 1073741824, MADV_SEQUENTIAL) = 0 <0.000006>
13:11:53.730549 madvise(0x7755e000, 1073741824, MADV_RANDOM) = 0 <0.000013>
...

I am using 32-bit linux kernel: 3.4.52-9

[root@mymachine ~]# free -lk
             total       used       free     shared    buffers     cached
Mem:       4034412    3419344     615068          0      55712     767824
Low:        853572     495436     358136
High:      3180840    2923908     256932
-/+ buffers/cache:    2595808    1438604
Swap:      4192960     218624    3974336


[root@mymachine ~]# cat /proc/buddyinfo 
Node 0, zone      DMA     89     23      9      4      5      4      4      1      0      2      0 
Node 0, zone   Normal   9615   7099   3997   1723    931    397     78      0      0      1      1 
Node 0, zone  HighMem   7313   8089   2187    420    206     92     41     15      8      3      6
Peter Cordes
  • 328,167
  • 45
  • 605
  • 847
denizeren
  • 934
  • 8
  • 20
  • The question as it stands is too broad - please post some relevant code. A minimal example would be nice. – Michael Foukarakis Aug 06 '14 at 10:28
  • library is close-source so I can't see what is inside the library and this call done in the library blocks my code. – denizeren Aug 06 '14 at 10:32
  • Can you at least show the output of `strace -ttT`? – 1000 Bites Aug 10 '14 at 10:30
  • I have edited the question and added strace -ttT output – denizeren Aug 11 '14 at 10:48
  • Are you sure it's the `madvise` call that takes 1-2 minutes? It's just a hint to the kernel on how to handle the memory range. Furthermore, you are allocating 1gb of memory, so maybe some pages are swapped out and that's causing the slowdown? – Björn Lindqvist Aug 14 '15 at 16:46
  • I'm pretty sure, but since I don't have the source code of the library I cannot be 100% sure. What you say makes sense, but the test environment was not using swap area. My test environment is not available right now, but when I could test it again I will test it again with swap in my mind. – denizeren Aug 17 '15 at 04:57

0 Answers0