4

I am trying to figure out the reason of an out of memory issue. I am having with a kernel. I am trying to build (info: from Freescale SabreSD to Solidrun Hummingboard, version 3.10, android distribution). My system has 512MB phisical memory.

My kernel boots with this virtual memory layout:

Memory: 447MB 64MB = 511MB total
Memory: 238916k/238916k available, 285372k reserved, 0K highmem
Virtual kernel memory layout:
    vector  : 0xffff0000 - 0xffff1000   (   4 kB)
    fixmap  : 0xfff00000 - 0xfffe0000   ( 896 kB)
    vmalloc : 0xa0800000 - 0xff000000   (1512 MB)
    lowmem  : 0x80000000 - 0xa0000000   ( 512 MB)
    pkmap   : 0x7fe00000 - 0x80000000   (   2 MB)
    modules : 0x7f000000 - 0x7fe00000   (  14 MB)
      .text : 0x80008000 - 0x80e7a48c   (14794 kB)
      .init : 0x80e7b000 - 0x80ed0600   ( 342 kB)
      .data : 0x80ed2000 - 0x80f56000   ( 528 kB)
       .bss : 0x80f56000 - 0x80feacf8   ( 596 kB)

While a working 3.0.35 kernel boots with the following virtual memory layout:

Memory: 431MB = 431MB total
Memory: 424264k/424264k available, 100024k reserved, 31744K highmem
Virtual kernel memory layout:
    vector  : 0xffff0000 - 0xffff1000   (   4 kB)
    fixmap  : 0xfff00000 - 0xfffe0000   ( 896 kB)
    DMA     : 0xfbe00000 - 0xffe00000   (  64 MB)
    vmalloc : 0xd9800000 - 0xf2000000   ( 392 MB)
    lowmem  : 0xc0000000 - 0xd9000000   ( 400 MB)
    pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
    modules : 0xbf000000 - 0xbfe00000   (  14 MB)
      .init : 0xc0008000 - 0xc0049000   ( 260 kB)
      .text : 0xc0049000 - 0xc0994048   (9517 kB)
      .data : 0xc0996000 - 0xc0a38858   ( 651 kB)
       .bss : 0xc0a3887c - 0xc0bc3af0   (1581 kB)

From my understanding, the vmalloc area is too large, and lowmem matches the phisical available memory. It my assumption right? What kind of configuration should I look at?

I am trying to understand what configuration changes I should do to the kernel or bootloader in order to modify this memory map (and if it is really wrong as I am guessing). Kernel ignores the command line vmalloc parameter (currently is a 400M default, maybe because it is configured to be this large at least). Is the memory layout configured somewhere in the KConfig or is it initialized in Uboot? Can you help me to figure out the right places?

EDIT

After changing CONFIG_VMSPLIT_2G to CONFIG_VMSPLIT_3G, I can see the following memory map:

Memory: 447MB 64MB = 511MB total
Memory: 238884k/238884k available, 285404k reserved, 0K highmem
Virtual kernel memory layout:
    vector  : 0xffff0000 - 0xffff1000   (   4 kB)
    fixmap  : 0xfff00000 - 0xfffe0000   ( 896 kB)
    vmalloc : 0xe0800000 - 0xff000000   ( 488 MB)
    lowmem  : 0xc0000000 - 0xe0000000   ( 512 MB)
    pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
    modules : 0xbf000000 - 0xbfe00000   (  14 MB)
      .text : 0xc0008000 - 0xc0e7a48c   (14794 kB)
      .init : 0xc0e7b000 - 0xc0ed0600   ( 342 kB)
      .data : 0xc0ed2000 - 0xc0f56000   ( 528 kB)
       .bss : 0xc0f56000 - 0xc0feacf8   ( 596 kB)

But I still have an "Out of memory" exception in Init.

Here is the boot log details:

request_suspend_state: wakeup (3->0) at 6224194667 (1970-01-01 00:00:05.154978000 UTC)
init: untracked pid 240 killed by signal 9
request_suspend_state: wakeup (0->0) at 10022039001 (1970-01-01 00:00:08.952822334 UTC)
init: untracked pid 287 killed by signal 9
request_suspend_state: wakeup (0->0) at 14912863668 (1970-01-01 00:00:13.843647334 UTC)
init: untracked pid 332 killed by signal 9
request_suspend_state: wakeup (0->0) at 20705884002 (1970-01-01 00:00:19.636667335 UTC)
init: untracked pid 376 killed by signal 9
request_suspend_state: wakeup (0->0) at 26695295003 (1970-01-01 00:00:25.626078336 UTC)
init: untracked pid 411 killed by signal 9
request_suspend_state: wakeup (0->0) at 31950397337 (1970-01-01 00:00:30.881180670 UTC)
init: untracked pid 454 killed by signal 9
request_suspend_state: wakeup (0->0) at 37370094004 (1970-01-01 00:00:36.300875670 UTC)
init: untracked pid 490 exited with status 1
request_suspend_state: wakeup (0->0) at 42182273338 (1970-01-01 00:00:41.113056337 UTC)
init: untracked pid 535 killed by signal 9
request_suspend_state: wakeup (0->0) at 47580075672 (1970-01-01 00:00:46.510858671 UTC)
netd invoked oom-killer: gfp_mask=0x2084d0, order=0, oom_score_adj=-1000
CPU: 0 PID: 588 Comm: netd Not tainted 3.10.53-g4f0dd92 #1
[<c0013fc8>] (unwind_backtrace+0x0/0xf8) from [<c00118f4>] (show_stack+0x10/0x14)
[<c00118f4>] (show_stack+0x10/0x14) from [<c07d9e28>] (dump_header.isra.10+0x6c/0x194)
[<c07d9e28>] (dump_header.isra.10+0x6c/0x194) from [<c00a8418>] (out_of_memory+0x29c/0x2d4)
[<c00a8418>] (out_of_memory+0x29c/0x2d4) from [<c00ac20c>] (__alloc_pages_nodemask+0x834/0x85c)
[<c00ac20c>] (__alloc_pages_nodemask+0x834/0x85c) from [<c00c16c0>] (__pte_alloc+0x24/0x13c)
[<c00c16c0>] (__pte_alloc+0x24/0x13c) from [<c00c4724>] (handle_mm_fault+0xdc/0xf0)
[<c00c4724>] (handle_mm_fault+0xdc/0xf0) from [<c00c490c>] (__get_user_pages+0x110/0x424)
[<c00c490c>] (__get_user_pages+0x110/0x424) from [<c00e2c20>] (get_arg_page+0x50/0xb4)
[<c00e2c20>] (get_arg_page+0x50/0xb4) from [<c00e2d88>] (copy_strings+0x104/0x260)
[<c00e2d88>] (copy_strings+0x104/0x260) from [<c00e2f04>] (copy_strings_kernel+0x20/0x28)
[<c00e2f04>] (copy_strings_kernel+0x20/0x28) from [<c00e4718>] (do_execve+0x3d8/0x4b8)
[<c00e4718>] (do_execve+0x3d8/0x4b8) from [<c00e4a0c>] (SyS_execve+0x30/0x44)
[<c00e4a0c>] (SyS_execve+0x30/0x44) from [<c000e0c0>] (ret_fast_syscall+0x0/0x30)
Mem-info:
DMA per-cpu:
CPU    0: hi:   90, btch:  15 usd:  22
CPU    1: hi:   90, btch:  15 usd:  42
active_anon:55770 inactive_anon:3 isolated_anon:0
 active_file:650 inactive_file:1024 isolated_file:0
 unevictable:342 dirty:0 writeback:0 unstable:0
 free:30058 slab_reclaimable:733 slab_unreclaimable:1420
 mapped:870 shmem:9 pagetables:263 bounce:0
 free_cma:29780
DMA free:120232kB min:1952kB low:2440kB high:2928kB active_anon:223224kB inactive_anon:12kB active_file:2600kB inactive_file:4168kB unevictable:1368kB isolated(anon):0kB isolated(file):0kB present:523264kB managed:238884kB mlocked:0kB dirty:0kB writeback:0kB mapped:3532kB shmem:36kB slab_reclaimable:2932kB slab_unreclaimable:5680kB kernel_stack:880kB pagetables:1052kB unstable:0kB bounce:0kB free_cma:119120kB writeback_tmp:0kB pages_scanned:9097 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 2124*4kB (UC) 2122*8kB (MC) 2124*16kB (UC) 1875*32kB (UMC) 1*64kB (M) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB 0*32768kB = 119520kB
2181 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
131072 pages of RAM
29928 free pages
5552 reserved pages
1542 slab pages
2939 pages shared
0 pages swap cached
[ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
[  134]     0   134      655      109       4        0         -1000 ueventd
[  135]     0   135      654       30       4        0         -1000 watchdogd
[  146]  1036   146     3060      186       8        0         -1000 logd
[  147]     0   147      922       73       4        0         -1000 healthd
[  148]     0   148     1118      121       4        0         -1000 lmkd
[  149]  1000   149      812      124       3        0         -1000 servicemanager
[  150]     0   150     2445      196       7        0         -1000 vold
[  153]     0   153     1228      241       4        0         -1000 debuggerd
[  154]  1019   154     3495      307       8        0         -1000 drmserver
[  156]  1012   156      801       68       3        0         -1000 installd
[  157]  1017   157     1571      144       5        0         -1000 keystore
[  158]     0   158     1310      165       4        0         -1000 rild
[  161]  1023   161     2317      119       6        0         -1000 sdcard
[  163]  2000   163     1935       32       6        0         -1000 adbd
[  402]     0   402    32816    17695      48        0         -1000 main
[  444]     0   444    30768    15807      46        0         -1000 main
[  487]     0   487    29744    13571      41        0         -1000 main
[  528]     0   528    29744     7304      31        0         -1000 main
[  583]     0   583     1103      148       4        0         -1000 netd
[  584]  1000   584     2173      286       5        0         -1000 surfaceflinger
[  585]  1013   585     3026      619       7        0         -1000 mediaserver
[  586]     0   586     2168      356       6        0         -1000 app_process
[  588]     0   588     1103       91       4        0         -1000 netd
Kernel panic - not syncing: Out of memory and no killable processes...

CPU0: stopping
CPU: 0 PID: 528 Comm: main Not tainted 3.10.53-g4f0dd92 #1
[<c0013fc8>] (unwind_backtrace+0x0/0xf8) from [<c00118f4>] (show_stack+0x10/0x14)
[<c00118f4>] (show_stack+0x10/0x14) from [<c0013214>] (handle_IPI+0x178/0x1a4)
[<c0013214>] (handle_IPI+0x178/0x1a4) from [<c0008564>] (gic_handle_irq+0x54/0x5c)
[<c0008564>] (gic_handle_irq+0x54/0x5c) from [<c000de7c>] (__irq_usr+0x3c/0x60)
Exception stack(0xc3ff3fb0 to 0xc3ff3ff8)
3fa0:                                     f9ab7147 b61cf1a0 00000000 f8d0b34a
3fc0: b5324c28 b5324c28 b5325360 b61cf1a0 00000000 00000000 b5325300 b5310400
3fe0: b6eca758 be809c00 b6ce96a1 b6cda996 200e3430 ffffffff
Rebooting in 5 seconds..

Any idea about the reason I am running out of memory?

Fab
  • 139
  • 11
  • I've found that the init script or the child services seem to be re-launched again if something is going wrong, without freeing the internal memory... And that "main" process should be the init script itself. So the issue might be related only to init scripts... – Fab Oct 23 '15 at 11:39
  • Could you share dtb and cat /proc/iomem? – Alex Hoppus Oct 23 '15 at 16:38
  • Thanks Alex for your interest. I had an issue in dtb, and framebuffer was not initialized. Init services were keeping restarting and filling the available memory. Now I have the console prompt working as expected, and I am trying to work on an issue with framebuffer/HDMI output. Tomorrow I'll check again DTB files. If you are so kind to have a look at them, I'll post here the link to our repository when they're ready or if we are not able to find the issue. Do you have notes to add to the memory layouts above? Is the first memory map acceptable? Where does the 1512MB memory maps? – Fab Oct 25 '15 at 14:20

0 Answers0