0

I have a memory leak process which cause OOM kill to other process. The OOM log below:

 [3669.400 s]4,1153,3669403826,-;test_process: vmalloc: allocation failure, allocated 102400 of 135168 bytes, mode:0x6000c1(GFP_KERNEL|GFP_DMA), nodemask=(null)

 [3669.400 s]6,1154,3669403836,-;test_process cpuset=/ mems_allowed=0

 [3669.400 s]4,1155,3669403950,-;CPU: 0 PID: 346 Comm: test_process Tainted: G           O      4.19.73 #1

 [3669.400 s]4,1156,3669403956,-;Hardware name: Generic DRA74X (Flattened Device Tree)

 [3669.400 s]4,1157,3669403962,-;Backtrace: 

 [3669.400 s]4,1158,3669403974,-;Function entered at [<c020c974>] from [<c020cc9c>]

 [3669.400 s]4,1159,3669403984,-; r7:ffffe000 r6:60030013 r5:00000000 r4:c0e45958

 [3669.400 s]4,1160,3669403990,-;Function entered at [<c020cc84>] from [<c0834980>]

 [3669.400 s]4,1161,3669403995,-;Function entered at [<c08348f0>] from [<c02f8498>]

 [3669.400 s]4,1162,3669404003,-; r7:ffffe000 r6:c0a8f28c r5:00000000 r4:c0e04c48

 [3669.400 s]4,1163,3669404009,-;Function entered at [<c02f83d4>] from [<c0330e9c>]

 [3669.400 s]4,1164,3669404016,-; r3:00019000 r2:c0a8f28c

 [3669.400 s]4,1165,3669404023,-; r7:ffffffff r6:006000c1 r5:00000019 r4:ce342800

 [3669.400 s]4,1166,3669404029,-;Function entered at [<c0330d50>] from [<c0330fec>]

 [3669.400 s]4,1167,3669404037,-; r9:00000000 r8:00020000 r7:00020000 r6:02024200 r5:bf031f18 r4:cd53fc00

 [3669.400 s]4,1168,3669404043,-;Function entered at [<c0330f9c>] from [<bf0052c4>]

 [3669.400 s]4,1169,3669404049,-;Function entered at [<bf00523c>] from [<bf005388>]

 [3669.400 s]4,1170,3669404055,-;Function entered at [<bf00533c>] from [<bf0009b4>]

 [3669.400 s]4,1171,3669404063,-; r8:00020000 r7:02024200 r6:00020000 r5:c0e04c48 r4:02024200

 [3669.400 s]4,1172,3669404068,-;Function entered at [<bf0008bc>] from [<bf008b84>]

 [3669.400 s]4,1173,3669404074,-; r6:d7da1f80 r5:c0e04c48 r4:00000000

 [3669.400 s]4,1174,3669404080,-;Function entered at [<bf0088c8>] from [<bf01316c>]

 [3669.400 s]4,1175,3669404090,-; r10:bf0088c8 r9:d7679bdc r8:00000000 r7:00020000 r6:00001000 r5:c0e04c48

 [3669.400 s]4,1176,3669404095,-; r4:d7eaf700

 [3669.400 s]4,1177,3669404101,-;Function entered at [<bf013094>] from [<bf008f74>]

 [3669.400 s]4,1178,3669404110,-; r10:0000020b r9:d7eaf700 r8:ce342100 r7:d7da1f80 r6:00000040 r5:c0e04c48

 [3669.400 s]4,1179,3669404115,-; r4:00000000

 [3669.400 s]4,1180,3669404120,-;Function entered at [<bf008e74>] from [<bf009790>]

 [3669.400 s]4,1181,3669404129,-; r10:d7da1f80 r9:0000000b r8:00020000 r7:d7679cb0 r6:d07ed100 r5:00000000

 [3669.400 s]4,1182,3669404133,-; r4:c0e04c48

 [3669.400 s]4,1183,3669404139,-;Function entered at [<bf0096b8>] from [<bf00a074>]

 [3669.400 s]4,1184,3669404147,-; r10:d7679d10 r9:d8a6d700 r8:d7da1f80 r7:00000000 r6:00000000 r5:0000000b

 [3669.400 s]4,1185,3669404152,-; r4:c0e04c48

 [3669.400 s]4,1186,3669404157,-;Function entered at [<bf009f88>] from [<bf0157cc>]

 [3669.401 s]4,1187,3669404165,-; r10:d7679e2c r9:00000000 r8:d762e180 r7:f1eec000 r6:c0e04c48 r5:00000000

 [3669.401 s]4,1188,3669404170,-; r4:f1eeb000

 [3669.401 s]4,1189,3669404176,-;Function entered at [<bf015650>] from [<bf017b04>]

 [3669.401 s]4,1190,3669404184,-; r9:0000015a r8:f1eeb000 r7:f1eec000 r6:bf015650 r5:d762e180 r4:d7679e2c

 [3669.401 s]4,1191,3669404190,-;Function entered at [<bf017a70>] from [<bf0048e8>]

 [3669.401 s]4,1192,3669404198,-; r8:d762e180 r7:d7eaff00 r6:c0e04c48 r5:c01c6707 r4:d7679e2c

 [3669.401 s]4,1193,3669404204,-;Function entered at [<bf004808>] from [<c0590564>]

 [3669.401 s]4,1194,3669404212,-; r9:bf004808 r8:00000030 r7:00000000 r6:d865b800 r5:d7eaff00 r4:c0e04c48

 [3669.401 s]4,1195,3669404217,-;Function entered at [<c05904b4>] from [<c05909b4>]

 [3669.401 s]4,1196,3669404225,-; r10:bf0312b0 r9:00000000 r8:bea3cc2c r7:401c6440 r6:d7eaff00 r5:d7edc240

 [3669.401 s]4,1197,3669404230,-; r4:c0e04c48

 [3669.401 s]4,1198,3669404235,-;Function entered at [<c059070c>] from [<c0366f4c>]

 [3669.401 s]4,1199,3669404243,-; r10:d8646f60 r9:d7678000 r8:bea3cc2c r7:00000013 r6:d7edc240 r5:bea3cc2c

 [3669.401 s]4,1200,3669404248,-; r4:c0e04c48

 [3669.401 s]4,1201,3669404253,-;Function entered at [<c0366e94>] from [<c0367690>]

 [3669.401 s]4,1202,3669404261,-; r10:00000036 r9:d7678000 r8:bea3cc2c r7:401c6440 r6:00000013 r5:d7edc240

 [3669.401 s]4,1203,3669404266,-; r4:d7edc241

 [3669.401 s]4,1204,3669404272,-;Function entered at [<c0367654>] from [<c03676cc>]

 [3669.401 s]4,1205,3669404280,-; r9:d7678000 r8:c0201204 r7:00000036 r6:401c6440 r5:bea3cc2c r4:c01c6707

 [3669.401 s]4,1206,3669404285,-;Function entered at [<c03676bc>] from [<c0201000>]

 [3669.401 s]4,1207,3669404291,-;Exception stack(0xd7679fa8 to 0xd7679ff0)

 [3669.401 s]4,1208,3669404299,-;9fa0:                   c01c6707 bea3cc2c 00000013 401c6440 bea3cc2c 0000001c

 [3669.401 s]4,1209,3669404307,-;9fc0: c01c6707 bea3cc2c 401c6440 00000036 0000000b 010b8b80 00000040 00000004

 [3669.401 s]4,1210,3669404314,-;9fe0: b6447094 bea3cc04 b643219b b4c2815c

 [3669.401 s]6,1211,3669404603,-;PVR_K:(Error): NewVMallocLinuxMemArea: failed!

 [3669.401 s]6,1212,3669404617,-;PVR_K:(Error): BM_ImportMemory: OSAllocPages(0x20000) failed

 [3669.401 s]6,1213,3669404667,-;PVR_K:(Error): AllocMemory: RA_Alloc(0x20000) FAILED

 [3669.401 s]6,1214,3669404679,-;PVR_K:(Error): BM_Alloc: AllocMemory FAILED

 [3669.401 s]6,1215,3669404689,-;PVR_K:(Error): AllocDeviceMem: BM_Alloc Failed

 [3669.401 s]6,1216,3669404702,-;PVR_K:(Error): PVRSRVAllocDeviceMemBW: PVRSRVAllocDeviceMemKM failed with eError = 1

 [3669.514 s]4,1217,3669518796,-;test_process invoked oom-killer: gfp_mask=0x6000c0(GFP_KERNEL), nodemask=(null), order=0, oom_score_adj=0

 [3669.514 s]6,1218,3669518809,-;test_process cpuset=/ mems_allowed=0

 [3669.514 s]4,1219,3669519201,-;CPU: 0 PID: 346 Comm: test_process Tainted: G           O      4.19.73 #1

 [3669.514 s]4,1220,3669519208,-;Hardware name: Generic DRA74X (Flattened Device Tree)

 [3669.514 s]4,1221,3669519215,-;Backtrace: 

 [3669.514 s]4,1222,3669519231,-;Function entered at [<c020c974>] from [<c020cc9c>]

 [3669.514 s]4,1223,3669519241,-; r7:d758bb44 r6:60030013 r5:00000000 r4:c0e45958

 [3669.514 s]4,1224,3669519247,-;Function entered at [<c020cc84>] from [<c0834980>]

 [3669.514 s]4,1225,3669519253,-;Function entered at [<c08348f0>] from [<c02f3dac>]

 [3669.514 s]4,1226,3669519262,-; r7:d758bb44 r6:d7679df4 r5:d758b600 r4:d7679df4

 [3669.514 s]4,1227,3669519269,-;Function entered at [<c02f3d34>] from [<c02f3194>]

 [3669.514 s]4,1228,3669519280,-; r10:00000000 r9:0000015b r8:c0a8cae0 r7:d758bb44 r6:d7679df4 r5:d758b608

 [3669.514 s]4,1229,3669519285,-; r4:d758b600

 [3669.514 s]4,1230,3669519290,-;Function entered at [<c02f30cc>] from [<c02f3ac0>]

 [3669.514 s]4,1231,3669519300,-; r10:00000000 r9:0000112c r8:c0e04e44 r7:c0e080c0 r6:c0e04c48 r5:d7679df4

 [3669.514 s]4,1232,3669519306,-; r4:d758b600

 [3669.514 s]4,1233,3669519313,-;Function entered at [<c02f3954>] from [<c02f90fc>]

 [3669.514 s]4,1234,3669519321,-; r7:00000002 r6:c0e04c7c r5:00404100 r4:00000000

 [3669.514 s]4,1235,3669519328,-;Function entered at [<c02f85e4>] from [<c02f94c8>]

 [3669.514 s]4,1236,3669519337,-; r10:02065e48 r9:00000025 r8:d7679f60 r7:d7ebb200 r6:c0e04c48 r5:d758b600

 [3669.514 s]4,1237,3669519343,-; r4:bea3de64

 [3669.514 s]4,1238,3669519349,-;Function entered at [<c02f94ac>] from [<c03b771c>]

 [3669.514 s]4,1239,3669519356,-;Function entered at [<c03b755c>] from [<c03537d0>]

 [3669.514 s]4,1240,3669519365,-; r10:00000003 r9:00000000 r8:d7679f60 r7:34dae64e r6:c03b755c r5:d7f4acc0

 [3669.514 s]4,1241,3669519371,-; r4:c0e04c48

 [3669.514 s]4,1242,3669519378,-;Function entered at [<c0353790>] from [<c0353984>]

 [3669.514 s]4,1243,3669519387,-; r9:00000000 r8:00000001 r7:d7679f60 r6:02065e48 r5:d7f4acc0 r4:00000400

 [3669.514 s]4,1244,3669519394,-;Function entered at [<c03538f4>] from [<c0353ee0>]

 [3669.514 s]4,1245,3669519404,-; r9:00000000 r8:00000000 r7:02065e48 r6:d7f4acc3 r5:c0e04c48 r4:d7f4acc0

 [3669.514 s]4,1246,3669519410,-;Function entered at [<c0353e74>] from [<c0353f64>]

 [3669.514 s]4,1247,3669519418,-; r9:d7678000 r8:c0201204 r7:00000003 r6:0000002b r5:02065e48 r4:00000400

 [3669.514 s]4,1248,3669519425,-;Function entered at [<c0353f54>] from [<c0201000>]

 [3669.514 s]4,1249,3669519432,-;Exception stack(0xd7679fa8 to 0xd7679ff0)

 [3669.514 s]4,1250,3669519441,-;9fa0:                   00000400 02065e48 0000002b 02065e48 00000400 00000000

 [3669.514 s]4,1251,3669519452,-;9fc0: 00000400 02065e48 0000002b 00000003 00000002 02065ac8 bea3d2f4 b4f0980c

 [3669.515 s]4,1252,3669519461,-;9fe0: 00000000 bea3d000 b2dd94d0 b4c1f1a4

 [3669.515 s]4,1253,3669519622,-;Mem-Info:

 [3669.515 s]4,1254,3669519643,-;active_anon:48392 inactive_anon:3552 isolated_anon:0\x0a active_file:10973 inactive_file:3258 isolated_file:0\x0a unevictable:0 dirty:16 writeback:0 unstable:0\x0a slab_reclaimable:1389 slab_unreclaimable:3786\x0a mapped:45654 shmem:3559 pagetables:949 bounce:0\x0a free:13805 free_pcp:94 free_cma:12383

 [3669.515 s]4,1255,3669519656,-;Node 0 active_anon:193568kB inactive_anon:14208kB active_file:43892kB inactive_file:13032kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:182616kB dirty:64kB writeback:0kB shmem:14236kB writeback_tmp:0kB unstable:0kB all_unreclaimable? yes

 [3669.515 s]4,1256,3669519671,c;DMA free:17316kB min:2016kB low:2520kB high:3024kB active_anon:77820kB inactive_anon:8168kB active_file:92kB inactive_file:68kB unevictable:0kB writepending:0kB present:687104kB managed:437000kB mlocked:0kB kernel_stack:2872kB pagetables:1216kB bounce:0kB free_pcp:260kB local_pcp:252kB free_cma:15480kB

 [3669.515 s]4,1257,3669519677,-;lowmem_reserve[]: 0 0 125 125

 [3669.515 s]4,1258,3669519714,c;HighMem free:37904kB min:128kB low:380kB high:632kB active_anon:115748kB inactive_anon:6040kB active_file:43812kB inactive_file:12884kB unevictable:0kB writepending:156kB present:259072kB managed:259072kB mlocked:0kB kernel_stack:0kB pagetables:2580kB bounce:0kB free_pcp:116kB local_pcp:100kB free_cma:34052kB

 [3669.515 s]4,1259,3669519754,-;lowmem_reserve[]: 0 0 0 0

 [3669.515 s]4,1260,3669519784,c;DMA: 658*4kB (UMEC) 365*8kB (UMEC) 184*16kB (UMC) 102*32kB (MC) 61*64kB (C) 11*128kB (C) 2*256kB (C) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 17584kB

 [3669.515 s]4,1261,3669519923,c;HighMem: 1791*4kB (UMC) 1291*8kB (UMC) 438*16kB (UMC) 74*32kB (C) 7*64kB (C) 1*128kB (C) 1*256kB (C) 0*512kB 2*1024kB (C) 2*2048kB (C) 1*4096kB (C) = 37940kB

 [3669.515 s]4,1262,3669520074,-;17794 total pagecache pages

 [3669.515 s]4,1263,3669520080,-;0 pages in swap cache

 [3669.515 s]4,1264,3669520088,-;Swap cache stats: add 0, delete 0, find 0/0

 [3669.515 s]4,1265,3669520158,-;Free swap  = 0kB

 [3669.515 s]4,1266,3669520168,-;Total swap = 0kB

 [3669.515 s]4,1267,3669520178,-;236544 pages RAM

 [3669.515 s]4,1268,3669520185,-;64768 pages HighMem/MovableOnly

 [3669.515 s]4,1269,3669520191,-;62526 pages reserved

 [3669.515 s]4,1270,3669520199,-;77824 pages cma reserved

 [3669.515 s]6,1271,3669520208,-;Tasks state (memory values in pages):

 [3669.515 s]6,1272,3669520218,-;[  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name

 [3669.515 s]6,1273,3669520281,-;[    101]     0   101     2930      329    32768        0         -1000 systemd-udevd

 [3669.515 s]6,1274,3669520295,-;[    111]     0   111    13068     4913   118784        0             0 weston

 [3669.515 s]6,1275,3669520305,-;[    133]     0   133     2496      787    40960        0             0 weston-keyboard

 [3669.515 s]6,1276,3669520314,-;[    134]     0   134     3556     1877    53248        0             0 weston-desktop-

 [3669.515 s]6,1277,3669520325,-;[    135]     0   135     1719      957    36864        0             0 superman

 [3669.515 s]6,1278,3669520336,-;[    137]     0   137     7647     1067    53248        0             0 hal

 [3669.515 s]6,1279,3669520344,-;[    140]     0   140     1380      699    28672        0             0 keyman

 [3669.515 s]6,1280,3669520353,-;[    141]     0   141     5670     1102    73728        0             0 mediaservice

 [3669.515 s]6,1281,3669520375,-;[    142]     0   142     6358     1081    73728        0             0 inforbroker

 [3669.515 s]6,1282,3669520410,-;[    143]     0   143     4216      931    36864        0             0 diagman

 [3669.515 s]6,1283,3669520421,-;[    144]     0   144     1706      953    32768        0             0 appman

 [3669.515 s]6,1284,3669520431,-;[    149]     0   149   178696     2000   675840        0             0 apps.out

 [3669.515 s]6,1285,3669520440,-;[    150]     0   150    42577    12144   229376        0             0 mediaplayer

 [3669.515 s]6,1286,3669520449,-;[    151]     0   151     2523     1422    40960        0             0 mediaplayer

 [3669.515 s]6,1287,3669520458,-;[    152]     0   152     2523     1434    36864        0             0 mediaplayer

 [3669.515 s]6,1288,3669520467,-;[    155]     0   155     2523     1422    40960        0             0 mediaplayer

 [3669.515 s]6,1289,3669520476,-;[    156]     0   156     2523     1424    36864        0             0 mediaplayer

 [3669.515 s]6,1290,3669520485,-;[    158]     0   158    61243     8169   393216        0             0 mediaplayer

 [3669.515 s]6,1291,3669520493,-;[    159]     0   159    11884     1874    77824        0             0 mediaplayer

 [3669.515 s]6,1292,3669520543,-;[    162]     0   162    33177     5213   143360        0             0 mediaplayer

 [3669.515 s]6,1293,3669520552,-;[    163]     0   163     2523     1416    40960        0             0 mediaplayer

 [3669.515 s]6,1294,3669520563,-;[    165]     0   165     2523     1416    40960        0             0 mediaplayer

 [3669.515 s]6,1295,3669520571,-;[    211]     0   211    27478      723   237568        0             0 thrive_storm

 [3669.516 s]6,1296,3669520584,-;[    339]     0   339    15930     1335    77824        0             0 avm_controller

 [3669.524 s]6,1297,3669520593,-;[    346]     0   346   178977    60276   970752        0             0 test_process

 [3669.524 s]6,1298,3669520604,-;[    486]     0   486      587      183    28672        0             0 dropbear

 [3669.524 s]6,1299,3669520612,-;[    487]     0   487     1172      306    32768        0             0 login

 [3669.524 s]6,1300,3669520625,-;[    488]     0   488    57375     1248   176128        0             0 cvf_new_app

 [3669.524 s]6,1301,3669520670,-;[    516]     0   516     7820      932    61440        0             0 cvf_controller

 [3669.524 s]6,1302,3669520682,-;[    523]     0   523     1410      719    32768        0             0 installman

 [3669.524 s]6,1303,3669520689,-;[    524]     0   524    28712     1844   118784        0             0 dvr

 [3669.524 s]6,1304,3669520700,-;[    567]     0   567      708      363    28672        0             0 sh

 [3669.524 s]6,1305,3669520710,-;[  11642]     0 11642      590      297    24576        0             0 tail

 [3669.524 s]3,1306,3669520720,-;Out of memory: Kill process 346 (test_process) score 347 or sacrifice child

 [3669.524 s]3,1307,3669520882,-;Killed process 346 (test_process) total-vm:715908kB, anon-rss:82476kB, file-rss:153224kB, shmem-rss:5404kB

 [3669.580 s]6,1308,3669577721,-;oom_reaper: reaped process 346 (test_process), now anon-rss:0kB, file-rss:138388kB, shmem-rss:5404kB

 [3669.619 s]connection closed

From log, it shows vmalloc with (GFP_KERNEL|GFP_DMA) mode get 135168 bytes size memory failed then triggered OOM kill.
But from "Mem-Info", it shows DMA free have 17316kB which is should have enough size for this vmalloc.
So if it possible that DMA zone have no continuous memory now? Or what others would possible trigger this kill?
Thanks a lot!

BR/Tim

Tim Cheng
  • 33
  • 8

0 Answers0