- Ubuntu 18.04.2 LTS
- Linux node-b02-08-06 5.3.0-45-generic #37~18.04.1-Ubuntu SMP Fri Mar 27 15:58:10 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
- Using XFS with Ceph rdb and mount by OverlayFS, sometimes system hung up with kernel crash like below:
[2492329.917737] VFS: Busy inodes after unmount of overlay. Self-destruct in 5 seconds. Have a nice day...
[2492369.349682] XFS (rbd16): Unmounting Filesystem
[2492398.245190] rbd: rbd16: capacity 2199023255552 features 0x13d
[2492398.455244] XFS (rbd16): Mounting V5 Filesystem
[2492398.885585] XFS (rbd16): Starting recovery (logdev: internal)
[2492400.922504] XFS (rbd16): Ending recovery (logdev: internal)
[2492809.899680] BUG: Dentry 000000000d0cd936{i=50430cc2,n=EMUI6.0_Kirin960_20170803} still in use (1) [unmount of xfs rbd28]
[2492809.900021] ------------[ cut here ]------------
[2492809.900045] WARNING: CPU: 52 PID: 1116291 at /build/linux-hwe-UUUsac/linux-hwe-5.3.0/fs/dcache.c:1596 umount_check+0x69/0x80
[2492809.900046] Modules linked in: xfs ip6table_nat ip6_tables rbd libceph xt_set xt_multiport ipt_rpfilter iptable_raw ip_set_hash_ip ip_set_hash_net ip_set veth nbd xt_nat xt_statistic ipt_REJECT nf_reject_ipv4 bluetooth ip_
[2492809.900098] mei lpc_ich acpi_power_meter mac_hid sunrpc sch_fq_codel ip_tables x_tables autofs4 btrfs xor zstd_compress raid6_pq libcrc32c megaraid_sas mlx5_ib ses enclosure ib_uverbs ib_core hid_generic mlx5_core tls mlx
[2492809.900120] CPU: 52 PID: 1116291 Comm: umount Tainted: G OE 5.3.0-45-generic #37~18.04.1-Ubuntu
[2492809.900121] Hardware name: Huawei 2488H V5/BC62MBHA0, BIOS 7.55 04/21/2020
[2492809.900123] RIP: 0010:umount_check+0x69/0x80
[2492809.900125] Code: 08 48 8b 46 30 48 85 c0 74 27 48 8b 50 40 51 48 c7 c7 50 78 f4 ac 48 89 f1 e8 a6 11 e2 ff 48 c7 c7 a0 3e f0 ac e8 9a 11 e2 ff <0f> 0b 58 31 c0 c9 c3 31 d2 eb d9 66 90 66 2e 0f 1f 84 00 00 00 00
[2492809.900125] RSP: 0018:ffffb8a6b68a3d50 EFLAGS: 00010282
[2492809.900126] RAX: 0000000000000024 RBX: ffff8b102f8f3c50 RCX: 0000000000000006
[2492809.900127] RDX: 0000000000000000 RSI: 0000000000000082 RDI: ffff8ae9ff117440
[2492809.900128] RBP: ffffb8a6b68a3d58 R08: 0000000000001a82 R09: 0000000000000004
[2492809.900128] R10: ffffb8a6b68a3c68 R11: 0000000000000001 R12: ffff8b102f8f3bc0
[2492809.900129] R13: ffff8b102f8f2150 R14: ffff8a86697c2300 R15: ffff8b102f8f3c18
[2492809.900130] FS: 00007f8d92b55080(0000) GS:ffff8ae9ff100000(0000) knlGS:0000000000000000
[2492809.900131] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[2492809.900131] CR2: 000000a165d0f018 CR3: 0000002ea4f7c003 CR4: 00000000007606e0
[2492809.900132] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[2492809.900133] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[2492809.900133] PKRU: 55555554
[2492809.900133] Call Trace:
[2492809.900138] d_walk+0xd5/0x270
[2492809.900139] ? shrink_lock_dentry.part.18+0xf0/0xf0
[2492809.900142] do_one_tree+0x24/0x40
[2492809.900143] shrink_dcache_for_umount+0x2d/0x90
[2492809.900147] generic_shutdown_super+0x1f/0x120
[2492809.900148] kill_block_super+0x2c/0x80
[2492809.900149] deactivate_locked_super+0x48/0x80
[2492809.900151] deactivate_super+0x40/0x60
[2492809.900155] cleanup_mnt+0xbd/0x160
[2492809.900156] __cleanup_mnt+0x12/0x20
[2492809.900163] task_work_run+0x9d/0xc0
[2492809.900170] exit_to_usermode_loop+0x109/0x130
[2492809.900171] do_syscall_64+0x115/0x130
[2492809.900177] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[2492809.900179] RIP: 0033:0x7f8d91b8bf5b
[2492809.900181] Code: ff 2b 00 f7 d8 64 89 01 48 83 c8 ff c3 66 90 f3 0f 1e fa 31 f6 e9 05 00 00 00 0f 1f 44 00 00 f3 0f 1e fa b8 a6 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d fd fe 2b 00 f7 d8 64 89 01 48
[2492809.900181] RSP: 002b:00007ffe445a8238 EFLAGS: 00000246 ORIG_RAX: 00000000000000a6
[2492809.900182] RAX: 0000000000000000 RBX: 00005582b77a3970 RCX: 00007f8d91b8bf5b
[2492809.900183] RDX: 0000000000000001 RSI: 0000000000000000 RDI: 00005582b77a3b50
[2492809.900183] RBP: 0000000000000000 R08: 00005582b77a49d0 R09: 00007f8d91c0cd40
[2492809.900184] R10: 0000000000000000 R11: 0000000000000246 R12: 00005582b77a3b50
[2492809.900184] R13: 00007f8d92937184 R14: 0000000000000000 R15: 00000000ffffffff
How to fix this problme. We try to use apport-collect to report this to ubuntu, but there is no log in dir.
