3

I'm stuck with this problem and I can't solve it, we have a website so people can upload their videos on it in some path (Temp directory for example) then our admins go and move these video to another location in our servers, backend of this storage is glusterfs suddenly we are getting this error on our server's and I don't know why and how I can solve it, by the way after a moment our admin's retries to move these file's and the problem will resolve, also I should say I'm a new sysadmin in this company and I'm not familiar with gluster.

glusterfs 3.12.14

mnt_sv3.log:[2020-08-02 23:01:54.869832] W [fuse-bridge.c:1381:fuse_err_cbk] 0-glusterfs-fuse: 72213904: FLUSH() ERR => -1 (Read-only file system)
mnt_sv3.log:[2020-08-02 23:01:57.105487] W [fuse-bridge.c:1445:fuse_unlink_cbk] 0-glusterfs-fuse: 72219047: UNLINK() /archives/videos_file/a9/0f/9Gok6_a90f19fcb8f03399d11548430d53b435f3f1e27f_o_720.mp4 => -1 (Read-only file system)
mnt_sv3.log:[2020-08-02 23:01:57.128186] W [fuse-bridge.c:779:fuse_truncate_cbk] 0-glusterfs-fuse: 72219050: TRUNCATE() /archives/videos_file/a9/0f/9Gok6_a90f19fcb8f03399d11548430d53b435f3f1e27f_o_720.mp4 => -1 (Read-only file system)
mnt_sv3.log:[2020-08-02 23:01:59.247595] W [fuse-bridge.c:1445:fuse_unlink_cbk] 0-glusterfs-fuse: 72221389: UNLINK() /archives/videos_file/a9/0f/9Gok6_a90f19fcb8f03399d11548430d53b435f3f1e27f_o_720.mp4 => -1 (Read-only file system)
mnt_sv3.log:[2020-08-03 14:52:54.175746] W [fuse-bridge.c:2402:fuse_writev_cbk] 0-glusterfs-fuse: 72227619: WRITE => -1 gfid=e2b5ff7e-f20b-49bd-98a8-c04e53aa76ad fd=0x7f27a8059c80 (Read-only file system)
mnt_sv3.log:[2020-08-03 14:52:54.176387] W [fuse-bridge.c:2402:fuse_writev_cbk] 0-glusterfs-fuse: 72227621: WRITE => -1 gfid=e2b5ff7e-f20b-49bd-98a8-c04e53aa76ad fd=0x7f27a8059c80 (Read-only file system)
mnt_sv3.log:[2020-08-03 14:52:54.176901] W [fuse-bridge.c:1381:fuse_err_cbk] 0-glusterfs-fuse: 72227622: FLUSH() ERR => -1 (Read-only file system)
mnt_sv3.log:[2020-08-03 14:52:56.590612] W [fuse-bridge.c:1445:fuse_unlink_cbk] 0-glusterfs-fuse: 72232405: UNLINK() /archives/videos_file/8e/e4/e8zKy_8ee485ac49a1274fef650b3cf7fc8e11850218a6_o_720.flv => -1 (Read-only file system)
mnt_sv3.log:[2020-08-03 14:52:56.616059] W [fuse-bridge.c:779:fuse_truncate_cbk] 0-glusterfs-fuse: 72232408: TRUNCATE() /archives/videos_file/8e/e4/e8zKy_8ee485ac49a1274fef650b3cf7fc8e11850218a6_o_720.flv => -1 (Read-only file system)
mnt_sv3.log:[2020-08-03 14:52:57.772769] W [fuse-bridge.c:1445:fuse_unlink_cbk] 0-glusterfs-fuse: 72233499: UNLINK() /archives/videos_file/8e/e4/e8zKy_8ee485ac49a1274fef650b3cf7fc8e11850218a6_o_720.flv => -1 (Read-only file system)
mnt_sv3.log:[2020-08-03 15:20:50.062578] W [fuse-bridge.c:2402:fuse_writev_cbk] 0-glusterfs-fuse: 72321509: WRITE => -1 gfid=adb4cfda-78ed-4489-aac7-a79b56ff94f8 fd=0x7f27a8036890 (Read-only file system)
mnt_sv3.log:[2020-08-03 15:20:50.063432] W [fuse-bridge.c:2402:fuse_writev_cbk] 0-glusterfs-fuse: 72321514: WRITE => -1 gfid=adb4cfda-78ed-4489-aac7-a79b56ff94f8 fd=0x7f27a8036890 (Read-only file system)
mnt_sv3.log:[2020-08-03 15:20:50.063945] W [fuse-bridge.c:1381:fuse_err_cbk] 0-glusterfs-fuse: 72321515: FLUSH() ERR => -1 (Read-only file system)
mnt_sv3.log:[2020-08-03 15:20:51.598748] W [fuse-bridge.c:1445:fuse_unlink_cbk] 0-glusterfs-fuse: 72322464: UNLINK() /archives/videos_file/3e/3e/VKAnx_3e3ec1f32627b57c0d31d07746c8cd581670cdb7_o_240.mp4 => -1 (Read-only file system)
mnt_sv3.log:[2020-08-03 15:20:51.615692] W [fuse-bridge.c:779:fuse_truncate_cbk] 0-glusterfs-fuse: 72322467: TRUNCATE() /archives/videos_file/3e/3e/VKAnx_3e3ec1f32627b57c0d31d07746c8cd581670cdb7_o_240.mp4 => -1 (Read-only file system)
mnt_sv3.log:[2020-08-03 15:20:51.768261] W [fuse-bridge.c:1445:fuse_unlink_cbk] 0-glusterfs-fuse: 72322474: UNLINK() /archives/videos_file/3e/3e/VKAnx_3e3ec1f32627b57c0d31d07746c8cd581670cdb7_o_240.mp4 => -1 (Read-only file system)
Gerald Schneider
  • 23,274
  • 8
  • 57
  • 89
GKaveH
  • 31
  • 4

1 Answers1

-3

This is likely because the underlying block device that GlusterFS uses has encountered I/O errors and it is remounted as read-only filesystem (to prevent further corruption). I would first go to the machine that hosts bricks and do a dmesg there to see if kernel has report such errors.

netghost
  • 1
  • 4
  • 3
    yep already I figure it out highly it's I/O issue because with iostat I have avg-cpu above 5% with this `grep avg-cpu -A1 ~/iostat.txt | grep -v "avg-cpu" | grep -v "-" | awk '$4>5.0{print $4}' | wc -l` – GKaveH Aug 04 '20 at 19:29