Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [ceph]

Ceph is a free software storage platform designed to present object, block, and file storage from a single distributed computer cluster.

Ceph is a free software storage platform designed to present object, block, and file storage from a single distributed computer cluster. Ceph's main goals are to be completely distributed without a single point of failure, scalable to the exabyte level, and freely-available.

179 questions
1
vote
0 answers

Computing stripe count for erasure coded storage

I'm setting up a ceph cluster (first time for me) which in the end will be made of ~100 disks spread over 10 hosts. I'm going with a single erasure coded data pool to maximize disk space; my constraints are ~80% efficiency and a fault tolerance of 2…
Nicola Mori
  • 281
  • 1
  • 7
1
vote
0 answers

Ceph RGW: slow `list_bucket` requests

I have a ceph-rgw installation with a large bucket (~60M objects) and 16 osds, the bucket index is sharded into 997 shards. In this environment single directory listing takes more than 30 seconds: $ time rclone lsd t:bucket/non/existent/path/…
Taha Jahangir
  • 2,122
  • 1
  • 15
  • 16
1
vote
1 answer

Ceph: is setting lower "size" parameter on a live pool possible?

I have a 5-node Proxmox cluster using Ceph as the primary VM storage backend. The Ceph pool is currently configured with a size of 5 (1 data replica per OSD per node) and a min_size of 1. Due to the high size setting, much of the available space in…
Matthew U.
  • 13
  • 4
1
vote
1 answer

Recommended values for mon osd down out in ceph

Are there any recommended values for "mon osd down out"? The cluster has 15 nodes and 300 osds. It's running ceph Jewl. The default value in Jewel is 300 seconds. I was wondering, are there any benefits or downsides to changing the default value?
Alex Elshamouty
  • 41
  • 2
1
vote
2 answers

Proxmox Ceph - Got timeout on separate network

I've installed on 4 nodes a completly fresh OS with Proxmox. Every node has 2xNVMe und 1xHD, one NIC public, one NIC private. On the public network there is an additional wireguard interface running for PVE cluster communication. The private…
TRW
  • 488
  • 3
  • 16
1
vote
0 answers

how to sync ceph bucket with scaleway s3 bucket or any s3 bucket for?

i,m trying to sync a ceph cluster to a scaleway s3 bucket i have the secret, access token and https endpoint for my s3 bucket ceph version: mimic i followed the cloud sync module guide at the following link…
moses
  • 83
  • 1
  • 12
1
vote
0 answers

Ceph-ansible: using the OS-SSD as cache

In ceph-ansible, you can specify a disk to be used for databases in the hosts file using the dedicated_devices directory. Example: hosts: stor01.hostname.com devices: ['/dev/sdd', '/dev/sde', '/dev/sdf'] dedicated_devices:…
aphid
  • 139
  • 8
1
vote
0 answers

CEPH cluster became unresponsive: e5 handle_auth_request failed to assign global_id

I've created an Octopus 15.2.4 cluster with 3 monitors and 3 OSDs (6 hosts in total, all ESXi VMs). It lived through a couple of reboots without problem, then I've reconfigured the main host a bit: set iptables-legacy as current option in…
XNRL
  • 65
  • 8
1
vote
1 answer

Can my Ceph nodes have the same hostname?

I have a high availability solution which requires me to have all my nodes using the same hostname. I've been tasked to use ceph cluster storage, but I just reached a dead end. I can't figure out how to configure ceph so that I can use the same…
Malazzar
  • 21
  • 3
1
vote
1 answer

Poor performance with rook, ceph and RBD

I have a k8s cluster on 4 VMs. 1 master and 3 workers. On each of the workers, I use rook to deploy a ceph OSD. The OSDs are using the same disk as the VM Operating System. The VM disks are remote (the underlaying infrastructure is again a Ceph…
Laurentiu Soica
  • 75
  • 1
  • 6
1
vote
1 answer

Ceph with mainstream (desktop) SSDs?

I am searching for an answer about using SSDs drivers, which are build for the mainstream. Afaik ceph uses for its marketing to buy cheap disks for entprise ready storages. But I cannot find any details or tests about this. E.g. if you take a look…
cilap
  • 277
  • 5
  • 14
0
votes
1 answer

How to benchmark small file (5kb) in ceph

I have read this article: https://tracker.ceph.com/projects/ceph/wiki/Benchmark_Ceph_Cluster_Performance But the problems is they benchmark a 4MB file while i need like to benchmark 1000 5kb file Is there anyway to do it, like the dd if=/dev/null…
Lê Minh Quân
  • 301
  • 2
  • 13
0
votes
1 answer

Ceph installation experiences high swap usage

Currently I'm running a 8 server Ceph setup consisting out off 3 Ceph monitors and 5 Ceph nodes. Performance wise the cluster runs great but after time the nodes start swapping the ceph-osd process to disk. When this happens I experience very pore…
bk207
  • 173
  • 1
  • 2
  • 9
0
votes
1 answer

Compaction error: Corruption: block checksum mismatch: expected 862584094, got 1969278739

Currently trying to handle this right on my 3-day memorial weekend :D Ceph 13.2.4 (Filestore) Rook 0.9 Kubernetes 1.14.1 https://gist.github.com/sfxworks/ce77473a93b96570af319120e74535ec My setup is a Kubernetes cluser with rook handling Ceph.…
sfxworks
  • 157
  • 1
  • 8
0
votes
1 answer

CEPH for video streaming?

So i have a video streaming app (HLS) and was wondering if using CEPH or GlusterFS would be a good idea vs using plain SAN storage? (from performance prospective) In general is software defined the silver bullet for storage (can it be used…
SHM
  • 109
  • 4
1 2 3
…
11 12