Questions tagged [ceph]

Ceph is a free software storage platform designed to present object, block, and file storage from a single distributed computer cluster.

Filter by
Sorted by
Tagged with
0
votes
1answer
27 views

How can I extend or resize lvm from disk space

We use ceph to manage the storage and recently increased /dev/sdb's disk size. I'm not sure how to utilize the newly available space in my ceph's lvm. sdb ...
2
votes
0answers
33 views

CEPH/ RGW encryption with mutliple tenants

I'm currently planning a small OpenStack deployment and I want to use Ceph for object storage (via Rados Gateway) and block storage. Ceph supports multitenancy. It also supports encryption using key ...
1
vote
1answer
250 views

bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (1) Operation not permitted

When I have upgrade the from Octopus 15.2.10 to Pacific 16.2.0, the mon nodes starts successfully using the manual upgrade process (by installing the packages with no orch) however when I have upgrade ...
0
votes
0answers
21 views

Ceph mon ms_dispatch segmentation fault

I have faced the following error message as I try to start the monitoring node (there is only 1 monitoring node) any help is much appreciated. I am running Ceph on Ubuntu 20.04 with latest update
2
votes
1answer
93 views

Orchestrator is not available with fresh Rook instance

I'm trying to set up a rook Ceph cluster on my kubernetes cluster. Topography: 3 kubernetes nodes (all are master/worker pods) Each node has /dev/vdX on it for ceph Each node is intended to work as ...
0
votes
1answer
100 views

Multi site ceph + proxmox

We have 2 proxmox clusters in production. They contain Ceph cluster with cephfs is on it. The task is to sync files between sites. Site A is master and site B is for backup. On site A cephfs files are ...
0
votes
1answer
46 views

How to know how many files can Ceph delete per 1 minute?

As the question says. Is there a way to find out how many files can Ceph delete per 1 minute? A little backstory: Image hosting website, running Ceph FS on 8TB WD RED 3.5 HDD's, around 51 TiB already ...
0
votes
0answers
136 views

Cinder volume service goes down while creating/deleting multiple volumes at the same time

Environment: Openstack Rocky Ceph Mimic openstack-cinder-13.0.8-0 Cinder volume status goes down while creating or deleting multiple volumes at the same time manually or while Rally test is running. ...
0
votes
1answer
351 views

Ceph Cluster - Reduced data availability: 96 pgs inactive And All OSD Nodes Are Down

I set up my Ceph Cluster by following this document. I have one Manager Node, one Monitor Node, and three OSD Nodes. The problem is that right after I finished setting up the cluster, the ceph health ...
0
votes
0answers
131 views

Create a fio job with multiple read and writes

I am trying to create a fio job that: will write 10% of size with verify=crc32c (to save the hash for next steps). how to set the location where the hash files are stored? After step one, there ...
0
votes
1answer
83 views

Pipe rbd export comand to tar to create compressed export

Is it possible to pipe the output of rbd export directly to something else like tar to create a compressed archive of the .img file without having to write to disk first? Something like rbd export ...
1
vote
1answer
181 views

Ceph: is setting lower “size” parameter on a live pool possible?

I have a 5-node Proxmox cluster using Ceph as the primary VM storage backend. The Ceph pool is currently configured with a size of 5 (1 data replica per OSD per node) and a min_size of 1. Due to the ...
0
votes
1answer
53 views

Openstack Using Ceph Erasure Code Pool

I tried configuring openstack with ceph pool erasure code backend I'm trying to using —data-pool on ceph.conf using rbd default data pool like the following blog https://themeanti.me/technology/2018/...
1
vote
1answer
61 views

Recommended values for mon osd down out in ceph

Are there any recommended values for "mon osd down out"? The cluster has 15 nodes and 300 osds. It's running ceph Jewl. The default value in Jewel is 300 seconds. I was wondering, are there ...
0
votes
1answer
401 views

Proxmox Ceph - Got timeout on separate network

I've installed on 4 nodes a completly fresh OS with Proxmox. Every node has 2xNVMe und 1xHD, one NIC public, one NIC private. On the public network there is an additional wireguard interface running ...
1
vote
0answers
110 views

how to sync ceph bucket with scaleway s3 bucket or any s3 bucket for?

i,m trying to sync a ceph cluster to a scaleway s3 bucket i have the secret, access token and https endpoint for my s3 bucket ceph version: mimic i followed the cloud sync module guide at the ...
1
vote
0answers
21 views

Ceph-ansible: using the OS-SSD as cache

In ceph-ansible, you can specify a disk to be used for databases in the hosts file using the dedicated_devices directory. Example: hosts: stor01.hostname.com devices: ['/dev/sdd', '/dev/...
0
votes
0answers
113 views

ensure cinder-api is available timeout

I can't install a new openstack-ansible (with ceph-ansible) cloud. The "setup-openstack" playbook gets an timeout at the task: os_cinder : Ensure cinder api is available timeout. It doesn't ...
1
vote
1answer
1k views

Ceph e5 handle_auth_request failed to assign global_id after a host outage

I have a small 3-host Ceph cluster with Ubuntu 20.04.1 and Ceph 15.2.5 using docker containers and deployed with cephadm. Yesterday one of the hosts (s65-ceph) had a power outage. The other two ...
0
votes
0answers
54 views

cephfs path on samba cluster not found

I have a ceph cluster running a cephfs. I also have a samba cluster (ctdb) and want to share different folders (lets say projects and plm) on the cephfs. But I can only share the root directory of the ...
0
votes
1answer
69 views

ceph-deploy install --release fails to upgrade on debian stretch

this is my first post on serverfault! I'm trying to setup a test ceph cluster to see if it can meet our storage needs. I've been following the instructions here I ran into some permissions issues when ...
0
votes
3answers
390 views

Data resiliency in Ceph or Gluster

Say I am most concerned with keeping my data safe from physical hard drive failure and the computer it connects to directly. RAID1 and RAID5 I understand. They provide protections against data loss ...
0
votes
1answer
104 views

Memory level for Ceph logs. What's this for?

I want to increase the logging level of some OSDs, but I don't understand what the memory level is for. I didn't find an exact explanation in the documentation. Is there a need to increase this level? ...
0
votes
0answers
814 views

CEPH cluster became unresponsive: e5 handle_auth_request failed to assign global_id

I've created an Octopus 15.2.4 cluster with 3 monitors and 3 OSDs (6 hosts in total, all ESXi VMs). It lived through a couple of reboots without problem, then I've reconfigured the main host a bit: ...
1
vote
1answer
51 views

Can my Ceph nodes have the same hostname?

I have a high availability solution which requires me to have all my nodes using the same hostname. I've been tasked to use ceph cluster storage, but I just reached a dead end. I can't figure out how ...
3
votes
2answers
505 views

How to best use large NVMe array for VMWare Datastore

I recently acquired a Supermicro SYS-1029P-N32R with 16 14TB NVMe SSD's (Micron 9300 Pro) at work. We have 100 gigabit networking on the box to our VMWare hosts. We've tried using FreeNAS to host ZFS ...
3
votes
0answers
50 views

Ceph Rados GW Proxy

I've been given an access key to a Ceph cluster that runs radoswgw to provide S3. The key allows bucket creation, object reading etc. I don't admin the Ceph or radosgw. I have a group of users that ...
0
votes
1answer
206 views

Does Kubeadm reset delete persistent volumes

I need to change the advertise address of my Kubernetes master node. I can do this with a kubeadm reset, but I don't want to lose my (rook-ceph) persistent volumes on the master node. Is it safe to do ...
0
votes
1answer
47 views

OpenStack redundant shared storage backend solution

Which storage backends are recommended for OpenStack which allows live migration and shared storage? Is it NFS / Ceph / storage backend drivers to commercial storage solutions? Which of these is ...
0
votes
1answer
215 views

Error running ceph-deploy install

I have been following the following tutorial to setup ceph on CentOS https://wiki.centos.org/SpecialInterestGroup/Storage/ceph-Quickstart When I try to run ceph-deploy to install to my other nodes, ...
-2
votes
1answer
57 views

Backup solution for medium infrastructure with commodity hardware

I'm looking for recommendations for a backup solution for our Infrastructure. We have lots of old Hardware, which is not very fast but just laying around. We have like 20 Servers for backup with 8-...
1
vote
1answer
1k views

Poor performance with rook, ceph and RBD

I have a k8s cluster on 4 VMs. 1 master and 3 workers. On each of the workers, I use rook to deploy a ceph OSD. The OSDs are using the same disk as the VM Operating System. The VM disks are remote (...
0
votes
1answer
4k views

Ceph hardware sizing calculator

I would like to calculate the Hardware sizing for a ceph cluster. There are so few references to the sizing, that I try to get these details here in the community. E.g. what shall i have depending on ...
1
vote
1answer
105 views

Ceph with mainstream (desktop) SSDs?

I am searching for an answer about using SSDs drivers, which are build for the mainstream. Afaik ceph uses for its marketing to buy cheap disks for entprise ready storages. But I cannot find any ...
1
vote
0answers
48 views

Ceph Clock Skew

I want to know where I can configure the limit of skew time for my ceph monitors. Also,how does CEPH throw this clock skew error? Specifically from which file and where can I find the file so that I ...
1
vote
0answers
52 views

CEPH netfilter connection tracking issues

We have a CEPH cluster (Ubuntu 18.04, Luminous) for Openstack images and volumes. As I was taking it into production I found many performance issues, slow OSDs, and throughput down to a trickle; this ...
1
vote
0answers
3k views

MountVolume.MountDevice failed operation with the given Volume ID already exists

Environment: Kubernetes cluster with 1 master and 3 nodes Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-66-generic x86_64) (VMWARE VMs) screenshot from dashboard Pod (simple nginx image) cannot be mounted ...
2
votes
1answer
188 views

Setup ceph for storage mailserver?

I have read some article but none of them tell how to use ceph for mail server will be like. The most close to it: https://www.7layer.org/?p=313, but it was glusterFS So is there any different when ...
4
votes
1answer
2k views

High availability for ceph storage?

Do ceph have high availability, I config 2 node like this cluster: id: 07df97db-f315-4c78-9d2a-ab85007a1856 health: HEALTH_WARN Reduced data availability: 32 pgs inactive ...
0
votes
1answer
42 views

How to benchmark small file (5kb) in ceph

I have read this article: https://tracker.ceph.com/projects/ceph/wiki/Benchmark_Ceph_Cluster_Performance But the problems is they benchmark a 4MB file while i need like to benchmark 1000 5kb file Is ...
1
vote
2answers
660 views

GlusterFS or Ceph RBD for storing Virtual Machine image

I am using glusterfs 5.3 for storing images of virtual machines in Cloudstack/KVM environment, majority of VMs are DB Servers (Sql Server & MariaDB). But I am facing performance issue on VMs ...
2
votes
0answers
466 views

Unable to resize Ceph RBD PVC in Kubernetes

So I have a 4 node (VMs) Kubernetes cluster spun up with Kubespray. I have a Ceph cluster set up from Proxmox, and a pool is available to k8s. I can make deployments using Ceph just fine. But when ...
0
votes
1answer
678 views

Ceph installation experiences high swap usage

Currently I'm running a 8 server Ceph setup consisting out off 3 Ceph monitors and 5 Ceph nodes. Performance wise the cluster runs great but after time the nodes start swapping the ceph-osd process to ...
2
votes
1answer
688 views

Proxmox Ceph OSD Partition Created With Only 10GB

How do you define the Ceph OSD Disk Partition Size? It always creates with only 10 GB usable space. Disk size = 3.9 TB Partition size = 3.7 TB Using ceph-disk prepare and ceph-disk activate (See ...
3
votes
1answer
322 views

What is wrong with my initial setup to install ceph?

I have downloaded ceph-ansible to install ceph for an evaluation and I ran on an error with package dependencies and I can't find anything about how to fix this using either ceph-ansible or yum. The ...
0
votes
1answer
590 views

Compaction error: Corruption: block checksum mismatch: expected 862584094, got 1969278739

Currently trying to handle this right on my 3-day memorial weekend :D Ceph 13.2.4 (Filestore) Rook 0.9 Kubernetes 1.14.1 https://gist.github.com/sfxworks/ce77473a93b96570af319120e74535ec My setup ...
2
votes
0answers
617 views

ceph shows wrong USED space in a single replicated pool

We are using ceph version 14.2.0. We have 4 hosts with 24 BlueStore OSDs, each is 1.8TB (2TB spinning disk). We have only a single pool with size 2 and I am absolutely sure that we are using more ...
0
votes
1answer
489 views

CEPH for video streaming? [closed]

So i have a video streaming app (HLS) and was wondering if using CEPH or GlusterFS would be a good idea vs using plain SAN storage? (from performance prospective) In general is software defined the ...
0
votes
1answer
165 views

Indicating multiple clusters with ceph-ansible

I am working through a Ceph course right now (ceph learning path from packtpub). The course is ok, but has a lot of errors, and isn't always the most accurate. The course expects that you use ceph-...
0
votes
1answer
850 views

Is more memory than the required beneficial for Ceph BlueStore OSDs?

I have a cluster of servers, each of them having 128GB or RAM and 6 x 2TB spinning disks dedicated for BlueStore OSDs. The servers also act like KVM hosts, so they are not dedicated to Ceph. In the ...