Questions tagged [ceph]

Ceph is a free software storage platform designed to present object, block, and file storage from a single distributed computer cluster.

Filter by
Sorted by
Tagged with
2
votes
0answers
15 views

Ceph Rados GW Proxy

I've been given an access key to a Ceph cluster that runs radoswgw to provide S3. The key allows bucket creation, object reading etc. I don't admin the Ceph or radosgw. I have a group of users that ...
0
votes
1answer
21 views

Does Kubeadm reset delete persistent volumes

I need to change the advertise address of my Kubernetes master node. I can do this with a kubeadm reset, but I don't want to lose my (rook-ceph) persistent volumes on the master node. Is it safe to do ...
0
votes
1answer
31 views

OpenStack redundant shared storage backend solution

Which storage backends are recommended for OpenStack which allows live migration and shared storage? Is it NFS / Ceph / storage backend drivers to commercial storage solutions? Which of these is ...
0
votes
0answers
11 views

How to configure a fail safe ceph Journal?

I am assuming a redundant 2x 10GbE network with a realistic 2GB/s network bandwidth for this question (details here Ceph sizing for 2x 10GbE network with SSDs?) Means if we can achieve that a bunch ...
0
votes
0answers
21 views

Ceph sizing for 2x 10GbE network with SSDs?

building a ceph cluster results into designing the network first So I have some questions about the ceph cluster network sizing Let's assume we build a redundant NIC with 2x 10bE NIC Ports per ceph ...
0
votes
1answer
37 views

Error running ceph-deploy install

I have been following the following tutorial to setup ceph on CentOS https://wiki.centos.org/SpecialInterestGroup/Storage/ceph-Quickstart When I try to run ceph-deploy to install to my other nodes, ...
-2
votes
1answer
41 views

Backup solution for medium infrastructure with commodity hardware

I'm looking for recommendations for a backup solution for our Infrastructure. We have lots of old Hardware, which is not very fast but just laying around. We have like 20 Servers for backup with 8-...
1
vote
1answer
318 views

Poor performance with rook, ceph and RBD

I have a k8s cluster on 4 VMs. 1 master and 3 workers. On each of the workers, I use rook to deploy a ceph OSD. The OSDs are using the same disk as the VM Operating System. The VM disks are remote (...
0
votes
0answers
30 views

CEPH on 10Gbit QNAP NAS [duplicate]

I am planing to setup a storage cluster for CEPH with a very limited budget. Fast networking connection is essential for a CEPH cluster. So the idea came to my mind to use a low budget NAS with fast ...
0
votes
1answer
684 views

Ceph hardware sizing calculator

I would like to calculate the Hardware sizing for a ceph cluster. There are so few references to the sizing, that I try to get these details here in the community. E.g. what shall i have depending on ...
0
votes
0answers
34 views

Ceph with mainstream (desktop) SSDs?

I am searching for an answer about using SSDs drivers, which are build for the mainstream. Afaik ceph uses for its marketing to buy cheap disks for entprise ready storages. But I cannot find any ...
0
votes
0answers
28 views

Virtual machines in proxmox did not work when proxmox was rebooted after power cut

Virtual machines in proxmox did not work when proxmox was rebooted after power cut The Proxmox Machine was shut down suddenly (for example, when the electricity was cut off). Then this machine was ...
0
votes
0answers
28 views

ceph pg dump report zero PG_SUM for newly added osd

i'm currently using ceph mimic version. I added some osd to the cluster. but unfortunately they did not have any PG so they could not store any object. any idea?
1
vote
0answers
25 views

Ceph Clock Skew

I want to know where I can configure the limit of skew time for my ceph monitors. Also,how does CEPH throw this clock skew error? Specifically from which file and where can I find the file so that I ...
1
vote
0answers
34 views

CEPH netfilter connection tracking issues

We have a CEPH cluster (Ubuntu 18.04, Luminous) for Openstack images and volumes. As I was taking it into production I found many performance issues, slow OSDs, and throughput down to a trickle; this ...
0
votes
0answers
1k views

MountVolume.MountDevice failed operation with the given Volume ID already exists

Environment: Kubernetes cluster with 1 master and 3 nodes Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-66-generic x86_64) (VMWARE VMs) screenshot from dashboard Pod (simple nginx image) cannot be mounted ...
0
votes
0answers
41 views

Change Maildir location for Postfix

I'm currently using Maildir format for postfix and dovecot, but I not wanted to use it in home/$user/Maildir, I wanted it to be in the shared file storage like cephFS. My cephFS mounted folder is /...
2
votes
0answers
58 views

Setup ceph for storage mailserver?

I have read some article but none of them tell how to use ceph for mail server will be like. The most close to it: https://www.7layer.org/?p=313, but it was glusterFS So is there any different when ...
4
votes
1answer
832 views

High availability for ceph storage?

Do ceph have high availability, I config 2 node like this cluster: id: 07df97db-f315-4c78-9d2a-ab85007a1856 health: HEALTH_WARN Reduced data availability: 32 pgs inactive ...
0
votes
1answer
27 views

How to benchmark small file (5kb) in ceph

I have read this article: https://tracker.ceph.com/projects/ceph/wiki/Benchmark_Ceph_Cluster_Performance But the problems is they benchmark a 4MB file while i need like to benchmark 1000 5kb file Is ...
0
votes
0answers
40 views

Ceph command exit after timeout

When I run ceph health --connect-timeout 10, it gives me a message about timeout after 10 seconds, as expected. But the process does not exit. It hangs there forever. How can I make the command ...
1
vote
1answer
325 views

GlusterFS or Ceph RBD for storing Virtual Machine image

I am using glusterfs 5.3 for storing images of virtual machines in Cloudstack/KVM environment, majority of VMs are DB Servers (Sql Server & MariaDB). But I am facing performance issue on VMs ...
2
votes
0answers
305 views

Unable to resize Ceph RBD PVC in Kubernetes

So I have a 4 node (VMs) Kubernetes cluster spun up with Kubespray. I have a Ceph cluster set up from Proxmox, and a pool is available to k8s. I can make deployments using Ceph just fine. But when ...
0
votes
1answer
356 views

Ceph installation experiences high swap usage

Currently I'm running a 8 server Ceph setup consisting out off 3 Ceph monitors and 5 Ceph nodes. Performance wise the cluster runs great but after time the nodes start swapping the ceph-osd process to ...
0
votes
0answers
34 views

Reassign old disk to new ceph cluster

we have HW problems with ceph cluster server, old ceph frontend is gone. Now we have new cluster and disks with old osd and data (rdb). How to reassign disk to new cluster without losing data? Trying ...
2
votes
0answers
434 views

Proxmox Ceph OSD Partition Created With Only 10GB

How do you define the Ceph OSD Disk Partition Size? It always creates with only 10 GB usable space. Disk size = 3.9 TB Partition size = 3.7 TB Using ceph-disk prepare and ceph-disk activate (See ...
3
votes
1answer
238 views

What is wrong with my initial setup to install ceph?

I have downloaded ceph-ansible to install ceph for an evaluation and I ran on an error with package dependencies and I can't find anything about how to fix this using either ceph-ansible or yum. The ...
0
votes
1answer
298 views

Compaction error: Corruption: block checksum mismatch: expected 862584094, got 1969278739

Currently trying to handle this right on my 3-day memorial weekend :D Ceph 13.2.4 (Filestore) Rook 0.9 Kubernetes 1.14.1 https://gist.github.com/sfxworks/ce77473a93b96570af319120e74535ec My setup ...
2
votes
0answers
417 views

ceph shows wrong USED space in a single replicated pool

We are using ceph version 14.2.0. We have 4 hosts with 24 BlueStore OSDs, each is 1.8TB (2TB spinning disk). We have only a single pool with size 2 and I am absolutely sure that we are using more ...
0
votes
1answer
307 views

CEPH for video streaming? [closed]

So i have a video streaming app (HLS) and was wondering if using CEPH or GlusterFS would be a good idea vs using plain SAN storage? (from performance prospective) In general is software defined the ...
0
votes
1answer
102 views

Indicating multiple clusters with ceph-ansible

I am working through a Ceph course right now (ceph learning path from packtpub). The course is ok, but has a lot of errors, and isn't always the most accurate. The course expects that you use ceph-...
0
votes
1answer
459 views

Is more memory than the required beneficial for Ceph BlueStore OSDs?

I have a cluster of servers, each of them having 128GB or RAM and 6 x 2TB spinning disks dedicated for BlueStore OSDs. The servers also act like KVM hosts, so they are not dedicated to Ceph. In the ...
3
votes
1answer
216 views

Libvirt pool cannot see or create rbd clones

While trying to get a cloned disk running from my OS snapshot I run into the problem that Libvirt cannot see existing images cloned from a snapshot. Created via: $ rbd -p vmdisks clone vmdisks/...
1
vote
0answers
155 views

ceph-deploy osd create --data /dev/vdb fails

From my ceph (mimic release) administrative node I ran ceph-deploy osd create --data /dev/vdb ceph0 against a bare metal ceph node and it worked without error. Now I run the same command against a ...
1
vote
0answers
276 views

Hung kernel tasks after unclean shutdown of ceph cluster

I am running ceph (created by the rook-ceph operator v0.9.3) on kubernetes v1.13. After an unclean shutdown of our cluster, some processes randomly go into uninterruptible sleep. After some time, the ...
0
votes
1answer
390 views

Ceph - Erasure Coded Pools - Always get inactive pgs

I'm trying to achieve similar thing to raid6 on ceph. But When I'm creating erasure coded pools (k=3 + m=2(or k=4) ) I always get inactive pgs. ceph health details are: HEALTH_WARN Reduced data ...
1
vote
1answer
1k views

Ceph iSCSI Gateway installation on Ubuntu 18.04

I'm installing Ceph using Ansible with ceph-ansible project, branch static-3.2. There is a problem with iSCSI Gateway installation. If you are using iscsigws name in inventory file - it shows that ...
0
votes
0answers
16 views

How do I use a non-standard ceph system user?

I configured a ceph user on my cluster named "cepher." I ran ceph-deploy as this user to deploy some servers. Then I see this: [block][INFO ] Running command: sudo ceph --cluster ceph --name client....
1
vote
0answers
50 views

How wide (data + parity) would you go for erasure coded pools / raid 6?

I am currently doing k=8 and m=2 or in other words data = 8 + parity = 2 shards I am using an all hdd ceph cluster. I have thought about going data = 11 + parity = 2 which would save a lot of ...
0
votes
1answer
65 views

Ceph Erasure Profile with k+m> 6 disks -> PGs to be stuck on creating+incomplete state forever

So my pools has 5 nodes with 12 osds each of 8tb. Currently I am trying to create an erasure coded pool of k=8 m=2, however after adding this profile and create an ecpool with this profile the pools ...
1
vote
2answers
2k views

How to add Hetzner Cloud Disk Volume to extend /var partition?

I have a new Hetzner cloud server and added an extra 50GB disk volume (CEPH) to my server. The default disk is only 40GB. I thought it would be added to the default storage like the traditional ...
2
votes
1answer
2k views

Ceph or Gluster for implementing big NAS

We are planning to build NAS solution which will be primarily used via NFS and CIFS and workloads ranging from various archival application to more “real-time processing”. The NAS will not be used as ...
0
votes
1answer
71 views

mount cephfs or rbd via Internet

Is there any suitable solution for this? I know about security issues, but is it possible? Or other better storage solutions? We tested s3fs+cephrgw, but It's very slow and unstable.
1
vote
1answer
291 views

Ceph architecture for small HPC cluster

I want to build Ceph Storage Cluster for HPC use. (CentOS 7 based) For now I have enterprise SAS RAID enclosure with 3 shelves by 12 4TB disks(36 total). Now it is configured as default RAID6 rig. And ...
0
votes
1answer
232 views

How to upgrade librbd1 and librados2 for oVirt 4.2.x (node ng)

oVirt 4.2 comes with librbd1 und librados2 from the Ceph Hammer release which is 0.94.5. I need to update both libraries to the luminous version which is 12.x. because my ceph server is not able to ...
0
votes
1answer
471 views

Why does ovirt VM using ceph disk stay in “waiting for launch” status

My setup comprises of ceph mimic (centos 7, setup with ceph-ansible), cinder/keystone combo on pike release, and ovirt 4.2.5.1. The external cinder provider is setup and I can create disks. When ...
2
votes
1answer
204 views

Ceph status health ok, but has flag “nearfull”

Recently in Ceph there was a status of WARN because 3 disks were 85-87% full. I expanded the cluster by adding the server to the storage. But now I see flag "nearfull". Previously, this wasn't see. ...
1
vote
0answers
36 views

openstack instance's image name lost after integration with ceph rbd

after integrated with ceph rbd,my openstack could not show "image name" of my instance, is there any option that controlling this feature?
1
vote
0answers
48 views

Tried to recover a Ext4 Superblock to a CEPH disk

We have a server with Proxmox installed with Debian, it's an dedicated server (sda). The VM's are being backed up every night on a CEPH disk (sdb). After an reboot of the dedicated, the ext4 drive ...
7
votes
1answer
4k views

What is the difference between Ceph and NFS when it comes to shared file storage?

Can someone explain to me what is Ceph compared to NFS? From a shared file storage perspective? What is the added value of using Ceph instead of NFS?