Questions tagged [ceph]
Ceph is a free software storage platform designed to present object, block, and file storage from a single distributed computer cluster.
115
questions
2
votes
0answers
15 views
Ceph Rados GW Proxy
I've been given an access key to a Ceph cluster that runs radoswgw to provide S3. The key allows bucket creation, object reading etc. I don't admin the Ceph or radosgw.
I have a group of users that ...
0
votes
1answer
21 views
Does Kubeadm reset delete persistent volumes
I need to change the advertise address of my Kubernetes master node.
I can do this with a kubeadm reset, but I don't want to lose my (rook-ceph) persistent volumes on the master node.
Is it safe to do ...
0
votes
1answer
31 views
OpenStack redundant shared storage backend solution
Which storage backends are recommended for OpenStack which allows live migration and shared storage?
Is it NFS / Ceph / storage backend drivers to commercial storage solutions? Which of these is ...
0
votes
0answers
11 views
How to configure a fail safe ceph Journal?
I am assuming a redundant 2x 10GbE network with a realistic 2GB/s network bandwidth for this question (details here Ceph sizing for 2x 10GbE network with SSDs?)
Means if we can achieve that a bunch ...
0
votes
0answers
21 views
Ceph sizing for 2x 10GbE network with SSDs?
building a ceph cluster results into designing the network first
So I have some questions about the ceph cluster network sizing
Let's assume we build a redundant NIC with 2x 10bE NIC Ports per ceph ...
0
votes
1answer
37 views
Error running ceph-deploy install
I have been following the following tutorial to setup ceph on CentOS
https://wiki.centos.org/SpecialInterestGroup/Storage/ceph-Quickstart
When I try to run ceph-deploy to install to my other nodes, ...
-2
votes
1answer
41 views
Backup solution for medium infrastructure with commodity hardware
I'm looking for recommendations for a backup solution for our Infrastructure.
We have lots of old Hardware, which is not very fast but just laying around.
We have like 20 Servers for backup with 8-...
1
vote
1answer
318 views
Poor performance with rook, ceph and RBD
I have a k8s cluster on 4 VMs. 1 master and 3 workers. On each of the workers, I use rook to deploy a ceph OSD.
The OSDs are using the same disk as the VM Operating System.
The VM disks are remote (...
0
votes
0answers
30 views
CEPH on 10Gbit QNAP NAS [duplicate]
I am planing to setup a storage cluster for CEPH with a very limited budget. Fast networking connection is essential for a CEPH cluster. So the idea came to my mind to use a low budget NAS with fast ...
0
votes
1answer
684 views
Ceph hardware sizing calculator
I would like to calculate the Hardware sizing for a ceph cluster. There are so few references to the sizing, that I try to get these details here in the community.
E.g. what shall i have depending on
...
0
votes
0answers
34 views
Ceph with mainstream (desktop) SSDs?
I am searching for an answer about using SSDs drivers, which are build for the mainstream.
Afaik ceph uses for its marketing to buy cheap disks for entprise ready storages. But I cannot find any ...
0
votes
0answers
28 views
Virtual machines in proxmox did not work when proxmox was rebooted after power cut
Virtual machines in proxmox did not work when proxmox was rebooted after power cut
The Proxmox Machine was shut down suddenly (for example, when the electricity was cut off). Then this machine was ...
0
votes
0answers
28 views
ceph pg dump report zero PG_SUM for newly added osd
i'm currently using ceph mimic version. I added some osd to the cluster. but unfortunately they did not have any PG so they could not store any object. any idea?
1
vote
0answers
25 views
Ceph Clock Skew
I want to know where I can configure the limit of skew time for my ceph monitors. Also,how does CEPH throw this clock skew error? Specifically from which file and where can I find the file so that I ...
1
vote
0answers
34 views
CEPH netfilter connection tracking issues
We have a CEPH cluster (Ubuntu 18.04, Luminous) for Openstack images and volumes. As I was taking it into production I found many performance issues, slow OSDs, and throughput down to a trickle; this ...
0
votes
0answers
1k views
MountVolume.MountDevice failed operation with the given Volume ID already exists
Environment:
Kubernetes cluster with 1 master and 3 nodes Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-66-generic x86_64) (VMWARE VMs)
screenshot from dashboard
Pod (simple nginx image) cannot be mounted ...
0
votes
0answers
41 views
Change Maildir location for Postfix
I'm currently using Maildir format for postfix and dovecot, but I not wanted to use it in home/$user/Maildir, I wanted it to be in the shared file storage like cephFS.
My cephFS mounted folder is /...
2
votes
0answers
58 views
Setup ceph for storage mailserver?
I have read some article but none of them tell how to use ceph for mail server will be like.
The most close to it: https://www.7layer.org/?p=313, but it was glusterFS
So is there any different when ...
4
votes
1answer
832 views
High availability for ceph storage?
Do ceph have high availability, I config 2 node like this
cluster:
id: 07df97db-f315-4c78-9d2a-ab85007a1856
health: HEALTH_WARN
Reduced data availability: 32 pgs inactive
...
0
votes
1answer
27 views
How to benchmark small file (5kb) in ceph
I have read this article: https://tracker.ceph.com/projects/ceph/wiki/Benchmark_Ceph_Cluster_Performance
But the problems is they benchmark a 4MB file while i need like to benchmark 1000 5kb file
Is ...
0
votes
0answers
40 views
Ceph command exit after timeout
When I run ceph health --connect-timeout 10, it gives me a message about timeout after 10 seconds, as expected. But the process does not exit. It hangs there forever.
How can I make the command ...
1
vote
1answer
325 views
GlusterFS or Ceph RBD for storing Virtual Machine image
I am using glusterfs 5.3 for storing images of virtual machines in Cloudstack/KVM environment, majority of VMs are DB Servers (Sql Server & MariaDB). But I am facing performance issue on VMs ...
2
votes
0answers
305 views
Unable to resize Ceph RBD PVC in Kubernetes
So I have a 4 node (VMs) Kubernetes cluster spun up with Kubespray. I have a Ceph cluster set up from Proxmox, and a pool is available to k8s. I can make deployments using Ceph just fine.
But when ...
0
votes
1answer
356 views
Ceph installation experiences high swap usage
Currently I'm running a 8 server Ceph setup consisting out off 3 Ceph monitors and 5 Ceph nodes. Performance wise the cluster runs great but after time the nodes start swapping the ceph-osd process to ...
0
votes
0answers
34 views
Reassign old disk to new ceph cluster
we have HW problems with ceph cluster server, old ceph frontend is gone. Now we have new cluster and disks with old osd and data (rdb). How to reassign disk to new cluster without losing data?
Trying ...
2
votes
0answers
434 views
Proxmox Ceph OSD Partition Created With Only 10GB
How do you define the Ceph OSD Disk Partition Size?
It always creates with only 10 GB usable space.
Disk size = 3.9 TB
Partition size = 3.7 TB
Using ceph-disk prepare and ceph-disk activate (See ...
3
votes
1answer
238 views
What is wrong with my initial setup to install ceph?
I have downloaded ceph-ansible to install ceph for an evaluation and I ran on an error with package dependencies and I can't find anything about how to fix this using either ceph-ansible or yum.
The ...
0
votes
1answer
298 views
Compaction error: Corruption: block checksum mismatch: expected 862584094, got 1969278739
Currently trying to handle this right on my 3-day memorial weekend :D
Ceph 13.2.4 (Filestore)
Rook 0.9
Kubernetes 1.14.1
https://gist.github.com/sfxworks/ce77473a93b96570af319120e74535ec
My setup ...
2
votes
0answers
417 views
ceph shows wrong USED space in a single replicated pool
We are using ceph version 14.2.0. We have 4 hosts with 24 BlueStore OSDs, each is 1.8TB (2TB spinning disk).
We have only a single pool with size 2 and I am absolutely sure that we are using more ...
0
votes
1answer
307 views
CEPH for video streaming? [closed]
So i have a video streaming app (HLS) and was wondering if using CEPH or GlusterFS would be a good idea vs using plain SAN storage?
(from performance prospective)
In general is software defined the ...
0
votes
1answer
102 views
Indicating multiple clusters with ceph-ansible
I am working through a Ceph course right now (ceph learning path from packtpub). The course is ok, but has a lot of errors, and isn't always the most accurate. The course expects that you use ceph-...
0
votes
1answer
459 views
Is more memory than the required beneficial for Ceph BlueStore OSDs?
I have a cluster of servers, each of them having 128GB or RAM and 6 x 2TB spinning disks dedicated for BlueStore OSDs.
The servers also act like KVM hosts, so they are not dedicated to Ceph.
In the ...
3
votes
1answer
216 views
Libvirt pool cannot see or create rbd clones
While trying to get a cloned disk running from my OS snapshot I run into the problem that Libvirt cannot see existing images cloned from a snapshot. Created via:
$ rbd -p vmdisks clone vmdisks/...
1
vote
0answers
155 views
ceph-deploy osd create --data /dev/vdb fails
From my ceph (mimic release) administrative node I ran
ceph-deploy osd create --data /dev/vdb ceph0
against a bare metal ceph node and it worked without error.
Now I run the same command against a ...
1
vote
0answers
276 views
Hung kernel tasks after unclean shutdown of ceph cluster
I am running ceph (created by the rook-ceph operator v0.9.3) on kubernetes v1.13. After an unclean shutdown of our cluster, some processes randomly go into uninterruptible sleep. After some time, the ...
0
votes
1answer
390 views
Ceph - Erasure Coded Pools - Always get inactive pgs
I'm trying to achieve similar thing to raid6 on ceph.
But When I'm creating erasure coded pools (k=3 + m=2(or k=4) ) I always get inactive pgs.
ceph health details are:
HEALTH_WARN Reduced data ...
1
vote
1answer
1k views
Ceph iSCSI Gateway installation on Ubuntu 18.04
I'm installing Ceph using Ansible with ceph-ansible project, branch static-3.2. There is a problem with iSCSI Gateway installation.
If you are using iscsigws name in inventory file - it shows that ...
0
votes
0answers
16 views
How do I use a non-standard ceph system user?
I configured a ceph user on my cluster named "cepher." I ran ceph-deploy as this user to deploy some servers.
Then I see this:
[block][INFO ] Running command: sudo ceph --cluster ceph --name client....
1
vote
0answers
50 views
How wide (data + parity) would you go for erasure coded pools / raid 6?
I am currently doing k=8 and m=2
or in other words data = 8 + parity = 2 shards
I am using an all hdd ceph cluster.
I have thought about going data = 11 + parity = 2 which would save a lot of ...
0
votes
1answer
65 views
Ceph Erasure Profile with k+m> 6 disks -> PGs to be stuck on creating+incomplete state forever
So my pools has 5 nodes with 12 osds each of 8tb.
Currently I am trying to create an erasure coded pool of k=8 m=2, however after adding this profile and create an ecpool with this profile the pools ...
1
vote
2answers
2k views
How to add Hetzner Cloud Disk Volume to extend /var partition?
I have a new Hetzner cloud server and added an extra 50GB disk volume (CEPH) to my server. The default disk is only 40GB. I thought it would be added to the default storage like the traditional ...
2
votes
1answer
2k views
Ceph or Gluster for implementing big NAS
We are planning to build NAS solution which will be primarily used via NFS and CIFS and workloads ranging from various archival application to more āreal-time processingā. The NAS will not be used as ...
0
votes
1answer
71 views
mount cephfs or rbd via Internet
Is there any suitable solution for this?
I know about security issues, but is it possible?
Or other better storage solutions?
We tested s3fs+cephrgw, but It's very slow and unstable.
1
vote
1answer
291 views
Ceph architecture for small HPC cluster
I want to build Ceph Storage Cluster for HPC use. (CentOS 7 based)
For now I have enterprise SAS RAID enclosure with 3 shelves by 12 4TB disks(36 total).
Now it is configured as default RAID6 rig. And ...
0
votes
1answer
232 views
How to upgrade librbd1 and librados2 for oVirt 4.2.x (node ng)
oVirt 4.2 comes with librbd1 und librados2 from the Ceph Hammer release which is 0.94.5.
I need to update both libraries to the luminous version which is 12.x. because my ceph server is not able to ...
0
votes
1answer
471 views
Why does ovirt VM using ceph disk stay in “waiting for launch” status
My setup comprises of ceph mimic (centos 7, setup with ceph-ansible), cinder/keystone combo on pike release, and ovirt 4.2.5.1.
The external cinder provider is setup and I can create disks.
When ...
2
votes
1answer
204 views
Ceph status health ok, but has flag “nearfull”
Recently in Ceph there was a status of WARN because 3 disks were 85-87% full. I expanded the cluster by adding the server to the storage. But now I see flag "nearfull". Previously, this wasn't see.
...
1
vote
0answers
36 views
openstack instance's image name lost after integration with ceph rbd
after integrated with ceph rbd,my openstack could not show "image name" of my instance, is there any option that controlling this feature?
1
vote
0answers
48 views
Tried to recover a Ext4 Superblock to a CEPH disk
We have a server with Proxmox installed with Debian, it's an dedicated server (sda). The VM's are being backed up every night on a CEPH disk (sdb). After an reboot of the dedicated, the ext4 drive ...
7
votes
1answer
4k views
What is the difference between Ceph and NFS when it comes to shared file storage?
Can someone explain to me what is Ceph compared to NFS?
From a shared file storage perspective? What is the added value of using Ceph instead of NFS?
