Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [ceph]

Ceph is a free software storage platform designed to present object, block, and file storage from a single distributed computer cluster.

0
votes
0answers
10 views

How do I use a non-standard ceph system user?

I configured a ceph user on my cluster named "cepher." I ran ceph-deploy as this user to deploy some servers. Then I see this: [block][INFO ] Running command: sudo ceph --cluster ceph --name client....
0
votes
2answers
1k views

Ceph PG's stuck creating / Pool creation slow

I've created a small ceph cluster 3 servers each with 5 disks for osd's with one monitor per server. The actual setup seems to have gone OK and the mons are in quorum and all 15 osd's are up and in ...
0
votes
1answer
869 views

install grub in a usb and boot proxmox from another drive

Good morning my friends! I'm tryng a solution before buy some new hardware. So this is my situation: I have some beautifull hp dl360p gen8 that comes with an p420i hardware raid controller. What ...
-1
votes
1answer
83 views

How to add Hetzner Cloud Disk Volume to extend /var partition?

I have a new Hetzner cloud server and added an extra 50GB disk volume (CEPH) to my server. The default disk is only 40GB. I thought it would be added to the default storage like the traditional ...
1
vote
1answer
1k views

Set CRUSHMAP for 3 way on a two node cluster

I have an SSD storage system that contains two nodes with 6 SSD's drives. Not ideal, and so some point I will introduce another node. Right now I'm wanting 3 way replication though. Under a default ...
0
votes
0answers
12 views

Ceph replicated file permissions

I got a problem with ceph file permissions after replication. For ex. i copied files with permissions 755, but after it got replicated to other ceph osd it got 700, so how to fix it or is there any ...
0
votes
2answers
1k views

how to use ceph in a backup environment

I want to pool a large number of hard drives over the network to provide a large backup space. I want also to partition this space and make it available to several machines as a backup space. I ...
1
vote
0answers
16 views

How wide (data + parity) would you go for erasure coded pools / raid 6?

I am currently doing k=8 and m=2 or in other words data = 8 + parity = 2 shards I am using an all hdd ceph cluster. I have thought about going data = 11 + parity = 2 which would save a lot of ...
0
votes
1answer
14 views

Ceph Erasure Profile with k+m> 6 disks -> PGs to be stuck on creating+incomplete state forever

So my pools has 5 nodes with 12 osds each of 8tb. Currently I am trying to create an erasure coded pool of k=8 m=2, however after adding this profile and create an ecpool with this profile the pools ...
0
votes
1answer
50 views

mount cephfs or rbd via Internet

Is there any suitable solution for this? I know about security issues, but is it possible? Or other better storage solutions? We tested s3fs+cephrgw, but It's very slow and unstable.
0
votes
0answers
56 views

How can I mount cephfs on proxmox 5.3?

I have added the pxmx-cephfs.secret (the admin keyring) to /etc/pve/priv/ceph/. Then I proceeded to add cephfs through gui with id pxmx-cephfs and the monitors. While it is added successfully ...
0
votes
2answers
3k views

Is Ceph too slow and how to optimize it?

The setup is 3 clustered Proxmox for computations, 3 clustered Ceph storage nodes, ceph01 8*150GB ssds (1 used for OS, 7 for storage) ceph02 8*150GB ssds (1 used for OS, 7 for storage) ceph03 8*250GB ...
3
votes
1answer
479 views

CTDB Samba failover not highly available

TL;DR Failing a node in a ctdb + samba cluster while interacting with a share interrupts share connection. here and here state there is work in progress to make this possible here states it is ...
0
votes
2answers
731 views

RAID-0 stripe size for Ceph OSD

I have 3 servers that I will use for a new Ceph cluster. It's my first Ceph "playground"... Each server has 2x1TB and 6x2TB HDDs connected to two separate 4-channel SAS controllers, each with 1GB ...
0
votes
0answers
107 views

Ceph vs GlusterFS for elasticsearch storage

Elasticsearch will be holding logs mostly, so we expect sequential read/writes. Does anyone have experience on this? Elasticsearch will run via openshift, if it makes any difference.
1
vote
1answer
45 views

Ceph architecture for small HPC cluster

I want to build Ceph Storage Cluster for HPC use. (CentOS 7 based) For now I have enterprise SAS RAID enclosure with 3 shelves by 12 4TB disks(36 total). Now it is configured as default RAID6 rig. And ...
1
vote
1answer
436 views

Ceph or Gluster for implementing big NAS

We are planning to build NAS solution which will be primarily used via NFS and CIFS and workloads ranging from various archival application to more “real-time processing”. The NAS will not be used as ...
0
votes
0answers
27 views

Is it safe to use CephFS on Bluestore with erasure coding?

Is it safe to use CephFS on Bluestore with erasure coding? There are lots of 3-5 year old articles about bluestore as dev preview. Are there any changes? Also there are some disaster recovery posts ...
0
votes
1answer
166 views

Why does ovirt VM using ceph disk stay in “waiting for launch” status

My setup comprises of ceph mimic (centos 7, setup with ceph-ansible), cinder/keystone combo on pike release, and ovirt 4.2.5.1. The external cinder provider is setup and I can create disks. When ...
0
votes
1answer
99 views

How to upgrade librbd1 and librados2 for oVirt 4.2.x (node ng)

oVirt 4.2 comes with librbd1 und librados2 from the Ceph Hammer release which is 0.94.5. I need to update both libraries to the luminous version which is 12.x. because my ceph server is not able to ...
0
votes
0answers
17 views

Can I manually remove this from crushmap of Ceph?

I see the removed OSD in crushmap how device124 device124. device 120 osd.120 device 121 osd.121 device 122 osd.122 device 123 osd.123 device 124 device124 <-- device 125 osd.125 device 126 osd....
2
votes
1answer
96 views

Ceph status health ok, but has flag “nearfull”

Recently in Ceph there was a status of WARN because 3 disks were 85-87% full. I expanded the cluster by adding the server to the storage. But now I see flag "nearfull". Previously, this wasn't see. ...
1
vote
0answers
29 views

openstack instance's image name lost after integration with ceph rbd

after integrated with ceph rbd,my openstack could not show "image name" of my instance, is there any option that controlling this feature?
0
votes
0answers
56 views

Why ceph isn't use all 1Gbit/s bandwith on interface of public network?

We have two networks in Ceph. Public network on 1Gbit/s and Cluster network on 10Gbit/s. But I don't see that clients use whole 1Gbit bandwith interface in OSD node. We use Ceph storage for OpenStack. ...
1
vote
0answers
27 views

Tried to recover a Ext4 Superblock to a CEPH disk

We have a server with Proxmox installed with Debian, it's an dedicated server (sda). The VM's are being backed up every night on a CEPH disk (sdb). After an reboot of the dedicated, the ext4 drive ...
0
votes
0answers
69 views

How to clone a KVM guest which is RBD back-end?

I was using KVM with libvirt(virsh) & file back-end,I can easily clone a guest with ''virt-clone -o vm01 -n vm02 -f /tmp/vm02.img''. Now I move to ceph's rbd back-end,here comes the question,...
4
votes
1answer
999 views

What is the difference between Ceph and NFS when it comes to shared file storage?

Can someone explain to me what is Ceph compared to NFS? From a shared file storage perspective? What is the added value of using Ceph instead of NFS?
2
votes
1answer
105 views

Which distributed filesystem will actually fit my needs?

Hear me out I have seen the question asked (in different forms) here, here, and perhaps the best one I found was here, but I do not think this is a duplicate because quite some time has past since ...
1
vote
0answers
227 views

Slow samba write speed to ceph backend

I have backend on cephfs cluster, mounted on gateway. Mount path is shared with samba 4.7.4. But my speed is low, about 1,5MB/s with small files (1k - 100k). I copy ot with win server 2008 client. ...
1
vote
0answers
254 views

Ceph is full but disk is almost empty

I have a ceph cluster with 2 nodes and 3 osd's each. Each osd is on one partion on a 8TB disk. The server is limited in amount of disk so I even have system on the same disks. On install I used ...
1
vote
2answers
182 views

broken ceph dependency with centos7 and epel

Okay, I'm trying to install ceph, using just "yum install ceph" from the epel repositories. This is what I get: --> Finished Dependency Resolution Error: Package: 1:python-cephfs-0.80.7-0.8.el7....
1
vote
1answer
341 views

On what network do the clients connect to the CEPH cluster (public/private)?

It's recommended to have a public network and cluster network when setting up CEPH. For what I understand this cluster network is what the nodes use to replicate data accross so that would preferably ...
1
vote
1answer
136 views

CephFS needed if you only need block storage?

I'm looking into building a CEPH cluster as a storage solution for our proxmox cluster. It's unclear to me if I need to set up a metadata server too, because this is needed for CephFS but I think ...
6
votes
3answers
1k views

is ceph possible to handle hardware RAID arrays (LUNs) as OSD drives?

I am pretty new to ceph and try to find out if ceph supports hardware level raid HBAs. Sadly could not find any information. What I found is, that it is recommended to use plain disks for OSD. But ...
0
votes
1answer
688 views

OpenShift Origin and Ceph persistent volume

I have installed OpenShift Origin from latest ansible install. (CentOS 7 - 3 masters and 7 nodes) [root@master-1 ~]# openshift version openshift v1.1.0.1-1-g2c6ff4b kubernetes v1.1.0-origin-1107-...
2
votes
1answer
96 views

Virtual Hosting Cluster File System Confusion

My title probably doesn't encompass the full scope of what I'm needing, so I'll lay out what I want to accomplish. I have two Linux servers with large drive arrays, multiple CPUs and a large amount ...
3
votes
1answer
3k views

ceph - can't start osd on rebooted cluster host

I've rebooted the server (one of ceph's hosts). Started the cluster, but the osd, that's on the host, which was rebooted is down. The osd's # is 2, so when I try: sudo /etc/init.d/ceph start osd.2 ...
1
vote
1answer
449 views

Proxmox Ceph OSD Fault

I am running Proxmox with 2 nodes and Ceph. Recently I had a drive die and replaced it. In the process of trying to initialize it as an OSD, all I get are fault errors: 2017-07-31 15:05:05.278499 ...
4
votes
0answers
1k views

Better performance when HDD write cache is disabled? (HGST Ultrastar 7K6000 and Media Cache behavior)

Attention, please. Long read. During initial performance tests of Hitachi Ultrastar 7K6000 drives that I'm planning to use in my Ceph setup I've noticed a strange thing: write performance is better ...
1
vote
2answers
1k views

Is ceph replication based on nodes or disks?

Im currently evaluating storage systems for xenserver. Because data replication is important in the case of a failure i have a question regarding replication in ceph. As far as i know every disk in a ...
2
votes
0answers
273 views

What is the correct usage of ceph-objectsore-tool?

I am trying to export data for pg, but am getting errors from ceph-objectstore-tool. I see some examples online of usage which seem to match this, and reading through the usage statement that the ...
0
votes
1answer
163 views

Does Swift or Ceph have “Vault Lock”-like capabilities?

AWS Glacier offers Vault Lock, which enables compliance policies like “write once read many” (WORM). Google Cloud Platform Storage does not. Does OpenStack Swift or Ceph offer any similar compliance ...
1
vote
1answer
2k views

How do I mount one of multiple filesystems in a ceph cluster?

Ceph now includes (experimental) support for multiple filesystems within a single storage cluster; but the mount options don't seem to allow specifying which filesystem to mount. I have configured ...
0
votes
1answer
572 views

Deleting files in Ceph does not free up space

I am using Ceph, uploading many files through radosgw. After, I want to delete the files. I am trying to do that in Python, like this: bucket = conn.get_bucket(BUCKET) for key in bucket.list(): ...
3
votes
2answers
134 views

Should io and cpu intensive servers be separated in kubernetes cluster?

We are designing a new cluster architecture for our web service and are planing to use Ceph object storage and kubernetes for our services. for optimizing our servers We have different options: Use ...
0
votes
1answer
2k views

Ceph OSDs and journal drives

I have a separate drive for each of my ceph OSD servers. Each OSD host has 4 data drives. Does one journal drive serve the 4? Is the journal drive shared? Should there be a partition for each data ...
2
votes
0answers
497 views

Slow fsync() with ceph (cephfs)

I have built an experimental ceph cluster - 12 nodes, 50 osds, 3 mons, 3 mds, for which I'm trying to run a samba gateway. It seems that when writing lots of small files, the fsync() system calls from ...
1
vote
0answers
143 views

Using NFS Mounts as Long Term Container Storage

this is more of a question of best practice than anything. I currently have a proxmox clustered deployment of three servers, all accessing a ceph cluster (that is self hosted on the same servers). ...
1
vote
1answer
2k views

Ceph Pool Snapshot for CephFS Backup and/or Data Recovery

Since CephFS does not support snapshot yet, could we use Ceph Pool Snapshot for backup purpose from accidental deletion of files inside CephFS? ceph osd pool mksnap {pool-name} {snap-name} ceph osd ...
0
votes
1answer
1k views

Using available space in a Ceph pool

Here's what my Ceph situation looks like (from ceph df): GLOBAL: SIZE AVAIL RAW USED %RAW USED 596G 593G 3633M 0.59 POOLS: NAME ...