Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [ceph]

Ceph is a free software storage platform designed to present object, block, and file storage from a single distributed computer cluster.

3
votes
1answer
408 views

CTDB Samba failover not highly available

TL;DR Failing a node in a ctdb + samba cluster while interacting with a share interrupts share connection. here and here state there is work in progress to make this possible here states it is ...
0
votes
2answers
577 views

RAID-0 stripe size for Ceph OSD

I have 3 servers that I will use for a new Ceph cluster. It's my first Ceph "playground"... Each server has 2x1TB and 6x2TB HDDs connected to two separate 4-channel SAS controllers, each with 1GB ...
0
votes
0answers
34 views

Ceph vs GlusterFS for elasticsearch storage

Elasticsearch will be holding logs mostly, so we expect sequential read/writes. Does anyone have experience on this? Elasticsearch will run via openshift, if it makes any difference.
0
votes
2answers
2k views

Is Ceph too slow and how to optimize it?

The setup is 3 clustered Proxmox for computations, 3 clustered Ceph storage nodes, ceph01 8*150GB ssds (1 used for OS, 7 for storage) ceph02 8*150GB ssds (1 used for OS, 7 for storage) ceph03 8*250GB ...
1
vote
1answer
36 views

Ceph architecture for small HPC cluster

I want to build Ceph Storage Cluster for HPC use. (CentOS 7 based) For now I have enterprise SAS RAID enclosure with 3 shelves by 12 4TB disks(36 total). Now it is configured as default RAID6 rig. And ...
1
vote
1answer
104 views

Ceph or Gluster for implementing big NAS

We are planning to build NAS solution which will be primarily used via NFS and CIFS and workloads ranging from various archival application to more “real-time processing”. The NAS will not be used as ...
0
votes
0answers
20 views

Is it safe to use CephFS on Bluestore with erasure coding?

Is it safe to use CephFS on Bluestore with erasure coding? There are lots of 3-5 year old articles about bluestore as dev preview. Are there any changes? Also there are some disaster recovery posts ...
0
votes
0answers
18 views

mount cephfs or rbd via Internet

Is there any suitable solution for this? I know about security issues, but is it possible? Or other better storage solutions? We tested s3fs+cephrgw, but It's very slow and unstable.
0
votes
1answer
89 views

Why does ovirt VM using ceph disk stay in “waiting for launch” status

My setup comprises of ceph mimic (centos 7, setup with ceph-ansible), cinder/keystone combo on pike release, and ovirt 4.2.5.1. The external cinder provider is setup and I can create disks. When ...
0
votes
1answer
51 views

How to upgrade librbd1 and librados2 for oVirt 4.2.x (node ng)

oVirt 4.2 comes with librbd1 und librados2 from the Ceph Hammer release which is 0.94.5. I need to update both libraries to the luminous version which is 12.x. because my ceph server is not able to ...
0
votes
0answers
17 views

Can I manually remove this from crushmap of Ceph?

I see the removed OSD in crushmap how device124 device124. device 120 osd.120 device 121 osd.121 device 122 osd.122 device 123 osd.123 device 124 device124 <-- device 125 osd.125 device 126 osd....
2
votes
1answer
86 views

Ceph status health ok, but has flag “nearfull”

Recently in Ceph there was a status of WARN because 3 disks were 85-87% full. I expanded the cluster by adding the server to the storage. But now I see flag "nearfull". Previously, this wasn't see. ...
1
vote
0answers
28 views

openstack instance's image name lost after integration with ceph rbd

after integrated with ceph rbd,my openstack could not show "image name" of my instance, is there any option that controlling this feature?
0
votes
0answers
48 views

Why ceph isn't use all 1Gbit/s bandwith on interface of public network?

We have two networks in Ceph. Public network on 1Gbit/s and Cluster network on 10Gbit/s. But I don't see that clients use whole 1Gbit bandwith interface in OSD node. We use Ceph storage for OpenStack. ...
1
vote
0answers
26 views

Tried to recover a Ext4 Superblock to a CEPH disk

We have a server with Proxmox installed with Debian, it's an dedicated server (sda). The VM's are being backed up every night on a CEPH disk (sdb). After an reboot of the dedicated, the ext4 drive ...
0
votes
0answers
49 views

How to clone a KVM guest which is RBD back-end?

I was using KVM with libvirt(virsh) & file back-end,I can easily clone a guest with ''virt-clone -o vm01 -n vm02 -f /tmp/vm02.img''. Now I move to ceph's rbd back-end,here comes the question,...
4
votes
1answer
572 views

What is the difference between Ceph and NFS when it comes to shared file storage?

Can someone explain to me what is Ceph compared to NFS? From a shared file storage perspective? What is the added value of using Ceph instead of NFS?
2
votes
1answer
96 views

Which distributed filesystem will actually fit my needs?

Hear me out I have seen the question asked (in different forms) here, here, and perhaps the best one I found was here, but I do not think this is a duplicate because quite some time has past since ...
0
votes
2answers
1k views

Ceph PG's stuck creating / Pool creation slow

I've created a small ceph cluster 3 servers each with 5 disks for osd's with one monitor per server. The actual setup seems to have gone OK and the mons are in quorum and all 15 osd's are up and in ...
1
vote
0answers
197 views

Slow samba write speed to ceph backend

I have backend on cephfs cluster, mounted on gateway. Mount path is shared with samba 4.7.4. But my speed is low, about 1,5MB/s with small files (1k - 100k). I copy ot with win server 2008 client. ...
1
vote
0answers
191 views

Ceph is full but disk is almost empty

I have a ceph cluster with 2 nodes and 3 osd's each. Each osd is on one partion on a 8TB disk. The server is limited in amount of disk so I even have system on the same disks. On install I used ...
1
vote
2answers
169 views

broken ceph dependency with centos7 and epel

Okay, I'm trying to install ceph, using just "yum install ceph" from the epel repositories. This is what I get: --> Finished Dependency Resolution Error: Package: 1:python-cephfs-0.80.7-0.8.el7....
1
vote
1answer
246 views

On what network do the clients connect to the CEPH cluster (public/private)?

It's recommended to have a public network and cluster network when setting up CEPH. For what I understand this cluster network is what the nodes use to replicate data accross so that would preferably ...
1
vote
1answer
107 views

CephFS needed if you only need block storage?

I'm looking into building a CEPH cluster as a storage solution for our proxmox cluster. It's unclear to me if I need to set up a metadata server too, because this is needed for CephFS but I think ...
6
votes
3answers
836 views

is ceph possible to handle hardware RAID arrays (LUNs) as OSD drives?

I am pretty new to ceph and try to find out if ceph supports hardware level raid HBAs. Sadly could not find any information. What I found is, that it is recommended to use plain disks for OSD. But ...
0
votes
1answer
681 views

OpenShift Origin and Ceph persistent volume

I have installed OpenShift Origin from latest ansible install. (CentOS 7 - 3 masters and 7 nodes) [root@master-1 ~]# openshift version openshift v1.1.0.1-1-g2c6ff4b kubernetes v1.1.0-origin-1107-...
0
votes
1answer
787 views

install grub in a usb and boot proxmox from another drive

Good morning my friends! I'm tryng a solution before buy some new hardware. So this is my situation: I have some beautifull hp dl360p gen8 that comes with an p420i hardware raid controller. What ...
2
votes
1answer
84 views

Virtual Hosting Cluster File System Confusion

My title probably doesn't encompass the full scope of what I'm needing, so I'll lay out what I want to accomplish. I have two Linux servers with large drive arrays, multiple CPUs and a large amount ...
3
votes
1answer
3k views

ceph - can't start osd on rebooted cluster host

I've rebooted the server (one of ceph's hosts). Started the cluster, but the osd, that's on the host, which was rebooted is down. The osd's # is 2, so when I try: sudo /etc/init.d/ceph start osd.2 ...
1
vote
1answer
397 views

Proxmox Ceph OSD Fault

I am running Proxmox with 2 nodes and Ceph. Recently I had a drive die and replaced it. In the process of trying to initialize it as an OSD, all I get are fault errors: 2017-07-31 15:05:05.278499 ...
4
votes
0answers
1k views

Better performance when HDD write cache is disabled? (HGST Ultrastar 7K6000 and Media Cache behavior)

Attention, please. Long read. During initial performance tests of Hitachi Ultrastar 7K6000 drives that I'm planning to use in my Ceph setup I've noticed a strange thing: write performance is better ...
1
vote
2answers
1k views

Is ceph replication based on nodes or disks?

Im currently evaluating storage systems for xenserver. Because data replication is important in the case of a failure i have a question regarding replication in ceph. As far as i know every disk in a ...
2
votes
0answers
253 views

What is the correct usage of ceph-objectsore-tool?

I am trying to export data for pg, but am getting errors from ceph-objectstore-tool. I see some examples online of usage which seem to match this, and reading through the usage statement that the ...
0
votes
1answer
148 views

Does Swift or Ceph have “Vault Lock”-like capabilities?

AWS Glacier offers Vault Lock, which enables compliance policies like “write once read many” (WORM). Google Cloud Platform Storage does not. Does OpenStack Swift or Ceph offer any similar compliance ...
1
vote
1answer
1k views

How do I mount one of multiple filesystems in a ceph cluster?

Ceph now includes (experimental) support for multiple filesystems within a single storage cluster; but the mount options don't seem to allow specifying which filesystem to mount. I have configured ...
0
votes
1answer
534 views

Deleting files in Ceph does not free up space

I am using Ceph, uploading many files through radosgw. After, I want to delete the files. I am trying to do that in Python, like this: bucket = conn.get_bucket(BUCKET) for key in bucket.list(): ...
3
votes
2answers
128 views

Should io and cpu intensive servers be separated in kubernetes cluster?

We are designing a new cluster architecture for our web service and are planing to use Ceph object storage and kubernetes for our services. for optimizing our servers We have different options: Use ...
0
votes
1answer
2k views

Ceph OSDs and journal drives

I have a separate drive for each of my ceph OSD servers. Each OSD host has 4 data drives. Does one journal drive serve the 4? Is the journal drive shared? Should there be a partition for each data ...
2
votes
0answers
455 views

Slow fsync() with ceph (cephfs)

I have built an experimental ceph cluster - 12 nodes, 50 osds, 3 mons, 3 mds, for which I'm trying to run a samba gateway. It seems that when writing lots of small files, the fsync() system calls from ...
1
vote
0answers
140 views

Using NFS Mounts as Long Term Container Storage

this is more of a question of best practice than anything. I currently have a proxmox clustered deployment of three servers, all accessing a ceph cluster (that is self hosted on the same servers). ...
1
vote
1answer
2k views

Ceph Pool Snapshot for CephFS Backup and/or Data Recovery

Since CephFS does not support snapshot yet, could we use Ceph Pool Snapshot for backup purpose from accidental deletion of files inside CephFS? ceph osd pool mksnap {pool-name} {snap-name} ceph osd ...
0
votes
1answer
1k views

Using available space in a Ceph pool

Here's what my Ceph situation looks like (from ceph df): GLOBAL: SIZE AVAIL RAW USED %RAW USED 596G 593G 3633M 0.59 POOLS: NAME ...
1
vote
1answer
122 views

Can VM hard disk size be bigger than single SSD in Ceph Storage?

I have a 3 node ceph proxmox cluster for storage with node1 7*143GB ssds, node2 7*143 ssds, node3 7*218 ssds. Now I have read through the http://docs.ceph.com/docs/jewel/architecture/#mapping-pgs-to-...
3
votes
1answer
5k views

Ceph OSD always 'down' in Ubuntu 14.04.1

I am trying to install and deploy a ceph cluster. As I don't have enough physical servers, I create 4 VMs on my OpenStack using official Ubuntu 14.04 image. I want to deploy a cluster with 1 monitor ...
1
vote
1answer
2k views

Will running Ceph (or similar systems) virtualized degrade its performance?

I am setting up a Ceph cluster. The client is asking for it to be done in virtual machines, one hypervisor / VM per server. Given my previous (minor) experience with virtual machines, I wonder if ...
3
votes
1answer
43 views

Shared and replication filesystem support POSIX

I'm looking for opensource solution that support my use case. I have now 4 node on my cluster network and i need this. Store file system (huge list) Replication my file save on one node and replicate ...
1
vote
2answers
1k views

How do I mount rbd devices with the fuse driver?

This is a follow-up question from: How do I mount an rbd device from fstab Which I asked some time back. How do I mount rbd devices with the fuse driver? (I can't use the kernel driver from docker ...
1
vote
4answers
13k views

Unable to add initial monitor to Ceph in Ubuntu

I am trying to set up a Ceph cluster. I have 4 nodes - 1 admin-node, 1 monitor and 2 object storage devices. The installation guide I am using is given at the following location: http://ceph.com/docs/...
2
votes
1answer
2k views

Why is GlusterFS so slow here?

We've set up a mirroring pair of GlusterFS servers. No special tuning, whatever came "out of the box" with GlusterFS-3.5.1 in the official RHEL6 RPM, that's what we have. The cluster works, but the ...
0
votes
1answer
595 views

OpenStack Redundant Ceph/Cinder Storage Architecture

Hello Serverfault community, I'm currently designing an OpenStack cluster. The part where I'm currently stuck is the Storage Architecture. I thought of building two redundant Ceph clusters in ...