Questions tagged [ceph]
Ceph is a free software storage platform designed to present object, block, and file storage from a single distributed computer cluster.
0
votes
0answers
14 views
Filesystem of Proxmox VMs get corrupted when Ceph Node goes down
first of all my highlevel Setup:
Proxmox 3 Node Cluster Virtual Environment 5.2-6
on some VMs I use write back caching.
ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e) mimic (stable) ...
0
votes
1answer
9 views
Ceph - Erasure Coded Pools - Always get inactive pgs
I'm trying to achieve similar thing to raid6 on ceph.
But When I'm creating erasure coded pools (k=3 + m=2(or k=4) ) I always get inactive pgs.
ceph health details are:
HEALTH_WARN Reduced data ...
0
votes
1answer
37 views
Ceph iSCSI Gateway installation on Ubuntu 18.04
I'm installing Ceph using Ansible with ceph-ansible project, branch static-3.2. There is a problem with iSCSI Gateway installation.
If you are using iscsigws name in inventory file - it shows that ...
0
votes
0answers
10 views
How do I use a non-standard ceph system user?
I configured a ceph user on my cluster named "cepher." I ran ceph-deploy as this user to deploy some servers.
Then I see this:
[block][INFO ] Running command: sudo ceph --cluster ceph --name client....
0
votes
0answers
12 views
Ceph replicated file permissions
I got a problem with ceph file permissions after replication.
For ex. i copied files with permissions 755, but after it got replicated to other ceph osd it got 700, so how to fix it or is there any ...
1
vote
0answers
20 views
How wide (data + parity) would you go for erasure coded pools / raid 6?
I am currently doing k=8 and m=2
or in other words data = 8 + parity = 2 shards
I am using an all hdd ceph cluster.
I have thought about going data = 11 + parity = 2 which would save a lot of ...
0
votes
0answers
103 views
How can I mount cephfs on proxmox 5.3?
I have added the pxmx-cephfs.secret (the admin keyring) to /etc/pve/priv/ceph/.
Then I proceeded to add cephfs through gui with id pxmx-cephfs and the monitors.
While it is added successfully ...
0
votes
1answer
16 views
Ceph Erasure Profile with k+m> 6 disks -> PGs to be stuck on creating+incomplete state forever
So my pools has 5 nodes with 12 osds each of 8tb.
Currently I am trying to create an erasure coded pool of k=8 m=2, however after adding this profile and create an ecpool with this profile the pools ...
-1
votes
2answers
162 views
How to add Hetzner Cloud Disk Volume to extend /var partition?
I have a new Hetzner cloud server and added an extra 50GB disk volume (CEPH) to my server. The default disk is only 40GB. I thought it would be added to the default storage like the traditional ...
0
votes
0answers
140 views
Ceph vs GlusterFS for elasticsearch storage
Elasticsearch will be holding logs mostly, so we expect sequential read/writes. Does anyone have experience on this? Elasticsearch will run via openshift, if it makes any difference.
1
vote
1answer
655 views
Ceph or Gluster for implementing big NAS
We are planning to build NAS solution which will be primarily used via NFS and CIFS and workloads ranging from various archival application to more “real-time processing”. The NAS will not be used as ...
0
votes
1answer
53 views
mount cephfs or rbd via Internet
Is there any suitable solution for this?
I know about security issues, but is it possible?
Or other better storage solutions?
We tested s3fs+cephrgw, but It's very slow and unstable.
1
vote
1answer
56 views
Ceph architecture for small HPC cluster
I want to build Ceph Storage Cluster for HPC use. (CentOS 7 based)
For now I have enterprise SAS RAID enclosure with 3 shelves by 12 4TB disks(36 total).
Now it is configured as default RAID6 rig. And ...
0
votes
0answers
30 views
Is it safe to use CephFS on Bluestore with erasure coding?
Is it safe to use CephFS on Bluestore with erasure coding?
There are lots of 3-5 year old articles about bluestore as dev preview. Are there any changes?
Also there are some disaster recovery posts ...
0
votes
1answer
130 views
How to upgrade librbd1 and librados2 for oVirt 4.2.x (node ng)
oVirt 4.2 comes with librbd1 und librados2 from the Ceph Hammer release which is 0.94.5.
I need to update both libraries to the luminous version which is 12.x. because my ceph server is not able to ...
0
votes
1answer
204 views
Why does ovirt VM using ceph disk stay in “waiting for launch” status
My setup comprises of ceph mimic (centos 7, setup with ceph-ansible), cinder/keystone combo on pike release, and ovirt 4.2.5.1.
The external cinder provider is setup and I can create disks.
When ...
0
votes
0answers
18 views
Can I manually remove this from crushmap of Ceph?
I see the removed OSD in crushmap how device124 device124.
device 120 osd.120
device 121 osd.121
device 122 osd.122
device 123 osd.123
device 124 device124 <--
device 125 osd.125
device 126 osd....
2
votes
1answer
103 views
Ceph status health ok, but has flag “nearfull”
Recently in Ceph there was a status of WARN because 3 disks were 85-87% full. I expanded the cluster by adding the server to the storage. But now I see flag "nearfull". Previously, this wasn't see.
...
1
vote
0answers
29 views
openstack instance's image name lost after integration with ceph rbd
after integrated with ceph rbd,my openstack could not show "image name" of my instance, is there any option that controlling this feature?
0
votes
0answers
57 views
Why ceph isn't use all 1Gbit/s bandwith on interface of public network?
We have two networks in Ceph. Public network on 1Gbit/s and Cluster network on 10Gbit/s. But I don't see that clients use whole 1Gbit bandwith interface in OSD node. We use Ceph storage for OpenStack. ...
1
vote
0answers
29 views
Tried to recover a Ext4 Superblock to a CEPH disk
We have a server with Proxmox installed with Debian, it's an dedicated server (sda). The VM's are being backed up every night on a CEPH disk (sdb). After an reboot of the dedicated, the ext4 drive ...
0
votes
0answers
77 views
How to clone a KVM guest which is RBD back-end?
I was using KVM with libvirt(virsh) & file back-end,I can easily clone a guest with ''virt-clone -o vm01 -n vm02 -f /tmp/vm02.img''.
Now I move to ceph's rbd back-end,here comes the question,...
4
votes
1answer
1k views
What is the difference between Ceph and NFS when it comes to shared file storage?
Can someone explain to me what is Ceph compared to NFS?
From a shared file storage perspective? What is the added value of using Ceph instead of NFS?
1
vote
0answers
243 views
Slow samba write speed to ceph backend
I have backend on cephfs cluster, mounted on gateway. Mount path is shared with samba 4.7.4. But my speed is low, about 1,5MB/s with small files (1k - 100k). I copy ot with win server 2008 client.
...
0
votes
2answers
788 views
RAID-0 stripe size for Ceph OSD
I have 3 servers that I will use for a new Ceph cluster. It's my first Ceph "playground"... Each server has 2x1TB and 6x2TB HDDs connected to two separate 4-channel SAS controllers, each with 1GB ...
1
vote
0answers
275 views
Ceph is full but disk is almost empty
I have a ceph cluster with 2 nodes and 3 osd's each. Each osd is on one partion on a 8TB disk. The server is limited in amount of disk so I even have system on the same disks.
On install I used ...
1
vote
1answer
384 views
On what network do the clients connect to the CEPH cluster (public/private)?
It's recommended to have a public network and cluster network when setting up CEPH.
For what I understand this cluster network is what the nodes use to replicate data accross so that would preferably ...
1
vote
1answer
162 views
CephFS needed if you only need block storage?
I'm looking into building a CEPH cluster as a storage solution for our proxmox cluster.
It's unclear to me if I need to set up a metadata server too, because this is needed for CephFS but I think ...
6
votes
3answers
1k views
is ceph possible to handle hardware RAID arrays (LUNs) as OSD drives?
I am pretty new to ceph and try to find out if ceph supports hardware level raid HBAs.
Sadly could not find any information. What I found is, that it is recommended to use plain disks for OSD. But ...
3
votes
1answer
503 views
CTDB Samba failover not highly available
TL;DR
Failing a node in a ctdb + samba cluster while interacting with a share interrupts share connection.
here and here state there is work in progress to make this possible
here states it is ...
0
votes
1answer
894 views
install grub in a usb and boot proxmox from another drive
Good morning my friends!
I'm tryng a solution before buy some new hardware.
So this is my situation:
I have some beautifull hp dl360p gen8 that comes with an p420i hardware raid controller.
What ...
2
votes
1answer
107 views
Which distributed filesystem will actually fit my needs?
Hear me out
I have seen the question asked (in different forms) here, here, and perhaps the best one I found was here, but I do not think this is a duplicate because quite some time has past since ...
2
votes
1answer
98 views
Virtual Hosting Cluster File System Confusion
My title probably doesn't encompass the full scope of what I'm needing, so I'll lay out what I want to accomplish.
I have two Linux servers with large drive arrays, multiple CPUs and a large amount ...
1
vote
1answer
478 views
Proxmox Ceph OSD Fault
I am running Proxmox with 2 nodes and Ceph. Recently I had a drive die and replaced it. In the process of trying to initialize it as an OSD, all I get are fault errors:
2017-07-31 15:05:05.278499 ...
4
votes
0answers
1k views
Better performance when HDD write cache is disabled? (HGST Ultrastar 7K6000 and Media Cache behavior)
Attention, please. Long read.
During initial performance tests of Hitachi Ultrastar 7K6000 drives that I'm planning to use in my Ceph setup I've noticed a strange thing: write performance is better ...
1
vote
2answers
188 views
broken ceph dependency with centos7 and epel
Okay, I'm trying to install ceph, using just "yum install ceph" from the epel repositories.
This is what I get:
--> Finished Dependency Resolution
Error: Package: 1:python-cephfs-0.80.7-0.8.el7....
2
votes
0answers
284 views
What is the correct usage of ceph-objectsore-tool?
I am trying to export data for pg, but am getting errors from ceph-objectstore-tool. I see some examples online of usage which seem to match this, and reading through the usage statement that the ...
1
vote
2answers
1k views
Is ceph replication based on nodes or disks?
Im currently evaluating storage systems for xenserver. Because data replication is important in the case of a failure i have a question regarding replication in ceph.
As far as i know every disk in a ...
0
votes
1answer
168 views
Does Swift or Ceph have “Vault Lock”-like capabilities?
AWS Glacier offers Vault Lock, which enables compliance policies like “write once read many” (WORM). Google Cloud Platform Storage does not. Does OpenStack Swift or Ceph offer any similar compliance ...
3
votes
2answers
134 views
Should io and cpu intensive servers be separated in kubernetes cluster?
We are designing a new cluster architecture for our web service and are planing to use Ceph object storage and kubernetes for our services. for optimizing our servers We have different options:
Use ...
0
votes
2answers
1k views
Ceph PG's stuck creating / Pool creation slow
I've created a small ceph cluster 3 servers each with 5 disks for osd's with one monitor per server.
The actual setup seems to have gone OK and the mons are in quorum and all 15 osd's are up and in ...
0
votes
1answer
3k views
Ceph OSDs and journal drives
I have a separate drive for each of my ceph OSD servers. Each OSD host has 4 data drives. Does one journal drive serve the 4? Is the journal drive shared? Should there be a partition for each data ...
2
votes
0answers
521 views
Slow fsync() with ceph (cephfs)
I have built an experimental ceph cluster - 12 nodes, 50 osds, 3 mons, 3 mds, for which I'm trying to run a samba gateway. It seems that when writing lots of small files, the fsync() system calls from ...
1
vote
0answers
144 views
Using NFS Mounts as Long Term Container Storage
this is more of a question of best practice than anything.
I currently have a proxmox clustered deployment of three servers, all accessing a ceph cluster (that is self hosted on the same servers). ...
0
votes
2answers
3k views
Is Ceph too slow and how to optimize it?
The setup is 3 clustered Proxmox for computations, 3 clustered Ceph storage nodes,
ceph01 8*150GB ssds (1 used for OS, 7 for storage)
ceph02 8*150GB ssds (1 used for OS, 7 for storage)
ceph03 8*250GB ...
1
vote
1answer
144 views
Can VM hard disk size be bigger than single SSD in Ceph Storage?
I have a 3 node ceph proxmox cluster for storage with node1 7*143GB ssds, node2 7*143 ssds, node3 7*218 ssds. Now I have read through the http://docs.ceph.com/docs/jewel/architecture/#mapping-pgs-to-...
1
vote
1answer
2k views
Will running Ceph (or similar systems) virtualized degrade its performance?
I am setting up a Ceph cluster. The client is asking for it to be done in virtual machines, one hypervisor / VM per server. Given my previous (minor) experience with virtual machines, I wonder if ...
3
votes
1answer
44 views
Shared and replication filesystem support POSIX
I'm looking for opensource solution that support my use case. I have now 4 node on my cluster network and i need this.
Store file system (huge list)
Replication my file save on one node and replicate ...
1
vote
1answer
2k views
How do I mount one of multiple filesystems in a ceph cluster?
Ceph now includes (experimental) support for multiple filesystems within a single storage cluster; but the mount options don't seem to allow specifying which filesystem to mount.
I have configured ...
0
votes
1answer
1k views
Using available space in a Ceph pool
Here's what my Ceph situation looks like (from ceph df):
GLOBAL:
SIZE AVAIL RAW USED %RAW USED
596G 593G 3633M 0.59
POOLS:
NAME ...
