Questions tagged [storage]
Primarily questions about disk storage, sometimes questions about application storage or cloud storage. Always questions about storing data.
147
questions
7
votes
1
answer
5k
views
FIO Benchmarking - Inconsistent and Slower than Anticipated: Are my RAIDs Misconfigured?
TL;DR: I'm having some performance issues with my hypervisor storage. here are a bunch of test results from fio. Skip to the Results section to read about them and see my questions.
Summary
I ...
- 251
6
votes
8
answers
3k
views
What are SAS external storage options (Promise, Infortrend, SuperMircro, ...)?
We are looking for a ~32T external storage solution for two 48-core AMD servers. These will be used for a small Linux OpenVZ cloud for CPU intensive web-servers and data warehousing. Dual path with ...
- 2,495
6
votes
1
answer
11k
views
Can I determine IOPS on a disk array using bonnie++?
I would like to measure the read and write performance of a RAID6 array verses a RAID10 array. I understand that bonnie++ can be used to calculate disk performance, but the numbers produced by bonnie++...
- 24k
5
votes
4
answers
8k
views
multiple file systems for mysql
Does mysql support multiple file systems for a single database with most of the tables being on MyISAM?
Context: we have a 1.5TB mysql database, which is increasing at the rate of 200GB per month. ...
- 4,534
1
vote
4
answers
12k
views
Creating bootable Fedora USB with persistent storage
I am attempting to burn the full Fedora 19 x86_64 DVD iso to a USB drive and have a separate partition on it for a kickstart file / other media that will be installed in the kickstart process.
With ...
- 325
1
vote
3
answers
2k
views
HP DL585 G1 with VMware ESXi 5 - SATA solution, need more storage
I have the following:
2 x DL585 G1 (8 cores) with built in Smart Array 5i (2 x 74GB SCSI in Raid 0)
I installed VMware ESXi 5 and got it all working :)
However I need more space.
I was thinking ...
- 195
0
votes
3
answers
4k
views
3WARE 9650-4LPML JBOD DISK to RAID - 1 Mirror Migration How To?
It is possible to migrate a single disk, to Raid 1 as per Table 10 in the User Guide Page 122:
http://www.3ware.com/support/UserDocs/3ware9650SE-UsrGuide.pdf
I am trying to find what the best method ...
- 111
0
votes
3
answers
2k
views
dell 2T 7.2k nearline SAS 6Gbps- ingress/egress throughput
I would like to know, how to calculate sustained read and write speeds supported. I don't have access to machine/drive, but need to define specs to purchase one.
What all things would it be ...
- 19
62
votes
8
answers
7k
views
What are the advantages of tape drives?
IBM still develop and sell tape drives today. The capacity of them seems to be on a par with today's hard drives, but the search time and transfer rate are both significantly lower than that of hard ...
- 2,877
37
votes
2
answers
8k
views
What is a Storage Area Network, and which benefits does it have over different storage solutions?
I'm proposing this to be a canonical question about enterprise-level Storage Area Networks.
What is a Storage Area Network (SAN), and how does it work?
How is it different from a Network Attached ...
- 71.3k
35
votes
7
answers
13k
views
Best way to test new HDD's for a cheap storage server
I want to build a storage server and bought 10 x 2TB WD RED's. The HDD's just arrived.
Is there any tool you guys use to check for bad drives or to best defend against infant mortality before copying ...
- 1,155
27
votes
1
answer
43k
views
KVM and Libvirt - How do I hotplug a new virtio disk?
I am trying to hot-add a file-based disk to a running KVM virtual server. I've created a new disk from scratch using the command
dd of=/home/cloud/vps_59/test.img bs=1 seek=5G count=0
and I was ...
- 271
22
votes
3
answers
4k
views
Can "enterprise" drives be safely replaced by near/midline in some situations?
When specifying servers, like (I would assume) many engineers who aren't experts in storage, I'll generally play it safe (and perhaps be a slave to marketing) by standardising on a minimum of 10k SAS ...
- 1,872
20
votes
7
answers
1k
views
Is there still a place for tape storage?
We've backed up our data on LTO tapes for years and it's a real comfort to know we have everything on tape. A sister project and one of our data providers have both moved to 100% disk storage because ...
- 885
18
votes
3
answers
4k
views
150 TB and growing, but how to grow?
My group currently has two largish storage servers, both NAS running debian linux. The first is an all-in-one 24-disk (SATA) server that is several years old. We have two hardware RAIDS set up on it ...
- 283
17
votes
8
answers
13k
views
Understanding NVMe storage and hardware requirements
I'm a bit confused about the recent developments in PCIe-based storage, particularly as it relates to the NVMe specification and its hardware compatibility.
While I've worked extensively with SSDs in ...
- 198k
16
votes
3
answers
37k
views
Why are Large Form Factor (LFF) disks still fairly prevalent?
Small Form Factor (SFF) / 2.5" disks seem to have become more popular than LFF disks now due to them being preferable over LFF disks in many scenarios (lower power consumption, higher density, etc). ...
- 1,872
16
votes
6
answers
4k
views
Cost effective, long term archival of video and image data? ~50 TB
My lab is in the process of setting up a small server that holds data (mostly video and image data, plus a few documents) for the project our group is working on at a moment in time.
Historically, ...
- 875
15
votes
2
answers
28k
views
VMware ESXi SSD on RAID showing as non-ssd
I am replacing a KVM server with a ESXi server. I just installed ESXi 5.5u1 and added a data store. The new server has 4 SSD drives in RAID 6 using hardware RAID. When I added the data store VMware ...
- 1,505
13
votes
5
answers
13k
views
LVM and disaster recovery
I understand what LVM is and what it accomplishes, but I feel like I am missing some things.
Lets say we have two physical drives, sda and sdb. Both are 100 Megs. I put them into VolumeGroup1 and ...
- 2,977
12
votes
3
answers
10k
views
LVM volume group shared between KVM/libvirt host and guests: is this a bad idea?
I have just built a shiny new KVM/libvirt-based virtual machine host, containing 4 SATA II hard drives, and running CentOS 5.5 x86_64.
I have decided to create virtual machine disks as logical ...
- 143
11
votes
3
answers
16k
views
Can NetApp Snapshots be used as Backups?
Our shop relies very heavily on NetApp Volume Snapshots for backups. We use traditional agent-based tape backups for some of our data but by and large we rely on the Snapshots for the majority of our ...
user62491
10
votes
4
answers
10k
views
write hole: which RAID levels are affected?
In my journey to understanding the advantages of RAIDZ, i came across the concept of write hole.
As this page explains, a write hole is the inconsistency you get among the disks of the array, when ...
- 336
10
votes
1
answer
8k
views
mdadm and 4k sectors (advanced format)
There are numerous questions on Serverfault about aligning 4k sectors disks, but one thing is not really clear to me yet.
I successfully aligned my RAID1+LVM. One of the things I did was use mdadm ...
- 8,204
10
votes
2
answers
24k
views
Slow disk performance on Azure Virtual Machine
Okay, so first of all, allow me to say that I'm not an operations person, but a developer. So I'm going into a bit of unknown land here, so please bear with me.
I would like to use a Azure Virtual ...
- 203
9
votes
5
answers
2k
views
Is brand-name NAS overpriced?
NetGear's ReadyNAS 2100 has 4 disk slots and costs $2000 with no disks. That seems a bit too expensive for just 4 disk slots.
Dell has good network storage solutions too. PowerVault NX3000 has 6 disk ...
- 424
9
votes
4
answers
4k
views
Advantages of thick provisioning of storage over thin provisioning with virtual machines
I typically use thin-provisioning of storage space when building VMs: it provides a lot of flexibility, and seems to be faster to build.
The only potential definite advantage I can see is to ensure ...
- 18.8k
9
votes
3
answers
5k
views
Backup storage server with ZFS
I am IT everything man at a small company. I want to design a new infrastructure including a new server and a separate backup server with company wide backup policy.
The most important thing in the ...
- 190
8
votes
7
answers
8k
views
How do you limit the use of network storage in your company?
I think this is one area that is a pain for every sysadmin.
Users don't really know how much storage space they are using, and don't take the time to manage what is stored there, or how long it stays ...
8
votes
2
answers
3k
views
Server Rack vs. Server in PC "Towers"
A friend of mine told me that the company he made an internship at didn't have server racks, because the cooling just takes up too much energy and thus isn't effective.
That company was using "normal" ...
- 183
8
votes
6
answers
4k
views
hardware recommendations for a DIY storage system based on ZFS [closed]
Looking to make use of old server lying idle as a proof-of-concept
...here are specs
Dell PE 2900: Xeon 5110 - 2P, 12 GB RAM, 8x 300 GB 15K drives, Perc 5i+256 MB cache
what additional h/w would be ...
- 1,275
8
votes
6
answers
30k
views
Identify Disks on SuperMicro server running FreeBSD
I am running on a SuperMicro 2U server with 6 disks in a ZPool. I want to identify each drive bay using the Drive Carrier LED from running a command from a tool in FreeBSD. Is there a way to do this?
...
- 299
8
votes
2
answers
2k
views
Adding new virtual disks to a RHEL host in ESX "live"
I'm sure I've just missed which tutorial/manual page covers this, but how do you add get the guest OS to recognize that you've added new drives to it without a reboot?
I have a RHEL5 guest running on ...
- 18.8k
7
votes
2
answers
6k
views
Does adding more drives to a RAID 10 Array increase performance?
I'd like to know if I were to add additional drives to an existing Raid 10 if this would increase the speed of the entire array?
- 371
7
votes
1
answer
1k
views
How long is the "defined grace period" for the idle scheduling class of the CFQ io scheduler?
The ionice man page says
A program running with idle io priority will only get disk time
when no other program has asked for disk io for a defined grace period.
Where is this "grace period" ...
- 293
7
votes
3
answers
6k
views
SAS Expander backplane performance drops when more than 8 drives are active?
We are using an LSI 9211-8i raid controller (flashed in IT mode) in a PCI-e 2.0 x8 slot.
It's connected to a supermicro SAS 836EL1 backplane with 1 SFF-8087 cable.
We have 16 drives installed.
When ...
- 123
7
votes
12
answers
9k
views
Best way to build home NAS with redundancy [closed]
I need a NAS device for my home network to get these high value files off my laptop (family photos, music, coding projects, etc.). My goal is cheap and reliable.
I have a spare old PC laying around ...
- 2,183
6
votes
1
answer
1k
views
Completely disjoint iSCSI networks vs dedicated switches and VLANs
I'm looking for best practice or pros/cons for deploying an iSCSI storage network.
The two options are:
A completely disjoint iSCSI storage network
Dedicated switches with a separate VLAN for ...
- 101k
6
votes
1
answer
38k
views
PostgreSQL - Could not extend file No space left on device. HINT: Check free disk space
I've been inserting data into PostgreSQL DB, and got the error message:
psycopg2.OperationalError: could not extend file "base/16384/61892": No space left on device
HINT: Check free disk space
I am ...
- 173
6
votes
1
answer
4k
views
Replace HP Smart Array E200i without losing data
I've got a Smart Array 200i which seems to have some bad slots (slot 3 and slot 5). It doesn't matter what HD I put in these slots, it keeps telling me the drive is bad.
My question is two-fold:
Is ...
- 63
5
votes
1
answer
30k
views
How to mount external vfat drive as user?
I have a mountpoint /mnt/v1 and a user bak in group bak. I did this:
sudo chown bak:bak /mnt/v1
sudo chmod 775 /mnt/v1
Now I want to mount an external fat32 drive with partition /dev/sdb1 to this ...
- 937
5
votes
2
answers
1k
views
Windows File server heavy cpu usage - Render Farm with many clients reading data at once
I have a windows file sever with Dual Xeon E5-2650v4 cpus. They are 14 core each so 28 cores total.
The network interface is a Mellanox Connectx3 40gbs.
The disk drives in the server are 7.68TB Sata ...
- 51
5
votes
8
answers
3k
views
Best places to free space on Windows? [closed]
Prompted by Adam..
I would like your suggestions as to freeing up space on a Windows machine. Yes, yes, I know disk space is cheap but sometimes, you have older machines that just need more space and ...
- 523
5
votes
1
answer
2k
views
How do I test the effect of ionice (against a device using the cfq scheduler)?
I'm trying to construct an experiment to measure the effect of ionice. What I'd like to do is to (per another answer on serverfault) cause frequent enough I/O that a sufficiently "niced" process is ...
- 293
5
votes
2
answers
6k
views
Repurposed disk - Linux blkid command returns incorrect information
I'm working with an SSD drive removed from a VMware ESXi installation. I'm trying to reuse the disk in an existing Linux installation (CentOS 6.2). The drive sits behind an HP Smart Array P410 RAID ...
- 198k
4
votes
2
answers
3k
views
Resizing of volumes used as VMWare NFS datastores causes Linux VM issues
We had to perform a live volume resize of an NFS datastores VMWare uses on our Netapp. All of our Windows VMs were fine after the resize. However, some of our Linux VMs had issues.
Some Linux VMs ...
- 10.9k
4
votes
1
answer
4k
views
Troubleshooting Windows Server 2012 storage spaces
I'm trying the new "Storage Pools" feature on Windows Server 2012, and I've created several disks on the pool.
When I restart the server, some of the disks (two, out of four) do not attach ...
- 330
4
votes
1
answer
11k
views
Max drive size for PERC H700 in Dell r610
I'm having a hard time understanding the spec. sheet. It says there is a max storage cap of 12TB for the entire system, but does not specify a disk size restriction.
Should I assume the HDD cap is ...
- 143
4
votes
1
answer
21k
views
How to upgrade the firmware of HP SAS expander card without Smart Array controller or Proliant Server?
How can I update/upgrade/flash the firmware of an HP SAS expander card [468406-B21 a.k.a. 487738-001]?
I used to do this using Windows and a HP P410 Smart Array controller, however that controller is ...
- 994
4
votes
2
answers
2k
views
Beginner SAS and server questions
I am a c++ programmer and database administrator looking to expand my knowledge of server administration and maintenance. I have read the Wikipedia pages and several other documents I found by ...
- 151