Questions tagged [nvme]
The nvme tag has no usage guidance.
118
questions
4
votes
3
answers
6k
views
How can I override IRQ affinity for NVME devices
I am trying to move all interrupts over to cores 0-3 to keep the rest of my cores free for high speed, low latency virtualization.
I wrote a quick script to set IRQ affinity to 0-3:
#!/bin/bash
while ...
3
votes
2
answers
3k
views
Dell R630 supported PCIe to NVMe (m.2) adapter and drive?
I can't seem to find any information anywhere about supported PCIe to NVMe (m.2) adapter and a brand of drive that will also work.
I am aware third party adapters and drives tend to work with various ...
1
vote
0
answers
125
views
Read data off Apple NGFF M2
I have one of those propietary NGFF M2 SSDs found in older Macbooks. I bought a converter which is supposed to convert it to a standard NVMe. When I plug it in my Linux box only sees it as a SCSI ...
0
votes
2
answers
2k
views
Hyper-V Server SSD vs NVME M.2
I'm quite novice on setting up servers and VMs and I'm trying to setup a simple PC to act up as a server in an office. The requirements are low but I think this is a good opportunity for me to learn ...
1
vote
1
answer
96
views
Since VFRC feature in VMware is now end of lifed, what can I replace it with?
Now that VFRC (vflash read cache) is end of lifed in Vsphere 7, does anyone have suggestions on what I could replace it with. Looks like there is only Infinio and Virtunet in the market now, has ...
0
votes
0
answers
157
views
AMD Optane alternative
I want to have an Optane features with my Ryzen build: high TBW and high I/O. Disk space irrelevant (like MEMPEK1W064GA, 64GB would be enough for me). And it would be nice if I can utilize the full ...
7
votes
0
answers
2k
views
Write performance is 5 times worse with LUKS on top of mdadm RAID10 than without LUKS
I have servers with many NVMe disks. I am testing disk performance with fio using the following:
fio --name=asdf --rw=randwrite --direct=1 --ioengine=libaio --bs=16k --numjobs=8 --size=10G --runtime=...
0
votes
1
answer
46
views
VMware nvme writes on unallocated space
I just have a question related to the disk DWPD that I have a hard time finding the answer in google.
Consider this scenario:
1 server with only 1 nvme disk of 1TB (no SATA, no Optane, no SD cards, ...
1
vote
0
answers
296
views
IPXE boot on amd based architecture with NVME
We have a quite big infrastructure and now we started to use AMD based servers instead of the intel ones with 1 nvme only for os.
We are using ipxe to kickstart the servers but for some reason with ...
1
vote
0
answers
2k
views
Very slow read/write on software raid1 with nvme device on Ubuntu focal
I have a server with Ubuntu 20.04.1 LTS with NVME ssd devices running on raid1 and the hard drive operates so slow! to open a gzip file to of 500MB to a 3.7GB takes a while.. way more then it should. ...
2
votes
3
answers
3k
views
NVMe disk problems - smartctl 0x2002 status
My NVMe disk works but it is very slow compared to what it was. I benchmarked it 6 months ago at 2000MB/s using fio, the exact same test now is giving me 350MB/s
I ran smartctl but it's giving a ...
1
vote
2
answers
250
views
Do modern NVMes correct and reallocate bad data at the hardware level?
I'm planning on getting an NVMe for use to contain a swapfile to extend the RAM of my Linux system. Yes, I know this is a bad choice due to the finite number of writes an NVMe has in its lifetime, but ...
2
votes
1
answer
2k
views
Do I need to replace my NVME SSD?
I have a simple server setup:
2 NVME SSD disks (both SAMSUNG MZVLB1T0HALR-00000 for 1TB) united into RAID0.
OS Ubuntu 19.04
Today my system stopped responding. Reboot didn't help.
I connected via KVM ...
6
votes
2
answers
2k
views
How to explain these fio bandwidth results?
Been running a couple of fio tests on a new server with the following setup:
1x Samsung PM981a 512GB M.2 NVMe drive.
Proxmox installed with ZFS on root.
1x VM with 30GB space created and Debian 10 ...
1
vote
1
answer
6k
views
add M.2 NVMe enablement kit to HP DL360gen9
I have a HP DL360gen9 8sff
Applications running on the machine are saturating the current disks, all Intel Enterprise SSD disks.
I would like to add M.2 NVMe drives but I can't figure out which ...
3
votes
5
answers
4k
views
Linus/ext4/nvme crashes during high io
During mvn compilation, I have random crashes.
The problem seems related to high IO and in kern.log, I can see things like:
kernel: [158430.895045] nvme nvme1: controller is down; will reset: CSTS=...
2
votes
3
answers
3k
views
DELL r630 + NvMe M.2->PCie : Cannot boot Ubuntu installed on M.2 NvMe drive via. PCie adapter
I have plugged in a PNY XLR8 500GB M.2 NvMe drive in a Delock PCIe card and stuck it in Riser 3 on my Poweredge r630 - Although the drive doesn't show up anywhere in BIOS, I could boot the Ubuntu ...
0
votes
1
answer
2k
views
Nearly 100% disk utilization after upgrading to debian 10 without any reason
We recently upgraded our servers to debian 10. The setup is completely the same as with debian9 beside of course some new software packages.
We are running the default kernel:
$ uname -a
Linux ...
1
vote
1
answer
2k
views
How to derive bytes written from SMART's host_write_commands?
Using either smartctl 7.0 or nvme 1.7, I get the following data from the SMART log
data_units_written : 350,371,149
host_write_commands : 2,974,115,785
Via smartctl, ...
2
votes
2
answers
3k
views
Supermicro NVME ports - how to use?
So, my supermicro X11 Dai-N has 8 pci-e lanes routed to two dedicated nvme ports. I just can’t figure out how to connect my nvme harddisks to them.
What’s the name of this port specification? Is there ...
0
votes
1
answer
5k
views
nvme queue depth vs fio iodepth arg relation/explanation
This question is related to fio (flexible i/o tester) utility manages I/O queues for a NVME storage (SSD's in particular) whilst using libaio engine.
For testing i am using Ubuntu 14.04 and a ...
0
votes
1
answer
166
views
Servers with 24 NVMe drives - read throughput [duplicate]
We would like to buy some servers and we are looking into DELL server with 24 NVMe drives (like R740xd2 - 2 socket server) since we need very high read IOPS and throughput. We are however wondering ...
2
votes
3
answers
2k
views
Supermicro: New NVMe is not detected properly
We have a Supermicro SuperServer 2029U-TN24R4T with currently 8 U.2 NVMe drives (Samsung PM1725a 1.6 TB) running on CentOS 7 with kernel 5.0.10-1.el7.elrepo.x86_64. After adding a new one (PM1725b 1.6 ...
0
votes
0
answers
74
views
Does the HPE ML350 support 970 Pro?
I am going in for an IO intensive application and have though of HPE ML350 for the purpose. I need to know if the NVMe SSDs are supported as boot drivers or not. Also if the drives are supported ...
1
vote
1
answer
410
views
Is there ready S.M.A.R.T monitoring toolkit for NVME disks (exporter,source,board)?
I already found node_exporter and grafana board for HDD S.M.A.R.T. Can You please get advice or url for ready exporter or other toolkit?
I'm trying to wrote my own "messy" text exporter for ...
1
vote
0
answers
32
views
Can't scale write bandwidth on Linux with Intel Optanes
I have a computer with 3 Intel Optane drives and iotop shows that sequential write to each one separately achieves ~2.2 GB/sec. I suppose that simultaneous writing on all three would result in ~6.6 GB/...
1
vote
2
answers
1k
views
ASUS HyperX M.2 x4 only the first device is visible under Ubuntu
I have a Dell T320 with an ASUS HyperX M.2 x4 PCI-e card (in a PCI-e 3.0 x16 socket), and two Samsung 970 PRO in it (in slot 1 and slot 2 of Hyper M.2), but under Ubuntu 18.04.2 LTS I only see the ...
5
votes
3
answers
8k
views
How to limit ZFS writes on NVME SSD in RAID1 to avoid rapid disk wear?
Currently I'm running Proxmox 5.3-7 on ZFS with few idling debian virtual machines. I'm using two SSDPE2MX450G7 NVME drives in RAID 1. After 245 days of running this setup the S.M.A.R.T values are ...
1
vote
0
answers
2k
views
NVME vs SSD: Read/Write performance issue
We have rent some servers from Hetzner.de. A few of them have NVMe hard disks while the others have SSDs.
We becnchmarked the read/write performance on 4 of our servers using the following commands:
...
12
votes
3
answers
6k
views
Automatically mount SSD instance storage on AWS EC2 in Ubuntu 16.04
I am using an Ubuntu 16.04 EC2 from AWS on c5d.2xlarge HW. It comes with a 200 GB SSD called /dev/nvme1n1.
I am able to mount this drive using:
$ sudo mkfs.ext4 -E nodiscard /dev/nvme1n1
$ sudo ...
2
votes
0
answers
366
views
PCIe: Why pciehp gives contradictory log info
I am running fio jobs on my NVMe SSD and hotplug it then. The platform is hot-pluggable and the system is Centos 7.0.Several seconds after my plug-out, the system encounters a crash and gives these ...
3
votes
1
answer
2k
views
Intel NVME performance on Linux
I have an Intel 760P NVME drive hooked up to a Supermicro X11SRM-F with a Xeon W-2155 and 64GB of ddr4-2400 RAM. The specs for this drive claim 205K-265K IOPS (whatever 8GB span means) with about 3G/...
0
votes
1
answer
1k
views
SSD NVME RAID0 MDADM
My goal is to obtain good writing performances on SSD NVME Disks (benchmarking tools seem to give performances in specific optimized context but far from what we can obtained in real life)
My server:...
4
votes
2
answers
21k
views
What does iodepth in fio tests really mean? Is it the queue depth?
I understand queue depth which is the number of outstanding I/O requests that the storage controller can handle (https://www.tomshardware.com/reviews/ssd-gaming-performance,2991-3.html) i.e., this is ...
0
votes
1
answer
2k
views
"Symbol 'grub_efi_secure_boot' not found" after disk replacement on boot (CentOS 7.5, RAID1)
Recently I had to replace a faulty HD at a Centos 7.5 server with RAID1 (2 x Samsung NVMe disks)
After disk replacement, server booted using the other drive, I copied the same partitions at the new ...
1
vote
0
answers
63
views
Two NVMe devices appearing as one
When I run lsblk I see that I have two NVMe devices:
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
...
nvme0n1 259:0 0 1.1T 0 disk /home
nvme1n1 ...
6
votes
3
answers
3k
views
Disabling ext4 write barriers when using an external journal
I'm currently experimenting with different ways of improving write speeds to a fairly large, rotating disk-based, software-raid (mdadm) array on Debian using fast NVMe devices.
I found that using a ...
3
votes
0
answers
395
views
How to setup nvmetcli for FC loopback mode?
I have a NVMe SSD, KC1000, from Kingston and a HBA, QLE2672 2-port 16Gb Fibre Channel Adapter, from QLogic and are installed on Linux kernel 4.16.6.
Then I would like to set up a scenario which has a ...
1
vote
1
answer
2k
views
Questions about performance, NVMe drives and virtual machines
I currently have a 4770K with Samsung 850 Evo running Ubuntu 16.04. I create and manage VMs using vagrant with Virtual Box. I recreate VMs frequently and right now, it takes roughly 36 seconds to spin ...
6
votes
3
answers
4k
views
What is the Long term stability of NVMe or SSD stored data?
Background:
We need ready access to 30TB of audio data, although only a small fraction of it is ever requested for playback, that playback needs to be done immediately even for multi-year old data. ...
2
votes
2
answers
4k
views
SFF-8639 Backplane - Mixing SAS 12G and NVMe?
On a passive SFF-8639 backplane, with one port per drive, is it possible to connect some of the back side SFF-8643 ports to a SAS HBA and the others to an NVMe card like the Supermicro AOC-SLG3-2E4?
...
4
votes
2
answers
3k
views
Cannot connect to EC2 instance after converting Ubuntu 16 PV to Ubuntu 16 HVM
So it's been a few days and I still can not connect to my new HVM instance with EC2 running Ubuntu 16. For reference, I am trying to upgrade our server from an m3 instance running Ubuntu 16, to a C5 ...
3
votes
4
answers
10k
views
ZFS: Configuration advice 1x NVMe as ARC and ZIL and 4x SSDs for zvols for virtualization
So recently upon testing a ZoL system we found out poor performance of random and sequential reads and poor performance of random writes on our SSDs.
Our system is a stripe of 2x Samsung 1TB 850Evo ...
24
votes
2
answers
14k
views
Why is there both character device and block device for nvme?
I am trying to understand the configuration of nvme.
But I do not understand why there are two devices: nvme block and nvme character device:
crw------- 1 root root 243, 0 Dec 12 16:09 /dev/nvme0
...
0
votes
1
answer
4k
views
Installing 960 Evo NVMe to PowerEdge R730 [closed]
I'm trying to install Samsung 960 Evo NVMe M.2 SSD to Dell PowerEdge R730 server. When I googled it, I stumbled upon this manual, but it says that I should use a carrier to insert into a disk bay (...
2
votes
2
answers
882
views
How-To Chain Boot/Trampoline Boot into Ubuntu 17 on PCIe SSD without BIOS support
I have a server-grade SSD (Intel DC P3520) that will eventually end up in a server. Until the server is deployed in the data center I want to conduct some tests in a desktop machine.
I have an older ...
2
votes
2
answers
24k
views
Mounted disk disappears after reboot
I've got a Centos 7 dedicated server with a SSD + a nvme ssd with 400gb. I followed a tutorial to mount + format (as xfs filesystem) the nvme disk so that I could setup my NoSQL databases on that ...
4
votes
1
answer
1k
views
Encrypt disks using SED and store keys in TPM?
I'm buying servers lately and all of them have disks that support TCG Opal full-disk encryption (aka SED). What I'd like to do is:
Store data encrypted-at-rest on the disks (NVMe & SAS).
Not be ...
-1
votes
1
answer
286
views
NVMe-Dedicated Server without Virtualisation: Big speed benefit? [closed]
I am currently using a VPS which is virtualised with Virtuozzo and sometimes my blog is starting to deliver in 150ms till 350ms (server response).
I assume that the virtualisation is the bottle neck ...
2
votes
1
answer
3k
views
NVMe on old HP Proliant DL380 G6
Do NVMe drives work on HP Proliant DL380 G6 boards?
These servers have:
1 x PCIe Gen2 X8 Full Length, Full Height Slot
2 x PCIe Gen2 X4 Half Length, Full Height Slot
Is there any interference with ...