Questions tagged [nvme]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
4 votes
3 answers
6k views

How can I override IRQ affinity for NVME devices

I am trying to move all interrupts over to cores 0-3 to keep the rest of my cores free for high speed, low latency virtualization. I wrote a quick script to set IRQ affinity to 0-3: #!/bin/bash while ...
Hubro's user avatar
  • 1,148
3 votes
2 answers
3k views

Dell R630 supported PCIe to NVMe (m.2) adapter and drive?

I can't seem to find any information anywhere about supported PCIe to NVMe (m.2) adapter and a brand of drive that will also work. I am aware third party adapters and drives tend to work with various ...
user avatar
1 vote
0 answers
125 views

Read data off Apple NGFF M2

I have one of those propietary NGFF M2 SSDs found in older Macbooks. I bought a converter which is supposed to convert it to a standard NVMe. When I plug it in my Linux box only sees it as a SCSI ...
Shawn Welch's user avatar
0 votes
2 answers
2k views

Hyper-V Server SSD vs NVME M.2

I'm quite novice on setting up servers and VMs and I'm trying to setup a simple PC to act up as a server in an office. The requirements are low but I think this is a good opportunity for me to learn ...
bennygenel's user avatar
1 vote
1 answer
96 views

Since VFRC feature in VMware is now end of lifed, what can I replace it with?

Now that VFRC (vflash read cache) is end of lifed in Vsphere 7, does anyone have suggestions on what I could replace it with. Looks like there is only Infinio and Virtunet in the market now, has ...
KVH Hospital's user avatar
0 votes
0 answers
157 views

AMD Optane alternative

I want to have an Optane features with my Ryzen build: high TBW and high I/O. Disk space irrelevant (like MEMPEK1W064GA, 64GB would be enough for me). And it would be nice if I can utilize the full ...
tchspprt's user avatar
7 votes
0 answers
2k views

Write performance is 5 times worse with LUKS on top of mdadm RAID10 than without LUKS

I have servers with many NVMe disks. I am testing disk performance with fio using the following: fio --name=asdf --rw=randwrite --direct=1 --ioengine=libaio --bs=16k --numjobs=8 --size=10G --runtime=...
tacos_tacos_tacos's user avatar
0 votes
1 answer
46 views

VMware nvme writes on unallocated space

I just have a question related to the disk DWPD that I have a hard time finding the answer in google. Consider this scenario: 1 server with only 1 nvme disk of 1TB (no SATA, no Optane, no SD cards, ...
NickPR's user avatar
  • 103
1 vote
0 answers
296 views

IPXE boot on amd based architecture with NVME

We have a quite big infrastructure and now we started to use AMD based servers instead of the intel ones with 1 nvme only for os. We are using ipxe to kickstart the servers but for some reason with ...
Rebron Treborn's user avatar
1 vote
0 answers
2k views

Very slow read/write on software raid1 with nvme device on Ubuntu focal

I have a server with Ubuntu 20.04.1 LTS with NVME ssd devices running on raid1 and the hard drive operates so slow! to open a gzip file to of 500MB to a 3.7GB takes a while.. way more then it should. ...
ufk's user avatar
  • 333
2 votes
3 answers
3k views

NVMe disk problems - smartctl 0x2002 status

My NVMe disk works but it is very slow compared to what it was. I benchmarked it 6 months ago at 2000MB/s using fio, the exact same test now is giving me 350MB/s I ran smartctl but it's giving a ...
GCPuser's user avatar
  • 23
1 vote
2 answers
250 views

Do modern NVMes correct and reallocate bad data at the hardware level?

I'm planning on getting an NVMe for use to contain a swapfile to extend the RAM of my Linux system. Yes, I know this is a bad choice due to the finite number of writes an NVMe has in its lifetime, but ...
Steve Mucci's user avatar
2 votes
1 answer
2k views

Do I need to replace my NVME SSD?

I have a simple server setup: 2 NVME SSD disks (both SAMSUNG MZVLB1T0HALR-00000 for 1TB) united into RAID0. OS Ubuntu 19.04 Today my system stopped responding. Reboot didn't help. I connected via KVM ...
Stalinko's user avatar
  • 121
6 votes
2 answers
2k views

How to explain these fio bandwidth results?

Been running a couple of fio tests on a new server with the following setup: 1x Samsung PM981a 512GB M.2 NVMe drive. Proxmox installed with ZFS on root. 1x VM with 30GB space created and Debian 10 ...
Woodgnome's user avatar
  • 213
1 vote
1 answer
6k views

add M.2 NVMe enablement kit to HP DL360gen9

I have a HP DL360gen9 8sff Applications running on the machine are saturating the current disks, all Intel Enterprise SSD disks. I would like to add M.2 NVMe drives but I can't figure out which ...
Max Favilli's user avatar
3 votes
5 answers
4k views

Linus/ext4/nvme crashes during high io

During mvn compilation, I have random crashes. The problem seems related to high IO and in kern.log, I can see things like: kernel: [158430.895045] nvme nvme1: controller is down; will reset: CSTS=...
Brimstedt's user avatar
  • 151
2 votes
3 answers
3k views

DELL r630 + NvMe M.2->PCie : Cannot boot Ubuntu installed on M.2 NvMe drive via. PCie adapter

I have plugged in a PNY XLR8 500GB M.2 NvMe drive in a Delock PCIe card and stuck it in Riser 3 on my Poweredge r630 - Although the drive doesn't show up anywhere in BIOS, I could boot the Ubuntu ...
Arni J's user avatar
  • 75
0 votes
1 answer
2k views

Nearly 100% disk utilization after upgrading to debian 10 without any reason

We recently upgraded our servers to debian 10. The setup is completely the same as with debian9 beside of course some new software packages. We are running the default kernel: $ uname -a Linux ...
Janning's user avatar
  • 1,551
1 vote
1 answer
2k views

How to derive bytes written from SMART's host_write_commands?

Using either smartctl 7.0 or nvme 1.7, I get the following data from the SMART log data_units_written : 350,371,149 host_write_commands : 2,974,115,785 Via smartctl, ...
Gaia's user avatar
  • 1,895
2 votes
2 answers
3k views

Supermicro NVME ports - how to use?

So, my supermicro X11 Dai-N has 8 pci-e lanes routed to two dedicated nvme ports. I just can’t figure out how to connect my nvme harddisks to them. What’s the name of this port specification? Is there ...
matthias_buehlmann's user avatar
0 votes
1 answer
5k views

nvme queue depth vs fio iodepth arg relation/explanation

This question is related to fio (flexible i/o tester) utility manages I/O queues for a NVME storage (SSD's in particular) whilst using libaio engine. For testing i am using Ubuntu 14.04 and a ...
Dexobox's user avatar
  • 103
0 votes
1 answer
166 views

Servers with 24 NVMe drives - read throughput [duplicate]

We would like to buy some servers and we are looking into DELL server with 24 NVMe drives (like R740xd2 - 2 socket server) since we need very high read IOPS and throughput. We are however wondering ...
Marek Milkovič's user avatar
2 votes
3 answers
2k views

Supermicro: New NVMe is not detected properly

We have a Supermicro SuperServer 2029U-TN24R4T with currently 8 U.2 NVMe drives (Samsung PM1725a 1.6 TB) running on CentOS 7 with kernel 5.0.10-1.el7.elrepo.x86_64. After adding a new one (PM1725b 1.6 ...
user3693367's user avatar
0 votes
0 answers
74 views

Does the HPE ML350 support 970 Pro?

I am going in for an IO intensive application and have though of HPE ML350 for the purpose. I need to know if the NVMe SSDs are supported as boot drivers or not. Also if the drives are supported ...
Utkarsh Narain Srivastava's user avatar
1 vote
1 answer
410 views

Is there ready S.M.A.R.T monitoring toolkit for NVME disks (exporter,source,board)?

I already found node_exporter and grafana board for HDD S.M.A.R.T. Can You please get advice or url for ready exporter or other toolkit? I'm trying to wrote my own "messy" text exporter for ...
Kein's user avatar
  • 131
1 vote
0 answers
32 views

Can't scale write bandwidth on Linux with Intel Optanes

I have a computer with 3 Intel Optane drives and iotop shows that sequential write to each one separately achieves ~2.2 GB/sec. I suppose that simultaneous writing on all three would result in ~6.6 GB/...
John Smith's user avatar
1 vote
2 answers
1k views

ASUS HyperX M.2 x4 only the first device is visible under Ubuntu

I have a Dell T320 with an ASUS HyperX M.2 x4 PCI-e card (in a PCI-e 3.0 x16 socket), and two Samsung 970 PRO in it (in slot 1 and slot 2 of Hyper M.2), but under Ubuntu 18.04.2 LTS I only see the ...
netmano's user avatar
  • 269
5 votes
3 answers
8k views

How to limit ZFS writes on NVME SSD in RAID1 to avoid rapid disk wear?

Currently I'm running Proxmox 5.3-7 on ZFS with few idling debian virtual machines. I'm using two SSDPE2MX450G7 NVME drives in RAID 1. After 245 days of running this setup the S.M.A.R.T values are ...
Ahupathi R.'s user avatar
1 vote
0 answers
2k views

NVME vs SSD: Read/Write performance issue

We have rent some servers from Hetzner.de. A few of them have NVMe hard disks while the others have SSDs. We becnchmarked the read/write performance on 4 of our servers using the following commands: ...
Sinai's user avatar
  • 213
12 votes
3 answers
6k views

Automatically mount SSD instance storage on AWS EC2 in Ubuntu 16.04

I am using an Ubuntu 16.04 EC2 from AWS on c5d.2xlarge HW. It comes with a 200 GB SSD called /dev/nvme1n1. I am able to mount this drive using: $ sudo mkfs.ext4 -E nodiscard /dev/nvme1n1 $ sudo ...
John Corson's user avatar
2 votes
0 answers
366 views

PCIe: Why pciehp gives contradictory log info

I am running fio jobs on my NVMe SSD and hotplug it then. The platform is hot-pluggable and the system is Centos 7.0.Several seconds after my plug-out, the system encounters a crash and gives these ...
Leo Erzhuo Chen's user avatar
3 votes
1 answer
2k views

Intel NVME performance on Linux

I have an Intel 760P NVME drive hooked up to a Supermicro X11SRM-F with a Xeon W-2155 and 64GB of ddr4-2400 RAM. The specs for this drive claim 205K-265K IOPS (whatever 8GB span means) with about 3G/...
Server Fault's user avatar
  • 3,774
0 votes
1 answer
1k views

SSD NVME RAID0 MDADM

My goal is to obtain good writing performances on SSD NVME Disks (benchmarking tools seem to give performances in specific optimized context but far from what we can obtained in real life) My server:...
akira2x3x's user avatar
4 votes
2 answers
21k views

What does iodepth in fio tests really mean? Is it the queue depth?

I understand queue depth which is the number of outstanding I/O requests that the storage controller can handle (https://www.tomshardware.com/reviews/ssd-gaming-performance,2991-3.html) i.e., this is ...
GP92's user avatar
  • 741
0 votes
1 answer
2k views

"Symbol 'grub_efi_secure_boot' not found" after disk replacement on boot (CentOS 7.5, RAID1)

Recently I had to replace a faulty HD at a Centos 7.5 server with RAID1 (2 x Samsung NVMe disks) After disk replacement, server booted using the other drive, I copied the same partitions at the new ...
Kostas Alexandreskos's user avatar
1 vote
0 answers
63 views

Two NVMe devices appearing as one

When I run lsblk I see that I have two NVMe devices: $ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT ... nvme0n1 259:0 0 1.1T 0 disk /home nvme1n1 ...
yong's user avatar
  • 111
6 votes
3 answers
3k views

Disabling ext4 write barriers when using an external journal

I'm currently experimenting with different ways of improving write speeds to a fairly large, rotating disk-based, software-raid (mdadm) array on Debian using fast NVMe devices. I found that using a ...
jcharaoui's user avatar
  • 332
3 votes
0 answers
395 views

How to setup nvmetcli for FC loopback mode?

I have a NVMe SSD, KC1000, from Kingston and a HBA, QLE2672 2-port 16Gb Fibre Channel Adapter, from QLogic and are installed on Linux kernel 4.16.6. Then I would like to set up a scenario which has a ...
codexplorer's user avatar
1 vote
1 answer
2k views

Questions about performance, NVMe drives and virtual machines

I currently have a 4770K with Samsung 850 Evo running Ubuntu 16.04. I create and manage VMs using vagrant with Virtual Box. I recreate VMs frequently and right now, it takes roughly 36 seconds to spin ...
mmvn's user avatar
  • 11
6 votes
3 answers
4k views

What is the Long term stability of NVMe or SSD stored data?

Background: We need ready access to 30TB of audio data, although only a small fraction of it is ever requested for playback, that playback needs to be done immediately even for multi-year old data. ...
Rowan Hawkins's user avatar
2 votes
2 answers
4k views

SFF-8639 Backplane - Mixing SAS 12G and NVMe?

On a passive SFF-8639 backplane, with one port per drive, is it possible to connect some of the back side SFF-8643 ports to a SAS HBA and the others to an NVMe card like the Supermicro AOC-SLG3-2E4? ...
PSpacer's user avatar
  • 23
4 votes
2 answers
3k views

Cannot connect to EC2 instance after converting Ubuntu 16 PV to Ubuntu 16 HVM

So it's been a few days and I still can not connect to my new HVM instance with EC2 running Ubuntu 16. For reference, I am trying to upgrade our server from an m3 instance running Ubuntu 16, to a C5 ...
Alex's user avatar
  • 221
3 votes
4 answers
10k views

ZFS: Configuration advice 1x NVMe as ARC and ZIL and 4x SSDs for zvols for virtualization

So recently upon testing a ZoL system we found out poor performance of random and sequential reads and poor performance of random writes on our SSDs. Our system is a stripe of 2x Samsung 1TB 850Evo ...
Ajay's user avatar
  • 61
24 votes
2 answers
14k views

Why is there both character device and block device for nvme?

I am trying to understand the configuration of nvme. But I do not understand why there are two devices: nvme block and nvme character device: crw------- 1 root root 243, 0 Dec 12 16:09 /dev/nvme0 ...
GP92's user avatar
  • 741
0 votes
1 answer
4k views

Installing 960 Evo NVMe to PowerEdge R730 [closed]

I'm trying to install Samsung 960 Evo NVMe M.2 SSD to Dell PowerEdge R730 server. When I googled it, I stumbled upon this manual, but it says that I should use a carrier to insert into a disk bay (...
ferit's user avatar
  • 133
2 votes
2 answers
882 views

How-To Chain Boot/Trampoline Boot into Ubuntu 17 on PCIe SSD without BIOS support

I have a server-grade SSD (Intel DC P3520) that will eventually end up in a server. Until the server is deployed in the data center I want to conduct some tests in a desktop machine. I have an older ...
i8086's user avatar
  • 21
2 votes
2 answers
24k views

Mounted disk disappears after reboot

I've got a Centos 7 dedicated server with a SSD + a nvme ssd with 400gb. I followed a tutorial to mount + format (as xfs filesystem) the nvme disk so that I could setup my NoSQL databases on that ...
kentor's user avatar
  • 163
4 votes
1 answer
1k views

Encrypt disks using SED and store keys in TPM?

I'm buying servers lately and all of them have disks that support TCG Opal full-disk encryption (aka SED). What I'd like to do is: Store data encrypted-at-rest on the disks (NVMe & SAS). Not be ...
Evan's user avatar
  • 497
-1 votes
1 answer
286 views

NVMe-Dedicated Server without Virtualisation: Big speed benefit? [closed]

I am currently using a VPS which is virtualised with Virtuozzo and sometimes my blog is starting to deliver in 150ms till 350ms (server response). I assume that the virtualisation is the bottle neck ...
frankhammer's user avatar
2 votes
1 answer
3k views

NVMe on old HP Proliant DL380 G6

Do NVMe drives work on HP Proliant DL380 G6 boards? These servers have: 1 x PCIe Gen2 X8 Full Length, Full Height Slot 2 x PCIe Gen2 X4 Half Length, Full Height Slot Is there any interference with ...
Luis's user avatar
  • 283