Questions tagged [io]

IO (or I/O) is an abbreviation for input/output, most commonly referring to disk input/output.

Filter by
Sorted by
Tagged with
0 votes
1 answer
95 views

Why Swap usage is high for Influxdb: 100% Disk I/O and Swap usage but only 50% memory?

We have an influxdb VM that is constantly under 100% swap. Even if we restart the VM, the swap usage reaches 100% in about 20 minutes. However, memory usage is only about 50%. (The VM has 32 CPU Cores ...
46 votes
6 answers
42k views

Measuring 'total bytes written' under Linux

We're quite interested in exploring the possibility of using SSD drives in a server environment. However, one thing that we need to establish is expected drive longevity. According to this article ...
2 votes
0 answers
47 views

How useful is ionice tool with modern linux kernel

Ionice tool is supposed to offer relieve on high I/O load by executing commands only when system is in specified state. https://www.linux.org/docs/man1/ionice.html states Linux supports I/O ...
-1 votes
0 answers
83 views

Proxmox I/O Error, dev sda, sector xxxxxxxxx HP ProLiant DL360 G6

I am running an HP Proliant DL360 G6 with 8 hard drives(SSD) installed in a RAID 5 configuration. After running for some time (a few hours to a couple of days) these errors are produced. Image of I/O ...
1 vote
0 answers
27 views

dmesg spamming "Power-on or device reset occurred"

So I have a DELL R730xd Server with 24 SSDs on a PERC H730P Mini controller in HBA mode. All 24 drives are in one big ZFS Raid-Z2 pool. I have noticed that dmesg gets spammed with a lot of these ...
19 votes
2 answers
3k views

What could explain unknown I/O on new 16TB HDD on a Debian server?

I recently obtained a new 16TB HDD for my Debian server. I first created a partition (gpt) on it via my Ubuntu-like desktop, formatted it (ext4), and rsync'ed old data on it. The disk is now usable, ...
1 vote
0 answers
272 views

MariaDB io_uring_queue_init() failed with errno 1 with kernel version 6.2.0-1018

I am running MariaDB using the official docker hub image: mariadb:lts (this is MariaDB 10.11.6-MariaDB-1:10.11.6+maria~ubu2204 source revision fecd78b83785d5ae96f2c6ff340375be803cd299) When the ...
2 votes
0 answers
87 views

Disk I/O over 100% (~400%)

I'm working with an Ubuntu OS cluster which is using a Job scheduler. Tens of jobs are running in parallel and all jobs read/write in the same disk Array, 3 SSD disks with software RAID 0. I do not ...
0 votes
0 answers
392 views

SSIS package cannot see network drive

I have an SSIS package that cannot see a network folder (with files) - when run directly from SSMS. (Windows Server 2019, SQL Server 2019, Visual Studio 2019, SSMS versions 17 & 18). I am using a ...
4 votes
1 answer
200 views

Unable to get my rsync of my software RAID 6 past 40MB/s. Looking for suggestions

Wondering what I could adjust to get my RAID 6 Software Raid to resync quicker. Currently it's proceeding at max of 64MB/s and averages to something around 25MB/s. Hoping to get it to 200MB: [...
0 votes
1 answer
298 views

ZFS I/O Error, Kernel Panic during import

I'm running a raidz1-0 (RAID5) setup with 4 data 2TB SSDs. During midnight, somehow 2 of my data disks experience some I/O error (from /var/log/messages). When I investigated in the morning, the zpool ...
1 vote
1 answer
72 views

Disk saturated by mariadb on CentOS 7, blocks everything

For the past few days I have been having a problem with a webserver running CentOS 7. Every 13-14 minutes some operation (which I identified with mariadb using iostat) seems to completely saturate the ...
0 votes
0 answers
185 views

Memory mapped files filesystem access over NFS performance

I'm debugging a system that is accessing files on AWS EFS, where the EFS reaches is peak throughput because of IO operations. I noticed that the application is sometime reading memory mapped files in ...
9 votes
1 answer
791 views

Finding which process is reading from disk constantly on FreeBSD

EDIT: This ended up being a scrub process. Run zpool status -v and read the scan status for your pool clearly. I have a TrueNAS server running FreeBSD and this morning at 12 AM the disks started to be ...
-3 votes
4 answers
3k views

figuring out high load cause from top and iotop

What should i do if I want to prevent this high load. My website becomes slow when load is higher than 8. Please see iotop result too. top - 20:23:10 up 127 days, 3:22, 1 user, load average: 9.92, ...
0 votes
0 answers
414 views

I have a new server but IO Speed is terrible based on FIO results

I ordered a new server because I have to move to higher performance server. However, this new server is totally awful. I tried to get the benchmark by using fio here is the result. [root@s206716 ~]# ...
2 votes
1 answer
1k views

contradictory results dd and iostat, ext4 and xfs, on EBS volumes

I'm seeing what seems to me to be contradictory results when looking at disk performance with dd versus iostat on two hosts (EC2 instances with an EBS drive). The hosts are identical except that one ...
-2 votes
2 answers
816 views

whats the difference between 1lane and 2lane with SATA-3 speeds? [closed]

I read the beauty of PCIe is that a designer can combine two, four, eight or sixteen of PCIe lanes into a single data port. 2.5 Gb/s PCIe×1 5 Gb/s PCIe×2 I have two sata 3 drives that I want to ...
73 votes
6 answers
247k views

How to check disk I/O utilization per process?

I'm having a problem with a Linux system and I have found sysstat and sar to report huge peaks of disk I/O, average service time as well as average wait time. How could I determine which process is ...
1 vote
1 answer
348 views

Strange IO activity

I have some strange IO activity on a server and I can't figure our where it's coming from. To provide some background, I had to replace an NVMe (Samsung PM81) from a server due to wear. I didn't ...
1 vote
0 answers
42 views

iotop PRIO column is showing ?sys

How can i fix iotop to show PRIO correct value?
0 votes
2 answers
641 views

High iowait, swap nearly always fully in use

I'm often experiencing slowness on a large server that is used by several users simultaneously, with many CPUs (72) and a decent amount of ram (125gb). Granted, the server runs a lot of stuff so gets ...
1 vote
1 answer
2k views

What are the parameters in iostat

I used iostat and trying to understand the iostat output using this link : https://linoxide.com/linux-command/linux-iostat-command/ but It is not clearly explained. PF the output at different times ...
9 votes
2 answers
9k views

Why do sequential writes have better performance than random writes on SSDs?

An LBA (logical block addresses) is a mapping table implemented in the FTL to match between logical and physical pages/blocks in SSDs, my guess is that most SSDs (at least when they are empty) keeps ...
27 votes
2 answers
33k views

kworker consuming +90% IO and zero disk write

this is a standard apache web server on AWS Linux AMI + EBS. We are noticing high load average (+8) and iotop -a shows: Total DISK READ: 0.00 B/s | Total DISK WRITE: 2.37 M/s TID PRIO USER ...
7 votes
2 answers
4k views

Identify which process is responsible for heavy disk i/o under FreeBSD

I wonder how I can monitor which process takes out all my IO bandwith. The only command i know is gstat but it shows cumulative info. Is there any tools i can use to inspect the running processes for ...
2 votes
1 answer
2k views

Weekly RAID check affecting my system - any way to mitigate?

I recently got my webhost (Hetzner) to add a paid of 16TB SATA drives to my webserver. Currently using 2.5TB of them. They're RAID 1 mirrored. I also have two 4TB nVME drives with 700GB currently on ...
6 votes
7 answers
38k views

I/O Performance Benchmarking Linux

Just finished setting up a poor mans SAN with iSCSI and want to benchmark it. What are some good I/O performance benchmarks for Linux besides: hdparm -tT /dev/sda1 How do I get a IOPS measurement? ...
0 votes
1 answer
324 views

Does my swapping server with ~80GB available RAM need even more?

I'm administering a Debian Linux x86_64 server with 125GB RAM, a 10GB swap partition and a swappiness value of 60. A free -gw output prints: total used free shared ...
3 votes
1 answer
1k views

Linux buffer memory always stays at zero

One of my hosted server is a Xen guest running Debian and linux kernel 2.6.32 then 2.6.38. It has been running fine for quite some time (more than a year) but I recently noticed something weird. For ...
0 votes
0 answers
377 views

Centos stream 9 local console login input shown as squares

I'm after a fresh centos stream 9 install as minimal install, with only English keyboard and only root user. After installing, I rebooted and got to the login screen. Trying to type "root" ...
0 votes
0 answers
476 views

High I/O wait in apache tomcat server

Facing high I/O wait in apache tomcat server when the load increases. Suspecting disk I/O is less when tomcat is running however. disk IO seems drops drastically when tomcat is running along with the ...
7 votes
1 answer
5k views

Linux - KVM - very slow disk io

The I/O performance of my new server is pretty low. I Googled and tried everything I can imagine. First of all, here are the Hardware Components: AMD Opteron(tm) Processor 6344 4x Seagate ...
1 vote
0 answers
84 views

Couchdb 3.1.1 increases disk latency gradually over weeks

I've been dealing with this issue for weeks now. I have the followin scenario: couchdb2.3.1-A <===> couchdb2.3.1-B <===> couchdb3.1.1-A <===> couchdb3.1.1-B where <===> ...
2 votes
1 answer
1k views

Looking for a NBD C/S for windows (Network Block Device) [closed]

Where can I find a NBD Server (and also a Client) that I can use in windows 10? (preferably FOSS source code for inspect/change/compile/...) Been looking for a while on the internet and seems that no ...
2 votes
2 answers
1k views

Dmesg is full of I/O errors on Ubuntu 18.04 VM, physical disks and host are fine

I have an Ubuntu 18.04 virtual machine on a Windows Server 2016 host (Dell PowerEdge R530, Intel Xeon E5-2620 @ 2.1GHz, 96GB RAM), running in VirtualBox 6.1.10. The host system has 2x2TB HDDs in RAID ...
3 votes
5 answers
4k views

Linus/ext4/nvme crashes during high io

During mvn compilation, I have random crashes. The problem seems related to high IO and in kern.log, I can see things like: kernel: [158430.895045] nvme nvme1: controller is down; will reset: CSTS=...
0 votes
1 answer
2k views

Testing zpool IO performance

To test the write speed of a 2.5 inch SATA3 spinning hard drive (some old Seagate one) installed in my TrueNAS system I used dd if=/dev/zero of=/mnt/pool/test.dd bs=1024 count=1m (found this in ...
-1 votes
1 answer
514 views

Simulate some process IO with fio or other tool

Is there any way to track a specific process and get stats for disk IO such as queue depth, total reading/writing threads, percent of reads/writes and so on? The main goal is to use all that ...
1 vote
1 answer
819 views

Interpreting SMART Logs of Newly Installed NVMe RAID0 Crashing Everyday

An Ubuntu 20.04 system has been stable for a year until a 2nd and 3rd NVMe drive is installed on the motherboard to form a 2x1TB RAID0 array. Ever since then, there is huge amount of IO load on this ...
156 votes
13 answers
77k views

Are networks now faster than disks?

This is a software design question I used to work on the following rule for speed cache memory > memory > disk > network With each step being 5-10 times the previous step (e.g. cache ...
0 votes
0 answers
173 views

Getting high read await in Zabbix monitoring, is this a common behavior for RAID5 disk array?

We are running a Oracle 11.2 DB on Redhat 6.8, and disk array is configured as RAID5, which is using a Lenovo 730i RAID adapter. I checked RAID policy and found that "Read Ahead Always" and &...
2 votes
0 answers
2k views

kvm virtualization, slow io in windows guest (with virtio drivers) on ubuntu host and HP 360 Server with P440 RAID controller

Problem: Very slow IO in Windows Guest vs Host Hardware: HP 360 G9, 2 x 10 cores Xeon CPU, 128GB Ram, 4 x 4TB SAS HDD, Raid 10, P440 Controller (with 2GB cache and battery) Host: Ubuntu 20.04 Server, ...
0 votes
1 answer
1k views

disk read io very high, but no process perform io read

We have experienced a very strange problem. The load of the server machine is very high due to high disk read io. But the processes running on this server do not perform any disk read operation. We ...
5 votes
1 answer
3k views

Linux Read-Ahead Downsides

Are there any downsides to raising the size of the read-ahead cache? On our farm, we're currently running at 256, and upon raising that higher, we are seeing significant throughput gains.   [root@...
0 votes
1 answer
252 views

It is possible that a process can not link a file, but I can with sudo for the same user?

I am working with Dovecot 2.3.4.1 and EncFS (encfs version 1.9.5) encrypted mailboxes on Debian 10 (no SELinux) Basic setup is: EncFS mounted with encfs --public /var/encrypted-mail /var/decrypted-...
2 votes
1 answer
487 views

About IOPS of a RAID5 disk array without cache

we are using a 530-8i RAID adapter,which is none-cache supported, on a Lenovo ThinkSystem SR590 server.And we have 4 HDD(600GB/10K RPM/SAS) in the array. I read an article about calculating the disk/...
2 votes
0 answers
796 views

Access disk io data in linux without tools like iotop/iostat to find processes or files with IO

We have regularly high load average. With the help of a zabbix agent, I could narrow it down to be mainly because of cpu iowait (and lastly disk io). I am not allowed to install any additional ...
0 votes
1 answer
205 views

ZFS raidz3 on proxmox having IO issues

I'm running Proxmox 6.2-11 with 256GB ECC RAM and the following raidz3 pool: pool: rpool state: ONLINE scan: scrub repaired 0B in 0 days 15:25:14 with 0 errors on Sun Jan 10 15:49:29 2021 config: ...
0 votes
0 answers
348 views

High %wa from top command while runing PyTorch

Everything was fine yesterday, but I noticed high dataloading time this morning. How do I go about trying to figure out how to fix this? top: process state: iotop: iostat:

1
2 3 4 5
9