Questions tagged [io]
IO (or I/O) is an abbreviation for input/output, most commonly referring to disk input/output.
368
questions
0
votes
0answers
7 views
Mysterious, intermittent Read I/O on Ubuntu 18.04 based EC2 server - iotop shows no disk reads
I have an EC2 server based on Ubuntu 18.04. With CloudWatch detailed monitoring enabled I'm seeing spikes of Disk Read Operations that last less than 1 minute (shown as 1-minute spikes on the graph) ...
1
vote
0answers
19 views
PHP high I/O after a DDoS attack
I've had a large scale DDoS attack on my website.
After I've enabled captcha challenge through CloudFlare and the attack stopped, PHP's I/O was still loaded at 99%, and the website was working very ...
0
votes
0answers
39 views
How does avgqu-sz size in iostat output relate to r_await and w_await?
Can someone explain what does avgqu-sz mean in the output of iostat? The manual page states like this.
avgqu-sz:
The average queue length of the requests that were issued to the device.
Can ...
0
votes
1answer
35 views
cat an file programs fail to acess a file but vi works properly (Debian)
I have getting some problems with my system after a bad reset via ctrl+alt+del. Since then errors related to ACPI and ata6 are reported. Trying to get a solution to this (in a savage way:I am n00b), I ...
1
vote
0answers
51 views
Huge IO performance degradation between proxmox ZFS host and WS19 VM
For more than a week, I am trying to determine the reason for the following IO performance degradation between proxmox host and a Windows Server 2019 VM(s).
I have to ask for your help guys because I'...
2
votes
3answers
123 views
Linus/ext4/nvme crashes during high io
During mvn compilation, I have random crashes.
The problem seems related to high IO and in kern.log, I can see things like:
kernel: [158430.895045] nvme nvme1: controller is down; will reset: CSTS=...
3
votes
1answer
68 views
High CPU Load and High I/O wait time : Mysql process
i am fairly new to the role and the technology. we have a Network server that receives the telemetry data from sensors and temeletry data is stored in MYSQL dabatase. we have a front end portal which ...
0
votes
1answer
39 views
nginx + php-fpm extremely high amounts of data written to disk
We are facing a problem with extremely high data writes to disk and huge amounts of traffic over loopback interface.
Environment:
Debian 10.3 (4.19.0.8) - virtualized in hyper-v
nginx/1.14.2
php-fpm ...
1
vote
2answers
255 views
RAID 50 vs RAID 10 for performance?
I have a server with 6 SSD's and a raid controller card that supports both RAID 10 and RAID 50 the plan is to use this as our Build server. It will be pulling in NPM Packages and many small code files,...
0
votes
1answer
19 views
jdb2 spike every hour for few seconds
I get IO hang up every hour on the exact same time on my Ubuntu 18.04 server.
iotop indicate that jbd2 spike to araong 30% around this time.
The hangup last only few seconds.
I started noticing this ...
1
vote
1answer
206 views
What is Split IO?
I'm trying to understand the meaning of split io/sec counter for physical disk in Windows Performance monitor and don't quite get the reason why the single IO operation might be split into several. ...
0
votes
1answer
40 views
Limitations to using many SSDs in computing server [closed]
This is a beginner's question:
We have a powerful computing server for roughly 10 users which will be increasingly used for data processing. In the past the main bottleneck was I/O.
I was wondering ...
17
votes
2answers
2k views
Total I/O cost of a process
I know that iotop lets me measure the disk bandwidth used by any or all processes in real time. iotop -a measures the accumulated disk bandwidth, which is closer to what I want.
However, when the ...
0
votes
1answer
49 views
Is it considered “safe” to run a task with high I/O on a server for 3 days straight, or would you add pauses in it?
I want to hash the contents of 4 million JPEG files taking up 32TB of a 48TB RAID6 partition in CentOS 7. It's a Hetzner SX131 10 x 6TB box.
Having run a test on 10,000 files I think the full job is ...
0
votes
1answer
533 views
nvme queue depth vs fio iodepth arg relation/explanation
This question is related to fio (flexible i/o tester) utility manages I/O queues for a NVME storage (SSD's in particular) whilst using libaio engine.
For testing i am using Ubuntu 14.04 and a ...
2
votes
1answer
420 views
How to accelerate metadata (getattr) requests on an NFS file server for fast locally-cached reads?
We have a storage server with a 12 disk RAID-6 pool of 100 TB. It is used by several client compute nodes via NFS, mostly running deep learning training (lots of images ~100 KB, loaded in random order)...
0
votes
1answer
44 views
Has anyone ever considered using a second partition on a disk in a ZFS mirror when that disk is known to be faster (SATA+NVMe)?
I was reading this question where I felt obligated to comment and poison the sanity of system administrators everywhere. Then I had an interesting idea.
Lets say that you have a laptop with a one ...
0
votes
0answers
25 views
QEMU disk io performance troubleshoot
I have Ubuntu 18.04 installed on dedicated server. It has 2 HDDs with 4 partitions each.
I created software raid1 on sda4 and sdb4 (md2) and created lvm volume on md2.
Then created logical volume of ...
0
votes
1answer
219 views
GCP App Engine Flexible Environment Slow Disk I/O
Background/Overview:
We have a Python Web Application with three environments in two projects staging/dev and production. These are hosted in GCP using Google App Engine Flexible Environment, Cloud ...
0
votes
0answers
66 views
Massive ZFS write and Checksum errors dmesg reports repeatedly aborted SCSI SYNCHRONIZE CACHE(10) commands
This morning my FreeBSD 12 server started reported lots of ZFS errors in writing and checksumming. It immediately began scrubbing and resilvering but came across unrecoverable errors and stopped. ...
8
votes
1answer
2k views
Dmesg full of I/O errors, smart ok, four disks affected
I'm working on a remote server (Dell Poweredge) that was a new install. It has four drives (2TB) and 2 SSD's (250 GB). One SSD contains the OS (RHEL7) and the four mechanical disks are eventually ...
1
vote
0answers
24 views
Can't scale write bandwidth on Linux with Intel Optanes
I have a computer with 3 Intel Optane drives and iotop shows that sequential write to each one separately achieves ~2.2 GB/sec. I suppose that simultaneous writing on all three would result in ~6.6 GB/...
1
vote
1answer
83 views
Bandwidth and IO issues when running hardware raid and zfs
I've inherited a setup using Linux Ubuntu 14, a megaraid_sas, and ZFS.
We're getting some performance problems (never being able to fully utilize the 6GB bandwidth from the raid) and I'm curious if ...
1
vote
1answer
50 views
What units are the I/O measures in GNU “time” in?
GNU time has optional display of I/O measurements:
TIME="%I:%O" /usr/bin/time cp filea fileb
0:5488
but what units is it measuring? Any ideas? The manual only says
%I Number of filesystem ...
1
vote
0answers
33 views
Azure Ubuntu VM - Unexpected short IOPS peaks (bursts)
I am not sure if such issue is well-known, I failed to find any straightforward explanations.
I have an ordinary VM on Azure of a small size (B1s) running a Ubuntu 18.04 from Canonical.
There's a ...
2
votes
2answers
49 views
Which process writes/reads the most from disk over time? [closed]
I want to know which process writes/reads the most from disk over time, I mean that I will start the monitoring process than after X time I will be able to see what was the process that used the disk ...
1
vote
0answers
325 views
What could cause Nginx to write this much I/O?
I have a high-traffic webserver (up to ~50 page loads per second) running nginx, which passes HTML requests to a PHP-FPM backend and serves static assets directly.
Under this load, I get a curiously ...
0
votes
0answers
20 views
Tool to generate IOWaits
In order to test our monitoring infrastructure, I'd like to generate enough IOs to trigger an alert. As the alert is based on IOWaits, I should find a way to 'overload' the IOs.
I found an article ...
1
vote
2answers
5k views
What does iodepth in fio tests really mean? Is it the queue depth?
I understand queue depth which is the number of outstanding I/O requests that the storage controller can handle (https://www.tomshardware.com/reviews/ssd-gaming-performance,2991-3.html) i.e., this is ...
3
votes
3answers
160 views
Are there any industry standards for network timeout or / IO timeout etc? [closed]
I was reading this article about microbursts: https://www.cisco.com/c/en/us/products/collateral/switches/nexus-5000-series-switches/white-paper-c11-733020.html
which can also cause traffic drops.
...
7
votes
2answers
10k views
what is the relation between block size and IO?
I have been reading about disk recently which led me to 3 different doubts. And I am not able to link them together. Three different terms I am confused with are block size, IO and Performance.
I ...
0
votes
0answers
54 views
Give read operations priority over write
I'm working on a system where I have a program that outputs high quality videos for a tv connected to an HDMI (raw 1080p data for instance) and I'm having an issue that when I copy some files to my ...
1
vote
2answers
2k views
Backup with rsync painfully slow
I'm using rsync to perform backups from an NFS server hosted in AWS to another EC2 instance in the same AZ, the rsync command that I'm using in the backup server is the following:
rsync -avzb --...
0
votes
2answers
177 views
Should sector-unaligned HDD writes be slower than sector-aligned writes?
Background: I'm planning to use ZFS and I need to find the correct ashift parameter for my harddrive, which should be log2(sector_size), e.g. 9 for 512 byte sectors.
My harddrive reports a physical ...
1
vote
1answer
351 views
Linux: how can I determine if an application is doing sequential or random disk I/O?
Everyone knows that databases tend to do lots of small random I/O while big data things like Kafka tend to do large sequential I/O, but if I'm approaching this as a sysadmin without making assumptions,...
4
votes
2answers
1k views
Mariadb 10.3 high disk write rates
With the help of some other fine people here, I have found the high loads we have been experiencing lately are related to mysqld high disk write rate seen in iotop. With a load average now 3-6 where ...
0
votes
1answer
719 views
What are the parameters in iostat
I used iostat and trying to understand the iostat output using this link : https://linoxide.com/linux-command/linux-iostat-command/ but It is not clearly explained.
PF the output at different times ...
2
votes
1answer
507 views
contradictory results dd and iostat, ext4 and xfs, on EBS volumes
I'm seeing what seems to me to be contradictory results when looking at disk performance with dd versus iostat on two hosts (EC2 instances with an EBS drive). The hosts are identical except that one ...
1
vote
1answer
132 views
How can I prevent high-volume filesystem writes from ovewhelming memory on my Linux system?
In several contexts, I've seen a behavior on Linux systems in which large volumes of filesystem write operations (e.g., many gigabytes of writes, very quickly) will overwhelm memory, apparently ...
0
votes
1answer
81 views
Disk/Controller Performance Comparison
Since 2012 Iam using the HP Proliant Dl385 G7 with Smart Array P410i.
2xAMD Opteron 2.4GHz, 24 Cores
System Disk C: 2xSAS 15k in RAID1
Some time ago I purchased a new HP Proliant Dl380 G9 with Smart ...
5
votes
1answer
388 views
Misunderstanding about Page Cache and dirty_background_bytes
I've been looking at this for a while now and things aren't lining up with my expectations, but I don't know if it's because something is off, or if my expectations are wrong.
So, I've got a system ...
1
vote
1answer
779 views
file system corrupt in a virtual machine
I have a virtual machine hosted by vmware esxi. It has been started for over 80 days. And about ten days ago, its file system got corrupted. Although I can use fsck to fix this file system, I am ...
1
vote
1answer
718 views
Does ionice has any effect on NFS client?
I need to remove millions of directories recursively on a NFS share and to avoid any performance issues, I ran the command with:
ionice -c 3 -t find /dir -type f -exec rm {};
which will delete all ...
-1
votes
1answer
4k views
VMWare - VM won't start - 5 (Input/output error) - failed drive
Basics:
Server with 4 drives, 2 solid state. One of the solid state drives appears to have failed. Running VMWare 5.0
We tried to distribute the VMs over several disks and using RAID, but I'm not ...
1
vote
0answers
2k views
Troubleshooting `Buffer I/O error` on software-raid/md-device
I've got a RAID-5 mdadm array that reliably causes Buffer I/O error on dev md0, logical block 1598030208, async page read to be written do dmesg when reading that block. Of course, the read also ...
0
votes
2answers
75 views
How to determine if I/O is slow for server
I have used I/O stat to check whether my server's I/O is slow or not. The % util is high but so is the read/sec - write/sec. Can anyone help ?
Following is the output of iostat -mx command
Linux 2.6....
1
vote
1answer
2k views
How to check why a write failed? [closed]
If I was to debug a service on a machine and I determined that a write was failing, how could I check why the write failed?
I know the system call in C will return -1 and then we can check what ...
6
votes
1answer
399 views
RAID6 resync with fast writes but slow reads
I'm using Debian Jessie.
# uname -a
Linux host 4.9.0-0.bpo.3-amd64 #1 SMP Debian 4.9.30-2+deb9u5~bpo8+1 (2017-09-28) x86_64 GNU/Linux
And have setup a RAID6.
# cat /proc/mdstat
Personalities : [...
1
vote
1answer
2k views
Google compute: Why persistent ssd disks are slower than persistent standard disks?
I ran several IO benchmarks and I cannot understand how pd-ssd offers a worse IO throughtput than pd-standard.
In my tests I boot a VM with a bootable disk pd-ssd of 30 or 250GB, and I still get ...
google-cloud-platform
2
votes
0answers
2k views
KVM/QUEMU guest slow disk performance
I'm really confused of QEMU/KVM behavior. I have a server - about 100GB RAM, 2 CPU - 6 core + HT, and three HDDs - one for root, and two for BTRFS-mirror for virtual machines (HGST HDN724030ALE640 if ...
