Questions tagged [iowait]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
252 votes
7 answers
296k views

Can anyone explain precisely what IOWait is?

As much as I have read about iowait, it is still mystery to me. I know it's the time spent by the CPU waiting for a IO operations to complete, but what kind of IO operations precisely? What I am also ...
23 votes
4 answers
19k views

Does iowait include time waiting for network calls?

The proc(5) manpage describes iowait as "time waiting for IO to complete". This was mostly explained in an earlier question. My question is: while waiting in blocking IO, does this include waiting on ...
  • 2,804
20 votes
1 answer
2k views

Why higher await time for DM multipath device than underlying device?

We have a CentOS 6.4 based server attached to Hitachi HNAS 3080 storage and observed the kernel remount the filesystem in read-only mode: May 16 07:31:03 GNS3-SRV-CMP-001 kernel: [1259725.675814] ...
  • 778
12 votes
1 answer
26k views

what percentage of iowait is considered to be high?

when iowait is considered to be high ? iostat -x avg-cpu: %user %nice %system %iowait %steal %idle 2,89 0,01 5,45 49,83 0,00 41,83 Device: rrqm/s wrqm/s ...
9 votes
1 answer
16k views

Windows "iowait" CPU utilization reporting

Does Windows account for and report "iowait" in the same manner as Linux? That is, processes enter "uninterruptible sleep" and their time spent is subtracted from "CPU free"? If so, which Perfmon ...
8 votes
1 answer
2k views

high load average, high wait, dmesg raid error messages (debian nfs server)

Debian 6 on HP proliant (2 CPU) with raid (2*1.5T RAID1 + 2*2T RAID1 joined RAID0 to make 3.5T) running mainly nfs & imapd (plus samba for windows share & local www for previewing web pages); ...
7 votes
4 answers
29k views

How to measure IOwait per device?

I have a server, which exports home directories over NFS. They are on software RAID1 (/dev/sdb and /dev/sdc) and the OS is on /dev/sda. I noticed that my %iowait as reported by top and sar are ...
  • 2,235
7 votes
4 answers
10k views

What is causing the unusual high load average?

I noticed on Tuesday night of last week, the load average went up sharply and it seemed abnormal since the traffic is small. Usually, the numbers usually average around .40 or lower and my server ...
  • 71
6 votes
4 answers
3k views

What would cause IO Wait on a SAN?

I understand IO wait when I see it on a server, it means that the CPU is blocked while I wait for the IO to catch up [source]. I am trying to understand why a SAN stats would show a high IO wait - ...
5 votes
4 answers
7k views

High iowait on Amazon EC2 MySQL instance with EBS volume

We have a MySQL server running on an Amazon EC2 c1.medium instance relying on a single EBS volume using the ext3 filesystem for storage. This MySQL server is queried ~500/ps by several applications ...
5 votes
2 answers
18k views

Finding the root cause of 100% iowait in Linux

I have a system that is a few years old that is running Fedora 7 (I know, it's old too). Now I have PS3 Media Server running (and some other things, like Vmware and Plone) and I noticed that I was ...
  • 2,176
5 votes
2 answers
4k views

Strange Recurrent Excessive I/O Wait

I know quite well that I/O wait has been discussed multiple times on this site, but all the other topics seem to cover constant I/O latency, while the I/O problem we need to solve on our server occurs ...
  • 51
4 votes
2 answers
9k views

is CPU actually occupied during iowait (%wa in top) on Linux / EC2?

On an 8-way Amazon EC2 instance (running Linux 2.6.21) with 8 EBS volumes and a lot of disk traffic, we see high %wa in top (30-40%), and high load average (8-9). My understanding is that processes ...
user avatar
4 votes
2 answers
637 views

Can increased disk throughput increase CPU utilization?

We recently upgraded the storage in our data center and migrated all our production systems to it. We noticed that in one machine (an HP_UX IA-64 Oracle database server for SAP), the CPU utilization ...
  • 351
4 votes
1 answer
1k views

How can i view the disk latency of a ZFS pool, or of the disks in it?

I'm using FreeBSD. I would like to write a munin plugin to log it, but i don't understand how to get this data.
  • 690
3 votes
1 answer
3k views

How to combine different pidstat outputs (CPU, I/O, Disk Usage)

I would like to use pidstat to get both -u, -r and -d flags. Is there a way to do so? The idea, is to get both CPU, I/O and disk usage. Cheers Ps. Combining the flags as -urd doesn't work in Ubuntu ...
  • 203
3 votes
3 answers
1k views

Is slow IO on my cloud server causing my slow server restart issues?

I'm running two servers on Rackspace cloud one for the web app and one for the db and redis instances. The web server has 1Gb of ram and single core. Nginx sits in front of unicorn which is running 2 ...
  • 131
3 votes
1 answer
10k views

CentOS6 and long Wait IO time on jbd2/dm-0-8

Linux box running CentOS 6.5 and 2.6.32-431.3.1.el6.i686 kernel recently got CPU spikes up to 99% causing by Wait IO. Executing while true; do date; ps auxf | awk '{if($8=="D") print $0;}'; sleep 1; ...
3 votes
4 answers
1k views

MySQL on VM, High IO Wait but low TPS

We are trying to figure out a strange situation on our MySQL installation on VM. We are on Mysql 5.7 and RHEL 7 with XFS. We are observing when we run a few select queries the IOWait races up to 40-...
2 votes
2 answers
11k views

What iowait values are ok?

I am trying to find the bottleneck of a server running a fairly busy php/mysql site. My first culprit was io but iostat shows that on average iowait consumes only %3.60 of cpu time. here is the ...
  • 3,047
2 votes
1 answer
820 views

RHEL8: Does NFS I/O count as part of "%iowait" calculations?

Customer telling us that they see high %iowait in top and iostat output on a busy RHEL8 NFS client. iostat man page clearly says that iowait is related to % of time kernel is waiting on "disk&...
  • 141
2 votes
2 answers
4k views

Drawbacks of setting a high I/O timeout?

I'm working on several Linux VMs whose partitions are mounted on a NetApp NAS. This NAS periodically experiences a very high iowait which causes the VM disks to switch to read-only mode, crash, or be ...
  • 1,055
2 votes
2 answers
223 views

Would software RAID0'ing SSDs help or hinder maximization of useful CPU work throughput?

My question is in the title... here is some background: [OS is Linux.] [UPDATE: The contents of this RAID0 is redundant (sync'd from SCM). I am not worried about the increased risk of data loss.] ...
2 votes
3 answers
1k views

MySQL on ubuntu very high IOWAIT

Can anyone advice what is going on with my setup. I have mysql server on ubuntu which periodically produce very high iowait. for long time up to 10-20 minutes. During this time system and database ...
2 votes
3 answers
1k views

Apache 2 handling requests very slow after Restart

We have a GPS device socket handling server using NodeJS which sends the received HTTPS request to Apache installed server and saving the data into MySQL Database. When we restart NodeJS or Apache ...
  • 23
2 votes
1 answer
1k views

Debugging I/O issues on a linux server

What are some ways to debug I/O issues on a linux server? I've been using: # nohup top -b -d 10 > /var/log/top.log & # nohup iotop -b -d 5 -o -t > /var/log/iotop.log & PS: hardware is ...
2 votes
1 answer
2k views

no disk io but iowait very high

there is no disk io going results of iotop Total DISK READ: 0.00 B/s | Total DISK WRITE: 0.00 B/s TID PRIO USER DISK READ DISK WRITE SWAPIN IO< COMMAND 1 be/4 root 0.00 ...
  • 21
2 votes
1 answer
746 views

IOwait peaks blocks the whole server for 1-3 minutes

On my Debian VPS, I am getting peaks of IO wait, which blocks my server for a few minutes. On the screenshot, you can see the Idle (green) which suddenly drops for IOwait (pink), which eventually ...
  • 275
2 votes
1 answer
667 views

Read IO Wait Increase

While troubleshooting a performance issue on a Debian Linux box running ext3 on software RAID 1 I redirected the output of a process writing useless temporary log files to /dev/null (ln -s /dev/null/ ...
2 votes
1 answer
2k views

IO-intensive processes hang with iowait, but no activity going on

I have a bunch of IO-intensive jobs, and to boost performance, I just installed two SSDs in a compute server, one as a scratch file system, one as swap. After running for some time, all my processes ...
  • 21
1 vote
4 answers
6k views

Resolving high iowait on Samba server

We've got a centralised file cache which a cluster of web servers use to store "heavyweight" pages. Each webserver uses Samba to mount this shared area. We're getting a lot of iowait on the server, ...
  • 1,446
1 vote
2 answers
237 views

CentOS 5 extremely slow when is running a download

When my server is downloading a file or executing other IO operations the other server functions become extremely slow (almost locked). Even other functions that do not use the network. When a ...
1 vote
3 answers
7k views

MySQL and high jdb2 io/wait

I am running MySQL 5.5 with InnoDB tables. There are about 200 queries per second. There are also tables with 500 000 or more rows. But I have big issues with server load and io/wait especially with ...
  • 29
1 vote
2 answers
2k views

High Iowait + High load average on a monitoring server

I've a nagios server which was perfectly working up to a few days ago. I've stopped it and restarted it to increase its RAM, and since then, iowait increased dramatically on the server (more than 20%, ...
  • 115
1 vote
1 answer
604 views

How to manage iowait over cifs?

For backup purposes we have Cifs file Server running that contains encrypted containers for backing up the more sensitive data. The container is mounted with cryptsetup and loop as a local filesystem ...
  • 23
1 vote
1 answer
1k views

Monitoring/Warning for IOWait

I had a downtime today on my server because of a high IOWait. I could not do mostly anything on the server, I managed only to run top to see the IOWait, but I did not have iotop installed at that time,...
  • 275
1 vote
1 answer
346 views

Why does md resync cause no iowait?

I've known the defintion of %iowait as Time the CPU spent idling/blocking, because it had to wait on an IO operation to complete. Now take a simple Linux system with an mdraid1 and two disks. /...
  • 168
1 vote
1 answer
3k views

Copying cause big iowait

I have dedicated server with 3 HDD. System disk, backup disk (same as system disk) and data disk. When I copy lot of data with cp (say, between backup disk and data disk) load average goes sky high. ...
  • 53
1 vote
0 answers
765 views

What is a "normal" latency/performance of inserts on mariadb or mysql?

Hadn't used MySQL family for years. Now need to move from OracleDB to either MariaDB or PostgreSQL. So on a similar "hardware" (Openstack VM: RHEL 7.3, 4HT cores of 2.5GHz Xeon E312xx, "local" SAN &...
  • 111
1 vote
0 answers
604 views

Issues with sudden high iowait on RabbitMQ Cluster

we run a RabbitMQ Cluster with 6 c5.2xlarge nodes on AWS. The Version is 3.6.14 on Erlang 19.1. They run in docker containers and mount a local volume to /var/lib/rabbitmq/mnesia. The docker run ...
  • 11
1 vote
0 answers
1k views

qemu-kvm VM hangs at iowait with virtio drive

We are seeing instances of a VM hanging with the following symptoms: load average 800, processes stuck, CPU 100% in iowait reading files work, writing files hang the system RAM utilization is high, ...
1 vote
0 answers
164 views

high linux server load

I have a problem with high server load (load average from 10 to 100+) when there is a lot of web traffic. There are also noticable delays when reading/writing files. Machine is a Apache web server. ...
  • 137
1 vote
1 answer
94 views

What happened to hard disk

I suspect that with my hard drives is something wrong. iostat -d -x -m root@srv2:~# iostat -d -x -m Linux 2.6.32-5-amd64 (srv2.host.com) 04/23/13 _x86_64_ (2 CPU) Device: ...
  • 285
1 vote
0 answers
368 views

Why does nginx 1.0.0 appear to not use the kernels open file cache on Gentoo? (high iowait)

free -m total used free shared buffers cached Mem: 7996 2043 5952 0 73 140 -/+ buffers/cache: 1830 6165 ...
  • 119
0 votes
2 answers
1k views

mysqld causing machine to lockup with high iowait

I've inherited a LAMP server running centos, which randomly hangs from time to time despite low web traffic. I've installed Server Density to monitor what's going on, and have found during a hang the ...
  • 178
0 votes
1 answer
731 views

One process takes all server resources (iowait?)

I have multiple processes running simultaneously on the same box under CentOS 7 (each one on behalf of the separate Linux user). I use Zabbix for monitoring. Sometimes the following pattern ...
  • 113
0 votes
2 answers
109 views

High iowait, swap nearly always fully in use

I'm often experiencing slowness on a large server that is used by several users simultaneously, with many CPUs (72) and a decent amount of ram (125gb). Granted, the server runs a lot of stuff so gets ...
  • 101
0 votes
1 answer
2k views

Tracing down High Linux Load - HDD at fault or too many interrupts ? (ksoftirqd Time 437:44.13)

Server Stats: "cat /proc/version" Output Linux version 2.6.18-308.24.1.el5 (mockbuild@builder17.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-52)) #1 SMP Tue Dec 4 17:43:34 EST 2012 ...
  • 202
0 votes
3 answers
740 views

JVM hangs with 100 IO wait on Ubuntu Maverick (AWS EC2 xlarge instance)

Asked this question on on stackoverflow, reposting: Have an xlarge instance in AWS running 9 Tomcats with heaps from 256M to 4G. With Ubuntu 10.04 the box sporadically hangs for a few hours with huge ...
  • 11
0 votes
3 answers
3k views

100% iowait + drive faults in dmesg

I have a server on which resides a fairly visited web app. It has a raid1 of 2 HDDs, 64MB Buffer, 7200 RPM. Today it started throwing out errors like: kernel: ata2.00: exception Emask 0x0 SAct ...
  • 1,144