Questions tagged [io]

IO (or I/O) is an abbreviation for input/output, most commonly referring to disk input/output.

Filter by
Sorted by
Tagged with
0
votes
0answers
7 views

Simulate some process IO with fio or other tool

Is there any way to track specific process and get stats for disk IO such as queue death, total reading/writing threads, percent of reads/writes and so on? Main goal is to use all that information to ...
1
vote
1answer
30 views

Interpreting SMART Logs of Newly Installed NVMe RAID0 Crashing Everyday

An Ubuntu 20.04 system has been stable for a year until a 2nd and 3rd NVMe drive is installed on the motherboard to form a 2x1TB RAID0 array. Ever since then, there is huge amount of IO load on this ...
-2
votes
0answers
24 views

Why are random reads on NVME SSD slower than sequential reads?

NVME flash SSDs can operate on data in parallel. So why is it that random IO is still slower than sequential IO on SSDs?
0
votes
0answers
11 views

Interpreting /proc/diskstats - IO time magnitude higher than sum of read and write times

This question is an extension of the existing: Interpreting read, write and total IO time in /proc/diskstats Original documentation of the diskstats I found: https://www.kernel.org/doc/html/latest/...
0
votes
0answers
16 views

Getting high read await in Zabbix monitoring, is this a common behavior for RAID5 disk array?

We are running a Oracle 11.2 DB on Redhat 6.8, and disk array is configured as RAID5, which is using a Lenovo 730i RAID adapter. I checked RAID policy and found that "Read Ahead Always" and &...
2
votes
0answers
66 views

kvm virtualization, slow io in windows guest (with virtio drivers) on ubuntu host and HP 360 Server with P440 RAID controller

Problem: Very slow IO in Windows Guest vs Host Hardware: HP 360 G9, 2 x 10 cores Xeon CPU, 128GB Ram, 4 x 4TB SAS HDD, Raid 10, P440 Controller (with 2GB cache and battery) Host: Ubuntu 20.04 Server, ...
0
votes
1answer
68 views

disk read io very high, but no process perform io read

We have experienced a very strange problem. The load of the server machine is very high due to high disk read io. But the processes running on this server do not perform any disk read operation. We ...
0
votes
0answers
12 views

ionice: “none: prio 4” vs “realtime: prio 4”

If run a process using ionice -c realtime processname I see: # ionice -p 4472 realtime: prio 4 On the other hand, neighbor process has: # ionice -p 4004 none: prio 4 The question: Is realtime 4 ...
2
votes
1answer
57 views

About IOPS of a RAID5 disk array without cache

we are using a 530-8i RAID adapter,which is none-cache supported, on a Lenovo ThinkSystem SR590 server.And we have 4 HDD(600GB/10K RPM/SAS) in the array. I read an article about calculating the disk/...
0
votes
1answer
40 views

It is possible that a process can not link a file, but I can with sudo for the same user?

I am working with Dovecot 2.3.4.1 and EncFS (encfs version 1.9.5) encrypted mailboxes on Debian 10 (no SELinux) Basic setup is: EncFS mounted with encfs --public /var/encrypted-mail /var/decrypted-...
2
votes
0answers
44 views

Access disk io data in linux without tools like iotop/iostat to find processes or files with IO

We have regularly high load average. With the help of a zabbix agent, I could narrow it down to be mainly because of cpu iowait (and lastly disk io). I am not allowed to install any additional ...
0
votes
1answer
45 views

ZFS raidz3 on proxmox having IO issues

I'm running Proxmox 6.2-11 with 256GB ECC RAM and the following raidz3 pool: pool: rpool state: ONLINE scan: scrub repaired 0B in 0 days 15:25:14 with 0 errors on Sun Jan 10 15:49:29 2021 config: ...
0
votes
0answers
24 views

High %wa from top command while runing PyTorch

Everything was fine yesterday, but I noticed high dataloading time this morning. How do I go about trying to figure out how to fix this? top: process state: iotop: iostat:
0
votes
1answer
30 views

Does add more memory RAM decrease I/O wait?

My server is Ubuntu KVM and has 2 cores, 4GB RAM, and I use HDD, at moment I can't swap HDD to SSD. I found this answear (https://unix.stackexchange.com/a/463819/351968) saying that adding more memory ...
-2
votes
1answer
30 views

SSD I/O slows down system

My HDD was getting full, so I added a new SSD and tried moving files to it. For some reason, this bogs down the whole system to the point where it's not usable. This never happened with HDDs when ...
0
votes
1answer
268 views

Looking for a NBD C/S for windows (Network Block Device) [closed]

Where can I find a NBD Server (and also a Client) that I can use in windows 10? (preferably FOSS source code for inspect/change/compile/...) Been looking for a while on the internet and seems that no ...
0
votes
1answer
34 views

Is RAID 6 a good choice regarding performance on server used for computation?

We are operating a server for doing computations with large data sets (100GB - 5TB) for a small group of researchers (10-20 people). At the moment we are running 4x8TB spinning drives in a RAID 6 ...
0
votes
0answers
62 views

Interpreting /proc/diskstats values to identify potential I/O bottleneck

I'm investigating some semi-regular unit test timeouts on our test runners, with test cleanup (close temp database, remove temp directory) taking well over 10s. I can reproduce the issue when using a ...
0
votes
1answer
291 views

Fio brandswitch measure

In disk benchmarking tool fio there is statistics for brandswitch Run status group 0 (all jobs): READ: bw=20.9MiB/s (21.9MB/s), 10.4MiB/s-10.8MiB/s (10.9MB/s-11.3MB/s), io=64.0MiB (67.1MB), run=...
2
votes
1answer
71 views

MYSQL vs. InnoDB [closed]

mysqltuner output. -------- Storage Engine Statistics ----------------------------------------------------------------- [--] Status: +ARCHIVE +BLACKHOLE +CSV -FEDERATED +InnoDB +MEMORY +MRG_MYISAM +...
1
vote
0answers
29 views

disk I/O latency spikes (contains log files from amplify)

Ubuntu 18.04 LTS server, disk is ssd. Application stack is: mongodb, node, nignx and some python. Server is up 1 month and already had 3 OS disk fails where Ubuntu would go in read only mode. After ...
0
votes
0answers
60 views

What are the best practices for device-mapper multipath?

Background: We are setting up the new storage for an HPC Compute cluster for applied statistics, bioinformatics, and genomics. Contrary to our expectations, an active-active configuration did not ...
4
votes
2answers
554 views

Use F2FS on RAID0 within HDD?

My server contains 128 monitors and that caused many random write IO be submitted. If I use SSD, many writes will make it broken soon. I read some articles about F2FS and know it works better on flash ...
0
votes
0answers
42 views

Fetch leading to i/o timeout during get

I'm trying to set up solr and I'm getting the following errors when running docker-compose build on a DockerFile with the command RUN python3 manage.py build_solr_schema: error: could not load from ...
0
votes
1answer
137 views

Low performance on file copy operations due large idle times (rsync, cp, ..)

I'm currently copying large directory tree from an external HDD to to internal storage of my server. The performance of the copy operation is basically quite good, i.e. 35MB/s with the external HDD ...
1
vote
0answers
54 views

Mysterious, intermittent Read I/O on Ubuntu 18.04 based EC2 server - iotop shows no disk reads

I have an EC2 server based on Ubuntu 18.04. With CloudWatch detailed monitoring enabled I'm seeing spikes of Disk Read Operations that last less than 1 minute (shown as 1-minute spikes on the graph) ...
1
vote
0answers
48 views

PHP high I/O after a DDoS attack

I've had a large scale DDoS attack on my website. After I've enabled captcha challenge through CloudFlare and the attack stopped, PHP's I/O was still loaded at 99%, and the website was working very ...
0
votes
0answers
172 views

How does avgqu-sz size in iostat output relate to r_await and w_await?

Can someone explain what does avgqu-sz mean in the output of iostat? The manual page states like this. avgqu-sz: The average queue length of the requests that were issued to the device. Can ...
0
votes
1answer
46 views

cat an file programs fail to acess a file but vi works properly (Debian)

I have getting some problems with my system after a bad reset via ctrl+alt+del. Since then errors related to ACPI and ata6 are reported. Trying to get a solution to this (in a savage way:I am n00b), I ...
1
vote
0answers
539 views

Huge IO performance degradation between proxmox ZFS host and WS19 VM

For more than a week, I am trying to determine the reason for the following IO performance degradation between proxmox host and a Windows Server 2019 VM(s). I have to ask for your help guys because I'...
3
votes
3answers
1k views

Linus/ext4/nvme crashes during high io

During mvn compilation, I have random crashes. The problem seems related to high IO and in kern.log, I can see things like: kernel: [158430.895045] nvme nvme1: controller is down; will reset: CSTS=...
3
votes
1answer
246 views

High CPU Load and High I/O wait time : Mysql process

i am fairly new to the role and the technology. we have a Network server that receives the telemetry data from sensors and temeletry data is stored in MYSQL dabatase. we have a front end portal which ...
0
votes
1answer
144 views

nginx + php-fpm extremely high amounts of data written to disk

We are facing a problem with extremely high data writes to disk and huge amounts of traffic over loopback interface. Environment: Debian 10.3 (4.19.0.8) - virtualized in hyper-v nginx/1.14.2 php-fpm ...
1
vote
2answers
3k views

RAID 50 vs RAID 10 for performance?

I have a server with 6 SSD's and a raid controller card that supports both RAID 10 and RAID 50 the plan is to use this as our Build server. It will be pulling in NPM Packages and many small code files,...
0
votes
1answer
54 views

jdb2 spike every hour for few seconds

I get IO hang up every hour on the exact same time on my Ubuntu 18.04 server. iotop indicate that jbd2 spike to araong 30% around this time. The hangup last only few seconds. I started noticing this ...
1
vote
1answer
1k views

What is Split IO?

I'm trying to understand the meaning of split io/sec counter for physical disk in Windows Performance monitor and don't quite get the reason why the single IO operation might be split into several. ...
0
votes
1answer
41 views

Limitations to using many SSDs in computing server [closed]

This is a beginner's question: We have a powerful computing server for roughly 10 users which will be increasingly used for data processing. In the past the main bottleneck was I/O. I was wondering ...
17
votes
2answers
2k views

Total I/O cost of a process

I know that iotop lets me measure the disk bandwidth used by any or all processes in real time. iotop -a measures the accumulated disk bandwidth, which is closer to what I want. However, when the ...
0
votes
1answer
60 views

Is it considered “safe” to run a task with high I/O on a server for 3 days straight, or would you add pauses in it?

I want to hash the contents of 4 million JPEG files taking up 32TB of a 48TB RAID6 partition in CentOS 7. It's a Hetzner SX131 10 x 6TB box. Having run a test on 10,000 files I think the full job is ...
0
votes
1answer
2k views

nvme queue depth vs fio iodepth arg relation/explanation

This question is related to fio (flexible i/o tester) utility manages I/O queues for a NVME storage (SSD's in particular) whilst using libaio engine. For testing i am using Ubuntu 14.04 and a ...
3
votes
1answer
1k views

How to accelerate metadata (getattr) requests on an NFS file server for fast locally-cached reads?

We have a storage server with a 12 disk RAID-6 pool of 100 TB. It is used by several client compute nodes via NFS, mostly running deep learning training (lots of images ~100 KB, loaded in random order)...
0
votes
1answer
61 views

Has anyone ever considered using a second partition on a disk in a ZFS mirror when that disk is known to be faster (SATA+NVMe)?

I was reading this question where I felt obligated to comment and poison the sanity of system administrators everywhere. Then I had an interesting idea. Lets say that you have a laptop with a one ...
0
votes
1answer
314 views

GCP App Engine Flexible Environment Slow Disk I/O

Background/Overview: We have a Python Web Application with three environments in two projects staging/dev and production. These are hosted in GCP using Google App Engine Flexible Environment, Cloud ...
8
votes
1answer
2k views

Dmesg full of I/O errors, smart ok, four disks affected

I'm working on a remote server (Dell Poweredge) that was a new install. It has four drives (2TB) and 2 SSD's (250 GB). One SSD contains the OS (RHEL7) and the four mechanical disks are eventually ...
1
vote
0answers
25 views

Can't scale write bandwidth on Linux with Intel Optanes

I have a computer with 3 Intel Optane drives and iotop shows that sequential write to each one separately achieves ~2.2 GB/sec. I suppose that simultaneous writing on all three would result in ~6.6 GB/...
1
vote
1answer
124 views

Bandwidth and IO issues when running hardware raid and zfs

I've inherited a setup using Linux Ubuntu 14, a megaraid_sas, and ZFS. We're getting some performance problems (never being able to fully utilize the 6GB bandwidth from the raid) and I'm curious if ...
4
votes
2answers
121 views

What units are the I/O measures in GNU “time” in?

GNU time has optional display of I/O measurements: TIME="%I:%O" /usr/bin/time cp filea fileb 0:5488 but what units is it measuring? Any ideas? The manual only says %I Number of filesystem ...
1
vote
0answers
37 views

Azure Ubuntu VM - Unexpected short IOPS peaks (bursts)

I am not sure if such issue is well-known, I failed to find any straightforward explanations. I have an ordinary VM on Azure of a small size (B1s) running a Ubuntu 18.04 from Canonical. There's a ...
2
votes
2answers
57 views

Which process writes/reads the most from disk over time? [closed]

I want to know which process writes/reads the most from disk over time, I mean that I will start the monitoring process than after X time I will be able to see what was the process that used the disk ...
1
vote
0answers
561 views

What could cause Nginx to write this much I/O?

I have a high-traffic webserver (up to ~50 page loads per second) running nginx, which passes HTML requests to a PHP-FPM backend and serves static assets directly. Under this load, I get a curiously ...

1
2 3 4 5
8