Tag Info

New answers tagged

2

You are most likely having a disk problem. The disk is failing and one fairly common failure method is to have higher latencies due to increased number of retries on certain problematic areas on the disk, these areas when hit will cause a chain reaction of other IOs waiting on them and if there were multiple IOs to the affected area you'll see such a problem ...


0

The drive lost your data, you could ofcourse replace it but if you just want to continue using the drive (at risk of future data loss again) you can try to write zeros in that location. This is what a RAID setup would do for you, it would find it can't read, zero out the place and recover the data from the RAID structure and then rewrite it back into the bad ...


1

If as per your comment the error happens all the time on the same sector than it is a disk problem, for some reason the disk times out on writing there (based on the smart error log) and either it can't reallocate it or for some unknown reasons it decides not to reallocate. In a SAS disk you could use an explicit command to reallocate the sector, I'm not ...


0

You should configure udev to automatically set permissions. See http://unix.stackexchange.com/questions/23955/permanently-changing-the-ownership-or-group-of-lvm-volume (reported here) run udevadm info --query=all --name /dev/dm-1 to get volume group name (E: DM_VG_NAME=... line) and logical volume name (E: DM_LV_NAME=... line) of the partition (you may ...


1

The disk did not respond in time and was reset by the OS, it can mean many things but the two most common are: Media error -- some location(s) on the disk cannot be read from or written to Link errors -- Bad cable This specific error with no previous errors on other errors or increased latency may indicate the media error. You can use smartctl to see if ...


0

This is not a false alert. It shows disc latency in excess of 40ms. Whether or not that is a PROBLEM is your definition - and you should adjust things. Acceptable disc latency really depends on whether the systems get too slow and it is a broad topic and one that has to be defined by you. On secondary VMs I accept 50 or even 100ms at times, on my main ...


5

In many cases, enterprise storage frames actually employ three different kinds of disks- SSD, SAS and SATA (or LSAS drives which are SATA with a SAS controller), with the goal being to optimize the data being stored with the I/O it actually needs. In other words, data accessed frequently ends up living in the highest speed areas (flash) and data accessed ...


4

Many servers still use spinning disks with a SAS interface, but most (all?) servers you buy these days should have options for SSDs as well. The price comparison you're making is a little tricky - an enterprise SAS disk is definitely more expensive than most consumer SSDs, but an enterprise SSD is a whole lot more expensive than a consumer one - they're ...


9

Cost, capacity and reliability are factors for why SSD adoption hasn't occurred at all levels. SSDs cost more than SAS disks for a given capacity. But in general, servers don't actually come with a particular type of disk. Storage is something that is configured afterwards. Some background information: Are SSD drives as reliable as mechanical drives ...


0

I would make sure I have backups. boot the server from some live media, such as Ubuntu installer or sysrescuecd, and force a thorough fsck to that file system with e2fsck -f.


0

I would never rely on the HCTL numbers to be stable. The host may change by addition of a new host or detection of a new device that will expose a new host that was unused until that moment. The kernel may have slightly different timings which will cause different hosts to be detected in a different order and devices being removed and reinserted may get ...


0

This error doesn't affect the health of the drive itself. If you will move the drive to another chassis that doesn't have the link problem the drive will be fine. That is assuming that the link problems do not originate from the drive port itself. These errors mean that there is a problem in the link between the drive and the upstream port, if you have a ...


0

Try to use cat /proc/diskstats as it more predictable. If you automate monitoring with zabbix or nagios you can always calculate diffirence between previous value and new one. If i remember correctly iostat also use /proc/diskstats.


2

Citing the iostat manpage: The first report generated by the iostat command provides statistics concerning the time since the system was booted, unless the -y option is used, when this first report is omitted. Each subsequent report covers the time since the previous report. This means that the first read will show a low value because ...


1

I have seen something similar myself, although it was a few years ago. In my case it was problem memory and when copying data, there was some form of caching going on which I suspected was slowly using memory until it got to the problem memory and... boom! crashed computer. A memory test should identify this easily enough, or perhaps removing some memory ...


1

The reason you are in read-only mode is that: a) Your file system was found to be damaged, then b) Your OS tried to fix it at boot, but failed, so c) Dropped you into a read-only shell so that you can extract and backup your files without modifying original file system hence preventing you from damaging it further. If I were you, I would try to fix the ...


0

The problem here is you put the LVM (physical volume) into a partition. If it were on the device directly, you could simply do pvextend lvextend resize2fs In your case you will have to boot using a live-cd or such delete the /dev/sda2 partition and recreate with a bigger size then proceed as outlined above The safer way would however be to ...


3

Both previous answers are correct, I'd only like to emphasize one point - if the controller on the array is battery backed, you can shutdown immediately. Otherwise - you must "wait" for the controller's cache to flush to disks. This "wait" IIRC is mostly in the realm of 5 seconds at most, and probably just a few milliseconds in normal circumstances. In any ...


4

Once the server is halted, there's no need for you to do anything else. Depending on the external enclosure, you'll want to make sure it's powered on and the drives spun-up before you start the server... so the reverse of of the shutdown process.


6

Probably not. Once the operating system has written out its caches, closed the journal, and unset the dirty bit, as applicable for the filesystem in use, things are pretty much done. If the external disk array isn't caching, this is definitely true and you can just cut power. If it is caching, it would normally provide its own mechanism for clean ...


3

So, to figure this out I did the following. Take your block number, multiply by four and add one (130856866 * 4) + 1 = 523427465 This represents the sector reported as producing an I/O error. The block size being 2k, sectors being 512 bytes. The additional one extra accounts for the starting sector offset for the partition. To correlate with SMART, ...


0

Scale horizontally, not vertically. Instead of having one server do all of the work, setup many to do the same. They may not all host the same data, which allows a smaller set to access. Hadoop may be one way to do this. Another could be just in the way it's coded. putting a lot of little servers behind varnish would definitely be something that could help ...


1

When using mount you have to mount the partition not the drive itself eg mount -t ext4 /dev/sda1 /data


0

The answer is more complex because it's a subtle problem. Not all storage is created equal, and different types of work require different types of storage. The most common types of storage and their measurement characteristics are: Small block intensive random IO: typically databases, sometimes office file shares, the bottleneck for many of the workloads ...


0

You can use the following commands for disk monitoring: # sar -p -d 1 1 (if sar package is installed on your system), with sar also you can check your disk read/write in the history from the /var/log/sa/sar-01 file. and # iostat -d -x 3 3 where the last two numbers are interval in seconds and number of repetition and -d : Display the device ...


0

I haven't used parted, but with fdisk, if I want to be exact i have to change the units to cylinders or sectors (assuming same disk geometry). How about trying that and seeing if you get closer? You should be able to put a suffix on your number to indicate unit type. According to the man page it is "s" for sectors and "cyl" for cylinders.


9

Unfortunately it looks like we may not get to the bottom of what the application was, but to get some value from this incident, I wanted to create a reference answer. This is VMware and virtual layer management centric. A lot of admins are in segregated, and cannot get guest or storage access quickly, and this is for them :) ...



Top 50 recent answers are included