Hot answers tagged

23

Honestly your best approach here is to either fix the power-cuts, or deploy a different system in a better location. Yes there are systems such as redis which will store data in an append-only-log for replay, but you risk corruption at lower levels - e.g. if your filesystem is scrambled then the data on disk is potentially at risk. I appreciate any ...


13

This is stupid. I find myself answering my own question again. It says, Leaving filesystem unchanged. suggesting that nothing is changed. This actually mean that typing 1 followed by pressing enter inside the fsck prompt did not work. Anyway, the following does work: # fsck.vfat -v -a -w /dev/sda1 The above command automatically write changes to ...


12

xfs_db has an option blocktrash which Trash randomly selected filesystem metadata blocks. Trashing occurs to randomly selected bits in the chosen blocks. This command is available only in debugging versions of xfs_db. It is useful for testing xfs_repair(8) and xfs_check(8). For example xfs_db -x -c blockget -c "blocktrash -s 512109 -n 1000" /dev/...


12

Perhaps try ntfsfix (or similar program) in your favourite linux (or knoppix). Usually located in package 'ntfsprogs'. Sometimes trying to delete from Linux (with ntfs-3g or other ntfs-write-support) helps. Knoppix (Linux-Live-CD/DVD): http://www.knoppix.net/


11

Your approach can work. Let me suggest some enhancements to it. There was a question in stack overflow on atomic writing to file. Essentially you save each packet of data to a temporary file and then you rename it to it's final name. Renaming is an atomic operation that will be safe from power failures. That way you are guaranteed that all your files in your ...


10

Always run: "cd /filesystem; find . -type f -exec md5sum {} \; >& /filesystem-md5.log" and then "cd /filesystem-new; md5sum -c /filesystem-md5.log" before and after copying a large amount of data. You'll be surprised how much random data corruption you experience in the real world. When you find a corrupt file, "cmp -l badfile goodfile" to attempt to ...


10

Very simply: the universe (and/or Murphy) is punishing you for continuing to use a thoroughly obsolete and insecure protocol like FTP. When you connect to your web server using FTP, your username and password are sent unencrypted, in the clear, for everyone and Big Brother to read. So are the contents of all the files that you upload or download using FTP. ...


9

Pulling the power causes everything to stop in flight, with no warning. kill -9 has the same effect on a single process, forcefully terminating it with a SIGKILL. If a process is killed by kernel or power outage, it doesn't do any clean-up. That means you could have half-written files, inconsistent states, or lost caches. You usually don't have to worry ...


8

CHKDSK (autochk.exe and chkdsk.exe, which share dll's) runs in a few stages (the below is for NTFS volumes, and a bit simplified): Stage 1 - "Verifying Files". In this stage it examines the Master File Table (MFT). The MFT stores file attributes about each file and directory such as the file name, creation date, time stamps, etc. in File Record Segments (...


8

COM2 is a reserved keyword, quite literally for the hardware port COM2 and I'm not sure how that file got there in the first place, since Windows won't let you create a file called COM2.log. Try it now, you'll get an error message. For fun, also try creating LPT1.txt. According to Naming Files, Paths, and Namespaces, the following are all reserved names and ...


8

Remount it as READ-ONLY and copy your data of NOW. That is the only sane thing to do with a bad drive. Any attempt to keep using it is madness.


7

cat DB1.myd /dev/random > DB2.myd


7

In my experience, each file type needs its own checks to determine if something is indeed corrupt. Data is just dots and dashes at its heart, and what determines "corruption" is entirely file dependent. You will need to determine what file types are most important, and then determine if it is reasonably possible to create automation that checks for the file ...


7

I've been troubled by OP's installer message on my laptop for a bit, but got it to run a few minutes ago. In addition to the lines suggested by John Zablocki I added lines for the installers in Wow6432Node since it's a 64 bit machine, even if I aren't sure it's needed. The most important key is however the last one, the UpgradeCode left behind by an old or ...


7

You could use the block2mtd driver to use the transactional jffs2 or yaffs(2) filesystems you seem to be employing elsewhere for your SD card, which would solve your problem of data loss or filesystem corruption at poweroff. Doing so might incur other problems, though. As the SD card is likely to have own mechanisms for wear levelling and sector remapping ...


7

This sounds like someone or something deleted the accounts, and then restored them. (Picture an admin saying "oh shit" - we've all been there.) This is the same kind of behavior you'd see when you restore/reanimate objects that have been deleted, back in the old days before the AD Recycle Bin. The object is restored with a blanked password and is disabled as ...


6

Orphaned inodes are benign and perfectly normal whenever you have an unclean dismount. They are simply files that had been deleted, but were still open when the fs was remounted read only. They are not the cause, but merely a symptom. You need to check your kernel logs to see what the actual problem was that caused the read only remount. You also might ...


6

The write cache has usually nothing to do with the BIOS, mostly there is no option for switching disk cache settings in there. With linux, using hdparm -W 0 should help. The setting is persistent, so if you don't have hdparm to play around with in your production systems, you should be able to disable the disk write cache on a different system and replug ...


6

Listing the directories does not mean that the filesystem is ok, you're just viewing metadata (which is a small percent of a filesystem). Create a backup of everything you can Run a filesystem check (fsck) Create another backup (in case fsck managed to save some more files) Replace the disk


6

You install a UPS or a RAID card with a battery-backed write cache to the system, and for as little as $49.95, you accomplish what is simply impossible to accomplish in software alone. Your claim that it's somehow not possible to hook this server up to a UPS or battery... is simply not believable.


6

You could try using -i which will present you with a question rm -i 1386141318.M854059P* rm: remove regular file '1386141318.M854059PΈ3Ώ62.email.vantagetitle.com,S?11304,W11473:2,Sc'? and answer y You may be able to delete the file using it's inode number. use ls -li to get the inode number of the file. Once you have it, use find to delete it find . -...


6

I'm not sure what might cause the corrupt packet problem that drops your connection, but you might find rsync's --partial or --partial-dir option helpful when transferring large files so that when you restart the transfer it will continue where the transfer left off instead of having to start over transferring the whole file again: --partial-dir=.rsync-...


6

The (sqlite) database used by pkg(ng) is corrupted. You should be able to restore the pkg db file using its daily backup: have a look at /var/backups/ # ls -lrt /var/backups/pkg* -rw-r--r-- 1 root wheel 16462211 May 3 03:01 pkgdb.bak.tbz -rw-r--r-- 1 root wheel 16462853 Apr 16 03:01 pkgdb.bak.tbz.2 -rw-r--r-- 1 root wheel 17824768 May 3 03:02 ...


5

You can keep track of the repair progress by watching the contents of your data directory; you'll see tables with names starting with # which are used during repair. You can also check SHOW PROCESSLIST to make sure it's not doing a "Repair with keycache" because that is much worse than "Repair by sorting" Repair is very slow with large tables; you should ...


5

The cable should be ruled out by the network protocol - I'm not that deep into tcp, but would expect that tcp handles cable errors through some kind of checksums, so that corrupt packets are detected. Disintermediate the cabling theory and (almost) everybody on the outside: Test downloads through https. This way you can be sure that the problem is either ...


5

In an unexpected shutdown, the only files which should be corrupted are files which are open for writing. On most systems at any given instant in time, you're probably not writing to a file. Probably. 1 kill -9 is POSIX SIGKILL and is implementation dependent. The process that receives this signal will not be given an opportunity to handle it. 1 ...


5

You can force a check of (eg) md0 with echo "check" > /sys/block/md0/md/sync_action You can check the state of the test with cat /sys/block/md0/md/sync_action while it returns check the check is running, once it returns idle you can do a cat /sys/block/$dev/md/mismatch_cnt to see if the mismatch count is zero or not. Many distros automate this ...


5

For the improper shutdown, you should be able to test using virtual machines. Write a script to load up all your applications and services. Then use an API to kill the VM and then boot it back up. Repeat... let your script run and see how long it takes to die. For kernel errors: find an old beta kernel, eg from early Reiser or XFS or EXT4 days. Look through ...


5

I assume you are using MyISAM storage engine which is not transactional, so the risk of data loss is implied. But repairing a corrupted MyISAM table is a standard procedure, so you should not lose more than several latest records (usually one or two).


5

The write cache suggestion is a good start but this sounds like an architectural design flaw. On an embedded system the internal flash should probably NOT be mounted R/W except in rare circumstances. You should really be doing most of the work in a memory filesystem and syncing changes back to the RW flash upon some user command or regular interval. It is ...



Only top voted, non community-wiki answers of a minimum length are eligible