Questions tagged [data-integrity]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
1 answer
112 views

Linux file system(s) with ACID guarantees? [closed]

What could I do, to obtain a Linux file system, with the same guarantees as ACID guarantees for databases? It seems file systems haven’t advanced to this point that I had assumed would be industry ...
user avatar
0 votes
1 answer
278 views

Linux server power outage - data integrity and OS corruption checks

What are the best practice steps to check for / recover from potential Linux OS corruption, in the event of unexpected power outage (or VM host failure)? Of course it "depends" on the ...
Jaans's user avatar
  • 175
0 votes
1 answer
431 views

Does zfs zfs recv validate checksums?

It seems that zfs send includes a checksum for all included data (stated for instance here). Is this checksum automatically verified by zfs recv? I would assume it is, but I cannot find any ...
joaerl's user avatar
  • 397
1 vote
2 answers
892 views

Ext4 metadata checksums: current feature status

The kernel wiki page for ext4 metadata checksums is marked "last modified on 22 October 2013". I couldn't find more recent status information on this feature, except for this question from ...
MichaelK's user avatar
  • 123
3 votes
3 answers
556 views

Is checking data integrity with hashes useful for my scenario?

I have a NAS as production system (mostly video footage and project files) and a 1:1 backup. I only need to retrieve the backup data in cases of emergency. Such a case didn't occur yet. Lately I'm ...
Martin's user avatar
  • 31
34 votes
7 answers
12k views

How beneficial are self-healing filesystems for general usage?

I have recently looked into advanced filesystems (Btrfs, ZFS) for data redundancy and availability and got interested in the additional functionality they provide, especially their "self-healing" ...
Prototype700's user avatar
1 vote
2 answers
803 views

How to test Snapraid data corruption?

I'm running Snapraid on my hard drives. However, I want to test it's data integrity features, to see how/if they actually work. But I don't really understand how to 'corrupt' my data. I can just ...
RazorHail's user avatar
  • 165
0 votes
1 answer
257 views

live server disk image backup data integrity

I've been looking for a way to do a disk image backup of a running linux server. There are several questions which answer this, like How to make a disk image, Mirror Live Linux (Debian) Server, Linux ...
1gd_1bs37coua9dd's user avatar
0 votes
1 answer
2k views

Logrotate with `copytruncate` "data loss" - is it serious?

Manpage of logrotate claims that using copytruncate option leads to "a very small time slice between copying the file and truncating it, so some logging data might be lost". Is it actually ...
Lapsio's user avatar
  • 363
0 votes
1 answer
791 views

MySQL Table Checksum - Before or after MySQL dump?

The company I'm currently working for has a script that runs at night that takes a MySQL dump of the production environment and imports it into the staging environment. This is a very basic setup - No ...
Justin's user avatar
  • 137
1 vote
1 answer
525 views

Where do I find the verified checksums for all files in all packages for Ubuntu?

I am looking to verify whether or not my binaries, installed by the Aptitude package manager under Ubuntu 16.04.2 LTS, are either in-tact or have been modified / compromised. I cannot find any list of ...
Brendan's user avatar
  • 83
0 votes
1 answer
886 views

How to repair a mailbox with public folders?

We have an Exchange 2013 organization with sort of legacy public folder set transferred via import/export from Exchange 2003 times. It has several linked mailboxes for our staff, and several dozens of ...
Vesper's user avatar
  • 794
2 votes
1 answer
243 views

How do you store VMs on HDDs with data integrity AND performance?

First off I know this topic easily gets subjective, but I'm trying to avoid that, as there should be at least 1 good answer to this in a sea of bad answers and it's hard to find. At first sight my ...
Alex's user avatar
  • 399
3 votes
1 answer
1k views

Ensuring VMware ESXi storage integrity?

I'm planning a small ESXi standalone server to host a number of VMs I use here. I haven't decided what facilities to use from the wider vSphere system. The underlying VM storage is local HDD RAID ...
Stilez's user avatar
  • 714
1 vote
2 answers
314 views

Do I need btrfs scrub if a lower layer also scrubs?

I have a multi-TB btrfs made up of virtual disk images that live on NetApp NFS filers. The data are redundant at the NetApp level, so the btrfs is RAID-0. In addition to providing redundancy, the ...
Reid's user avatar
  • 302
4 votes
1 answer
1k views

Drawbacks of a single drive split into partitions and partitions joined into a ZFS raidz1, vs. single drive ZFS with data duplication?

I'm looking for a minimal ZFS setup to achieve self-healing on a single device. Since I back up my data, drive failures are no concern, but bit-rot is. So far, the only suggestions I've been able ...
bz9js's user avatar
  • 41
0 votes
1 answer
135 views

Managing uploaded files on a server [duplicate]

I have a server hosting an intranet web site where one of the features will be the ability to upload files. The files will be saved in a restricted access folder and managed through the web back-end. ...
Big_Al_Tx's user avatar
  • 101
1 vote
2 answers
2k views

restore mongoDB databases from manual backup (not used mongodump): strange permission problems

Is it possible to restore databases which were manually copied from /var/lib/mongodb to eg. /home/foo/mongobackup ? I am not sure if the backup was made after stopping the mongod service. Either way, ...
nylki's user avatar
  • 203
1 vote
0 answers
393 views

Verify rdiff backup integrity

I am using rdiff for my differential backup. Recently an issue happened where all versions of my backup failed to patch correctly and rdiff returning an error 106 which after investigation is a code ...
Zaid Amir's user avatar
  • 179
9 votes
2 answers
2k views

What is a safe way to back up a sparsebundle that is exported via afpd?

I work with a number of OSX client machines that back up via TimeMachine to AFP shares on an Ubuntu Linux file server, exported by netatalk/afpd. These clients back up at arbitrary times of the day, ...
davidA's user avatar
  • 393
-1 votes
1 answer
82 views

Does RAID1 also checks for corruption? [closed]

I have two 1.5 TB disks, and I don't trust both of them because of weird noises and bad clusters. I was wondering if I put them in a software RAID1 array (mirrored volume in Win7) it will write my ...
Maestro's user avatar
  • 285
0 votes
3 answers
1k views

Which SSD to choose, when unexpected powerlosses are frequent?

I am working on a embedded system where there can be unexpected power losses. So far it has been using a Corsair F80 SSD with Windows Embedded Standard 7 (NTFS), and after a unexpected power loss ...
Walkingsteak's user avatar
17 votes
2 answers
16k views

How to get checksum data integrity on ext4?

On file systems like btrfs you can preform a scrub, which will go though all the data to see if the data still matches the file system checksum. I would like to check the data on ext4 if it is ...
Sandra's user avatar
  • 10.4k
2 votes
2 answers
157 views

Single file changed: intrusion or corruption?

rkhunter reported a single file change on a virtual server (netstat binary). It didn't report any other warning. The change was not the result of a package upgrade (I reinstalled it and the checksum ...
Michaël Witrant's user avatar
2 votes
1 answer
4k views

DB2 Integrity Checks and Exception Tables

I am working on planning a migration of a DB2 8.1 database from a horrible IBM encoding to UTF-8 to support further languages etc. I am encountering an issue that I am stuck on. A few notes on this ...
imthefirestartr's user avatar
4 votes
2 answers
2k views

Detecting data corruption so we're not backing up corrupt files

I've been thinking about data integrity, I currently backup about 2tb of data and always have one backup of data from a year ago. My concern is if a file became corrupt on our production file server ...
Sean Bannister's user avatar
0 votes
1 answer
2k views

Best practices for mass file transfer with verification?

I'm looking for a bit of advice on transferring a several files from Windows-based workstations to a Linux Server running Ubuntu Server LTS 10.04. At the moment I've been mounting the shares on the ...
toolshed's user avatar
  • 237
1 vote
3 answers
1k views

Best practices for backing up Windows workstations to Linux server?

I was wondering what strategies others in the community use when backing up their Windows-based workstations to Linux servers. At the moment I'm doing a bit of a test with Ubuntu Server LTS 10.04 and ...
toolshed's user avatar
  • 237
2 votes
2 answers
1k views

What happens to InnoDB and MyISAM tables if the power fails during a transaction?

We have two MySQL tables, one is InnoDB and the other is MyISAM. If I execute a chunk of inserts in a single transaction on both tables and the power fails in the middle, what will be the status? ...
Ash's user avatar
  • 21
1 vote
0 answers
84 views

Archival storage tool: fast, queryable, has api, reliable

I have large quantities of structured data coming in over the network from disparate sources, and that data is archived in the filesystem, parsed/analyzed, and eventually makes it's way into a ...
mikewaters's user avatar
  • 1,195
195 votes
6 answers
85k views

LVM dangers and caveats

I have recently started using LVM on some servers for hard drives larger than 1 TB. They're useful, expandable and quite easy to install. However, I could not find any data about the dangers and ...
Adam Matan's user avatar
  • 13.4k
6 votes
3 answers
6k views

Making SATA disk write cache safe

Supposedly (see, e.g., a question about it here), with NCQ enabled drives, the drive write cache is supposed to be safe, as in it doesn't lie to the OS about data being committed to the platters when ...
derobert's user avatar
  • 1,308
5 votes
2 answers
3k views

PCI Compliance: File Integrity Checkers [closed]

We're looking for a file integrity checker for our windows server environment. Anyone have any good suggestions?
I.T. Support's user avatar
3 votes
2 answers
4k views

Upload over HTTPS is reliable?

Sometimes we are facing a problem with corrupted files after upload through HTTP protocol. Today I’ve read an article that says that TCP protocol is not 100% reliable due to weak CRC algorithm and ...
Eugene's user avatar
  • 133
3 votes
5 answers
9k views

Is there any value in checking MD5 of files after they have been unzipped?

A project I am working on currently requires that the user runs an MD5 hash checking tool on the entire project, after it has been unzipped. They do not currently request that the ZIP itself is ...
Craig Mason's user avatar
0 votes
2 answers
1k views

SQL: Find Last Integrity Check Date

Is there a way to check when was the last integrity check run for each database? I know when the service startup it checks for these values, so they exist, and they are probably writen on the ...
Gabriel Guimarães's user avatar
7 votes
2 answers
8k views

It is XFS reliable? In case of a power gone off is xfs riskier that ext3 in data corruption/safey?

I have been having some electrical issues,mostly power going out suddenly for a few months,and although ups solve the thing mostly. But still i am worried about filesystem corruption,and data lost. ...
Abel Coto's user avatar
10 votes
3 answers
32k views

How to check PostgreSQL database integrity?

Is there a way to check PostgreSQL database(s) integrity and consistency? I know about SQL Server DBCC CHECKDB and wonder if there is something similar to PostgreSQL.
FooBar's user avatar
  • 653
0 votes
3 answers
48 views

How two or more guest OS's hosted in a virtual environment handles editing of single resource?

I know this sounds very general question. Consider 2 OS's running using virtualization say operating system level virtualization. How concurrency is handled if both hosted OS using same resource and ...
pramodc84's user avatar
  • 103
0 votes
1 answer
434 views

Why does DBCC CHECKTABLE report errors on non-existent pages?

When I run DBCC CHECKDB, I get a slew of error messages like this: Table error: object ID 2020918271, index ID 1, partition ID 72057594196590592, alloc unit ID 72057594190233600 (type In-row data), ...
Aaronaught's user avatar
1 vote
4 answers
11k views

How to verify integrity of a remote file transfer

Is there any way I can verify the integrity of a remotely transferred file? I am currently transferring a large amount of files and folders from an ftp server to another remote server using wget, but ...
user avatar
0 votes
2 answers
93 views

Is there a service I can run which will automatically test CPU/RAM/hardware in the background?

Is there any service which runs in the background, and does the job of prime95 (i.e. it tests server CPU/RAM for integrity now and again)? Regards, Shane. p.s. The reason I ask: we recently had some ...
Contango's user avatar
  • 1,150
0 votes
1 answer
829 views

Best way to add a new non-nullable column?

I'm a web developer, with admin rights on our SQLServer, and I need to add a non-nullable integer column (foreign key) to an existing table. I'm looking for the best way to create this column and ...
ThoseDarnKids's user avatar
5 votes
4 answers
555 views

Disaster Recovery/Sabotage Protection for a small business

I've been contacted by two partners in a small professional firm. They are concerned about their other partner and want to take some steps to be absolutely sure that the company's data and systems ...
Ward - Trying Codidact's user avatar
6 votes
4 answers
2k views

USB storage media - data integrity issues on linux

I've always been somewhat paranoid about verifying data backed up to removable media, so after copying stuff to a USB flash drive or portable HDD, I invariably unmount the drive, remount it and diff -...
timday's user avatar
  • 866
15 votes
6 answers
13k views

SATA Disks that handle write caching properly?

It's pretty common to see advice to disable the write cache on individual disks used for databases because otherwise some disks will acknowledge writes that haven't yet made it to the disk surface. ...
eas's user avatar
  • 268