Questions tagged [data-integrity]
The data-integrity tag has no usage guidance.
46
questions
0
votes
1
answer
112
views
Linux file system(s) with ACID guarantees? [closed]
What could I do, to obtain a Linux file system, with the same guarantees as ACID guarantees for databases?
It seems file systems haven’t advanced to this point that I had assumed would be industry ...
anon
0
votes
1
answer
278
views
Linux server power outage - data integrity and OS corruption checks
What are the best practice steps to check for / recover from potential Linux OS corruption, in the event of unexpected power outage (or VM host failure)?
Of course it "depends" on the ...
- 175
0
votes
1
answer
431
views
Does zfs zfs recv validate checksums?
It seems that zfs send includes a checksum for all included data (stated for instance here). Is this checksum automatically verified by zfs recv? I would assume it is, but I cannot find any ...
- 397
1
vote
2
answers
892
views
Ext4 metadata checksums: current feature status
The kernel wiki page for ext4 metadata checksums is marked "last modified on 22 October 2013". I couldn't find more recent status information on this feature, except for this question from ...
- 123
3
votes
3
answers
556
views
Is checking data integrity with hashes useful for my scenario?
I have a NAS as production system (mostly video footage and project files) and a 1:1 backup. I only need to retrieve the backup data in cases of emergency. Such a case didn't occur yet.
Lately I'm ...
- 31
34
votes
7
answers
12k
views
How beneficial are self-healing filesystems for general usage?
I have recently looked into advanced filesystems (Btrfs, ZFS) for data redundancy and availability and got interested in the additional functionality they provide, especially their "self-healing" ...
- 443
1
vote
2
answers
803
views
How to test Snapraid data corruption?
I'm running Snapraid on my hard drives. However, I want to test it's data integrity features, to see how/if they actually work.
But I don't really understand how to 'corrupt' my data. I can just ...
- 165
0
votes
1
answer
257
views
live server disk image backup data integrity
I've been looking for a way to do a disk image backup of a running linux server.
There are several questions which answer this, like How to make a disk image, Mirror Live Linux (Debian) Server, Linux ...
0
votes
1
answer
2k
views
Logrotate with `copytruncate` "data loss" - is it serious?
Manpage of logrotate claims that using copytruncate option leads to "a very small time slice between copying the file and truncating it, so some logging data might be lost". Is it actually ...
- 363
0
votes
1
answer
791
views
MySQL Table Checksum - Before or after MySQL dump?
The company I'm currently working for has a script that runs at night that takes a MySQL dump of the production environment and imports it into the staging environment. This is a very basic setup - No ...
- 137
1
vote
1
answer
525
views
Where do I find the verified checksums for all files in all packages for Ubuntu?
I am looking to verify whether or not my binaries, installed by the Aptitude package manager under Ubuntu 16.04.2 LTS, are either in-tact or have been modified / compromised. I cannot find any list of ...
- 83
0
votes
1
answer
886
views
How to repair a mailbox with public folders?
We have an Exchange 2013 organization with sort of legacy public folder set transferred via import/export from Exchange 2003 times. It has several linked mailboxes for our staff, and several dozens of ...
- 794
2
votes
1
answer
243
views
How do you store VMs on HDDs with data integrity AND performance?
First off I know this topic easily gets subjective, but I'm trying to avoid that, as there should be at least 1 good answer to this in a sea of bad answers and it's hard to find.
At first sight my ...
- 399
3
votes
1
answer
1k
views
Ensuring VMware ESXi storage integrity?
I'm planning a small ESXi standalone server to host a number of VMs I use here. I haven't decided what facilities to use from the wider vSphere system. The underlying VM storage is local HDD RAID ...
- 714
1
vote
2
answers
314
views
Do I need btrfs scrub if a lower layer also scrubs?
I have a multi-TB btrfs made up of virtual disk images that live on NetApp NFS filers. The data are redundant at the NetApp level, so the btrfs is RAID-0.
In addition to providing redundancy, the ...
- 302
4
votes
1
answer
1k
views
Drawbacks of a single drive split into partitions and partitions joined into a ZFS raidz1, vs. single drive ZFS with data duplication?
I'm looking for a minimal ZFS setup to achieve self-healing on a single device. Since I back up my data, drive failures are no concern, but bit-rot is. So far, the only suggestions I've been able ...
- 41
0
votes
1
answer
135
views
Managing uploaded files on a server [duplicate]
I have a server hosting an intranet web site where one of the features will be the ability to upload files. The files will be saved in a restricted access folder and managed through the web back-end. ...
- 101
1
vote
2
answers
2k
views
restore mongoDB databases from manual backup (not used mongodump): strange permission problems
Is it possible to restore databases which were manually copied from /var/lib/mongodb to eg. /home/foo/mongobackup ?
I am not sure if the backup was made after stopping the mongod service. Either way, ...
- 203
1
vote
0
answers
393
views
Verify rdiff backup integrity
I am using rdiff for my differential backup. Recently an issue happened where all versions of my backup failed to patch correctly and rdiff returning an error 106 which after investigation is a code ...
- 179
9
votes
2
answers
2k
views
What is a safe way to back up a sparsebundle that is exported via afpd?
I work with a number of OSX client machines that back up via TimeMachine to AFP shares on an Ubuntu Linux file server, exported by netatalk/afpd. These clients back up at arbitrary times of the day, ...
- 393
-1
votes
1
answer
82
views
Does RAID1 also checks for corruption? [closed]
I have two 1.5 TB disks, and I don't trust both of them because of weird noises and bad clusters.
I was wondering if I put them in a software RAID1 array (mirrored volume in Win7) it will write my ...
- 285
0
votes
3
answers
1k
views
Which SSD to choose, when unexpected powerlosses are frequent?
I am working on a embedded system where there can be unexpected power losses. So far it has been using a Corsair F80 SSD with Windows Embedded Standard 7 (NTFS), and after a unexpected power loss ...
17
votes
2
answers
16k
views
How to get checksum data integrity on ext4?
On file systems like btrfs you can preform a scrub, which will go though all the data to see if the data still matches the file system checksum.
I would like to check the data on ext4 if it is ...
- 10.4k
2
votes
2
answers
157
views
Single file changed: intrusion or corruption?
rkhunter reported a single file change on a virtual server (netstat binary). It didn't report any other warning. The change was not the result of a package upgrade (I reinstalled it and the checksum ...
- 358
2
votes
1
answer
4k
views
DB2 Integrity Checks and Exception Tables
I am working on planning a migration of a DB2 8.1 database from a horrible IBM encoding to UTF-8 to support further languages etc. I am encountering an issue that I am stuck on.
A few notes on this ...
4
votes
2
answers
2k
views
Detecting data corruption so we're not backing up corrupt files
I've been thinking about data integrity, I currently backup about 2tb of data and always have one backup of data from a year ago.
My concern is if a file became corrupt on our production file server ...
- 751
0
votes
1
answer
2k
views
Best practices for mass file transfer with verification?
I'm looking for a bit of advice on transferring a several files from Windows-based workstations to a Linux Server running Ubuntu Server LTS 10.04. At the moment I've been mounting the shares on the ...
- 237
1
vote
3
answers
1k
views
Best practices for backing up Windows workstations to Linux server?
I was wondering what strategies others in the community use when backing up their Windows-based workstations to Linux servers. At the moment I'm doing a bit of a test with Ubuntu Server LTS 10.04 and ...
- 237
2
votes
2
answers
1k
views
What happens to InnoDB and MyISAM tables if the power fails during a transaction?
We have two MySQL tables, one is InnoDB and the other is MyISAM. If I execute a chunk of inserts in a single transaction on both tables and the power fails in the middle, what will be the status?
...
- 21
1
vote
0
answers
84
views
Archival storage tool: fast, queryable, has api, reliable
I have large quantities of structured data coming in over the network from disparate sources, and that data is archived in the filesystem, parsed/analyzed, and eventually makes it's way into a ...
- 1,195
195
votes
6
answers
85k
views
LVM dangers and caveats
I have recently started using LVM on some servers for hard drives larger than 1 TB. They're useful, expandable and quite easy to install. However, I could not find any data about the dangers and ...
- 13.4k
6
votes
3
answers
6k
views
Making SATA disk write cache safe
Supposedly (see, e.g., a question about it here), with NCQ enabled drives, the drive write cache is supposed to be safe, as in it doesn't lie to the OS about data being committed to the platters when ...
- 1,308
5
votes
2
answers
3k
views
PCI Compliance: File Integrity Checkers [closed]
We're looking for a file integrity checker for our windows server environment. Anyone have any good suggestions?
- 611
3
votes
2
answers
4k
views
Upload over HTTPS is reliable?
Sometimes we are facing a problem with corrupted files after upload through HTTP protocol.
Today I’ve read an article that says that TCP protocol is not 100% reliable due to weak CRC algorithm and ...
- 133
3
votes
5
answers
9k
views
Is there any value in checking MD5 of files after they have been unzipped?
A project I am working on currently requires that the user runs an MD5 hash checking tool on the entire project, after it has been unzipped. They do not currently request that the ZIP itself is ...
- 133
0
votes
2
answers
1k
views
SQL: Find Last Integrity Check Date
Is there a way to check when was the last integrity check run for each database?
I know when the service startup it checks for these values, so they exist, and they are probably writen on the ...
7
votes
2
answers
8k
views
It is XFS reliable? In case of a power gone off is xfs riskier that ext3 in data corruption/safey?
I have been having some electrical issues,mostly power going out suddenly for a few months,and although ups solve the thing mostly.
But still i am worried about filesystem corruption,and data lost.
...
- 81
10
votes
3
answers
32k
views
How to check PostgreSQL database integrity?
Is there a way to check PostgreSQL database(s) integrity and consistency? I know about SQL Server DBCC CHECKDB and wonder if there is something similar to PostgreSQL.
- 653
0
votes
3
answers
48
views
How two or more guest OS's hosted in a virtual environment handles editing of single resource?
I know this sounds very general question.
Consider 2 OS's running using virtualization say operating system level virtualization. How concurrency is handled if both hosted OS using same resource and ...
- 103
0
votes
1
answer
434
views
Why does DBCC CHECKTABLE report errors on non-existent pages?
When I run DBCC CHECKDB, I get a slew of error messages like this:
Table error: object ID 2020918271, index ID 1, partition ID 72057594196590592, alloc unit ID 72057594190233600 (type In-row data), ...
- 492
1
vote
4
answers
11k
views
How to verify integrity of a remote file transfer
Is there any way I can verify the integrity of a remotely transferred file? I am currently transferring a large amount of files and folders from an ftp server to another remote server using wget, but ...
Rafael
0
votes
2
answers
93
views
Is there a service I can run which will automatically test CPU/RAM/hardware in the background?
Is there any service which runs in the background, and does the job of prime95 (i.e. it tests server CPU/RAM for integrity now and again)?
Regards,
Shane.
p.s. The reason I ask: we recently had some ...
- 1,150
0
votes
1
answer
829
views
Best way to add a new non-nullable column?
I'm a web developer, with admin rights on our SQLServer, and I need to add a non-nullable integer column (foreign key) to an existing table. I'm looking for the best way to create this column and ...
- 103
5
votes
4
answers
555
views
Disaster Recovery/Sabotage Protection for a small business
I've been contacted by two partners in a small professional firm. They are concerned about their other partner and want to take some steps to be absolutely sure that the company's data and systems ...
- 12.9k
6
votes
4
answers
2k
views
USB storage media - data integrity issues on linux
I've always been somewhat paranoid about verifying data backed up to removable media, so after copying stuff to a USB flash drive or portable HDD, I invariably unmount the drive, remount it and diff -...
- 866
15
votes
6
answers
13k
views
SATA Disks that handle write caching properly?
It's pretty common to see advice to disable the write cache on individual disks used for databases because otherwise some disks will acknowledge writes that haven't yet made it to the disk surface.
...
- 268