Questions tagged [data-integrity]
The data-integrity tag has no usage guidance.
42
questions
2
votes
3answers
59 views
Is checking data integrity with hashes useful for my scenario?
I have a NAS as production system (mostly video footage and project files) and a 1:1 backup. I only need to retrieve the backup data in cases of emergency. Such a case didn't occur yet.
Lately I'm ...
30
votes
7answers
5k views
How beneficial are self-healing filesystems for general usage?
I have recently looked into advanced filesystems (Btrfs, ZFS) for data redundancy and availability and got interested in the additional functionality they provide, especially their "self-healing" ...
0
votes
1answer
142 views
How to test Snapraid data corruption?
I'm running Snapraid on my hard drives. However, I want to test it's data integrity features, to see how/if they actually work.
But I don't really understand how to 'corrupt' my data. I can just ...
-1
votes
1answer
51 views
live server disk image backup data integrity
I've been looking for a way to do a disk image backup of a running linux server.
There are several questions which answer this, like How to make a disk image, Mirror Live Linux (Debian) Server, Linux ...
0
votes
1answer
1k views
Logrotate with `copytruncate` “data loss” - is it serious?
Manpage of logrotate claims that using copytruncate option leads to "a very small time slice between copying the file and truncating it, so some logging data might be lost". Is it actually ...
0
votes
1answer
425 views
MySQL Table Checksum - Before or after MySQL dump?
The company I'm currently working for has a script that runs at night that takes a MySQL dump of the production environment and imports it into the staging environment. This is a very basic setup - No ...
1
vote
1answer
103 views
Where do I find the verified checksums for all files in all packages for Ubuntu?
I am looking to verify whether or not my binaries, installed by the Aptitude package manager under Ubuntu 16.04.2 LTS, are either in-tact or have been modified / compromised. I cannot find any list of ...
0
votes
1answer
695 views
How to repair a mailbox with public folders?
We have an Exchange 2013 organization with sort of legacy public folder set transferred via import/export from Exchange 2003 times. It has several linked mailboxes for our staff, and several dozens of ...
2
votes
1answer
154 views
How do you store VMs on HDDs with data integrity AND performance?
First off I know this topic easily gets subjective, but I'm trying to avoid that, as there should be at least 1 good answer to this in a sea of bad answers and it's hard to find.
At first sight my ...
3
votes
1answer
740 views
Ensuring VMware ESXi storage integrity?
I'm planning a small ESXi standalone server to host a number of VMs I use here. I haven't decided what facilities to use from the wider vSphere system. The underlying VM storage is local HDD RAID ...
1
vote
2answers
189 views
Do I need btrfs scrub if a lower layer also scrubs?
I have a multi-TB btrfs made up of virtual disk images that live on NetApp NFS filers. The data are redundant at the NetApp level, so the btrfs is RAID-0.
In addition to providing redundancy, the ...
4
votes
1answer
706 views
Drawbacks of a single drive split into partitions and partitions joined into a ZFS raidz1, vs. single drive ZFS with data duplication?
I'm looking for a minimal ZFS setup to achieve self-healing on a single device. Since I back up my data, drive failures are no concern, but bit-rot is. So far, the only suggestions I've been able ...
0
votes
1answer
130 views
Managing uploaded files on a server [duplicate]
I have a server hosting an intranet web site where one of the features will be the ability to upload files. The files will be saved in a restricted access folder and managed through the web back-end. ...
1
vote
2answers
1k views
restore mongoDB databases from manual backup (not used mongodump): strange permission problems
Is it possible to restore databases which were manually copied from /var/lib/mongodb to eg. /home/foo/mongobackup ?
I am not sure if the backup was made after stopping the mongod service. Either way, ...
1
vote
0answers
312 views
Verify rdiff backup integrity
I am using rdiff for my differential backup. Recently an issue happened where all versions of my backup failed to patch correctly and rdiff returning an error 106 which after investigation is a code ...
8
votes
2answers
2k views
What is a safe way to back up a sparsebundle that is exported via afpd?
I work with a number of OSX client machines that back up via TimeMachine to AFP shares on an Ubuntu Linux file server, exported by netatalk/afpd. These clients back up at arbitrary times of the day, ...
-1
votes
1answer
77 views
Does RAID1 also checks for corruption? [closed]
I have two 1.5 TB disks, and I don't trust both of them because of weird noises and bad clusters.
I was wondering if I put them in a software RAID1 array (mirrored volume in Win7) it will write my ...
0
votes
3answers
1k views
Which SSD to choose, when unexpected powerlosses are frequent?
I am working on a embedded system where there can be unexpected power losses. So far it has been using a Corsair F80 SSD with Windows Embedded Standard 7 (NTFS), and after a unexpected power loss ...
14
votes
2answers
11k views
How to get checksum data integrity on ext4?
On file systems like btrfs you can preform a scrub, which will go though all the data to see if the data still matches the file system checksum.
I would like to check the data on ext4 if it is ...
2
votes
2answers
149 views
Single file changed: intrusion or corruption?
rkhunter reported a single file change on a virtual server (netstat binary). It didn't report any other warning. The change was not the result of a package upgrade (I reinstalled it and the checksum ...
2
votes
1answer
3k views
DB2 Integrity Checks and Exception Tables
I am working on planning a migration of a DB2 8.1 database from a horrible IBM encoding to UTF-8 to support further languages etc. I am encountering an issue that I am stuck on.
A few notes on this ...
3
votes
2answers
1k views
Detecting data corruption so we're not backing up corrupt files
I've been thinking about data integrity, I currently backup about 2tb of data and always have one backup of data from a year ago.
My concern is if a file became corrupt on our production file server ...
0
votes
1answer
2k views
Best practices for mass file transfer with verification?
I'm looking for a bit of advice on transferring a several files from Windows-based workstations to a Linux Server running Ubuntu Server LTS 10.04. At the moment I've been mounting the shares on the ...
1
vote
3answers
963 views
Best practices for backing up Windows workstations to Linux server?
I was wondering what strategies others in the community use when backing up their Windows-based workstations to Linux servers. At the moment I'm doing a bit of a test with Ubuntu Server LTS 10.04 and ...
2
votes
2answers
1k views
What happens to InnoDB and MyISAM tables if the power fails during a transaction?
We have two MySQL tables, one is InnoDB and the other is MyISAM. If I execute a chunk of inserts in a single transaction on both tables and the power fails in the middle, what will be the status?
...
1
vote
0answers
80 views
Archival storage tool: fast, queryable, has api, reliable
I have large quantities of structured data coming in over the network from disparate sources, and that data is archived in the filesystem, parsed/analyzed, and eventually makes it's way into a ...
195
votes
6answers
76k views
LVM dangers and caveats
I have recently started using LVM on some servers for hard drives larger than 1 TB. They're useful, expandable and quite easy to install. However, I could not find any data about the dangers and ...
6
votes
3answers
6k views
Making SATA disk write cache safe
Supposedly (see, e.g., a question about it here), with NCQ enabled drives, the drive write cache is supposed to be safe, as in it doesn't lie to the OS about data being committed to the platters when ...
5
votes
2answers
3k views
PCI Compliance: File Integrity Checkers [closed]
We're looking for a file integrity checker for our windows server environment. Anyone have any good suggestions?
2
votes
2answers
3k views
Upload over HTTPS is reliable?
Sometimes we are facing a problem with corrupted files after upload through HTTP protocol.
Today I’ve read an article that says that TCP protocol is not 100% reliable due to weak CRC algorithm and ...
3
votes
5answers
7k views
Is there any value in checking MD5 of files after they have been unzipped?
A project I am working on currently requires that the user runs an MD5 hash checking tool on the entire project, after it has been unzipped. They do not currently request that the ZIP itself is ...
0
votes
2answers
1k views
SQL: Find Last Integrity Check Date
Is there a way to check when was the last integrity check run for each database?
I know when the service startup it checks for these values, so they exist, and they are probably writen on the ...
7
votes
2answers
6k views
It is XFS reliable? In case of a power gone off is xfs riskier that ext3 in data corruption/safey?
I have been having some electrical issues,mostly power going out suddenly for a few months,and although ups solve the thing mostly.
But still i am worried about filesystem corruption,and data lost.
...
7
votes
3answers
20k views
How to check PostgreSQL database integrity?
Is there a way to check PostgreSQL database(s) integrity and consistency? I know about SQL Server DBCC CHECKDB and wonder if there is something similar to PostgreSQL.
0
votes
3answers
46 views
How two or more guest OS's hosted in a virtual environment handles editing of single resource?
I know this sounds very general question.
Consider 2 OS's running using virtualization say operating system level virtualization. How concurrency is handled if both hosted OS using same resource and ...
0
votes
1answer
393 views
Why does DBCC CHECKTABLE report errors on non-existent pages?
When I run DBCC CHECKDB, I get a slew of error messages like this:
Table error: object ID 2020918271, index ID 1, partition ID 72057594196590592, alloc unit ID 72057594190233600 (type In-row data), ...
1
vote
4answers
10k views
How to verify integrity of a remote file transfer
Is there any way I can verify the integrity of a remotely transferred file? I am currently transferring a large amount of files and folders from an ftp server to another remote server using wget, but ...
0
votes
2answers
86 views
Is there a service I can run which will automatically test CPU/RAM/hardware in the background?
Is there any service which runs in the background, and does the job of prime95 (i.e. it tests server CPU/RAM for integrity now and again)?
Regards,
Shane.
p.s. The reason I ask: we recently had some ...
0
votes
1answer
816 views
Best way to add a new non-nullable column?
I'm a web developer, with admin rights on our SQLServer, and I need to add a non-nullable integer column (foreign key) to an existing table. I'm looking for the best way to create this column and ...
5
votes
4answers
529 views
Disaster Recovery/Sabotage Protection for a small business
I've been contacted by two partners in a small professional firm. They are concerned about their other partner and want to take some steps to be absolutely sure that the company's data and systems ...
6
votes
4answers
2k views
USB storage media - data integrity issues on linux
I've always been somewhat paranoid about verifying data backed up to removable media, so after copying stuff to a USB flash drive or portable HDD, I invariably unmount the drive, remount it and diff -...
15
votes
6answers
12k views
SATA Disks that handle write caching properly?
It's pretty common to see advice to disable the write cache on individual disks used for databases because otherwise some disks will acknowledge writes that haven't yet made it to the disk surface.
...
