Hot answers tagged diff
46
You can do it with Bash's process substitution:
diff foo <(ssh myServer 'cat foo')
Or, if both are on remote servers:
diff <(ssh myServer1 'cat foo') <(ssh myServer2 'cat foo')
31
Finally I've found great solution: vimdiff
vimdiff /path/to/file scp://remotehost//path/to/file
thanks to http://linux.spiney.org/remote_diff_with_vim_and_ssh
see also http://www.vim.org/scripts/script.php?script_id=1075 .
21
Another way to look at it:
Show lines that only exist in file a:
comm -23 a b
Show lines that only exist in file b:
comm -13 a b
Show lines that only exist in one file or the other:
comm -3 a b | sed 's/^\t//'
21
The technical issues are pretty straightforward. Using a combination of SHA and MD5 hashes is pretty typical in the forensics industry.
If you're talking about text files that might've been modified-- say source code files, etc, then performing some type of structured "diff" would be pretty common. I can't cite cases, but there's definitely precedent out ...
18
If you are somewhat comfortable with vim, I would strongly encourage you to use vimdiff:
vimdiff file1 file2
This will open a vim session with two panes, with one file on each side. Highlights and color will indicate differences between the files, and all identical parts will be hidden (folded, but expandable).
Then, if you want to selectively merge ...
14
You're looking for
diff -rq (dir1) (dir2)
Proof of concept:
#!/bin/sh
#create our test
mkdir -p /tmp/a/b
echo "test" >> /tmp/a/c
mkdir -p /tmp/a/d/e
echo "blah" >> /tmp/a/d/e/f #only exists here
mkdir -p /tmp/q/b
echo "testing" >> /tmp/q/c #/tmp/a/c shouldnt match
mkdir -p /tmp/q/d/e
echo "blah" >> /tmp/q/d/e/g #only exists here
...
12
diff <(grep -v '^#' f1) <(grep -v '^#' f2)
To avoid blank lines, and lines containing nothing but spaces, in addition to identical lines that have a single difference of added leading spaces...
diff -b \
<(grep -vE '^([ \t]*#|^[ \t]*$)' f1)\
<(grep -vE '^([ \t]*#|^[ \t]*$)' f2)
By this point though, I'd probably put that into a script ...
7
Use scp to bring the files to a common machine and diff them there?
Or, if you just want to know if the files are different or not, hash them with md5sum on each machine.
You could also look into something like SSHFS, but I don't know how well an algorithm like diff performs over that.
5
Beyond Compare is the ultimate tool for this!
Link:
http://www.scootersoftware.com/
Available for Windows and Linux.
Jeff wrote a good overview article about the tool awhile back:
http://www.codinghorror.com/blog/archives/000454.html
5
Expanding on nima's one-liner, you could do that as a shell function and drop it in your .bashrc
diff <(grep -v '^#' f1) <(grep -v '^#' f2)
becomes (using -u because I like unified diffs)
function cleandiff {
diff -u <(grep -v '^#' $1| grep -v '^ *$') <(grep -v '^#' $2 | grep -v '^ *$')
}
If you like GUI diff viewers, meld is nice, and ...
5
Typically your attorney should already have a lot of this under control.
To prove the files are the same, md5 should be used. But even more than that, you need to prove chain of custody using auditable trails. If someone else has had the files in their custody, then you will have a hard time proving in court that the evidence wasn't 'planted'.
There are ...
5
ZFS has no feature to report that however, the undocumented zdb tool can certainly be used to get the blocks used by a file in a particular dataset (filesystem or snapshot) so achieving what you look for with a little bit of scripting is doable, although it would probably take a very long time to process zdb output.
Here is a blog showing how to use zdb to ...
5
Use --compare-dest.
From the man page:
--compare-dest=DIR -
This option instructs rsync to use DIR on the destination machine as an additional hierarchy to compare destination files against doing transfers (if the files are missing in the destination directory). If a file is found in DIR that is identical to the sender's file, the file will NOT ...
4
It sounds like you're looking for btrfs send/receive, which will appear in Linux 3.6. The send command creates a log file of the differences between two snapshots, and the receive command applies the changes from a file. Note that send/receive uses a custom file format, so the file won't look exactly like, say, diff or tar.
4
You could always do something like this to get the differences.
diff -u <( cd path1 ; find . -printf "chown %U:%G %p; chmod %m %p \n" | sort ) \
<( cd path2 ; find . -printf "chown %U:%G %p; chmod %m %p \n" | sort )
It would then be trivial to hack up something that performs the require changes.
4
If you always do a before and after such that the end of the after file will contain the new content then you could use comm.
$ cat before
aaa
bbb
$ cat after
aaa
bbb
ccc
$ comm -3 after before
ccc
4
Vim has vimdiff mode which is quite good. It obviously works in console mode too.
It give you color highlighting of differences and the possibility to edit the diffed files efficiently. Vim is also installable or even installed on most *nix system these days...
It is better if you already know Vim though...
4
rsync with the --dry-run and -v parameters will report the files and directories, whether they are present on the remote side and also detect differences in files (by checksumming). There is plenty of documentation on the net about how to get it running over SSH.
If you prefer a different approach, you could mount a remote directory locally using FUSE/SSHfs ...
4
There might be a better way, but this might work, albeit not that efficiently:
rsync -vrn / dest:/ > ~/file_list
Then edit test to remove the stats, then:
while read file; do
diff $file <(ssh dest "cat $file")
done < ~/edited_file_list
Another Option:
You might also consider mounting the file system with something like sshfs/fuse, and ...
3
There might be a more elegant way to do it, but pragmatically (and quickly):
grep -v '^#' server1-snmpd.conf | grep -v '^ *$' > server1-snmpd.conf-clean
grep -v '^#' server2-snmpd.conf | grep -v '^ *$' > server2-snmpd.conf-clean
diff server1-snmpd.conf-clean server2-snmpd.conf-clean
3
For finding duplications, you one use:
fdupes -r1 dir1 dir2
Although the others gave you numerous good tips, you should give it it a try too.
If you use
fdupes -rd dir1 dir2
it will prompt you which file to keep (the others will be deleted). Extremely useful for removing duplications (I did make a good use of it with my photos)
NOTE: yes, I know ...
3
It almost sounds like what you want is a tool, that will retrieve each block of both files, and then do a bitwise OR on each block, and send the output to a new file.
The psuedo-code might look like below. Nothing would happen to identical bits, and bits that where not identical a bit would be set to 1.
while not end-of-files:
read block file_a
read ...
3
Those are the line numbers and the operation to change one to the other. "5c5" means "line five changed (replaced) to line five", "7d6" means "delete line seven", "21a22" means "add (append) line 22"
You can change the output format using options such as --context=NUM or --unified.
Here is some more information on output formats from the info file for ...
3
comm might do what you want. From its man page:
DESCRIPTION
Compare sorted files FILE1 and FILE2 line by line.
With no options, produce three-column output. Column one
contains lines unique to FILE1, column
two contains lines unique to FILE2,
and column three contains lines common
to both files.
These columns are suppressable with ...
3
for i in `cat /tmp/invent.txt`; do grep ^$i$ /tmp/backup.txt >/dev/null || echo $i ; done
This will echo all names in /tmp/invent.txt which is not in /tmp/backup.txt
Edit
I found another way using diff :- (diff -r dir1 dir2)
-r, --recursive means Recursively compare any subdirectories found.
$ mkdir -p dir1/dir dir2/dir/dir
$ diff -r dir1 dir2
...
3
Diff your files, then use awk to find the lines you want (/^<), strip the leading bit (gsub(/^</,"")), and print the matched line.
diff file1 file2 | awk '/^</ {gsub(/^</,""); print}'
So if you want a one liner you could even do something like this.
diff <(cd /store ; find . -type f -print | sort) \
<(cd /fbkup ; find . -type f ...
3
Don't use diff, use join instead. Since your input files are already sorted, the following should produce exactly the output you requested:
join -v 2 /tmp/invent.txt /tmp/backup.txt > /tmp/in-backup-but-not-invent.txt
(If the file paths in either /tmp/backup.txt or /tmp/invent.txt contain whitespace, then join may not work correctly with the options ...
3
This little script should work:
folder1="firsts_folder/"
folder2="second_folder/"
folder3="target_folder"
for x in `rsync -rcnC --out-format="%n" $folder1 $folder2`
do
if [ -d "$folder1/$x" ]; then
mkdir -p "$folder3/$x"
else
cp -frv $folder1/$x $folder3/$x
fi
done
But I also think that parsing output ...
2
How about something simple like calculating the md5sum for each file and sorting based on the hash.
md5deep -r . | sort
d921223ccbe759a632973962bc15a497 /root/.bash_history
dcac40478a92e87cd08a42a6425acea6 /root/testsrv/keys/04.pem
dcac40478a92e87cd08a42a6425acea6 /root/testsrv/keys/client2.crt
e12f5739f81b08c470f20890304bf53e /root/.bashrc
...
2
rsync -rcnC --out-format="%f" . ../folder2/ |xargs cp --parents -rt ../folder3/
That's the simplest form I could come up with, if you're willing to first descend into folder1 before issuing the command.
This is the slower option, but I believe it meets your requirements, and it could be executed using non-relative directory names:
rsync -rcnC ...
Only top voted, non community-wiki answers of a minimum length are eligible
