3

I am seeing a random behavior when copying from one server to another using rsync to the /dev/null of the destination server. This is as part my network performance testing, and I want to avoid Disk I/O in my testings. I have the same script in all my servers, but for some of them the destination /dev/null gets overwritten and becomes a regular file, and that breaks other things.

For the same server scp does not overwrite the destination /dev/null.

Before:

[root@localhost ~]# ls -l /dev/null
crw-rw-rw-. 1 root root 1, 3 Sep 11 09:24 /dev/null

Here is my rsync command:

rsync -v -e ssh --progress 500MB root@destination-server:/dev/null

After:

[root@localhost ~]# ls -l /dev/null
-rw-r--r--. 1 root root 104857600 Sep 11 10:09 /dev/null

Doing scp to the same server, does not break /dev/null

scp 500MB root@destination-server:/dev/null

Any idea why causes the random behavior? My servers are a mix of CentOS6 and 7

Thank You, Eilbron

New contributor
Eilbron is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
  • if you aren't already doing so using a tool like iperf will give a better understanding of network performance than rsync over ssh – Mark Wagner Sep 11 at 20:20
3

rsync doesn't just copy files' contents, it also syncs file attributes. This is somewhat limited by the options you pass (or don't pass) to rsync, but only somewhat. rsync has a --devices option that makes it syncing device files as device files, but apparently not passing that option doesn't prevent it from syncing plain files as plain files.

More generally: /dev/null is a safe place to write data into; it is not safe to use it as a target for things that mess with file attributes or structure.

1

By default, rsync will

  • create a new file, and overwrite the original file with the new one
  • not treat any file as special (device, link, fifo etc).

Here is how syncing a file called passwd from directory a to directory b goes down:

Reads the source file

a/ OPEN passwd
a/ ACCESS passwd
a/ CLOSE_NOWRITE,CLOSE passwd

Creates temp file and moves data:

b/ CREATE .passwd.cBzlfu
b/ OPEN .passwd.cBzlfu
b/ ATTRIB .passwd.cBzlfu
b/ MODIFY .passwd.cBzlfu
b/ CLOSE_WRITE,CLOSE .passwd.cBzlfu
b/ ATTRIB .passwd.cBzlfu

Finalizes filename

b/ MOVED_FROM .passwd.cBzlfu
b/ MOVED_TO passwd

The last operation is what overwrites your /dev/null, because it simply overwrites the device file. You expect that it will write (append) into it, but it doesn't.

Switches --inplace and --partial are influencing the synchronization behavior.

The switch --devices also helpful

This option causes rsync to transfer character and block device files to the remote system to recreate these devices. This option has no effect if the receiving rsync is not run as the super-user (see also the --super and --fake-super options).

Perhaps you should also consider not syncing a file onto a device :) rsync can sync a file with a file, a device with a device, but you appear to sync a file onto a device. That will probably don't go.

0

/dev/null is a character special file and, as you have discovered, copying to it using rsync as root will overwrite it. AFAIK, rsync will not do what you want it to do. It might be possible to pipe through ssh to cat.

time tar cf - file | ssh server "cat > /dev/null"
New contributor
scarville is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.

Your Answer

Eilbron is a new contributor. Be nice, and check out our Code of Conduct.

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.