sshfs won't help with this; what you need to do is add the
-W option to tell rsync to just transfer whole files, without trying to figure out where the specific differences are. You might also need to remove
--inplace (not sure about that).
rsync is really designed for remote operations where there's an rsync instance (or daemon) on each computer, each with a fast link to its disk, but with a slow link between the rsync instances. In your case, the slow link is between the "remote" rsync and its disk, so a different set of optimization choices are needed. Specifically, rsync generally defaults to reading the entire source and destination files so it can figure out just which sections need to be transferred over the "slow" middle link; in your case, slows it down more than just blindly transferring the entire file would.
Also, make sure you have the
-t option (or
-a, which implies
-t), and are not using
-c would make it read the entire file to see if it had changed at all; without that, it'll just look at the file size and modification time, which