rsync - Jul 2014 - rsync 3.1.0/3.1.1 incompatible with 2.5.7

I develop GNU Parallel. GNU Parallel uses rsync to transfer files. GNU
Parallel has a design goal of not requiring the users to change their
setup to be able to use GNU Parallel. In other words: If you are
called as a consultant to work on a Centos3 server untouched since
2007, then you can expect even the newest version of GNU Parallel will
work flawlessly.

Centos3 has rsync 2.5.7 installed.

On my development machine I have so far been using rsync prior to
3.1.0 and there has been no hiccups: Rsync has worked flawlessly
against Centos3.

Yesterday I upgraded my development machine, which also upgraded rsync to 3.1.0.

This breaks rsync against Centos3. I get the following error:

rsync-3.1.0     protocol version mismatch - is your shell clean?
rsync-3.1.0     (see the rsync man page for an explanation)
rsync-3.1.0     rsync error: protocol incompatibility (code 2) at compat.c(62)
rsync-3.1.0     rsync: connection unexpectedly closed (0 bytes
received so far) [sender]
rsync-3.1.0     rsync error: error in rsync protocol data stream (code
12) at io.c(226) [sender=3.1.0]

This is also the case for 3.1.1:

rsync-3.1.1     protocol version mismatch - is your shell clean?
rsync-3.1.1     (see the rsync man page for an explanation)
rsync-3.1.1     rsync error: protocol incompatibility (code 2) at compat.c(62)
rsync-3.1.1     rsync: connection unexpectedly closed (0 bytes
received so far) [sender]
rsync-3.1.1     rsync error: error in rsync protocol data stream (code
12) at io.c(226) [sender=3.1.1]

But this is not the case for 3.0.[0-9]: They all work fine against rsync 2.5.7.

The workaround seems to be adding --protocol 30 if you are running
rsync 3.1.x against 2.5.7, but that workaround fails when running
2.5.7 against 2.5.7, so it cannot always be used:

  [centos3]$ rsync --protocol 30 /tmp/foo localhost:
  rsync: --protocol: unknown option
  rsync error: syntax or usage error (code 1) at main.c(994)

Obviously as a consultant you cannot require the customer to neither
upgrade nor downgrade their rsync installation, just so that you can
use GNU Parallel.

I am quite surprised that rsync-3.1.x does not simply negotiate the
protocol and downgrade to protocol 30 automatically when 31 fails. And
I find the error message could be more helpful in finding the
workaround.

My question is now: What should I do as a developer?

How do I make sure that GNU Parallel's use of rsync never fails
because of protocol issues when I cannot predict which version of
rsync is used by the remote machine?

If this is a bug that will be fixed in 3.1.2, then that solves it for
future versions. But there will still be machines with 3.1.[0-1] out
there for years to come. How do your recommend I support those in GNU
Parallel?

Do you have a compatibility matrix between all version of rsync?

Are other incompatibilities (that will be turned on by default and
which does not get negotiated with the remote) planned?


/Ole

On Fri, Jul 25, 2014 at 11:50 AM, Ole Tange <tange at gnu.org> wrote:
> I am quite surprised that rsync-3.1.x does not simply negotiate
> the protocol and downgrade to protocol 30 automatically when 31 fails.

Sadly, it can't do any of that because it is the 2.5.7 version that is
killing the connection due to the advent of protocol 31 -- 2.5.7 has some
code in it to die if what it thinks is a protocol number isn't in a
particular range of values that maxes out at protocol 30.

And I find the error message could be more helpful in
finding
> the workaround.
>
Again, it's the old 2.5.7 assuming that the bytes it is encountering are
crap instead of valid data, which usually means that some ascii text got
into the transmission stream.  A sad effect of having no decent starting
byte sequence to the protocol, and no way to change it without becoming
incompatible with all prior versions of the software.

My question is now: What should I do as a developer?
>
There aren't any particularly easy answers for dealing with an old bit of
software that is refusing to play nice with newer software.  One
possibility is for your wrapping code to do a configure pass that checks
the rsync version on both of the hosts and figures out if one is too old
(2.6.0 and newer are fine, so only 2.5.7 and older have this issue) and one
is too new (3.1.0 is the first to start using protocol 31). If that occurs,
you can specify the --protocol=30 option on the newer rsync to avoid the
issue.

Depending on the client's willingness to tweak their rsync, the fix for
such an old 2.5.7 would really be as simple as changing one number in the
executable from comparing against 30 to comparing against 40.  If someone
were to take the code for 2.5.7 and tweak that one use of the
MAX_PROTOCOL_VERSION
(it only gets used in compat.c) then the 2.5.7 version would work fine.
 That would essentially be a 1-byte patch to the executable, but obviously
you can't expect everyone to be willing to apply such a change, even if you
were able to figure out exactly what byte to twiddle.

..wayne..
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.samba.org/pipermail/rsync/attachments/20140725/f755837b/attachment.html>