rsync - Jun 2021 - Permission denied errors with --append-verify run as non-root user

I noticed that my latest  "syncHome" script log scontained a number of
unexpected "Permission denied" errors for some modified files.  The
cause
seems to be that those files were read-only at the target.

The syncHome script is used to maintain a partial mirror of my home
directory tree on another host.

Further investigation and some experiments led me to suspect the
--append-verify functionality.  Specifically, those files were not rsync'd
when the --append-verify rsync flag was specified but were properly rsync'd
when --append-verify was NOT specified.

Other rsync scripts that use --append-verify and are run as root seem not
to exhibit this behavior.

FYI, I downloaded and built the "latest" version (rsync  version 3.2.3
 protocol version 31) and with this version of rsync, the modified files
are silently NOT rsync'd for both root and non-root users when
--append-verify is specified.  When --append-verify is not specified, the
updated files are rsync'd correctly.

That seems to me to be a serious bug, IMHO, because I "LIKE' to use
--append-verify to ensure that large files are copied entirely without
having to resync the entire file if there is an error.  If the
--append-verify option *silently fails* to rsync some updated files that
*should* be rsync'd, I cannot use --append-verify!

Please advise whether I am misunderstanding the purpose of, or misusing
the  --append-verify flag or if this is a bug.   It is really easy to
reproduce, BTW...

Dave Knight
Radford, VA
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.samba.org/pipermail/rsync/attachments/20210604/5c099fe9/attachment.htm>

Short answer: Don't use --append[-verify]

If you aren't syncing files that aren't tagged with 'chattr +a'
you
don't want --append.  --append-verify is essentially "I think --append
is a good idea to use on general files but rsync keeps corrupting
stuff!" but it still allows files to be out of sync since it ignores any
change to a file that doesn't make it a bigger file.  Essentially it is
for things like append only filesytem images and non-rotated log files.

All of that being said, rsync by default only transfers what is
different between the source and the target.  If a file has gotten
bigger only by adding stuff at the end rsync will figure that out and
transfer only the stuff that matters.  Any performance improvement you
are getting from --append-verify is either because it is ignoring file
changes that didn't result in a bigger file and because it causes file
updates to work like --inplace (which is what is causing your problem).

On 6/4/21 11:50 AM, Knight, Dave via rsync wrote:
> I noticed that my latest ?"syncHome" script log scontained a
number of
> unexpected "Permission denied" errors for some modified files.?
The
> cause seems to be that those files were read-only at the target. ?
> 
> The syncHome script is used to maintain a partial mirror of my home
> directory tree on another host.
> 
> Further investigation and some experiments led me to suspect the
> --append-verify functionality.? Specifically, those files were not
> rsync'd when the --append-verify rsync flag was specified but were
> properly rsync'd when --append-verify was NOT specified. ?
> 
> Other rsync scripts that use --append-verify and are run as root seem
> not to exhibit this behavior.
> 
> FYI, I?downloaded?and built the "latest" version (rsync ?version
3.2.3
> ?protocol version 31) and with this version of rsync, the modified files
> are silently NOT rsync'd?for both root and non-root users when
> --append-verify is specified.? When --append-verify is not specified,
> the updated files are rsync'd?correctly.
> 
> That seems to me to be a serious bug, IMHO, because I "LIKE' to
use
> --append-verify to ensure that large files are copied entirely without
> having to resync the entire file if there is an error.? If the
> --append-verify option */silently fails/* to rsync some updated files
> that */should/*?be rsync'd, I cannot use --append-verify!
> 
> Please?advise whether I am misunderstanding the purpose of, or misusing
> the? --append-verify flag or if this is a bug.? ?It is really easy to
> reproduce, BTW...?
> 
> Dave Knight?
> Radford, VA
> 
> 
> 
-- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			https://sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,