rsync - Feb 2013 - A --exclude-checksum option?

Hi,

I use rsync with hardlinks for backup, once a week doing checksums
to ensure there's no filesystem corruption in the
backed-up data.  

I also use tmpwatch, or something similar, to clean up /tmp,
it removes files that have not been accessed recently.
(atime older than some configured limit).
I backup /tmp because I throw stuff in tmp that
I might possibly need again but don't want to bother
having to remember to delete -- and if I'm expecting
to have useful data somewhere I want it backed up.

However, rsync's checksumming (naturally) updates
the atimes of the files in /tmp, and so tmpwatch
never deletes them.

It occurs to me that a handy solution might be
to have an rsync option, similar to
the --exclude option, which would allow checksumming
to happen throughout most of the backup process but
would do "regular" size/timestamp based backups
on certain directories.

What do people think of such an option?
Is there a better design.  (E.g. an option that, er,
preserves atime when checksumming?)
Is rsync just too overloaded with options already
and it would be better instead to run
two instances of rsync?  Is there a bug/feature
in process already that would address the use-case
above?

I'd like to have a sensible design before even
thinking about patching.

Thanks for the feedback.

Regards,

Karl <kop at meme.com>
Free Software:  "You don't pay back, you pay forward."
                 -- Robert A. Heinlein

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

My first thought is why are you backing up /tmp at all?

My second thought is why are you using atime for anything?  It can be
touched by almost anything and running a filesystem with atime enabled
is a huge performance detriment as it adds a directory write operation
to every file read operation.

My final thought is maybe you want a file verification tool (I like
cfv) instead of rsync --checksum.  Rsync's --checksum is kinda
mindless in terms of performance.  It checksums everything.  This is
rather pointless as a file that is a different size will obviously
have a different checksum.  Rsync even checksums files that only exist
on one side of the transfer.

On 02/12/13 15:42, Karl O. Pinc wrote:
> Hi,
> 
> I use rsync with hardlinks for backup, once a week doing checksums 
> to ensure there's no filesystem corruption in the backed-up data.
> 
> 
> I also use tmpwatch, or something similar, to clean up /tmp, it
> removes files that have not been accessed recently. (atime older
> than some configured limit). I backup /tmp because I throw stuff in
> tmp that I might possibly need again but don't want to bother 
> having to remember to delete -- and if I'm expecting to have useful
> data somewhere I want it backed up.
> 
> However, rsync's checksumming (naturally) updates the atimes of the
> files in /tmp, and so tmpwatch never deletes them.
> 
> It occurs to me that a handy solution might be to have an rsync
> option, similar to the --exclude option, which would allow
> checksumming to happen throughout most of the backup process but 
> would do "regular" size/timestamp based backups on certain
> directories.
> 
> What do people think of such an option? Is there a better design.
> (E.g. an option that, er, preserves atime when checksumming?) Is
> rsync just too overloaded with options already and it would be
> better instead to run two instances of rsync?  Is there a
> bug/feature in process already that would address the use-case 
> above?
> 
> I'd like to have a sensible design before even thinking about
> patching.
> 
> Thanks for the feedback.
> 
> Regards,
> 
> Karl <kop at meme.com> Free Software:  "You don't pay back,
you pay
> forward." -- Robert A. Heinlein
> 
- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlEaqqMACgkQVKC1jlbQAQeERgCghHJ4sa3XPCg72KABmvX4/1+y
/4sAn19bpO+kKOW4/PNgcmPm2AoChjAf
=I+LJ
-----END PGP SIGNATURE-----

On Tue, Feb 12, 2013 at 12:42 PM, Karl O. Pinc <kop at meme.com> wrote:
> It occurs to me that a handy solution might be to have an rsync option,
> similar to the --exclude option, which would allow checksumming to happen
> throughout most of the  ackup process but would do "regular"
size/timestamp
> based backups on certain directories.
>
One thing you could do is to apply the checksum-reading.diff patch and then
generate a checksum file (using the included perl script) for the /tmp dir
right before doing your rsync run.  That would cache the file's checksums
and not re-read the files unless the perl script notices that the ctime has
changed (which will cause it to update the cache file).  A patched rsync
will read the checksum from the cache file instead of re-computing it,
which should solve your issue.

..wayne..
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.samba.org/pipermail/rsync/attachments/20130217/b67f9ecd/attachment.html>