hans.troost at telfort.nl
2010-Sep-02 09:38 UTC
Minimize log file: get rid of ".d...p..... send"
I'm just starting with rsync, googled and experienced a lot, but get stuck with minimizing the log-file: I only want to see the "real" changes: new, updated and deleted files/directories in the log-file. The LOG-file always contains a lot of lines like: 2010/08/31 20:19:01 [24573] .d...p..... send home/hans/Desktop 2010/08/31 20:19:05 [24573] .d...p..... send home/hans/Documents/PCPa I did not change anything, also not the permissions. So these directories are unchanged! Question: how to get rid of these lines, while keeping lines like: 2010/08/31 20:19:01 [24573] >f.stp..... send home/hans/2bedited.txt 2010/08/31 20:19:01 [24573] >f+++++++++ send home/hans/Documents/newfile.txt Want to use rscync for a "manual backup" (now and then) and my situation is: Source: workstation with PCLinuxOS 2010. Destination: a NAS (LaCie Ethernet Disk mini) with a share called Backup, mounted as /mnt/backup: [hans at localhost mnt]$ ls -l ... drwxr-xr-x 4 hans hans 0 2010-08-31 20:18 backup/ ... [hans at localhost mnt]$ ls -l backup total 0 drwxr-xr-x 2 hans hans 0 2010-08-31 19:39 deleted/ drwxr-xr-x 9 hans hans 0 2010-08-31 20:18 hans/ drwxr-xr-x 2 hans hans 0 2010-08-31 20:18 updated/ [hans at localhost mnt]$ the rsync command I use is: rsync -axhhX --stats --delete --delete-excluded --backup --backup-dir-dels=deleted/`date +%Y-%m-%d` --backup-dir=updated/`date +%Y-%m-%d` --suffix=.`date +%H%M` --log-file=/home/hans/rsync-files/rsync-log.`date +%Y%m%d-%H%M` --log-file-format="%i %o %f" --exclude-from=/home/hans/rsync-files/exclude.txt /home/hans /mnt/backup NB: also the old mailing list thread "rsync 2.6.4 - directory always updated" from John Stoffel in 2005 did not help me (I'm not that good in *nix file system stuff) Best regards, Hans Troost
On Thu, 2010-09-02 at 11:38 +0200, hans.troost at telfort.nl wrote:> I'm just starting with rsync, googled and experienced a lot, but get stuck > with > minimizing the log-file: I only want to see the "real" changes: new, updated > and > deleted files/directories in the log-file. > The LOG-file always contains a lot of lines like: > > 2010/08/31 20:19:01 [24573] .d...p..... send home/hans/Desktop > 2010/08/31 20:19:05 [24573] .d...p..... send home/hans/Documents/PCPa > > I did not change anything, also not the permissions. So these directories > are > unchanged!Yes, but are they the same on source and destination? The behavior you observe could happen if the destination filesystem is unable to store permissions and silently ignores chmod(2) calls. In that case, you should disable the -p option.> Question: how to get rid of these lines, while keeping lines like: > > 2010/08/31 20:19:01 [24573] >f.stp..... send home/hans/2bedited.txt > 2010/08/31 20:19:01 [24573] >f+++++++++ send home/hans/Documents/newfile.txtTo get rid of lines with only attribute changes, you can set a log format that does not contain %i. But I would suggest first looking into why they are being printed. -- Matt
On Fri, 2010-09-03 at 17:05 +0200, Hans Troost wrote:> and indeed - although I specified rsync -vvv (most verbose output) - rsync > "silently ignores chmod(2) calls" with emphasis on "silently": > > 2010/09/03 16:45:49 [7882] send_files(15, /home/hans/rsync-files/rsync-test/New Folder) > 2010/09/03 16:45:49 [7882] cd+++++++++ send home/hans/rsync-files/rsync-test/New Folder > 2010/09/03 16:45:49 [7882] set modtime of rsync-test/New Folder to (1283525117) Fri Sep 3 16:45:17 2010 > 2010/09/03 16:45:49 [7882] delete_in_dir(rsync-test/New Folder)Rsync is calling chmod, there just isn't a log message for it even on high verbosity. Run rsync under strace if you want to see all the system calls it is making. -- Matt
Thanks Matt, Tested again a bit and indeed: permissions on source and destination are different: [hans at localhost rsync-test]$ cd ~/rsync-files/rsync-test/ [hans at localhost rsync-test]$ ls -l total 24 drwxrwxr-x 2 hans hans 4096 2010-09-02 18:24 2bchanged/ drwxrwxr-x 2 hans hans 4096 2010-09-03 16:45 New Folder/ drwxrwxr-x 2 hans hans 4096 2010-09-03 16:42 New Folder (2)/ [hans at localhost rsync-test]$ cd /mnt/backup//rsync-test/ [hans at localhost rsync-test]$ ls -l total 8 drwxr-xr-x 1 hans hans 0 2010-09-02 18:24 2bchanged/ drwxr-xr-x 1 hans hans 0 2010-09-03 16:45 New Folder/ drwxr-xr-x 1 hans hans 0 2010-09-03 16:42 New Folder (2)/ and indeed - although I specified rsync -vvv (most verbose output) - rsync "silently ignores chmod(2) calls" with emphasis on "silently": 2010/09/03 16:45:49 [7882] send_files(15, /home/hans/rsync-files/rsync-test/New Folder) 2010/09/03 16:45:49 [7882] cd+++++++++ send home/hans/rsync-files/rsync-test/New Folder 2010/09/03 16:45:49 [7882] set modtime of rsync-test/New Folder to (1283525117) Fri Sep 3 16:45:17 2010 2010/09/03 16:45:49 [7882] delete_in_dir(rsync-test/New Folder) Since it is just a backup of my own personal workstation I decided to to use rsync -a --no-p .... : when restoring I just have to re-apply the correct permissions (if required at all). I decided to also remove the --log-file-format at all. This gives the log-file I prefer. Thanks for your comments and recommendations, Kind regards, Hans Troost
On Fri, 2010-09-03 at 20:49 +0200, Hans Troost wrote:> Indeed, strace -f option helps: > > the chmod status "unfinished" shows up several times now. no real "warnings" > or "errors", but nevertheless: information about what's going wrong."Unfinished" does not indicate that anything is wrong. It just means that something happened in another process between when the call was issued and when it returned, so the output had to be broken up. The problem is that the chmod calls return 0 (success) without actually setting the permissions. -- Matt