Hi,
My name is Vu Huynh. I'm currently work for Atmel Corporation.
We're using Rsync v2.5.7 to
rsync design kits between our remote sites. It works fine for all small kit
directories. However,
there is 1 BIG kit directory (But there is no file has the size of bigger than
100MB) that always
gets "rsync: connection unexpectedly closed" Error. The Full Error
Message is below:
% rsync --password-file -avz user@host::KIT1 KIT1 --exclude '5.*'
--exclude '6.*'
receiving file list ... done
./
7.0/
7.0/bin/
7.0/bin/AWK/
7.0/bin/IA32,Linux_2.4.7-10,32bits/
..............
..............
7.0/tools_sources/
7.0/tools_sources/templates/
rsync: connection unexpectedly closed (1892713 bytes read so far)
rsync error: error in rsync protocol data stream (code 12) at io.c(165)
Does file-list has a size limitation? Or something was not setup right in
this kit directory?
I hope you can have any clue/suggestion of what was wrong or limitation of
rsync.
Thank you and have a good day!
Vu Huynh
On Tue, Dec 09, 2003 at 01:54:25PM -0500, Vu Huynh wrote:> > > Hi, > > My name is Vu Huynh. I'm currently work for Atmel Corporation. We're > using Rsync v2.5.7 to rsync design kits between our remote sites. It works > fine for all small kit directories. However, there is 1 BIG kit directory > (But there is no file has the size of bigger than 100MB) that always gets > "rsync: connection unexpectedly closed" Error. The Full Error Message is > below: > > % rsync --password-file -avz user@host::KIT1 KIT1 --exclude '5.*' > --exclude '6.*' > receiving file list ... done > ./ > 7.0/ > 7.0/bin/ > 7.0/bin/AWK/ > 7.0/bin/IA32,Linux_2.4.7-10,32bits/ > .............. > .............. > 7.0/tools_sources/ > 7.0/tools_sources/templates/ > rsync: connection unexpectedly closed (1892713 bytes read so far) > rsync error: error in rsync protocol data stream (code 12) at io.c(165) > > Does file-list has a size limitation? Or something was not setup right > in this kit directory? I hope you can have any clue/suggestion of what was > wrong or limitation of rsync.If you have gotten to the point of processing files it is unlikely that the size of the file list is the issue. With the information given my best guess is that something is causing a delay where no data is being sent and the firewall is dropping the session. You might try another -v option to see if you can get more specific info about what file rsync is processing or other possible causes for a stall. -- ________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: jw@pegasys.ws Remember Cernan and Schmitt
Vu, I see the code 12 exit quite frequently over out WAN (using ssh as the transport). It is generally a network hiccup. What I ended up doing was including rsync in a shell script which retries if the return code is not 0, i.e.: #!/usr/bin/ksh count=0 while : do ((count=count+1)) if [ $count -gt 5 ] then echo "TOO MANY RETRIES #$count" break fi rsync $your $options $here rc=$? if [ $rc -ne 0 ] then echo "RETRY #$count (rsync FAILED with rc=$rc)" else break fi done Unfortunately for large directories, this does mean that you reprocess everything. I've rarely had more than one retry unless there was a serious network outage. A. Daniel King, System Analyst HP-UX, Solaris, Linux -----Original Message----- From: jw schultz [mailto:jw@pegasys.ws] Sent: Tuesday, December 09, 2003 2:07 PM To: rsync@lists.samba.org Subject: Re: Rsync died on big directories/file-lists On Tue, Dec 09, 2003 at 01:54:25PM -0500, Vu Huynh wrote:> > > Hi, > > My name is Vu Huynh. I'm currently work for Atmel Corporation.We're> using Rsync v2.5.7 to rsync design kits between our remote sites. Itworks> fine for all small kit directories. However, there is 1 BIG kit directory> (But there is no file has the size of bigger than 100MB) that always gets > "rsync: connection unexpectedly closed" Error. The Full Error Message is > below: > > % rsync --password-file -avz user@host::KIT1 KIT1 --exclude '5.*' > --exclude '6.*' > receiving file list ... done > ./ > 7.0/ > 7.0/bin/ > 7.0/bin/AWK/ > 7.0/bin/IA32,Linux_2.4.7-10,32bits/ > .............. > .............. > 7.0/tools_sources/ > 7.0/tools_sources/templates/ > rsync: connection unexpectedly closed (1892713 bytes read so far) > rsync error: error in rsync protocol data stream (code 12) at io.c(165) > > Does file-list has a size limitation? Or something was not setupright> in this kit directory? I hope you can have any clue/suggestion of whatwas> wrong or limitation of rsync.If you have gotten to the point of processing files it is unlikely that the size of the file list is the issue. With the information given my best guess is that something is causing a delay where no data is being sent and the firewall is dropping the session. You might try another -v option to see if you can get more specific info about what file rsync is processing or other possible causes for a stall. -- ________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: jw@pegasys.ws Remember Cernan and Schmitt -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
On Tue, Dec 09, 2003 at 01:54:25PM -0500, Vu Huynh wrote:> % rsync --password-file -avz user@host::KIT1 KIT1 --exclude '5.*' > --exclude '6.*'Since you're using compression, be sure to try the compression bugfix: http://www.blorf.net/rsync-2.5.7-compress-fix.patch ..wayne..