dovecot - Jan 2008 - Time just moved backwards error even with ntpd

Scenario: server PC abruptly switched off due to power cable problems 
(an UPS cannot solve this issue), so during shutdown Linux was not 
able to resinchronize the system clock. After a few hours the server 
come back on, Linux booted and the services (ntpd, dovecot and many 
others) started

But the system clock was 45 minutes ahead, so:

Jan 19 11:13:39 gw ntpd[2112]: synchronized to LOCAL(0), stratum 10
Jan 19 11:13:39 gw ntpd[2112]: kernel time sync disabled 0041
Jan 19 11:14:43 gw ntpd[2112]: synchronized to 62.48.35.100, stratum 2
Jan 19 10:31:55 gw ntpd[2112]: time reset -3600.221385 s
Jan 19 10:31:55 gw ntpd[2112]: kernel time sync enabled 0001
Jan 19 10:31:55 gw dovecot: Time just moved backwards by 3600 seconds. 
This might cause a lot of problems, so I'll just kill myself now. 
http://wiki.dovecot.org/TimeMovedBackwards




Ciao,
luigi

-- 
/
+--[Luigi Rosa]--
\

Spoon boy: Do not try and bend the spoon. That's impossible. 
Instead... only try to realize the truth.
Neo: What truth?
Spoon boy: There is no spoon.
Neo: There is no spoon?
Spoon boy: Then you'll see, that it is not the spoon that bends, it is 
only yourself.
     --The Matrix

* Luigi Rosa <lrosa at hypertrek.info>:
> Scenario: server PC abruptly switched off due to power cable problems (an 
> UPS cannot solve this issue), so during shutdown Linux was not able to 
> resinchronize the system clock. After a few hours the server come back on, 
> Linux booted and the services (ntpd, dovecot and many others) started
>
> But the system clock was 45 minutes ahead, so:
>
> Jan 19 11:13:39 gw ntpd[2112]: synchronized to LOCAL(0), stratum 10
> Jan 19 11:13:39 gw ntpd[2112]: kernel time sync disabled 0041
> Jan 19 11:14:43 gw ntpd[2112]: synchronized to 62.48.35.100, stratum 2
> Jan 19 10:31:55 gw ntpd[2112]: time reset -3600.221385 s
> Jan 19 10:31:55 gw ntpd[2112]: kernel time sync enabled 0001
> Jan 19 10:31:55 gw dovecot: Time just moved backwards by 3600 seconds. This
> might cause a lot of problems, so I'll just kill myself now. 
> http://wiki.dovecot.org/TimeMovedBackwards
Solution: Start ntpdate/ntpd FIRST.

-- 
Ralf Hildebrandt (Ralf.Hildebrandt at charite.de)             plonk at
charite.de
Postfix - Einrichtung, Betrieb und Wartung       Tel. +49 (0)30-450 570-155
http://www.arschkrebs.de
I realise computers suck. The only reason why they are a hobby of mine
is because I enjoy pain!

On Sat, 2008-01-19 at 12:46 +0100, Luigi Rosa wrote:
> Scenario: server PC abruptly switched off due to power cable problems 
> (an UPS cannot solve this issue), so during shutdown Linux was not 
This sounds like rather extreme, exceptional circumstances. And actually
an infrastructure problem, rather than software. ;)
> able to resinchronize the system clock. After a few hours the server 
> come back on, Linux booted and the services (ntpd, dovecot and many 
> others) started
> 
> But the system clock was 45 minutes ahead, so:
No. :)
> Jan 19 11:13:39 gw ntpd[2112]: synchronized to LOCAL(0), stratum 10
> Jan 19 11:13:39 gw ntpd[2112]: kernel time sync disabled 0041
> Jan 19 11:14:43 gw ntpd[2112]: synchronized to 62.48.35.100, stratum 2
> Jan 19 10:31:55 gw ntpd[2112]: time reset -3600.221385 s
> Jan 19 10:31:55 gw ntpd[2112]: kernel time sync enabled 0001
> Jan 19 10:31:55 gw dovecot: Time just moved backwards by 3600 seconds. 
> This might cause a lot of problems, so I'll just kill myself now. 
> http://wiki.dovecot.org/TimeMovedBackwards
Exactly one hour. Doesn't strike me as a coincidence...

  guenther


-- 
char *t="\10pse\0r\0dtu\0. at
ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8?
c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){
putchar(t[s]);h=m;s=0; }}}

Hi!

Am Samstag, 19. Januar 2008 schrieb Luigi Rosa:
> > How about first starting ntpd, forcing a time-update using ntpdate
> > even before starting ntpd, and delaying the dovecot start until after
> > the time has been adjusted?
> The problem is tha you cannot tell when the time is adjusted because
> it take some time to the ntp protocol to synchronize.
That's why I suggested to use ntpdate to force the correct time to be set 
before ntpd even starts and the ntp init script returns.

Admittedly, this won't work if you have no network connectivity during 
boot up, but if you have, this paticular problem is most likely solved.

ntpdate optionally can be given a timeout value and indicates success or 
failure via its return code so that a calling script might initiate 
appropriate measures.

Greetings,

  Gunter

-- 
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
"You're dead," he said. Keli waited. She couldn't think of any
suitable
reply. "I'm not" lacked a certain style, while "Is it
serious?" seemed
somehow too frivolous.        -- Princess Keli in trouble           
(Terry Pratchett, Mort)
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+                   PGP-verschl?sselte Mails bevorzugt!                 +
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part.
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20080119/06780553/attachment-0002.bin>

Gunter Ohrner said the following on 19/01/08 16:51:
> That's why I suggested to use ntpdate to force the correct time to be
set
> before ntpd even starts and the ntp init script returns.
> Admittedly, this won't work if you have no network connectivity during 
> boot up, but if you have, this paticular problem is most likely solved.
I don't remember if there already is an ntpdat on that particular machine,
but
many ntpd startup script have an ntpdate entry. I'll check.

But I don't think that this is the point.

According to Dovecot Wiki, "If all else fails, you can just go and remove
the
error checking code from src/lib/ioloop.c. It's unlikely that anything will 
break badly, but you might get some errors logged once in a while."

On one hand we have Dovecot that kill itself until someone kicks it back on, on 
the other hand we have just some errors on the logfile.

Well, IMHO at least there could be a configuration parameter that allows me to 
choose between a service that kills itself and some errors on a logfile and 
maybe a temporary service failure.

Each SysAdmin could weight the consequences and choose the option he/she thinks 
is more appropriate to his/her server.




Ciao,
luigi

-- 
/
+--[Luigi Rosa]--
\

Success is 99% failure.
     --Soichiro Honda

On some FreeBSD 6.2 systems I was solved this problem just by adding to crontab:
@hourly        /sbin/sysctl `/sbin/sysctl -e machdep.adjkerntz` >/dev/null
2>&1

For some explanation, see:
http://lists.freebsd.org/pipermail/freebsd-hackers/2005-August/013383.html

Quote follows:

... It seems that CMOS clock is updated using resettodr(9) function. There
seem to be only a few occasions when this function is called:

1. clock_settime(ClOCK_REALTIME) // through kern_time.c:settime()
2. settimeofday() // through kern_time.c:settime()
3. machdep.adjkerntz sysctl is set

I believe that ntpd calls *settime-family* functions *only* if time offset
is *larger* than *128ms* (or maybe I am thinking about ntpdate), but
normally it uses ntp_adjtime() to adjust time keeping. Obviosuly, a
system running ntpd with good enough hardware clock and good enough
connection to good enough ntp server(s) would use the latter method all
of the time after some initial stabilization period. But it seems that
time changes are *never propagated to CMOS* in this case, ...

Alexander