Pekka L.J. Jalkanen
2013-May-10 11:04 UTC
[Samba] Sudden authentication failures, hex dumps in log.samba
In a leap of faith, I decided to relax the iptables rules on our Samba DC (4.0.5) on Wednesday, permitting some of our production clients to actually authenticate against it (in addition to our W2k3R2 DC). After all, there are no replication errors and no errors either in log.samba or Windows event log, so things _should've_ been generally working, and various test clients also have had no problems. To limit the fallout of potential failures I chose to do this on the eve of the Ascension Day (a public holiday where I live), knowing that almost all people would be off work on the following day, and that many people would also be having an extra day off today. Alas, things didn't go entirely smoothly. One person, who had came to work on Thursday afternoon despite the holiday, complained to me that he was having login problems (wrong username or password) and that only after first (successfully) logging on to a different workstation he, on a second attempt, managed to log on to his normal workstation. He also said that these problems had been repeated this morning. Given this information, I investigated log.samba and found the following: [2013/05/09 12:39:57, 0] ../lib/util/util.c:457(dump_data) [0000] 00 00 00 00 62 00 00 00 00 00 00 00 20 00 20 00 ....b... .... . . [0010] 20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 . . . . . . . . [0020] 20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 . . . . . . . . [0030] 20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 . . . . . . . . [0040] 20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 . . . . . . . . [0050] 20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 . . . . . . . . [0060] 20 00 20 00 20 00 20 00 20 00 20 00 50 00 00 . . . . . .P.. That hexdump with exactly the same contents was repeated 10 times yesterday afternoon and another 31 times this morning. The times of the dumps roughly matched the times of the logon failures. Question: how much more verbosity for log.samba would be needed to further investigate this problem? I'd rather not log everything with "-d10" for extended periods of time, because I really can't know how long it will take for the problem to reappear. I've now increased logging from the default level to "-d3". I also wish to turn on Kerberos logging in Samba so that I could have something akin to Windows's security log and see all successful and failed login attempts. Can this be achieved by normal krb5 logging settings in krb5.conf (as described on man 3 krb5_openlog)? Any recommended logging settings? Pekka L.J. Jalkanen
Pekka L.J. Jalkanen
2013-May-10 13:32 UTC
[Samba] Sudden authentication failures, hex dumps in log.samba
On 10.5.2013 14:04, Pekka L.J. Jalkanen wrote:> Question: how much more verbosity for log.samba would be needed to > further investigate this problem? I'd rather not log everything with > "-d10" for extended periods of time, because I really can't know how > long it will take for the problem to reappear. I've now increased > logging from the default level to "-d3"."-d3" logging pays off: [2013/05/10 14:31:05, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper) Kerberos: TGS-REQ someuser at MYDOMAIN.SITE from ipv4:10.10.59.151:4736 for cifs/w2k3r2dc.mydomain.site at MYDOMAIN.SITE [renewable, forwardable] [2013/05/10 14:31:06, 1] ../librpc/ndr/ndr.c:412(ndr_pull_error) ndr_pull_error(11): Pull bytes 2 (../librpc/ndr/ndr_basic.c:103) [2013/05/10 14:31:06, 0] ../lib/util/util.c:457(dump_data) [0000] 00 00 00 00 62 00 00 00 00 00 00 00 20 00 20 00 ....b... .... . . [0010] 20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 . . . . . . . . [0020] 20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 . . . . . . . . [0030] 20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 . . . . . . . . [0040] 20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 . . . . . . . . [0050] 20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 . . . . . . . . [0060] 20 00 20 00 20 00 20 00 20 00 20 00 50 00 00 . . . . . .P.. [2013/05/10 14:31:06, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper) Kerberos: Client no longer in database: someuser at MYDOMAIN.SITE [2013/05/10 14:31:06, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper) Kerberos: Failed building TGS-REP to ipv4:10.10.59.151:4736 [2013/05/10 14:31:06, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper) Kerberos: TGS-REQ someuser at MYDOMAIN.SITE from ipv4:10.10.59.151:4737 for cifs/w2k3r2dc.mydomain.site at MYDOMAIN.SITE [renewable, forwardable] [2013/05/10 14:31:06, 1] ../librpc/ndr/ndr.c:412(ndr_pull_error) ndr_pull_error(11): Pull bytes 2 (../librpc/ndr/ndr_basic.c:103) [2013/05/10 14:31:06, 0] ../lib/util/util.c:457(dump_data) [0000] 00 00 00 00 62 00 00 00 00 00 00 00 20 00 20 00 ....b... .... . . [0010] 20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 . . . . . . . . [0020] 20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 . . . . . . . . [0030] 20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 . . . . . . . . [0040] 20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 . . . . . . . . [0050] 20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 . . . . . . . . [0060] 20 00 20 00 20 00 20 00 20 00 20 00 50 00 00 . . . . . .P.. [2013/05/10 14:31:06, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper) Kerberos: Client no longer in database: someuser at MYDOMAIN.SITE [2013/05/10 14:31:06, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper) Kerberos: Failed building TGS-REP to ipv4:10.10.59.151:4737 [2013/05/10 14:31:20, 3] ../source4/dsdb/repl/drepl_service.c:202(_drepl_schedule_replication) Client is Windows XP. I've yet to see this problem on newer clients... this and the other one that previously failed are the last two XP clients here that still remain in heavy production use. What is also common with this client and the other that previously failed is that they both have once been migrated from a different domain (that no longer exists) using MS ADMT. This also applies to the users' accounts that were used. Don't know if that really matters, but just for the record. Any ideas how to resolve this problem? Pekka L.J. Jalkanen