Author: Ofer Nissim, Solution Architect, HP

Date: 11/08/2008

Symptom

When configuring KCD (Kerberos Constrained Delegation) on ISA/IAG server, users that were trying to authenticate, prompted for a client certificate and then got HTTP 401 Unauthorized error message.

There are a lot of troubleshooting documents on HTTP 401error message over the internet but none of them was helpful in the following scenario.

After performing debugging processes on the client side and on the ISA/IAG server side we found that the client certificate wasn't trusted by the Windows OS on the ISA/IAG server

Cause

The problem was due to the "Autoenrollement" component having been disabled in the Group Policy. The "Autoenrollement" engine controls the population of the NTAuth registry key which is the key that determines which Root CA's are trusted for client certificate authentication and mapping.

"AutoEnrollement" is ENABLED by default so this is something that has been explicitly disabled in a GPO.

Resolution Procedures

Enabling auto-enrollment for the ISA/IAG server via GPO resolved this problem

To force the machine to attempt repopulate the NTAuth registry key, please do the following:

Delete Registry Key

1. Delete the AEDirectoryCache key to force the client to consider downloading the "NTAuthCertificates" container in AD

    Delete the following key:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Cryptography\AutoEnrollment\AEDirectoryCache

    Note: This key hold a count of objects in the AD, to avoid unneeded downloads

Force Download using Certificates MMC for local machine

1. Load the MMC
2. Add the Certificates Snap-in
3. Select "Computer Account" when prompted with choice for "always manage certificates for"
4. Click on the "certificates(local computer)" icon
5. Right click and select "all tasks"  - automatically Enroll Certificates

    This should repopulate the following keys….

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Cryptography\AutoEnrollment\AEDirectoryCache

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\EnterpriseCertificates\NTAuth

If you have any issues with these procedures please go to IAG Technical Forums in the following link: http://Forums.ForefrontSecurity.ORG