Force Azure DevOps Server 2019 to manually sync with ActiveDirectory

Question

We have an on-premise Azure DevOps Server that works with a corporate ActiveDirectory. When adding new users, Azure DevOps Server pulls their information from ActiveDirectory. But the user's information was later updated in ActiveDirectory to fix an issue - their email account was missing.

In the past, I have been able to remove and re-add the user to Azure DevOps Server to fix the problem, as my administrative account has access and can see the user's email in ActiveDirectory. But the users are not being picked up by the sync job in Azure DevOps Server anymore, so their email address continues to be blank. (Users have been added for weeks or months without the update being picked up.)

We have verified that the Azure DevOps Server service account can see the email address in ActiveDirectory when logged into the server. So it's not an access issue with the service account.

How do I manually force Azure DevOps Server to run an ActiveDirectory sync? There used to be a JobService web service that I could access for this in previous versions of TFS, but that service doesn't appear to be available anymore, or is no longer scheduled to run.

I'm still looking for a solution to this issue. So far nothing that has been tried has worked. And we now have other sections about a user, such as a last name change, that aren't being picked up either. — Jeff, May 06 '20 at 20:41

score 3 · Accepted Answer · answered Jun 17 '20 at 21:04

Since no solution has worked, I decided to see what could be done from a coding perspective. The answer turned out to be straightforward. NOTE: Please make sure that you check the solutions provided below prior to the coding approach, as Azure DevOps Server is supposed to be refreshing identities automatically.

First, I found a Stack Overflow article about finding users by name:

TFS get user by name

This can be used to fetch a user or a group by its display name, among other attributes, using the ReadIdentity method.

This same IIDentityServiceProvider also has a method on it called RefreshIdentity. This method, when called with the IdentityDescriptor of the user, forces the identity to be immediately refreshed from its provider. See the documentation here:

https://learn.microsoft.com/en-us/previous-versions/visualstudio/visual-studio-2013/ff734945(v=vs.120)?redirectedfrom=MSDN

This method returns true if the refresh was successful, or false if the refresh failed. it is also possible for the refresh to throw an exception. For example, the Azure DevOps identity named "Project Collection Build Service" is listed as a user when retrieved, but this identity throws an exception when refreshed.

Using these methods, a complete tool was able to be constructed to repair the identities of individual users, or to scan through all users in the group "Project Collection Valid Users" and refresh the entire system. Using this tool, we were able to fix our synchronization issues between Azure DevOps Server and Active Directory.

Here's some sample code showing how to use these methods:

string rootSourceControlUrl = "TODO: Root URL of Azure DevOps";
string projectCollection = "TODO: Individual project collection within Azure DevOps";    

TfsTeamProjectCollection tfsCollection = TfsTeamProjectCollectionFactory.GetTeamProjectCollection(new Uri($"{rootSourceControlUrl}/{projectCollection}"));
IIdentityManagementService ims = (IIdentityManagementService)tfsCollection.GetService(typeof(IIdentityManagementService));
TeamFoundationIdentity foundUser = ims.ReadIdentity(IdentitySearchFactor.DisplayName, 
                                       "TODO: Display name of user", MembershipQuery.Direct, 
                                       ReadIdentityOptions.ExtendedProperties);
if(foundUser != null)
{
    try
    {
        if (ims.RefreshIdentity(foundUser.Descriptor))
        {
            // Find the user by its original IdentityDescriptor, which shouldn't change during the refresh
            TeamFoundationIdentity refreshedUser = ims.ReadIdentity(foundUser.Descriptor, 
                          MembershipQuery.Direct, ReadIdentityOptions.ExtendedProperties);

            // TODO : Display changes from foundUser to refreshedUser, using individual properties 
            //        and the method foundUser.GetProperties(), which returns an 
            //        IEnumerable<KeyValuePair<string, object>> collection.
        }
        else
        {
             // TODO : Notify that user failed to refresh
        }
    }
    catch(Exception exc)
    {
        // TODO : Notify that exception occurred
    }
}
else
{
    // TODO : Notify that user was not found
}

score 1 · Answer 2 · answered Mar 06 '20 at 06:56

1

TFS/Azure DevOps Server uses a background synchronization job, scheduled every hour, to look for changes in Active Directory. So changes you make to Active Directory groups do not get reflected in TFS immediately. Instead, TFS will synchronize those groups regularly (by default every hour).

You may try to restart TFS Job Agent service to see whether it helps.

answered Mar 06 '20 at 06:56

Cece Dong - MSFT

29,631
1
24
39

Actually, we did try restarting the TFS Job Agent service, and it did not fix the issue. We have also rebooted the Azure DevOps Server multiple times to apply Windows patches to the machine, so the service has been stopped and started multiple times. If there is no direct way to force the job, can you tell me what the background job is triggering off of to apply updates to users? It's almost like the job isn't seeing that the user has been updated in Active Directory, so it doesn't think it has anything to update. – Jeff Mar 09 '20 at 15:58
Could you try to delete the user and wait for one or two hours to re-add it? – Cece Dong - MSFT Mar 10 '20 at 09:07
I did as you suggested, but no luck. I removed the users and waited over 7 hours before adding them back. The users still did not synchronize, even after waiting an additional 2 hours for the Azure DevOps Server jobs to run. I did locate the appropriate job history table in the database, and I can see the periodic sync job updating the table, so it seems to be running. But it is not performing the sync. I also tried recycling the service again, and I'll check in the morning to see if it has an effect later. (Nothing immediately changed.) – Jeff Mar 11 '20 at 08:28
I should also mention that we are on Update 1 of Azure DevOps Server 2019, the latest version of the system. – Jeff Mar 11 '20 at 08:29
And I just noticed that update 1.1 came out yesterday. – Jeff Mar 11 '20 at 08:30
We're planning to do that. In the meantime, I went into Operational Intelligence and happened to find the Team Foundation Server Periodic Identity Synchronization Job. I can see one job running, the average run time is between 15 to 20 seconds from 3/9 through 3/11. But the Job History section isn't showing any job results over that timeframe. I would have thought the job would have been logging something. I also checked the Azure DevOps Server logs, and I'm not seeing anything being logged in those. Are there logs for this job somewhere? (BTW, these pages could use some search tools.) – Jeff Mar 11 '20 at 15:50
You may check the event log in Event Viewer on the server machine to see whether there is any useful information. – Cece Dong - MSFT Mar 12 '20 at 09:27
If I am reading the Event Viewer logs correctly, it is starting and completing the group synchronization for Sync Tree, and the times seem to correspond with that particular job history entry on the database. No errors were logged there. – Jeff Mar 13 '20 at 15:40
Have you upgraded your server? Does the issue persist after upgrade? – Cece Dong - MSFT Mar 16 '20 at 02:15
My apologies for the long delay, but the earliest that we could apply the patch was this weekend. The upgrade to Azure DevOps Server Update 1.1 did not fix the issue. I also tried removing and re-adding the users and stopping and starting the service again after Update 1.1 was applied. I also checked the event viewer, and I am not seeing any errors from the service. – Jeff Mar 16 '20 at 14:51
To investigate this problem from another direction, can you tell me which field(s) in Active Directory that the Azure DevOps Server is using to determine whether a user needs to be synchronized? Since Azure DevOps Server doesn't seem to want to synchronize these users, I'm wondering if there is an issue with the way our system administrators updated these users. Maybe Azure DevOps Server isn't seeing that there's something to update when there actually is. – Jeff Mar 16 '20 at 14:59
TFS synchronizes with AD depend on SID. Mr. Hinsh has a good troubleshooting guide, you may take a look at it: https://nkdagility.com/active-directory-groups-not-syncing-with-team-foundation-server-2010/. You may try to ask the system administrators delete and re-add this user. – Cece Dong - MSFT Mar 17 '20 at 07:18
I was able to verify that the SIDs between Azure DevOps and AD lined up. No errors in the event logs. And the Result code for the sync jobs are all 0. I turned on verbose logging for the job agent. It seems to be scanning every group that we have in TFS, both globally and the individual project level. But not every user is being picked up for synchronization. The first job only seemed to pick up 14 users. The second job only picked up 1, and he was in the original 14. I was expecting a lot more users to be picked up. Checking AD didn't seem to reveal anything obvious either. – Jeff Mar 17 '20 at 21:11
Would users no longer with the company be an issue? If the SID cannot be found and the user is still in a group, would the process just halt and not synchronize instead of generating an error in the logs? – Jeff Mar 17 '20 at 21:13
To be honest, I'm not sure about this scenario, I don't have environment to test this right now. Did you try to ask the system administrators delete and re-add this user? Will the user sync with AD? – Cece Dong - MSFT Mar 19 '20 at 09:24
Not yet. I am trying everything short of that, as our system administrators would not be happy having to do that. I am trying to eliminate any other possible sources of issues first. – Jeff Mar 23 '20 at 15:15
On Friday, I performed a security scrub of the users on the system, removing any users from Azure DevOps who were no longer with our company. I then let the sync run all weekend without touching it, then checked the logs this morning. No change in behavior. – Jeff Mar 23 '20 at 15:20
I asked our system administrators this morning, and they said no. This would have extensive impacts on our systems, including integrations with other software like Office 365. – Jeff Mar 23 '20 at 16:05
@Jeff We are currently on the newest version (2020.1.1 Patch 4) and are experiencing the same problem. Have you ever found a solution or how did you proceed with that? – roli09 Feb 07 '22 at 08:16

score -3 · Answer 3 · edited Jan 06 '21 at 19:22

-3

I tried all of above recommendations and none of them works!

Finally, this code solved my problem:

update [Tfs_Configuration].dbo.tbl_Identity
set 
    AccountName = 'New Name', 
    DistinguishedName = 'CN=*New Name*, OU=..., OU= ... ,OU=... ,OU=...,OU=...,DC=...,DC=...',
    MailAddress = 'New eMail'
where *Your Condition*

edited Jan 06 '21 at 19:22

Jeremy Caney

7,102
69
48
77

answered Jan 06 '21 at 08:59

Maziar Chitsaz

1

As I understand it, direct database updates to Azure DevOps (TFS) are not recommended and is considered a violation of the licensing/service agreement. – Jeff Jan 07 '21 at 17:02

Force Azure DevOps Server 2019 to manually sync with ActiveDirectory

3 Answers3