ECP & OWA aren't working - "Incorrect username/password"

Question

I haven't been here for some time.. This time I think I have one of those "Rocket Science" problems, so shall I start?

alright, tl;dr - I started to work in a company as a Sysadmin and the last guy that I replaced really messed some stuff around and I'm spinning around trying to fix them..

I'm going to try to sum up everything in one post to avoid being asked the same questions over and over again.

The Problem:

I cannot access ECP/OWA, no matter which credentials I give it (and they are validated as correct vs Outlook itself) - Outlook works, ECP/OWA does not.

The error I get, no matter where I access it from (Internally / Locally) -

"The user name or password you entered isn't correct. Try entering it again."

- I think the problem relies within owa (Exchange Back End) / ecp (Exchange Back End), as I tried various solution suggestions I may have deleted the back end Virtual Directory to recreate them.

Some Info:

OS and Exchange: Windows Server 2016, Exchange 2016

Exchange CU Version: CU6

Logs & Debugging:

Event Viewer:

The Outlook Web App configuration settings couldn't be read and updated. Virtual directory: "owa". Web site: "Exchange Back End".
Error message:
"The Active Directory configuration settings couldn't be accessed for virtual directory "owa" under Web site "Exchange Back End"."

-> Source: MSExchangeOWA
-> Event ID: 64
--> Qualifiers: 49152

Image -

IIS:

W3SVC1 (Default Web Site?) + W3SVC2 (Exchange Back End?) log files don't say much actually , no indication of errors when I try to login. Here's a few lines I found (but its about health mail boxes);

2018-07-19 00:28:34 ::1 POST /owa/proxylogon.owa &ClientId=Some_Content_Here&ClientRequestId=&ActID=Some_Content_Here&CorrelationID=<empty>&userContextLogonIdentityName=DOMAIN_NAME\HealthMailboxc66d8b0&userContextLogonIdentitySid=Some_Content_Here&userContextMbGuid=Some_Content_Here&redir=lang 444 DOMAIN_NAME\HealthMailboxc66d8b0 ::1 Mozilla/4.0+(compatible;+MSIE+11.0;+Trident/7.0;+rv:11.0;+Windows+NT+6.1;+MSEXCHMON;+ACTIVEMONITORING;+EACBACKENDLOGON) - 302 0 0 3768
2018-07-19 00:28:34 ::1 GET /ecp/About.aspx ActID=Some_Content_Here 444 - ::1 Mozilla/4.0+(compatible;+MSIE+11.0;+Trident/7.0;+rv:11.0;+Windows+NT+6.1;+MSEXCHMON;+ACTIVEMONITORING;+EACBACKENDLOGON) - 401 1 2148074254 3
2018-07-19 00:28:34 ::1 GET /ecp/About.aspx ActID=Some_Content_Here 444 DOMAIN_NAME\HealthMailboxc66d8b0 ::1 Mozilla/4.0+(compatible;+MSIE+11.0;+Trident/7.0;+rv:11.0;+Windows+NT+6.1;+MSEXCHMON;+ACTIVEMONITORING;+EACBACKENDLOGON) - 302 0 0 82
2018-07-19 00:28:34 ::1 GET /owa/languageselection.aspx url=%2fecp%2fAbout.aspx&ClientId=Some_Content_Here&ClientRequestId=&ActID=Some_Content_Here&CorrelationID=<empty> 444 - ::1 Mozilla/4.0+(compatible;+MSIE+11.0;+Trident/7.0;+rv:11.0;+Windows+NT+6.1;+MSEXCHMON;+ACTIVEMONITORING;+EACBACKENDLOGON) - 401 1 2148074254 2
2018-07-19 00:28:34 ::1 GET /owa/auth/error.aspx url=%2fecp%2fAbout.aspx&ClientId=Some_Content_Here&ClientRequestId=&ActID=Some_Content_Here&CorrelationID=<empty> 444 DOMAIN_NAME\HealthMailboxc66d8b0 ::1 Mozilla/4.0+(compatible;+MSIE+11.0;+Trident/7.0;+rv:11.0;+Windows+NT+6.1;+MSEXCHMON;+ACTIVEMONITORING;+EACBACKENDLOGON) - 200 0 0 17

ADSI vs IIS:

You can see that there is no "owa (Exchange Back End) / ecp (Exchange Back End)", that might be the problem.. didn't have time to compare these vs my local hosted mail server to confirm.

This is in:

CN=HTTP,CN=Protocols,CN=Mail_Server,CN=Servers,CN=Exchange Administrative Group (GUID_HERE),CN=Administrative Groups,CN=DOMAIN_NAME,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=DOMAIN_NAME,DC=local

IIS:

• Default Web Site

• Exchange Back End

I think it'll be important to mind that I've had a lot of problems before that and they have been fixed and that one popped up (probably my mistake) recently after solving a lot of errors that came before that about OWA.

Believe me I dug every hole in the internet to find a solution without success, I have a final solution planned (as a Plan B at the moment) which is upgrading Exchange from CU6 to CU10 (planned to happen soon) but I can't really do that at the moment, keeping in mind that those are production servers and I cannot do whatever I want.

Tried solutions:

1. Recreating virtual directories (including webApplications) & Recycling AppPools (OWA & ECP)

2. Changing authentication methods and SSL settings back to default (https://learn.microsoft.com/en-us/exchange/clients/default-virtual-directory-settings) + comparing to a local mail server hosted at home.

3. Checking permissions (permissions are fine)

4. Checking Bindings and SSL cert attached to https bindings

5. Comparing IIS config files found at C:\Windows\System32\inetsrv\config\ vs My local hosted Mail Server (didn't really find much difference)

6. Restarting IIS ofcourse (tons of times) and Rebooting

7. Analyzing with Exchange Analyzer (https://gallery.technet.microsoft.com/office/Exchange-Analyzer-6e20132e) - no critical errors or anything noticeable relating ECP / OWA / Webservices

8. Updating CAS (C:\Program Files\Microsoft\Exchange Server\V15\Bin\UpdateCas.ps1)

9. Testing Exchange connectivity (https://testconnectivity.microsoft.com/) - No errors whatsoever

10. More (can't remember anymore.. too much)

I hope all of this helps analyzing the problem and fixing it , hope we can find a fix for this without having to upgrading exchange / reinstalling and thanks for reading

Answer 1

I have finally fixed the problem!

Here's what I did for reference to people having the same or familiar problem:

NOTE: You are going to need to have an Exchange 2016 server with a working ECP/OWA to make a comparison between the broken Machine's files and fix the problem (I have installed a local Virtual Machine at my home's PC, you can do so too)

Fixing EventID 64 @ Event Viewer: This is for people getting this error @ Event Viewer

The Outlook Web App configuration settings couldn't be read and updated. Virtual directory: "XXX". Web site: "XXX".
Error message:
"The Active Directory configuration settings couldn't be accessed for virtual directory "XXX" under Web site "XXX"."

-> Source: MSExchangeOWA
-> Event ID: 64
--> Qualifiers: 49152

I was suspecting that this was the problem and after some research I have found this article (follow the article): https://dirteam.com/dave/2010/12/23/fixing-a-broken-owa-2010-virtual-directory/

In my situation, after doing the steps in the article the errors went away but I still Couldn't log-in!

I have had no errors anymore, not in the Event Viewer or IIS logs so, I have been thinking to myself that maybe the same way I have been doing in https://dirteam.com/dave/2010/12/23/fixing-a-broken-owa-2010-virtual-directory/ to fix the ADSI Object of ECP and OWA That I would do the same concept but instead of comparing between ADSI's, this time maybe Comparing between an example machine's working ECP/OWA config files and a broken ECP/OWA config files may reveal the problem to me!

So, I fired up my local Exchange 2016 server back at home and compared 3 Files using https://www.diffchecker.com/ to check what is wrong.

I have gone ahead and compared between those 3 web.config files located at: Text

[Exchange_Install_Drive]\Program Files\Microsoft\Exchange Server\V15\FrontEnd\HttpProxy\ecp

[Exchange_Install_Drive]\Program Files\Microsoft\Exchange Server\V15\FrontEnd\HttpProxy\owa

C:\inetpub\wwwroot

To my surprise I have found some wrong and empty parameters in those files , so I went ahead and made a backup for those files and carefully removed those parameters, saved those files and restarted the IIS service (iisreset)

ECP and OWA are now fully working for me!

Hope this helps anyone!