45

The certificate for our Azure blob storage expired today. This is not a certificate provided by us but provided by Microsoft as show in the picture below. How does one go about fixing this? I have tried searching for a solution but found nothing. Our app cannot connect to the storage as the certificate has expired and we are getting an error indicating: Could not establish trust relationship for the SSL/TLS secure channel

enter image description here

Nick Olsen
  • 6,299
  • 11
  • 53
  • 75
  • Wow, thanks to your post I realized that I am not able to use some Azure Blob browsing tools such as CloudBerry (SSL issue: `The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.`). – Tom Feb 22 '13 at 20:59
  • 3
    I opened a support case with Microsoft. This affects the *.blob.core.windows.net SSL cert, which in turn affects all Blob Storage accounts Worldwide. Someone at Microsoft should lose their job for this. – Brian Clark Feb 22 '13 at 21:12
  • Scott Hanselman is working on it now, you can track it on Twitter: https://twitter.com/search/realtime?q=azure+blob&src=typd – Nico Westerdale Feb 22 '13 at 21:37
  • 5
    The problem is being investigated seriously and dashboard is updated with the current status. Please keep an eye on dashboard for frequent updates: http://www.windowsazure.com/en-us/support/service-dashboard/ – AvkashChauhan Feb 22 '13 at 21:12
  • 12
    **Moderator Note:** Please keep the noise on this post to a minimum. You know about it, I know about it, and now Microsoft knows about it. We all agree it's a travesty; save your editorial comments for Twitter. *There's a temporary workaround already posted below.* – Robert Harvey Feb 22 '13 at 22:18

4 Answers4

26

As a temporary measure I was able to log into the azure portal and change the protocol part of the connection string from https to http.

Robert Harvey
  • 178,213
  • 47
  • 333
  • 501
dr. foot fist headknocker
  • 386
  • 4
  • 7
  • 13
    Keep blobs containing sensitive data accessible only over HTTPS. – Bill Hoag Feb 22 '13 at 21:40
  • How do you change the protocol? I don't see it in the portal. – Nico Westerdale Feb 22 '13 at 21:43
  • 3
    On the configure tab scroll down to settings, find your connection string and change `DefaultEndpointsProtocol=https;` to `DefaultEndpointsProtocol=http;` – dr. foot fist headknocker Feb 22 '13 at 21:45
  • 3
    Be careful. If you're changing it in portal, your previously deployed package may not reload because it is stored in Azure storage (I believe). Doing this caused the first 2 worker roles in my pool to become unresponsive and continuously recycle. – aarondcoleman Feb 22 '13 at 22:22
  • If you do this, be sure to leave the connection string for Diagnostics running with HTTPS. If you change it to HTTP, the monitor throws an exception on startup that will cause your roles to recycle endlessly. – Brian Reischl Feb 23 '13 at 00:03
5

Two more possible solutions if you can RDP into your roles.

  1. Change the configuration manually in the c:\Config directory.
  2. Build a DLL that's patched to work around the problem, and manually upload it via RDP. The workaround could be hardcoded connection strings, or put in code to accept expired certs. For example: ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };

(Hat tips to AlexNS on MSDN forums for idea #2 and to Jason Vallery for the cert validation callback code)

As noted in the comments, disabling HTTPS and/or ignoring certificate validation errors can compromise the security of your communications. So think hard before you do this, and change it back as soon as Microsoft fixes this problem.

Brian Reischl
  • 7,216
  • 2
  • 35
  • 46
  • 3
    Just be warned to all that this could effectively enable [MITM](http://en.wikipedia.org/wiki/Man-in-the-middle_attack) type attacks. So if you do this, do this knowing the risk you expose yourself to and revert it the second Microsoft fixes this!! – Jaxidian Feb 22 '13 at 23:04
1

We were able to dodge most of this in the first place through explicit use of HTTP endpoints for storage (we don't store anything too sensitive there).

In case you're in a similar situation and can do with HTTP endpoints, there is a workaround that allows you to upgrade your roles permanently. It involves Azure Powershell deployments with local packages and seems to work even when upgrades via the both portals continue to fail.

Nariman
  • 6,368
  • 1
  • 35
  • 50
0

Just as a note - if you switch to http from https then the transfer mechanism no longer makes sure the data is transferred correctly, and you may need to check the MD5 of the blob.

StorageClient < 2.0 manages this sometimes with uploads, but reading this article, never from downloads.

For StorageClient 2.0, you may need to change the BlobRequestOptions to UseTransactionalMD5 (as detailed here)

Rob Church
  • 6,783
  • 3
  • 41
  • 46