Over the weekend I completed the installation and testing of a Cisco ASA 5512 to replace our old router/firewall. Today, I've been wrestling with an issue of browsers inconsistently failing to connect via SSL to exchange OWA over the VPN. Sometimes computers will boot up just fine, connect to the VPN, and connect to exchange OWA just fine, other times they won't connect. Once they've either connected or not, they appear to work that way consistently until the computer is rebooted.
Running a packet capture on the ASA appears to show that when it's successful it's using a previously established SSL session and when it fails it appears to be during the establishment of the session, but I have to admit my network diagnostic skills aren't exactly flawless. The packet capture during failure doesn't show any ACKs for the packets being sent out by the server. The server is sending out ACKs for the client's packets however.
A simple pcap file that contains a successful connection and a failed connection can be found Here. Is my assessment of the cause accurate? What might be causing the issue and what might be a solution of further steps to troubleshoot the issue?
Edit: Day two updates.
The problem appears related to MTU sizes, the VPN, and the ASA not tranmitting ICMP unreachable packets when it needs to fragment data but cannot.
Using pings of various sizes with the Don't Fragment bit set and changes to the ASA's MTU on the internet facing port I discovered the following:
I can ping both internet and VPN connected computers just fine with small packets.
When the router's internet-facing port is set to an MTU of 1500 I can ping computers on the internet with packets up to 1472, above which my computer (which is also set to an MTU of 1500) tells me that the packet must be fragmented. This is exactly as I would expect so far.
When the router's internet-facing port is set to an MTU of 1500 I can only ping computers connected to VPN with packets up to 1356, after which the packets time out. This is not what I would expect. Even if the packet size was to large, I should be getting a ICMP response back stating that the packet was dropped because it needed to be fragmented and the DF bit was set.
Once I drop the router's internet-facing port down to an MTU of 1400, I can ping computers on the internet with packets on to 1372, after which the packets time out. Once again, we've got a problem. I would expect to only be able to send packets up to 1372, but at 1373 (where we're below my computer's MTU but below the router's MTU with the 28 byte headers) the router should be sending me back a response saying the packet needs to be fragmented but the DF bit is set.
There's also a corresponding drop to 1256 when attempting to ping computer's connected via VPN, with no response for pings with more bytes.
Shouldn't the ASA be replying with ICMP packets when it can't forward a packet? During the process of setting up the router is there a setting somewhere that I accidentally activated that disabled this functionality?
