How to configure and enable Azure Service Fabric Reverse Proxy for an existing on-premises cluster?

Question

Is the Azure Service Fabric Reverse Proxy available in an on-premises cluster? If so, how can I enable it for an existing cluster?

The Service Fabric Reverse Proxy is described here. It allows clients external to the cluster to access application services by name with a special URL, without needing to know the exact host:port on which an instance of the service is running (which may change as services are automatically moved around).

By default the Service Fabric Reverse Proxy does not appear to be enabled for my on-prem cluster with two instances of a stateless service. I tried using the documented port 19008 but could not reach the service using the recommended URI syntax.

To wit, this works:

http://fqdn:20001/api/odata/v1/$metadata

but this does not:

http://fqdn:19008/MyApp/MyService/api/odata/v1/$metadata

In the NodeTypes section of the ClusterConfig JSON used to set up my on-prem cluster, there is a property "httpGatewayEndpointPort": "19080", but that port does not appear to work as a reverse proxy (it is the Service Fabric Explorer web-app endpoint). I am guessing that the needed configuration is specified somehow in the cluster config JSON. There are instructions in the referenced article that explain how to configure the reverse proxy in the cloud, but not on-premises.

What I am looking for are instructions on how to set up the Service Fabric reverse proxy in an on-premises multi-machine cluster or dev cluster.

Answer 1

Yes, the reverse proxy is available on-premises.

To get it working for an existing cluster, it must be configured and enabled in the cluster config XML and then the new config must be deployed, as described below.

For a new cluster, set it up in the cluster config JSON before creating the cluster, as described by @Scott Weldon.

@Senj provided the clue (thanks!) that led me to the answer. I had recently updated my Service Fabric bits on my dev box to 5.1.163.9590. When I looked in C:\SfDevCluster\Data\FabricHostSettings.xml, I noticed the following:

 <Section Name="FabricNode">
    ...
    <Parameter Name="NodeVersion" Value="5.1.163.9590:1.0:0" />
    ...
    <Parameter Name="HttpApplicationGatewayListenAddress" Value="19081" />
    <Parameter Name="HttpApplicationGatewayProtocol" Value="http" />
    ...
  </Section>

Interesting! With the dev cluster fired up, I browsed to:

http://localhost:19081/MyApp/MyService/api/odata/v1/$metadata

and voila! My API returned the expected data. So @Senj was correct that it has to do with the HttpApplicationGateway settings. I am guessing that in the latest SDK version it is pre-configured and enabled by default. (What threw me off is all the docs refer to port 19008, but the actual configured port was 19081!)

In order to get the reverse proxy to work on the 'real' multi-machine (VM) cluster, I did the following (Note: I don't think upgrading the cluster codepackage was necessary, but since I had nothing in my image store for the cluster upgrade, and the cluster upgrade process requires a code package, I used the latest version):

1. Copy the existing cluster manifest (from the Manifest tab in Service Fabric Explorer), paste into a new XML file, bump the version number and modify as follows:

To the NodeType Endpoints section, add:

<NodeTypes>
    <NodeType Name="NodeType0">
      <Endpoints>
        <HttpApplicationGatewayEndpoint Port="19081" Protocol="http" />
        ...
      </Endpoints>
    </NodeType>
</NodeTypes>

and under <FabricSettings>, add the following section:

<Section Name="ApplicationGateway/Http">
  <Parameter Name="IsEnabled" Value="true" />
</Section>

2. Using Service Fabric PowerShell commands:

   • Copy the new cluster config (the previously copied manifest.xml) to the fabric image store
   • Register the new cluster config
   • Copy the Service Fabric Runtime cluster codepackage (available here - see the release notes for the link to the MSI) to the image store
   • Register the cluster codepackage
   • Start and complete cluster upgrade (I used unmonitored manual mode, which does one VM at a time and requires a manual Resume command after each node is complete)

After the cluster upgrade was complete, I was able to query my service API using the reverse proxy endpoint and appname/servicename URL syntax:

http://fqdn:19081/MyApp/MyService/api/odata/v1/$metadata

Answer 2

I enabled this in the standalone installer version (5.1.156) by adding the following line to the JSON configuration file under the nodeTypes element (I used ClusterConfig.Unsecure.MultiMachine.json but I assume any of the JSON files would work):

"httpApplicationGatewayEndpointPort": "19081"

So the final nodeTypes looked like this:

"nodeTypes": [
    {
        "name": "NodeType0",
        "clientConnectionEndpointPort": "19000",
        "clusterConnectionEndpoint": "19001",
        "httpGatewayEndpointPort": "19080",
        "httpApplicationGatewayEndpointPort": "19081",
        "applicationPorts": {
            "startPort": "20001",
            "endPort": "20031"
        },
        "ephemeralPorts": {
            "startPort": "20032",
            "endPort": "20062"
        },
        "isPrimary": true
    }
]

Answer 3

I think it has something to do with the HttpApplicationGatewayEndpoint property, see also my question on https://github.com/Azure/service-fabric-issues/issues/5 But it doesn't work for me..

Also notice that

<Section Name="ApplicationGateway/Http">
            <Parameter Name="IsEnabled" Value="true" />
</Section>

is true for me.

Edit:

I noticed that on my Windows-Only installation, HttpApplicationGatewayListenAddress has value 0 in the FabricHostSettings.xml

<Parameter Name="HttpGatewayListenAddress" Value="19080" />
<Parameter Name="HttpGatewayProtocol" Value="http" />
<Parameter Name="HttpApplicationGatewayListenAddress" Value="0" />
<Parameter Name="HttpApplicationGatewayProtocol" Value="" />