How to correctly set APIM Store in a distributed setup with apache front-end?

Question

Scenario of WSO2 APIM 2.6.0:
1 VM for API Publisher -> Front-END: https://mgt.mydomain (manager of gateways workers)
1 VM for API Store -> Front-END: https://store.mydomain
1 VM for Traffic Manager -> Front-END: https://trafficmanager.mydomain
2 VM for Gateway cluster -> LB : https://api.mydomain (Gateways workers)
2 VM for IS as KM cluster -> LB : https://security.mydomain

carbon.xml at Store:

<HostName>store.mydomain</HostName>
<MgtHostName>store.mydomain</MgtHostName>

apimanager.xml at Store:

        <Environment type="hybrid" api-console="true">
            <Name>MyGateway</Name>

            <ServerURL>https://api.mydomain:443/services/</ServerURL>
            <!-- Admin username for the API gateway. -->
            <Username>${admin.username}</Username>
            <!-- Admin password for the API gateway.-->
            <Password>${admin.password}</Password>
            <!-- Endpoint URLs for the APIs hosted in this API gateway.-->
           <GatewayEndpoint>http://api.mydomain:80,https://api.mydomain:443</GatewayEndpoint>
            <!-- Endpoint of the Websocket APIs hosted in this API Gateway -->
            <GatewayWSEndpoint>ws://${carbon.local.ip}:9099</GatewayWSEndpoint>
        </Environment>
    </Environments>
</APIGateway>

After published my test API, when I try to test It, the store use It´s as Request URI:

Request URL: https://store.mydimain/testAPI/v1.0.0/info
Sample Request from API Store:
curl -k -X GET "https://store.mydomain/testAPI/v1.0.0/info" -H "accept: application/json" -H "Authorization: Bearer 1fcf7e3f-8292-3110-8cff-27226caa59a8"

The request maded directly to Gateway LB are OK.
Sample request:
curl -k -X GET "https://api.mydomain/testAPI/v1.0.0/info" -H "accept: application/json" -H "Authorization: Bearer 1fcf7e3f-8292-3110-8cff-27226caa59a8"
Response:{"version":"1.0.0","nome":"test-api"}

Additional Info: There is a NFS mounted between Gateways Workers
There is a NFS mounted between IS-KMs
The Pubisher should be the Gateway manager

Additional Info It was possible to observer that, after published the API, the overview in APIM Publisher and APIM store don´t show the endpoints:

But my apimanager.xml in Publisher Server looks like correctly config as follow:

apimanager.xml inside PUBLISHER:

<APIGateway>
        <Environments>
            <Environment type="hybrid" api-console="true">
                <Name>Production and Sandbox</Name>
                <Description>This is a hybrid gateway that handles both production and sandbox token traffic.</Description>
                <!-- Server URL of the API gateway -->
                <ServerURL>https://api.mydomain:443/services/</ServerURL>
                        <!-- Admin username for the API gateway. -->
                <Username>${admin.username}</Username>
                <!-- Admin password for the API gateway.-->
                <Password>${admin.password}</Password>
                <!-- Endpoint URLs for the APIs hosted in this API gateway.-->
                <GatewayEndpoint>http://api.mydomain:80,https://api.mydomain:443</GatewayEndpoint>
                <!-- Endpoint of the Websocket APIs hosted in this API Gateway -->
                <GatewayWSEndpoint>ws://${carbon.local.ip}:9099</GatewayWSEndpoint>
            </Environment>
        </Environments>
    </APIGateway>

New Update

for VirtualServer api.mydomain:443 insideapache my configs are:

<VirtualHost api.mydomain:443>

        SSLEngine on
        SSLCipherSuite HIGH:!aNULL:!MD5
        SSLHonorCipherOrder on
        ...
        ServerName api.mydomain

        CustomLog /var/log/httpd/api.log combined
        ErrorLog /var/log/httpd/api.error.log
        LogLevel debug

        # disable forward proxy requests
        ProxyRequests off
        SSLProxyEngine On
        SSLProxyCheckPeerCN off
        SSLProxyCheckPeerName off
        SSLProxyCheckPeerExpire off
        ProxyPreserveHost On

        UseCanonicalName On

        Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" env=BALANCER_ROUTE_CHANGED

    <Proxy balancer://apissl_nio>
        # Add a member to the load balancing group
        BalancerMember https://gtw01.mydomain:8243 route=1
        BalancerMember https://gtw02.mydomain:8243 route=2
        ProxySet stickysession=ROUTEID
        ProxySet lbmethod=byrequests
    </Proxy>
    ProxyPass /revoke balancer://apissl_nio/revoke
    ProxyPassReverse /revoke balancer://apissl_nio/revoke

    ProxyPass /token balancer://apissl_nio/token
    ProxyPassReverse /token balancer://apissl_nio/token

    <Proxy balancer://apissl_mgt>
        # Add a member to the load balancing group
        BalancerMember https://gtw01.mydomain:9443 route=3
        BalancerMember https://gtw02.mydomain:9443 route=4
        ProxySet stickysession=ROUTEID
        ProxySet lbmethod=byrequests
    </Proxy>

    ProxyPass /services balancer://apissl_mgt/services
    ProxyPassReverse /services balancer://apissl_mgt/services


</VirtualHost>

That's strange. It should use the URL you set in ``. Do you have any other ``s? — Bee, Aug 30 '19 at 16:26

score 0 · Answer 1 · answered Sep 03 '19 at 07:07

The ServerURL is used for the management services of the Gateway, not the NIO. You should enable access to 9443/9763 port or create a new virtual host to balance 9443 and/or 9763 ports of the gateway instance.

    <Environment type="hybrid" api-console="true">
        <Name>Production and Sandbox</Name>
        <Description>This is a hybrid gateway that handles both production and sandbox token traffic.</Description>
        <!-- Server URL of the API gateway -->

        <ServerURL>https://gwmng:9443${carbon.context}services/</ServerURL>
                <!-- Admin username for the API gateway. -->
        <Username>${admin.username}</Username>
        <!-- Admin password for the API gateway.-->
        <Password>${admin.password}</Password>
        <!-- Endpoint URLs for the APIs hosted in this API gateway.-->
        <GatewayEndpoint>http://gw:80,https://gw:443</GatewayEndpoint>
        <!-- Endpoint of the Websocket APIs hosted in this API Gateway -->
        <GatewayWSEndpoint>ws://gw:9099</GatewayWSEndpoint>

Hi falcon! thanks for the answer, but my load balancer is maping the **https://api.mydomain:443/services/** for **9443** port in the Gateways worker. I had make a upgrade I my question to clearly this point... — ChelloFera, Sep 03 '19 at 19:06
Exactly. But I found the problem as you can see in my answer. Thanks.... — ChelloFera, Sep 04 '19 at 13:41

score 0 · Answer 2 · answered Sep 04 '19 at 13:25

First, Thanks for all comments and answers. I found the problem and its lies on

<APIGateway>
    <Environments>
      <Environment>
          <Name>****</Name>

I had deploy the Store with different Environment Name than the others components. Since the Publisher uses the Name to choose the correct gateways to deploy the API, The Stores don´t find the endpoints, even if your ServerURL and GatewayEndpoint are correct filled.

How to correctly set APIM Store in a distributed setup with apache front-end?

2 Answers2