Aggregator gets empty responses when using Advanced Ocelot Request Aggregation

Question

My ocelot config looks like below (only relevant parts):

"Routes": [
    {
      "DownstreamPathTemplate": "/api/service1/json/{pageSize}/{pageNo}/all/{partnerId}",
      "DownstreamScheme": "http",
      "UpstreamPathTemplate": "/a",
      "UpstreamHttpMethod": [ "Get" ],
      "ServiceName": "Service1",
      "LoadBalancerOptions": {
        "Type": "LeastConnection"
      },
      "Key": "v0-service1",
      "Priority": 1
    },
    {
      "DownstreamPathTemplate": "/api/service2/json/{pageSize}/{pageNo}/{partnerId}",
      "DownstreamScheme": "http",
      "UpstreamPathTemplate": "/b",
      "UpstreamHttpMethod": [ "Get" ],
      "ServiceName": "Service2",
      "LoadBalancerOptions": {
        "Type": "LeastConnection"
      },
      "Key": "v0-service2",
      "Priority": 1
    }
    ],
    "Aggregates": [
    {
      "RouteKeys": [
        "v0-service1",
        "v0-service2"
      ],
      "UpstreamPathTemplate": "/api/services/{partnerId}?pageNo={pageNo}&pageSize={pageSize}",
      "Aggregator": "GetDataAggregator"
    }
]

The aggregator, for the time being, is defined as follows:

public class GetOffersAggregator : IDefinedAggregator
{
    public async Task<DownstreamResponse> Aggregate(List<HttpContext> responses)
    {
        var readers = responses.Select(r => new StreamReader(r.Response.Body)).ToList();
        var objects = await Task.WhenAll(readers.Select(r => r.ReadToEndAsync()));

        readers.ForEach(r => r.Dispose());

        throw new NotImplementedException();
    }
}

Now, if I put a breakpoint in both of the downstream services, it gets hit when I call out to "/api/services/{partnerId}?pageNo={pageNo}&pageSize={pageSize}". The issue is that both responses are 404 (even api gateway console logs 200 OK responses which is right, because both breakpoints are getting hit and services are returning data). If I inspect HttpContexts passed to my aggregator, I can see that the request object contains totally wrong address. It all looks as if the Ocelot was correctly invoking my endpoints but at the same time invoking some non existent ones and passing their results to my aggregator. Any ideas?

EDIT:

I can see my services responses inside the Items property of a HttpContext. Damn, it really looks like some serious error on the library authors side, or is it by design?

score 1 · Answer 1 · answered Mar 02 '21 at 20:54

I ran into the same situation and you just need to update how you reference the data. The data actually exists inside the responses[index].Items.DownstreamResponse().Content.

Here is a small example of what I've done that seems to work. Since the downstream content comes in compressed, I have two helper functions that decompress and convert the content into a Json object.

internal class DemoAggregator : IDefinedAggregator
    {
        public async Task<DownstreamResponse> Aggregate(List<HttpContext> responses)
        {
            List<Header> header = new List<Header>();
            try
            {
                var headers = responses.SelectMany(x => x.Items.DownstreamResponse().Headers).ToList();

                var oneByteArray = await responses[0].Items.DownstreamResponse().Content.ReadAsByteArrayAsync();
                var oneData = Decompress(oneByteArray);
                var oneObj = ConvertToJson(oneData);
                var oneContent = new StringContent(JsonConvert.SerializeObject(oneObj), Encoding.UTF8, "application/json");

                return new DownstreamResponse(oneContent, HttpStatusCode.OK, headers, "OK");
            }
            catch (Exception ex)
            {
                return new DownstreamResponse(null, System.Net.HttpStatusCode.InternalServerError, header, null);
            }
        }

        private static byte[] Decompress(byte[] data)
        {
            using (var compressedStream = new MemoryStream(data))
            using (var zipStream = new GZipStream(compressedStream, CompressionMode.Decompress))
            using (var resultStream = new MemoryStream())
            {
                zipStream.CopyTo(resultStream);
                return resultStream.ToArray();
            }
        }

        private static JObject ConvertToJson(byte[] data)
        {
            JObject jObj;
            using (var ms = new MemoryStream(data))
            using (var streamReader = new StreamReader(ms))
            using (var jsonReader = new JsonTextReader(streamReader))
            {
                jObj = (JObject)JToken.ReadFrom(jsonReader);
            }
            return jObj;
        }
    }

Aggregator gets empty responses when using Advanced Ocelot Request Aggregation

1 Answers1