Flow architecture backend in python

Question

I'm currently trying to refactor a program from monolith to micro-service architecture and I'm not sure how to proceed:

The goal of the program is straight forward, it receives raw data (bytes) , split it and convert it to something meaningful and attach it back to the original message, some sort of enrichment. For example:

Input, message A:
{
    Id:
    Size:
    Data:
}

Output, message A:
{
    Id:
    Parsed Data: 
    {
        Field 1
        Field 2
    }
}

I'll add that according to the value Size a different app is needed (different routing).

And this can be separated to several layers of splitting and enriching. Up until now everything happened in one solution and it was huge and hard to maintain. So I decided to move to micro-service architecture. I started with Nameko with has build in support for rabbitmq but after a while I've noticed and I'm only using the @handle_event decorator and Dispatcher to publish the event and not the @rpc decorator.

So I have 3 questions:

1. Do you think micro service architecture is the best way to go here? Because I fill it's less request-recive and more publish-subscribe flow, one application manipulates the data then sends it to another application that dose the same.
2. What is the best practices and libraries to achieve this without wasting time working with Rabbitmq.
3. What is the best way to handle complex (more than just topic routing) routing with rabbitmq?

Answer 1

This sounds a lot more like a data processing pipeline than a microservice architecture. You could build one with Nameko but you'd be reinventing the wheel.

If you are building a data pipeline then something like apache beam would be more appropriate.