Flutter FirebaseAnimatedList rebuilds and moves widgets that have specific index to another place

Question

I have a chat which queries messages from firebase.

FirebaseAnimatedList(
        query: ChatService.instance.getMessagesQuery(widget.recipientUid),
        reverse: false,
        controller: scrollController,
        itemBuilder: (BuildContext context, DataSnapshot snapshot,
            Animation<double> animation, int x) {
          return buildRow(x, snapshot);
        })

I am trying to build a Text widget every time a message comes from new sent on day compared to previous as so

Widget buildRow(int index, snapshot) {
    List<Widget> rows = [];
    ChatMessage chatMessage = fromMap(snapshot);
    if (chatMessage.messageTimeStamp.day != lastMessage?.messageTimeStamp.day) {
      rows.add(Text(chatMessage.messageTimeStamp.toShortString()));
      lastMessage = chatMessage;
    }
    rows.add(buildChatRow(index, chatMessage));
    return Column(
      children: [...rows],
    );
}

When I open my chat, it seems ok

But as soon as I scroll down and scroll up a bit, it gets rebuilt and places time on different place.

Ideas please