Android TabLayout scroll to random position when new Tabs are added

Question

I have a TabLayout with scrollable tabs that I setup with a TabLayoutMediator attached to a ViewPager2, and when I add new tabs to the TabLayout it scrolls to a random position.

When I create the TabLayoutMediator I tried to set autoRefresh parameter to false and handle the addition/removal of the tabs manually but scrolling issue is still there.

My code is the following:

  tabLayoutMediator = TabLayoutMediator(tabs, viewPager, false) { tab, position ->
        tab.text = items[0].title
  }
  tabLayoutMediator.attach()

and when I add new tab:

  viewModel.loadedTabsStrings.observe(this) {
        tabs.removeAllTabs()
        it?.forEach { title ->
            val tab: TabLayout.Tab = tabs.newTab().setText(title)
            tabs.addTab(tab)
        }
    }

What I want is to add and remove tabs in the TabLayout keeping the same scroll position that user left.