ImageView SharedElement Resized Before Transition

Question

I have a MainActivity holding 2 fragments - MainFragment and SlideshowFragment. Both fragments are using ViewPager with FragmentStatePagerAdapter to hold childFragments.

When an ImageView in ChildFragmentA of MainFragment is clicked, it is to expand and fill the PhotoView in ChildFragmentB of SlideshowFragment. The ImageView is held in a RecyclerView.

I've referred to the Android Official Guide here, looked that its GitHub code and had made the required change to make sure the transition names and callback are properly set.

However, ImageView is using centerCrop in xml, while PhotoView is using fitCenter. The result is that before the transition happen, the image will expand first, which is very unpleasant. Gif here (irrelevant, but appreciate if someone can let me know how to embed gif).

Below are some of the relevant codes.

Loading of image into view with Glide

GlideApp.with(getActivityContext())
    .load(filePath)
    .dontTransform()    // I found some posts that say this should be set
    .listener(object : RequestListener<Drawable> {
        override fun onLoadFailed(e: GlideException?, model: Any?, target: Target<Drawable>?,
                                  isFirstResource: Boolean): Boolean {
            startPostponedEnterTransition()
            return false
        }

        override fun onResourceReady(resource: Drawable?, model: Any?, target: Target<Drawable>?,
                                     dataSource: DataSource?, isFirstResource: Boolean): Boolean {
            startPostponedEnterTransition()
            return false
        }
    })
    .into(imageView)

Preparation of transition in SlideshowFragment

private fun prepareSharedElementTransition() {
    val transition = TransitionInflater.from(getActivityContext())
            .inflateTransition(R.transition.image_shared_element_transition)
    transition.duration = 1000
    sharedElementEnterTransition = transition
    sharedElementReturnTransition = transition    // it wasn't working, so I tried adding this line too
    setEnterSharedElementCallback(
            object : SharedElementCallback() {
                override fun onMapSharedElements(names: List<String>, sharedElements: MutableMap<String, View>) {
                    // mapping stuff here done correctly
                }
            })
}

XML

<!-- In ChildFragmentA -->
<ImageView
    android:id="@+id/thumbnailLayout_thumbnail"
    android:layout_width="match_parent"
    android:adjustViewBounds="true"
    android:layout_height="match_parent"
    android:scaleType="centerCrop" />

<!-- In ChildFragmentB -->
<com.github.chrisbanes.photoview.PhotoView
    android:id="@+id/childFragmentB_photoView"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:adjustViewBounds="true"
    android:layout_centerInParent="true"
    android:scaleType="fitCenter" />

<!-- In image_shared_element_transition.xml -->
<transitionSet xmlns:android="http://schemas.android.com/apk/res/android"
    android:interpolator="@android:interpolator/fast_out_slow_in"
    android:transitionOrdering="together">
    <changeBounds/>
    <changeClipBounds/>
    <changeTransform/>
</transitionSet>

Really appreciate any help as I have spent 2 days on this.