Removing views at other adapter positions

Question

Problem: I'm trying to conditionally remove a linear layout component of an item in a recycler view inside the onBindViewHolder section of the adapter but I can't figure out how to access the previous view to do that. How do I go about that? The following works for the most recently added item, but how do I do this for previously added items? This is taking place inside the adapter so I don't have direct access to the recyclerView or layoutmanager.

if (holder.mContainer.getParent() != null) {
       ((ViewGroup) holder.mEmotionLayout.getParent()).removeView(holder.mEmotionLayout);
}
holder.mMainLayout.addView(holder.mEmotionLayout, 1); 

Please let me know if you need me to add anything to the post. Thank you.

Update: I tried passing the linear layout manager to the adapter as shown below and then tried to remove the views in question with the subsequent code but the findViewById always returns null.

private LinearLayoutManager linearLayoutManager;

public ChatAdapter(LinearLayoutManager linearLayoutManager) {
   
    this.linearLayoutManager = linearLayoutManager;

Inside onBindViewHolder:

((ViewGroup) linearLayoutManager.getChildAt(position-1).findViewById(R.id.emotionLayout).getParent()).removeView(linearLayoutManager.getChildAt(position-1).findViewById(R.id.emotionLayout));

Solved: Had some success using array lists to hold the old viewHolders but I ran into out of bounds exceptions occasionally that crashed app after restarting it or reloading the activity.

java.lang.IndexOutOfBoundsException: Index: 1, Size: 1

//within onBindViewHolder

private ArrayList mainLinearLayouts = new ArrayList();
private ArrayList emotionLinearLayouts = new ArrayList();

mainLinearLayouts.add(holder.mMainLayout);
emotionLinearLayouts.add(holder.mEmotionLayout);

if (position > 0) {
         if (chatList.get(position - 1).getCurrentUser()) {
                    LinearLayout mll = (LinearLayout) mainLinearLayouts.get(position - 1);
                    LinearLayout ell = (LinearLayout) emotionLinearLayouts.get(position - 1);
                    mll.removeView(ell);
                }
            }