This is because the onTouchListener of ScrollView is Overridden by the onTouchListener of Swipe.
Try to use this lib:
https://github.com/daimajia/AndroidSwipeLayout
Is very simple, download the file: AndroidSwipeLayout-v1.1.8.jar, paste in your libs folder and use like this in your xml item of RecyclerView:
<com.daimajia.swipe.SwipeLayout xmlns:android="http://schemas.android.com/apk/res/android"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_gravity="center_vertical">
<LinearLayout
android:id="@+id/bottom_wrapper"
android:layout_width="40dp"
android:layout_height="match_parent"
android:background="@color/primary_color"
android:tag="Bottom1"
android:weightSum="1">
<ImageView
android:id="@+id/delete"
android:layout_width="30dp"
android:layout_height="30dp"
android:layout_gravity="center_vertical"
android:layout_marginLeft="5dp"
android:layout_marginRight="5dp"
android:background="@android:color/transparent"
android:src="@android:drawable/ic_delete"
android:tint="@android:color/white" />
</LinearLayout>
<RelativeLayout
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:gravity="center_vertical"
android:paddingBottom="10dp"
android:paddingTop="10dp">
<TextView
android:id="@+id/exercise_item"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_marginLeft="20dp"
android:clickable="false"
android:text="ExampleText"
android:textAppearance="?android:attr/textAppearanceMedium"
android:textColor="@color/primary_color_dark" />
</RelativeLayout>
</com.daimajia.swipe.SwipeLayout>
You have to use 2 Layouts, the first is the hidden before the swipe.