NestedScrollView not Scrolling with Viewpager, Fragment and RecylerView

Question

I have an activity,it loads up one fragment A. This fragment A will have some header content and a Tablayout and a Viewpager. The Viewpager will have a Fragment B which will have a recyler view in it. The scroll is not working after implementation.

This is my layout of Fragment A.

<?xml version="1.0" encoding="utf-8"?>
<android.support.v4.widget.NestedScrollView xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/tools"
android:id="@+id/profile_logged_in_view"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:fillViewport="true"
android:fitsSystemWindows="true"
app:layout_behavior="@string/appbar_scrolling_view_behavior"
android:orientation="vertical">

<RelativeLayout
    android:id="@+id/logged_in_container"
    android:layout_width="match_parent"
    android:layout_height="match_parent">

    <TextView
        android:id="@+id/logged_in"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:gravity="center"
        android:text="Hi "
        android:textSize="18sp"
        android:textStyle="bold" />

    <TextView
        android:id="@+id/info"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_below="@+id/logged_in"
        android:layout_centerInParent="true"
        android:gravity="center"
        android:text="You are currently logged in using"
        android:textStyle="normal" />

    <Button
        android:id="@+id/logo"
        android:layout_width="130dp"
        android:layout_height="30dp"
        android:layout_below="@+id/info"
        android:layout_centerInParent="true"
        android:layout_marginTop="15dp"
         />

    <TextView
        android:id="@+id/edit_profile_notice_text"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_below="@+id/logo"
        android:layout_centerInParent="true"
        android:gravity="center"
        android:padding="10dp"
        android:text="Edit your profile on"

        android:textStyle="normal" />

    <android.support.design.widget.TabLayout
        android:id="@+id/info_tabs"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:layout_below="@+id/edit_profile_notice_text"
        app:tabMode="fixed" />

    <android.support.v4.view.ViewPager
        android:id="@+id/info_pager"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        app:layout_behavior="@string/appbar_scrolling_view_behavior"
        android:layout_below="@+id/info_tabs" />

    <View
        android:layout_width="match_parent"
        android:layout_height="15dp"
        android:layout_below="@id/info_pager"></View>

</RelativeLayout>

The full source code is here https://github.com/msomu/NestedScrollView

If i don't include recyclerView.setNestedScrollingEnabled(false); in the child fragments its scrolling itself if I include no layout is scrolling.