Android drawerLayout close using touch listener doesnt close all the way

Question

I am trying to close the navigation layout when user touches outside the navigation. I added a touchListener to the layout and call drawerLayout.closeDrawer(drawer) in the touch listener. However, this doesnt close the drawer all the way. If I keep clicking down the screen outside the drawer, it continues closing. So the touch event is not long enough to close the drawer all the way. How can I accomplish closing the drawer when clicking outside?

drawerLayout.setOnTouchListener(new OnTouchListener(){
        @Override
        public boolean onTouch(View v, MotionEvent event) {
            // TODO Auto-generated method stub              
            if(drawerLayout.isDrawerOpen(drawer)){
                drawerLayout.closeDrawer(drawer);
            }
            return false;
        }           
    }); 

Snippet of XML:

<android.support.v4.widget.DrawerLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools"
android:id="@+id/drawer_layout"
android:clickable="true"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context=".MainActivity" >
.
.
.
<LinearLayout
    android:id="@+id/drawer"
    android:layout_width="300dp"
    android:layout_height="match_parent"
    android:layout_gravity="start"
    android:background="#80000000"
    android:choiceMode="singleChoice"
    android:orientation="vertical" >