@David Wasser is correct ... cell re-use is causing multiple listview rows to draw with gray background.
However, if you're trying to set your background based on SELECTION STATE, consider this technique:
// set single or multi-select on your list (CHOICE_MODE_SINGLE = single row selectable)
// do this in onCreate
listView.setChoiceMode(ListView.CHOICE_MODE_SINGLE);
.
.
.
// in your onItemClick, set the checked state of the item
// you DO NOT need to call notifyDataSetChanged
listView.setItemChecked(position, true);
And, set the background on your listview cell layout to the built-in or a custom selector
BUILT-IN:
android:background="?android:attr/activatedBackgroundIndicator"
CUSTOM:
android:background="@drawable/myListBackground"
drawable/myListBackground.xml:
<?xml version="1.0" encoding="utf-8"?>
<selector xmlns:android="http://schemas.android.com/apk/res/android">
<item android:state_activated="true" android:drawable="@color/lightgray" />
<item android:drawable="@color/transparent" />
</selector>
the key is the state_activated entry, which will be used when the item is selected/checked. You can also specify colors for other states, the above example references colors from a colors.xml table.
For more details on this, please check out How does "?android:attr/activatedBackgroundIndicator" work?