My app suffers from this and it turns out so does the Wiktionary sample from http://developer.android.com/resources/samples/Wiktionary/index.html. Install the app, create a widget, uninstall the app - observe text as stated where widget used to be. Tried on Android 2.3.4.
I understand Widgets can't be rendered unless all their resources are available to them - are they trying to be drawn after the uninstall, when those resources clearly are gone?
Whilst struggling with this I saw the AppWidgetProvider.onDeleted() explanation says it's called when the ACTION_APPWIDGET_DELETED broadcast is sent. On my app this is true when I manually drag a widget to the trashcan. Is it also supposed to fire when you leave the widget on the home screen but uninstall the app?