I guess you are experiencing a similar issue as me:
https://stackoverflow.com/questions/9875935/android-2-3-4-sipaudiocall-listener-oncallestablished-not-firing-every-time
Even in your case I think that the Listener that you connect to incoming calls (which are passed from the call receiver to your activity) is not always firing the methods. Therefore the call is closed in your application, but not in asterisk / on the other device.
I am sorry that I can't give you a solution, I am stuck by myself with this problem...
Best Regards,
Armin
EDIT 02/04/2012:
I have now - just for curiosity - changed VoIP server using the 3CX software and I am not able to reproduce the problem anymore!!! Maybe there is something in the replies of asterisk that causes the strange behavior! I will now start some wireshark tests to understand the difference between the messages of asterisk and 3CX and keep you updated!