I don't know of any reason why you can't use transactions with HornetQ. The only way currently to have the client send a NACK is to use a transaction and Rollback the Transaction on error. Once you reach the configured Max allowed redeliveries the client will NACK the Message.
In most implementations if your client doesn't ACK a Message and then disconnects the message would be redelivered to another client. In the case of ActiveMQ you can configure a maximum number of times that the broker would attempt this before it moved the message into a Dead Letter Queue.
As it stands right now the NMS.Stomp implementation doesn't NACK a message when an exception is thrown from the asynchronous message callback however this could be a sensible enhancement, the project does accept contributions so dive in.