Edit: Found an ugly bandaid fix for now, but still need help solving the bug. Details at the end of the question.
I'm currently developing a db/site between neo4j and flask, using py2neo. I'm in the middle of developing a notification system, where upon X action, users get notified that X took place. To do this I'm creating a new "Update" node in the DB with merge.
The thing though is if a similar action, Y, takes place in the same way, I want to instead of creating a new update node, update the old one. Normally merge would work just fine for this, because it should detect if a node w/ these properties already exists before making a new one. The problem I'm having is that the properties of this node change, for example the "date_time" property will be different each time. Because of this, it's creating a second update node for the same type of action, rather than simply updating the date_time.
To avoid this, I do the following:
update = Node('Update', updateUUID = other_node['uuid'])
graph.merge(update)
update['date_time'] = new_date_time
graph.push(update) ## Same error if I use update.push()
rel = Relationship(other_node, 'has_update', update)
graph.create(rel)
This was a method suggested by others to update existing nodes. Specifically, calling merge with only the unique property of the node, then pushing other data to it separately with the push call.
This works perfectly if the node in question already existed ahead of time. In fact, if I try to do this it fails only on the first try, as the node gets partially created before failing. If I repeat the action X a second time, the update node gets changed successfully, overwriting the first incomplete node with the new properties.
Finally, this is the actual error I'm getting. As noted, the results of running the above will create the node initially with the merge successfully, but fail to perform the updates, leaving me with a nearly blank node in the DB.
2017-02-15 00:40:25.636+0000 WARN /db/data/batch Transaction was marked as successful, but unable to commit transaction so rolled back.
org.neo4j.graphdb.TransactionFailureException: Transaction was marked as successful, but unable to commit transaction so rolled back.
at org.neo4j.kernel.impl.coreapi.TopLevelTransaction.close(TopLevelTransaction.java:101)
at org.neo4j.server.rest.transactional.CommitOnSuccessfulStatusCodeRepresentationWriteHandler.closeTransaction(CommitOnSuccessfulStatusCodeRepresentationWriteHandler.java:65)
at org.neo4j.server.rest.transactional.CommitOnSuccessfulStatusCodeRepresentationWriteHandler.onRepresentationFinal(CommitOnSuccessfulStatusCodeRepresentationWriteHandler.java:60)
at org.neo4j.server.rest.web.BatchOperationService$1.write(BatchOperationService.java:137)
at com.sun.jersey.core.impl.provider.entity.StreamingOutputProvider.writeTo(StreamingOutputProvider.java:71)
at com.sun.jersey.core.impl.provider.entity.StreamingOutputProvider.writeTo(StreamingOutputProvider.java:57)
at com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:302)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1510)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:808)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669)
at org.neo4j.server.rest.dbms.AuthorizationEnabledFilter.doFilter(AuthorizationEnabledFilter.java:122)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.neo4j.server.rest.web.CollectUserAgentFilter.doFilter(CollectUserAgentFilter.java:69)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:497)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.neo4j.kernel.api.exceptions.TransactionFailureException:
at org.neo4j.kernel.impl.api.KernelTransactionImplementation.commit(KernelTransactionImplementation.java:553)
at org.neo4j.kernel.impl.api.KernelTransactionImplementation.closeTransaction(KernelTransactionImplementation.java:484)
at org.neo4j.kernel.api.KernelTransaction.close(KernelTransaction.java:135)
at org.neo4j.kernel.impl.coreapi.TopLevelTransaction.close(TopLevelTransaction.java:79)
... 35 more
Caused by: org.neo4j.kernel.api.exceptions.TransactionHookException: Transaction handler failed.
at org.neo4j.kernel.impl.api.TransactionHooks$TransactionHooksState.add(TransactionHooks.java:102)
at org.neo4j.kernel.impl.api.TransactionHooks.beforeCommit(TransactionHooks.java:61)
at org.neo4j.kernel.impl.api.KernelTransactionImplementation.commit(KernelTransactionImplementation.java:549)
... 38 more
Caused by: com.graphaware.runtime.module.DeliberateTransactionRollbackException: You are not allowed to remove the uuid property
at com.graphaware.module.uuid.UuidModule.beforeCommit(UuidModule.java:143)
at com.graphaware.module.uuid.UuidModule.beforeCommit(UuidModule.java:40)
at com.graphaware.runtime.manager.BaseTxDrivenModuleManager.beforeCommit(BaseTxDrivenModuleManager.java:193)
at com.graphaware.runtime.TxDrivenRuntime.beforeCommit(TxDrivenRuntime.java:76)
at com.graphaware.runtime.TxDrivenRuntime.beforeCommit(TxDrivenRuntime.java:39)
at org.neo4j.kernel.internal.TransactionEventHandlers.beforeCommit(TransactionEventHandlers.java:128)
at org.neo4j.kernel.internal.TransactionEventHandlers.beforeCommit(TransactionEventHandlers.java:50)
... 40 more
After initial research, I had been lead to believe that this is due to a lack of heap space, which made sense because I was on a pretty weak system at the time. However, after moving to a new system (and installing the database nearly from scratch again, with a new config file, which was made sure not to have any heap limiting settings used), I'm still having the same issue.
I'm at a bit of a loss here, so if anyone has suggestions on how to either fix this, or perform what I'm trying to do in a different way to avoid this issue, it would be greatly appreciated.
Update: As described, the error only gets thrown on nodes I'm creating for the first time. So, as a temporary work until a real solution is found, I simply emulate the behavior necessary to make it behave like an "existing" node, even if it's brand new. Aka..
graph.merge(update_node)
graph.merge(update_node)
Adding a second merge call avoids the bug for now... even if it's kind of dirty.
Update 2:
As requested, here is my graphaware settings
com.graphaware.runtime.enabled=true
com.graphaware.module.UIDM.1=com.graphaware.module.uuid.UuidBootstrapper
com.graphaware.module.UIDM.uuidProperty=uuid
com.graphaware.module.UIDM.stripHyphens=false
com.graphaware.module.UIDM.uuidIndex=uuidIndex
com.graphaware.module.UIDM.uuidRelationshipIndex=uuidRelIndex
Update 3: Continuing debug, here's the result of replacing updateUUID with simply 'uuid', a property graphaware normally creates and assigns on its own.
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/Flask-0.11.1-py2.7.egg/flask/app.py", line 2000, in __call__
return self.wsgi_app(environ, start_response)
File "/usr/local/lib/python2.7/dist-packages/Flask-0.11.1-py2.7.egg/flask/app.py", line 1991, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/usr/local/lib/python2.7/dist-packages/Flask-0.11.1-py2.7.egg/flask/app.py", line 1567, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/Flask-0.11.1-py2.7.egg/flask/app.py", line 1988, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/Flask-0.11.1-py2.7.egg/flask/app.py", line 1641, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python2.7/dist-packages/Flask-0.11.1-py2.7.egg/flask/app.py", line 1544, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/Flask-0.11.1-py2.7.egg/flask/app.py", line 1639, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/Flask-0.11.1-py2.7.egg/flask/app.py", line 1625, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File ".../views.py", line 608, in updateMem
answer = mem.update(results)
File ".../models.py", line 363, in update
graph.merge(update)
File "/usr/local/lib/python2.7/dist-packages/py2neo/database/__init__.py", line 639, in merge
self.begin(autocommit=True).merge(subgraph, label, *property_keys)
File "/usr/local/lib/python2.7/dist-packages/py2neo/database/__init__.py", line 1169, in merge
subgraph.__db_merge__(self, primary_label, primary_key)
File "/usr/local/lib/python2.7/dist-packages/py2neo/types.py", line 443, in __db_merge__
tx.run(statement, parameters)
File "/usr/local/lib/python2.7/dist-packages/py2neo/database/__init__.py", line 1277, in run
self.finish()
File "/usr/local/lib/python2.7/dist-packages/py2neo/database/__init__.py", line 1296, in finish
self._sync()
File "/usr/local/lib/python2.7/dist-packages/py2neo/database/__init__.py", line 1286, in _sync
connection.fetch()
File "/usr/local/lib/python2.7/dist-packages/py2neo/packages/neo4j/v1/bolt.py", line 344, in fetch
handler(*fields)
File "/usr/local/lib/python2.7/dist-packages/py2neo/database/__init__.py", line 961, in on_failure
raise GraphError.hydrate(metadata)
ClientError: [Neo.ClientError.Transaction.TransactionHookFailed]
