I am trying to insert/update a graph db (~ 500K nodes & relationships) using py2neo. Somewhere in this process, the transaction failed & neo4j crashed. As per the log file - the root cause was reported as
"Caused by: java.io.IOException: File too large"
Restarting the neo4j server did not work.
"Failed to start within 120 seconds"
console.log >> 2015-01-04 03:29:40.061+0000 INFO [API] Setting startup timeout to: 120000ms based on -1 Detected incorrectly shut down database, performing recovery..
Tried shutting down the server - to perform a clean exit. Got the following error.
"Caused by: java.io.IOException: File too large"
./neo4j-shell -path ../data/graph.db -c quit -v
ERROR (-v for expanded information): Error starting org.neo4j.kernel.EmbeddedGraphDatabase,/media/shard02/neo4j-2.1.6/bin/../data/graph.db java.lang.RuntimeException: Error starting org.neo4j.kernel.EmbeddedGraphDatabase, /media/shard02/neo4j-2.1.6/bin/../data/graph.db at org.neo4j.kernel.InternalAbstractGraphDatabase.run(InternalAbstractGraphDatabase.java:366) at org.neo4j.kernel.EmbeddedGraphDatabase.(EmbeddedGraphDatabase.java:59) at org.neo4j.graphdb.factory.GraphDatabaseFactory$1.newDatabase(GraphDatabaseFactory.java:91) at org.neo4j.graphdb.factory.GraphDatabaseBuilder.newGraphDatabase(GraphDatabaseBuilder.java:181) at org.neo4j.shell.kernel.GraphDatabaseShellServer.instantiateGraphDb(GraphDatabaseShellServer.java:167) at org.neo4j.shell.kernel.GraphDatabaseShellServer.(GraphDatabaseShellServer.java:63) at org.neo4j.shell.StartClient.tryStartLocalServerAndClient(StartClient.java:239) at org.neo4j.shell.StartClient.startLocal(StartClient.java:226) at org.neo4j.shell.StartClient.start(StartClient.java:158) at org.neo4j.shell.StartClient.main(StartClient.java:120) Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.impl.transaction.XaDataSourceManager@66c9c89f' was successfully initialized, but failed to start. Please see attached cause exception. at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:513) at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:115) at org.neo4j.kernel.InternalAbstractGraphDatabase.run(InternalAbstractGraphDatabase.java:343) ... 9 more Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.impl.nioneo.xa.NeoStoreXaDataSource@1bf0eebd' was successfully initialized, but failed to start. Please see attached cause exception. at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:513) at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:115) at org.neo4j.kernel.impl.transaction.XaDataSourceManager.start(XaDataSourceManager.java:164) at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:507) ... 11 more Caused by: org.neo4j.kernel.impl.nioneo.store.UnderlyingStorageException: Unable to write record[1248649] @[42454066] at org.neo4j.kernel.impl.nioneo.store.PersistenceRow.writeContents(PersistenceRow.java:198) at org.neo4j.kernel.impl.nioneo.store.PersistenceRow.force(PersistenceRow.java:214) at org.neo4j.kernel.impl.nioneo.store.LockableWindow.writeOutAndClose(LockableWindow.java:60) at org.neo4j.kernel.impl.nioneo.store.LockableWindow.writeOutAndCloseIfFree(LockableWindow.java:162) at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.release(PersistenceWindowPool.java:241) at org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.releaseWindow(CommonAbstractStore.java:557) at org.neo4j.kernel.impl.nioneo.store.RelationshipStore.updateRecord(RelationshipStore.java:169) at org.neo4j.kernel.impl.nioneo.xa.command.NeoXaCommandExecutor$NeoCommandExecutor.visitRelationshipCommand(NeoXaCommandExecutor.java:99) at org.neo4j.kernel.impl.nioneo.xa.command.Command$RelationshipCommand.accept(Command.java:202) at org.neo4j.kernel.impl.nioneo.xa.command.NeoXaCommandExecutor.execute(NeoXaCommandExecutor.java:55) at org.neo4j.kernel.impl.nioneo.xa.NeoStoreTransaction.executeModified(NeoStoreTransaction.java:916) at org.neo4j.kernel.impl.nioneo.xa.NeoStoreTransaction.applyCommit(NeoStoreTransaction.java:683) at org.neo4j.kernel.impl.nioneo.xa.NeoStoreTransaction.doCommit(NeoStoreTransaction.java:604) at org.neo4j.kernel.impl.transaction.xaframework.XaTransaction.commit(XaTransaction.java:327) at org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.injectOnePhaseCommit(XaResourceManager.java:460) at org.neo4j.kernel.impl.transaction.xaframework.XaLogicalLog$LogApplier.onePhaseCommitEntry(XaLogicalLog.java:1580) at org.neo4j.kernel.impl.nioneo.xa.command.LogHandler$Filter.onePhaseCommitEntry(LogHandler.java:74) at org.neo4j.kernel.impl.transaction.xaframework.EntryCountingLogHandler.onePhaseCommitEntry(EntryCountingLogHandler.java:59) at org.neo4j.kernel.impl.transaction.xaframework.XaLogicalLog$RecoveryConsumer.accept(XaLogicalLog.java:1798) at org.neo4j.kernel.impl.transaction.xaframework.XaLogicalLog$RecoveryConsumer.accept(XaLogicalLog.java:1774) at org.neo4j.kernel.impl.nioneo.xa.RecoveryLogDeserializer$RecoveryCursor.next(RecoveryLogDeserializer.java:72) at org.neo4j.kernel.impl.transaction.xaframework.XaLogicalLog.doInternalRecovery(XaLogicalLog.java:722) at org.neo4j.kernel.impl.transaction.xaframework.XaLogicalLog.open(XaLogicalLog.java:284) at org.neo4j.kernel.impl.transaction.xaframework.XaLogicalLog.open(XaLogicalLog.java:239) at org.neo4j.kernel.impl.transaction.xaframework.XaContainer.openLogicalLog(XaContainer.java:64) at org.neo4j.kernel.impl.nioneo.xa.NeoStoreXaDataSource.start(NeoStoreXaDataSource.java:426) at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:507) ... 14 more Caused by: java.io.IOException: File too large at sun.nio.ch.FileDispatcherImpl.pwrite0(Native Method) at sun.nio.ch.FileDispatcherImpl.pwrite(FileDispatcherImpl.java:66) at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:89) at sun.nio.ch.IOUtil.write(IOUtil.java:65) at sun.nio.ch.FileChannelImpl.writeInternal(FileChannelImpl.java:739) at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:724) at org.neo4j.kernel.impl.nioneo.store.StoreFileChannel.write(StoreFileChannel.java:51) at org.neo4j.kernel.impl.nioneo.store.PersistenceRow.writeContents(PersistenceRow.java:186) ... 40 more
- Increasing the heapsize (to 2GB on the 8GB CentOS 6.0 box) did not help. Also tried bumping up values in neo4j-wrapper.conf. Server fails to start..
I am pretty much stuck, and would really appreciate if any pointers can be suggested.
