-
Notifications
You must be signed in to change notification settings - Fork 34
Description
We get a really weird behavior with store-utils (tried in release 3.5.20 and release 3.5.6). It removes all relationships --millions of them-- and reconnects all to a single node, that is the one with id 0. It basically destroys the entire graph database. This is a Neo4j 3.5.5 database.
@jexp any idea why this might be happening? Thanks!
Running store utils throws error "Maximum id limit for RELATIONSHIP_TYPE_TOKEN has been reached. Generated id 65536 is out of permitted range [0, 65535]." Then, when I set dbms.record_format=high_limit
in neo4j.config, store-util runs ok. However, the store util processs reconnects all relationship to a single node (node id 0).
What should be the right approach to deal with this problem?
./copy-store.sh enterprise /Users/arikan/Sites/store-util-3.5.6/graph.db /Users/arikan/Sites/graph-fixed.db '' '' '' '' true
To use your existing Neo4j 3.3.x installation set NEO4J_HOME to your Neo4j directory. Currently set to: /Users/arikan/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-4a52d26d-7e7c-4c60-bbd5-b90ea4b87722/installation-3.5.5
Usage: copy-store.sh [community|enterprise] source.db target.db [RELS,TO,SKIP] [props,to,skip] [Labels,To,Skip] [Labels,To,Delete,Nodes]
Using: Heap 4G Pagecache 2G Edition 'enterprise' from '/Users/arikan/Sites/store-util-3.5.6/graph.db' to '/Users/arikan/Sites/graph-fixed.db' skipping labels: '', removing nodes with labels: '' rels: '' props '' Keeping Node Ids: true
Please note that you will need this memory (2G + 1G + 4G) as it opens 2 databases one for reading and one for writing.
Copying from /Users/arikan/Sites/store-util-3.5.6/graph.db to /Users/arikan/Sites/graph-fixed.db ingoring rel-types [true] ignoring properties [] ignoring labels [] removing nodes with labels [] keep node ids true
Exception in thread "main" java.lang.RuntimeException: Error starting org.neo4j.graphdb.facade.GraphDatabaseFacadeFactory, /Users/arikan/Sites/store-util-3.5.6
at org.neo4j.graphdb.facade.GraphDatabaseFacadeFactory.initFacade(GraphDatabaseFacadeFactory.java:229)
at org.neo4j.graphdb.facade.GraphDatabaseFacadeFactory.newFacade(GraphDatabaseFacadeFactory.java:146)
at org.neo4j.graphdb.factory.GraphDatabaseFactory.newDatabase(GraphDatabaseFactory.java:133)
at org.neo4j.graphdb.factory.GraphDatabaseFactory.newEmbeddedDatabase(GraphDatabaseFactory.java:122)
at org.neo4j.graphdb.factory.GraphDatabaseFactory$EmbeddedDatabaseCreator.newDatabase(GraphDatabaseFactory.java:178)
at org.neo4j.graphdb.factory.GraphDatabaseBuilder.newGraphDatabase(GraphDatabaseBuilder.java:210)
at org.neo4j.graphdb.factory.GraphDatabaseFactory.newEmbeddedDatabase(GraphDatabaseFactory.java:79)
at org.neo4j.tool.StoreCopy.getHighestNodeId(StoreCopy.java:121)
at org.neo4j.tool.StoreCopy.copyStore(StoreCopy.java:74)
at org.neo4j.tool.StoreCopy.main(StoreCopy.java:49)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.NeoStoreDataSource@369c9bb' was successfully initialized, but failed to start. Please see the attached cause exception "Maximum id limit for RELATIONSHIP_TYPE_TOKEN has been reached. Generated id 65536 is out of permitted range [0, 65535].".
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:473)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:111)
at org.neo4j.kernel.impl.transaction.state.DataSourceManager.start(DataSourceManager.java:116)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:452)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:111)
at org.neo4j.graphdb.facade.GraphDatabaseFacadeFactory.initFacade(GraphDatabaseFacadeFactory.java:224)
... 9 more
Caused by: java.lang.RuntimeException: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageEngine@60e21209' was successfully initialized, but failed to start. Please see the attached cause exception "Maximum id limit for RELATIONSHIP_TYPE_TOKEN has been reached. Generated id 65536 is out of permitted range [0, 65535].".
at org.neo4j.kernel.NeoStoreDataSource.start(NeoStoreDataSource.java:450)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:452)
... 14 more
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageEngine@60e21209' was successfully initialized, but failed to start. Please see the attached cause exception "Maximum id limit for RELATIONSHIP_TYPE_TOKEN has been reached. Generated id 65536 is out of permitted range [0, 65535].".
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:473)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:111)
at org.neo4j.kernel.NeoStoreDataSource.start(NeoStoreDataSource.java:434)
... 15 more
Caused by: org.neo4j.kernel.impl.store.id.validation.IdCapacityExceededException: Maximum id limit for RELATIONSHIP_TYPE_TOKEN has been reached. Generated id 65536 is out of permitted range [0, 65535].
at org.neo4j.kernel.impl.store.id.validation.IdValidator.assertIdWithinCapacity(IdValidator.java:90)
at org.neo4j.kernel.impl.store.id.IdGeneratorImpl.setHighId(IdGeneratorImpl.java:170)
at org.neo4j.kernel.impl.store.CommonAbstractStore.setHighId(CommonAbstractStore.java:660)
at org.neo4j.kernel.impl.store.CommonAbstractStore.rebuildIdGenerator(CommonAbstractStore.java:464)
at org.neo4j.kernel.impl.store.CommonAbstractStore.makeStoreOk(CommonAbstractStore.java:678)
at org.neo4j.kernel.impl.store.NeoStores.lambda$makeStoreOk$0(NeoStores.java:459)
at org.neo4j.kernel.impl.store.CommonAbstractStore.visitStore(CommonAbstractStore.java:973)
at org.neo4j.kernel.impl.store.NeoStores.visitStore(NeoStores.java:504)
at org.neo4j.kernel.impl.store.NeoStores.makeStoreOk(NeoStores.java:457)
at org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageEngine.start(RecordStorageEngine.java:376)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:452)
... 17 more
Suppressed: org.neo4j.kernel.lifecycle.LifecycleException: Exception during graceful attempt to stop partially started component. Please use non suppressed exception to see original component failure.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:464)
... 17 more
Caused by: java.lang.NullPointerException
at org.neo4j.kernel.impl.storageengine.impl.recordstorage.id.BufferedIdController.stop(BufferedIdController.java:59)
at org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageEngine.stop(RecordStorageEngine.java:401)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:460)
... 17 more
Here is a view from the store-copy.log
Here is how all roads lead to node id 0: