Java Zookeeper API weird ZNode behavior. Unable to delete ZNode properly. It has unexpected results

Question

I am trying to create a persistent ZNode and storing the number of lines of a particular file that I have processed. Creation works just like it should, so does reading data from the node, but deletion doesn't work if it's in the same code. I'd explain what I mean.

I have created functions:

setOrCreateFileCheckpoint(fileName: String, lineNumber: Int) :- checks if the ZNode exists, creates it if it doesn't and sets the stored value to lineNumber getFileCheckpoint(fileName: String) :- returns the value stored in the ZNode deleteFileCheckpoint(fileName: String) :- deletes the ZNode

below is the code for all three:

/*
updates or creates a checkpoint for a file being processed
 */
def setOrCreateFileCheckpoint(fileName: String, lineNumber: Int): Unit =
    {
        val fileCheckpointPath = checkpointPoolPath + "/" +fileName
        val zk = getZookeeper
        val zkCuratorClient = getZookeeperCuratorClient

        if ( zk.exists(fileCheckpointPath, false) == null)
            {
                val node = new PersistentNode(zkCuratorClient, CreateMode.PERSISTENT, false, fileCheckpointPath, lineNumber.toString.getBytes())
                node.start()
            }
        else
            zk.setData(fileCheckpointPath, lineNumber.toString.getBytes(), -1)
    }

/*
gets checkpoint for a file
 */
def getFileCheckpoint(fileName: String): Int =
    {
        val fileCheckpointPath = checkpointPoolPath + "/" +fileName
        val zk = getZookeeper
        val zkCuratorClient = getZookeeperCuratorClient

        if ( zk.exists(fileCheckpointPath, false) != null)
            new String(zk.getData(fileCheckpointPath, false, null)).toInt

        else
            0

    }

/*
deletes the file checkpoint so that we don't keep accumulating zNodes on the zookeeper
 */
def deleteFileCheckpoint(fileName: String): Unit =
    {
        val fileCheckpointPath = checkpointPoolPath + "/" +fileName

        val zk = getZookeeper

        if ( zk.exists(fileCheckpointPath, false) == null)
        {
            throw RuntimeException("Trying to delete checkpoint that doesn't exist for file: " + fileName)
        }
        else
            {
                /*println(zk.exists(fileCheckpointPath, false).getVersion)
                zk.delete(fileCheckpointPath, zk.exists(fileCheckpointPath, false).getVersion)*/
                deleteChildren(zk, fileCheckpointPath, true)

            }
    }

Following is the code I am testing and am perplexed by:

        ZookeeperUtility.setOrCreateFileCheckpoint("file1", 2000) //let's call it cre1

        println(ZookeeperUtility.getFileCheckpoint("file1")) //let's call it get1

        ZookeeperUtility.deleteFileCheckpoint("file1") //let's call it del1
        println("del1")

        ZookeeperUtility.deleteFileCheckpoint("file1") //let's call in del2
        println("del2")

Run 1:

Step1: I Run the code shown above

Result: Error encountered on the del2

Step2: Comment out cre1 and run the code again

Result: Node is fetched, gives the correct value as result error encountered on del2. This is mind boggling. I can't understand why. The node is supposed to be deleted.

Step3: with cre1 still commented, same as previous step, run the code again

Result: Node doesn't exist gives 0 at get1 which means node doesn't exist. error is encountered at del1. Which is what should've happened in step2 itself

Run2:

Step1: Comment out del2, run the code

Result: creates node, fetches correct data, exits normally

Step2: Comment out cre1, run the code

Result: Fetches the value 2000 from a node that was supposed to be deleted. exits normally

Step3: Run the same code as step2 again

Result: fetches 0, error encountered on del1.

If I run the code one step at a time, if I only create in one run, only fetch in the next run and only delete in the run after that, everything works just like it should. I am on the brink of pulling my hair out.

P.S. The code is written in Scala but I am using the Java API. Scala can seemlessly work with Java classes.

If you look at the deleteFileCheckpoint function I have commented out a part, I have tried that approach as well. It has the exact same behvaior.

score 1 · Accepted Answer · answered Apr 23 '20 at 03:47

1

This is mind boggling. I can't understand why. The node is supposed to be deleted.

I'm not sure why you're surprised. You are creating a PersistentNode which exists to automatically recreate the node should it get deleted. In fact, all the surrounding code is very puzzling. It's duplicating what PersistentNode does internally. You don't need to do all that other stuff. Just use PersistentNode.

Further, code like checkExists() followed by an action based on the result will almost never work in production. ZooKeeper is highly concurrent and eventually consistent. This is why you should always use Curator's recipes instead of hand-coding solutions.

answered Apr 23 '20 at 03:47

Randgalt

2,907
1
17
31

What should I do instead? I am trying to checkpoint how many lines of a file I have already processed. So another instance can pick up where this instance left off if case of a code failure. Any tips for that? – Vipul Rajan Apr 23 '20 at 04:08
Should I not be able to delete a persistent node if I want to? If I'm not supposed to delete it why is there and API for deletion though. If it's there it should work like it's supposed to, right? – Vipul Rajan Apr 23 '20 at 04:19
According to this answer I should be able to delete persistent nodes on request, it'd be a huge help if you could shed some light on it: https://stackoverflow.com/questions/26773316/expiration-of-zookeeper-persistent-node. – Vipul Rajan Apr 23 '20 at 04:24
It seems you've confused ZooKeeper "persistent nodes" with Curator's `PersistentNode` class. They're different things. Curator's `PersistentNode` is a high level recipe that will completely manage a ZNode for you - recreating it if necessary when deleted. – Randgalt Apr 23 '20 at 13:25
Curator's `PersistentNode` has a method to change the value/data of the managed node: `setData()`. To delete the node you would `close()` the `PersistentNode` instance, etc. – Randgalt Apr 23 '20 at 13:26
Oh. Thanks a huge ton sir. You're a savior. That was exactly it. I have not worked with zookeeper before :-) – Vipul Rajan Apr 23 '20 at 13:57
Would you know anything about this, my testing server keeps failing : https://stackoverflow.com/questions/61389209/apache-curator-testingserver-creation-throws-java-lang-nosuchmethoderror – Vipul Rajan Apr 23 '20 at 17:45
That's a problem with dependencies and Maven - nothing to do with Curator or ZooKeeper. Unfortunately, you really need to learn your tools better. – Randgalt Apr 24 '20 at 18:06
I think I do. I'd keep at it. Thanks for your help anyway – Vipul Rajan Apr 25 '20 at 18:08
"Further, code like checkExists() followed by an action based on the result will almost never work in production" Any tips on what I should do instead of it? If I just want to check if a node exists and decide my actions based on that? – Vipul Rajan Apr 25 '20 at 18:26
It's difficult to do in ZooKeeper. That's why I say "friends don't let friends write ZooKeeper recipes". Use Curator's recipes. Use one of the cache recipes which have listeners, etc. – Randgalt Apr 25 '20 at 23:45

Java Zookeeper API weird ZNode behavior. Unable to delete ZNode properly. It has unexpected results

1 Answers1