How can I get an accurate height value from the gradient noise in shader graph based on an object's world x and z position values in unity?

Question

So the solution I currently have actually works for a couple of seconds, then it goes out of sync. Here is how the shader graph looks:

Here is how the code looks (the code is attached to the object). I assign a global time float to the shader each frame to ensure that the time on shader and in the code doesn't go out of sync.

Here is how the gradient noise code looks like. I don't really understand this part (I got it from google), but it SHOULD be at least a bit accurate since the object does sync up with the noise for a couple of seconds before slowly going out of sync.

Here is how it looks in game. You can see how towards the end it starts to get out of sync:

https://i.stack.imgur.com/PFOUd.jpg

Does anyone see how I could manage to fix this? Or if anyone has a better way to go about doing this, I'd really appreciate it :)

I haven't really tried anything else to be honest, I've looked all around google and I didn't find a solution for this, only people asking questions about the same problem and not getting responses.