2

I would like to know if there is a way for checking if a .ksql script is syntactically correct?

I know that you can send a POST request to the server, this however would also execute the containing ksql commands. I would love to have some kind of endpoint where you can pass your statement and it returns you either an error code or an OK like:

curl -XPOST <ksqldb-server>/ksql/validate -d '{"ksql": "<ksql-statement>"}' .

My question aims for a way to check the synatx in an automated fashion without the need to cleaning up everything afterwards.

Thanks for you help!

Note: I am also well aware that I could run everything separately using, e.g., a docker-compose file and tear everything down again. This however is quite resource heavy and and harder to maintain.

XH_P
  • 31
  • 1
  • 8

2 Answers2

1

one option could be to use the ksql test runner (see here: https://docs.ksqldb.io/en/latest/how-to-guides/test-an-app/) and look at the errors to check if the statement is valid. Let me know if it works for your scenario.

Jean-Sébastien
  • 737
  • 3
  • 7
  • One problem I have with the ksql-test-runner is that I cannot make assumptions on how the `.ksql` will look like, so defining an input/output file is not really possible. The most concrete assumptions I can have is how the topics are called. Although I like this test-runner and will try to make use of it! Thanks – XH_P Dec 20 '21 at 14:25
1

By now I've found a way to test for my use case. I had a ksqldb cluster already in place with all other systems needed for the Kafka ecosystem (Zookeeper, Broker,...). I had to compromise a little but and go through the effort of deploying everything but here is my approach:

  1. Use proper naming (let it be prefixed with test or whatever suits your use case) for your streams, tables,... the queries' sink property should include the prefixed topic in order to find it easily, or you simply assign an QUERY_ID (https://github.com/confluentinc/ksql/pull/6553).
  2. Deploy the streams, tables,... to your ksqldb server using its rest API. Since I was programming in Python, I made use of the ksql package using pip (https://github.com/bryanyang0528/ksql-python).
  3. Cleanup the ksqldb server by filtering for the naming that you assigned to the ksql resources and run the corresponding DROP or TERMINATE statement. Consider also, you will have dependencies that result in multiple streams/tables reusing a topics. The statements can be looked up in the official developer guide (https://docs.ksqldb.io/en/latest/developer-guide/ksqldb-reference/quick-reference/).
  4. If you had errors in step 2, step 3 should have cleaned up the leftovers so that you can adjust your ksql scripts until they run through smoothly.

Note: I could not make any assumptions on how the streams,... look like. If you can, I would prefer the ksql-test-runner.

XH_P
  • 31
  • 1
  • 8