0

I have created a workspace in AWS Databricks with private link. When we launch a cluster we get the following error:

Security Daemon Registration Exception: Failed to set up the spark container due to an error when registering the container to security daemon.

The log (in the EC2 instance) stays in a loop at:

[ 348.935443] audit: kauditd hold queue overflow
[ 348.940183] audit: kauditd hold queue overflow
[ 348.946439] audit: kauditd hold queue overflow
[ 354.013288] audit: kauditd hold queue overflow
[ 354.019590] audit: kauditd hold queue overflow
[ 354.025712] audit: kauditd hold queue overflow
[ 475.998259] audit: kauditd hold queue overflow
[ 476.004062] audit: kauditd hold queue overflow
[ 476.009494] audit: kauditd hold queue overflow
[ 638.997129] audit: kauditd hold queue overflow
[ 639.005763] audit: kauditd hold queue overflow
[ 639.012236] audit: kauditd hold queue overflow

Databricks error

Tsyvarev
  • 60,011
  • 17
  • 110
  • 153
jonro
  • 1

1 Answers1

0

This issue typically isn't because of the kauditd overflow - it's typically because the sts private link is associated with the wrong subnet.

Best link I've been able to find so far: https://community.databricks.com/s/question/0D58Y00009J6n1uSAB/error-cluster-launch-security-daemon-registration

If it's a terraform deployment remove the sts vpc-endpoint

If you copy/paste terraform solution from databricks official website and get this error then the sts vpc endpoint is your issue.

The error itself is not listed anywhere else besides there, I do not have the information as to why this is the issue.

smoot
  • 21
  • 5
  • As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Mar 29 '23 at 00:11