3

Recently my Undertow application is triggering Cloud Run to report the following:

Container Sandbox Limitation: Unsupported syscall setsockopt(0x13,0x1,0xa,0x3e05747fe5a0,0x4,0xfc1abc10). Please, refer to https://gvisor.dev/c/linux/amd64/setsockopt for more information.

I've done an strace and it seems that the socket option to enable Out-of-band Inlining (SO_OOBINLINE) is being sent by Undertow. I've explicitly told it not to do that in the configuration (two ways) but it's still happening. Using Undertow with Cloud Run seems like a reasonable use case but without more insight into what Out-of-band Inlining is for Undertow and why gVisor doesn't support this, I'm blocked on which program is being unreasonable. Is it Undertow for doing something that other webservers are not or is it gVisor being too immature to handle this particular socket function? Maybe gVisor will support it in the future and I just need to wait?

  def main(args: Array[String]): Unit = {
    val server = Undertow.builder
      .addHttpListener("8080", "0.0.0.0")
      .setHandler(defaultHandler)
      .setSocketOption[java.lang.Boolean](XnioOptions.TCP_OOB_INLINE, false)
      .setWorkerOption[java.lang.Boolean](XnioOptions.TCP_OOB_INLINE, false)
      .build
    server.start()
  }
mgenereu
  • 33
  • 3
  • Were you able to resolve this issue? Which version of Undertow were you using? I am also seeing the same error in Cloud Run while using `2.0.28.Final`. However, I don't see anywhere in`Undertow.java` which it sets TCP_OOB_INLINE. – user482594 Dec 02 '19 at 11:56

1 Answers1

2

My answer covers services "listening" within a container in Cloud Run Managed. My answer does not include Anthos or custom applications where your container is connecting outside of Cloud Run via TCP, gRPC or WebSockets. In your question, your example is an HTTP server which is a listener and not a client.

This is not a gVisor problem. The first step is to understand what SO_OOBINLINE does.

If this option is enabled, out-of-band data is included in the receive data stream. Otherwise, you must use the flag MSG_OOB during a recv() call to get out-of-band data.

Now, who is going to send you out-of-band (msg) data? The front end to Google Cloud Run Managed is the Google Frontend (GFE). This is Google's proxy and load balancer for Cloud Run Managed (and many other Google services). The interface to the GFE is HTTP/HTTPS. The client/browser cannot generate out-of-band data. The client connects to the GFE. The GFE connects to your service running in Cloud Run.

If gVisor were to support SO_OOBINLINE, who can send out-of-band data? Nobody/nothing in the TCP/IP chain that you can control/manage.

There is an Internet Draft Out-Of-Band' Content Coding for HTTP. I have only been following this document. There might be support for this in the future for HTTP but not today.

In your question, you are setting SO_OOBINLINE to false. This is the default case (false), so setting this to false is not required.

Note: There are a few good reasons to use OOB, but they are rare. OOB is only one byte of data. In the HTTP world, if something goes wrong, the standard expectation is a status code to indicate a problem or retry situation.

How to stop Undertow triggering warnings from gVisor in Cloud Run

Don't call the API setSocketOption() and equivelent. There is no method to disable gVisor warnings.

John Hanley
  • 74,467
  • 6
  • 95
  • 159
  • I think your last line hit it on the head. I think Undertow is setting it to false regardless of its default and that's a unnecessary call to `setsockopt`. – mgenereu Oct 17 '19 at 18:19
  • @mgenereu subclass and handle setSocketOption yourself and make it a noop. – John Hanley Oct 17 '19 at 18:35
  • It's in a few places in the code and the threaded sequence of events combined with the HTTP 100 Continue code that actually does use it lead me to just open a ticket. I'm going to see if there's an option to filter syslog at some point in the process before StackDriver gets a hold of it. – mgenereu Oct 18 '19 at 15:21