0

A TypeScript application sends a Uint8Array object through an HTTP POST request to a Scala Play application.

How to convert the Uint8Array object into a Scala object in the Play application?

For example, the TypeScript code sends the following object:

{stuff: new Uint8Array([0, 1, 2])}

Inside the Scala Play controller:

case class StuffData(stuff: List[Byte])

implicit val reads: Reads[StuffData] = Json.reads[StuffData]

def processStuff  = Action.async(parse.json[StuffData]) { request =>
  val stuffData = request.body
  println(stuffData)
}

This does not work... the error message from Play is:

For request 'POST /send-stuff' 
[Json validation error List((obj.key,List(JsonValidationError(List(error.expected.jsarray),WrappedArray()))))] 

UPDATE:

To be able to get on the Scala side exactly the same byte array as was on the JavaScript side (i.e. a byte for a byte, in the original order), the current code in the accepted answer should be updated to something like:

  implicit val stuffReader = new Reads[StuffData] {
    def reads(js: JsValue): JsResult[StuffData] = {
      JsSuccess(StuffData(
        (js \ "stuff").as[Map[String, Int]].toList.sortBy(_._1.toInt).map(_._2.toByte)
      ))
    }
  }

Now, the reader populates a

case class StuffData(stuff: List[Byte])

with the same order as the original JavaScript Uint8Array.

It's possible to convert all the Ints into Bytes without losing information, because we know that all the Ints are in the range [0..255].

rapt
  • 11,810
  • 35
  • 103
  • 145

1 Answers1

1

By default, Unit8Array is encoded in JSON as {"0":1,"1":2,"2":3,"3":4}, so you can decode it in Scala as a Map or write your custom reader, that can translate this object into an array type. Or you could make changes from the other side, instead of using Uint8Array you can use an array or a custom stringify function that makes expected JSON.

In my opinion, the easiest one is writing the custom reader. Custom reader example:


  implicit val stuffReader = new Reads[StuffData] {
    def reads(js: JsValue): JsResult[StuffData] = {
      JsSuccess(StuffData(
        (js \ "stuff").as[Map[String, Int]].toList.map(_._2)
      ))
    }
  }
  • If you look inside JavaScript's Unit8Array, it's more complicated than what you wrote. I don't think writing my own converter is the way to go here. – rapt Feb 11 '23 at 19:17
  • I saw the object in the web inspector. Sure it's the hardest option, I just wanted to be clear about all options that I have found out. In my opinion, the easiest one is writing the custom reader. – Stanislav Kovalenko Feb 11 '23 at 19:22
  • Regarding your original comment before the edit, note that if the JavaScript object is something like `{stuff: [0, 1, 2]}`, then the code works. – rapt Feb 11 '23 at 19:24
  • Yep, you are right. First, when I wrote my first answer I thought that this is an array. But after my little research about Unit8Array type, I think I got your problem. – Stanislav Kovalenko Feb 11 '23 at 19:25
  • There are libraries such as `Scala.js` that seem to know how to do conversions between JavaScript and Scala. Can it be used in this case? – rapt Feb 11 '23 at 19:27
  • I worked with Scala.js a lot and I don't think that there is any Scala.js feature which could help you with this problem. Scala.js is about macking javascript code from your Scala code and using this code inside your JS app. This generated js code won't help Play json to decode this object – Stanislav Kovalenko Feb 11 '23 at 19:32
  • With scalajs, there is still be a problem with decoding – Stanislav Kovalenko Feb 11 '23 at 19:33
  • I didn't realize that the data of Uint8Array is actually a map, because when inspecting the object in the browser, it shows other properties as well such as `byteLength`. The corrected code you suggested works, under some conditions. You can also do instead `case class StuffData(stuff: Map[String, Byte])`. The problem is that when the array of numbers is longer than 100, the code fails. It seems like Unit8Array structure is being changed by JavaScript when it's longer (I first thought it just changes the display, but not the structure). It adds another map level. Any idea how to overcome this? – rapt Feb 11 '23 at 21:48
  • A custom reader can handle through adding additional step in parsing like if the parsing into a map fails then try other parser. But it seems that this type is quite complex and you don’t have tools that could make readable json from it. I think the easiest way is to change JS data structure in your JS app before sending. Like stuff.toArray or smth and send a simple array – Stanislav Kovalenko Feb 11 '23 at 22:59
  • Could be easier to do it on the JS side. But I have 2 issues with it: 1. I don't have control over the JS app, I just get Uint8Array from it on the Scala/Play app. 2. I have tested what JS does with long arrays. It seems like it also does the insertion of another level of keys (not sure if it's actually keys) that look like `[0-99], [100-199]...`. Since I need to process the Uint8Array on the Scala side, I noticed that Uint8Array has 2 fields `length/byteLength` (not sure what the difference is). I tried something like `if ((js \ "stuff" \ "byteLength").as[Int] <= 100)` but it fails, any idea? – rapt Feb 12 '23 at 13:08
  • Yep, try to check Uint8Array through `JSON.stringify(obj.stuff)`. It shouldn't have a length method. – Stanislav Kovalenko Feb 12 '23 at 14:08
  • I checked the reader on Unit8Array with 184 elements and it works. Are you sure, that the problem is a length? Could you add an example of your Unit8Array that the reader couldn't handle? – Stanislav Kovalenko Feb 12 '23 at 14:19
  • Same code as you suggested. Works when JS sends object `{stuff: new Uint8Array([...Array(50).keys()])}`. Fails when you replace 50 with 200. Scala error: `[JsResultException: JsResultException(errors:List((/128,List(JsonValidationError(List(error.expected.byte),WrappedArray()))), (/129,List(JsonValidationError(List(error.expected.byte),WrappedArray()))), ... (/199,List(JsonValidationError(List(error.expected.byte),WrappedArray())))))] ` – rapt Feb 12 '23 at 16:00
  • I think I found out why it failed. I had to use `Int` instead of `Byte` in the Scala code. Now it works with every length of array. – rapt Feb 12 '23 at 16:26
  • Yep possible values of a byte from -127 to 127, it's not about a length of an array – Stanislav Kovalenko Feb 12 '23 at 16:27
  • 1
    You are right, the problem was trying to put an Int inside a Byte, which stops working for integers > 127. I'm glad it's less complicated than I first thought and that we've found an easy way to do it. Thank you for the ideas! – rapt Feb 12 '23 at 18:44
  • Trying to work with the result in Scala app, I realized that I got list/array of Scala `Int` (4 bytes), which is different than what JS provided, which was a list of bytes (that JS presents as unsigned integers in range [0..255]). Any idea why Scala code didn't work with `Byte`? Scala was supposed to be able to accept each JS byte and put it into a `Byte` (because the Scala code said `.as[Byte]`). But for some reason Scala looks at it as a (signed) integer, sees that its value > 127, and then throws an Exception. Any idea how to make Scala code accept input as `Byte` (like I originally did)? – rapt Feb 13 '23 at 11:42
  • Unfortunately, Java doesn't have unsigned bytes (0 to 255). The closest is Short -32,768 to 32,767 – Stanislav Kovalenko Feb 13 '23 at 12:21
  • But I am referring to Byte (not Int (or Integer), which may have the concept of signed/unsigned number). A Byte should be allowed to contain every one of the possible 256 8-bit permutations. And a Byte is not supposed to be interpreted as a number (whether signed or unsigned) – rapt Feb 13 '23 at 12:33
  • You JS byte is unsigned, Java byte is signed (-127 to 127, contains 255). So you can't use unsigned byte because it doesn't exist in Java. You have to use the next by capacity data type - Short or Int – Stanislav Kovalenko Feb 13 '23 at 14:37
  • I think what you say is inaccurate. JS Unit8Array contains unsigned int, not signed byte. Sign is concept of numbers, not of byte, which is just permutation of 8 bits. I think Unit8Array uses term uint instead of byte, although they are equivalent, because it's easier to provide bytes as uints. Now, I give Scala as input a series of bytes, and by specifying `.as[Byte]`, I tell it to interpret input as `Byte`s, rather than `Int`s. Interpreting as `Byte` should always work, because all data is bytes. If JS sees `11111111` as 255 it shouldn't prevent Scala Byte from having value of `11111111`. – rapt Feb 13 '23 at 15:05
  • I'm sorry I can't follow you. Does it relate to the question or it's discussion about the difference between js and java types. You can do whatever you want in reader. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Uint8Array Uint8Array contains 8-bit unsigned integers, so if you expect that you are going to have only 0,0,1,0 etc. then a Byte should work. But if you want to write universal reader for Uint8Array, you need to use a Long type in Scala. – Stanislav Kovalenko Feb 13 '23 at 15:25
  • Let's focus on the 2nd part of your comment. The size of a Uint8Array element is 1 byte (i.e. 8 bit) and it is interpreted by JS as an unsigned int whose value is in the range [0..255]. Is there a byte value in the universe that cannot be assigned to a Scala `Byte`? The answer is NO. So we should be able to assign every element (i.e. byte) of Uint8Array to a Scala `Byte`. Scala is not supposed to care how that byte is interpreted by JS. For Scala it's just a byte that we want to assign to a Scala `Byte` variable. There should not be a problem to do it. However, it does fail here. Why? – rapt Feb 13 '23 at 18:02
  • Because, Scala Byte range [-127..127] and JS unsigned int range is [0..255]. For Scala is the process converting a string e.g. JSON to a number. It sees smth more than 127 and fails. – Stanislav Kovalenko Feb 13 '23 at 18:37
  • This shouldn't have been a problem though. You should be able to put a byte from one environment into a byte in another environment. After looking into it, I noticed that JS sends the value of each key in the Uint8Array map as a text that represents a number, not as byte. E.g. `"1"=1` instead of `"1"=`. Then, on Scala side Json library translates this string to Int, which is the reason for the problem. – rapt Feb 21 '23 at 14:19
  • Please see my update above. – rapt Feb 21 '23 at 14:37