3

there is similar question (how to process a HTTP stream with Dart) about processing streams in dart2js. I am focused on vm.

I can read from spec that:

The content of message can be: primitive values (null, num, bool, double, String), instances of SendPort, and lists and maps whose elements are any of these. List and maps are also allowed to be cyclic.

In the special circumstances when two isolates share the same code and are running in the same process (e.g. isolates created via spawn), it is also possible to send object instances (which would be copied in the process). This is currently only supported by the dartvm. For now, the dart2js compiler only supports the restricted messages described above.

I ve learnt I cannot send to isolates and back following objects: HttpRequest and HttpResponse objects, I cannot send streams.

Q: I cannot really understand how should I process big chunk of data in isolate and then send it back to main isolate and in turn it can be send back to client.

Normally if I want to read a file. I can obtain a stream, apply transforms and then pipe a stream to http response. what is best practice to do that using isolates?

thanks for help!

Community
  • 1
  • 1
kamiseq
  • 593
  • 4
  • 17

1 Answers1

3

I have provided an example but Ill give a quick overview on how you can achieve this although it may not necessarily be the best practise - its simply the only way I personally know how.

We are given a method called Isolate.spawnUri(Uri uri, List<String> args, dynamic message)

The message parameter can hold any of the values the extract in your first post mentions. What we want to do is in the main thread create a new ReceivePort and listen for incoming data, then we want to spawn an isolate with the message as the ReceivePorts .sendPort.

The isolate should then create its own ReceivePort and use the message value to send back its own sendPort and listen on its receive port. What this essentially does is creates a 2 way communication. Allowing us to keep the isolate alive and send work back and forth.

Any response from your isolate will come through your original receive port and any data you want to send to the isolate will be through the send port the isolate just sent back.

You can then directly send the stream data to the isolate for it to process at will and it can send back data as its made (or you can just close the ports after isolate has sent its response - up to you how you want to go about it).

I have provided an example on GitHub: https://github.com/TomCaserta/ExampleIsolate

Please note, if you want your isolates to work in dart2js its important that you run dart2js on each of your isolate files via the following command:

dart2js -o <filename>.dart.js <filename>.dart

Tom C
  • 822
  • 7
  • 27
  • no im not interested in running this in browsers. I ll investigate your example and come back with any questions. thanks ;-) – kamiseq Feb 28 '14 at 19:02
  • Oh wow, I completely misread your first line and took it the other way around. It should still be relatively correct for the VM too I believe however there may be better ways of doing it as you are not limited to the primitive types. – Tom C Feb 28 '14 at 19:34
  • hi, what I understand you are not sending stream through sendPort, you just send simple objects and you feed your stream with whatever come from isolate. you could actually return ResponsePort as it implements Stream already. I just wonder if this is not adding additional performance overhead as you need to copy each event (whatever it is) between isolates. I was looking for solution where you process a file (large one) and you do it in spawn isolate and when you done you just pass output stream back to main isolates without copying data :/ I guess it is not possible. – kamiseq Mar 01 '14 at 12:14
  • Hmm, yeah I dont think that would be possible as all the objects get copied anyway. So even if you could send through the stream object it would just get copied on the other side. That being said, I wouldnt have thought it would have been a too bad overhead. All you'd need to do is move the file loading to the isolate, and get it to spit out the data as it processes it. – Tom C Mar 03 '14 at 16:49
  • well copying means you uses more memory and cpu, am I wrong? – kamiseq Mar 03 '14 at 18:52
  • In this answer http://stackoverflow.com/questions/21792901 @GregLowe writes that WebWorkers pass a reference. Would be interesting how this works on the server. – Günter Zöchbauer Mar 07 '14 at 05:08