2

I have two sides for my projects. One is a camera, with which we use OpenCV, to capture frames, in 8 bit 3 channel (R,G,B), format and write it to socket to be streamed. Image is 640 x 480 resolution, so 640x480x3 bytes are written to the socket.

On the other end, I have a Flutter app which is listening to this socket. I am able to successfully gather the bytes, but I am not able to restructure an image using them. How can I do this? I am sharing what I have so far, but I am not able to display any visual.

import 'dart:typed_data';
import 'package:flutter/material.dart';
import 'package:ferenova_flutter_app/plan_display_page/plan_display_page.dart';
import 'package:ferenova_flutter_app/vars.dart' as vars;

import 'dart:io';

// ignore: must_be_immutable
class CameraStreamPage extends StatefulWidget {
  PlanDisplayPageState pdps;
  Socket socket;
  String title;

  CameraStreamPage(
    String title,
    PlanDisplayPageState pdps,
  ) {
    this.title = title;
    this.pdps = pdps;
  }

  @override
  _CameraStreamPage createState() => _CameraStreamPage();
}

class _CameraStreamPage extends State<CameraStreamPage> {
  // VlcPlayerController _vlcViewController;
  PlanDisplayPageState pdps;
  Socket socket;
  Image img;
  Uint8List bytes;

  @override
  void initState() {
    // TODO: implement initState
    super.initState();
    pdps = widget.pdps;
    this.socket = widget.socket;

    img = Image.network(
        'https://am23.mediaite.com/tms/cnt/uploads/2020/01/babyyoda.jpg');
    Socket.connect(vars.streamIP, int.parse(vars.streamPORT))
        .then((Socket sock) {
      socket = sock;
      socket.listen(dataHandler,
          onError: errorHandler, onDone: doneHandler, cancelOnError: false);
    }).catchError((Object e) {
      print("Unable to connect: $e");
    });
  }

  void dataHandler(data) {
    String str = new String.fromCharCodes(data).trim();
    print(str);

    vars.picture += str;
    int lim = 640 * 480 * 3;
    if (vars.picture.length >= (640 * 480 * 3)) {
      bytes = Uint8List.fromList(vars.picture.codeUnits);
      print('BBBBBBBBBBBBBBBBB: ' + bytes.length.toString());
      setState(() {
        img = new Image.memory(
          bytes,
          width: 640,
          height: 480,
          scale: 1,
          fit: BoxFit.contain,
        );
        vars.picture = vars.picture.substring(lim);
      });
    }
  }

  void errorHandler(error, StackTrace trace) {
    img = Image.network(
        'https://am23.mediaite.com/tms/cnt/uploads/2020/01/babyyoda.jpg');
    print('ERRORORORORORORORROROROR');

    print(error);
  }

  void doneHandler() {
    img = Image.network(
        'https://am23.mediaite.com/tms/cnt/uploads/2020/01/babyyoda.jpg');
    print('DONEDONEDONEDONEDONEDONE');
    vars.picture = '';
    socket.destroy();
  }

  @override
  Widget build(BuildContext context) {
    return Center(
      // child: SingleChildScrollView(
      child: Container(
        width: 1000,
        height: 600,
        color: Colors.green[400],
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          // mainAxisSize: MainAxisSize.min,
          children: <Widget>[
            Container(
              width: 900,
              height: 550,
              color: Colors.purple,
              child: img,
            ),
          ],
        ),
      ),
    );
  }
}

Also Following is a piece of what I receive:

P]rP]rT\rU^sX^tY_uYcvXbuVcsTaqP^lP^lR_mP^lR_mR_mU^jU^jS]iR\hQYjS[lR\oT]pT]pT]pR]mR]mR]mR]mU_pU_pU`nU`nYar[ctZd{]hYiaqhzw¤¡ §«©ª¨ ¬£ ¬£ ª¢ ª¢¡©¡© ¨§§§¢¥¢¥££¡¤¡¤¢¢¢¢¢¢  ¢¢¢¢¢¢         ¢¢¢¢¢¢   ¢¢£¢¡  ¡¡¡ ¡¢¢¢¢£¢£££££¤¤£¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¥¦¦ § § §¦¦ § §¡¨¢ §¡ §¡ §¡ §¡ §¡ §¡¡¨¢¡¨¢¡¨¢¡¨¢¡¨¢¡¨¢¡¨¢¢©£¢©£¤ª¤¤ª¤¤ª¤¤ª¤¢©£¡¨¢¢©£¢©£¢©£¤ª¤¥«¦¥«¦¤ª¤¤ª¤¤«£¤«£¤«£¤«£¨ª£¨ª£¨ª£¨ª£¨©¤¨©¤¨©

 ¡¡¡ ¡¡    .'/,$,-&..'/0(33+5MGM`Z_OFE&)!,$!*#$+$%.&)/'*/&,.%+*!'*!',$*+"(*!'*!'("')#()#('!&/).7164.13-0.(+,&),%+)#()"*(!)' ()"**#+.'/3.65085166288286054-31+1.'--&,/&,.%+.%+/&,2)/5,26-38/5:28:28:28=4:=4:<398045-02*.0(+,$(,$(+#'+#'+#'( #'"( #+#'+#')#()#(*$*,%+-)..*/,(-*&+,(-,(-,(--).-(0+&.*%,*%,'"*($+'"*'"**%,+&.+',,(-.'-.'-.'-.'-/).-&,-&,.'-.'-.'-/).-&,,%+,%+.'-.'-/'*+#'*
dramaticlook
  • 653
  • 1
  • 12
  • 39
  • 1) Why is your port stored as a String? This seems like a minor oversight, but it is actually an omen of what is about to happen. 2) Is your camera sending a String? - No, then don't use a String to store and use it further on. 3) What exactly where you expecting to see in the console when you told it to printout image bytes as a String? Your question takes [Stringly typed](https://blog.codinghorror.com/new-programming-jargon/) to a whole new level.... – Uroš Jan 07 '21 at 19:11
  • 1) what I shared is only a small portion of my project, since for some use cases, I need to integrate the PORT to a URL, which is a String. So if I define it as an int, I will need to convert it to a String in many more use cases. 2) It is sending a Uint8List. To be able to see and print it, I convert it to a String.3) it should print a bunch of bytes, as I shared. The question is clearly asking for guidance to form an image using such bytes, obviously I wasn't expecting to see an image on the console :D – dramaticlook Jan 08 '21 at 14:15
  • 1) I'd put that ip,port etc. data in a class and call an appropriate getter. 2) You are needlessly converting from byte data to String and then back to bytes, beside wasting resources you could introduce errors. 3) Meaningful log would be: `void logBytesAsInt(Uint8List bytes) {final StringBuffer sb = new StringBuffer(); for(final int u8 in bytes){ sb.write('$u8 ');} print(sb.toString());}` or `void logBytesAsHex(Uint8List bytes) { final StringBuffer sb = new StringBuffer(); for(final int u8 in bytes){ sb.write('0x${u8.toRadixString(16)} ');} print(sb.toString());}`. – Uroš Jan 09 '21 at 08:27
  • You might wanna save that log to a file or split the sb into smaller chunks so you can see the whole byte array in your console. Anyhow when you do log the byte data you will either see a) some [magic bytes](https://en.wikipedia.org/wiki/List_of_file_signatures) at the beginning, because images don't consist only of pixel data, there is also format, compression etc. involved. In which case `vars.picture.substring(lim);` is incorrect. Or b) you truly are getting rasterized image (bitmap) and you can't pass a raw bitmap to [Image class](https://api.flutter.dev/flutter/widgets/Image-class.html) – Uroš Jan 09 '21 at 08:52
  • Oh wow, thank you for the detailed explanation :) Since I have the working OpenCV C++ code for the client to receive the bytes and display it, I will give Flutter's OpenCV package a chance. Hopefully that will be a quick solution. Thanks – dramaticlook Jan 10 '21 at 19:57

1 Answers1

13

You can try this

Image.memory(Uint8List.fromList(YOUR_BYTES));
dm_tr
  • 4,265
  • 1
  • 6
  • 30