Why does InputStreamReader giving me U-FFFD? How can I fix it?

Question

I am trying to receive data from bluetooth device which is expected to be in this format:

00 FE 80 01 01 A0 0A B0 0B C0 0C D0 0D E0 0E F0 0F 00 01 02 03 04 05 06 07 08 09
0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F 20 21 22 23 24 25
26 27 28 29 2A 2B 2C 2D 2E 2F 30 31 32 33 34 35 36 37 38 39 3A 3B 3C 3D 3E 3F 40 41
42 43 44 45 46 47 48 49 4A 4B 4C 4D 4E 4F 50 51 52 53 54 55 56 57 58 59 5A 5B 5C 5D
5E 5F 60 61 62 63 64 65 66 67 68 69 6A 6B 6C 6D 6E 6F 70 71 FF FF 68 FF

and I would like it to be converted to decimal after reception, should be look like this:

   0 254 128 1 1 160 10 176 11 192 12 208 13 224 14 240 15 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 
   16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 
   48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 
   80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 
   109 110 111 112 113 255 255 104 255

However, what I am actually receiving is in this form:

�����F���t?���������������������������������������������

then if I try to convert it into hex:

FFFD FFFD FFFD FFFD FFFD 46 06 FFFD FFFD FFFD 01 74 3F FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD 
FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD 
FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD FFFD

and if in decimal, I will get these:

65533 65533 65533 65533 65533 70 6 65533 65533 65533 1 116 63 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533

This is the part of the code (I got from this library) where the data is being read using InputStreamReader and BufferedReader:

override fun openMessageStream(): Flowable<String> {
    checkNotClosed()
    return Flowable.create({ emitter ->
        val reader = BufferedReader(InputStreamReader(inputStream, charset))
        while (!emitter.isCancelled && !closed.get()) {
            synchronized(inputStream) {
                try {
                    val receivedString = reader.readLine()
                    if (!TextUtils.isEmpty(receivedString)) {
                        emitter.onNext(receivedString)
                    }
                } catch (e: Exception) {
                    if (!emitter.isCancelled && !closed.get()) {
                        emitter.onError(e)
                    }
                }
            }
        }
        emitter.onComplete()
    }, BackpressureStrategy.BUFFER)
}

I have tried other suggestions to change the charset being used in InputStreamReader. I already tried all the charsets listed in this documentation but still did not fixed the issue.

How can I read the data properly? Which part should I change in handling the incoming data from stream reader?

EDIT:

I am now able to receive the right values using the code below and following Andy's suggestion. Now I can proceed to scanning the start and end bits and take the data between. I know this code is not efficient, I am new to Java so I would highly appreciate any suggestion to make this code better.

   override fun openMessageStream(): Flowable<String> {
    checkNotClosed()
        return Flowable.create({ emitter ->
            val reader = BufferedInputStream(inputStream)
            while (!emitter.isCancelled && !closed.get()) {
                synchronized(inputStream) {
                    try {
                        val receivedString = reader.read()
                        if (!TextUtils.isEmpty(receivedString.toString())) {
                            emitter.onNext(receivedString.toString())
                        }
                    } catch (e: Exception) {
                        if (!emitter.isCancelled && !closed.get()) {
                            emitter.onError(e)
                        }
                    }
                }
            }
            emitter.onComplete()
        }, BackpressureStrategy.BUFFER)
    }

`I am trying to receive data from bluetooth device which is expected to be in this format:` No. The sender will certainly not send data in hexadecimal notation. And also not in decimal. What the sender sends is bytes. And bytes are just eight bits. Now its to you how you want to display the value represented by those bits. And you can do that in hexadecimal notation or decimal if you want. If you want to receive the bytes unharmed/unchanged then do not use a reader as that is for receiving text like Andy told you. — blackapps, May 27 '21 at 19:14

score 0 · Accepted Answer · answered May 27 '21 at 18:48

0

It doesn't look like you're reading binary data, but you're trying to read it as a string in a particular charset.

The FFFD is a replacement character, basically saying "the input data couldn't be interpreted in this charset".

Read it as binary. Instead of using an InputStreamReader, wrap your InputStream in a BufferedInputStream, and then read the bytes a chunk at a time.

You seem to want to return String, though: I can't comment on what you should do instead, or how to get your bytes into Strings if that's what you really need.

answered May 27 '21 at 18:48

Andy Turner

137,514
11
162
243

Thank you for your answer, I will try it. I actually need to return it as byte for further processing. – CarloDp May 27 '21 at 18:52
I am now able to get the correct data. I edited my post with the changes in my code. I think It is not efficient though. – CarloDp May 28 '21 at 13:27

Why does InputStreamReader giving me U-FFFD? How can I fix it?

1 Answers1