0

Is there an internal size limit of TIdtcpserver buffer? How come whatever method I use, It reaches a limit of 65535?

I have encountered this buffer issue of TidTCPSever these days. My code is very basic: preset the size of buffer array, extract server byte from InputBuffer, and copy buffer array to workspace. Here is the code

TByteDynArray buffer;  // decliared in private    
void __fastcall TmodWifiCom::IdServerExecute(TIdContext *AContext)
{
        long readLength;
        int c, s;
        byte b;
        DataH->FDataReceivedBytes=0;
        AContext->Connection->IOHandler->CheckForDataOnSource(10);
        while (!AContext->Connection->IOHandler->InputBufferIsEmpty())  {
                // get hint of size of buffer
                s = AContext->Connection->IOHandler->InputBuffer->Size;
                buffer.set_length(s);
                AContext->Connection->IOHandler->InputBuffer->ExtractToBytes(buffer,-1,false);
                readLength = buffer.Length;
                for (long i = 0; i < readLength; i++) {
                        b = buffer[i];
                        DataH->FDataBuffer[DataH->FDataReceivedBytes++]=b; // copy buffer bytes to workspace
                }
                // process workspace
        }
}

The code appears to work fine, readLength and s are equal. FDataBuffer appears to recieve every bytes. However, as TidTCPSever reaches a limit deque fails.

// private of head file in other class 
frameQue_Type frameQue0;
deque<frameQue_Type> frameQue;

// cpp file in other class 
frameQue.push_back(frameQue0);
...
frameQue0 = DataH->frameQue.pop_front(); // [ERROR STOPS HERE]

The error message was: access volation #0048D893

I don't understand:

  • TidTCPSever and deque are in different classes
  • struct values in deque seem fine
  • Error occured as soon as buffer size reaches 65535 bytes

Am I using the buffer right?

user3859012
  • 21
  • 4
  • Your `buffer` variable is not being used in a thread-safe manner. Each client runs in its own thread, but you only have one `buffer` variable. You need a separate `buffer` per client. What is `DataH`, and do multiple clients share it? How is `FDataBuffer` declared and allocated? Is it 65535 bytes in size? How does it get emptied between TCP reads? You are not taking its size into account when copying `buffer` into it, so you could easily be causing buffer overflows and/or data corruption. – Remy Lebeau Sep 07 '16 at 01:35
  • Personally, I would suggest re-writing this code. This is not how you should be using Indy to read socket data. And your deque system appears to be implemented without any regard to thread safety. What does the actual communication protocol look like that your TCP peer is using to transmit its data? TCP is a byte stream, there has to be some kind of structure to the data to denote where one message ends and the next begins. Following that structure in your code would allow for more efficient and safer reading operations. What you have shown so far is a dangerous mess. – Remy Lebeau Sep 07 '16 at 01:40
  • Thanks Remy, You’re obviously very knowledgeable in this area so all of your feedback is well received, I hope that I can address your queries. This piece of code is for a tablet dedicated to a single sensor device. The device continuously sends a stream of non-packed bytes. So to answer your questions: • My TCPserver has one thread and one buffer, because this tablet only has one client (device) • FDataBuffer has allocated 131072 bytes. And it has itself emptied before every server execute cycle – user3859012 Sep 07 '16 at 09:43
  • • The content of FDataBuffer is processed after each cycle. Linear bytes are assembled into frames. Incomplete bytes of frame will be left for next cycle • Buffer is pre-allocated before copying (buffer.set_length(s)), and ExtractToBytes() ABytesCount parameter is set to -1. (I tried s here, it worked the same) – user3859012 Sep 07 '16 at 09:44
  • I am not sure of a better way to rewrite this code to read the indy socket. As I said, the incoming data from the device is like a stream of frames attached to each other, start to end. My program is expected to construct frames back from any length of the buffer. What I don’t understand is why buffer only reaches as much as 65535 bytes? What happens to the buffer if more data comes? What have I missed in the handling of this buffer? Also, what do you think I can do to make thread-safe? Thanks again. – user3859012 Sep 07 '16 at 09:44

0 Answers0