I have to puts large amounts of data to a file in TCL, and it takes very long. I tried increasing the buffer capacity from 4KB to 1MB using fconfigure, but noticed no improvement whatsoever.
I am not sure if I could flush my buffer at intervals, as I was guessing some of my data would be lost if I do so.
Is there some way I could increase the speed of puts without losing any data?
Generally the output speed is going to be limited by your disk drive's speed and computer system's i/o bandwidth.
Increasing the buffer size is probably the only thing you can do to help.
flush will slow down the write, as it will force-push the write buffer to the operating system.
If your incoming data stream ever pauses or comes in one big chunk that can be fit into memory, you can buffer the incoming data internally, and let the write catch up later.
If your data is coming from another channel (file, socket, whatever) then you can use fcopy to move it across. The fcopy command is careful to work as efficiently as possible, and if you configure both sides (incoming and outgoing) to use binary data transfer — so no encoding conversion or EOL/EOF character processing — then it can do it with minimal data copies; it's as efficient as a user-process level system can copy data (and you'd have to do hackery to move the copy into the OS kernel to do better). Obviously, having to process encoding conversion and transformation of end-of-line markers will slow things down.
Otherwise, the main bottleneck will still (probably) be device to which the output is being written. If it is going to a file, moving to writing to an SSD is the simplest option (but not necessarily the cheapest!) When writing over the network, better networking will make a gigantic difference. You really have to identify what the bottleneck really is; if Tcl is spending most of its time waiting for the hardware, there's very little point in working hard to make Tcl faster as you'll see virtually no results for that work. Fixing hardware bottlenecks is out of the scope of Stack Overflow, though some sister sites might be able to assist.
puts will not lose data unless you do something really evil like doing a force kill (kill -9) on the process, or doing a reset on the location of the file pointer from C code.
