Best way to avoid output truncation when using IJulia-notebook

Question

From the source I can see that IJulia

Truncate the output to max_output_per_request bytes per execution request since excessive output can bring browsers to a grinding halt.

I also agreed that having very long prints during interactive sections is not necessarily useful, but sometimes I want to print information that will take only one line for tracking the progress.

IJulia track the number of written bytes using stdio_bytes:

using IJulia

N = 10_000
slen = 100

println("\nPrinting ", N, " lines")
for i in 1:N
    IJulia_counter = IJulia.stdio_bytes[]; # IJulia byte counter
    print(rpad(string("line: ", i, ", iJulia counter: " , IJulia_counter), slen), "\r")
    mod(i, 50) == 0 && flush(stdout) # just forcing to print
    IJulia_counter > 524000 && flush(stdout)
end
flush(stdout)
# Final output
# Printing 10000 lines
# line: 5190, iJulia counter: 524089    
# Excessive output truncated after 524291 bytes.

Im currently working around by resetting this counter manually

println("\nPrinting $N lines")
for i in 1:N
    IJulia_counter = IJulia.stdio_bytes[]; # IJulia byte counter
    print(rpad(string("line: ", i, ", iJulia counter: " , IJulia_counter), slen), "\r")
    mod(i, 50) == 0 && flush(stdout) # just forcing to print
    IJulia_counter > 524000 && (IJulia.stdio_bytes[] = 0) # Reset counter
end
flush(stdout)
# Final output
# Printing 10000 lines
# line: 10000, iJulia counter: 485810   

What is the best approach to address this issue? Is any external configuration possible? I don't want to import IJulia just to prevent this in the interactive sections.

Thanks!!!

Answer 1

You need to add:

if isdefined(Main, :IJulia)
    Main.IJulia.stdio_bytes[] = 0
end

This allows to avoid I/O throttling by IJulia when IJulia is loaded and will be ignored otherwise (and hence you do not have to load IJulia if you do not need it). This recommendation is based on its code: https://github.com/stevengj/ProgressMeter.jl/commit/4bfc3e5a7e4ea889022ba704e741f3b8eaca0752 so this is probably the best you can get.

Full sample code:

using IJulia

N = 10_000
slen = 100

println("\nPrinting ", N, " lines")
for i in 1:N
    print(rpad(string("line: ", i ), slen), "\r")
    if mod(i, 50) == 0 
        flush(stdout) # just forcing to print
        if isdefined(Main, :IJulia)
            Main.IJulia.stdio_bytes[] = 0
        end
    end
end
flush(stdout)

I recommend you also trying to consider ProgressMeter.jl for monitoring your progress:

using ProgressMeter
N=10000
p = Progress(N, 0.1)
for i in 1:N
    next!(p)
    rand(100_000) #added so something is happening here
end