0

I'm trying to make a graphical engine in console. I don't use any of the GLs, but soon hoping to start. It is written in C and as for now, it is capable of building semi-3D environment in color, but I found, that win cmd supports True color scheme, using ANSI escape sequences.

Throughout my research on this topic I've found WriteFile(), on which all the printfs and putcs are based in Win10. But for all goods it does not provide it is still slow. As internal code shows, writefile is essentially based on this code:

NtWriteFile:

mov r10,rcx 
mov eax,8 
test byte ptr [7FFE0308h],1 
jne NtWriteFile+15h (07FFBA80CAAA5h) 
syscall                                 
ret

it is good, but slow and inefficient. What do I mean: I'm passing output handle of my console to WriteFile() and the buffer, but for the screen of 160*60 (considering every character has its own ESC[48;2;r;g;bm, which is 20 bytes long) we get 192000 bytes. The system call has 84ms of execution.

Then I returned to my previous setup with WriteConsoleOutputW(). It uses the same output ptr of the console, but this time for the buffer of 320*84 (it uses CHAR_INFO ptr as a buffer, so 4 bytes each)-107 520 bytes it lasts only 1ms!

WriteConsoleOutputW() is based on similar looking syscall code:

NtDeviceIoControlFile:

mov r10,rcx 
mov eax,7                           ; notice here is the 7 instead of 8 
test byte ptr [7FFE0308h],1 
jne NtDeviceIoControlFile+15h (07FFBA80CAAA5h) 
syscall                                 
ret

but this time it outputs roughly similar amounts of info 80 times faster! My assumption, that it's just reallocating a pointer to console buffer, because when I start it with buffer of single character it outputs some strange symbols with colors

I've tried to output ESC sequences as a CHAR_INFO but it outputs them as a text. So the question is: Can I somehow output an escape sequence using WriteConsoleOutput or NtDeviceIoControlFile()

In Addition: my frame buffer will consist only of escape sequences and space symbol after it. The Width and Height of the buffer, as well as a rectangle is defined, all I need to do is output correct foreground colors on each of spaces.(so the length will be 20 * Height*Width)

Ilya Pakhmutov
  • 159
  • 11
  • `As internal code shows, writefile is essentially based on this code:` and `WriteConsoleOutputW() is based on similar looking syscall code:`: Looking at this code is pointless. The "magic" happens when `syscall` is hit, which jumps to the kernel where the real code is. – tkausl Apr 07 '19 at 16:05
  • @tkausl I know, but all those functions are callable from code and have their documentation, but in case of writefile esc are read, bun in second they aren't. What I'm trying to show is where the slowness is – Ilya Pakhmutov Apr 07 '19 at 16:07
  • How did you measure those times? How do you know the OS did not pe-empt your task? – Paul Ogilvie Apr 07 '19 at 16:14
  • @PaulOgilvie with 2 measurments: 1) internal debug tool of Visual studio 2015, 2) QueryPerformanceCounter on wrapped functions (they looked similar enough) – Ilya Pakhmutov Apr 07 '19 at 16:16
  • That just gives you high resolution time stamps but you still don't know if the OS swapped your task out during your measuring. With assembly you just not are on the "bare metal" and you still go through the OS. And with a console "window" on your desktop window you are still windowing and so you have to go through the OS. So I believe there is no way to speed up. – Paul Ogilvie Apr 07 '19 at 16:27
  • Note also that escape sequences are intercepted by the terminal or terminal _emulator_ such as the original VT100. There is no hardware that you can give escape sequences. – Paul Ogilvie Apr 07 '19 at 16:29
  • @PaulOgilvie the thing is, how do I access a VT100? If the values are just intercepted there must be a way to just put those values directly. They are put by NtWriteFile(), as I haven't found any format processing – Ilya Pakhmutov Apr 07 '19 at 16:32
  • 1
    It's documented that VT support is implemented only for the high-level stream functions `WriteFile` (i.e. `NtWriteFile` in Windows 8+, but the syscall is an implementation detail; it was different in Windows 7) and `WriteConsole` (i.e. `NtDeviceIoControlFile` in Windows 8+, for a screen-buffer file, using an undocumented IOCTL code). The low-level console API is character-cell based instead of stream-like, so it's not compatible with a virtual-terminal interface. The character-cell API performs better, but it comes at the cost of losing high-level processing. – Eryk Sun Apr 08 '19 at 09:19
  • @eryksun As experiments shown, yes, the terminal itself is slow, I made a program, which used syscall 7 and it still ran for 88ms, thx for the answer – Ilya Pakhmutov Apr 08 '19 at 10:16
  • We shouldn't make the system call directly, and not even via the `NtDeviceIoControlFile` stub in ntdll.dll. It provides no significant performance advantage at the cost of portability. The IOCTL code that's assigned to the `WriteConsoleOutput` function is undocumented (note that this is not the system call number; `NtDeviceIoControlFile` is an extensible call interface for devices). Moreover, the console didn't even use a device driver until Windows 8. Previously it used an LPC port, and `WriteFile` to a console file was internally redirected to `WriteConsole`. – Eryk Sun Apr 08 '19 at 10:59

1 Answers1

0

CMD is a VT100 emulator (that information is from the Telnet server app help).

I'm not sure why you keep talking about ANSI and the function not having features.

ANSI is not supported except for very recent versions of Windows 10.

Not using ANSI one uses SetConsoleTextAttribute (which means colour). And there are other functions to move the cursor.

    Dim hOut as IntPtr
    Dim Ret as Integer
    hOut  = GetStdHandle(STD_OUTPUT_HANDLE)
    Ret = SetConsoleTextAttribute(hOut,  &hfA)
    Console.Out.Write("*")
    Ret = SetConsoleTextAttribute(hOut, &hfC)
    Console.Out.Write("WARNING")
    Ret = SetConsoleTextAttribute(hOut, &hfA)
    Console.Out.Write("*" & vbcrlf)

To move the cursor use

Pos.X = CInt(Xpos)
Pos.Y= CInt(Ypos)
Ret = SetConsoleCursorPosition(hOut,  Pos)

The behaviour of writefile (and there are three ways to write) is controlled by SetConsoleMode (which also can turn ANSI on). See https://learn.microsoft.com/en-us/windows/console/setconsolemode.

The two high level modes are writefile and writeconsole. The low level is writeconsoleoutputcharacter.

Noodles
  • 264
  • 2
  • 3
  • 2
    CMD is a shell and scripting language that uses standard I/O. It's not a console or terminal, and definitely not a virtual-terminal emulator. It uses a console by default for standard I/O, which it will allocate if it doesn't inherit one, just like any other console application. – Eryk Sun Apr 08 '19 at 09:09
  • Telnet into Windows and see what shell you get. – Noodles Apr 08 '19 at 19:06
  • 1
    A shell is neither a terminal/console nor a telnet/ssh server. If we telnet or ssh into a Windows 7/8 system, the server has to allocate a console and spawn the target program (any console-based app, not just cmd.exe) such that it inherits the console. Then it has to poll and read the screen buffer and transform this into a stream that can be interpreted by the terminal of the client. Thankfully Windows 10 has pseudoconsole support. Now the server can call `CreatePseudoConsole` to get VT streams instead of implementing this from scratch with inefficient polling. – Eryk Sun Apr 09 '19 at 02:41