0

As soon as I try to access a folder/file containing an emoji in its name from my Lua 5.2 script, for example like this:

os.execute('start "" "' .. path .. "\\scripts\\menu\\ My Scripts" .. '"')

The Windows' Command Prompt simply refuses to open it with the following error message:

CMD and emoji in path issue

I'm aware Windows' Command Prompt doesn't support emojis and therefore is not possible to make it work just like that, but my doubt is if won't exist some workaround or whatever I can do to ensure any Windows/Unix user is going to able to get the folder open by my Lua script without any problem.

I have tried i.e. things like use the codes instead (1246 and U+1F4F0 in this page facing up case) without success. Couldn't I for example simply use some kind of "wildcard" instead? I mean, knowing it's always going to be the very first character in the name. Or, well, any other ideas will be welcomed, cause nothing I'm trying really seems to work...

Of course if it's going to represent any problem I'll simply refuse to use them, but it came in handy for some "first sight" folder distinction and, if possible, I'd like to can count this little visual resource

Rai
  • 314
  • 1
  • 2
  • 9
  • _"aware Windows' Command Prompt doesn't support emojis"_ - what makes you think `os.execute` invokes programs via `cmd.exe` instead of via Win32 directly? – Dai Oct 30 '22 at 02:33
  • Well, the fact that when I use it the CMD jumps and remains visible for a second or so and then closes itself just before the folder gets open... So I kind of assumed that's how it worked after all... but, if that's not right, of course I'll be happy of learning something new! – Rai Oct 30 '22 at 02:39
  • Ah, I was wrong - and you're right: [Lua's docs say `os.execute` works the same as C's `system`](https://www.lua.org/pil/22.2.html), which really does invoke the shell rather than using a syscall (e.g. [`execve`](https://man7.org/linux/man-pages/man2/execve.2.html)) or "normal" OS API call (e.g. `CreateProcess` in Win32) – Dai Oct 30 '22 at 02:43
  • Here's the answer to your question: [try using `io.popen`](http://www.lua.org/manual/5.1/manual.html#pdf-io.popen) - it looks like that starts a process via whatever the host's OS API is instead of via `system`, which means it should preserve Unicode text (I don't know what Lua uses internally, but Windows is thoroughly UTF-16 which means most emoji will be a Surrogate Pair - which does potentially comlicate things... - whereas Linux is UTF-8 and Windows 95 [was a mess](https://learn.microsoft.com/en-us/cpp/text/mbcs-programming-tips)). – Dai Oct 30 '22 at 02:48
  • 1
    You can try it with Bytecode? - To get the Bytecode use: ```print((''):byte(1,-1))``` and to use it escape the numbers with a backslash: ```print('\240\159\147\132')``` – koyaanisqatsi Oct 30 '22 at 06:51
  • With utf8 you can do: ```for p, c in utf8.codes('') do print(c) end``` and ```utf8.char(128196)``` - https://www.lua.org/manual/5.4/manual.html#6.5 – koyaanisqatsi Oct 30 '22 at 06:59
  • 1
    There could be used `os.execute('for /D %I in ("' .. path .. '\\scripts\\menu\\* My Scripts") do start %SystemRoot%\\explorer.exe /root,"%~sI"')` to run in background `C:\Windows\System32\cmd.exe /c for /D %I in ("C:\Whatever Path\scripts\menu\* My Scripts") do start %SystemRoot%\explorer.exe /root,"%~sI"` if the creation of short 8.3 names is enabled for the NTFS formatted partition which is usually the case for the system drive. – Mofi Oct 30 '22 at 07:18
  • @Dai OK, thanks for clarifying how it works! About trying io.popen, well, for what I've read for now, it seems to be used for opening files and I don't see how it could be used for opening a directory instead, but if I can find something of course I'll try... Greetings. – Rai Oct 31 '22 at 00:32
  • @koyaanisqatsi First off thanks! But I didn't specified I'm forced to use Lua 5.2 (just added now) for this, and things like "utf8.codes()" seem to be only available from 5.3 and above, a pity... Anyway, I've tried you other suggestions and those doesn't seem to work either since, even thought in this case Lua version is not a problem, it seems that try to use the Bytecode instead the emoji itself doesn't make any difference and the CMD ends up throwing just the very same error. But thanks again for the info! Because I wasn't aware about that and I think it will come in handy at some point. – Rai Oct 31 '22 at 00:45
  • @Mofi OK, this: `os.execute('for /D %I in ("' .. path .. '\\scripts\\menu\\* My Scripts") do start %SystemRoot%\\explorer.exe /root,"%~sI"')` is the first thing I try that does seem to work, but I think that opening the window in background will make difficult for the user to know what's going on... Is there a way to can make the explorer window appear over the main app like os.execute usually did in the other case? About the other variants, I'm having trouble to make them work here, but I'll continue making more tests tomorrow and I'll see... Thanks! – Rai Oct 31 '22 at 01:03
  • 1
    The window of `explorer.exe` is displayed by default on top of all other application windows. Neither `explorer.exe` nor `cmd.exe` control the Z-layer of all opened application windows. I don't have LUA installed at all. But if the Explorer window is not shown at top of all other application windows, the reason is LUA or your LUA script. Sorry, I can't help further. – Mofi Oct 31 '22 at 09:39
  • @Mofi Ahhh... It may be due to the host application then, no idea why it'll work as expected in the other cases, tho. But don't worry, you already helped a lot! I'll continue investigating, still have to do some more testing about all this... Thanks a lot! – Rai Oct 31 '22 at 11:52
  • @Mofi may I ask you a last question regarding all this I simply can't grasp? If what os.execute() requires (if I'm right) is a string with a simple valid CMD command after all, why can't one simply do i.e.: `os.execute('start "" "' .. path .. "\\scripts\\menu\\* My Scripts" .. '"')` and make the wildcard * do its magic by simply accepting any character instead as it normally would do? I mean, what is the reason for having to add all that loop complexity and all? Sorry if it's too basic, but I'm not being able to find the answer or figure out and it really intrigues me... Thanks for your time. – Rai Oct 31 '22 at 18:58
  • 1
    It is of course also possible to pass to `cmd.exe` a command line to execute with a short 8.3 path of the folder consisting only of ASCII characters instead of the long folder path with the emoji. But it is not possible to pass a command line with a folder path with a wildcard pattern character like `*` or `?` to `cmd.exe`. `cmd.exe` does not expand wildcard patterns anywhere in a command line in comparison to Linux/Mac shell interpreters which do that on argument string with `*` or `?` not enclosed in `'` (straight single quotes). – Mofi Oct 31 '22 at 19:20
  • 1
    The command to process a file system entry matching a wildcard pattern requires the usage of the internal command `for` of `cmd.exe`. Further note that Linux/Mac shell interpreters use by default UTF-8 encoding while `cmd.exe` uses by default a single byte per character encoding using a code page according to the country/region configured for the account used to run `cmd.exe`. This is [code page 437](https://en.wikipedia.org/wiki/Code_page_437) In North American countries and [code page 850](https://en.wikipedia.org/wiki/Code_page_850) in Western European countries. – Mofi Oct 31 '22 at 19:25
  • 1
    There is no code page supporting emojis. Emojis as characters can be encoded only with Unicode encodings like UTF-8 or UTF-16 using the appropriate surrogate pairs. `cmd.exe` respectively the classic Windows console API are designed to be compatible with programs and batch scripts written years before the emojis become popular and could be encoded with Unicode instead of small images. – Mofi Oct 31 '22 at 19:32
  • @Mofi OK, I could somehow glimpse some of the reasons but with your explanation I think everything is crystal-clear now (at least as far as I'm able to understand not being too familiarized on this CMD/batch/etc. matters). A pity even nowadays there is no way to make this possible like in the other systems. But, well... it is what it is, I guess! Anyway, thank you very much for you time again and have a nice day :) – Rai Oct 31 '22 at 23:55

2 Answers2

0

This is a Problem about how the string is constructed.
I found only one solution with [[command "path"]] (on Windows 11 and Lua 5.3)...

os.execute([[start ]] .. path .. [["\scripts\menu\ My Scripts"]])
-- My Testpath is/was: os.execute([[dir "%localappdata%\nvim\ Lua"]])

...the long string ([[]]) will not be interpreted (coercionated) by Lua.
That also have the side effect that you can use single backslashs with that kind of string.
Environment variable expansion (e.g. Windows: %localappdata%) only works inside doublequotes.
Single quotes instead ([[command '%localappdate%\path\']]) will not work (expanded).

koyaanisqatsi
  • 2,585
  • 2
  • 8
  • 15
  • As much as I'd like to end up using a simple solution like this I'm not being able to make it work, under Windows 10 22H2 & Lua 5.2.4 at least. No matter how many variations of i.e. `os.execute('start "" "' .. [[D:\Ramon0\Projects\Animation\Moho Pro\scripts\menu\ My Scripts"]])` I try, it just throws the same error as soon as an emoji is present. I'll continue testing and combining it with the other proposals, but it doesn't seem to make a difference in the way these special characters are passed to the CMD after all. Good tip about using [[]] to can avoid double backslashes, tho... Thanks! – Rai Oct 31 '22 at 13:30
  • Hm - Maybe it has someting to do what locale is present? Because i tested on a german Windows 11 (Lua Standalone 5.3) and using it not so often. Under Linux OS i can get the Locale with Lua: ```print(os.setlocale())``` -- Try it under Windows should put out by default: ```C``` – koyaanisqatsi Oct 31 '22 at 17:40
  • Yeah, it prints `C` here too. Sincerely, I don't know what to do at this point... I think I should start thinking on simply give up with this. I mean, the idea was only use emojis should they couldn't cause any trouble at all, and the only thing that seems to work as I commented above is Mofi's proposal: `os.execute('for /D %I in ("' .. path .. '\\scripts\\menu\\* My Scripts") do start %SystemRoot%\\explorer.exe /root,"%~sI"')`, but with some issues as well; plus, it's kinda "complex", and not being able to can test it under UNIX represents a risk that I don't think is worth taking after all.. – Rai Oct 31 '22 at 18:23
  • Under Windows - I think - It is worth to take a look at the current Environment* and maybe you find a way to set up a ```%localluaappdata%``` that includes the Emoji? Anyway - It is only a way for Windows - It will fail on mobile devices ( Linux ;- ) ) and iOS. *(Open cmd.exe and type ```set```) – koyaanisqatsi Oct 31 '22 at 20:18
  • With LuaJIT and ffi it is possible to decide what to do depending on the OS - https://luajit.org/ext_ffi_tutorial.html - On first look i thought it is only for Windows - No No No - Windows is an: Exception ;-) - Beside the example i like to say: Q: Why is Lua so fast? - A: It have no ```sleep``` or ```wait``` instructions – koyaanisqatsi Oct 31 '22 at 20:46
  • So what I was afraid in the end, that it wouldn't worth all the mess and possible risk of breaking things under other systems only for this. It would have been nice if there would have been a reasonably easy way, but well... At least I know now that I was not missing something obvious or a _hidden_ CMD trick or whatever. About LuaJIT, I don't know much about it (basically that it existed ) and it seems like an interesting reading, but a little too advanced for me, at this point at least, I'm afraid... But thank you very much for all the given attention and time! – Rai Nov 01 '22 at 00:12
0

os.execute accepts only ANSI-encoded strings (win-1252 in European Windows), but it is unable to encode an emoji.

Hint: you can create .bat-file to do the task for you and invoke it from Lua with os.execute.

Egor Skriptunoff
  • 23,359
  • 2
  • 34
  • 64
  • Hi, at this point I think I'm given up with this, but just to ensure I understood your hint... Are you suggesting that using a .bat file as an intermediate for the command would somehow help to skip the characters limitation? Because I thought such limitation was more from part of the CMD itself than from Lua's `os.execute`. Anyway, that would entail another file only for that and, again, I don't think it would worth after all... But thank you for pointing it out, of course! – Rai Nov 01 '22 at 00:24
  • @Ramon0 - `os.execute` input is always in Win ANSI codepage, but inside a .bat file you can `chcp` to Unicode codepage in the first line and execute any unicode command in the second line – Egor Skriptunoff Nov 04 '22 at 11:33
  • Ahhh... I think I get it, good to know. Maybe not this time for the previously exposed reasons, but I'm sure that info will come in handy at some point in the future. Thank you! – Rai Nov 04 '22 at 14:41