Windows Batch File : Convert Structures into Single Lines Strings

Question

This hard task I need to do as a batch file, it's not the most difficult to do with C but in DOS is a hell (at least for me!) I need to convert a structure to a single var (string) to be able to convert them into that structure again inside my program. Don't worry about the comeback, it's already done.

The struct WILL vary in size and have an index inside [0] which need to be the first item in my var.

Here an exemple of these structures (just 3):

level.waypoints[0] = spawnstruct();
level.waypoints[0].origin = (1774.5,834.7,67.6);
level.waypoints[0].type = "stand";
level.waypoints[0].childCount = 2;
level.waypoints[0].children[0] = 1;
level.waypoints[0].children[1] = 6;
level.waypoints[1] = spawnstruct();
level.waypoints[1].origin = (1780.6,639.1,65.5);
level.waypoints[1].type = "stand";
level.waypoints[1].childCount = 2;
level.waypoints[1].children[0] = 7;
level.waypoints[1].children[1] = 0;
level.waypoints[2] = spawnstruct();
level.waypoints[2].origin = (1902.2,-345.2,74.2);
level.waypoints[2].type = "stand";
level.waypoints[2].childCount = 2;
level.waypoints[2].children[0] = 3;
level.waypoints[2].children[1] = 97;
level.waypoints[3] = spawnstruct();
level.waypoints[3].origin = ... (goes on...)

So the first structure:

level.waypoints[0] = spawnstruct();
level.waypoints[0].origin = (1774.5,834.7,67.6);
level.waypoints[0].type = "stand";
level.waypoints[0].childCount = 2;
level.waypoints[0].children[0] = 1;
level.waypoints[0].children[1] = 6;

I need to convert this structure into this line:

set flwp_0 "0,1774.5,834.7,67.6,stand,2,1,6"

Where 0 (flwp_) it's the index and also the first item inside "".

So the other 2 in this exemple will be:

set flwp_1 "1,1780.6,639.1,65.5,3,7,0,30"
set flwp_2 "2,1902.2,-345.2,74.2,2,3,97"

The childCount could be any value, but I don't believe will pass than 9. So, how many childCount we have, more children will be in the next lines.

But I just need the values after the childCount value, so I with that value I know how much more items will be available in that string, so no problem.

I'm thinking removing everything I don't need first like (in order):

level.waypoints[
] = spawnstruct();
].origin = (
);
].type = "
";
].childCount = 
].children[0] = 
].children[1] = 
].children[2] = 
].children[3] = 
].children[4] = 
].children[5] = 
].children[6] = 
].children[7] = 
].children[8] = 
].children[9] = 
;

This will left something like this in the first structure:

0
01774.5,834.7,67.6
0stand
02
01
06

Now I need to read that and transform into my single line string. But I'm having problems to know the right index because could be the first value, but if the index have 2 or 3 chars?

I also was thinking in replace with some info to track me, so will not replace the "]" of the files, so I'll know what is before that is the index and after that the values.

I was also trying to a loop without success creating a index like:

level.waypoints[X]

Where X is from 0 to 1000 (I don't believe will pass that number)

It's not working, maybe my first idea was better... :\

I need a batch file (or several) to convert several files which have structures like that inside to new files with the values as strings as I wrote in details above.

Any ideas?

Thank you!

---

@Squashman Also the result is a little messed up in the order as you can see:

set flwp_0 "0,1774.5,834.7,67.6,stand,2,1,6"
set flwp_100 "100,1636.76,371.924,240.125,stand,2,101,99"
set flwp_101 "101,1861.45,437.846,240.125,stand,2,102,100"
set flwp_102 "102,1843.93,557.03,240.125,stand,2,103,101"
set flwp_103 "103,1504.58,553.357,64.125,stand,2,8,102"
set flwp_104 "104,653.17,1675.32,64.125,stand,2,26,105"
set flwp_105 "105,338.784,1680.49,232.125,stand,2,104,133"
set flwp_106 "106,-919.398,1537.7,80.125,stand,3,107,109,150"
set flwp_107 "107,-928.311,1111.47,80.125,stand,3,108,106,149"
set flwp_108 "108,-696.488,1095.93,80.125,stand,2,36,107"
set flwp_109 "109,-787.781,1566.87,80.125,stand,2,106,110"
set flwp_10 "10,1423.3,-403.8,64.3,stand,4,11,12,15,3"
set flwp_110 "110,-754.274,1716.71,80.125,stand,2,109,34"
set flwp_111 "111,-736.201,1887.87,64.125,stand,4,112,34,151,152"
set flwp_112 "112,-454.293,1879.4,64.125,stand,4,111,33,34,152"
set flwp_113 "113,125.395,-451.579,58.6958,stand,3,51,114,142"
set flwp_114 "114,282.593,-447.87,68.125,stand,2,115,113"
set flwp_115 "115,304.311,-271.206,68.125,stand,2,116,114"
set flwp_116 "116,447.128,-292.167,68.125,stand,3,118,115,117"
set flwp_117 "117,437.415,-443.822,68.125,stand,2,67,116"
set flwp_118 "118,582.932,-297.811,126.125,stand,2,119,116"
set flwp_119 "119,561.586,-485.701,204.125,stand,2,120,118"
set flwp_11 "11,1240.1,-249.7,74.1,stand,5,9,16,10,12,15"
set flwp_120 "120,423.422,-470.754,204.125,stand,2,119,135"
set flwp_121 "121,1164.82,-1203.64,72.125,stand,4,123,14,126,122"
set flwp_122 "122,1064.12,-1299.76,72.125,stand,3,123,125,121"
set flwp_123 "123,1117.59,-1476.49,72.125,stand,3,124,122,121"
set flwp_124 "124,861.878,-1497.7,72.125,stand,2,125,123"
set flwp_125 "125,878.58,-1294.38,72.125,stand,2,124,122"

Is possible to write in the file in the right order?

thank you very much again for the help @Squashman

cheers

---

Ok, really doesn't matter the order, but I'm getting trouble to make do it to everyfile into a new one... I tryed this without success:

@echo off
 setlocal enabledelayedexpansion

for %%f in (*.gsc) do ( 

 FOR /F "tokens=1-2 delims==;^(^) " %%G IN (%%f) do set %%~G=%%~H
 FOR /F "tokens=2 delims=[]" %%G IN ('set level.waypoints ^|find /I "spawnstruct"') do (
    SET waypoints=!waypoints! %%G
 )

 FOR %%G IN (%waypoints%) do (
    set line=%%G,!level.waypoints[%%G].origin!,!level.waypoints[%%G].type!,!level.waypoints[%%G].childCount!
    FOR /F "tokens=2 delims==" %%H IN ('set level.waypoints[%%G].children') DO set line=!line!,%%H
    set line=set flwp_%%G "!line!"
    echo !line! >> %%f.cfg
 )

)
 pause

Any clues?

=======================================

Ok, now we are VERY close! Need to process ONLY the lines with level.waypoints[ and the rest must be ignored, and also add a final close var that I already did with success below:

@echo off
setlocal enableDelayedExpansion

set findtext="level.waypoints["
for %%F in (*.gsc) do (
  set "out="
  set "i=x"
  > "%%~nF.cfg" (
    for /f usebackq^ tokens^=2^,4^,5^ delims^=[]^=(^)^;^"^  %%A in ("%%F") do (
      if %%A neq !i! (
        if defined out echo !out!"
        set /a "i=%%A, j=0"
        set "out=set flwp_!i! "!i!"

      ) else (
        set /a j+=1
        if !j! leq 3 (set "out=!out!,%%B") else set "out=!out!,%%C"
      )

    )
    if defined out echo !out!"
    set /a "fim=i+1"
    echo set flwp_!fim! "eof"
  )
)

Now how can I add findstr %findtext% into these loops? I tryed in many different ways, but the syntax is wrong... Could you help me? :D

Thank you very much @Squashman and @dbenham for all the help until now.

Cheers

Answer 1

Edited the code with your new requests.

 @echo off

 FOR %%F IN (*.gsc) DO (
    setlocal enabledelayedexpansion
    FOR /F "usebackq tokens=1-2 delims==;^(^) " %%G IN (`find /I "level.waypoints["^<"%%F"`) do set %%~G=%%~H
    (FOR /F "tokens=2 delims=[]" %%G IN ('find /I "spawnstruct" ^<"%%F"') do (
        set line=%%G,!level.waypoints[%%G].origin!,!level.waypoints[%%G].type!,!level.waypoints[%%G].childCount!
        FOR /F "tokens=2 delims==" %%H IN ('set level.waypoints[%%G].children') DO set line=!line!,%%H
        set line=set flwp_%%G "!line!"
        echo !line!
    ))>"%%~nF.cfg"
    endlocal
 )
 pause

Answer 2

Here is a relatively fast pure batch solution that uses a single FOR /F loop for each file. It is significantly faster than Squashman's solution.

I set DELIMS and TOKENS to parse out all needed data elements, no matter which line I am parsing. I don't worry about the value of childCount. Instead I simply detect the change of the waypoint index to signal the start of a new record, and an incrementing waypoint line number (j variable) is used to identify which tokens are needed from each line. Each line adds new text to the out variable.

@echo off
setlocal enableDelayedExpansion
for %%F in (*.gsc) do (
  set "out="
  set "i=x"
  > "%%~nF.cfg" (
    for /f usebackq^ tokens^=2^,4^,5^ delims^=[]^=(^)^;^"^  %%A in ("%%F") do (
      if %%A neq !i! (
        if defined out echo !out!"
        set /a "i=%%A, j=0"
        set "out=set flwp_!i! "!i!"
      ) else (
        set /a j+=1
        if !j! leq 3 (set "out=!out!,%%B") else set "out=!out!,%%C"
      )
    )
    if defined out echo !out!"
  )
)

The above is able to process a 23 MB file on my machine in 70 seconds.

But I can do better if you allow yourself to go outside the bounds of pure batch. Here is an efficient JREPL.BAT solution that makes use of new version 6.0 features that were released just moments ago.

JREPL.BAT is a regular expression text processing utility for Windows. It is pure script (hybrid batch/Jscript) that runs natively on any Windows machine from XP onward. No 3rd party exe files are required.

Extensive documentation is available from the command line via jrepl /?, or jrepl /?? for paged help.

This solution should be able to handle files approaching 1 GB in size. Each file is processed completely by a single call to JREPL.BAT. It should be very efficient, as there is minimal back-tracking required. For small files, it will be slower than the pure batch method due to the startup time of the CSCRIPT engine. But for large files it is much faster than pure batch.

The code below processes the same 23 MB file in only 11 seconds :-)

This first version breaks construction of the complicated regex terms into steps, and includes commented numbers above each captured group to give you a chance at fathoming how it works. In particular, read the documentation for the /T and /P options. And also bone up on your regex skills!

@echo off
setlocal

set "filter=(.*?\[\d+])[\s\S]*?(?:\n(?!\1)|(?![\s\S]))"

::   1           2                3                  4
set "find1=l.*?\[(\d+)\][\s\S]*?\((\d.*?)\)[\s\S]*?\q(.*?)\q"
set "repl1=set flwp_$2 \q$2,$3,$4"

::   5                 6
set "find2=;[\s\S]*? = (\d+)"
set "repl2=,$6"

::   7
set "find3=[\s\S]+"
set "repl3=\q\n"

set "find=%find1%|%find2%|%find3%"
set "repl=%repl1%|%repl2%|%repl3%"

for %%F in (*.gsc) do (
  call jrepl.bat "%find%" "%repl%" /t "|" /p "%filter%" /x /m /f "%%F" /o "%%~nF.cfg"
)

And here is the exact same solution, but without showing any of the intermediate steps. I did include line continuation to improve readability.

@echo off
for %%F in (*.gsc) do (
  call jrepl.bat "l.*?\[(\d+)\][\s\S]*?\((\d.*?)\)[\s\S]*?\q(.*?)\q|;[\s\S]*? = (\d+)|[\s\S]+"^
                 "set flwp_$2 \q$2,$3,$4|,$6|\q\n"^
                 /p "(.*?\[\d+])[\s\S]*?(?:\n(?!\1)|(?![\s\S]))"^
                 /t "|" /x /m /f "%%F" /o "%%~nF.cfg"
)

I suppose the fastest script solution would be a custom built JScript or VBS script. But I enjoy working with JREPL :-)

Update: Here is a little bonus - a JREPL script that will convert *.cfg back to *.gsc

@echo off
setlocal

set "beg=var pre,i,q='\x22',n='\r\n'"

::        1     2     3             4     5     67
set "find=^.*?\q(\d+),(.*?,.*?,.*?),(.*?),(.*?),|(.*?)[,\q]"

set "repl=i=0;pre='level.waypoints['+$2+']';$txt=pre+' = spawnstruct();'"
set "repl=%repl%+n+pre+'.origin = ('+$3+');'"
set "repl=%repl%+n+pre+'.type = '+q+$4+q+';'"
set "repl=%repl%+n+pre+'.childCount = '+$5+';'"
set "repl=%repl%|$txt=n+pre+'.children['+(i++)+'] = '+$7+';'"

for %%F in (*.cfg) do (
  call jrepl.bat "%find%" "%repl%" /x /t "|" /jq /jbeg "%beg%" /f "%%F" /o "%%~nF.gsc"
)