0

So I was wondering how to remove the linenumbers and the ascii while outputting data with xxd.

The code:

chst1="$(xxd -u -p -l1 -s 765 "$d2s")"
chst2="$(xxd -u -p -l1 -s 766 "$d2s")"
chst3="$(xxd -u -p -l1 -s 767 "$d2s")"
chst4="$(xxd -u -p -l1 -s 768 "$d2s")"
chst5="$(xxd -u -p -l1 -s 769 "$d2s")"
chst6="$(xxd -u -p -l1 -s 770 "$d2s")"
chst7="$(xxd -u -p -l1 -s 771 "$d2s")"
chst8="$(xxd -u -p -l1 -s 772 "$d2s")"
chst9="$(xxd -u -p -l1 -s 773 "$d2s")"

echo "Hex"
echo " $chst1 $chst2 $chst3 $chst4 $chst5 $chst6 $chst7 $chst8 $chst9 "
echo "Binary"
echo " $chst1 $chst2 $chst3 $chst4 $chst5 $chst6 $chst7 $chst8 $chst9 " | xxd -r -p | xxd -b

Which outputs

Hex
 67 66 00 28 08 F0 80 80 0C 
Binary
00000000: 01100111 01100110 00000000 00101000 00001000 11110000  gf.(..
00000006: 10000000 10000000 00001100                             ...

But I want 00000000: and 00000006: and gf.(.. and ... removed from the output.

As for the amount of variables, this is correct and ugly, but it's the only real working solution I have as they each are a variable on their own within the code.

  • 1
    What's in `d2s`? What have you tried? This should be trivial with Awk, but your code looks like it needs a more drastic refactoring. Anything with repeated command substitutions like that smells suspicious. – tripleee Aug 24 '20 at 13:59
  • lol. d2s is the Diablo Save game file. In there are stuff like level, char name, stats like strength, vitality, etcetera. This part of the script will read out the stats data so I can show this in my terminal while choosing a character. Which in turn is part of a larger (already working script) that allows me to select a character, move that to the official Savegame folder and play with it. Why? More than 50 characters in the game makes searching in the game character select menu very... eerie, at times it even crashed on me. –  Aug 24 '20 at 14:03
  • Also, the echo of Hex and Binary are merely a test to show what it looks like. In the end they'll just be converted to the data I want to extract (show in terminal) and the echo's as they currently show will be removed. So for now it's testing and the purpose to explain what I see. –  Aug 24 '20 at 14:14

2 Answers2

2

Bash has quite decent base arithmetic built in. But I can see no need to extract the hex digits one by one, either.

echo Hex:
xxd -u -p -l 9 -s 765 "$d2s" | sed 's/../ &/g'
echo Bin:
xxd -u -b -l 9 -s 765 "$d2s" | cut -c 11-65

If you really insist on plucking out the bytes one by one (maybe they are in reality at different non-sequential offsets?) you could collect the values into two arrays.

hex=()
bin=()
for offset in 123 234 345 456; do
    hex+=("$(xxd -p -s "$offset" -l 1 -u "$d2s")")
    bin+=("$(xxd -b -s "$offset" -l 1 "$d2s" | awk '{ print $2 }')")
done
echo Hex:
printf " %s" "${hex[@]}"
printf "\n"
echo Bin:
printf " %s" "${bin[@]}"
printf "\n"

For variation mainly, this demonstrates how to use Awk. I would generally go with cut for simple extraction where you know how far into each line you want to go, but Awk is definitely more versatile. It works very well when the substring offsets are not necessarily fixed, but you know how many fields there are on each line.

Parallel arrays are also convenient if you want to associate a label with each byte. Perhaps

labels=(Hunger Thirst "Need to pee" Arms Legs Velocity Strength)
...
for ((i=0; i<=${#labels[@]}; ++i)); do
  echo "${labels[i]}: ${hex[i]} (${bin[i]})"
done
tripleee
  • 175,061
  • 34
  • 275
  • 318
  • As shown in my script already. Indeed they are at different offsets. So a length of 1 bit at offset 765, then 766, etc. (and trust me, it gets worse than this haha). But thanks for your help, it saved me a CRAPLOAD of coding as I see from your code. ^_^ As for the bits. They each hold a value. Example: Strength: 40 Dexterity: 56 Energy: 83, etc. It's to visually show the data represented by those values. –  Aug 24 '20 at 14:46
  • Your code looks like the bytes are successive; but the array should work if you have a collection of non-sequential offsets where you want to pick out a single byte. (Fixed a typo in the code just now, sorry.) – tripleee Aug 24 '20 at 14:48
  • Aye, am going to work on your code tonight. Thanks man. ^_^ Edit: What if I want to change the bit lengths? let's say I need 4 bit lengths. Could I then add for offset in 123 234 345 456; for length in 1 4; do ?? Could it be this simple and am I overthinking things? –  Aug 24 '20 at 14:52
  • Not sure what you mean. Do you want to extract the high and low nybble of a byte separately? – tripleee Aug 24 '20 at 15:38
  • With Awk, `substr($1, 1, 4)` gets the first four characters of the first field. From the hex digits, each digit is a nybble. There is no way with `xxd` to fetch less than one byte. – tripleee Aug 24 '20 at 15:42
  • Let's give an example as the script above isn't fully complete, it just focused on the issue at hand. I can read the character name with this: charname="$(xxd -u -p -l16 -s 20 $d2s | tr '0' 'n'| xxd -r -p)". A length of 16 at offset 20. The total length of a name is 15 characters. .. Trust me. In the end it's a LOT of output gathered from the savegame file. Different bit lengths, different outputs, etc. Even right now with the binary output, which will be reversed (using | rev) and then translated to the value in question. –  Aug 24 '20 at 16:05
  • Sounds like you should post a new question with detailed requirements and properly formatted code. The above definitely looks malformatted, and I still don't think I understand what you are trying to achieve. Why do you extract 16 bytes if you only need or expect 15? What does this have with bit lengths to do? – tripleee Aug 24 '20 at 16:33
  • Character names are storted as a [16]byte which will contain the name, one letter per byte. The name can be 15 characters long, and a name that's shorter will have padded 0x00's behind the name until we reach 16 bytes. Ask Blizzard this question. :P –  Aug 24 '20 at 16:35
  • That's a null-terminated C string, but you don't need the null to extract it. (If the value is shorter than 15 characters the null could come earlier, so I guess it does make sense to extract all 16 and then parse up to the null.) – tripleee Aug 24 '20 at 16:37
0

and welcome to SO!

First, let's not spawn so many redundant processes.

 xxd -u -p -l9 -s 765 "$d2s" |
   perl -ne 'for my $o (0..16) { next if $o%2; printf "%08b ", substr($_,$o,2) }'
 01000011 01000010 00000000 00011100 00001000 00000000 01010000 01010000 00000000

I didn't add a newline, but we could. If you don't have perl (go get it!), then

for offset in {0..16}
do (( offset%2)) && continue
   printf "%08d " $(bc <<< "ibase=16;obase=2;${sample:$offset:2}")
done
01100111 01100110 00000000 00101000 00001000 11110000 10000000 10000000 00001100

Again, no newlines, but you seem able to figure that out... but now we're spawning a bunch of procs again. Rather than that, use tripleee's cut.
Let me know if you want more specifics.

Paul Hodges
  • 13,382
  • 1
  • 17
  • 36
  • Thanks. I will install Perl. As for the newlines. Yes they're needed. Example: Strength: 40 Dexterity: 56 Energy: 83, etc. It's to visually show the data represented by those values. –  Aug 24 '20 at 14:49
  • 1
    Perl should be *de facto* standard on most platforms these days, unless you are on something really strapped like Busybox (or something really stupid like Windows). – tripleee Aug 24 '20 at 14:57
  • Yeah, I noticed that. I never used it and basically began a few weeks ago with bash scripting. I run Debian 10, it has perl 5, version 28 installed. –  Aug 24 '20 at 16:01
  • I only started looking at learning `awk` a year or so ago, since it's so widely used. I started using `perl` on the first unix system I worked on a couple decades ago, and never saw the point after that... I have yet to find anything `awk` does that `perl` won't, though it is admittedly sometimes nice not to have to add the command line options to get there. – Paul Hodges Aug 24 '20 at 16:25
  • *"As for the newlines. Yes they're needed."* Where? – Paul Hodges Aug 24 '20 at 16:28