How to Write Block of Random Data to File using Bash

Question

I need to write a 5MB block of data obtained from /dev/urandom to a partition, at a specific location in the partition. I then need to check that the write executed correctly. I was previously successfully doing this in C++, but now I would like to implement it in a bash script.

My C++ code consisted of:

create 10MB array of data populated from /dev/urandom (RANDOM_ARRAY)
open partition with open()
use lseek() to navigate to desired position in partition
use write() to write the array into the partition
close and reopen partition, use lseek to navigate back to desired position
use read() to read 5MB at this position and populate another array with this data (WRITTEN_ARRAY)
compare each element in (RANDOM_ARRAY) with (WRITTEN_ARRAY)

Im not experienced with writing bash scripts but this is what I've got so far, although it doesn't seem to work:

random_data="$(cat /dev/urandom | head -c<5MB>)"

printf $random_data | dd of=<partition_path> bs=1 seek=<position_in_partition> count=<5MB>

file_data=$(dd if=<partition_path> bs=1 skip=<position_in_partition> count=<5MB>)

if [ "$random_data" == "$file_data" ]
then
 echo "data write successful"
fi

Thanks to the helpful commenters my script now looks something like this:

# get 10MB random data 
head -c<10MB> /dev/urandom > random.bin
# write 10MB random data to partition
dd if=random.bin of=<partition_location>
# copy the written data
dd if=<partition_location> count=<10MB/512 bytes> of=newdata.bin
# compare 
cmp random.bin newdata.bin

At this point cmp returns that the first char is different. Looking at a verbose output of cmp and turns out all values in newdata.bin are 0.

You cannot `printf` your binary data like this. Use an intermediate file: `head -c 5MB /dev/urandom > data.bin`. And then: `dd if=data.bin of=...` Same in the other direction: `dd if= of=newdata.bin...` — Renaud Pacalet, Jun 09 '22 at 11:14
The variables in bash are string terminated by ```\0```: this is not suitable for binary. This is why ```random_data=...``` is shorter than 5 MB. As @RenaudPacalet wrote, intermediary files need to be used instead. — Jay jargot, Jun 09 '22 at 12:08
Thanks @RenaudPacalet. That worked when I made the partition location a binary file. The partition is a RAM block, and for some reason, the newdata.bin file is all 0. It is like the writing did not execute properly for some reason. — Mickthekangaroo, Jun 09 '22 at 16:07
@Jayjargot Thanks, that makes sense and I made changes to my script according to what you have said, so I believe I am now a lot closer to getting a working solution. — Mickthekangaroo, Jun 09 '22 at 16:09

score 2 · Accepted Answer · answered Jun 11 '22 at 16:21

Here's a simpler approach which just saves the data in a temporary file.

#!/bin/sh

set -e

random_data=$(mktemp -t ideone.XXXXXXXX) || exit
trap 'rm -rf "$t"' EXIT

dd if=/dev/urandom bs=10240 count=512 of="$random_data"

dd if="$random_data" of=<partition_path> bs=1 seek=<position_in_partition> count=<5MB>

if dd if=<partition_path> bs=1 skip=<position_in_partition> count=<5MB> |
    cmp "$random_data"
then
 echo "data write successful"
fi

score 1 · Answer 2 · answered Jun 09 '22 at 15:12

Bash strings cannot hold arbitrary binary data because the ASCII NUL character is used as a string terminator.

One way to do what you want to do is to put the data in files instead of variables and use cmp to compare the files.

Another option is to store cryptographic hashes of the data in Bash variables. This Shellcheck-clean code demonstrates the idea:

#! /bin/bash -p

partition_path=testpart
position_in_partition=10

sha256_1=$( exec 3>&1
            head -c 5MB /dev/urandom                \
                | tee >(sha256sum >&3)              \
                | dd of="$partition_path" bs=1  \
                        seek="$position_in_partition" count=5MB)

sha256_2=$(dd if="$partition_path" bs=1 \
                skip="$position_in_partition" count=5MB \
            | sha256sum)

[[ $sha256_1 == "$sha256_2" ]] && echo 'data write successful'

You'll need to set the partition_path and position_in_partion variables to values that are appropriate for you.
exec 3>&1 connects file descriptor 3 to the stream that is used to read the value of sha256_1.
tee >(sha256sum >&3) uses the standard tee utility and Bash process substitution to copy the pipeline data as input to a sha256sum process whose output is redirected to file descriptor 3. The effect of this is that the sha256sum output (with the trailing newline removed) becomes the value of the sha256_1 variable.
You can use a stronger cryptographic hash function by replacing sha256sum with (for example) sha512sum.

Thanks for your answer, I will try the cryptographic hash approach. Would you mind having a look at the edit I made to my post where I modified my script for intermediary files? — Mickthekangaroo, Jun 09 '22 at 16:20
@Mickthekangaroo, the intermediary files code looks basically sound to me, but it's obviously not complete. I'd recommend creating some real code that actually works, using an ordinary disk file like 'test.dat', before trying it on your real partition. The code that I posted ran successfully and printed 'data write successful'. — pjh, Jun 09 '22 at 16:41

How to Write Block of Random Data to File using Bash

2 Answers2