Why are the result files not being generated for each counter in the new directory?

Question

I am attempting to use the Linux perf tool to monitor counters. I am measuring counters in intervals of 500 milliseconds over a span of 10 minutes. I am trying to output the results of each counter into their own respective output files in a new directory. The new directory is created but I am only getting a hidden .txt file within it. I would like for files to be generated for each counter and their respective results to populate those files.

Here is the script:

     #!/bin/bash

     # Set the duration (in seconds) for monitoring
     duration=$((10 * 60))

     # Create an array of counters to monitor
     counters=(
               "cpu/event=0x00,umask=0x01,name=branch-instructions/"
               "cpu/event=0x00,umask=0x02,name=branch-misses/"
               "cpu/event=0x00,umask=0x0e,name=bus-cycles/"
               "cpu/event=0x00,umask=0x08,name=cache-misses/"
               "cpu/event=0x00,umask=0x00,name=instructions/"
               "msr/aperf/"
               "msr/mperf/"
               "msr/pperf/"
               "msr/smi/"
               "msr/tsc/"
               "power/energy-cores/"
               "power/energy-pkg/"
               "power/energy-ram/"
    )

    # Get the start time
    start_time=$(date +%s)

    # Create a directory to store the output files
    output_dir="perf_results"
    mkdir -p "$output_dir"

    # Iterate through each counter and monitor them
    for counter in "${counters[@]}"; do
    # Extract the counter name for file storage
         counter_name=$(echo "$counter" | awk -F"/" '{print $NF}')

         # Create an output file for the counter
         output_file="${output_dir}/${counter_name}.txt"

         # Run perf command to monitor the counter at each interval and append to the file
         run=1
         while true; do
              measurement=$(perf stat -a -e "$counter" -I 500 sleep 0.5 2>&1)
              {
                   echo "Run: $run"
                   echo "Time: $(date)"
                   echo "Counter: $counter_name"
                   echo "Measurement: $measurement"
                   echo
              } >> "$output_file"

              # Check if the duration has elapsed
              current_time=$(date +%s)
              elapsed_time=$((current_time - start_time))
              if [ "$elapsed_time" -ge "$duration" ]; then
                  break
              fi

              ((run++))
         done &
     done

    # Wait for the monitoring to complete
    wait

    # Display a message indicating the monitoring is finished
    echo "Monitoring completed."

The output directory is created properly, but the files for each counter are not generated at all. Can someone assist with where I am going wrong?

I attempted to modify the script for just one counter and one file, but I ended up with the same result. A directory is generated, but with no result file inside.

Viewing the hidden .txt file that is generated, I see that all of the results are there. What I want is all of the results for each counter to be placed into their own .txt file. For example, branch-instructions would have a file in the output directory called branch-instructions.txt and so on.

`$counter_name` is empty because of the trailing `/` in `$counter` .. you have to change `$NF` in `awk -F"/" '{print $NF}'` or removing trailing `/` from `$counter` — ufopilot, May 22 '23 at 20:51
Thank you for your response! From my understanding, the trailing / is already removed? I'm imagining this: cpu/event=0x00,umask=0x01,name=batch-instructions/ will become this: batch-instructions — Tre' Jeter, May 22 '23 at 20:58
Already removed by what? You didn't take any steps to remove it before passing it to `awk`. — tjm3772, May 22 '23 at 21:02
when you pass `msr/aperf/` to `awk`, and the input field delimiter is defined as `/`, `awk` sees 3 fields .... `msr`, `aperf` and `''` (blank == nothing after last `/`), so when you `print $NF` you're doing `print $3` which becomes `print ''`; this is going to happen for each entry in the array; I would expect a single output file named `.txt` — markp-fuso, May 22 '23 at 21:33
there's nothing in your code that comes anywhere close to parsing an input of `cpu/event=0x00,umask=0x01,name=batch-instructions/` to generate `batch-instructions`; the `awk` code parses by `/` (see previous comment) but other than that there's nothing that parses by `=` (or any other delimiter other than `/`) — markp-fuso, May 22 '23 at 21:34
for each `$counter` value please show the expected `$counter_name` — markp-fuso, May 22 '23 at 21:38
Thanks @tjm3772 the $(NF-1) got what I was originally expecting. — Tre' Jeter, May 22 '23 at 21:43
The counter names are expected to be branch-instructions, branch-misses, bus-cycles, cache-misses, instructions, msr/aperf, etc @markp-fuso — Tre' Jeter, May 22 '23 at 21:44

score 0 · Answer 1 · answered May 23 '23 at 03:10

Modify your code like this (I only show the modified part, integrate the rest of your code in this):

#!/bin/bash

counters=(
    "cpu/event=0x00,umask=0x01,name=branch-instructions/"
    "cpu/event=0x00,umask=0x02,name=branch-misses/"
    "cpu/event=0x00,umask=0x0e,name=bus-cycles/"
    "cpu/event=0x00,umask=0x08,name=cache-misses/"
    "cpu/event=0x00,umask=0x00,name=instructions/"
    "msr/aperf/"
    "msr/mperf/"
    "msr/pperf/"
    "msr/smi/"
    "msr/tsc/"
    "power/energy-cores/"
    "power/energy-pkg/"
    "power/energy-ram/"
)

for counter in "${counters[@]}"
do
    counter_name=$(echo "$counter" | awk -F"[/=]" '{print $(NF-1)}')

    echo "$counter_name"
done

The difference is in the awk like that sets the value of counter_name.
-F"[-=]": tells awk to use both / and = as deliminters.
like you already found out, $(NF-1) gives you the second to last delimited value.

The output of my code above is:

$ ./so.bash 
branch-instructions
branch-misses
bus-cycles
cache-misses
instructions
aperf
mperf
pperf
smi
tsc
energy-cores
energy-pkg
energy-ram

score 0 · Answer 2 · answered May 23 '23 at 04:18

just skip the shell for … loop entirely and do it in one-shot :

printf '%s\n' "${counters[@]}" |

{m,n}awk '$!NF = $--NF' FS='/|=' 

gawk -F'=|/' '$_=$--NF'

branch-instructions
branch-misses
bus-cycles
cache-misses
instructions
aperf
mperf
pperf
smi
tsc
energy-cores
energy-pkg
energy-ram

Why are the result files not being generated for each counter in the new directory?

2 Answers2