0

Has anyone an idea how to read this table and parse out to CSV file or something. I need to convert it into graph(if possible).

I really confused how to read this table, not like 'show inventory' or 'show version' there's have a TextFSM Template to parse out the data, how about 'show proc cpu history' there's have TextFSM Template or anything?

show proc cpu history(from show tech.txt) enter image description here

wp78de
  • 18,207
  • 7
  • 43
  • 71
Nedy Suprianto
  • 201
  • 1
  • 6
  • 14

1 Answers1

0

You need to read the text file in and extract the 3 lines above each graph containing the numerical values. These need to be "rotated" and converted into integers.

The following code would read the text file in and extract the necessary lines. It first skips over all of the initial lines until it finds the header line for the three graphs.

It rotates the values (using *zip(*data)). Next it writes the data out to three separate CSV files, based on the source filenames.

Lastly, it uses the data to display three graphs using the Python matplotlib library (which would need to be installed).

from itertools import dropwhile, takewhile, izip_longest
import matplotlib.pyplot as plt
import csv
import re
import os
from collections import deque

def process_block(block):
    # Determine number of leading spaces (varies between files)
    start_offset = min(len(block[0]) - len(block[0].lstrip()), len(block[1]) - len(block[1].lstrip()))
    # Extract just data. Skip over leading spaces, remove trailing newline and replace any spaces with 0
    block = [line[start_offset:].rstrip('\n').replace(' ', '0') for line in block]
    # Convert to values
    return [int(''.join(entry)) for entry in zip(*block)]


def process_switch_file(figure, switch_filename):

    with open(switch_filename) as f_input:
        # Extract just the graph lines
        lines = list(takewhile(lambda x: "- show logging -" not in x, dropwhile(lambda x: "- show process cpu history -" not in x, f_input)))

    headings = ["CPU% per second (last 60 seconds)", "CPU% per minute (last 60 minutes)", "CPU% per hour (last 72 hours)"]

    # Keep reading each until a line starting 100 is found, keep the previous 2 lines
    data = []
    entry = iter(lines)

    while True:
        block = deque(takewhile(lambda x: not re.match(' *?100 ', x), entry), 2)

        if len(block[0]) <= 1:
            break
        data.append(block)

    data = [process_block(block) for block in data]
    x_values = [range(0, len(block)) for block in data]

    # Get the base filename (without path or extension)
    csv_filename = os.path.splitext(os.path.basename(switch_filename))[0]

    # Write data to a csv
    with open('{}_cpu_seconds.csv'.format(csv_filename), 'wb') as f_output:
        csv_output = csv.writer(f_output)
        csv_output.writerow(['Second', 'Value'])
        csv_output.writerows(list(zip(x_values[0], data[0])))

    with open('{}_cpu_minutes.csv'.format(csv_filename), 'wb') as f_output:
        csv_output = csv.writer(f_output)
        csv_output.writerow(['Minute', 'Value'])
        csv_output.writerows(list(zip(x_values[1], data[1])))

    with open('{}_cpu_hours.csv'.format(csv_filename), 'wb') as f_output:
        csv_output = csv.writer(f_output)
        csv_output.writerow(['Hour', 'Value'])
        csv_output.writerows(list(zip(x_values[2], data[2])))

    fig = plt.figure(figure)
    fig.canvas.set_window_title(csv_filename)

    for subplot, x, block, heading in zip(range(311, 314), x_values, data, headings):
        # Display as graphs
        ax = fig.add_subplot(subplot)
        ax.set_xlabel(heading)
        ax.set_ylim(0, 100)
        plt.plot(x, block)

    plt.tight_layout()


for figure, filename in enumerate(['switch-2.txt', "switch-3.txt", "Switch.txt"], start=1):
    process_switch_file(figure, filename)

plt.show()

The CSV files would start something like:

Minute,Value
0,0
1,0
2,0
3,0
4,3
5,11

And the display would show:

matplotlib graphs

matplotlib can usually be installed using:

pip install matplotlib
Martin Evans
  • 45,791
  • 17
  • 81
  • 97
  • i got error when i try u'r code : data2 = [int(''.join(entry)) for entry in zip(*data2)] ValueError: invalid literal for int() with base 10: 'SSS' – Nedy Suprianto May 19 '17 at 03:01
  • Your file obviously contained a lot of other information, so the script needed to be updated to skip over this until the graphs were found. I have updated it to work with the file you provided. – Martin Evans May 19 '17 at 07:12
  • can this code will work with the other 'show tech' files or only for this file? – Nedy Suprianto May 22 '17 at 02:30
  • hmm.. actually i worked with much files (show tech) in switch/router, and parse out the cli output and create a graph from the files like 'sh memory statistic','sh flash', 'sh process cpu history', etc. – Nedy Suprianto May 22 '17 at 03:41
  • Example of my files https://drive.google.com/open?id=0B_jl0iXmYwS_YzBmZG1NcldoOGs – Nedy Suprianto May 22 '17 at 03:59
  • I see the problem, each file has a slightly different format. I have updated the script to cope with your 3 different files. – Martin Evans May 22 '17 at 20:41
  • yap, that's what i mean, i worked with much files(around 50 files) and diferrent switch/router, so i need a code that can be use for all files like a template. – Nedy Suprianto May 23 '17 at 02:51
  • I don't think TextFSM could be used in this case. This appears to use simple RegEx to extract single entries. The data here is vertical, and not delimited well. Which file does the current script fail on? – Martin Evans May 23 '17 at 09:23
  • hhmmm... you're script is worked, i have >50 files show tech(each company), so i need to change the script manually if not maybe this code doesn't work right? – Nedy Suprianto May 24 '17 at 02:21