0

I am new to python. I have a python script that reads a csv file with data from a camera. The csv file contains a row of data for each event captured. The rows change dynamically with the 'object' field growing as more objects are detected in each row instance. Each object is assigned an 'iD'. I would like to load all these rows into a database like pandas and then be able to graphically display data by using a query on the data.

My python code in streamlit so far

if option == 'Subscribed Data':
        st.subheader("Latest Subscribed Data from csv.")
        pass
        data_load_state = st.text('Loading data...')

        df = pd.read_csv('my_file.csv', delimiter=',')
        list_of_rows = [list(row) for row in df.values]       
        data_load_state.text("Collected the following data.")
        st.json(list_of_rows)

One line of the csv file.

{'dataNumber': '109233', 'frameCounter': '218530', 'messageType': 'Data', 'object': 
[{'center': [{'latitude': '0', 'longitude': '0', 'x': '-1.21', 'y': '0', 'z': '23.2'}], 
'class': 'CAR', 'height': '1.5', 'iD': '728', 'leftBack': [{'latitude': '-36.753566', 
'longitude': '174.70354', 'x': '-2.3', 'y': '0', 'z': '25.6'}], 'leftFront': [{'latitude': 
'-36.753611', 'longitude': '174.70354', 'x': '-2.12', 'y': '0', 'z': '20.6'}], 'length': '5', 
'rightBack': [{'latitude': '-36.753566', 'longitude': '174.70356', 'x': '-0.304', 'y': '0',0 
'z': '25.7'}], 'rightFront': [{'latitude': '-36.753611', 'longitude': '174.70356', 'x': 
'-0.125', 'y': '0', 'z': '20.7'}], 'speed': '0.000486', 'width': '2', 'yaw': '1.61'}, 
{'center': [{'latitude': '0', 'longitude': '0', 'x': '-9.26', 'y': '0', 'z': '23.2'}], 
'class': 'CAR', 'height': '1.5', 'iD': '821', 'leftBack': [{'latitude': '-36.753559', 
'longitude': '174.70345', 'x': '-10.1', 'y': '0', 'z': '25.8'}], 'leftFront': [{'latitude': 
'-36.753604', 'longitude': '174.70345', 'x': '-10.4', 'y': '0', 'z': '20.8'}], 'length': '5', 
'rightBack': [{'latitude': '-36.753562', 'longitude': '174.70348', 'x': '-8.07', 'y': '0', 
'z': '25.7'}], 'rightFront': [{'latitude': '-36.753606', 'longitude': '174.70347', 'x': 
'-8.45', 'y': '0', 'z': '20.7'}], 'speed': '33.4', 'width': '2', 'yaw': '1.49'}], 'time': 
'2022-03-07T15:31:36.320+13:00', 'type': 'WorldData'}

Here is the output I get in Streamlit app.

    [0:[
    0:"{'dataNumber': '109197'"
    1:" 'frameCounter': '218460'"
    2:" 'messageType': 'Data'"
    3:" 'object': [{'center': [{'latitude': '0'"
    4:" 'longitude': '0'"
    5:" 'x': '-1.21'"
    6:" 'y': '0'"
    7:" 'z': '23.2'}]"
    8:" 'class': 'CAR'"
    9:" 'height': '1.5'"
    10:" 'iD': '728'"
    11:" 'leftBack': [{'latitude': '-36.753566'"
    12:" 'longitude': '174.70354'"
    13:" 'x': '-2.3'"
    14:" 'y': '0'"
    15:" 'z': '25.6'}]"
    16:" 'leftFront': [{'latitude': '-36.753611'"
    17:" 'longitude': '174.70354'"
    18:" 'x': '-2.12'"
    19:" 'y': '0'"
    20:" 'z': '20.6'}]"
    21:" 'length': '5'"
    22:" 'rightBack': [{'latitude': '-36.753566'"
    23:" 'longitude': '174.70356'"
    24:" 'x': '-0.304'"
    25:" 'y': '0'"
    26:" 'z': '25.7'}]"
    27:" 'rightFront': [{'latitude': '-36.753611'"
    28:" 'longitude': '174.70356'"
    29:" 'x': '-0.125'"
    30:" 'y': '0'"
    31:" 'z': '20.7'}]"
    32:" 'speed': '0.000349'"
    33:" 'width': '2'"
    34:" 'yaw': '1.61'}"
    35:" {'center': [{'latitude': '0'"
    36:" 'longitude': '0'"
    37:" 'x': '-11.4'"
    38:" 'y': '0'"
    39:" 'z': '12.3'}]"
    40:" 'class': 'CAR'"
    41:" 'height': '1.5'"
    42:" 'iD': '821'"
    43:" 'leftBack': [{'latitude': '-36.753656'"
    44:" 'longitude': '174.70342'"
    45:" 'x': '-12.2'"
    46:" 'y': '0'"
    47:" 'z': '14.9'}]"
    48:" 'leftFront': [{'latitude': '-36.7537'"
    49:" 'longitude': '174.70341'"
    50:" 'x': '-12.5'"
    51:" 'y': '0'"
    52:" 'z': '9.9'}]"
    53:" 'length': '5'"
    54:" 'rightBack': [{'latitude': '-36.753658'"
    55:" 'longitude': '174.70344'"
    56:" 'x': '-10.2'"
    57:" 'y': '0'"
    58:" 'z': '14.8'}]"
    59:" 'rightFront': [{'latitude': '-36.753703'"
    60:" 'longitude': '174.70344'"
    61:" 'x': '-10.5'"
    62:" 'y': '0'"
    63:" 'z': '9.8'}]"
    64:" 'speed': '39.1'"
    65:" 'width': '2'"
    66:" 'yaw': '1.52'}]"
    67:" 'time': '2022-03-07T15:31:35.150+13:00'"
    68:" 'type': 'WorldData'}"]
Francoiss
  • 11
  • 3
  • Please simplify your question. – ferdy Mar 08 '22 at 02:34
  • Question is now simplified. – Francoiss Mar 08 '22 at 18:46
  • By `df = pd.read_csv('my_file.csv', delimiter=',')` you now have a pandas dataframe. So this step is now solved. Next you want to display the data graphically. Have you researched about it? What kind of graphics do you want? – ferdy Mar 09 '22 at 00:09
  • Thanks for the reply. I have been looking at Ag-Grid. This gives me the tables and queries I want, but the columns are not displaying the correct data as the 'object' key in my data row grows dynamically as more objects are detected. this causes the data to shift in columns. Is there a better way of formatting the data first? I would also like to display the track GPS info on a scatter graph, or map, to create a visual representation. – Francoiss Mar 09 '22 at 19:12
  • It seems you have an issue with your data. First check your csv file. Once it is fixed go to the next step. – ferdy Mar 10 '22 at 00:02
  • I have tried to recursively flatten the json file to include the nested lists, but I can't seem to get it to do just that. The nested data is then displayed within the column. – Francoiss Mar 15 '22 at 23:50

0 Answers0