OOP workflow for data

Question

I've worked with the Python basics for some time and fall back to mysql for data analysis. Now I want to learn how to do OOP the Python way, but with all the reading about classes, objects and their attributes: I got lost on the way experimenting and am looking for directions.

I use Python module ciscoconfparse, reading all interfaces, and for each interface going through spreadsheets to filter and get more (supplier) data I need. As an example I can have the following data:

dict = {    
    'Customer'        :   'customer1',
    'supplier-con'    :   'id823985',
    'hostname'        :   'router01',
    'interface'       :   'gig0/1',
    'subinterface'    :   '101',
    'dot1q'           :   '111',
    'qinq'            :   '10101' 
}

Tree-wise to show the relations:
the keys would look like the example below without the values:

Customer 1 : customer1  
----supplier-con : id823985
-------hostname : router 1
----------interface : gi0/1
--------------subinterface : 101
--------------subinterface : 111

Customer 1 : customer1  
----supplier-con : id45223
-------hostname : router 5
----------interface : gi0/3
--------------subinterface : 107
--------------subinterface : 888

Customer 2 : customer2  
----supplier-con : id625544
-------hostname : router 2
----------interface : gi0/2
--------------subinterface : 202
--------------subinterface : 222

You can see the interface is used multiple times with more subinterfaces. This also counts for the hostname and could be for other details along the way.

In what kind of way should I be thinking of handling 50000 entries in memory? Or am I better of with a database?

I know how to create dict-in-dicts, but not how to make relations between each other using actual objects.

See [my answer](http://stackoverflow.com/a/15420716/355230) to the question [_What is the best data structure for storing a set of four (or more) values?_](http://stackoverflow.com/questions/15418386/what-is-the-best-data-structure-for-storing-a-set-of-four-or-more-values) which creates lookup tables for every unique value of every field in the database (if and when needed). — martineau, Jul 24 '16 at 01:48
Also note that a key can be associated with a value that is a container, such as the dict-in-dicts you mentioned, by as lists-in-dicts—so for example the key `'subinterfaces'` value could be a list of items, like `['101', '111']` or even a list of dictionaries and those dictionaries can be other elements in the data structure. Python does reference counting, so the data in the dictionaries doesn't actually get doubled every time it's referenced. — martineau, Jul 24 '16 at 01:58
Thank you Martineau, those are very good answers I can work with, especially your well written csv example. That's what I'm going to try. — Gaston, Jul 24 '16 at 02:06
You're welcome. Another useful thing to keep in mind is that one can effectively create persistent dictionaries by using the [shelve](https://docs.python.org/3/library/shelve.html#module-shelve) module to create "shelf" objects that save themselves on disk. — martineau, Jul 24 '16 at 02:17

score 1 · Answer 1 · answered Jul 24 '16 at 07:17

Object-orientation is about objects which are bundles of data and functions (or messages that the objects understand) that operate on the data. Instead of starting with plain data structures and dividing them, you should start to think which operations you want to do in your program. Then, cluster these into entities with clear, distinct responsibilities. The later will be your objects.

It could well be that the final design does not look similar to the data-grouping you presented and that you will not see the same flow as in a data-oriented approach. If this is a pro or con is open for discussion.

I have started to build first the class relations in UML. . The next easy phase could still be having all information in a list of dicts on which I run functions to create relations that I store in different dicts. But still I am wondering if I can manage to get information like
print customer1.all-interfaces
without running through a dicts, but purely fishing them through relations... Got to work on that part — Gaston, Jul 24 '16 at 08:22

OOP workflow for data

1 Answers1