3

I'm working on building a SQL emulator in Python and to store the rows, I'd like to use namedtuples since I can easily handle complex queries with select, order by, and where. I started with normal tuples but I often found myself looking for an attribute of a row and needing to maintain the order of the columns, so I arrived at namedtuples.

The issue is that some of my column names have leading underscores which causes me to end up with ValueError: Field names cannot start with an underscore: '_col2'

I'm looking for either a way to use namedtuples with underscores (maybe some type of override) or a suitable alternative container that allows me to easily convert to a tuple of values in original column order or to access individual values by their field names.

I thought about appending a leading character string to every tuple and then writing a middleware function to serve as the getattr function but by first removing the leading character string - but that seems incredibly hacky.

  • Have a look at [`pandas.Series`](https://pandas.pydata.org/pandas-docs/version/0.23.4/generated/pandas.Series.html). – a_guest Jan 25 '19 at 18:43
  • i think your situation is similar to alchemy, check their design considerations https://groups.google.com/forum/#!topic/sqlalchemy/E0uA91vODdI – Dyno Fu Jan 25 '19 at 18:44
  • I should have specified better - pandas Series and Keyed Tuples booth look great, but I'm unable to use any external libraries, only from the STL. –  Jan 25 '19 at 18:50
  • Just use custom classes. – juanpa.arrivillaga Jan 25 '19 at 19:17

3 Answers3

4

You can avoid ValueError using rename=True argument

from collections import namedtuple

a = namedtuple("Table", "_col1 _col2 _col3 col4", rename=True)

print(a._fields)

('_0', '_1', '_2', 'col4')

@Edit1 You might want to keep track of which fields have changed

from collections import namedtuple

columns = "_col1 _col2 _col3 col4"
a = namedtuple("Table", columns, rename=True)

old_feilds = columns.split()
new_feilds = a._fields

mapper = {}

for f1,f2 in zip(old_feilds, new_feilds):
    mapper[f1] = f2

print(mapper)

{'_col3': '_2', '_col1': '_0', 'col4': 'col4', '_col2': '_1'}

HariUserX
  • 1,341
  • 1
  • 9
  • 17
  • This worked great. Similar to what I had originally considered (appending a leading string) but this seems like a much cleaner way to go. –  Jan 25 '19 at 19:29
1

You could always use type:

obj = type('Foo', tuple(), {'_closed': False})()

Now I can access it:

obj._closed

Let's add a few utility functions:

from collections import deque
from itertools import islice


it_consumes = (lambda it, n=None: deque(it, maxlen=0) or None if n is None
               else next(islice(it, n, n), None))                                  

def update_d(d, **kwargs):
    d.update(kwargs)
    return d

Okay, now let's take an arbitrary tuple of t = 'a', 'b', 'c', '_can' and do this:

MyClass = type('MyClass', tuple(),
               update_d({k      : None for k in t},
                        __init__=lambda self, **kwargs: it_consumes(
                                     setattr(self, k, v) 
                                     for k,v in kwargs.items())))

Which can then be used like so:

obj = MyClass(a=5, b=6, _can='haz')
print('obj._can:', obj._can)
Samuel Marks
  • 1,611
  • 1
  • 20
  • 25
0

Consider using the OrderedDict type. You can access fields with any string name via bracket syntax, and convert it into a tuple eventually using ".items".

from collections import OrderedDict
ordered_form = OrderedDict([("col1", 'Apple'), ("col2", 'Orange'), ("_col3", 'Banana')])
ordered_form["_col3"] = 'Grapefruit'
tuple_form = tuple([i[1] for i in list(ordered_form.items())])
print(tuple_form)
Recaiden
  • 58
  • 1
  • 4
  • That's a great suggestion. I initially considered it but ended up not using it because I was concerned about the sorting. Right now I'm sorting my list of named tuples with `ordered = sorted(self.rows, key=attrgetter(*order)) if order else self.rows`. Can I still use attrgetter with an ordered dict or how would I need to change it? –  Jan 25 '19 at 19:05
  • It looks like you would want to use `itemgetter` in place of `attrgetter` – Recaiden Jan 25 '19 at 19:20