Storing (or fetching) multiple values at once from Riak with Python and Flask

Question

I am trying to write a small app that gets visitor IP and stores in Riak database. On its /show route it should list all visitors, that is, stored IP's and associated dates. However, for some reason, I am either storing only last visitor, or fetching only last one. What am I doing wrong here? Im sure its not only wrong code issue, but also wrong way - I am trying to switch from die-hard sql user and I cant wrap my head about not having 'select * from ...' available ;)

import riak
import json
from flask import Flask, request, render_template
from datetime import datetime

app = Flask(__name__)
riak = riak.RiakClient(host='some.domain', port=8087, transport_class=riak.RiakPbcTransport)


def date(obj):
    return obj.isoformat() if hasattr(obj, 'isoformat') else obj


@app.route('/')
@app.route('/index')
def index():
    db = riak.bucket('visitor')
    ip = request.remote_addr
    time = json.dumps(datetime.now(), default=date)
    new_visitor = db.new('log', data={
        'ip': ip,
        'time': time,
    })
    new_visitor.store()
    return ip


@app.route('/show')
def show():
    db = riak.bucket('visitor')
    visitors = db.get('log')
    return render_template("show.html", visitors=visitors.get_data())


if __name__ == '__main__':
    app.run(debug=True, host='0.0.0.0')

score 0 · Answer 1 · answered Jan 12 '13 at 17:23

0

You're storing everything under the same name 'log' so you're overwriting the value every time you get a new request.

Riak bucket's are like python dict's and you're basically doing visitors['log'] = {...data...} on each request. You might want to use a different data structure for that or maybe use the time as a key if it's discrete enough.

answered Jan 12 '13 at 17:23

Oin

6,951
2
31
55

Um... then how should I store them? If if'd give them unique name each time, then I wouldnt know what to fetch from db, since its nosql and there is no 'select * from' available. – SpankMe Jan 12 '13 at 17:25
As with a python dict you can list all keys in a bucket: http://docs.basho.com/riak/latest/references/apis/http/HTTP-List-Keys/ – Oin Jan 12 '13 at 17:32
That doc has huge "Not for production - This operation requires traversing all keys stored in the cluster and should not be used in production." warning. I assume, this is not the way to do it. – SpankMe Jan 12 '13 at 17:33
So then either use a map reduce on the date or see this blog post on using secondary indexes: http://www.paperplanes.de/2011/12/13/list-all-of-the-riak-keys.html – Oin Jan 12 '13 at 17:47
Using map reduce is, as I understand it now, way to fetch the keys. Would that work in the way I store them now, or is there a better (universally, perhaps) way to do it? – SpankMe Jan 12 '13 at 17:56
It depends on what you want to accomplish. If you want to fetch only a subset (e.g. visits on a specific day) then Map Reduce is the way to go; the database will do the filtering for you (and you can use your existing "schema"). If you want to get all the keys, then it makes no sense to use Map Reduce or Riak for that matter... – Oin Jan 12 '13 at 18:25
The thing is, I have to use riak, because I have no other choice - in the project I am working on the db persistence and high availability is the highest priority, and neither mysql nor postgresql gets close to riak when it comes to clustering that's not networking and OS based (various combinations of heartbeat, drbd, replication). Therefore, I have to use a db that can cluster and replicate itself on its own - and that's only nosql riak. – SpankMe Jan 12 '13 at 18:28

Storing (or fetching) multiple values at once from Riak with Python and Flask

1 Answers1