1

I am trying to do some data analysis on Pythonista 3 (iOS app for python), however because of the C libraries of pandas it does not compile in the iOS device.

Is there any substitute for Pandas? Would numpy be an option for data of type string?

The data set I have at the moment is the history of messages between my friends and I.

The whole history is in one csv file. Each row has the columns 'day_of_the_week', 'date', 'time_of_message', 'author_of_message', 'message_body'

The goal of the analysis is to produce a report of our chat for the past year.

I want be able to count number of messages each friend sent. I want to be able to plot a histogram of the hours in which the messages where sent by each friend. Then, I want to do some word counting individually and as a group.

In Pandas I know how to do that. For example:

df = read_csv("messages.csv")
number_of_messages_friend1 = len(df[df.author_of_message == 'friend1']

How can I filter a csv file without Pandas?

zeh
  • 1,197
  • 2
  • 14
  • 29
  • I don't know the exact limitations for Pythonista, but numpy has similar requirements to pandas - so it's not very likely to work if pandas doesn't. Adding some details about what you want to do may help give more suitable results – Ophir Yoktan Nov 20 '16 at 10:58
  • I thought the same about numpy and Pandas. But that's what pythonista offers. I think the developer of the app compiled numpy in python to enable this package. Anyway, I edited the question to make my problem clearer. – zeh Nov 20 '16 at 11:15
  • If numpy does not work you could use standard library tools to read the csv into lists of 'records' (namedtuple can be your friend here) but this will entail a lot of for-loops when you want to query things. I don't know whether a lightweight dbms as sqlite can remedy this. – Maarten Fabré Nov 20 '16 at 13:20
  • If numpy works than *tabel* should work as well, as its based on numpy. https://pypi.org/project/tabel/ – Bastiaan Sep 13 '18 at 05:19

3 Answers3

3

Since Pythonista does have numpy, you will want to look at recarrays, which are numpy's approach to this type of problem. The following worked out of the box in Pythonista for me:

import numpy as np
df=np.recfromcsv('messages.csv')
len(df[df.author_of_message==b'friend1'])

Depending on your data format, tou may find that recsfromcsv "just works", since it tries to guess data types, or you might need to customize things a bit. See genfromtext for a number of options, such as explictly specifying data types or for using converters for converting string dates to datetime objects. recsfromcsv is just a convienece wrapper around genfromtext

https://docs.scipy.org/doc/numpy/user/basics.io.genfromtxt.html#

Once in recarray, many of the simple indexing operations work the same as in pandas. Note you may need to do string compares using b-prefixed strings (bytes objects), unless you convert to unicode strings, as shown above.

JonB
  • 350
  • 1
  • 7
0

Use the csv module from the standard library to read the messages. You could store it into a list of collections.namedtuple for easy access.

import csv

messages = []
with open('messages.csv') as csvfile:
    reader = csv.DictReader(csvfile, fieldnames=('day_of_the_week', 'date', 'time_of_message', 'author_of_message', 'message_body'))
    for row in reader:
        messages.append(row)

That gives you all the messages as a list of dictionaries.

Alternatively you could use a normal csv reader combined with a collections.namedtuple to make a list of named tuples, which are slightly easier to access.

import csv
from collections import namedtuple

Msg = namedtuple('Msg', ('day_of_the_week', 'date', 'time_of_message', 'author_of_message', 'message_body'))

messages = []
with open('messages.csv') as csvfile:
    msgreader = csv.reader(csvfile)
    for row in msgreader:
        messages.append(Msg(*row))
Roland Smith
  • 42,427
  • 3
  • 64
  • 94
0

Pythonista now has competition on iOS. The pyto app provides python 3.8 with pandas. https://apps.apple.com/us/app/pyto-python-3-8

elwarren
  • 327
  • 4
  • 8