Using if statements to filter data?

Question

Lets say I have an excel document with the following format. I'm reading said excel doc with pandas and plotting data using matplotlib and numpy. Everything is great!

Buttttt..... I wan't more constraints. Now I want to constrain my data so that I can sort for only specific zenith angles and azimuth angles. More specifically: I only want zenith when it is between 30 and 90, and I only want azimuth when it is between 30 and 330

Air Quality Data
Azimuth Zenith    Ozone Amount
230    50         12   
0      81         10    
70     35         7
110    90         17
270    45         23
330    45         13
345    47         6
175    82         7
220    7          8

This is an example of the sort of constraint I'm looking for.

 Air Quality Data
Azimuth Zenith    Ozone Amount
230    50         12   
70     35         7
110    90         17
270    45         23
330    45         13
175    82         7

The following is my code:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import datetime

P_file = file1
out_file = file2
out_file2 = file3

data = pd.read_csv(file1,header=None,sep=' ')
df=pd.DataFrame(data=data)
df.to_csv(file2,sep=',',header = [19 headers. The three  that matter for this question are 'DateTime', 'Zenith', 'Azimuth', and 'Ozone Amount'.]
df=pd.read_csv(file2,header='infer')
mask = df[df['DateTime'].str.contains('20141201')] ## In this line I'm sorting for anything containing the locator for the given day.
mask.to_csv(file2) ##I'm now updating file 2 so that it only has the data I want sorted for.
data2 = pd.read_csv(file2,header='infer')
df2=pd.DataFrame(data=data2)

def tojuliandate(date):
   return.... ##give a function that changes normal date of format %Y%m%dT%H%M%SZ to julian date format of %y%j
def timeofday(date):
    changes %Y%m%dT%H%M%SZ to %H%M%S for more narrow views of data

df2['Time of Day'] = df2['DateTime'].apply(timeofday)
df2.to_csv(file2) ##adds a column for "timeofday" to the file

So basically at this point this is all the code that goes into making the csv I want to sort. How would I go about sorting

'Zenith' and 'Azimuth'

If they met the criteria I specified above?

I know that I will need if statements to do this. I tried something like this but it didn't work and I was looking for a bit of help:

if you want constraint why don't you do on the load level...like data[data['Zenith'] > 30 and data['Zenith'] <90].. similarly for Azimuth — iamklaus, Oct 11 '18 at 15:04

score 3 · Answer 1 · answered Oct 11 '18 at 15:03

You can use series between:

df[(df['Zenith'].between(30, 90)) & (df['Azimuth'].between(30, 330))]

Yields:

   Azimuth  Zenith  Ozone Amount
0      230      50            12
2       70      35             7
3      110      90            17
4      270      45            23
5      330      45            13
7      175      82             7

Note that by default, these upper and lower bounds are inclusive (inclusive=True).

score 3 · Accepted Answer · answered Oct 11 '18 at 15:36

3

df[(df["Zenith"]>30) & (df["Zenith"]<90) & (df["Azimuth"]>30) & (df["Azimuth"]<330)]

Basically a duplicate of Efficient way to apply multiple filters to pandas DataFrame or Series

answered Oct 11 '18 at 15:36

Ma.Na

59
3

please explain your answer briefly so people know what the code is doing. – Adam B Oct 11 '18 at 15:48
Thanks Ma. Na! That quick edit to my code made it perfect! – haramassive Oct 11 '18 at 17:09

score 2 · Answer 3 · answered Oct 11 '18 at 15:05

You can only write those entries of the dataframe to your file, which are meeting your boundary conditions

# replace the line df.to_csv(...) in your example with
df[((df['Zenith'] >= 3) & (df['Zenith'] <= 90)) and 
   ((df['Azimuth'] >= 30) & (df['Azimuth'] <= 330))].to_csv('my_csv.csv')

score 2 · Answer 4 · answered Oct 11 '18 at 15:17

Using pd.DataFrame.query:

df_new = df.query('30 <= Zenith <= 90 and 30 <= Azimuth <= 330')

print(df_new)

   Azimuth  Zenith  OzoneAmount
0      230      50           12
2       70      35            7
3      110      90           17
4      270      45           23
5      330      45           13
7      175      82            7

Using if statements to filter data?

4 Answers4