3

Source file is unable to upload data using "pd.read_csv" when in a certain directory, whether it's using a Python IDE (Pycharm) or a Jupyter notebook. If I create a source file outside of it, and point it to whatever data I want to upload (whether the data is in the dodgy directory or somewhere else), it works.

I'm a novice, so I'm sure what I could have changed to make source files in this directory behave this way.

I copied the directory, and tried the same in the copied directory and the behavior still stands.

But if I create a new folder, and copy the data files there, and create a source file and operate within the fresh new directory, it works.

This is the code that I was trying to run, and the dataframe came simply as 'not defined':

travel_df = pd.read_csv('/Users/Anon/Projects/Travel/FullData.csv')

travel_df.head()

which returns nothing, and:

if travel_df.empty:
    print('DataFrame is empty!')

which returns:

NameError                                 Traceback (most recent call last)
<ipython-input-2-29f39da7c7dc> in <module>()
----> 1 if travel_df.empty:
      2     print('DataFrame is empty!')

NameError: name 'travel_df' is not defined

I'm ok with this workaround solution, but I was wondering what about the original directory had been changed that causes this behavior so I can check for it/avoid it in the future.

exlo
  • 315
  • 1
  • 8
  • 20
  • 1
    Seems to me like it's a relative/absolute path issue. That said, the path you provided in your slightly "anonymised" code is an absolute path so I'm not sure why. Also, if you look at your error message, the names of the dataframes are mismatched. – absolutelydevastated Jul 08 '19 at 02:17
  • Edited. Thank you, pasted from two Jupyter notebooks from different troubleshooting attempts. I used absolute paths for troubleshooting and the problem remains so I'm stumped. – exlo Jul 08 '19 at 02:32
  • 1
    Can you post the location of your script and data for the two different configurations? No need for the actual path, just show the structure of your project setup, because it's not apparent what you did to fix the issue from your description. As in, I'm not sure if the problem arises when you have your script and data in different locations or when your script is in a certain directory and that directory only. – absolutelydevastated Jul 08 '19 at 03:02
  • In the `notebook` execute `%pwd` to see the current working directory. `%ls` will show the files in that directory. `pd.read_csv('FullData.csv')` should work for a file in that list. – hpaulj Jul 08 '19 at 05:25
  • @absolutelydevastated Of course, but to put it simply, it's when the script is in a certain directory and that directory only. Let's say the location of script is in the problem directory "travel" - no data, whether in "travel" dir or anywhere - gets read by the script. This is also true if I duplicate "travel" and try with a script in the duplicated directory. But if I make a new directory (like "travel new") or use another unrelated dir to house the script and point to *any* data, then the script is able to upload the data via `pd.read_csv`. – exlo Jul 08 '19 at 06:50
  • @hpaulj, the data is located in the current dir, but unable to be read. The pathway isn't the problem, the source file being located in the problematic dir is most likely is but I'm not sure why – exlo Jul 08 '19 at 06:53
  • 1
    @exlo The interesting thing is that your code fails on `.empty` instead of `.head()`. You'll need to relook at that notebook. Does this mean that the read is successful? Try doing something like `os.path.exists(csv_path)`. You might also want to check the permissions of the folder. In any case, I think the name error is a bug in your code since it should only occur if the variable was not defined previously. – absolutelydevastated Jul 08 '19 at 08:01
  • @absolutelydevastated I think you're right! I realize that the cell that the read code was in was in the wrong format ('Raw NBConvert' as opposed to 'Code'). I wouldn't have caught that unless you mentioned that it must be a mistake in the original notebook. Thank you for the help. – exlo Jul 09 '19 at 01:13

0 Answers0