6

While working through the Scrapy tutorial here I noticed that Python scripts are not resulting in any output to the Windows command prompt.

I did check the path environment variable to confirm that "c:\python27" and "c:\python27\Scripts" are both included. And do I believe I successfully installed Scrapy and the required dependencies.

I was able to follow along with the tutorial right up to Trying Selectors in the Shell. Up to that point most things went as anticipated: the output files were created as expected; however, no console output appeared. According to the tutorial, I should have seen something like the following:

2014-01-23 18:13:07-0400 [scrapy] INFO: Scrapy started (bot: tutorial)
2014-01-23 18:13:07-0400 [scrapy] INFO: Optional features available: ...
...

Initially I did not notice there was no output to the console and moved on to trying to launch the shell as described in the tutorial:

...\tutorial\>scrapy shell "http://www.dmoz.org/Computers/Programming/Languages/Python/Books/"

...\tutorial\>

There is no output, but there is a pause before the prompt re-appears, so it is obviously thinking...

I searched with Google and here at SO; but had no luck finding a solution. Though, in the process of trying to diagnose this issue, I noticed that other scripts that should output to the console are not doing so (python.exe, however, does).

For example, as expected:

C:\>python --version

results in:

Python 2.7.6

On the other hand, I get nothing from pip:

C:\>pip --version

C:\>

Pip is installed. I can re-direct output to a text file: 

C:\>pip list 1>output.txt 2>&1    

C:\>

The resulting text file includes the following, as expected:

...
nose (1.3.4)
numpy (1.8.1)
pip (6.0.8)
...

I saved a script 'time.py' containing the following:

import datetime
print datetime.datetime.now()

And running it works fine:

...\Desktop>time.py
2015-02-28 21:23:18.088000

...\Desktop>

So now I am thoroughly confused. How can I get pip, Scrapy, etc. to output to windows command prompt? Thanks in advance!

Dave G
  • 163
  • 1
  • 5
  • What do you get for `where pip` and `where scrapy`? Are they EXE files in the Scripts directory? – Eryk Sun Mar 01 '15 at 04:54
  • hmm ... have you considered cygwin? or running a linux vm on your windows box with VirtualBox or VMplayer? – Red Cricket Mar 01 '15 at 04:55
  • @eryksun yes, they are both exe's in the scripts directory. Interesting - I had assumed these were both .py files; what's the implication of these being executables? – Dave G Mar 01 '15 at 08:03
  • @RedCricket - I have not used cygwin or a virtual machine before - i'll read about them tomorrow morning; thank you for the input – Dave G Mar 01 '15 at 08:13
  • Using an .exe enables running the script more generally, since it works directly via Win32 `CreateProcess`, whereas a .py script only runs indirectly via `ShellExecuteEx`. The .exe files are created by setuptools and pip, so you should try updating to the latest versions: `python -m pip install -U setuptools pip`. Then reinstall scrapy. – Eryk Sun Mar 01 '15 at 08:17
  • @RedCricket thanks; I updated/re-installed but it did not solve the issue – Dave G Mar 01 '15 at 18:40
  • oops - sorry! Thanks to @eryksun! – Dave G Mar 01 '15 at 19:07

2 Answers2

1

By the way, U said after you run "pip --version" nothing was displayed, I think this means you didn't install python correctly, pip should be installed automatically after you install Python. Just remind, make sure set the right Python path to your computer system path, set the "path" variable to the correct python istalled path.
In my opinion, you should uninstall python, and reinstall again.Any problem please let me know.

tolerious
  • 144
  • 1
  • 10
  • Yeah, nothing was displayed; however, I could redirect the output to a text file with '1>output.txt 2>&1'. The content of that file was 'pip 6.0.8 from C:\Python27\lib\site-packages (python 2.7)'. I agree that it looks like time for a full uninstall and reinstall. I did so this morning and the python fresh install works just fine. Thanks for your suggestions. – Dave G Mar 01 '15 at 19:04
0

I think you may need a virtual machine, take a look about Vagrant.

You can set up a whole python environment on Linux and use the Vagrant package on Windows.
It's an amazing tool.

tolerious
  • 144
  • 1
  • 10