3

I am a Microsoft Windows user and new to Apache Hadoop. Most of the Hadoop tutorials I found are Linux based so I am finding it difficult to learn Hadoop on Windows. Any pointers to learn Hadoop on Windows would be best.

  • possible duplicate of [Hadoop on windows server](http://stackoverflow.com/questions/467911/hadoop-on-windows-server) – Yuval F Apr 25 '12 at 06:56
  • I would install virtual machine with linux and run Hadoop on this virtual machine. Check: http://www.virtualbox.org and http://virtualboxes.org/images/ –  Apr 25 '12 at 06:56
  • I would say it's a good investment get in contantact with Linux anyway; buy a book about Linux/Ubuntu and start reading. – Raphael Bossek Apr 25 '12 at 07:00

7 Answers7

4

If your primary objective is to learn Hadoop then it does not matter you learn it on Windows or Linux, because everything is exactly same on both platforms. I have extensively used Hadoop on both platform and found all the commands and processing are identical on Windows and Linux. So here are my suggestions:

  1. Download VMware VMPlayer on your Windows Machine
  2. Download CDH Virtual Machine for VMware https://ccp.cloudera.com/display/SUPPORT/Downloads

Access virtual machine in your Windows box and follow the tutorials exactly they are on Linux.

AvkashChauhan
  • 20,495
  • 3
  • 34
  • 65
  • Started downloading virtual machine now and give a quick try. It is nice to have instant help. –  Apr 25 '12 at 07:08
  • Thanks to all and I have Cloudera Virtual Machine running fine in my Windows Box. –  Apr 26 '12 at 17:19
  • What if I want to do a production setup. Would it be alright to use a Hadoop cluster setup on Windows. – frewper Apr 26 '17 at 12:09
1

You can also try "Syncfusion BigData Studio" to run a single node cluster in your local machine along with required ecosystems installed with it; Also "Syncfusion BigData Platform" allows you to manage clusters in a much easy way without any manual configuration from user's end; These 2 setups are "100% free for everyone";

To download setup and know more, please refer: http://www.syncfusion.com/products/big-data

Dinesh Kumar P
  • 1,128
  • 2
  • 18
  • 32
0

You can also try Amazon Elastic MapReduce, this is more efficient if you don't have any linux experience.

Shumin Guo
  • 184
  • 1
  • 3
  • 11
0

I managed to port Hadoop-1.0.1 on windows-7, cygwin-1.7, jdk1.7_x64.

but it's not for beginners: you will need to patch and recompile hadoop.

http://sourceforge.net/p/win-hadoop/wiki/Hadoop-on-Cygwin/

FKorning
  • 166
  • 2
0

I use Hadoop natively on Windows as a virtual 2-node cluster running on one machine. It runs inside Cygwin (so no VM). Works well to try Hadoop out and I still use it to test new code in small before putting it on the cluster. You basically get every bit of functionality as with a full cluster. Getting it to work can be a bit tricky though.

I used the following short guide: Stanford Hadoop for Windows guide

Which worked fine for me. Very important is that you use 0.20.0! Higher version do not run under Cygwin. I think it is best to leave the number of default nodes to 2. This way you can test if splitting the work across multiple nodes works, but more simultaneous nodes can give you memory problems.

Jeroen Vuurens
  • 1,171
  • 1
  • 9
  • 10
0

With the latest release of Hadoop 2.2 I see that the release notes mentions that this version has significant improvements for running Hadoop on Windows. I downloaded Hadoop 2.2 yesterday and I saw lot of .cmd file alon with .sh files which ensures that this version has scripts and batch files for running Hadoop on Windows environment. However while looking at the Apache Hadoop documentation I couldn't find any step-by-step instructions on how to install and run this newer version on Windows. Besides this it looks like that the newer version has YARN architecture embedded in it and the old configurations provided on some of the tutorials online may be outdated and not applicable anymore. Is there any good documentation for Hadoop 2.2 available online ? I want it specifically for running Hadoop under Win

VIKAS UIKEY
  • 1
0

I compiled Hadoop on Windows and it's released as zetabyte's "Apache Hadoop for Windows". There is a core/common package available and also a package with a GNU environment (bash, etc.)

veganaiZe
  • 539
  • 5
  • 13