Setup Hadoop in distributed mode

This tutorial explains How to Setup and configure Hadoop on Multiple machines, i.e. Installation of Hadoop in Distributed Mode. In the cluster setup there is one master and 2 slaves will be configured. During the deployment all the pre-requisites will be installed. Hadoop installation is done on Amazon cloud (AWS).
Follow following video tutorial for the installation and configuration of Hadoop 1 in distributed mode  (real cluster mode)on Amazon Cloud:

In this video following topics has been covered:
 - Installation and configuration of Hadoop 1.x or Cloudera CDH3Ux in Distributed mode (on multiple node cluster).
 - Launch 3 instances on AWS (Amazon Cloud), on which we will setup the real cluster. One instance will act as Master and rest all the instances will act as slaves.
 - Prerequisites for hadoop Installation.
   -- Installation of Java.
   -- Setup of password-less ssh.
 - Important configurations properties.
 - Setup Configuration in core-site.xml, hdfs-site.xml, map-red-site.xml.
 - Format name-node.
 - Start hadoop services: NameNode, DataNode, secondary-namenode, JobTracker, TaskTracker.
 - Setup environment variables for  Hadoop,
 - Submit Map-Reduce Job.

Installing YARN (Hadoop 2.4.0) on Ubuntu

Steps to install Hadoop 2.x release (Yarn) on single node cluster setup (pseudo-distributed mode)

Hadoop 2.x release involves many changes to Hadoop and MapReduce. The centralized JobTracker service is replaced with a ResourceManager that manages the resources in the cluster and an ApplicationManager that manages the application lifecycle. These architectural changes enable hadoop to scale to much larger clusters. For more details on architectural changes in Hadoop next-gen (a.k.a. Yarn), watch this video or visit this blog.

This post explains on installing Hadoop 2.x a.k.a. Yarn on a single-node cluster.

Prerequisites:

1. Java 7 installed
2. Dedicated user for hadoop (not mandatory)
3. SSH configured

Steps to install Hadoop 2.x:

1. Download tarball

You can download tarball for hadoop 2.x from http://apache.cs.utah.edu/hadoop/common/current2/ (hadoop-2.4.0.tar.gz ).

Extract it to a folder in your home directory say, $HOME/yarn.

$ cd $HOME/yarn (Optional)

$ sudo chown -R hduser:hadoop hadoop-2.4.0

    READ MORE >>