Simple MapReduce Program for Hadoop

Overview

Previous post we went through basics of big data processing and Hadoop. This post I describe some basic map reduce program structure. Here I have used example of counting words in large number of books. I have configured a two node cluster with distributed file system. Here we are using three classes as,

1. Mapper class 
2. Reducer Class 
3. Main words Count Class 

Assume I have 10 documents having only words “hello” and “world”. Documents will be shared by two nodes. How much documents for each node will be decide by Hadoop framework.

Pacemaker, DRBD Active-Passive Cluster

Overview

Active Passive cluster consists of two nodes: active node and passive node. Active passive cluster ensures constant service of a certain application keeping redundant nodes to back up system in the event of a failure. When the active node goes down one of the backup nodes takes the responsibility and runs those services. In the process of fail over, the services are restarting on another node immediately without administration intervention.

Big Data processing with Apache Hadoop

Overview

Earlier software applications have been developed to run on single computer. Some examples are calculators, word processing packages, drawing applications etc. With the introduction of client server architecture vast amount of software systems were developed with databases. Most of web applications, business systems are built databases and concurrency, transaction handling are some new terms introduced with the architecture. Now world has moved to new computer era with concepts of high performance computing.

Apache Load balancing with HAProxy

Overview

HAProxy is an open source load balancing software run on Linux. It will divide the load which may be web page or database query requests upon slave servers from the master. Therefore usual Linux machine can be configured as a load balance without any hardware load balancing capabilities.

What is Computer Cluster

Computer cluster is a collection of computers connected as a grid to perform special tasks. Rather than using one high performed computer, collection of computers so called cluster provides considerable advantages through high availability and performance. Computer cluster can be specified according to different criteria. Here is a summary of classification of clusters.