Integrating LVM with Hadoop

LVM(Logical Volume Manager) is a piece of software, which helps resizing,scraping and changing Logical partition in Linux.

We will use LVM to dynamically increase/decrease hadoop slave node partition so that we don’t need to stop nodes for unmounting and static partitioning, which will waste our time.

Step 1 :- Create a Hard disk and connect with RedHat OS.

/dev/sdb contains the 10GiB, we created for dynamically resizing of slave node.

Step 2 :- Format The Partition

Create A Physical Volume to access it inside RHEL OS.

Create a Volume Group to differentiate it from other partitions.

Step 3:- Create A Logical Partition

Use lvcreate — size 2G — name task task_dev to create a logical volume of name “task”.

Format the logical Partition

Finally, Make a folder /dr, and mount the logical partition on the folder.

Step 4:- Start the hadoop cluster

Connect the DataNode with the cluster

Step 5 :- Use LVM to Increase the size

Use command lvextend — size +4G /dev/task_dev/task to extend the partition by 4GB.

Step 6 :- Use LVM to reduce the partition

Use the command lvreduce -L 4G /dev/task_dev/task to reduce the Logical Volume Size by 4GB.

So, we can Dynamically Increase and Decrease the size by using LVM software and even edit static partitions.