Hadoop ecosystem contain of Hadoop Distributed File System-HDFS and HDFS components, MapReduce, YARN, Hive, Apache Pig, Apache HBase and HBase components, HCatalog, Avro, Thrift, Drill, Apache mahout, Sqoop, Apache Flume, Ambari, Zookeeper and Apache OOzie. YARN is the basic prerequisite for Enterprise Hadoop Infrastructure, providing resource management and a central platform to offer consistent processes, security, and data domination tools across all clusters. It also provides new tools found within the data center so that they can take benefit of cost effective, linear-scale storage and processing. YARN allows different data processing engines containing graph processing, interactive processing, stream processing and batch processing to run, track and process data stored in HDFS.
YARN offers support for additional processing models by implementing a flexible execution engine.
• YARN improves the performance of the Hadoop compute cluster.
• YARN resource manager emphases completely on scheduling making it easy to manage large Hadoop clusters.
Yarn Architecture
Cluster utilization
YARN’s dynamic sharing of cluster resources progresses utilization over more static MapReduce rules used in initial versions of Hadoop
Reservation System
Hadoop YARN also comprises a Reservation System feature that lets users stand-in cluster resources in advance for significant processing jobs to guarantee they run smoothly. To escape overloading a cluster with reservations, administrators can limit the amount of resources that can be reserved by individual users and can set automated policies to discard reservation requests that surpass the limits.
Multi-tenancy
YARN permits multiple access engines (either open-source or proprietary) to practice Hadoop as the common standard for batch, interactive and real-time engines that can concurrently admission the same data set.
ResourceManager
ResourceManager (RM) is the master that judges all the available cluster resources and thus aids to manage the distributed applications running on the YARN.
ApplicationMaster
It manages the life-cycle of a job by guiding the NodeManager to make or terminate a container for a job. There is only one ApplicationMaster for a job.
NodeManagers
NodeManagers yield orders from the ResourceManager and manage resources accessible on a single node.
Scalability
YARN helps to significantly improve data center processing power. YARN’s ResourceManager efforts exclusively on scheduling and retains pace as clusters expand to thousands of nodes managing petabytes of data.
For details see Official web site of Hadoop here
Nub8 Hadoop Consulting Services
Nub8 can work with you to start a big data strategy that realizes specific business goals and objectives. Nub8 consultants can help you plan, manage, configure, install and run your Apache Hadoop customization, development and deployment projects, ensuring it is optimized to achieve your business goals. Our team of experts has the experience necessary to ensure your unique challenges are met. We specialize in Hadoop deployment to Amazon EMR, Microsoft HDInsight, Google Cloud DataProc and more. Nub8’s big data team implements solutions that help clients derive value and gain actionable insights from large data volumes stored in their Hadoop cluster.