Dashboard - MapReduce Overview

In this Chapter

Dashboard shows the monitoring details for QueryIO cluster.

Dashboard displays:

MapReduce Overview

MapReduce is a framework for processing parallelizable problems across huge datasets using a large number of computers (nodes), collectively referred to as a cluster (if all nodes are on the same local network and use similar hardware) or a grid (if the nodes are shared across geographically and administratively distributed systems, and use more heterogenous hardware).

A MapReduce job usually splits the input data-set into independent chunks which are processed by the map tasks in a completely parallel manner. The framework sorts the outputs of the maps, which are then input to the reduce tasks. Typically both the input and the output of the job are stored in a file-system. The framework takes care of scheduling tasks, monitoring them and re-executes the failed tasks.

The MapReduce framework consists of a single master ResourceManager and one NodeManager per cluster-node. The ResourceManager is the ultimate authority that arbitrates resources among all the applications in the system. The NodeManager is the per-machine framework agent who is responsible for containers, monitoring their resource usage (cpu, memory, disk, network) and reporting the same to the ResourceManager/Scheduler.

MapReduce Overview displays Container summary and Application Summary. It also displays ResourceManager summary and NodeManager summary.


Container Summary

Container Summary is gathered form NodeManager. It also displays a pie chart containing container information.

Application Summary

Application Summary is gathered from ResourceManager. It also displays a pie chart showing application information.

ResourceManager Summary

It displays certain attributes about the ResourceManager in the cluster in a tabular form.
Summary attributes are:

NodeManager Summary

It displays certain attributes about the NodeManager in the cluster in a tabular form.
Summary attributes are:

Copyright 2018 QueryIO Corporation. All Rights Reserved.

QueryIO, "Big Data Intelligence" and the QueryIO Logo are trademarks of QueryIO Corporation. Apache, Hadoop and HDFS are trademarks of The Apache Software Foundation.