Performance

Configuration Parameters

File descriptor limits

A busy Hadoop process might need to open a lot of files. The open files limit [ulimit] in Linux is 1024 by default, which might be too low. You should set to a higher value [let's say 16384].

You can view your ulimits in bash by running:

$ ulimit -a

To set the fd ulimit for a process, you'll need to be logged in as root. As root user, open a shell, and run:

# ulimit -n 16384

You can also set the ulimit for the hadoop user in /etc/security/limits.conf; it will set the value persistently.



Copyright 2017 QueryIO Corporation. All Rights Reserved.

QueryIO, "Big Data Intelligence" and the QueryIO Logo are trademarks of QueryIO Corporation. Apache, Hadoop and HDFS are trademarks of The Apache Software Foundation.