Performance

Configuration Parameters

File descriptor limits

A busy Hadoop process might need to open a lot of files. The open files limit [ulimit] in Linux is 1024 by default, which might be too low. You should set to a higher value [let's say 16384].

You can view your ulimits in bash by running:

$ ulimit -a

To set the fd ulimit for a process, you'll need to be logged in as root. As root user, open a shell, and run:

# ulimit -n 16384

You can also set the ulimit for the hadoop user in /etc/security/limits.conf; it will set the value persistently.



Copyright © 2018 QueryIO Corporation. All Rights Reserved.

QueryIO, "Big Data Intelligence" and the QueryIO Logo are trademarks of QueryIO Corporation. Apache, Hadoop and HDFS are trademarks of The Apache Software Foundation.