192

This is happening on Ubuntu Release 12.04 (precise) 64-bit Kernel Linux 3.2.0-25-virtual

I'm trying to increase the number of open files allowed for a user. This is for my eclipse java application where the current limit of 1024 is not enough.

According to the posts I've found so far, I should be able to put lines into

/etc/security/limits.conf like this:

soft nofile 4096
hard nofile 4096

to increase the number of open files allowed for all users.

But that's not working for me and I think the problem is not related to that file.

For all users, the default limit is 1024, regardless of what is in /etc/security/limits.conf (I rebooted after changing that file)

$ ulimit -n
1024

Now, despite the entries in /etc/security/limits.conf I can't increase that:

$ ulimit -n 2048

-bash: ulimit: open files: cannot modify limit: Operation not permitted The weird part is that I can change the limit downwards, but can't change it upwards - even to go back to a number which is below the original limit:

$ ulimit -n 800
$ ulimit -n
800

$ ulimit -n 900

-bash: ulimit: open files: cannot modify limit: Operation not permitted

As root, I can change that limit to whatever I want, up or down. It doesn't even seem to care about the supposedly system-wide limit in /proc/sys/fs/file-max

# cat /proc/sys/fs/file-max
188897

# ulimit -n 188898
# ulimit -n 
188898

But even I get eclipse to run as root, my application still crashes because of "Too Many Open File" exception!

So far, I haven't found any way to increase the open files limit for a non-root user.

How should I properly do this? I have looked at several other posts but no luck!

Jr.
  • 3
iCode
  • 2,121
  • 8
    Also, note that after you edit /etc/security/limits.conf, you may have to logout and then back in before you can use the new max limit. I did this, and was baffled by ulimit -Hs still showing 1000 when I had just raised it to 1000000! Then I logged out and back in, and ulimit showed the new amount. – Cerin Mar 15 '14 at 21:28
  • 2
    For GUI session you may find also important (Ubuntu 16+) this thread. – Waldemar Wosiński Mar 08 '18 at 07:57
  • Very important! My solution came from Waldemar's link on https://superuser.com/questions/1200539/cannot-increase-open-file-limit-past-4096-ubuntu Also, logging out was insufficient, I needed to reboot (restart) for it to work. – Chee Loong Soon Mar 10 '23 at 22:14

4 Answers4

175

The ulimit command by default changes the HARD limits, which you (a user) can lower, but cannot raise.

Use the -S option to change the SOFT limit, which can range from 0-{HARD}.

I have actually aliased ulimit to ulimit -S, so it defaults to the soft limits all the time.

alias ulimit='ulimit -S'

As for your issue, you're missing a column in your entries in /etc/security/limits.conf.

There should be FOUR columns, but the first is missing in your example.

* soft nofile 4096
* hard nofile 4096

The first column describes WHO the limit is to apply for. '*' is a wildcard, meaning all users. To raise the limits for root, you have to explicitly enter 'root' instead of '*'.

You also need to edit /etc/pam.d/common-session* and add the following line to the end:

session required pam_limits.so
landroni
  • 5,941
  • 7
  • 36
  • 58
lornix
  • 2,036
  • 3
    How do I change the Hard limit for the maximum number of open file descriptors? My problem is that I need to allow my Eclipse application to have as many as open file as it wants but the limit of 4096 is not enough and I want to set it to as high as possible maybe 500000. My setting in the limits.conf is as you mentioned with the star and still the Hard limit is not moving. – iCode Jul 11 '12 at 04:13
  • did you logout/reboot this time? ulimit -a will show you all the limits, so you can compare to see if they've changed. – lornix Jul 11 '12 at 04:20
  • yes, I modified the /etc/security/limits.conf file only and reboot. after reboot, the ulimit -Hn500000 command still throws bash: ulimit: open files: cannot modify limit: Operation not permitted error. – iCode Jul 11 '12 at 04:53
  • 1
    Well, 500,000 IS a bit large. I doubt you've got enough RAM to contain all the file descriptors for that many open files. Try something a bit smaller, say 65,000 or less. Although the docs say it should accept unlimited as a value too, also infinity, both should remove restrictions. – lornix Jul 11 '12 at 07:51
  • If the value you supply is too large, it probably just ignores the setting. – lornix Jul 11 '12 at 08:05
  • You are correct. I set it to 65000 and that worked but the issue is that I am still getting too many open file error! The library that I am using does this (does ask me why but it is valid) and I can not change it. What is the maximum number that I can set this number to? – iCode Jul 11 '12 at 20:59
  • 2
    Well, you discovered the value of /proc/sys/fs/file-max was 188,897... So I'd say your upper limit is between 188,000 (allowed) and 500,000 (not allowed). Boggles the mind that you've got something that requires over 65000 open files simultaneously. Wow. Usually that's handled on big-iron. I'd be suspicious of a programming, uh, misconception. (With all due respect, please) – lornix Jul 11 '12 at 21:13
  • Lovely - this worked like a charm. – Rakesh Sankar Oct 25 '13 at 05:38
  • 4
    that was it: session required pam_limits.so, thanks! – lethalman Nov 14 '13 at 11:15
  • 18
    At first I missed the wildcard on the end of /etc/pam.d/common-session* and just edited common-session, and even after a reboot it didn't work. But after adding the same line (for pam_limits.so) to common-session-noninteractive, ulimit -n displayed the new value after a fresh login (no reboot required). FWIW I was trying to change the limit for root (only). – Lambart Jun 26 '14 at 18:12
  • I needed to add session required pam_limits.so to the xrdp-sesman file in order to have this word over RDP. – Dave Jul 16 '15 at 18:31
  • 3
    also check if value different if logged in as root.
    https://superuser.com/questions/1200539/cannot-increase-open-file-limit-past-4096-ubuntu     – Robbo_UK Jul 21 '17 at 13:29
  • I have the same issue if using root user... – Abbadon Oct 25 '17 at 15:15
  • I followed the instructions in this post and now my virtual box virtual machine doesn't start up. I don't think it has to do with the fact that I applied the changes on my VM. Has this happened to anyone ? – ggulgulia Dec 27 '18 at 14:13
  • Can you explain what session required pam_limits.so does and why it's necessary? I've never used this and I seem to achieve an increased limit for root and all other users. Thanks! – Joshua Pinter Dec 15 '22 at 19:29
24

If you use soft and hard limits on a per user basis you can use something like:

su USER --shell /bin/bash --command "ulimit -n"

to check wether your settings are working for that specific user or not.

David Foerster
  • 36,264
  • 56
  • 94
  • 147
Jeroen Teeuwen
  • 341
  • 2
  • 2
  • 1
    I was also able to use this to unzip a file against ulimit rules su i --shell /bin/bash --command "ulimit -f 8000000 && gunzip -f file_name.gz" – Íhor Mé Mar 11 '20 at 22:38
10
ulimit -S -n 4096

That should increase your soft limit for open files (-n) to 4096.

ekeyser
  • 239
  • 4
    The soft limit is rarely the problem, in my experience. It's the hard limit that has to be increased. – James Moore May 30 '20 at 23:29
  • This helped in my case. My soft limit was 1024 and the hard limit was way higher than that (default Ubuntu 22.04 installation) – Orlando Nov 06 '23 at 10:04
6

I have lots of trouble getting this to work.

Using the following allows you to update it regardless of your user permission. 

sudo sysctl -w fs.inotify.max_user_watches=100000
David Foerster
  • 36,264
  • 56
  • 94
  • 147
Layke
  • 1,379
  • 4
    I tried this since I'm having same issue but it did not work $ sudo sysctl -w fs.inotify.max_user_watches=100000 fs.inotify.max_user_watches = 100000 $ ulimit -n 1024 – grepmaster Dec 07 '15 at 11:12
  • This also worked for me on Ubuntu 20.04 inside WSL2, on a largeish React project which is using parcel for packaging. It does not necessarily seem to be the "open files" for me, but rather the "watched" files. – donmartin Sep 17 '20 at 14:33
  • 1
    I don't think that inotify is related to ulimits. – jjmontes Feb 10 '21 at 21:34