21

I'm trying to go into an irb session with specific environment variables from a file with this command:

$ env $(cat env.sh) irb

But when I try press Tab after I type env. to complete it, I get this following error:

$ env $(cat env.-bash: unexpected EOF while looking for matching `)'
-bash: syntax error: unexpected end of file

Another interesting thing is that if I'm logged in as root, this error does not occur.

Here's the output of find ~ -uid 0:

$ find ~ -uid 0
/home/(redacted)/.rpmdb
/home/(redacted)/.rpmdb/Group
/home/(redacted)/.rpmdb/Conflictname
/home/(redacted)/.rpmdb/Installtid
/home/(redacted)/.rpmdb/Sha1header
/home/(redacted)/.rpmdb/Providename
/home/(redacted)/.rpmdb/__db.002
/home/(redacted)/.rpmdb/Requirename
/home/(redacted)/.rpmdb/Sigmd5
/home/(redacted)/.rpmdb/__db.001
/home/(redacted)/.rpmdb/Obsoletename
/home/(redacted)/.rpmdb/.dbenv.lock
/home/(redacted)/.rpmdb/Name
/home/(redacted)/.rpmdb/Basenames
/home/(redacted)/.rpmdb/Triggername
/home/(redacted)/.rpmdb/Packages
/home/(redacted)/.rpmdb/Dirnames
/home/(redacted)/.rpmdb/__db.003

Can anyone explain to me why this is happening and if so, how do I fix it in when I'm not a root user?

αғsнιη
  • 35,660
eldosoa
  • 745

1 Answers1

36

You found a bug in the Bash Completion library used by Ubuntu.

What does this mean?

Ubuntu uses a bash completion library to make bash completion smart. This library lives in /usr/share/bash-completion/bash_completion.

Essentially, this library declares a few clever functions that know about typical commands and how to complete them. Whenever you press Tab, functions within this library get called and attempt to complete your current command line. So for example if you type apt-get iTab it will complete that to apt-get install. If you don't source that library, you only have the standard, primitive bash completion - so for example if you type apt-get iTab without having sourced it, bash will simply look for files in the current directory starting with i and attempt to complete your command according to these filenames.

Why isn't it happening as root?

Because when you use sudo su to make yourself root, the bash completion library isn't sourced. This would be different if you used sudo -i to make yourself root. I bet you see the bug then, don't you? See for example 'sudo su -' vs 'sudo -i' vs 'sudo /bin/bash' - when does it matter which is used, or does it matter at all? if you aren't familiar with the differences.

In my case, as a normal user, the library gets sourced when I start a Bash shell because ~/.bashrc sources /etc/bash_completion which sources /usr/share/bash-completion/bash_completion.

If I use sudo -i to login as root, the library gets sourced because /etc/profile sources /etc/profile.d/bash_completion.sh which sources /usr/share/bash-completion/bash_completion.

Why is that bug happening?

Try to execute this command:

$ eval 'quoted=$(cat' env.
bash: unexpected EOF while looking for matching `)'
bash: syntax error: unexpected end of file

Looks familiar? ;-) Indeed, that's exactly what happened behind the scenes when you hit Tab in the context you described. More precisely, the bug is in the function _quote_readline_by_ref declared by /usr/share/bash-completion/bash_completion. If you sourced that file you should have that function available. So next try this:

$ _quote_readline_by_ref '$(cat env.' quoted
bash: unexpected EOF while looking for matching `)'
bash: syntax error: unexpected end of file

Given these arguments, the function _quote_readline_by_ref performs, among other things, that eval mentioned above. You can have a look if you like. And when you typed env $(cat env.Tab, behind the scenes that function got called with exactly those arguments. So that's what happened.

This eval hack was supposed to fix another issue, but I guess it introduced this other bug in the process.

How do I fix it?

It turns out that this bug has already been reported. After reading that bug report, I see three ways to fix it:

  1. Patch it: In one of the comments in that bug report, someone suggests replacing the line

    [[ ${!2} == \$* ]] && eval $2=${!2}
    

    within function _quote_readline_by_ref in the file /usr/share/bash-completion/bash_completion by the line

    [[ ${!2} == \$\'* ]] && eval $2=${!2}
    

    I recommend against doing this. The person who wrote that comment does not appear to be a developer of bash-completion. This hotfix will simply cause the left operand of the statement to evaluate to false and thus prevent that eval from happening. However without a good knowledge of what that function is supposed to do and in what contexts it is called, it is unclear whether this will not potentially break some other intended functionality.

  2. Get the newest version: As also mentioned in that bug report, this bug is not present in git head (wherein among other changes the function _quote_readline_by_ref has been simplified). You can simply clone the current revision from Git:

    git clone https://salsa.debian.org/debian/bash-completion.git
    

    ...and then copy the newest version of the bash_completion script to /usr/share/bash-completion (no urgent need to backup the old version unless it makes you feel safer - if you experience some problems, sudo apt-get install --reinstall bash-completion should revert any changes you made just fine.) This is the way I recommend if you're in a hurry to get this fixed. :-)

Note that none of those solutions will make bash completion inside command substitution work: as mentioned in that same bug report, this is broken in Bash 4.3.

  1. Sit back and wait: Sooner or later a new version will get released (which may even fix bash completion inside command substitution) and you will get it with some future Ubuntu version. That's what I'm going for ;-)
con-f-use
  • 18,813
Malte Skoruppa
  • 13,196
  • 5
  • 57
  • 65
  • Wow, thank you very much for the very comprehensive answer! – eldosoa Jan 24 '15 at 04:44
  • Its amazing that the bug is still present in 14.04. Could you link the git repository in answer? The bug-report requires a login and a quick google search finds a lot of git bash completion repositories. I suspect it's: git://git.debian.org/git/bash-completion/bash-completion.git – con-f-use Jun 23 '15 at 08:56
  • 1
    @con-f-use Yep, that's it! The Git repo is also linked from the main site of bash-completion, which is why I hadn't linked to it in my answer. – Malte Skoruppa Jun 23 '15 at 09:05
  • 2
    @con-f-use Your comment caused me to do some more research on this bug. It turns out it is not, and was never, an upstream bug. Instead, it is actually a bug introduced by an Ubuntu patch applied against the upstream version. Noone appears to have narrowed down this bug to that particular patch so far. So I have reported my finding in the corresponding Ubuntu bug report: https://bugs.launchpad.net/ubuntu/+source/bash-completion/+bug/1312243. – Malte Skoruppa Jun 24 '15 at 10:53
  • 2
    Nice work Malte. – Barry Kelly Jul 23 '15 at 19:37
  • 1
    Thanks for the explanation. As of this writing, the git head version has removed the bug and autocompletes as desired. EDIT: actually, nevermind, it doesn't. – user3391564 Oct 02 '16 at 08:10
  • You say that when you run it as sudo it doesn't source the library. Is there a way to open a new shell where you explicitly make it not source the library, without that shell having to operate in root (sudo)? – Max von Hippel Oct 03 '16 at 05:12
  • 1
    @MaxvonHippel I said that when you use sudo su to become root, it doesn't source the library, but it will get sourced when you use sudo -i instead, which is the intended way to get an interactive root session. As for your question: Since any bash shell reads ~/.bashrc which eventually sources the library, and there is no way to "un-source" a file, I see no completely straightforward way. Here's a possible hack: Make the sourcing of the library conditional on some environment variable, say, NOCOMPL, not being defined in your ~/.bashrc, ... – Malte Skoruppa Oct 03 '16 at 10:10
  • @MaxvonHippel ... then you can start a shell with this environment variable defined (e.g., NOCOMPL=true bash) so as to not have the library sourced whenever you want this behavior. For more details, please ask this as a new question. – Malte Skoruppa Oct 03 '16 at 10:10
  • 1
    This is still a great answer. But the git repo has moved: git clone https://salsa.debian.org/debian/bash-completion.git – Kirk Feb 20 '19 at 20:55
  • 2
    It's still there in 19.04 and the latest commit has just disabled that type of completions in $(-constructs. Very disappointing. – con-f-use May 11 '19 at 10:44