Weird bug I cant figure out: permission denied

[matentzn]

Reported simultaneously today:

• Krishna to agro prj patterns for #1228 EnvironmentOntology/envo#1330 (comment)
• Error in the design pattern during the release CompositionalDietaryNutritionOntology/cdno#97 (comment)

https://github.com/Southern-Cross-Plant-Science/cdno/blob/master/src/ontology/Makefile#L114

I cant figure out what is wrong. Here is what I know:

If I run using the ODK docker wrapper:

./run.sh dosdp-tools generate --table-format=csv --template=./patterns/entity_attribute.yaml --outfile=modules/entity_attribute.tmp.owl --obo-prefixes=true --infile=modules/entity_attribute.csv

Everything works fine.

While adding the absolute identical command into a Makefile:

ttt:
	dosdp-tools generate --table-format=csv --template=./patterns/entity_attribute.yaml --outfile=modules/entity_attribute.tmp.owl --obo-prefixes=true --infile=modules/entity_attribute.csv

Does not work:

make: dosdp-tools: Permission denied
make: *** [Makefile:66: ttt] Error 127

Strangely enough, the makefile works without the docker wrapper.

Any help appreciated.

[balhoff]

Is the command not executable in the docker image?

[matentzn]

In both cases the command is run in the docker image.

[matentzn]

Moved this issue to ODK because this is where it belongs after all :)

[gouttegd]

So what seems to be the problem is that the dosdp-tools script has the same name than the dosdp-tools directory in /tools. And because /tools itself is also in the PATH when calling the ODK, when Make tries to find a dosdp-tools executable, it finds the directory /tools/dosdp-tools and tries to execute it (which of course it cannot do, since it is a directory).

I assume it works when passing through the shell (with exec or an if construct) because the shell must use a different logic when trying to find an executable (specifically, when finding a directory with the name it looks for, the shell must know to skip it and to continue searching further down the PATH).

I still don’t understand how it can only happen when going through Docker and why not everybody seems affected, but at least that gives us a clear solution: fix the ODK to change the name of the /tools/dosdp-tools folder to anything else (e.g. /tools/dosdptools or whatever).

[matentzn]

Thank you @gouttegd!

[pbuttigieg]

Nice catch @gouttegd - @matentzn is there a timeline for an update? Or should we find a workaround for EnvironmentOntology/envo#1330 (comment) ?

[matentzn]

I already made a pull request on ENVO!

[kaiiam]

Thanks again @matentzn and @gouttegd!

[gouttegd]

For information, contrary to what many of us (myself included) thought while we were debugging this issue yesterday on Slack, this is not a Docker-induced problem.

I can reproduce the problem without Docker on my GNU/Linux system:

1. Create a directory hierarchy somewhere (here, my home folder, but it doesn’t matter) with a dosdp-tools in the middle:

make -p ~/tools/dosdp-tools/bin

2. Create an executable file named dosdp-tools at the bottom of this hierarchy:

cat <<EOF > ~/tools/dosdp-tools/bin/dosdp-tools
#!/bin/sh
echo "I am here"
EOF
chmod 755 ~/tools/dosdp-tools/bin/dosdp-tools

3. Create a Makefile with a recipe that tries to call the dosdp-tools script:

cat <<EOF > ~/Makefile
test:
        @dosdp-tools
EOF

4. Try invoking that recipe with a custom PATH that includes both the ~/tools directory and the ~/tools/dosdp-tools/bin directory (in that order):

PATH=$HOME/tools:$HOME/tools/dosdp-tools/bin:$PATH make test
make: dosdp-tools: Permission denied
make: *** [Makefile:2: test] Error 127

5. Observe that if you invert the entries in the custom PATH so that the ~/tools/dosdp-tools/bin folder is before the ~/tools folder, everything is fine:

PATH=$HOME/tools/dosdp-tools/bin:$HOME/tools:$PATH make test
I am here

So there is no Docker trickery involved here. The problem lies purely within Make, which seems to have its own logic to find an executable in the PATH (instead of using, say, execvp(3)), and that logic is confused when it finds in the PATH a directory with the same name than the executable it is looking for.

Reproduced with GNU Make version 4.3 on GNU/Linux (which is also the version used in the ODK since version 1.3.1); not reproduced with GNU Make 3.81 on MacOS, and not reproduced with GNU Make 4.2.1 (which is the version used in the ODK up to version 1.3.0, before we upgraded to Ubuntu 22.04).

So, looks like a bug introduced in GNU Make between 4.2.1 and 4.3. I guess people who were not affected by the problem yesterday were still using a pre-1.3.1 ODK.

This also highlights a lack of coverage of our tests. We did a lot of testing after upgrading the base system to Ubuntu 22.04, but we completely missed this problem.

[gouttegd]

For those who are curious (or if someone lands on this ticket while looking for a similar problem), it is this bug in GNU Make. It was fixed in GNU Make’s master branch two years ago, but the fix is not available in any published version yet (last release of GNU Make was 4.3, back in January 2020, before the bug was reported).

So, nothing else to do here apart from the fix in #652 — and being careful with the ODK’s PATH, until we can upgrade to the next version of GNU Make.

[matentzn]

This is so super interesting. Thanks for getting to the bottom of this, I learned something today! :)