Serving static files is broken with a directory under /

[kccqzy]

Long story short

Serving static files is a feature of aiohttp that can greatly make development more convenient. Unfortunately, the behavior is broken when the directory is a relative path AND follow_symlinks is False AND the current directory is /.

Expected behaviour

All static files could be served specified in the directory normally.

Actual behaviour

None of the static files could be served. aiohttp returns 404.

Steps to reproduce

On a Linux machine or container, prepare the following file called serve.py at the root of the file system

from aiohttp import web
app = web.Application()
app.add_routes([web.static('/ui', './ui')])
web.run_app(app)

Create a file under the /ui directory, e.g. mkdir /ui && echo Hello > /ui/a.txt.

Use curl or any HTTP client to fetch localhost:8080/ui/a.txt.

I expect the file to be returned, but 404 is returned.

strace

Running it under strace, it uses readlink(2) on the file path. But the file is not a symlink. The Linux kernel returns EINVAL.

Relevant code

The relevant code causing the bug is here in web_urldispatcher.py:

    async def _handle(self, request: Request) -> StreamResponse:
        rel_url = request.match_info['filename']
        try:
            filename = Path(rel_url)
            if filename.anchor:
                # rel_url is an absolute name like
                # /static/\\machine_name\c$ or /static/D:\path
                # where the static dir is totally different
                raise HTTPForbidden()
            filepath = self._directory.joinpath(filename).resolve()
            if not self._follow_symlinks:
                filepath.relative_to(self._directory)
        except (ValueError, FileNotFoundError) as error:
            # relatively safe
            raise HTTPNotFound() from error
        except HTTPForbidden:
            raise
        except Exception as error:
            # perm error or other kind!
            request.app.logger.exception(error)
            raise HTTPNotFound() from error

When the self._follow_symlinks is False, it calls filepath.relative_to(self._directory) which raises a ValueError. Indeed I have confirmed that self._directory starts with two slashes. Here's why there are two slashes:

>>> import pathlib, os
>>> os.chdir('/')
>>> pathlib.Path('./ui').resolve()
PosixPath('//ui')

Since there are two slashes, the call to relative_to then raises an exception, which gets interpreted as a non-existent file.

Your environment

• Debian Linux (stretch)
• aiohttp-3.5.4

[aio-libs-bot]

GitMate.io thinks the contributor most likely able to help you is @asvetlov.

Possibly related issues are #1401 (Serving static files with add_static() breaks event loop on windows), #1093 (Problem with uvloop and static files), #3429 (CI is broken), #1404 (Use aiofiles in static file serving), and #1893 (ConnectionResetError when close browser, but not all static files was fetch).

[kccqzy]

I did some more research and let me clarify that the root cause of the issue is in fact this: https://bugs.python.org/issue33660

However, this upstream Python bug is affecting an important use case of aiohttp. I suggest we adopt a workaround in aiohttp to call resolve() twice.

[asvetlov]

Upgrading Python to 3.8+ solves the issue.
Sorry for the inconvenience.