sphinx-build -M isn't like other options

[nedbat]

(Sorry if this is a duplicate, it's hard to search for "-M"...)

The -M option isn't like other options:

First, it isn't mentioned at all in sphinx-build --help.

It doesn't interact with other options properly. If I run sphinx-build -M html "." "_build", my build starts. Great.

But if I run sphinx-build -vv -M html "." "_build", I get, sphinx-build: error: unrecognized arguments: -M
If I run sphinx-build -M html -vv "." "_build", I get, ApplicationError: config directory doesn't contain a conf.py file (./html)
(Note: adding -vv to the very end does work, but there's no help about -M to help me figure that out.)

Environment info

• OS: Mac
• Python version: 3.7.4
• Sphinx version: 2.1.1
• Sphinx extensions: none
• Extra tools: none

Reproduction:

$ mktmpenv -p python3.7 -n
Running virtualenv with interpreter /usr/local/bin/python3.7
Using base prefix '/usr/local/pythonz/pythons/CPython-3.7.4'
/usr/local/pythonz/pythons/CPython-2.7.14/lib/python2.7/site-packages/virtualenv.py:1047: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
New python executable in /usr/local/virtualenvs/tmp-be002eb96f07729d/bin/python3.7
Also creating executable in /usr/local/virtualenvs/tmp-be002eb96f07729d/bin/python
Installing setuptools, pip, wheel...done.
Requirement already up-to-date: pip in /usr/local/virtualenvs/tmp-be002eb96f07729d/lib/python3.7/site-packages (19.2.1)
Requirement already up-to-date: setuptools in /usr/local/virtualenvs/tmp-be002eb96f07729d/lib/python3.7/site-packages (41.0.1)
This is a temporary environment. It will be deleted when you run 'deactivate'.
(tmp-be002eb96f07729d)
$ pip install sphinx
Collecting sphinx
  Using cached https://files.pythonhosted.org/packages/6e/e5/c9ba68935cd2d72c553d49bc156bfb15ddb40e734ea7e3f238d8bd6ca6f1/Sphinx-2.1.2-py3-none-any.whl
Collecting sphinxcontrib-htmlhelp (from sphinx)
  Using cached https://files.pythonhosted.org/packages/e4/35/80a67cc493f4a8a9634ab203a77aaa1b84d79ccb1c02eca72cb084d2c7f7/sphinxcontrib_htmlhelp-1.0.2-py2.py3-none-any.whl
Collecting imagesize (from sphinx)
  Using cached https://files.pythonhosted.org/packages/fc/b6/aef66b4c52a6ad6ac18cf6ebc5731ed06d8c9ae4d3b2d9951f261150be67/imagesize-1.1.0-py2.py3-none-any.whl
Collecting alabaster<0.8,>=0.7 (from sphinx)
  Using cached https://files.pythonhosted.org/packages/10/ad/00b090d23a222943eb0eda509720a404f531a439e803f6538f35136cae9e/alabaster-0.7.12-py2.py3-none-any.whl
Collecting packaging (from sphinx)
  Using cached https://files.pythonhosted.org/packages/91/32/58bc30e646e55eab8b21abf89e353f59c0cc02c417e42929f4a9546e1b1d/packaging-19.0-py2.py3-none-any.whl
Requirement already satisfied: setuptools in /usr/local/virtualenvs/tmp-be002eb96f07729d/lib/python3.7/site-packages (from sphinx) (41.0.1)
Collecting sphinxcontrib-devhelp (from sphinx)
  Using cached https://files.pythonhosted.org/packages/b0/a3/fea98741f0b2f2902fbf6c35c8e91b22cd0dd13387291e81d457f9a93066/sphinxcontrib_devhelp-1.0.1-py2.py3-none-any.whl
Collecting sphinxcontrib-applehelp (from sphinx)
  Using cached https://files.pythonhosted.org/packages/13/9a/4428b3114d654cb1cd34d90d5e6fab938d5436f94a571155187ea1dd78b4/sphinxcontrib_applehelp-1.0.1-py2.py3-none-any.whl
Collecting sphinxcontrib-serializinghtml (from sphinx)
  Using cached https://files.pythonhosted.org/packages/57/b3/3648e48fa5682e61e9839d62de4e23af1795ceb738d68d73bd974257a95c/sphinxcontrib_serializinghtml-1.1.3-py2.py3-none-any.whl
Collecting sphinxcontrib-qthelp (from sphinx)
  Using cached https://files.pythonhosted.org/packages/ce/5b/4747c3ba98b3a3e21a66faa183d8f79b9ded70e74212a7988d236a6eb78a/sphinxcontrib_qthelp-1.0.2-py2.py3-none-any.whl
Collecting Jinja2>=2.3 (from sphinx)
  Using cached https://files.pythonhosted.org/packages/1d/e7/fd8b501e7a6dfe492a433deb7b9d833d39ca74916fa8bc63dd1a4947a671/Jinja2-2.10.1-py2.py3-none-any.whl
Collecting snowballstemmer>=1.1 (from sphinx)
Collecting babel!=2.0,>=1.3 (from sphinx)
  Using cached https://files.pythonhosted.org/packages/2c/60/f2af68eb046c5de5b1fe6dd4743bf42c074f7141fe7b2737d3061533b093/Babel-2.7.0-py2.py3-none-any.whl
Collecting Pygments>=2.0 (from sphinx)
  Using cached https://files.pythonhosted.org/packages/5c/73/1dfa428150e3ccb0fa3e68db406e5be48698f2a979ccbcec795f28f44048/Pygments-2.4.2-py2.py3-none-any.whl
Collecting sphinxcontrib-jsmath (from sphinx)
  Using cached https://files.pythonhosted.org/packages/c2/42/4c8646762ee83602e3fb3fbe774c2fac12f317deb0b5dbeeedd2d3ba4b77/sphinxcontrib_jsmath-1.0.1-py2.py3-none-any.whl
Collecting requests>=2.5.0 (from sphinx)
  Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting docutils>=0.12 (from sphinx)
  Downloading https://files.pythonhosted.org/packages/d6/08/e44fae2abfd2b9b6f4c77579d4bab43e6f4fcbfa2b642bcb90ee3db78b04/docutils-0.15.1.tar.gz (2.0MB)
     |████████████████████████████████| 2.0MB 3.0MB/s
Collecting pyparsing>=2.0.2 (from packaging->sphinx)
  Using cached https://files.pythonhosted.org/packages/dd/d9/3ec19e966301a6e25769976999bd7bbe552016f0d32b577dc9d63d2e0c49/pyparsing-2.4.0-py2.py3-none-any.whl
Collecting six (from packaging->sphinx)
  Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23 (from Jinja2>=2.3->sphinx)
  Using cached https://files.pythonhosted.org/packages/ce/c6/f000f1af136ef74e4a95e33785921c73595c5390403f102e9b231b065b7a/MarkupSafe-1.1.1-cp37-cp37m-macosx_10_6_intel.whl
Collecting pytz>=2015.7 (from babel!=2.0,>=1.3->sphinx)
  Using cached https://files.pythonhosted.org/packages/3d/73/fe30c2daaaa0713420d0382b16fbb761409f532c56bdcc514bf7b6262bb6/pytz-2019.1-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests>=2.5.0->sphinx)
  Using cached https://files.pythonhosted.org/packages/69/1b/b853c7a9d4f6a6d00749e94eb6f3a041e342a885b87340b79c1ef73e3a78/certifi-2019.6.16-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests>=2.5.0->sphinx)
  Using cached https://files.pythonhosted.org/packages/e6/60/247f23a7121ae632d62811ba7f273d0e58972d75e58a94d329d51550a47d/urllib3-1.25.3-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5 (from requests>=2.5.0->sphinx)
  Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests>=2.5.0->sphinx)
  Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Building wheels for collected packages: docutils
  Building wheel for docutils (setup.py) ... done
  Created wheel for docutils: filename=docutils-0.15.1-cp37-none-any.whl size=547603 sha256=e014dc01af2e9de9be8d126b32f925d5483eaba50343b40e1445e1098707b36c
  Stored in directory: /Users/ned/Library/Caches/pip/wheels/76/ae/2b/ee4711f8aa7aaadba139c183d5d9711d3bcded8efb69a9fa21
Successfully built docutils
Installing collected packages: sphinxcontrib-htmlhelp, imagesize, alabaster, pyparsing, six, packaging, sphinxcontrib-devhelp, sphinxcontrib-applehelp, sphinxcontrib-serializinghtml, sphinxcontrib-qthelp, MarkupSafe, Jinja2, snowballstemmer, pytz, babel, Pygments, sphinxcontrib-jsmath, certifi, urllib3, idna, chardet, requests, docutils, sphinx
Successfully installed Jinja2-2.10.1 MarkupSafe-1.1.1 Pygments-2.4.2 alabaster-0.7.12 babel-2.7.0 certifi-2019.6.16 chardet-3.0.4 docutils-0.15.1 idna-2.8 imagesize-1.1.0 packaging-19.0 pyparsing-2.4.0 pytz-2019.1 requests-2.22.0 six-1.12.0 snowballstemmer-1.9.0 sphinx-2.1.2 sphinxcontrib-applehelp-1.0.1 sphinxcontrib-devhelp-1.0.1 sphinxcontrib-htmlhelp-1.0.2 sphinxcontrib-jsmath-1.0.1 sphinxcontrib-qthelp-1.0.2 sphinxcontrib-serializinghtml-1.1.3 urllib3-1.25.3

$ sphinx-quickstart --quiet -p Sample -a Me

Finished: An initial directory structure has been created.

You should now populate your master file ./index.rst and create other documentation
source files. Use the Makefile to build the docs, like so:
   make builder
where "builder" is one of the supported builders, e.g. html, latex or linkcheck.

$ sphinx-build -M html "." "_build"
Running Sphinx v2.1.2
making output directory... done
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 1 source files that are out of date
updating environment: 1 added, 0 changed, 0 removed
reading sources... [100%] index
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index
generating indices... genindex
writing additional pages... search
copying static files... done
copying extra files... done
dumping search index in English (code: en) ... done
dumping object inventory... done
build succeeded.

The HTML pages are in _build/html.

$ sphinx-build -vv -M html "." "_build"
usage: sphinx-build [OPTIONS] SOURCEDIR OUTPUTDIR [FILENAMES...]
sphinx-build: error: unrecognized arguments: -M

$ sphinx-build -M html -vv "." "_build"

Traceback (most recent call last):
  File "/usr/local/virtualenvs/tmp-be002eb96f07729d/lib/python3.7/site-packages/sphinx/cmd/build.py", line 283, in build_main
    args.tags, args.verbosity, args.jobs, args.keep_going)
  File "/usr/local/virtualenvs/tmp-be002eb96f07729d/lib/python3.7/site-packages/sphinx/application.py", line 161, in __init__
    "conf.py file (%s)") % confdir)
sphinx.errors.ApplicationError: config directory doesn't contain a conf.py file (./html)

Application error:
config directory doesn't contain a conf.py file (./html)

[samuel-emrys]

For other people who come across this issue, it's caused by the fact that the -M command is special. it must come first, and the following two arguments must be the source dir and build dir, in that order. This is my interpretation of the following logic in sphinx/cmd/build.py and sphinx/cmd/make_mode.py:

    if argv[:1] == ['-M']:
        return make_main(argv)
    else:
        return build_main(argv)

and in make_main:

    make = Make(args[1], args[2], args[3:])

where the constructor for Make is defined as:

class Make:
    def __init__(self, srcdir: str, builddir: str, opts: List[str]) -> None:

Importantly, all optional arguments will be parsed if you append them to the end, it's just that the -M flag cares that the source dir and build dir immediately follow the builder value.

Help should be added for this to argparse. Ideally, both the make and build commands here should have the same interface - argument positions should be independent of whether -M or -b is specified. This is essentially the same findings that @nedbat has detailed above, but I thought i'd be specific about the requirements when using this command.

[AA-Turner]

-M is now documented, and there is a longer term aspiration to provide a unified sphinx command (#5618), so whilst still not ideal, I'm closing this issue.

A