 |
| DocFx output using Sphinx .yml files and Sphinx extensions |
Overview
The previous Sphinx/DocFx posts are:
In those posts, we talked about the Sphinx conf.py configuration file for configuring Sphinx to use extensions. An extension is simply a Python module that can be used to extend Sphinx functionality. We used the "sphinx.ext.autodoc" extension to get docstring comments from Python files and we used the "docfx_yaml.extension" extension to instruct Sphinx to export YAML files. In the Sphinx to DocFx post, we had a conf.py file with:
extensions = ["sphinx.ext.autodoc", "docfx_yaml.extension"]
In this post, we'll discuss two more extensions you can use so that the we have:
extensions = ["sphinx.ext.autodoc", "sphinx.ext.intersphinx",
"sphinx.ext.extlinks", "docfx_yaml.extension"]
where:
sphinx.ext.autodoc Import modules for documentation, including pulling in content from docstrings.
docfx_yaml.extension An exporter for the Sphinx autodoc module to produce YAML files for use with DocFX. Seems to need to be at the end of the extensions list. Order is important. The docfx extension needs to be at the end.
sphinx.ext.intersphinx Generate automatic links to the documentation in other projects like Python base classes. Depends on variable intersphinx_mapping variable in conf.py. See the interpret community repo for an example.
sphinx.ext.extlinks Allows creating a short link for commonly used links that go to subpages of one site.
sphinx.ext.intersphinx
The intersphinx_mapping configuration value is in the conf.py and it can be used to create mappings so that references to other documentation sets (outside of yours) can be referenced.
We'll use the following intersphinx_mapping:
intersphinx_mapping = {
'Python': ('https://docs.python.org/3', None),
'Pillow': ('https://pillow.readthedocs.io/en/latest/', None),
'NumPy': ('http://docs.scipy.org/doc/numpy/', None),
'pandas': ('http://pandas.pydata.org/pandas-docs/stable/', None),
'sklearn': ('http://scikit-learn.org/stable', None),
'matplotlib': ('http://matplotlib.sourceforge.net/', None)
}
To get a link to a Python builtin type at docs.python.org, we need only specify the type name in the type or rtype docstring field as follows:
""" :type: bool :rtype: list """
And the correct linkages will be made to the docset when Sphinx builds. And, if you wanted to link to these types in text outside of these fields? You can like so:
""" This is a link to the Python built-in string type: :class:`str`. """
It works the same for other doc sets specified in the intersphinx_mapping.
""" This is a link to :class:`pandas.DataFrame`, this to :mod:`matplotlib.image`, and this to :func:`numpy.array`. :type: pandas.DataFrame or numpy.array :rtype: matplotlib.image """
The big gotcha with using intersphinx_mapping is that if you look at the example above in the Sphinx rendered HTML, you would see pandas.DataFrame, matplotlib.image, and numpy.array are correctly linked to their library types for both the descriptive text and the :type: and :rtype: markup.
However, if you look in the DocFx rendered HTML (from the same docstring), you would see that while the links in the descriptive text do resolve, those in the :type: or :rtype: markup do not. To get around this problem, we need to use a cross reference file in DocFx as described here. This will be the subject of a future post. (For numpy links, see the stack overflow question.)
sphinx.ext.extlinks
This extension is a convenience for avoiding repeatedly typing a URL to a web site you are referencing frequently. The docs for extlinks show how you might use this to link back to GitHub issues. In our tutorial here, we'll create links back to Wikipedia.
In the conf.py file make sure you have extlinks configuration parameter defined like so:
extlinks = {'wiki':
('https://en.wikipedia.org/wiki/%s','Wikipedia: ')
}
Now, suppose in a docstring you want to reference these three Wikipedia pages: https://en.wikipedia.org/wiki/Machine_learning, https://en.wikipedia.org/wiki/Supervised_learning, and https://en.wikipedia.org/wiki/Unsupervised_learning. Your docstring would look like this:
""" Here are links using markup to make external links easier to work with. See :wiki:`Machine_learning`, :wiki:`Supervised_learning`, and :wiki:`Unsupervised_learning`. """
You can experiment with the presentation be modifying the extlinks configuration parameter.
Build Example
The steps below follow the post From Sphinx to DocFX - Generating Python API Documentation with DocFx. TIn this tutorial, our goal is to use the new extensions we enabled and show how they appear in the DocFX HTML. The prerequisites are:
- Sphinx installed
- DocFx installed
- Optional: read or ran the previous tutorial
Step 1: Clone the repo travelmarx-blog and start in the \sphinx-extensions-example folder.
Step 2: Create config.py and index.rst files.
Use the instructions in the Sphinx Quickstart post to generate these files, following the suggested answers for the prompts. Or, if you already have these files from that project, you can reuse them here. Your folder structure should look like this.
.
├───build
├───mycode
│ ├───core_api
│ │ ├───package1
│ │ └───package2
│ └───test_api
└───source
├───_static
└───_templates
In this tutorial, we are working with four of the extensions mentioned in the intro:
extensions = ['sphinx.ext.autodoc', 'sphinx.ext.intersphinx', 'sphinx.ext.extlinks', 'docfx_yaml.extension']
Point to the code folder:
import os
import sys
sys.path.insert(0, os.path.abspath('../mycode'))
Add intersphinx_mapping configuration:
intersphinx_mapping = {
'Python': ('https://docs.python.org/3', None),
'Pillow': ('https://pillow.readthedocs.io/en/latest/', None),
'NumPy': ('http://docs.scipy.org/doc/numpy/', None),
'pandas': ('http://pandas.pydata.org/pandas-docs/stable/', None),
'sklearn': ('http://scikit-learn.org/stable', None),
'matplotlib': ('http://matplotlib.sourceforge.net/', None)
}
Add extlinks configuration:
extlinks = {'wiki':
('https://en.wikipedia.org/wiki/%s','Wikipedia: ')
}
Some of these lines in the config.py file may already exist and you'll have to uncomment them and/or modify them.
In the root folder, run:
sphinx-apidoc -o source .\mycode
This creates .rst files in the \source folder.
Step 5: Modify the source\index.rst file to include the modules.rst file which is the entry point for code to document.
Running sphinx-apidoc will produce a source\modules.rst file by default. The modules.rst file is the entry point for documenting the code in \mycode.
source\index.rst (snippet, add the "modules" line)
Test documentation
=======================
.. toctree::
:maxdepth: 4
:caption: Table of Contents
modules
Step 6: Run sphinx-build to create Sphinx HTML and create YAML files for DocFx.
In the root folder, run
sphinx-build source build
You may not be interested in Sphinx's HTML, but it's this step which creates the YAML files. It doesn't hurt to check it to see what Sphinx's HTML looks like. To see it, run:
build\index.html
If you are interested in only the Sphinx generated HTML, then you can stop here.
Step 7: Confirm the YAML files were generated.
Starting in root folder, run:
dir build\docfx_yaml
You should see a listing of .yml files like "core_api.package1.someclass.SomeClass.yml".
In the root folder, run:
docfx init -q
This will create a docfx_project folder with the docfx.json configuration file.
Copy
.\build\docfx_yaml\* to .\docfx_project\api\*
Step 10: Build the DocFx HTML and serve the docs on localhost.
Starting in the root folder, run:
docfx docfx_project\docfx.json --serve
Go to http://localhost:8080.
You will note that there are some warnings in the DocFx output for things to address. Mostly invalid links.
Build Cycle
So running through the build once is great, but what about an authoring flow that works for the fix-build-verify cycle. Well that's where it gets messy with both Sphinx and DocFx we've found. We find that cleaning out previous build artifacts and building everything works the best. To this end we create a simple batch file (for Windows) that looks like the following:
REM For best results cleaning out directories works the best del build\* /Q del build\docfx_yaml\* /Q rmdir docfx_project\_site /S /Q rmdir docfx_project\obj\.cache /S /Q del source\core_*.rst /Q del source\modules.rst /Q del source\test_*.rst /Q REM Run sphinx commands sphinx-apidoc -o source .\mycode -f sphinx-build -a source build REM Copy the sphinx generated yaml files to the docfx folder copy .\build\docfx_yaml\* .\docfx_project\api\* REM Build and serve docfx html docfx docfx_project\docfx.json --serve
![[Apocynaceae] Vinca major – {Pervinca maggiore}](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitDKBbupXlF2XY3vEXEB753D9o-eK_VGx0yz290hne-jPqZgf-s_Df8e6MyrM52CNxcxnAJBmKBJV9gOTsy6pls2wzELcNvkjWI2g2yBNEdzAn8NFYmIMxCrSaMTIjVcUus7yT2ejKPN4/s1600/%255BApocynaceae%255D+Vinca+major.jpg "[Apocynaceae] Vinca major – {Pervinca maggiore}")
![[Boraginaceae] Pulmonaria sp. – {Polmonaria}](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhRsLrNd9cKUJ0wmRnmUe5ujkQPzT6dmroHOjmxbIlBJCn67DpLynr12aGowWDlqJc5gYEYOHtzRSdUB344e9qT3rOm8gIoG_YYmHL-07W0y-aPOA0APyKvj_MQ9dnUZEJcDz3_R66wNfY/s1600/%255BBoraginaceae%255D+Pulmonaria+sp..jpg "[Boraginaceae] Pulmonaria sp. – {Polmonaria}")
![[Ranunculaceae] Hepatica nobilis {Erba trinità}](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh58KgWVKMmpzH-gpcX0KIihzedcxRfBkgGsIWlOJhWG559n9H7w1z-dX4gyovry6TuWujzrELsFx4BWQrvwOwZuHcky-yfkRJRIMAkta3A87M7hnIBEx_fd0IlwRRJemGRHDh0Z_skeY8/s1600/%255BRanunculaceae%255D+Hepatica+nobilis.jpg "[Ranunculaceae] Hepatica nobilis {Erba trinità}")
![[Violaceae] Viola odoratum – {Viola mammola}](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjG0u0ObKG0qBM5mZZn0adamMCUy6jPAFRLQxLDY8Ixoo8GDzboxcmXR48_IbmiSUYkYmJQKcBHs4_0cDrwu-iIaRK7dL5oKTfro_rT1wm-Bh2QYx8X6I45iKXiTV5Dkfc0cZEl3wmMBeU/s1600/%255BViolaceae%255D+Viola+odoratum.jpg "[Violaceae] Viola odoratum – {Viola mammola}")
![[Asteraceae] Bellis perennis – {Pratolina comune}](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8u1-bwFdpnLmYF1BW0iIoX5N0jUYCrMaarA2O36cA0XL8wSzveffO3rGcTU8qbjk-A81_yZiqs1mY0YAFt8b5T2e5GJSbMxREtSaXsmBvLeCfft9l2g0mCgcXK6uDu1fah-icYaOGE6Y/s1600/%255BAsteraceae%255D+Bellis+perennis.jpg "[Asteraceae] Bellis perennis – {Pratolina comune}")
![[Euphorbiaceae] Euphorbia helioscopia – {Euforbia calenzuola} - This is just a guess on the species.](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidMZRt3SkWvzOXb21ok_j6RKaAQTwbTUanHlQGuLB0PAfiNRw_DnH8VAwyQUtiAe1YwJO2I3iSm42eW5wZr3yRxh9Jtf56xoyN8CAUzMBTSHZcip5MeGm4EV_-pi973EQZYvur2SI4V_c/s1600/%255BEuphorbiaceae%255D+Euphorbia+helioscopia.jpg "[Euphorbiaceae] Euphorbia helioscopia – {Euforbia calenzuola} - This is just a guess on the species.")
![[Liliaceae] Erythroninum dens-canis – {Dente di cane}](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgeuApdK0dyq-yjoLg61fmzE5iAxzNhAFZdWYIeRqZbGLnBlMlBmTTxGHl_yi9AWt-JhXDOwPqZo7LjnK9SnrnbHF0NzF1e8Btk_WmJHfLBcFWMYFIYa308gTPaoLJ1naS_0P7dCimBHNU/s1600/%255BLiliaceae%255D+Erythroninum+dens-canis+1.jpg "[Liliaceae] Erythroninum dens-canis – {Dente di cane}")
![[Liliaceae] Erythroninum dens-canis – {Dente di cane}](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjy2G8vbxaqV0gIrawSpyq0-B5a3rTCQOcWhO73TrEZq-3UoOzDhMR6axcW4NgZwhR7GkyL_-wUlJ6BMP0g1WlZT_S-_u6mb9thSQ4n0_DiwttmAouDYm-Sf344TZAUeqjLZC2LvqVf9tk/s1600/%255BLiliaceae%255D+Erythroninum+dens-canis+2.jpg "[Liliaceae] Erythroninum dens-canis – {Dente di cane}")
![[Polygalaceae] Polygala chamaebuxus – {Poligala falso bosso}](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4D3y7wkKbE7dqxz_f9BgFmWikTdyqHw-ZQdE76Cjb2nocPgdxWYHoP_GiZ2YTBtV-KQlURVY_vFLF5jCrWQu2PY51o4MZy6vOXEVu70V9pRczHkx9njo_qJIIqAhkf2wfQHT2wMQ_cKM/s1600/%255BPolygalaceae%255D+Polygala+chamaebuxus+1.jpg "[Polygalaceae] Polygala chamaebuxus – {Poligala falso bosso}")
![[Polygalaceae] Polygala chamaebuxus – {Poligala falso bosso}](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBgr6Jsjapk58I_TyMXI8OgsZfbvg8SNopcotf-pnzNcMN1Uh2wZLZGcHNAfRhSTz3HKjX8XQnttxou1rO_kyhw0h1rp0GaLKIBVj18TKmt5O13zn2w93RCpLd2ZaKIrPCZGLVrKvy4L4/s1600/%255BPolygalaceae%255D+Polygala+chamaebuxus+2.jpg "[Polygalaceae] Polygala chamaebuxus – {Poligala falso bosso}")
![[Primulaceae] Primula vulgaris – {Primula comune}](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivOHLniIQyj_RVHjGYXRj5rGLrvPLF1JSq_tXtabRNAZbFSxJaFucZ6lGc5j-1NFhQK8d1QWgThu8OMR2Tw5Jcq8qwt4Yz_VW3kcsGs-1yMH3XxbOT0qOSszOOctpM5HbtDKy0KTnAJuA/s1600/%255BPrimulaceae%255D+Primula+vulgaris.jpg "[Primulaceae] Primula vulgaris – {Primula comune}")
![[Ranunculaceae] Anemone nemorosa – {Anenome bianca}](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvMSQxjVDvRr12gk76GTG-uyD-3xJt8ehP3QgXA6A7rltBQTKG-OUTl8J9UnznawJndY0zMxHXhBzvGuygA_nkrcVPW4mqP-1YOf5hXfMj2bC6X2cb68A_AwWvq-5_DP8iBDfsZm2Kn8E/s1600/%255BRanunculaceae%255D+Anemone+nemorosa.jpg "[Ranunculaceae] Anemone nemorosa – {Anenome bianca}")
