[PATCH v2 0/5] make htmldocs in CI

This patch series provides a htmldocs target for Travis, Gitlab and Azure CI.
doc/sphinx/kerneldoc.py is updated to avoid a build warning with current Sphinx.
The last patch turns build warnings of 'make htmldocs' into errors.
While first four patches can be immediately merged the last patch requires an updated Docker image, cf.
[PATCH 1/1] Dockerfile: add imagemagick https://lists.denx.de/pipermail/u-boot/2020-February/400983.html
v2: update update doc/sphinx/kerneldoc.py change sequence of targets in Travis Ci change a commit message
Heinrich Schuchardt (5): doc: update doc/sphinx/kerneldoc.py travis: build HTML docs gitlab: build HTML documentation azure: build HTML documentation doc/Makefile: turn warnings into errors
.azure-pipelines.yml | 10 ++++++++ .gitlab-ci.yml | 7 ++++++ .travis.yml | 5 ++++ doc/Makefile | 1 + doc/sphinx/kerneldoc.py | 51 +++++++++++++++++++++++++++++++---------- 5 files changed, 62 insertions(+), 12 deletions(-)
-- 2.25.0

Update doc/sphinx/kerneldoc.py from Linux next-20200219 to avoid warnings like:
doc/sphinx/kerneldoc.py:125: RemovedInSphinx20Warning: AutodocReporter is now deprecated. Use sphinx.util.docutils.switch_source_input() instead. self.state.memo.reporter = AutodocReporter(result, self.state.memo.reporter)
Signed-off-by: Heinrich Schuchardt xypron.glpk@gmx.de --- v2: new patch --- doc/sphinx/kerneldoc.py | 51 +++++++++++++++++++++++++++++++---------- 1 file changed, 39 insertions(+), 12 deletions(-)
diff --git a/doc/sphinx/kerneldoc.py b/doc/sphinx/kerneldoc.py index e536360de1..4bcbd6ae01 100644 --- a/doc/sphinx/kerneldoc.py +++ b/doc/sphinx/kerneldoc.py @@ -37,7 +37,17 @@ import glob from docutils import nodes, statemachine from docutils.statemachine import ViewList from docutils.parsers.rst import directives, Directive -from sphinx.ext.autodoc import AutodocReporter + +# +# AutodocReporter is only good up to Sphinx 1.7 +# +import sphinx + +Use_SSI = sphinx.__version__[:3] >= '1.7' +if Use_SSI: + from sphinx.util.docutils import switch_source_input +else: + from sphinx.ext.autodoc import AutodocReporter
import kernellog
@@ -49,9 +59,10 @@ class KernelDocDirective(Directive): optional_arguments = 4 option_spec = { 'doc': directives.unchanged_required, - 'functions': directives.unchanged_required, 'export': directives.unchanged, 'internal': directives.unchanged, + 'identifiers': directives.unchanged, + 'functions': directives.unchanged, } has_content = False
@@ -67,6 +78,10 @@ class KernelDocDirective(Directive):
tab_width = self.options.get('tab-width', self.state.document.settings.tab_width)
+ # 'function' is an alias of 'identifiers' + if 'functions' in self.options: + self.options['identifiers'] = self.options.get('functions') + # FIXME: make this nicer and more robust against errors if 'export' in self.options: cmd += ['-export'] @@ -76,9 +91,13 @@ class KernelDocDirective(Directive): export_file_patterns = str(self.options.get('internal')).split() elif 'doc' in self.options: cmd += ['-function', str(self.options.get('doc'))] - elif 'functions' in self.options: - for f in str(self.options.get('functions')).split(): - cmd += ['-function', f] + elif 'identifiers' in self.options: + identifiers = self.options.get('identifiers').split() + if identifiers: + for i in identifiers: + cmd += ['-function', i] + else: + cmd += ['-no-doc-sections']
for pattern in export_file_patterns: for f in glob.glob(env.config.kerneldoc_srctree + '/' + pattern): @@ -121,13 +140,7 @@ class KernelDocDirective(Directive): lineoffset += 1
node = nodes.section() - buf = self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter - self.state.memo.reporter = AutodocReporter(result, self.state.memo.reporter) - self.state.memo.title_styles, self.state.memo.section_level = [], 0 - try: - self.state.nested_parse(result, 0, node, match_titles=1) - finally: - self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = buf + self.do_parse(result, node)
return node.children
@@ -136,6 +149,20 @@ class KernelDocDirective(Directive): (" ".join(cmd), str(e))) return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))]
+ def do_parse(self, result, node): + if Use_SSI: + with switch_source_input(self.state, result): + self.state.nested_parse(result, 0, node, match_titles=1) + else: + save = self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter + self.state.memo.reporter = AutodocReporter(result, self.state.memo.reporter) + self.state.memo.title_styles, self.state.memo.section_level = [], 0 + try: + self.state.nested_parse(result, 0, node, match_titles=1) + finally: + self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = save + + def setup(app): app.add_config_value('kerneldoc_bin', None, 'env') app.add_config_value('kerneldoc_srctree', None, 'env') -- 2.25.0

On Fri, Feb 21, 2020 at 06:23:59PM +0100, Heinrich Schuchardt wrote:
Update doc/sphinx/kerneldoc.py from Linux next-20200219 to avoid warnings like:
doc/sphinx/kerneldoc.py:125: RemovedInSphinx20Warning: AutodocReporter is now deprecated. Use sphinx.util.docutils.switch_source_input() instead. self.state.memo.reporter = AutodocReporter(result, self.state.memo.reporter)
Signed-off-by: Heinrich Schuchardt xypron.glpk@gmx.de
Applied to u-boot/master, thanks!

Several patches delivered incorrect restructured text as documentation. We should be able to discover this in Travis CI.
Provide a build step for 'make htmldocs'.
Add required package graphviz.
Signed-off-by: Heinrich Schuchardt xypron.glpk@gmx.de --- v2: change sequence of targets: put htmldocs after cppcheck --- .travis.yml | 5 +++++ 1 file changed, 5 insertions(+)
diff --git a/.travis.yml b/.travis.yml index e6db9d6a72..911c85e130 100644 --- a/.travis.yml +++ b/.travis.yml @@ -38,6 +38,7 @@ addons: - libisl15 - clang-7 - srecord + - graphviz
install: # Clone uboot-test-hooks @@ -352,6 +353,10 @@ matrix: - name: "cppcheck" script: - cppcheck --force --quiet --inline-suppr . + # build HTML documentation + - name: "htmldocs" + script: + - make htmldocs # search for TODO within source tree - name: "grep TODO" script: -- 2.25.0

On Sat, Feb 22, 2020 at 1:24 AM Heinrich Schuchardt xypron.glpk@gmx.de wrote:
Several patches delivered incorrect restructured text as documentation. We should be able to discover this in Travis CI.
Provide a build step for 'make htmldocs'.
Add required package graphviz.
Signed-off-by: Heinrich Schuchardt xypron.glpk@gmx.de
v2: change sequence of targets: put htmldocs after cppcheck
.travis.yml | 5 +++++ 1 file changed, 5 insertions(+)
Reviewed-by: Bin Meng bmeng.cn@gmail.com

On Fri, Feb 21, 2020 at 06:24:00PM +0100, Heinrich Schuchardt wrote:
Several patches delivered incorrect restructured text as documentation. We should be able to discover this in Travis CI.
Provide a build step for 'make htmldocs'.
Add required package graphviz.
Signed-off-by: Heinrich Schuchardt xypron.glpk@gmx.de Reviewed-by: Bin Meng bmeng.cn@gmail.com
Applied to u-boot/master, thanks!

Several patches delivered incorrect restructured text as documentation. We should be able to discover this in Gitlab CI.
Provide a build step for 'make htmldocs'.
Signed-off-by: Heinrich Schuchardt xypron.glpk@gmx.de Reviewed-by: Bin Meng bmeng.cn@gmail.com --- v2: no change --- .gitlab-ci.yml | 7 +++++++ 1 file changed, 7 insertions(+)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index d486e72042..652ed3b501 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -122,6 +122,13 @@ grep TODO/FIXME/HACK: # search for HACK within source tree and ignore HACKKIT board - grep -r HACK . | grep -v HACKKIT
+# build HTML documentation +htmldocs: + tags: [ 'all' ] + stage: testsuites + script: + - make htmldocs + # some statistics about the code base sloccount: tags: [ 'all' ] -- 2.25.0

On Fri, Feb 21, 2020 at 06:24:01PM +0100, Heinrich Schuchardt wrote:
Several patches delivered incorrect restructured text as documentation. We should be able to discover this in Gitlab CI.
Provide a build step for 'make htmldocs'.
Signed-off-by: Heinrich Schuchardt xypron.glpk@gmx.de Reviewed-by: Bin Meng bmeng.cn@gmail.com
Applied to u-boot/master, thanks!

Several patches delivered incorrect restructured text as documentation. We should be able to discover this in Azure CI.
Provide a build step for 'make htmldocs'.
Signed-off-by: Heinrich Schuchardt xypron.glpk@gmx.de Reviewed-by: Bin Meng bmeng.cn@gmail.com --- v2: no change --- .azure-pipelines.yml | 10 ++++++++++ 1 file changed, 10 insertions(+)
diff --git a/.azure-pipelines.yml b/.azure-pipelines.yml index c22095830c..129035a1a5 100644 --- a/.azure-pipelines.yml +++ b/.azure-pipelines.yml @@ -54,6 +54,16 @@ jobs: steps: - script: cppcheck --force --quiet --inline-suppr .
+ - job: htmldocs + displayName: 'Build HTML documentation' + pool: + vmImage: $(ubuntu_vm) + container: + image: $(ci_runner_image) + options: $(container_option) + steps: + - script: make htmldocs + - job: todo displayName: 'Search for TODO within source tree' pool: -- 2.25.0

On Fri, Feb 21, 2020 at 06:24:02PM +0100, Heinrich Schuchardt wrote:
Several patches delivered incorrect restructured text as documentation. We should be able to discover this in Azure CI.
Provide a build step for 'make htmldocs'.
Signed-off-by: Heinrich Schuchardt xypron.glpk@gmx.de Reviewed-by: Bin Meng bmeng.cn@gmail.com
Applied to u-boot/master, thanks!

Several patches delivered incorrect restructured text as documentation. We should be able to discover this in Travis CI, Gitlab CI, or Azure CI.
So let us turn all build warnings into errors.
Signed-off-by: Heinrich Schuchardt xypron.glpk@gmx.de --- v2: change commit message --- doc/Makefile | 1 + 1 file changed, 1 insertion(+)
diff --git a/doc/Makefile b/doc/Makefile index 5135a96e88..0e0da5666f 100644 --- a/doc/Makefile +++ b/doc/Makefile @@ -56,6 +56,7 @@ quiet_cmd_sphinx = SPHINX $@ --> file://$(abspath $(BUILDDIR)/$3/$4) PYTHONDONTWRITEBYTECODE=1 \ BUILDDIR=$(abspath $(BUILDDIR)) SPHINX_CONF=$(abspath $(srctree)/$(src)/$5/$(SPHINX_CONF)) \ $(SPHINXBUILD) \ + -W \ -b $2 \ -c $(abspath $(srctree)/$(src)) \ -d $(abspath $(BUILDDIR)/.doctrees/$3) \ -- 2.25.0

On Sat, Feb 22, 2020 at 1:25 AM Heinrich Schuchardt xypron.glpk@gmx.de wrote:
Several patches delivered incorrect restructured text as documentation. We should be able to discover this in Travis CI, Gitlab CI, or Azure CI.
So let us turn all build warnings into errors.
Signed-off-by: Heinrich Schuchardt xypron.glpk@gmx.de
v2: change commit message
doc/Makefile | 1 + 1 file changed, 1 insertion(+)
Reviewed-by: Bin Meng bmeng.cn@gmail.com

On Fri, Feb 21, 2020 at 06:24:03PM +0100, Heinrich Schuchardt wrote:
Several patches delivered incorrect restructured text as documentation. We should be able to discover this in Travis CI, Gitlab CI, or Azure CI.
So let us turn all build warnings into errors.
Signed-off-by: Heinrich Schuchardt xypron.glpk@gmx.de Reviewed-by: Bin Meng bmeng.cn@gmail.com
Applied to u-boot/master, thanks!
participants (3)
-
Bin Meng
-
Heinrich Schuchardt
-
Tom Rini