[U-Boot] [RFC 0/3] uboot-doc User's Manual Generation Tool
Hello all,
I've been working on writing a User's Manual generation tool, and before I get too far I'd like to see if this is something the U-Boot community would be interested in.
The thought was that everyone needs to develop User's Manuals for their customers, and there's no reason for us all to reinvent the wheel for each product we create. I've written a Perl script, uboot-doc, which is very loosely based on the kernel-doc documentation creator in Linux. (Take it easy on my Perl, this is my first script.) The specific syntax is described in uboot-doc, but the parser takes formatted comments with @'s and converts the information into DocBook XML. The DocBook XML can be used to generate HTML, XHTML, PDFs, and text, and can be extended to change formatting and appearance. The static portions of the manual are taken from .tmpl files in the doc/manual/ directory, which have basic precompiler support (#ifdef, #else, #endif) and support for accessing system environment variables (${TOPDIR}, for example) and any value found in autoconf.mk, (@CONFIG_BAUDRATE, for example). All template files in doc/manual/ can be over-ridden by a board specific file in board/$BOARDDIR/manual. For example, doc/manual/manual.tmpl would be over-ridden by board/$BOARDDIR/manual/manual.tmpl.
General Control Flow: C pre-processor -> doc_stream.pl : pre-processed code gets everything but comments with an @ stripped away doc_stream.pl -> autoconf_doc.txt : "interesting" comments get sent to this file autoconf_doc.txt -> uboot-doc : file is parsed and comments are turned to XML *.tmpl -> uboot-doc : template files are processed into XML *.xml -> xsltproc : XML is converted into desired output format *.fo -> fop : (PDFs only) fo files are translated to PDF
Steps to compile your very own manual (using Ubuntu package names): 1) Install xsltproc. I'm using version 1.1.24-2ubuntu2. If you've got no desire to make text, html, xhtml, or pdf you don't need this tool.
2) Install docbook-xsl. I'm using version 1.73.2.dfsg.1-5. If you've got no desire to make text, html, xhtml, or pdf you don't need this tool.
3) FOP needs to be installed with hyphenation support, which is not available from Synaptic or apt-get. Use the instructions from http://www.linuxfromscratch.org/blfs/view/svn/pst/fop.html to install your version of FOP (yes, you do need OFFO and JAI). You will need to install ant if you don't already have it (1.7.1-0ubuntu2). If you've got no desire to make a PDF, you don't need this tool.
4) If you follow the steps above, the default values of DOCBOOK_XSL_DIR, XSLT_PROCESSOR, and FO_PROCESSOR should be correct. If you decided to try to go it your own, set them accordingly. DOCBOOK_XSL_DIR points to your Docbook stylesheet directory, XSLT_PROCESSOR points to the default program which compiles XML into it's various forms, and FO_PROCESSOR points the the default program for turning FO images into PDFs.
5) Once you have the tools installed, you should be able to compile a manual. This documentation feature was tested on the X-ES's XPedite5370. make XPEDITE5370_config make {xml/html/xhtml/pdf/text} Note: text doesn't work yet as I believe there is an issue with the docbook style sheets (or possibly how I'm using them). It's being looked into. Something about varlistentries not belonging in varlists? I dunno.
The doc/manual directory also includes a PDF style sheet (uboot_manual.xsl). By creating your own book_info.tmpl, you can add a company logo and an address to the cover page. By default these two are left out. It is also possible to create a completely new style sheet to customize your manual look. Similarly, HTML output can be customized using CSS. The results of this output as well as the HTML output can be found on the X-ES website: www.xes-inc.com/sources/u-boot/index.html www.xes-inc.com/sources/u-boot/xpedite5370.pdf Note that I've defined a "mediaobject" and "address" in my custom board/xes/xpedite5370/book_info.tmpl to place the address and company logo on the cover page.
I see several advantages to adopting this scheme. - Documentation should be easier to keep in sync with code - DocBook XML output can be used to generate webpages, PDFs, text, etc, which are all extensible - People don't have to reinvent the wheel when writing documentation - Source code ends up being thouroughly commented
These patches are just meant to be an example of how in-code documentation could be used. You'll also notice there are many warnings regarding variables the manual is referencing which aren't defined yet. I wanted to get some feedback before diving in too far. What do others think? Is this worth pursuing?
Thanks, John
John Schmoller (3): uboot-doc: Initial support of user documentation generator uboot-doc: Add example support for uboot-doc xpedite5370: Add uboot-doc support
Makefile | 2 + board/xes/xpedite5370/manual/book_info.tmpl | 52 ++ board/xes/xpedite5370/manual/booting_linux.tmpl | 179 ++++ board/xes/xpedite5370/manual/booting_vxworks.tmpl | 166 ++++ board/xes/xpedite5370/manual/manual.tmpl | 30 + board/xes/xpedite5370/manual/redundant_images.tmpl | 50 ++ board/xes/xpedite5370/manual/scripting.tmpl | 168 ++++ common/cmd_i2c.c | 71 ++- common/cmd_mem.c | 45 +- common/cmd_net.c | 30 +- common/env_common.c | 35 + config.mk | 1 + doc/manual/85xx_program_flow.tmpl | 74 ++ doc/manual/book_info.tmpl | 35 + doc/manual/booting_linux.tmpl | 249 ++++++ doc/manual/config.mk | 74 ++ doc/manual/doc-stream.pl | 34 + doc/manual/introduction.tmpl | 136 +++ doc/manual/manual.tmpl | 26 + doc/manual/scripting.tmpl | 154 ++++ doc/manual/setup.tmpl | 294 +++++++ doc/manual/uboot-doc | 866 ++++++++++++++++++++ doc/manual/uboot_manual.xsl | 286 +++++++ include/configs/XPEDITE5370.h | 47 ++ post/tests.c | 16 + rules.mk | 7 + 26 files changed, 3122 insertions(+), 5 deletions(-) create mode 100644 board/xes/xpedite5370/manual/book_info.tmpl create mode 100644 board/xes/xpedite5370/manual/booting_linux.tmpl create mode 100644 board/xes/xpedite5370/manual/booting_vxworks.tmpl create mode 100644 board/xes/xpedite5370/manual/manual.tmpl create mode 100644 board/xes/xpedite5370/manual/redundant_images.tmpl create mode 100644 board/xes/xpedite5370/manual/scripting.tmpl create mode 100644 doc/manual/85xx_program_flow.tmpl create mode 100644 doc/manual/book_info.tmpl create mode 100644 doc/manual/booting_linux.tmpl create mode 100644 doc/manual/config.mk create mode 100644 doc/manual/doc-stream.pl create mode 100644 doc/manual/introduction.tmpl create mode 100644 doc/manual/manual.tmpl create mode 100644 doc/manual/scripting.tmpl create mode 100644 doc/manual/setup.tmpl create mode 100644 doc/manual/uboot-doc create mode 100644 doc/manual/uboot_manual.xsl
Dear John Schmoller,
In message cover.1248798202.git.jschmoller@xes-inc.com you wrote:
I've been working on writing a User's Manual generation tool, and before I get too far I'd like to see if this is something the U-Boot community would be interested in.
Anything wrong with the User's Manual generation tool we already have in use (DUTS) for example to generate the DULG?
General Control Flow: C pre-processor -> doc_stream.pl : pre-processed code gets everything but comments with an @ stripped away doc_stream.pl -> autoconf_doc.txt : "interesting" comments get sent to this file autoconf_doc.txt -> uboot-doc : file is parsed and comments are turned to XML *.tmpl -> uboot-doc : template files are processed into XML *.xml -> xsltproc : XML is converted into desired output format *.fo -> fop : (PDFs only) fo files are translated to PDF
What you describe does not sound like a user's manual to me, but more like some API documentation - whioch is a completely different thing (and something we really don't have yet).
Your approach may be suitable for standard documents, but in many years of working with U-Boot (and Linux) we found that it does not work so well with the specific needs we have for a User's Manual. One issue is that you have to support many different boards (well, maybe not you as a user, but we as a community). And you have to include examples. And examples must really fit the board. If you for example provide a manual entry for the "erase" command you better make sure that your example does not erase a range of flash that on some board happens to hold the U-Boot image. etc.
That's how we came to the DULG we have today. I'm still convinced that this is a really well-working approach.
It seems you don;t address this issue yet.
I see several advantages to adopting this scheme.
- Documentation should be easier to keep in sync with code
Better than what we have with the DULG? I doubt it.
- DocBook XML output can be used to generate webpages, PDFs, text, etc, which are all extensible
We do the same with the DULG.
- People don't have to reinvent the wheel when writing documentation
Well, you just did that, it seems ;-)
- Source code ends up being thouroughly commented
These patches are just meant to be an example of how in-code documentation could be used. You'll also notice there are many warnings regarding variables the manual is referencing which aren't defined yet. I wanted to get some feedback before diving in too far. What do others think? Is this worth pursuing?
Hm... we cannot look at your patches, you just posted the patch statistics, but no content.
The documentation itself seems to duplicate a lot of content we have in the DULG. I have to admit that I'd prefer to see the effort invested in improving the existing manual, than to come up with something different (and, as it seems, less flexible).
Best regards,
Wolfgang Denk
On Tue, 2009-07-28 at 19:49 +0200, Wolfgang Denk wrote:
Dear John Schmoller,
In message cover.1248798202.git.jschmoller@xes-inc.com you wrote:
I've been working on writing a User's Manual generation tool, and before I get too far I'd like to see if this is something the U-Boot community would be interested in.
Anything wrong with the User's Manual generation tool we already have in use (DUTS) for example to generate the DULG?
I have to admit, I'm not very familiar with DUTS, so read up a little on it before responding. So please correct me if I've misinterpreted something.
It seems to me that DUTS is designed to test U-Boot and also automates the running of commands whose output can be put online in the DULG. I didn't notice any documented procedure on how to turn the DULG into XML, extensible PDFs, etc. It also seems as if the DULG is a combination of hand-edited wiki pages as well as the DUTS output? I was hoping to develop a system that's completely automated. I also noticed in a quick probe around that a few items in the DULG seem to be out of sync (imd vs. i2c md, source vs. autoscr, etc.) and DHCP seems to be out of date as well. Is the process for updating the DULG automatic? If so, how is it done?
General Control Flow: C pre-processor -> doc_stream.pl : pre-processed code gets everything but comments with an @ stripped away doc_stream.pl -> autoconf_doc.txt : "interesting" comments get sent to this file autoconf_doc.txt -> uboot-doc : file is parsed and comments are turned to XML *.tmpl -> uboot-doc : template files are processed into XML *.xml -> xsltproc : XML is converted into desired output format *.fo -> fop : (PDFs only) fo files are translated to PDF
What you describe does not sound like a user's manual to me, but more like some API documentation - whioch is a completely different thing (and something we really don't have yet).
As I mentioned, I borrowed this idea from the kernel-doc script in Linux, which does do API documentation. But my hope for the uboot-doc tool would be to create user documentation, or a manual we'd provide to a customer when they purchased a product that would describe available commands, environment variables, common operations, etc.
Your approach may be suitable for standard documents, but in many years of working with U-Boot (and Linux) we found that it does not work so well with the specific needs we have for a User's Manual. One issue is that you have to support many different boards (well, maybe not you as a user, but we as a community). And you have to include examples. And examples must really fit the board. If you for example provide a manual entry for the "erase" command you better make sure that your example does not erase a range of flash that on some board happens to hold the U-Boot image. etc.
It's my hope that the documentation system I proposed can be made flexible enough to support things such as this without too much headache. That's the hope at least :)
That's how we came to the DULG we have today. I'm still convinced that this is a really well-working approach.
It seems you don;t address this issue yet.
That's correct, I wanted to get feedback before spending too much time digging into the details.
I see several advantages to adopting this scheme.
- Documentation should be easier to keep in sync with code
Better than what we have with the DULG? I doubt it.
- DocBook XML output can be used to generate webpages, PDFs, text, etc, which are all extensible
We do the same with the DULG.
This would make DUTS/DULG more appealing. What is the process of generating DocBook XML from DUTS/DULG?
- People don't have to reinvent the wheel when writing documentation
Well, you just did that, it seems ;-)
It's true, I may have. :) On the other hand, it seems that there are still a lot of people who are in the same boat. Most manuals I have seen from other embedded companies (Freescale, Analog Devices, etc.) seem to provide their own format/content. Additionally, most companies will prefer to have their content formatted to match the rest of their manuals, which is easily done from DocBook XML. Again, if you can do that with DUTS/DULG, then that's great and probably eliminates the need for this tool.
- Source code ends up being thouroughly commented
These patches are just meant to be an example of how in-code documentation could be used. You'll also notice there are many warnings regarding variables the manual is referencing which aren't defined yet. I wanted to get some feedback before diving in too far. What do others think? Is this worth pursuing?
Hm... we cannot look at your patches, you just posted the patch statistics, but no content.
This is just the patch series introduction, the actual content appears in 1-3.
The documentation itself seems to duplicate a lot of content we have in the DULG. I have to admit that I'd prefer to see the effort invested in improving the existing manual, than to come up with something different (and, as it seems, less flexible).
As I mentioned, I'm not all that familiar with the abilities of DUTS/DULG, but I got the impression that there was still a fair amount of manual labor involved for each manual. Thus most people (that I'm familiar with) provide their own documentation and "reinvent the wheel". I was hoping that creating documentation from comments in the source would be easier to maintain and providing DocBook output would allow others to more easily reuse the U-Boot manual contents.
Thanks for your time, John
Dear John,
in message 1248813631.3915.102.camel@johns you wrote:
It seems to me that DUTS is designed to test U-Boot and also automates the running of commands whose output can be put online in the DULG. I
Correct. And the DULG itself is a wiki (TWiki at the moment, to be converted to Foswiki ASAP) based framework which holds the "static" parts of the documentation.
didn't notice any documented procedure on how to turn the DULG into XML, extensible PDFs, etc. It also seems as if the DULG is a combination of hand-edited wiki pages as well as the DUTS output? I was hoping to
You probably can export the DULG into XML, too - but what would that be good for? At the moment we export it into PDF, PostScript, HTML and plain text.
develop a system that's completely automated. I also noticed in a quick
Hm... "completely automated" is a nice buzzword, but you still have to write the documentation in the first place.
The nice thing of the wiki based approach is that you can have a (even random) collection of pages which can be "linearized" and brought into arbitrary sets of linear doucumentation - this is what the WebOrder pages are good for - see http://www.denx.de/wiki/DULG/WebOrder
So you can use the same set of wiki pages an genreate for example the full fledged manual, a short version including only the most significant topics and an extended version including other stuff that is normally not part of the manual - all from the same envrionment.
probe around that a few items in the DULG seem to be out of sync (imd vs. i2c md, source vs. autoscr, etc.) and DHCP seems to be out of date as well. Is the process for updating the DULG automatic? If so, how is it done?
Soemone still has to write the documentation - and this is not always done in sync with the code. Your approach of including the documentation with the source code makes it more likely that one remebers to update the docs as well, but just remember how many places there are where code and comments don't agree - it's no guarantee either.
If you find such areas in the DULG that are out of sync or missing please feel free to add them - everybody can contribute and modify the pages or add new content. And everyboidy can modify and extend the DUTS test cases, too.
As I mentioned, I borrowed this idea from the kernel-doc script in Linux, which does do API documentation. But my hope for the uboot-doc tool would be to create user documentation, or a manual we'd provide to a customer when they purchased a product that would describe available commands, environment variables, common operations, etc.
That's what the DULG does, so you are truely duplicating efforts.
Your approach may be suitable for standard documents, but in many years of working with U-Boot (and Linux) we found that it does not work so well with the specific needs we have for a User's Manual. One issue is that you have to support many different boards (well, maybe not you as a user, but we as a community). And you have to include examples. And examples must really fit the board. If you for example provide a manual entry for the "erase" command you better make sure that your example does not erase a range of flash that on some board happens to hold the U-Boot image. etc.
It's my hope that the documentation system I proposed can be made flexible enough to support things such as this without too much headache. That's the hope at least :)
Heh. Look at the DULG implementation... it's simple for the first 3 or 4 boards, especially when these are similar. But try supporting ARM and PowerPC and MIPS boards from N different vendors, with different kernel versions etc.
This would make DUTS/DULG more appealing. What is the process of generating DocBook XML from DUTS/DULG?
What would you need DocBook XML for?
It's true, I may have. :) On the other hand, it seems that there are still a lot of people who are in the same boat. Most manuals I have seen from other embedded companies (Freescale, Analog Devices, etc.) seem to provide their own format/content. Additionally, most companies will prefer to have their content formatted to match the rest of their manuals, which is easily done from DocBook XML. Again, if you can do that with DUTS/DULG, then that's great and probably eliminates the need for this tool.
Ah, I see.
Well, we use htmldoc to generate .pdf and .ps files from HTML; you could for example use Tidy to convert HTML to DocBook (and I'm sure there are lots of other tools that can do that).
Hm... we cannot look at your patches, you just posted the patch statistics, but no content.
This is just the patch series introduction, the actual content appears in 1-3.
Yes, I know. Sorry, patches arrived shortly after I started replying, and I didn;t notice.
As I mentioned, I'm not all that familiar with the abilities of DUTS/DULG, but I got the impression that there was still a fair amount of manual labor involved for each manual. Thus most people (that I'm
Indeed there is a lot of manual labor involved in preparing and updating the documentation (the framework, that is; once a board has been configured and added to DUTS, generating the needed include files is mostly an automatic procedure - just the upload of the files to the web server has to be triggered manually, but even this is scripted).
But you have to spend at least the same amount of writing and updating the documentation, don't you?
I was hoping that creating documentation from comments in the source would be easier to maintain and providing DocBook output would allow others to more easily reuse the U-Boot manual contents.
A very, very old version of the DULG was set up like this, using a set of Perl scripts and generating DokBoot output. We switched to the wiki based approach some 6 years ago, and I never had the slightest wish to switch back.
Best regards,
Wolfgang Denk
On Tue 28 Jul 2009 17:27, Wolfgang Denk pondered:
Dear John,
in message 1248813631.3915.102.camel@johns you wrote:
It seems to me that DUTS is designed to test U-Boot and also automates the running of commands whose output can be put online in the DULG. I
Correct. And the DULG itself is a wiki (TWiki at the moment, to be converted to Foswiki ASAP) based framework which holds the "static" parts of the documentation.
While we are on the topics of wikis, and U-Boot docs, can we discuss the license?
http://www.denx.de/wiki/view/DULG/Introduction#Section_2.1.
Since it is "is not permitted to sell this document or the derivative work or to include it into any package or distribution that is not freely available to everybody."
Not meaning to interpret licensing, but to me that means I can't copy/paste sections into end product documentation, and ship the product for a fee...
Was that the intent?
Hi Robin,
On Tue 28 Jul 2009 17:27, Wolfgang Denk pondered:
Dear John,
in message 1248813631.3915.102.camel@johns you wrote:
It seems to me that DUTS is designed to test U-Boot and also automates the running of commands whose output can be put online in the DULG. I
Correct. And the DULG itself is a wiki (TWiki at the moment, to be converted to Foswiki ASAP) based framework which holds the "static" parts of the documentation.
While we are on the topics of wikis, and U-Boot docs, can we discuss the license?
http://www.denx.de/wiki/view/DULG/Introduction#Section_2.1.
Since it is "is not permitted to sell this document or the derivative work or to include it into any package or distribution that is not freely available to everybody."
Not meaning to interpret licensing, but to me that means I can't copy/paste sections into end product documentation, and ship the product for a fee...
That's how it is currently, yes.
Was that the intent?
Not really - but I'll better let Wolfgang answer that question as he came up with the original licence text.
It is on my priority list however to assemble a list of all contributors and ask them if they agree to relicence their contributions under the FDL.
Cheers Detlev
Hi,
In message m2eiryebkk.fsf@ohwell.denx.de Detlev Zundel wrote: ...
Not meaning to interpret licensing, but to me that means I can't copy/paste sections into end product documentation, and ship the product for a fee...
That's how it is currently, yes.
Was that the intent?
Not really - but I'll better let Wolfgang answer that question as he came up with the original licence text.
Originally this actualy has been my intent. But that was a long, long time ago, back when I was writing all these documentation only by myself.
It is on my priority list however to assemble a list of all contributors and ask them if they agree to relicence their contributions under the FDL.
Consider my permission as granted :-)
Best regards,
Wolfgang Denk
On Thu 30 Jul 2009 14:45, Wolfgang DenkVersion 1.3 pondered:
Hi,
In message m2eiryebkk.fsf@ohwell.denx.de Detlev Zundel wrote: ...
Not meaning to interpret licensing, but to me that means I can't copy/paste sections into end product documentation, and ship the product for a fee...
That's how it is currently, yes.
Was that the intent?
Not really - but I'll better let Wolfgang answer that question as he came up with the original licence text.
Originally this actualy has been my intent. But that was a long, long time ago, back when I was writing all these documentation only by myself.
It is on my priority list however to assemble a list of all contributors and ask them if they agree to relicence their contributions under the FDL.
Consider my permission as granted :-)
I assume - no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts? or did you desire something else?
Since Front-Cover Text is limited to 5 words, and Back-Cover Text is limited to 25 words - how about (as Back-Cover Text) "This manual includes sections from the Das U-Boot documentation, which can be found at http://www.denx.de/wiki/U-Boot and is copyright it's authors. Included under the Free Documentation License (v1.3)"
?
Dear Robin Getz,
In message 200907301550.40651.rgetz@blackfin.uclinux.org you wrote:
I assume - no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts? or did you desire something else?
We don't have such detailed plans yet. I see no need for Invariant Sections.
Since Front-Cover Text is limited to 5 words, and Back-Cover Text is limited to 25 words - how about (as Back-Cover Text) "This manual includes sections from the Das U-Boot documentation, which can be found at http://www.denx.de/wiki/U-Boot and is copyright it's authors. Included under the Free Documentation License (v1.3)"
?
Maybe something like that. Thanks for the proposal :-)
Best regards,
Wolfgang Denk
On Thu 30 Jul 2009 15:55, Wolfgang Denk pondered:
Dear Robin Getz,
In message 200907301550.40651.rgetz@blackfin.uclinux.org you wrote:
I assume - no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts? or did you desire something else?
We don't have such detailed plans yet.
Depending on the exact final plans, Analog Devices would be happy to donate anything from the U-Boot documentation we have created over the past few years....
https://docs.blackfin.uclinux.org/doku.php?id=bootloaders:u-boot
While the examples are Blackfin specific, most should be generic enough to ensure that the reader should understand what to do on their architecture. We see maintaining something separate a duplication of effort, and wasted resources spent on documentation creation (which is a task most developers don't like anyways)...
I think Mike brought this up awhile ago - http://www.mail-archive.com/u-boot-users@lists.sourceforge.net/msg04491.html
Which gets back to the original question...
On Thu, 17 Apr 2008 11:02:18 -0700 Mike Frysinger pondered:
On Thursday 17 April 2008, Detlev Zundel wrote:
Hi Mike,
On Monday 14 April 2008, Detlev Zundel wrote:
we maintain a Blackfin-specific u-boot wiki that goes into quite a bit of detail, some of which is duplicated with the main u-boot wiki. how do people feel about extending the u-boot wiki to allow for arch-specific details ?
What exactly do you have in mind? I surely don't see any principal problem here.
It would certainly be valuable to get all U-Boot related info collected in a central place and have pointers wherever that make sense...
from my reading of the wiki, it's more of a technical/command reference than a guide. the wiki we maintain is geared to be more of a guide. i think the two can be merged, i just dont want to convert things only to find out people dont want to take it that direction.
Just to be clear, we are discussing the DULG wiki, right?
is there any other worth talking about :)
I agree that in the current state the documentation is more a reference but IIRC that wasn't really a conscious design decision. It simply turned out this way in the end.
So I do not see any general problem in adding "guide style" sections in there. Maybe then most of the current documentation can then be shifted to a "commands reference" section.
OK
One problem I see though is how to correctly adapt such sections to the board specific nature of the DULG. Hopefully we can get away with mostly generic text passages and only a few ifdefs. It would be very helpful to know more concrete plans (outline!) to think further about these implications.
Are there any thoughts about this?
On Thursday 30 July 2009 05:59:07 Detlev Zundel wrote:
It is on my priority list however to assemble a list of all contributors and ask them if they agree to relicence their contributions under the FDL.
why not CC-BY-SA ? the wikipedia page has a bunch of references to FDL being a bad idea, and even the Debian guys declare it a non-free license. -mike
On Tue, 2009-07-28 at 23:27 +0200, Wolfgang Denk wrote:
Dear John,
in message 1248813631.3915.102.camel@johns you wrote:
It seems to me that DUTS is designed to test U-Boot and also automates the running of commands whose output can be put online in the DULG. I
Correct. And the DULG itself is a wiki (TWiki at the moment, to be converted to Foswiki ASAP) based framework which holds the "static" parts of the documentation.
didn't notice any documented procedure on how to turn the DULG into XML, extensible PDFs, etc. It also seems as if the DULG is a combination of hand-edited wiki pages as well as the DUTS output? I was hoping to
You probably can export the DULG into XML, too - but what would that be good for? At the moment we export it into PDF, PostScript, HTML and plain text.
Exporting to XML (and specifically DocBook XML, which is widely supported and accepted as a standard (at least that's what the tech writer next to me says :) ) is, I feel, a key to this whole process. Since XML doesn't specify a style, but rather what the content *means*, style sheets can be designed to turn the DocBook XML into anything, in any format, rearranged in any way, and with any style (fonts, colors, graphics). Again, I'm not familiar with the process you use to export HTML to PFD, PS, etc. but I'm guessing that the formatting and style end up being very similar.
develop a system that's completely automated. I also noticed in a quick
Hm... "completely automated" is a nice buzzword, but you still have to write the documentation in the first place.
You're in luck, I've already volunteered to write it in the first place :) and much of it is already done. Take a look at the manual posted on the X-ES website and see for yourself. http://www.xes-inc.com/sources/u-boot/xpedite5370.pdf This manual was completely generated with my tool.
The nice thing of the wiki based approach is that you can have a (even random) collection of pages which can be "linearized" and brought into arbitrary sets of linear doucumentation - this is what the WebOrder pages are good for - see http://www.denx.de/wiki/DULG/WebOrder
So you can use the same set of wiki pages an genreate for example the full fledged manual, a short version including only the most significant topics and an extended version including other stuff that is normally not part of the manual - all from the same envrionment.
This is definitely possible with DocBook XML. For those who care, I'm thinking the @role attribute available in standard DocBook, though there may be a better way. I'll talk this through with our tech writers.
probe around that a few items in the DULG seem to be out of sync (imd vs. i2c md, source vs. autoscr, etc.) and DHCP seems to be out of date as well. Is the process for updating the DULG automatic? If so, how is it done?
Soemone still has to write the documentation - and this is not always done in sync with the code. Your approach of including the documentation with the source code makes it more likely that one remebers to update the docs as well, but just remember how many places there are where code and comments don't agree - it's no guarantee either.
I would guess someone is several dozen times more likely to update a documentation snippet in the source over modifying a Wiki page.
If you find such areas in the DULG that are out of sync or missing please feel free to add them - everybody can contribute and modify the pages or add new content. And everyboidy can modify and extend the DUTS test cases, too.
As I mentioned, I borrowed this idea from the kernel-doc script in Linux, which does do API documentation. But my hope for the uboot-doc tool would be to create user documentation, or a manual we'd provide to a customer when they purchased a product that would describe available commands, environment variables, common operations, etc.
That's what the DULG does, so you are truely duplicating efforts.
As I mentioned earlier, it seems to me that most vendors are creating their own manuals. That either means knowledge of the existence of DULG is limited, or the tool isn't meeting people's needs. In order to use it, you need to download a completely separate package (DUTS), learn to use it, then hand edit the output into a Wiki page. The advantage with a documentation engine distributed with the sources is that there is no learning curve. "make pdf" creates a perfectly suitable PDF manual, and with a little work (first time only) they can make it look like anything and chug out manuals for all their boards in minutes.
Your approach may be suitable for standard documents, but in many years of working with U-Boot (and Linux) we found that it does not work so well with the specific needs we have for a User's Manual. One issue is that you have to support many different boards (well, maybe not you as a user, but we as a community). And you have to include examples. And examples must really fit the board. If you for example provide a manual entry for the "erase" command you better make sure that your example does not erase a range of flash that on some board happens to hold the U-Boot image. etc.
It's my hope that the documentation system I proposed can be made flexible enough to support things such as this without too much headache. That's the hope at least :)
Heh. Look at the DULG implementation... it's simple for the first 3 or 4 boards, especially when these are similar. But try supporting ARM and PowerPC and MIPS boards from N different vendors, with different kernel versions etc.
That, I believe, is one of the strengths of the tool I created. First, it uses the C pre-compiler to conditionally include comments from source. Secondly, the common XML templates support pre-compiler directives, so you can #ifdef in a section only when such-and-such is defined. And finally, if your manual requirements are dramatically different than others, a whole template can be overridden (or a whole new template added or deleted) in the board/${BOARDDIR}/manual directory. In my opinion, that's pretty flexible.
This would make DUTS/DULG more appealing. What is the process of generating DocBook XML from DUTS/DULG?
What would you need DocBook XML for?
As I mentioned above, I think the presence of DocBook XML is crucial to gain wide acceptance. When people see that this tool can generate manuals that look just like the manuals they offer now, I think that will make the sale.
It's true, I may have. :) On the other hand, it seems that there are still a lot of people who are in the same boat. Most manuals I have seen from other embedded companies (Freescale, Analog Devices, etc.) seem to provide their own format/content. Additionally, most companies will prefer to have their content formatted to match the rest of their manuals, which is easily done from DocBook XML. Again, if you can do that with DUTS/DULG, then that's great and probably eliminates the need for this tool.
Ah, I see.
Well, we use htmldoc to generate .pdf and .ps files from HTML; you could for example use Tidy to convert HTML to DocBook (and I'm sure there are lots of other tools that can do that).
XML specifies what the data means. HTML specifies what the data should look like. In my head, it's intuitive to say titles should be big and bold, but it's not intuitive to say big and bold should be a title. The conversion is imperfect. DocBook has a page on doing just that, and it's 5 steps which would probably several hours to get right for each manual. http://wiki.docbook.org/topic/Html2DocBook
Hm... we cannot look at your patches, you just posted the patch statistics, but no content.
This is just the patch series introduction, the actual content appears in 1-3.
Yes, I know. Sorry, patches arrived shortly after I started replying, and I didn;t notice.
As I mentioned, I'm not all that familiar with the abilities of DUTS/DULG, but I got the impression that there was still a fair amount of manual labor involved for each manual. Thus most people (that I'm
Indeed there is a lot of manual labor involved in preparing and updating the documentation (the framework, that is; once a board has been configured and added to DUTS, generating the needed include files is mostly an automatic procedure - just the upload of the files to the web server has to be triggered manually, but even this is scripted).
But you have to spend at least the same amount of writing and updating the documentation, don't you?
At least the way I see it, with your tool there's a fair amount of manual labor for every manual. With my tool, it's a similar amount of manual labor once for everyone in the whole community. Say we add a new command, it's extra work to comment it properly, but then everyone in the community reaps the benefits. Or more complexly, we add an entirely new interface of some sort, and a whole new manual section needs to be written. Someone writes it once and everyone in the community can reuse that section. And sure, you could do that with the DULG, but that's copy-and-paste-and-edit manual labor. For this tool, it would either be just run "make pdf" again, or possibly add a #define to your config file and then run "make pdf".
I was hoping that creating documentation from comments in the source would be easier to maintain and providing DocBook output would allow others to more easily reuse the U-Boot manual contents.
A very, very old version of the DULG was set up like this, using a set of Perl scripts and generating DokBoot output. We switched to the wiki based approach some 6 years ago, and I never had the slightest wish to switch back.
I do see the benefit of using a wiki, but I feel that it doesn't fit the needs of many U-Boot developers who need to generate documentation in a variety of styles and formats. I have to assume the vast majority of current U-Boot developers are rolling their own documentation as most boards are not supported by DULG. My hope is that if we generated U-Boot User's manuals in a different, easier manner with a more flexible output many more people could use it, thereby saving time which they can put to improving U-Boot itself :)
Thanks, John
participants (6)
-
Detlev Zundel -
John Schmoller -
jschmoller
-
Mike Frysinger -
Robin Getz -
Wolfgang Denk