On Tue, 2009-07-28 at 23:27 +0200, Wolfgang Denk wrote:
Dear John,
in message 1248813631.3915.102.camel@johns you wrote:
It seems to me that DUTS is designed to test U-Boot and also automates the running of commands whose output can be put online in the DULG. I
Correct. And the DULG itself is a wiki (TWiki at the moment, to be converted to Foswiki ASAP) based framework which holds the "static" parts of the documentation.
didn't notice any documented procedure on how to turn the DULG into XML, extensible PDFs, etc. It also seems as if the DULG is a combination of hand-edited wiki pages as well as the DUTS output? I was hoping to
You probably can export the DULG into XML, too - but what would that be good for? At the moment we export it into PDF, PostScript, HTML and plain text.
Exporting to XML (and specifically DocBook XML, which is widely supported and accepted as a standard (at least that's what the tech writer next to me says :) ) is, I feel, a key to this whole process. Since XML doesn't specify a style, but rather what the content *means*, style sheets can be designed to turn the DocBook XML into anything, in any format, rearranged in any way, and with any style (fonts, colors, graphics). Again, I'm not familiar with the process you use to export HTML to PFD, PS, etc. but I'm guessing that the formatting and style end up being very similar.
develop a system that's completely automated. I also noticed in a quick
Hm... "completely automated" is a nice buzzword, but you still have to write the documentation in the first place.
You're in luck, I've already volunteered to write it in the first place :) and much of it is already done. Take a look at the manual posted on the X-ES website and see for yourself. http://www.xes-inc.com/sources/u-boot/xpedite5370.pdf This manual was completely generated with my tool.
The nice thing of the wiki based approach is that you can have a (even random) collection of pages which can be "linearized" and brought into arbitrary sets of linear doucumentation - this is what the WebOrder pages are good for - see http://www.denx.de/wiki/DULG/WebOrder
So you can use the same set of wiki pages an genreate for example the full fledged manual, a short version including only the most significant topics and an extended version including other stuff that is normally not part of the manual - all from the same envrionment.
This is definitely possible with DocBook XML. For those who care, I'm thinking the @role attribute available in standard DocBook, though there may be a better way. I'll talk this through with our tech writers.
probe around that a few items in the DULG seem to be out of sync (imd vs. i2c md, source vs. autoscr, etc.) and DHCP seems to be out of date as well. Is the process for updating the DULG automatic? If so, how is it done?
Soemone still has to write the documentation - and this is not always done in sync with the code. Your approach of including the documentation with the source code makes it more likely that one remebers to update the docs as well, but just remember how many places there are where code and comments don't agree - it's no guarantee either.
I would guess someone is several dozen times more likely to update a documentation snippet in the source over modifying a Wiki page.
If you find such areas in the DULG that are out of sync or missing please feel free to add them - everybody can contribute and modify the pages or add new content. And everyboidy can modify and extend the DUTS test cases, too.
As I mentioned, I borrowed this idea from the kernel-doc script in Linux, which does do API documentation. But my hope for the uboot-doc tool would be to create user documentation, or a manual we'd provide to a customer when they purchased a product that would describe available commands, environment variables, common operations, etc.
That's what the DULG does, so you are truely duplicating efforts.
As I mentioned earlier, it seems to me that most vendors are creating their own manuals. That either means knowledge of the existence of DULG is limited, or the tool isn't meeting people's needs. In order to use it, you need to download a completely separate package (DUTS), learn to use it, then hand edit the output into a Wiki page. The advantage with a documentation engine distributed with the sources is that there is no learning curve. "make pdf" creates a perfectly suitable PDF manual, and with a little work (first time only) they can make it look like anything and chug out manuals for all their boards in minutes.
Your approach may be suitable for standard documents, but in many years of working with U-Boot (and Linux) we found that it does not work so well with the specific needs we have for a User's Manual. One issue is that you have to support many different boards (well, maybe not you as a user, but we as a community). And you have to include examples. And examples must really fit the board. If you for example provide a manual entry for the "erase" command you better make sure that your example does not erase a range of flash that on some board happens to hold the U-Boot image. etc.
It's my hope that the documentation system I proposed can be made flexible enough to support things such as this without too much headache. That's the hope at least :)
Heh. Look at the DULG implementation... it's simple for the first 3 or 4 boards, especially when these are similar. But try supporting ARM and PowerPC and MIPS boards from N different vendors, with different kernel versions etc.
That, I believe, is one of the strengths of the tool I created. First, it uses the C pre-compiler to conditionally include comments from source. Secondly, the common XML templates support pre-compiler directives, so you can #ifdef in a section only when such-and-such is defined. And finally, if your manual requirements are dramatically different than others, a whole template can be overridden (or a whole new template added or deleted) in the board/${BOARDDIR}/manual directory. In my opinion, that's pretty flexible.
This would make DUTS/DULG more appealing. What is the process of generating DocBook XML from DUTS/DULG?
What would you need DocBook XML for?
As I mentioned above, I think the presence of DocBook XML is crucial to gain wide acceptance. When people see that this tool can generate manuals that look just like the manuals they offer now, I think that will make the sale.
It's true, I may have. :) On the other hand, it seems that there are still a lot of people who are in the same boat. Most manuals I have seen from other embedded companies (Freescale, Analog Devices, etc.) seem to provide their own format/content. Additionally, most companies will prefer to have their content formatted to match the rest of their manuals, which is easily done from DocBook XML. Again, if you can do that with DUTS/DULG, then that's great and probably eliminates the need for this tool.
Ah, I see.
Well, we use htmldoc to generate .pdf and .ps files from HTML; you could for example use Tidy to convert HTML to DocBook (and I'm sure there are lots of other tools that can do that).
XML specifies what the data means. HTML specifies what the data should look like. In my head, it's intuitive to say titles should be big and bold, but it's not intuitive to say big and bold should be a title. The conversion is imperfect. DocBook has a page on doing just that, and it's 5 steps which would probably several hours to get right for each manual. http://wiki.docbook.org/topic/Html2DocBook
Hm... we cannot look at your patches, you just posted the patch statistics, but no content.
This is just the patch series introduction, the actual content appears in 1-3.
Yes, I know. Sorry, patches arrived shortly after I started replying, and I didn;t notice.
As I mentioned, I'm not all that familiar with the abilities of DUTS/DULG, but I got the impression that there was still a fair amount of manual labor involved for each manual. Thus most people (that I'm
Indeed there is a lot of manual labor involved in preparing and updating the documentation (the framework, that is; once a board has been configured and added to DUTS, generating the needed include files is mostly an automatic procedure - just the upload of the files to the web server has to be triggered manually, but even this is scripted).
But you have to spend at least the same amount of writing and updating the documentation, don't you?
At least the way I see it, with your tool there's a fair amount of manual labor for every manual. With my tool, it's a similar amount of manual labor once for everyone in the whole community. Say we add a new command, it's extra work to comment it properly, but then everyone in the community reaps the benefits. Or more complexly, we add an entirely new interface of some sort, and a whole new manual section needs to be written. Someone writes it once and everyone in the community can reuse that section. And sure, you could do that with the DULG, but that's copy-and-paste-and-edit manual labor. For this tool, it would either be just run "make pdf" again, or possibly add a #define to your config file and then run "make pdf".
I was hoping that creating documentation from comments in the source would be easier to maintain and providing DocBook output would allow others to more easily reuse the U-Boot manual contents.
A very, very old version of the DULG was set up like this, using a set of Perl scripts and generating DokBoot output. We switched to the wiki based approach some 6 years ago, and I never had the slightest wish to switch back.
I do see the benefit of using a wiki, but I feel that it doesn't fit the needs of many U-Boot developers who need to generate documentation in a variety of styles and formats. I have to assume the vast majority of current U-Boot developers are rolling their own documentation as most boards are not supported by DULG. My hope is that if we generated U-Boot User's manuals in a different, easier manner with a more flexible output many more people could use it, thereby saving time which they can put to improving U-Boot itself :)
Thanks, John