[asterisk-dev] Final Preview: docs.asterisk.org

[asterisk at phreaknet.org]

On 7/11/2023 4:34 PM, George Joseph wrote:
> On Sat, Jul 8, 2023 at 4:24 PM <asterisk at phreaknet.org 
> <mailto:asterisk at phreaknet.org>> wrote:
>
>     Hi, Geroge,
>          Just had a chance to look at this this afternoon. The
>     instructions
>     for the dynamic doc generation definitely made my head hurt a little
>     bit, but I have a few thoughts after putzing around a little bit.
>
>
> Your head should be much better now. :)

Much better... the A/C repairman just left too, which helps :)

>
>     My initial thought was that some of the make targets in the Makefile
>     could be split up a little bit. The version-dynamic target both
>     downloads documentation source and does the actual build of the
>     documentation. They could be split into different targets:
>
>       * In my case, I don't want to download the upstream Asterisk
>         documentation, I want to use the local core-en-us.xml, which is a
>         superset of the documentation available upstream.
>
>
> Uhm are you sure??   The core XML document generated during a build 
> doesn't have the xinclude references de-referenced.  I'd be interested 
> to know what you're seeing.

Mm... yes, you're right about that. I remember encountering that 
limitation when I wrote my converter. I planned to do something about 
it, but never did :)
My core-en_US.xml and full-en_US.xml files are very similar in size so I 
don't think which one I use would make any difference with this.

As far as "what I'm seeing", this is an example of what I get when I 
take my built XML file at the end of compiling and run it through my 
conversion script: https://asterisk.phreaknet.org/

I believe it handles most of the documentation except those 
xinclude/pointer references to other files. It's not fully on par with 
the real docs, but my main use case here was being able to view 
documentation for applications and functions that aren't upstreamed, and 
for that it's sufficed. Also, the parser I wrote is not super robust, an 
unintended benefit of which is occasionally it's caught bugs in the 
xmldocs that have needed to be fixed, that otherwise might have slipped 
through a more lenient parser.

I could keep using this, of course, since nothing has changed with the 
XML and there's no Confluence involved, but what you've done with the 
dynamic docs is way better (and the search capability is really nice), 
so I'd like to migrate to that for internal documentation.

How are the includes *supposed* to be handled, by the way i.e. what's 
supposed to dereference the xincludes? Is it one of the Asterisk build 
scripts for the docs piecing everything together, or is it expected that 
whatever consumes the XML files is able to handle those?

> There are now facilities in the Makefile to get the dynamic sources 
> from your
> local system.   In that case, gh is not required. Details in the README.

Found a typo in the new README, seems to be just on that branch so I 
guess I can't submit a PR: "CreteDocs"

>
>     Then again, when you boil it down to that, it seems like it really
>     comes
>     down to just:
>
>       * utils/astxml2markdown.py
>       * mike deploy
>
>     So it seems I'm better off avoiding the makefile and just running the
>     individual commands needed. I'll end up wrapping it in a script
>     anyways,
>     but just wondering if there's anything else about the process here
>     I've
>     missed that wouldn't be conducive to that?
>
>
> You don't need to avoid the Makefile.  You can specify where to get 
> the dynamic sources from and which branches to build.  Details in the 
> README.

I'm a little confused about the BRANCHES variable in Makefile.inc. I 
presume this is irrelevant if using a local XML doc? If only because 
there's only one version of Asterisk compiled (and from which the XML 
doc is sourced). It does specifically say if I'm using local sources I 
need separate Makefile.branch.inc's though so I think I'm not 
understanding something here.

I think I understand why the branches are needed - because now it's 
building all the docs for all the versions into the same site. I think 
my point of confusion is the dynamic doc builder only sees an XML file, 
it can't even possibly know what version of Asterisk that's for without 
being told, can it? I was thinking that make BRANCHES=XX just needs to 
match Makefile.XX.inc, but wasn't sure if there's any other significance 
to these values.

It seems to still be trying to download it for me despite adding the 
includes, though I just set up the includes and followed the 
instructions, haven't dug much deeper

root at debian11:/usr/src/documentation# nano Makefile.inc
root at debian11:/usr/src/documentation# nano Makefile.20.inc
root at debian11:/usr/src/documentation# make BRANCHES=20
Finding last CreateDocs job
/usr/bin/bash: line 1: gh: command not found
Setting Up Static Documentation
   Copying to temp build
   Applying link transformations
Building branch 20
Finding last CreateDocs job
/usr/bin/bash: line 1: gh: command not found
Creating ./temp/build-20
Setting Up Core Dynamic Documentation for branch '20'
   Generating markdown from Asterisk XML
No current docs job
make[1]: *** [Makefile:118: download-from-job] Error 1
make: *** [Makefile:61: dynamic-setup] Error 2

root at debian11:/usr/src/documentation# cat Makefile*.inc
ASTERISK_XML_FILE := /usr/src/asterisk-20.3.0/doc/core-en_US.xml
BRANCHES := 20
ASTERISK_XML_FILE := /usr/src/asterisk-20.3.0/doc/core-en_US.xml


>  <snip>
>
> File sizes have been significantly reduced by updating mkdocs and 
> using the new navigation.prune option.  I tried removing all of the 
> extra whitespace but after the prune option was applied it made very 
> little difference overall. We can keep looking at it.
>

Thanks, George, I thought this might be hard to track down but it looks 
like you all solved all the problems in one fell swoop. I need to figure 
out what I did wrong in building but it sounds like this should work 
great. Thank you!

  NA