[asterisk-dev] Final Preview: docs.asterisk.org

[George Joseph]

Comments in line ...

On Sat, Jul 8, 2023 at 4:24 PM <asterisk at phreaknet.org> wrote:

> Hi, Geroge,
>      Just had a chance to look at this this afternoon. The instructions
> for the dynamic doc generation definitely made my head hurt a little
> bit, but I have a few thoughts after putzing around a little bit.
>

Your head should be much better now. :)


>
> My initial thought was that some of the make targets in the Makefile
> could be split up a little bit. The version-dynamic target both
> downloads documentation source and does the actual build of the
> documentation. They could be split into different targets:
>
>   * In my case, I don't want to download the upstream Asterisk
>     documentation, I want to use the local core-en-us.xml, which is a
>     superset of the documentation available upstream.
>

Uhm are you sure??   The core XML document generated during a build doesn't
have the xinclude references de-referenced.  I'd be interested to know what
you're seeing.


>   * Since I'm not downloading the docs, I don't think I need the "gh"
>     tool, and so it doesn't need to be installed for purely generating
>     documentation. (Also, could be documented as a pre-req, if doing the
>     download step)
>

There are now facilities in the Makefile to get the dynamic sources from
your
local system.   In that case, gh is not required.  Details in the README.


>
> Then again, when you boil it down to that, it seems like it really comes
> down to just:
>
>   * utils/astxml2markdown.py
>   * mike deploy
>
> So it seems I'm better off avoiding the makefile and just running the
> individual commands needed. I'll end up wrapping it in a script anyways,
> but just wondering if there's anything else about the process here I've
> missed that wouldn't be conducive to that?
>

You don't need to avoid the Makefile.  You can specify where to get the
dynamic sources from and which branches to build.  Details in the README.


>
> In particular, it doesn't seem that finagling with Git repositories is
> necessary at all to build the dynamic docs. I was able to get it to work
> with just this:
>

No more git repo finagling.


>
> git clone https://github.com/asterisk/documentation.git --depth 1
> cd documentation
> pip3 install -r requirements.txt
> mv docs/favicon.ico docs/index.md .
> rm -rf docs/*
> mv favicon.ico index.md docs
> utils/astxml2markdown.py
> --file=/usr/src/asterisk-20.3.0/doc/core-en_US.xml
> --xslt=utils/astxml2markdown.xslt --directory=docs/ --branch=20
> --version=20.3.0
> mike deploy -F mkdocs.yml 20
> rm -rf /var/www/html/asterisk && mv site /var/www/html/asterisk
> cd .. && rm -rf documentation
>
> It seems to work really well. There were just a couple surprises or
> annoying aspects:
>
>   * Even with --depth 1, the documentation repo takes a hot minute to
>     clone, due to all the large PDF artifacts in it, though a tarball of
>     the repo wouldn't help either if it came with all the static
>     artifacts anyways. Could probably work around that by cloning it
>     using svn instead of git, but I was too lazy to do that today.
>

No longer needed.


>   * For just turning markdown into HTML, mike is pretty slow, it takes
>     over half a minute just for the dynamic docs (compared to ~1 second
>     for my previous method, though that was from the original XML to
>     HTML, not from intermediate markdown)
>

mike has bene eliminated.


>   * Most significantly, the webpages are *huge*. Even just the dynamic
>     docs are a whopping 228 MB. On average, a documentation page is
>     almost 250 KB (compared to 1.2 MB the old way for all the
>     applications, functions, AMI/AGI, and configs on a single webpage -
>     granted it's not a fair comparison since the menu and what not isn't
>     repeated that way). Taking a look at the page source of an
>     application[1], much of the page is whitespace. I know that docs
>     hosted on GitHub don't need to worry about disk consumption, but for
>     folks building the docs themselves, I think it might be worth trying
>     to clean this a little bit. Bloated webpages are obviously also
>     going to be slower to load as well.
>

File sizes have been significantly reduced by updating mkdocs and using the
new navigation.prune option.  I tried removing all of the extra whitespace
but after the prune option was applied it made very little difference
overall.  We can keep looking at it.


>
> I haven't yet figured out what's introducing all the extraneous
> whitespace. The markdown files seem okay, but things seem to blow up
> somewhere in the middle. Ideally we could prevent it from happening in
> the first place, but if not, then maybe some fancy recursive
> post-processing could strip it all out.
>
> [1] https://docs.asterisk.org/20/_Dialplan_Applications/ADSIProg/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20230711/3333bd76/attachment.html>