[asterisk-dev] Journald support for Asterisk

[Matthew Jordan]

On Sat, May 9, 2015 at 9:15 AM, Bruce Ferrell <bferrell at baywinds.org> wrote:
> On 05/09/2015 01:17 AM, Ludovic Gasc wrote:
>> Hi,
>>
>> Systemd and Journald is now by default on Debian Jessie and Ubuntu 15.04, as on RHEL/CentOS.
>> Journald supports syslog format, nevertheless, at least for us, the structured log system provided with journald helps us to debug the production.
>>
>> The idea behind that is to attach metadata with a log line to facilitate the search with journalctl, you can write queries to find the errors.
>>
>> For example, with Apache: http://httpd.apache.org/docs/trunk/mod/mod_journald.html
>>
>> For our Python daemons, for each log line, we store account_id, request_id, endpoint and method used to be easy to retrieve quickly interesting logs.
>>
>> Moreover, not yet used for us, but you can generate statistics about your source code real usage based
>> on: http://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html#CODE_FILE=
>>
>> For Asterisk, for example with dialplan logging, you should attach the context, the extension and the channel.
>>
>> You can simulate this journald feature with a specific format message for your logs, nevertheless, journalctl is more user-friendly to retrieve pertinent logs compare to the
>> classical grep usage.
>> ave no idea if i
>> I'm permitted to raise the question about an eventual journald support in Asterisk because I don't find anything about that in issues tracker nor mailing list.
>>
>> Moreover, I understand that even if you add the journald support, it's certainly necessary to change logging everywhere in Asterisk, however, I should help to do that.
>>
>> Regards
>> --
>> Ludovic Gasc (GMLudo)
>> http://www.gmludo.eu/
>>
>>
> systemd and journald SUCK to high heaven.  I have no idea if the issues I've had with them (OpenSUSE) are distro related or inherent.
>
> I have seen that the implementation of apache with a static systemd/journald module will no longer correctly serve content.
>
> Please do NOT do this.  It also breaks fail2ban site security.  Asterisk works well now.  It integrates easily.

Being very honest, my knowledge of systemd and journald is not
extensive. I have no real position on systemd/journald one way or the
other. I will say that I do not view it as an inevitable conclusion,
nor a necessary requirement for Asterisk. As such, I'm viewing this
only as a proposal for "we'd like to change the vast majority of log
statements in Asterisk."

First, as Bruno pointed out, a lot of external systems rely on parsing
log messages. Asterisk does not exist in a vacuum, and is often a
component in a much larger system. As such, any modification to
Asterisk's logging would have to be configurable.

Second, the logging system in Asterisk has a measurable impact on
performance. Unfortunately, adding a configuration option that is
evaluated on every log statement will further degrade performance when
logging verbosity is increased. A compile time option is not a
pleasant alternative either. Note that we experienced a similar
situation in Asterisk 11 with the remote console independent verbosity
setting, and spent a lot of time over many point releases getting the
performance back in line.

It is also telling that Apache's mod_journald states:
{quote}
Performance warning

Currently, systemd-journald is not designed for high-throughput
logging and logging access_log to systemd-journald could decrease the
performance a lot.
{quote}

Third, journald seems to be providing structured log data. While some
logging statements in Asterisk may benefit from such a system, many
may not. Asterisk can be *extremely* chatty. I'm not sure that every
log statement in Asterisk should be converted or support such a
system. A more concrete proposal over what messages are appropriate
for journald would be nice.

I'd be curious if another option would be to treat journald as we
currently treat syslog - not as a formatting option, but as a
destination. If that were possible, support could be added for it
without necessarily impacting the existing log statements. Asterisk
supports the concept of having dynamic log channels, and support for
journald could then also be added as an external module without
impacting the core - or impacting users who don't care for it.

-- 
Matthew Jordan
Digium, Inc. | Director of Technology
445 Jan Davis Drive NW - Huntsville, AL 35806 - USA
Check us out at: http://digium.com & http://asterisk.org