[asterisk-dev] Journald support for Asterisk
Ludovic Gasc
gmludo at gmail.com
Tue May 12 14:55:20 CDT 2015
I've two remarks about that:
1. Even if a journald support is one day included in Asterisk, it will be
for Asterisk 14 or 15 at least. The time you have that as stable version,
it will be 2016 or 2017.
Even if we already have some few CentOS on production because it's
mandatory for some products, we stopped to install new CentOS 6 a long time
ago.
At least for us, when we want to have a new version of Asterisk, we install
first a new VM with the latest stable version, in this case, CentOS 7,
because it's very easier to setup a new VM, migrate data, and finally
switch DNS records, instead of to cut the service.
2. Moreover, the support of journald doesn't mean you must drop syslog
logging: I don't know how the standard logging mechanism works in C, but in
Python:
a. You log has usual your message with logging.info(), and you add a
parameter "extra" with the metadata/dict of the log.
b. You have a list of handlers already included in Python to log in
syslog, console, file...:
https://docs.python.org/3.4/library/logging.handlers.html#module-logging.handlers
Most handlers don't support extra parameter, its drop extra data. For
handlers that support that like journald, it's included.
Ludovic Gasc (GMLudo)
http://www.gmludo.eu/
On 12 May 2015 07:25, "Corey Farrell" <git at cfware.com> wrote:
> CentOS/RHEL 6 are under production support until Nov 2020 [1] and journald
> is not available. Even if journald could improve Asterisk logging, that's
> not an acceptable reason to force Asterisk users to upgrade from a
> supported OS. If there was a module that allowed Asterisk logging to talk
> to journald I wouldn't care, but it should not replace the logging system
> or be required to run Asterisk.
>
> [1]
> http://wiki.centos.org/FAQ/General#head-fe8a0be91ee3e7dea812e8694491e1dde5b75e6d
>
> On Sun, May 10, 2015 at 5:22 PM, Ludovic Gasc <gmludo at gmail.com> wrote:
>
>> 2015-05-10 20:46 GMT+02:00 Bruce Ferrell <bferrell at baywinds.org>:
>>
>>> On 05/09/2015 01:53 PM, Ludovic Gasc wrote:
>>> > 2015-05-09 16:15 GMT+02:00 Bruce Ferrell <bferrell at baywinds.org
>>> <mailto:bferrell at baywinds.org>>:
>>> >
>>> > On 05/09/2015 01:17 AM, Ludovic Gasc wrote:
>>> > > Hi,
>>> > >
>>> > > Systemd and Journald is now by default on Debian Jessie and
>>> Ubuntu 15.04, as on RHEL/CentOS.
>>> > > Journald supports syslog format, nevertheless, at least for us,
>>> the structured log system provided with journald helps us to debug the
>>> production.
>>> > >
>>> > > The idea behind that is to attach metadata with a log line to
>>> facilitate the search with journalctl, you can write queries to find the
>>> errors.
>>> > >
>>> > > For example, with Apache:
>>> http://httpd.apache.org/docs/trunk/mod/mod_journald.html
>>> > >
>>> > > For our Python daemons, for each log line, we store account_id,
>>> request_id, endpoint and method used to be easy to retrieve quickly
>>> interesting logs.
>>> > >
>>> > > Moreover, not yet used for us, but you can generate statistics
>>> about your source code real usage based
>>> > > on:
>>> http://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html#CODE_FILE=
>>> > >
>>> > > For Asterisk, for example with dialplan logging, you should
>>> attach the context, the extension and the channel.
>>> > >
>>> > > You can simulate this journald feature with a specific format
>>> message for your logs, nevertheless, journalctl is more user-friendly to
>>> retrieve pertinent logs compare to the
>>> > > classical grep usage.
>>> > > ave no idea if i
>>> > > I'm permitted to raise the question about an eventual journald
>>> support in Asterisk because I don't find anything about that in issues
>>> tracker nor mailing list.
>>> > >
>>> > > Moreover, I understand that even if you add the journald
>>> support, it's certainly necessary to change logging everywhere in Asterisk,
>>> however, I should help to do that.
>>> > >
>>> > > Regards
>>> > > --
>>> > > Ludovic Gasc (GMLudo)
>>> > > http://www.gmludo.eu/
>>> > >
>>> > >
>>> > systemd and journald SUCK to high heaven.
>>> >
>>> >
>>> > I've no problems to believe you, nevertheless, could you be more
>>> precise ?
>>> > What are the facts that you arrive to this conclusion ?
>>> >
>>> >
>>> > I have no idea if the issues I've had with them (OpenSUSE) are
>>> distro related or inherent.
>>> >
>>> >
>>> > It's issues on your production with several servers ? Or on your
>>> laptop ?
>>> > Could give us links with unresolved bugs you have ?
>>> > For now, to my experience, with 6 VMs on Debian Jessie on production
>>> with small clients and Asterisk 13, I've no issues with systemd nor
>>> journald.
>>> >
>>> >
>>> > I have seen that the implementation of apache with a static
>>> systemd/journald module will no longer correctly serve content.
>>> >
>>> >
>>> > Could you give me a link about that ?
>>> I would suggest you try it for yourself. Procedure to reproduce:
>>>
>>> Perform fresh install of OpenSUSE 13.2
>>> assure apache/php/MySQL support are installed
>>> install wordpress from wordpress.org
>>> Observe CSS is NOT correctly served.
>>>
>>> I did this several times to assure my findings.
>>>
>>> I rebuilt the opensuse rpms from the source rpms to omit
>>> systemd/journald from the apache build. Correct operation was now observed
>>>
>>
>> Interesting, it seems like a regression bug.
>> The question now is the patchs from opensuse has broken this module, or
>> the bug is in Apache itself.
>> The bug seems to be too obvious to be present in Apache source code, but
>> who knows ?
>>
>>
>>> The response to the bug report was "Use Nginex"
>>>
>>
>> To be really honest with you, I've thought that when you explain your
>> problem, I've dropped Apache on our production a long time ago for
>> efficiency issues.
>>
>>
>>>
>>> > Please do NOT do this
>>> >
>>>
>>> > Why ? If it works for me, what's the problem for you ?
>>>
>>> Systemd integration has been insidious. Should you wish to discuss
>>> further, please off this list.
>>>
>>> >
>>> >
>>> > It also breaks fail2ban site security.
>>> >
>>> >
>>> > I don't understand the issue here, I see at least two solutions:
>>> > 1. You can continue to use rsyslog even if you have journald, BTW,
>>> it's the default setup of Debian Jessie, where journald forwards everything
>>> to rsyslog to avoid to perturb
>>> > sysadmins with log files.
>>> > 2. Apparently, fail2ban supports journald:
>>> https://github.com/fail2ban/fail2ban/pull/82
>>>
>>> 1.) I don't know of any asterisk systems that use any form of system
>>> logging. It can, and the fact that you keep bringing syslog logging into
>>> this indicates a lack of
>>> understanding of asterisk usage on your part.
>>> 2.) Not released code and still under test. This is disingenuous, at
>>> best.
>>> 3.) Just because everyone else is jumping off of bridges isn't a good
>>> enough reason to do it. It's NOT broken and this is NOT an improvement.
>>>
>>
>> Concretely, we have an infrastructure that for now handles more than 3
>> millions of SIP calls by month with a dedicated telco team for that, and
>> we're continuing to grow.
>>
>> For now, even with voipmonitor and homer, it's a nightmare for our telco
>> team to debug that.
>> One part of the problem is because we have an heterogeneous architecture
>> with kamailio, freeswitch and asterisk, but also, as most telco solutions
>> in the cloud, we're trying to stack the most clients as possible with the
>> less VMs: It means that each node is in tenant mode => no nodes are
>> dedicated for one client.
>>
>> To improve this situation, I'm looking for a solution to invert the
>> classical UNIX model: to split logs not by daemon, but by client => instead
>> of a telco guy becomes crazy to parse with grep each daemon log and track
>> one call with different ids, telco guys want to track easily all history to
>> understand why the system has taken this decision to do that. One of a
>> concrete example is Asterisk and FastAGI daemons interactions.
>>
>> journald itself isn't clearly a silver bullet to handle all problematic,
>> nevertheless, the first step is to have a better categorization of logs for
>> after build a tool behind that.
>> In the past, we've tried a little bit ELK but it was very complicated,
>> heavy process for a few ROI from our experience.
>>
>> We'd already made a small tool to parse Asterisk logs and categorize
>> them, but under high load, we had efficiency issues: it didn't parse enough
>> fastly to be useable for a direct live debugging.
>>
>> It's why I've made a little labo with some clients to validate my ideas
>> before a bigger production, because I don't trust the mainstream nor
>> hipster architectures promoted in IT world.
>> I "believe" only in concrete values I can measure on concrete use cases
>> useful for my users: clients or telco guys.
>>
>> Ok, while Asterisk doesn't support structured logs, the workaround is to
>> implement a more efficient parser for Asterisk logs to categorize that.
>>
>> Thanks for your attention.
>>
>>
>>> >
>>> >
>>> > Asterisk works well now. It integrates easily.
>>> >
>>> >
>>> > For this point, I'm agree with you, it's very important to continue to
>>> be integrated easily, I use that a lot.
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> >
>>> _____________________________________________________________________
>>> > -- Bandwidth and Colocation Provided by http://www.api-digital.com
>>> --
>>> >
>>> > asterisk-dev mailing list
>>> > To UNSUBSCRIBE or update options visit:
>>> > http://lists.digium.com/mailman/listinfo/asterisk-dev
>>> >
>>> >
>>> >
>>> >
>>>
>>>
>>> --
>>> _____________________________________________________________________
>>> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>>>
>>> asterisk-dev mailing list
>>> To UNSUBSCRIBE or update options visit:
>>> http://lists.digium.com/mailman/listinfo/asterisk-dev
>>>
>>
>>
>> --
>> _____________________________________________________________________
>> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>>
>> asterisk-dev mailing list
>> To UNSUBSCRIBE or update options visit:
>> http://lists.digium.com/mailman/listinfo/asterisk-dev
>>
>
>
> --
> _____________________________________________________________________
> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>
> asterisk-dev mailing list
> To UNSUBSCRIBE or update options visit:
> http://lists.digium.com/mailman/listinfo/asterisk-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20150512/602f5692/attachment-0001.html>
More information about the asterisk-dev
mailing list