[asterisk-dev] [Code Review] New application JabberReceive, implement SendText in chan_gtalk and chan_jingle

[John Todd]

On Dec 12, 2008, at 12:37 PM, Philippe Sultan wrote:

>> After some thought, I'd have to agree with Russell on this as well -
>> there needs to be some way to capture messages that would be sent
>> prior to the JabberRecieve being called.
>
> Agreed too :).
>
>> [note: the next two paragraphs are the start of what will become a
>> very, very long set of religious discussions and gnashing of teeth  
>> and
>> tearing of hair]
>>
>> Why?  Because I suspect that this will almost immediately become
>> another API method for Asterisk.  It's an ugly, horrible, nasty
>> hackish API but guess what - it'll work for most people doing
>> lightweight interactions who are already using Jabber.  Machine-to-
>> Machine interactions via XMPP is becoming common, and therefore the
>> responses back from JabberSend will be lightning fast in many
>> instances, and even the time it takes for a few dialplan events may  
>> be
>> too long to correctly receive an inbound reply from an outbound
>> message.  At the very least, the implementation that gets shipped
>> first should be able to handle immediate responses to transmitted
>> messages that may be generated by other systems otherwise the first
>> impressions will be the last impressions someone gets on using this
>> method, and it will create bogus summaries and manual pages that will
>> be difficult or impossible to correct later - "You can't use the
>> JabberSend/JabberReceive apps in Asterisk for fast responses because
>> they will lose messages." would be an example of what I do NOT want  
>> to
>> see.  There will certainly be problems with M2M systems using XMPP in
>> this way, but such a significant one I think we should avoid on our
>> first step into the minefield.
>>
>> Of course, an API should have the ability to create actions in a
>> system, but the elements Russell mentions (global receipt of XMPP
>> messages) makes that a fairly simple jump.  My opinion is that XMPP
>> messages should be received globally on any JIDs registered in
>> jabber.conf, and then should execute in the dialplan on a per-
>> registration context (just like any other channel) using the
>> registered JID as the $EXTEN, using the remote JID as the caller ID,
>> and setting the text value as a dialplan variable.   But... one step
>> at a time, I suppose.  :-)
>
> That's really interesting, but wouldn't you loose the ability to call
> JabberReceive within any place in the dialplan then? This is something
> that needs to be investigated.

Now that I think of it that way, yes, it does present a problem.  But  
I've got a solution a bit further down.

Regardless of any future implementation of XMPP messages creating new  
channels, I believe that JabberReceive might be best to understand two  
modes, a "non-buffered receive" and a "buffered receive".  It may make  
sense for default behavior to listen to buffered responses, and an  
optional declaration to not look at the buffer.   Perhaps (though I  
don't outline it below) there would be a timer option as well - "go  
back a maximum of X seconds to look for messages that match this  
criteria when executing a JabberReceive."

Danger, Danger: If there is a buffer for received XMPP messages, there  
should be some hand-tunable timeout for messages that linger in the  
buffer, otherwise there is a risk of memory exhaustion from attackers  
or unintentionally broken and chatty remote systems.


Of course, now that I've had some more time to think about this, more  
obvious problems come to light when using XMPP as a machine-to-machine  
transport in a high-volume system.  The biggest one you've already  
touched on, Philippe, is "threading" in a message.  If there are  
multiple calls happening that all use the XMPP send/receive methods,  
then there needs to be some way that the XMPP handler can ensure (or  
try to ensure) that the right messages are handed to the right call  
flows.  It seems that the "threading" option you mentioned is the best  
way to do it, but you also said that it's poorly implemented in  
clients.  This may not matter so much to anyone using an automated  
system, since they'll probably be building their own kit out of fairly  
modern code which undersatnds (or can be made to understand) the  
thread/ element.  For chats with humans, this may actually also be  
important - what happens if I get two calls at once and I have an  
automated system that sends me messages about sending calls to  
voicemail?  While my XMPP client might not handle it today (Adium on  
Mac) but it appears that as soon as libpurple is ready it may be  
useful (http://trac.adiumx.com/ticket/8319).

I know that XEP is in "draft" status, which makes me wonder how M2M  
implementations of XMPP are being handled currently.  Without knowing  
a thread identifier or some other transactional state message, how do  
two JIDs communicate about many different simultaneous sessions?   
Putting something in the message seems like a hack that is so bad that  
it would never be done.  Any XMPP experts out there?  It does seem  
like implementations are at least considering that XEP for similar  
reasons.

http://xmpp.org/extensions/xep-0201.html

Here's the hypothetical set of commands with the modification concepts  
I talked about:

*CLI> core show application JabberReceive

   -= Info about application 'JabberReceive' =-

[Synopsis]
JabberReceive will receive a text message from a remote XMPP client and
   store the contents in a given variable.

[Description]
Same as Synopsis.

[Syntax]
JabberReceive(JID,Variable[,Timeout][,Account][,ThreadID][,options])

[Arguments]
   JID - Jabber ID of the buddy to receive message from
   Variable - Variable to store message in
   Timeout - Defaults to 20 seconds
   Account - the local named account to listen on (specified in  
jabber.conf)
   ThreadID - the thread ID for this conversation (ignores other  
ThreadIDs if set)
   options -
     R - accept messages in "realtime", and do not match against  
buffered messages
         which may have been in the queue before this app was activated

[See Also]
JabberSend, JabberStatus

*CLI>



This would also require JabberSend to get a "ThreadID" setting:

core1*CLI>
   -= Info about application 'JabberSend' =-

[Synopsis]
JabberSend will send an arbitrary text message to a given XMPP  
destination

[Description]
Same as Description.

[Syntax]
JabberSend(Account,JID,Message[,ThreadID])

[Arguments]
   Account  - Client or transport Asterisk uses to connect to XMPP  
network.
    This is the named account in jabber.conf.
   JID      - XMPP/Jabber JID (Name) of recipient
   Message  - Text message to be sent to the buddy
   ThreadID - (optional) unique ThreadID to associate with message  
flow.  This
    should be either a unique universal value for new sessions or  
should be set
    to the value expected by the receiver to continue a particular  
thread.

[See Also]
JabberReceive, JabberStatus



Things Not To Be Done Right Now:  If and when a "call" (new message)  
is instantiated to some global receiver of XMPP messages to a specific  
JID, the process could be something like this: create a new "call" to  
the context= line IF there is no JabberReceive waiting that matches  
the correct sending JID AND that JabberReceive does not have a  
matching thread identifier.  If there are two JabberReceive  
applications waiting that match both the sender JID, receiver account  
JID, and have an empty or unknown thread/ element, then whatever  
JabberReceive has been waiting the longest has the message passed to  
it.  This, I believe, would lead to the reasonable "expected" response  
for all cases I can think of, and would allow XMPP messages to  
activate dialplan code.  The decisions on what the $EXTEN is and all  
that other stuff can be hashed out later.


>> Philippe - I understand that it works now the way you have it, but
>> would it delay implementation significantly?  Are you on a roll, and
>> could you keep that momentum going to add the pre-buffering or global
>> receipt that has been discussed above?  I think there is an
>> opportunity to have a big leap in functionality of Asterisk, and it's
>> your code that's at the heart of that advancement.
>
> No problem, I'll come up with the new implementation as soon as
> possible. Plus, given the way messages are stored (e.g. in a structure
> within the XMPP client), your suggestion of including the account JID
> we're listening messages on becomes mandatory :)


Sorry for the feature creep issues.  This is an interesting track to  
take, and I think many people are interested in getting better control  
of their Asterisk boxes via XMPP.  I'm getting to the point where  
almost 100% of my interactions are through XMPP - AOL, MSN, Yahoo, SIP  
(yes, SIP messaging), Twitter, Yammer, and one-way Asterisk are all  
gatewayed or directly connected via XMPP, and my IM client also  
controls Skype and Facebook via APIs (though I wish they were XMPP  
gateways as well.)  Being able to _control_ my Asterisk system via  
XMPP will really, really be useful and will open up another push of  
development in the Enterprise area which is now starting to see the  
wisdom of XMPP as the protocol of choice.  Getting the inbound  
messages today is already super-handy - Asterisk burps out missed  
calls and other statistical data without my having to be actively  
engaged with a console or a web browser or anything.  Being eventually  
able to TELL my Asterisk system to "call +14102241145" or "show  
missed" or "dial missed last" or "voicemail delete last" - that would  
be freakin' GREAT.  I'm sure that's all possible now with external  
programs and AGI scripts and callfiles, but I'd like to do it in a  
more integrated way.

PS: We should really change all these "Jabber" terms and app names to  
"XMPP".

JT


---
John Todd
jtodd at digium.com        +1-256-428-6083
Asterisk Open Source Community Director