[asterisk-dev] Contacts, Contact Status and Sorcery

[George Joseph]

On Thu, Apr 30, 2015 at 2:01 PM, Joshua Colp <jcolp at digium.com> wrote:

> George Joseph wrote:
>
>>
>> Change https://gerrit.asterisk.org/261 sparked some discussion about
>> contacts and contact status so I'd like to continue that here.
>>
>> Today, dynamic contacts (incoming registrations) are created on the fly
>> as full sorcery objects and stored in an astdb back-end so they survive
>> a restart.  Permanent contacts however, are created on the fly as
>> sorcery objects when the aor is created but instead of being managed by
>> sorcery, are stored in an aor-specific container. Contact statuses for
>> both types of contacts are created as full sorcery objects and stored in
>> a memory back-end.  They obviously do not survive a restart.  This makes
>> navigating the life-cycle of these objects a little convoluted.
>>
>> The handling of dynamic contacts is pretty straightforward and I have no
>> issue there.  Permanent contacts are another matter however.  Because
>> they aren't managed by sorcery, none of the sorcery life-cycle observers
>> get called and you can't retrieve them by id from sorcery like you can
>> for dynamic contacts.  When you're dealing with contacts in general, you
>> have to understand and account for that.   So, my first suggestion is
>> that we treat the permanent contacts exactly as dynamic ones.   I.E.
>> They become fully-managed objects and get stored in the astdb with a
>> flag or their expiration set such that when asterisk reloads, they get
>> immediately expired.  They'll then be immediately rebuilt as the aors
>> are parsed.  We can then get rid of the aor-specific containers and the
>> processing around them.
>>
>
> What if contacts are stored in an external database? Your proposal would
> also have them going there as well. This would require an explanation for
> deployers to know what is going on exactly and what they are. As well this
> means there would be a period of time where the permanent contacts would
> not be present if other instances are sharing the same database and one
> restarts.
>

They can't be today can they?  The dynamic ones are explicitly mapped to
astdb and the permanent ones are explicitly placed in the aor based
container.  Can you even use sorcery.conf to map contacts somewhere else
given that they're just strings in an aor?


>
>
>> Contact status is also pretty straightforward but there's one minor
>> complication related to the fact that status is mutable and sorcery
>> objects can't be...  So although status is backed up by a memory wizard
>> (which is just an ao2_container), in order to update the status you have
>> to clone the object and replace the original object with the new one
>> (well, you don't have to but Josh gets upset if you don't :)).   It's
>> easy enough but it's also extra code and an opportunity for reference
>> leaks.   My proposal here is that we modify the memory wizard such that
>> if the sorcery_update method is called, we compare the object reference
>> passed in with the object reference we have in the container and if it's
>> equal, we just noop.  You still need to call ast_sorcery_update to
>> trigger the observers but at least you don't have to clone the object
>> anymore.
>>
>
> As Richard said they are immutable so you don't need to lock. Having
> special cases that are excluded from the sorcery contract, just because you
> know the implementation is currently done in a specific manner is bad.
>
> Perhaps the answer is to just not use sorcery for contact status.
>

Like I said, it's a minor complication.  There's no pressing need to change
it right now.


>
>
>> Josh's Sorcery Caching proposal triggered another idea though.  What if
>> we created a layer, similar to the cache, that allowed in-memory changes
>> that don't persist.   I.E.  calling ast_sorcery_create_temp instead of
>> ast_sorcery_create would create the object only in memory and not
>> attempt to persist it to any other back-end.  When asterisk restarts,
>> you have the original versions.
>>
>
> Ummm, does this assume that sorcery objects are no longer immutable? Or
> are you just talking about permanent contacts like below?
>

Yes.  Permanent contacts would use all _temp functions.  Dynamic functions
could use the _temp functions, specifically update_temp for changes that
only need to remain in memory or the normal functions for changes that need
to be persisted.


>
>
>> Scenario:
>>
>> We get rid of contact_status and add those fields to contact.   When a
>> dynamic contact registers, we call ast_contact_create and it gets
>> persisted like today.  When it's status changes, we update the contact
>> with ast_contact_update_temp which stays in memory only.  When a contact
>> de-registers, we call ast_sorcery_delete and it gets removed from both
>> memory and astdb.  When asterisk restarts, we get only the remaining
>> persisted objects.
>>
>
> A contact is updated when the registration is refreshed, you have to
> persist that out to disk or if a restart occurs it'll be expired.
>

Yes, so for dynamic contacts, the expiration would be written with the
normal update and for status and rtt changes, with update_temp.


>
>> For permanent contacts, we do the same except we call
>> ast_sorcery_create_temp and ast_sorcery_delete_temp so they never get
>> persisted to astdb.
>>
>
> So. This is what I was originally trying to express when you did the
> wizard work, you've just gone about it in a different manner. I think this
> is a rather specialized addition that could be done in a better way.
>
> Right now sorcery wizards can't be programmatically added. They just
> can't. You have to do it from the sorcery.conf configuration file.
>

Not true.  ast_sorcery_apply_wizard_mapping allows yo to add a wizard to
any object type at run time.  That's what the config_wizard does.  When the
object type registers, it adds a memory wizard to the object type's wizard
list.  The config wizard then manipulates the memory wizard.


>
> What if they could be?
> What if you could add a wizard that's marked as caching (so it's only
> queried) but still have a handle to it to create/update/destroy?
>

You just described res_pjsip_config_wizard exactly.


> Wrap that up in a higher level API like you want above and you extend
> sorcery in a way that's useful in multiple ways.
>
>
Put the whole mutability thing aside for a minute.  I guess what I'm
suggesting that the default sorcery behavior could be an automatic
read/write cache in front of every sorcery wizard.  You can then control
whether how writes are handled (if at all) by options you give when you
register the object type, and options you give on CRUD calls.

For my contact scenario, I'd say writable cache and a writable back end
(astdb by default).  When writing a new dynamic contact or updating the
expiration, I'd say write-through.  When updating status, I'd say
cache-only.   For permanent contacts, it would always be cache-only.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20150430/276df5b9/attachment.html>