[Asterisk-Users] Everyone-- the scoop on Bison/Flex --

[Steve Murphy]

Hey, everybody---

Ignorance CAN be bliss, at least for a while, but, ....  Just so you
know...

A week or two ago, some upgrades to the expression parser (you know, the
expressions you put in $[ ... ]  in your extensions.conf file) that I
submitted, have been merged into the CVS HEAD of the source.

Hopefully, for around 99.9% of you, it won't make any difference to you.
The Makefile has also been "surgically altered" to detect which versions
of bison and flex are on your system, and compile the right versions of
the code to suit your environment.

The docs/README.variables outlines what you gain/lose either way, in
every thinkable way I could imagine.

Why did I/we muck around with this? Because, when I started writing my
own little $[ ] expressions, I had a devil of a time making them work,
and then when I read README.variables, I discovered why. You had to use
a single space to separate the tokens in the expressions. One more or
less, in the wrong spot, and bang! your expression fairly silently got
an error, and you had to figure out why.

So, I submitted, actually, two sets of fixes. The first, which was
entered months ago, added some code to the scanner, which was a hand-
written affair, to overlook spaces to a degree. Basically, the mods let
you have one or more spaces to separate tokens. You still needed to have
space between tokens, but this fix at least let you add extra spaces to
your heart's content. It also introduced some syntax error messages that
might help you locate the problem if you did have errors.  In the
logfile, it shows the POST-EVALUATED expression, and throws some extra
characters to literally point to where the syntax error was encountered.
This first submission was kind of painful, as it forced everyone to
upgrade their bison to 1.875, if they had bison problems. Some minor
upgrade to bison that allowed the better error messages demanded this.
There was some confusion, a bit of pain, but I hope that is past.

This next submission replaces the hand-written scanner with a full-
fledged flex scanner. The rub here is that Asterisk threading demands
that all code is "thread-safe", "re-entrant", etc., and a "pure" scanner
from flex wasn't introduced until 2.5.31. So, you can't use the flex
scanner until you have 2.5.31 installed, and frankly, no linux
distribution does this yet(AFAIK). You have to hand-install it if you
want the newer scanner. Of course the bison parser file needed to be
upgraded to work with the flex scanner, so the two come as a (new) set.

What do you get if you hand-install flex-2.5.31? Well, now, your
expressions don't have to have at least one space separating the tokens.
You can pack them tighter if you want, if the syntax allows it. 
For instance, you don't have to say $[2 * ${lala}]; you can say 
$[2*${lala}] or $[2*   ${lala}] or whatever you want.

Also, you get some extra operators. You can now use Unary Minus, like 
$[2 * -${lala}]; and You get a logical negation operator, "!". So, you
can say $[ !${var} : "HELLO" ]  to test if var DOESN'T begin with
"HELLO".  (or $[!${var}:"HELLO"] if you like!).  And, you also get the
binary "=~"  matching operator, which is like :, but isn't anchored to
the beginning of the string. You can read all about them in
README.variables.

And, one final plus is that if we ever need to add new operators or
syntax to the current $[ ] syntax, hopefully such can be done in bison
and flex more easily and robustly than it could via the hand-written
scanner, which was simple, yet a bit intricate, all at the same time.

So, if you never get around to installing flex-2.5.31, you don't need to
feel impelled to go and install it. Asterisk will work just fine with
what you've got now, without the upgrade. You still get the better error
messages in your log files. If you are excited to use the new operators,
then you'll want to use the new stuff. Otherwise, you can just let it
float as-is. Sooner or later, Linux distributions will start including
flex-2.5.31, and you'll silently get the new syntax, if you need it.

Now, for the .1% of you that have done "fancy stuff" with your
expressions up till I meddled: Beware the spaces, especially if you were
counting on them for anything. There may be subtle differences in
behavior, that will puzzle or just plain confound you. A few extra
double quotes (") may be useful to you. Find the fine points in the
README.variables.

So, I hope you found it worth your while to read this.

murf


-- 
Steve Murphy <murf at e-tools.com>
Electronic Tools Company
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3317 bytes
Desc: not available
Url : http://lists.digium.com/pipermail/asterisk-users/attachments/20050603/85ac5bba/smime.bin