[asterisk-users] Multiple readfile oddities, newlines etc

[Jonathan H]

Just by chance, I was browsing the mailing list and it looks like
Sebastian's reply made it to the list, but not to me because it's
apparently blacklisted by UCEPROTECTL3.

This is what he wrote:

> Theres always garbage in the end of the files.
> I do this when I want to read a file:
>     same => n,Set(featurefile=/home/test/feature-1.txt)
>     same => n,Set(unfilteredfeat2=${FILE(${featurefile},0,1,l,u)})
>     same => n,Set(feature2=${SHIFT(unfilteredfeat2)})
> After that, add a , inside end of the file, so
> Cat feature-1.txt
> Reads:
> radio,
> Thus if there is garbage in the file, it will happen after the ,. Same if garbage happens to find its way into the end of the variable for some reason.

Well, OK, that sort of works, but it revealed even more odd things.
And, if tools like wc are saying that a file is a certain length, how
can there be junk hiding in it?

So, then I found that having different lengths in the file messed
things up, too. Can anyone tell me this is normal?!?

Have a look, this is for changing the following words in the
features.txt. In all of the examples, the LEN function gets it wrong,
too.

radio < gets cut a letter short even with SHIFT trimmer
radio, < works, but only by chance because , makes it 6 characters
radui12345 < garbage at end
radio12345, < shift trimmer removes garbage
rad < junk at end


Context: fromvoipfone201 Exten:5
 Using a string, feature is set to ---radio--- and is 7 long
 Using a variable, feature2 is set to ---radi--- and is 8 long
 Using a string with shift method, feature3 is set to ---radi--- and is 8 long

Context: fromvoipfone201 Exten:5
 Using a string, feature is set to ---radio,--- and is 7 long
 Using a variable, feature2 is set to ---radio,--- and is 8 long
 Using a string with shift method, feature3 is set to ---radio--- and is 8 long

Context: fromvoipfone201 Exten:5
 Using a string, feature is set to ---radio12345--- and is 7 long
[Oct 17 15:30:29] ERROR[5813][C-00000073]: json.c:704 ast_json_vpack:
Error building JSON from '{s: s, s: s}': Invalid UTF-8 string.
[Oct 17 15:30:29] ERROR[5813][C-00000073]: stasis_channels.c:773
ast_channel_publish_varset: Error creating message
 Using a variable, feature2 is set to ---radio12345G▒--- and is 8 long
[Oct 17 15:30:29] ERROR[5813][C-00000073]: json.c:704 ast_json_vpack:
Error building JSON from '{s: s, s: s}': Invalid UTF-8 string.
[Oct 17 15:30:29] ERROR[5813][C-00000073]: stasis_channels.c:773
ast_channel_publish_varset: Error creating message
[Oct 17 15:30:29] ERROR[5813][C-00000073]: json.c:704 ast_json_vpack:
Error building JSON from '{s: s, s: s}': Invalid UTF-8 string.
[Oct 17 15:30:29] ERROR[5813][C-00000073]: stasis_channels.c:773
ast_channel_publish_varset: Error creating message
 Using a string with shift method, feature3 is set to
---radio12345G▒--- and is 8 long

Context: fromvoipfone201 Exten:5
 Using a string, feature is set to ---radio12345,--- and is 7 long
[Oct 17 15:30:35] ERROR[5814][C-00000074]: json.c:704 ast_json_vpack:
Error building JSON from '{s: s, s: s}': Invalid UTF-8 string.
[Oct 17 15:30:35] ERROR[5814][C-00000074]: stasis_channels.c:773
ast_channel_publish_varset: Error creating message
 Using a variable, feature2 is set to ---radio12345,▒--- and is 8 long
[Oct 17 15:30:35] ERROR[5814][C-00000074]: json.c:704 ast_json_vpack:
Error building JSON from '{s: s, s: s}': Invalid UTF-8 string.
[Oct 17 15:30:35] ERROR[5814][C-00000074]: stasis_channels.c:773
ast_channel_publish_varset: Error creating message
[Oct 17 15:30:35] ERROR[5814][C-00000074]: json.c:704 ast_json_vpack:
Error building JSON from '{s: s, s: s}': Invalid UTF-8 string.
[Oct 17 15:30:35] ERROR[5814][C-00000074]: stasis_channels.c:773
ast_channel_publish_varset: Error creating message
 Using a string with shift method, feature3 is set to ---radio12345---
and is 8 long

Context: fromvoipfone201 Exten:5
 Using a string, feature is set to ---rad--- and is 7 long
[Oct 17 15:30:43] ERROR[5815][C-00000075]: json.c:704 ast_json_vpack:
Error building JSON from '{s: s, s: s}': Invalid UTF-8 string.
[Oct 17 15:30:43] ERROR[5815][C-00000075]: stasis_channels.c:773
ast_channel_publish_varset: Error creating message
 Using a variable, feature2 is set to ---rad▒--- and is 8 long
[Oct 17 15:30:43] ERROR[5815][C-00000075]: json.c:704 ast_json_vpack:
Error building JSON from '{s: s, s: s}': Invalid UTF-8 string.
[Oct 17 15:30:43] ERROR[5815][C-00000075]: stasis_channels.c:773
ast_channel_publish_varset: Error creating message
[Oct 17 15:30:43] ERROR[5815][C-00000075]: json.c:704 ast_json_vpack:
Error building JSON from '{s: s, s: s}': Invalid UTF-8 string.
[Oct 17 15:30:43] ERROR[5815][C-00000075]: stasis_channels.c:773
ast_channel_publish_varset: Error creating message
 Using a string with shift method, feature3 is set to ---rad▒--- and is 8 long

So, using the SHIFT trimmer cuts the junk, but still loses the last
character in some cases.

Here's the dialplan

    same => n,Set(feature=${FILE(/home/test/feature-1.txt,0,1,l,u)})
    same => n,Verbose(1,Using a string, feature is set to
---${feature}--- and is ${LEN(feature)} long)

    same => n,Set(featurefile=/home/test/feature-1.txt)
    same => n,Set(feature2=${FILE(${featurefile},0,1,l,u)})
    same => n,Verbose(1,Using a variable, feature2 is set to
---${feature2}--- and is ${LEN(feature2)} long)

    same => n,Set(unfilteredfeat=${FILE(${featurefile},0,1,l,u)})
    same => n,Set(feature3=${SHIFT(unfilteredfeat)})
    same => n,Verbose(1,Using a string with shift method, feature3 is
set to ---${feature3}--- and is ${LEN(feature3)} long)

Bug or... "feature"?!

On 17 October 2016 at 14:31, Jonathan H <lardconcepts at gmail.com> wrote:
> I have a plain text file, ASCII, unix line breaks. 1 single line, and all
> that is in it is the word "radio".
>
> Here's some test dialplan:
>
> exten => 5,1,Verbose(Context: ${CONTEXT} Exten:${EXTEN})
>
>     same => n,Set(feature=${FILE(/home/test/feature-1.txt,0,1,l,u)})
>     same => n,Verbose(${feature})
>
>     same => n,Set(featurefile=/home/test/feature-1.txt)
>     same => n,Set(feature2=${FILE(${featurefile},0,1,l,u)})
>     same => n,Verbose(${feature2})
>
> Both should output "radio", right? Here's the output:
>
>     -- Executing [5 at fromvoipfone201:2] Set("PJSIP/6001-00000052",
> "feature=radio") in new stack
>     -- Executing [5 at fromvoipfone201:3] Verbose("PJSIP/6001-00000052",
> "radio") in new stack
> radio
>
>     -- Executing [5 at fromvoipfone201:4] Set("PJSIP/6001-00000052",
> "featurefile=/home/test/feature-1.txt") in new stack
>     -- Executing [5 at fromvoipfone201:5] Set("PJSIP/6001-00000052",
> "feature2=radi") in new stack
>                                                         GAAAAARRRGGGHHHH!
> ^^^^^
>
>     -- Executing [5 at fromvoipfone201:6] Verbose("PJSIP/6001-00000052",
> "radi") in new stack
>
> And this is what's just at the top of the script. If I put it way down in
> another context:
>
>     -- Executing [s at track-handler:3] Verbose("Local/s at root-00000026;2",
> "/home/test/feature-1") in new stack
> /home/test/feature-1
>     -- Executing [s at track-handler:4] Set("Local/s at root-00000026;2",
> "feature=radio▒▒") in new stack
>                                                     EVEN MORE
> GAAAAARRRGGGHHHH!  ^^^^^
>
> [Oct 17 13:29:33] ERROR[5093][C-0000003c]: json.c:704 ast_json_vpack: Error
> building JSON from '{s: s, s: s}': Invalid UTF-8 string.
> [Oct 17 13:29:33] ERROR[5093][C-0000003c]: stasis_channels.c:773
> ast_channel_publish_varset: Error creating message
>     -- Executing [s at track-handler:5] Verbose("Local/s at root-00000026;2",
> "radio▒▒") in new stack
> radio▒▒
>     -- Executing [s at track-handler:6] GotoIf("Local/s at root-00000026;2",
> "1?radio▒▒,s,1") in new stack
>
>
> So, at the top of the file, when it's just a straight file as the filename,
> it gives the full word. If the filename is a variable, it strips and
> character. And further down, same thing, but it adds two weird blocks, which
> appear side by side in the console, but weirdly, stacked up in this email.
>
> (Oh, by the way, if there is more than one line in the file, even if I used
> "u" for line breaks, it adds a newline to the variable, Is that correct?)
>
> What am I missing? I've opened it in both nano and notepad++, I've used
> iconv and all the tools I can think of to check that file, and all the
> asterisk conf files, too.
> It all looks as it should here:
>
> $ cat feature-1.txt
> radio$ file feature-1.txt
> feature-1.txt: ASCII text
> $ wc -l feature-1.txt
> 0 feature-1.txt
> wc -c feature-1.txt
> 5 feature-1.txt
>
> After 6 hours struggling with this, I think I'm starting to lose the plot.
> Can anyone tell me where I'm going wrong? Thanks.