[asterisk-bugs] [JIRA] (ASTERISK-28830) Incorrect UTF-8 handling when using function FILTER

[Richard Mudgett (JIRA)]

    [ https://issues.asterisk.org/jira/browse/ASTERISK-28830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=250330#comment-250330 ] 

Richard Mudgett edited comment on ASTERISK-28830 at 4/15/20 8:41 AM:
---------------------------------------------------------------------

Have you read the code and the output?
There is no problem with allowed characters. The problem is that FILTER function substitutes some characters while first argument is passed as a range of characters and the second argument passed as a variable (CALLERID in my case) which causes system to crash and error.

$\{FILTER(а-я,$\{CALLERID(name)})} - Crash, some characters in the second argument are broken and substituted with garbage

$\{FILTER(абвгдеёжзийклмнопрстуфхцчшщъыьэюя,$\{CALLERID(name)})} - No crash, no substitutes



was (Author: newborn838):
Have you read the code and the output?
There is no problem with allowed characters. The problem is that FILTER function substitutes some characters while first argument is passed as a range of characters and the second argument passed as a variable (CALLERID in my case) which causes system to crash and error.

${FILTER(а-я,${CALLERID(name)})} - Crash, some characters in the second argument are broken and substituted with garbage

${FILTER(абвгдеёжзийклмнопрстуфхцчшщъыьэюя,${CALLERID(name)})} - No crash, no substitutes


> Incorrect UTF-8 handling when using function FILTER
> ---------------------------------------------------
>
>                 Key: ASTERISK-28830
>                 URL: https://issues.asterisk.org/jira/browse/ASTERISK-28830
>             Project: Asterisk
>          Issue Type: Bug
>      Security Level: None
>          Components: Functions/General
>    Affects Versions: 13.31.0
>            Reporter: newborn
>
> As you can see, after using FILTER we're getting wrong symbols (Коп▒▒а")
> This causes dialplan errors and crash 
> How to filter UTF-8 (Russian alphabet) correctly?
> Code
> {noformat}
> NoOp(.. Processing call from ${CALLERID(all)} to ${EXTEN}@${CONTEXT}-GLOBAL ..);
> Set(CALLERID(name)=${FILTER(А-Яа-яA-Za-z\040\050\051\055\056,${CALLERID(name)})});
> {noformat}
> Result
> {noformat}
> [Apr 14 18:26:08]     -- Executing [0999 at inbound-trunk:1] NoOp("IAX2/02asterisk03-2864", ".. Processing call from "ООО Рога и Копыта" <7***> to 0299 at inbound-trunk-GLOBAL ..") in new stack
> [Apr 14 18:26:08]     -- Executing [0299 at inbound-trunk:2] Set("IAX2/msk02asterisk03-11656", "CALLERID(name)=ООО Рога и Коп▒▒а") in new stack
> [Apr 14 18:26:08] WARNING[1322]: res_odbc.c:550 ast_odbc_print_errors: SQL Execute returned an error: 22021: ERROR: invalid byte sequence for encoding "UTF8": 0xd1 0xd1;
> Error while preparing parameters
> [Apr 14 18:26:08] WARNING[1322]: res_odbc.c:445 ast_odbc_prepare_and_execute: SQL Execute error -1!
> [Apr 14 18:26:08] WARNING[1322]: cel_odbc.c:797 odbc_log: Insert failed on 'asterisk-local:cel'.  CEL failed: INSERT INTO cel 
> [Apr 14 18:26:16]     -- Executing [s at macro-inboundcall:5] Set("SIP/00001a28", "CALLFILENAME=in-2020-04-14-Tuesday--18h26m--from--ООО Рога и Коп▒▒а-7***--to-999--1586877968.66779") in new stack
> [Apr 14 18:26:16] ERROR[21790][C-00000df1]: json.c:870 ast_json_vpack: Error building JSON from '{s: s, s: s}': Invalid UTF-8 string.
> [Apr 14 18:26:16] ERROR[21790][C-00000df1]:   Got 19 backtrace records
> # 0: /usr/sbin/asterisk(ast_json_pack+0x99) [0x4ff6e9]
> # 1: /usr/sbin/asterisk(ast_channel_publish_varset+0x25) [0x5874a5]
> # 2: /usr/sbin/asterisk(pbx_builtin_setvar_helper+0x123) [0x54b143]
> # 3: /usr/sbin/asterisk(pbx_builtin_setvar+0xc6) [0x54b3f6]
> # 4: /usr/sbin/asterisk(pbx_exec+0xb9) [0x5432d9]
> # 5: /usr/sbin/asterisk() [0x53b92e]
> # 6: /usr/sbin/asterisk(ast_spawn_extension+0x30) [0x53c410]
> # 7: /usr/lib64/asterisk/modules/app_macro.so(+0x2d17) [0x7f89f309dd17]
> # 8: /usr/sbin/asterisk(pbx_exec+0xb9) [0x5432d9]
> # 9: /usr/sbin/asterisk(ast_app_exec_macro+0x5e) [0x443f6e]
> #10: /usr/lib64/asterisk/modules/app_dial.so(+0xbd0d) [0x7f89f6118d0d]
> #11: /usr/lib64/asterisk/modules/app_dial.so(+0xd2d6) [0x7f89f611a2d6]
> #12: /usr/sbin/asterisk(pbx_exec+0xb9) [0x5432d9]
> #13: /usr/sbin/asterisk() [0x53b92e]
> #14: /usr/sbin/asterisk() [0x53c95e]
> #15: /usr/sbin/asterisk() [0x53de1b]
> #16: /usr/sbin/asterisk() [0x5a3f4e]
> #17: /lib64/libpthread.so.0(+0x7e65) [0x7f8a2ec4fe65]
> #18: /lib64/libc.so.6(clone+0x6d) [0x7f8a2e22088d]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)