[asterisk-bugs] [JIRA] (ASTERISK-30500) Caller name corruption in encodings other than UTF-8

[Sean Bright (JIRA)]

    [ https://issues.asterisk.org/jira/browse/ASTERISK-30500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=261810#comment-261810 ] 

Sean Bright commented on ASTERISK-30500:
----------------------------------------

Packet captures of the invalid SIP messages would also be beneficial.

> Caller name corruption in encodings other than UTF-8
> ----------------------------------------------------
>
>                 Key: ASTERISK-30500
>                 URL: https://issues.asterisk.org/jira/browse/ASTERISK-30500
>             Project: Asterisk
>          Issue Type: Bug
>      Security Level: None
>          Components: Resources/res_pjsip
>    Affects Versions: 18.17.0
>         Environment: FreeBSD 13.2
>            Reporter: Basil Mi
>            Assignee: Basil Mi
>            Severity: Major
>
> After this change:   ASTERISK-27830
> ===================================
> {quote}
> 2023-02-16 10:05 +0000 [1ddfb7551a]  George Joseph <gjoseph at sangoma.com>
>         * res_pjsip: Replace invalid UTF-8 sequences in callerid name
>            * Added a new function ast_utf8_replace_invalid_chars() to
>             utf8.c that copies a string replacing any invalid UTF-8
>             sequences with the Unicode specified U+FFFD replacement
>             character.  For example:  "abc\xffdef" becomes "abc\uFFFDdef".
>             Any UTF-8 compliant implementation will show that character
>             as a � character.
>            * Updated res_pjsip:set_id_from_hdr() to use
>             ast_utf8_replace_invalid_chars and print a warning if any
>             invalid sequences were found during the copy.
>            * Updated stasis_channels:ast_channel_publish_varset to use
>             ast_utf8_replace_invalid_chars and print a warning if any
>             invalid sequences were found during the copy.
>            ASTERISK-27830
> {quote}
> ===================================
> Some legacy devices transmit the caller name in encodings other than UTF-8. For example, PBX Panasonic KX-TDE600 uses WINDOWS-1251 and it's not configurable.
> In this case we use a function ICONV in dialplan to convert caller name to UTF-8 (for incoming calls). And vice versa (for outcoming calls).
> Using new function {color:red} "ast_utf8_replace_invalid_chars" {color} distorts caller name to "�" characters before it can be converted to UTF-8 in dialplan.
> Users see “�������” on devices instead of valid the caller's name.
> This logic worked for almost 10 years and broke on 18.17.0.
> Need to be able to turn off the replacement of invalid UTF-8 sequences (f.e. from config). Or  be able to use the ICONV before replacement (before call {color:red} "ast_utf8_replace_invalid_chars" {color}).



--
This message was sent by Atlassian JIRA
(v6.2#6252)