[asterisk-bugs] [JIRA] (ASTERISK-30500) Caller name corruption in encodings other than UTF-8

[Asterisk Team (JIRA)]

    [ https://issues.asterisk.org/jira/browse/ASTERISK-30500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=261808#comment-261808 ] 

Asterisk Team commented on ASTERISK-30500:
------------------------------------------

WARNING

JIRA will be going read-only at the end of April, 2023. We will be starting fresh on Github at https://github.com/asterisk/asterisk at that time. No issues or patches will be copied to Github. If you file an issue on JIRA at this time you will need to recreate it on Github after this date. The same applies if you have a patch.

WARNING

Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution. Please note that log messages and other files should not be sent to the Sangoma Asterisk Team unless explicitly asked for. All files should be placed on this issue in a sanitized fashion as needed.

A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report.

Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process].

Please note that once your issue enters an open state it has been accepted. As Asterisk is an open source project there is no guarantee or timeframe on when your issue will be looked into. If you need expedient resolution you will need to find and pay a suitable developer. Asking for an update on your issue will not yield any progress on it and will not result in a response. All updates are posted to the issue when they occur.

Please note that by submitting data, code, or documentation to Sangoma through JIRA, you accept the Terms of Use present at [https://www.asterisk.org/terms-of-use/|https://www.asterisk.org/terms-of-use/].

> Caller name corruption in encodings other than UTF-8
> ----------------------------------------------------
>
>                 Key: ASTERISK-30500
>                 URL: https://issues.asterisk.org/jira/browse/ASTERISK-30500
>             Project: Asterisk
>          Issue Type: Bug
>      Security Level: None
>          Components: Resources/res_pjsip
>    Affects Versions: 18.17.0
>         Environment: FreeBSD 13.2
>            Reporter: Basil Mi
>            Severity: Major
>
> After this change:   ASTERISK-27830
> ===================================
> {quote}
> 2023-02-16 10:05 +0000 [1ddfb7551a]  George Joseph <gjoseph at sangoma.com>
>         * res_pjsip: Replace invalid UTF-8 sequences in callerid name
>            * Added a new function ast_utf8_replace_invalid_chars() to
>             utf8.c that copies a string replacing any invalid UTF-8
>             sequences with the Unicode specified U+FFFD replacement
>             character.  For example:  "abc\xffdef" becomes "abc\uFFFDdef".
>             Any UTF-8 compliant implementation will show that character
>             as a � character.
>            * Updated res_pjsip:set_id_from_hdr() to use
>             ast_utf8_replace_invalid_chars and print a warning if any
>             invalid sequences were found during the copy.
>            * Updated stasis_channels:ast_channel_publish_varset to use
>             ast_utf8_replace_invalid_chars and print a warning if any
>             invalid sequences were found during the copy.
>            ASTERISK-27830
> {quote}
> ===================================
> Some legacy devices transmit the caller name in encodings other than UTF-8. For example, PBX Panasonic KX-TDE600 uses WINDOWS-1251 and it's not configurable.
> In this case we use a function ICONV in dialplan to convert caller name to UTF-8 (for incoming calls). And vice versa (for outcoming calls).
> Using new function {color:red} "ast_utf8_replace_invalid_chars" {color} distorts caller name to "�" characters before it can be converted to UTF-8 in dialplan.
> Users see “�������” on devices instead of valid the caller's name.
> This logic worked for almost 10 years and broke on 18.17.0.
> Need to be able to turn off the replacement of invalid UTF-8 sequences (f.e. from config). Or  be able to use the ICONV before replacement (before call {color:red} "ast_utf8_replace_invalid_chars" {color}).



--
This message was sent by Atlassian JIRA
(v6.2#6252)