[asterisk-bugs] [JIRA] (ASTERISK-23917) res_http_websocket: Delay in client processing large streams of data causes disconnect and stuck socket

[Matt Jordan (JIRA)]

     [ https://issues.asterisk.org/jira/browse/ASTERISK-23917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matt Jordan updated ASTERISK-23917:
-----------------------------------

    Description: 
*Note:* This issue was originally seen when sending large volumes of data to a connected ARI client over a websocket, but it theoretically could occur with any version of Asterisk that uses a websocket.

When a client takes a long time to process information received from Asterisk, a write operation using fwrite may fail to write all information. This causes the underlying file stream to be in an unknown state, such that the socket must be disconnected. Unfortunately, there are two problems with this in Asterisk's existing websocket code:

# Periodically, during the read loop, Asterisk must write to the connected websocket to respond to pings. As such, Asterisk maintains a reference to the session during the loop. When {{ast_http_websocket_write}} fails, it may cause the session to decrement its ref count, but this in and of itself does not break the read loop. The read loop's write, on the other hand, does not break the loop if it fails. This causes the socket to get in a 'stuck' state, preventing the client from reconnecting to the server.
{code}
		if (*opcode == AST_WEBSOCKET_OPCODE_PING) {
			ast_websocket_write(session, AST_WEBSOCKET_OPCODE_PONG, *payload, *payload_len);
                        /* This needs an off nominal handler */
		}
{code}
# More importantly, however, is that the fwrite in {{ast_http_websocket_write}} fails with a large volume of data when the client takes awhile to process the information. When it does fail, it fails writing only a portion of the bytes. With some debugging, it was shown that this was failing in a similar fashion to ASTERISK-12767. Switching this over to {{ast_careful_fwrite}} with a long enough timeout solved the problem.

Patches for Asterisk 11 and 12:
* https://reviewboard.asterisk.org/r/3624
* 

  was:
*Note:* This issue was originally seen when sending large volumes of data to a connected ARI client over a websocket, but it theoretically could occur with any version of Asterisk that uses a websocket.

When a client takes a long time to process information received from Asterisk, a write operation using fwrite may fail to write all information. This causes the underlying file stream to be in an unknown state, such that the socket must be disconnected. Unfortunately, there are two problems with this in Asterisk's existing websocket code:

# Periodically, during the read loop, Asterisk must write to the connected websocket to respond to pings. As such, Asterisk maintains a reference to the session during the loop. When {{ast_http_websocket_write}} fails, it may cause the session to decrement its ref count, but this in and of itself does not break the read loop. The read loop's write, on the other hand, does not break the loop if it fails. This causes the socket to get in a 'stuck' state, preventing the client from reconnecting to the server.
{code}
		if (*opcode == AST_WEBSOCKET_OPCODE_PING) {
			ast_websocket_write(session, AST_WEBSOCKET_OPCODE_PONG, *payload, *payload_len);
                        /* This needs an off nominal handler */
		}
{code}
# More importantly, however, is that the fwrite in {{ast_http_websocket_write}} fails with a large volume of data when the client takes awhile to process the information. When it does fail, it fails writing only a portion of the bytes. With some debugging, it was shown that this was failing in a similar fashion to ASTERISK-12767. Switching this over to {{ast_careful_fwrite}} with a long enough timeout solved the problem.

Patches for Asterisk 11 and 12:
*
* 


> res_http_websocket: Delay in client processing large streams of data causes disconnect and stuck socket
> -------------------------------------------------------------------------------------------------------
>
>                 Key: ASTERISK-23917
>                 URL: https://issues.asterisk.org/jira/browse/ASTERISK-23917
>             Project: Asterisk
>          Issue Type: Bug
>      Security Level: None
>          Components: Resources/res_http_websocket
>    Affects Versions: 11.10.2, 12.3.2
>            Reporter: Matt Jordan
>            Assignee: Matt Jordan
>
> *Note:* This issue was originally seen when sending large volumes of data to a connected ARI client over a websocket, but it theoretically could occur with any version of Asterisk that uses a websocket.
> When a client takes a long time to process information received from Asterisk, a write operation using fwrite may fail to write all information. This causes the underlying file stream to be in an unknown state, such that the socket must be disconnected. Unfortunately, there are two problems with this in Asterisk's existing websocket code:
> # Periodically, during the read loop, Asterisk must write to the connected websocket to respond to pings. As such, Asterisk maintains a reference to the session during the loop. When {{ast_http_websocket_write}} fails, it may cause the session to decrement its ref count, but this in and of itself does not break the read loop. The read loop's write, on the other hand, does not break the loop if it fails. This causes the socket to get in a 'stuck' state, preventing the client from reconnecting to the server.
> {code}
> 		if (*opcode == AST_WEBSOCKET_OPCODE_PING) {
> 			ast_websocket_write(session, AST_WEBSOCKET_OPCODE_PONG, *payload, *payload_len);
>                         /* This needs an off nominal handler */
> 		}
> {code}
> # More importantly, however, is that the fwrite in {{ast_http_websocket_write}} fails with a large volume of data when the client takes awhile to process the information. When it does fail, it fails writing only a portion of the bytes. With some debugging, it was shown that this was failing in a similar fashion to ASTERISK-12767. Switching this over to {{ast_careful_fwrite}} with a long enough timeout solved the problem.
> Patches for Asterisk 11 and 12:
> * https://reviewboard.asterisk.org/r/3624
> * 



--
This message was sent by Atlassian JIRA
(v6.2#6252)