[asterisk-users] Asterisk 15, Jack, streams, speech recognition… so many questions!

[Jonathan H]

I know Asterisk has a speech recognition interface built in, but I
need to go beyond that, with APIs like Lex, Wit or Luis etc.

There's also the very cheap/free high quality speech synthesis
services like Amazon Polly, which can also return an audio stream
object (or save a file).

These APIs can respond really quickly and accurately as they can
receive and interpret an audio stream, but in all the Asterisk speech
recognition tools I can find, they all say that they need to save the
speech to a file, then convert, then upload via whatever API.

Removing the "record in Asterisk/store as file/convert file/upload
file  <> receive stream/save file/convert file/playback in Asterisk"
part of the sequence would save vital seconds of silence and caller
annoyance.

I am looking at, for example, Google's speech to text service, and it
can cope with the following codecs (below), some of which can be used
directly from Asterisk, yes? So there is no need to transcode? As
Google returns "live" word results, I could set an API to "watch" for
a trigger word, and then use the AGI to trigger something in Asterisk.

Well, that is how I am assuming it can/should be done!

Also, there is JackAudio. And I'm thinking to myself that surely
something could be done here, but then also, surely one of the people
far more skilled than me would have already done this?

So, have they either missed a trick here, or is there something in a
recent version of Asterisk which could do this? Some steer my in the
right direction!

LINEAR16 Uncompressed 16-bit signed little-endian samples (Linear PCM).
FLAC FLAC (Free Lossless Audio Codec) is the recommended encoding
because it is lossless--therefore recognition is not compromised--and
requires only about half the bandwidth of LINEAR16. FLAC stream
encoding supports 16-bit and 24-bit samples, however, not all fields
in STREAMINFO are supported.
MULAW 8-bit samples that compand 14-bit audio samples using G.711 PCMU/mu-law.
AMR Adaptive Multi-Rate Narrowband codec. sample_rate_hertz must be 8000.
AMR_WB Adaptive Multi-Rate Wideband codec. sample_rate_hertz must be 16000.
OGG_OPUS Opus encoded audio frames in Ogg container (OggOpus).
sample_rate_hertz must be 16000.
SPEEX_WITH_HEADER_BYTE Although the use of lossy encodings is not
recommended, if a very low bitrate encoding is required, OGG_OPUS is
highly preferred over Speex encoding. The Speex encoding supported by
Cloud Speech API has a header byte in each block, as in MIME type
audio/x-speex-with-header-byte. It is a variant of the RTP Speex
encoding defined in RFC 5574. The stream is a sequence of blocks, one
block per RTP packet. Each block starts with a byte containing the
length of the block, in bytes, followed by one or more frames of Speex
data, padded to an integral number of bytes (octets) as specified in
RFC 5574. In other words, each RTP header is replaced with a single
byte containing the block length. Only Speex wideband is supported.
sample_rate_hertz must be 16000.

Example services...
https://cloud.google.com/speech/reference/rpc/google.cloud.speech.v1#google.cloud.speech.v1.RecognitionConfig
https://pypi.python.org/pypi/SpeechRecognition/3.7.1
https://wit.ai/faq
https://wiki.asterisk.org/wiki/display/AST/Asterisk+15+Application_JACK