Skip to content

Audio Encoding API

Flex Video supports optional Codec2 audio encoding for SimplePipeline configurations. Codec2 is a low-bitrate vocal codec designed for voice communication at 1,200 to 3,200 bps.

Audio Devices

List Audio Devices

GET /flex/audio-devices

Returns all ALSA capture (input) devices detected on the system. Use these device paths in the audio.device field when creating a pipeline with audio.source: "local".

curl -k https://<encoder-ip>:3539/flex/audio-devices

Response (200):

{
  "devices": [
    {
      "device": "dsnoop:1,0",
      "name": "USB Audio",
      "card_name": "USB",
      "card": 1,
      "subdevice": 0
    }
  ]
}
Field Type Description
device string ALSA device path (use this in audio.device)
name string PCM device name
card_name string Sound card name
card integer Sound card index
subdevice integer Subdevice index

AudioConfig

Add the audio field to a SimplePipeline to enable audio encoding:

curl -k -X POST https://<encoder-ip>:3539/flex/pipeline \
  -H "Content-Type: application/json" \
  -d '{
    "id": "camera-with-audio",
    "mode": "simple",
    "on_demand": true,
    "source": {
      "uri": "udp://239.1.1.1:5004"
    },
    "encoding": {
      "codec": "av1",
      "bitrate": 500
    },
    "output": {
      "uri": "rtsp://0.0.0.0:8731/camera-with-audio"
    },
    "audio": {
      "source": "stream",
      "codec2_mode": 2400,
      "noise_filter": true,
      "noise_suppression": true,
      "noise_suppression_level": 2,
      "auto_gain_control": true
    }
  }'

Fields

Field Type Default Description
source string stream Audio source: stream (extract from input), local (ALSA device), test (tone generator)
device string — ALSA device path. Required when source: "local". Use GET /flex/audio-devices to list devices.
test_wave integer 0 Wave type for test source (0=sine, 1=square, 2=saw, 3=triangle, 4=silence, 0–12). Only used when source: "test".
codec2_mode integer 2400 Codec2 encoding bitrate in bps. Valid: 3200, 2400, 1600, 1400, 1300, 1200. Higher = better voice quality.
volume number — Post-processing volume (0.0–2.0). When omitted, no volume adjustment. 1.0 = normal.
noise_filter boolean true 200 Hz high-pass filter for wind/rumble removal. Skipped for test sources.
noise_suppression boolean true ML-based noise suppression. Skipped for test sources.
noise_suppression_level integer 2 Suppression aggressiveness (1=low, 2=moderate, 3=high, 4=aggressive).
auto_gain_control boolean true Automatic gain control with limiter. Skipped for test sources.

Audio Source Examples

Stream extraction (from input video):

{
  "audio": {
    "source": "stream",
    "codec2_mode": 2400
  }
}

Local ALSA device (USB microphone):

{
  "audio": {
    "source": "local",
    "device": "plughw:1,0",
    "codec2_mode": 2400,
    "noise_filter": true,
    "noise_suppression": true
  }
}

Test tone:

{
  "audio": {
    "source": "test",
    "test_wave": 0,
    "codec2_mode": 3200
  }
}

Output Behavior

Transport Audio Delivery Video Codec Constraint
RTSP (local) Separate audio stream Any codec
UDP / Multicast / TCP Muxed via FlexMux AV1 required

FlexMux Constraint

UDP, multicast, and TCP output with audio requires encoding.codec: "av1". Other codecs will return a 400 VALIDATION_ERROR.

Validation Errors

Audio-related validation errors are returned as 400 VALIDATION_ERROR with specific messages in the details.errors array:

Condition Error Message
Invalid Codec2 mode audio.codec2_mode must be one of: 3200, 2400, 1600, 1400, 1300, 1200.
Test wave out of range audio.test_wave must be between 0 and 12.
Volume out of range audio.volume must be between 0.0 and 2.0.
Suppression level out of range audio.noise_suppression_level must be between 1 and 4.
Local source without device audio.device is required when audio.source is "local".
Invalid ALSA device path audio.device must be a valid ALSA device path (e.g., "dsnoop:0,0", "plughw:0,0", "hw:1,0", "default").
Stream audio with test video audio.source cannot be "stream" when video source is a test source.
Non-AV1 codec with UDP audio Audio over UDP/multicast/TCP output requires AV1 video codec (FlexMux constraint).

Codec2 Macro (Advanced Pipelines)

For advanced (raw GStreamer) pipelines, use the <flex:codec2 /> macro to insert a Codec2 encoder:

curl -k -X POST https://<encoder-ip>:3539/flex/pipeline \
  -H "Content-Type: application/json" \
  -d '{
    "id": "advanced-audio",
    "mode": "advanced",
    "on_demand": true,
    "raw_pipeline": "... ! <flex:codec2 mode=2400 /> ! ..."
  }'
Macro Result
<flex:codec2 /> Codec2 encoder at mode 2400 (default)
<flex:codec2 mode=1600 /> Codec2 encoder at mode 1600

Mode must be one of: 3200, 2400, 1600, 1400, 1300, 1200. Invalid modes return a 400 VALIDATION_ERROR.