jambonz Adds Speech-to-Speech Support for Alibaba's Qwen Omni-Realtime

We’re happy to announce that jambonz now supports Alibaba’s Qwen Omni-Realtime models (the models behind the recent Qwen-Audio-3.0-Realtime launch) as a first-class speech-to-speech vendor. If you run jambonz v11 or later with the mediajam media engine, you can connect any phone call to a full-duplex Qwen voice agent with a single verb.

Why Qwen Omni-Realtime Matters for Voice Agents

Qwen Omni-Realtime is a native speech-to-speech model: one model listens, reasons, and speaks over a single persistent connection, rather than chaining STT → LLM → TTS. That architecture shows up where it counts on a phone call:

• Latency. In our live testing against the Singapore endpoint, the model’s first audio arrived about 400 ms after the caller stopped speaking, squarely in natural-conversation territory.
• Semantic turn detection. Beyond plain voice-activity detection, Qwen’s semantic_vad weighs whether the caller’s utterance actually sounds finished before taking the turn.
• Streaming transcripts both ways. Caller speech is transcribed incrementally while they speak, and the assistant’s speech arrives with a synchronized text transcript, so your application always has text to log, analyze, or act on.
• Scale of language support, function calling, and a catalog of 55 voices.

How to Add Qwen Omni-Realtime to a jambonz Call

If you’ve used the OpenAI Realtime API with jambonz, this will feel instantly familiar. Qwen’s wire protocol is an OpenAI-Realtime dialect, and jambonz exposes it the same way. Here’s a minimal application using the @jambonz/sdk WebSocket interface:

session
  .answer()
  .qwen_s2s({
    model: 'qwen3.5-omni-flash-realtime',
    auth: {
      apiKey: process.env.DASHSCOPE_API_KEY,
    },
    llmOptions: {
      session_update: {
        modalities: ['text', 'audio'],
        voice: 'Ethan',
        instructions: 'You are a friendly and helpful voice assistant. ' +
          'Keep your responses concise and conversational.',
        turn_detection: {
          type: 'semantic_vad',
          threshold: 0.5,
          silence_duration_ms: 800,
        },
      },
      // the agent speaks first; omit this to wait for the caller
      response_create: {
        instructions: 'Greet the caller warmly and ask how you can help.',
      },
    },
    actionHook: '/s2s-complete',
  })
  .send();

A few practical notes from our testing:

• Endpoints and regions. The default endpoint is Alibaba’s international one (dashscope-intl.aliyuncs.com). You can set auth.host to a workspace-scoped endpoint (ws-<workspaceId>.<region>.maas.aliyuncs.com, which Alibaba recommends for performance) or to the China (Beijing) endpoint. DashScope API keys are region-bound, so key and endpoint must match.
• Turn detection tuning. silence_duration_ms controls how long a pause ends the caller’s turn. Lower values respond faster but will jump into mid-sentence pauses; we found 800 ms (the vendor default) a good starting point with semantic_vad.
• Greeting behavior. The model stays silent until the caller speaks unless you provide a response_create, a deliberate choice so both agent-first and caller-first call flows are easy.

Where to Learn More About Qwen Omni-Realtime

• The Qwen Omni Realtime tutorial walks through the full setup, including function calling and events.
• A complete working example lives in jambonz/v10-examples.
• Qwen support ships in @jambonz/schema 0.4.1, @jambonz/sdk 0.10.0, and jambonz v11’s feature-server and mediajam.

As always, come find us in the jambonz community with questions. We’d love to hear what you build with it.