Commit 9295e3ef65 for ffmpeg
commit 9295e3ef651883e05ec9d30c6581b536ec8ada73
Author: Stefan Singer <stefan.singer@quartr.se>
Date: Tue Oct 20 15:24:34 2026 +0200
avfilter/whisper: include detected language in JSON output
The whisper filter exposes segment timestamps and text in JSON but omits the language selected by whisper.cpp. Clients consuming multilingual transcripts cannot distinguish the language of each transcription chunk without detecting it again.
Read whisper_full_lang_id() after each successful transcription and include its language code in JSON segments when available. This leaves the configured language and audio-frame metadata unchanged. Document that the code applies to the whole chunk, not to individual segments.
Tested with a minimal --enable-whisper build, make fate (without external SAMPLES), and a local WAV transcription using a multilingual tiny model.
Signed-off-by: Stefan Singer <stefan.singer@quartr.se>
diff --git a/doc/filters.texi b/doc/filters.texi
index be3058b1a1..79ac811744 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -7792,6 +7792,9 @@ If the destination is a file and it already exists, it will be overwritten.
@item format
The destination format string; it could be "text" (only the transcribed text
will be sent to the destination), "srt" (subtitle format) or "json".
+JSON segments include a @code{language} field containing the language
+code reported by whisper.cpp for the transcription chunk, or @code{unknown}
+if none is available. All segments from the same chunk have the same language.
Default value: @code{"text"}
@item max_len
diff --git a/libavfilter/af_whisper.c b/libavfilter/af_whisper.c
index 33d8a2112a..7f41ea1f16 100644
--- a/libavfilter/af_whisper.c
+++ b/libavfilter/af_whisper.c
@@ -240,6 +240,8 @@ static void run_transcription(AVFilterContext *ctx, AVFrame *frame, int samples)
return;
}
+ const int lang_id = whisper_full_lang_id(wctx->ctx_wsp);
+ const char *detected_language = lang_id >= 0 ? whisper_lang_str(lang_id) : NULL;
const int n_segments = whisper_full_n_segments(wctx->ctx_wsp);
char *segments_text = NULL;
@@ -292,7 +294,9 @@ static void run_transcription(AVFilterContext *ctx, AVFrame *frame, int samples)
wctx->index++;
} else if (!av_strcasecmp(wctx->format, "json")) {
- buf = av_asprintf("{\"start\":%" PRId64 ",\"end\":%" PRId64 ",\"text\":\"%s\"}\n", start_t, end_t, text_cleaned);
+ buf = av_asprintf("{\"start\":%" PRId64 ",\"end\":%" PRId64 ",\"text\":\"%s\",\"language\":\"%s\"}\n",
+ start_t, end_t, text_cleaned,
+ detected_language ? detected_language : "unknown");
} else
buf = av_asprintf("%s\n", text_cleaned);