VLC Developers Working on AI-Powered Real-Time Subtitles – OMG! Ubuntu!

Cloud-free & hassle-free captioning for any videoReal-time AI subtitling is in the works for VLC, the phenomenally popular open-source media player.VideoLAN, the non-profit in charge of VLC development, demoed automatic subtitle generation in VLC during CES 2025 —where not using AI made you stand out this year— and shared a clip of the feature in action at their CES booth on X last week.No cloud service or internet connection is needed for VLC to auto-generate subtitles with AI. The feature works offline to provide real-time translation support for more 100+ languages; is able to translate/display two languages at once; and can ‘save’ translations to an SRT file.“VLC automatic subtitles generation and translation based on local and open source AI models running on your machine working offline, and supporting numerous languages,” VideoLAN says in its X post.Photos shared in follow up posts show a video of British comedian Ricky Gervais’ (iconic/infamous, depending on your sensitivities) Golden Globes 2020 roast with real-time subtitles displayed in various different languages, including Japanese and French:Most of us are familiar with auto-generated captions on YouTube, social media “creators” use AI caption apps to ‘burn in’ caption on short videos, and many leading video editors have plugins and/or native real-time transcription features too.It makes sense to leverage similar tech in VLC. I like to watch local movies, TV shows, and DVDs with subtitles on, but sometimes the file/disc doesn’t have subtitles on it. VLC has a built-in subtitle download tool which saves the need to trawl a sea of malware-peddling free subtitle download sites to find an SRT file.Of course, you can only download subtitle file(s) if someone, somewhere shared/made one in the language you need. If they haven’t, or you want subtitles on a custom video (or another kind of content – not judging, but do they really have much talking?1), you’re on your own.I wonder if VLC’s auto-subtitle feature makes use of OpenAI’s Whisper, as it’s the leading open-source neural network for speech recognition tasks. Whatever model VLC uses for this, it’s nice that it runs offline, decoupled from the cloud, with no logins, signups, or network connection needed.Asked by one follower if they’d consider using a cloud service instead VideoLAN reply to say “absolutely not” as “the goal is to not depend on an expensive cloud operation!”At the time of writing this VLC hasn’t (that I’m aware) indicated when/if a public demo of the player with AI generated subtitles will be made available for testing, but indicate this feature may be available as part of VLC 4.0.For now, if that MP4 movie you want to watch lacks subtitles, you’ll need to stick to finding an SRT/VTT file. It’s not a leap to imagine this feature being used to general real-time unofficial AI dubs – though AI can’t act, and such a move could have wider industry implementations leading to streaming services replacing dubbing casts with AI.By buying me a $2 coffee, you’re directly funding continued independent coverage of the Linux ecosystem—content for you, and for everyone!© 2025 Ohso Media

Source: https://www.omgubuntu.co.uk/2025/01/vlc-demo-ai-real-time-subtitles-for-any-video