On-the-fly translations in Microsoft Edge

On-the-fly translations in Microsoft Edge

Most of the content on the web is in English, but the margin is not overwhelming: it’s about 59%. Thus, a lot is published in other languages, and if written text can be translated in any major browser without any fuss, with the help of respective integrated functions, it’s not that easy with videos. Microsoft understands that and joins the real-time video translation race.

Translation function in web browsers: state of affairs

Yandex browser, current leader in real-time video translation. Snapshot from the product's website

The company that currently (end of 2023) dominates the game of real-time translation in web browsers is Russia’s Yandex. In 2022, its namesake product learned to overlay videos with subtitles in your language of choice first, and then the developer enhanced it with on-the-fly voice-over video translation, and cemented its position as the leader in this field by connecting Yandex browser to neural networks (also known as AI) that do real-time synchronous translation of streamed videos.

There is a caveat to those Yandex browser capabilities, though: it all revolves around the Russian language. As of this writing, that is, since the company does not plan to stop and continues to further improve one of its flagship products.

As for Chrome, there are extensions that add somewhat similar functionality to the world’s most popular browser, but it feels like none of them delivers as expected, i.e., it’s not a seamless experience as with the Yandex browser. But, Google definitely has something in the works.

Microsoft’s expertise in the field of real-time translation

Microsoft has actually been one of the pioneers in on-the-fly speech translation. The respective service has been offered in its Skype messenger since 2015, when AI was still a combination of letters from sci-fi and ramblings of assorted visionaries.

Since then, the company has been improving this functionality steadily; nowadays, turning on real-time translation in Skype, you are pleasantly surprised by TruVoice, a technology that makes the translated speech sound much like that of the original speaker.

So far, subtitles only, and no voice-over?

Real-time translation of videos in a browser will soon be a must. Image by vectorjuice from Freepik

There was no official communication covering the new functionality, it was simply noticed in the Canary channel, one of those the company uses to grant users access to the latest versions of its products before they are released. This means the knowledge about the new feature is very limited.

So far, the company plans to automatically provide subtitles to videos in four languages:

– English;

– Russian;

– Spanish;

– French.

According to Statista, these are the four most common languages on the web, so the selection is well justified. However, it is not clear why Microsoft lags behind and rolls out subtitles only, given its extensive voice-over translation expertise mentioned above. A possible reason is that Edge is used primarily in the desktop realm, and there, it’s in third place, winning less than 10% of all users worldwide. On the other hand, the browser race has seen dramatic twists in the last couple decades, and Microsoft knows this first-hand. So, starting it slowly may be a strategy of a seasoned player.