Skype update translates English and Spanish in real time

The world is edging a little bit closer to Star Trek-style universal language translation, as the Skype Translator preview service goes live to select users on Monday.

Skype Translator tries to offer real-time translation of both spoken and written communications, so that users who speak completely different languages can converse with one another easily.

Right now, the service can handle near real-time audio translation between spoken English and Spanish, and text translation between more than 40 languages.

How does it work? As shown in the video above, Skype Translator will convert spoken English or Spanish sentences to text and display them (and their translation) in a sidebar next to the video stream. A computerized voice will also read the translated text aloud. The video shows students in Mexico and the US conversing in nearly real time, but the translation will introduce a delay between each conversation fragment. The video comes directly from Microsoft, so it probably represents a best-case scenario in terms of accurate speech recognition and translation.

Skype Translator uses machine learning to get better at converting text to speech, and also to become more adept at translation. That means the more it hears a word used in conversation, the better it will be at translating it accurately and using it in the proper context.

The children in the video are using pretty basic vocabulary and sentence construction, but in theory, given enough practice and fine-tuning Skype Translator will be able to translate more complicated phrases and sentences. Microsoft has also been working to allow the service to handle partial sentences, and scenarios in which one speaker cuts off another’s words. It’s still a long way from being able to handle all the complexities of everyday conversation, but Microsoft wants Skype Translator to be able to understand the nuances and intonation of language, not just the words.

How well does it work in real life? Surprisingly well, says Lance Ulanoff of Mashable. “The translation is about as close-to-instant as you can get and appears to be at least 90% accurate,” he writes. “Once I got used to it, it didn’t impede my conversation.... I even noted that many of my verbal ticks, like ‘uh’s’ and ‘umm’s,’ were missing from the spoken and written translations.”

Microsoft hopes that data gathered from early users will make the service even more accurate, and that it will be able to adjust to different accents and verbal habits.

The sign-up page for Skype Translator preview is still open, though Microsoft doesn’t promise instant access to everyone who signs up. The service works with mobile and desktop versions of Windows 8.1 and Windows 10 preview.