Language technologist: Europe needs a language infrastructure, not just Google Translate

Europe's language industry has lost ground to American technology companies. But it still has a chance to offer better services, says Jochen Hummel.

Jochen Hummel is Chairman of LT-Innovate, the European Association of the Language Technology Industry, he is also the CEO of ESTeam and Coreon.

He spoke with Christophe Leclercq, EurActiv’s Founder.

Could you start by telling me what LT-Innovate is about?

LT-Innovate is an association of European language technology companies. We have about 180 members, primarily SMEs in Europe, and companies from different areas of language technology like translation, text analytics, text recognition. All areas of language technology.

It seems that the European translation industry is losing out to American initiatives, notably Google Translate. Why is that?

You are referring to the European machine translation industry and technology, because when it comes to language services, Europe is of course very strongly positioned. When it comes to technology, indeed, European companies are not that good at marketing their products and deploying them as commercial solutions. We develop a lot. Even Google Translate was originally developed by a European. But when it comes to converting these into marketable products, we are not as good as the Americans. That is also what we are trying to address in LT-Innovate, to help our members to develop better market strategies and alliances.

It seems that there is a market opportunity combining machine translation, and very cheap post-editing by humans, which could be a kind of Uber for translation. Who could do that? Could it be a European company?

It surely could be. I have been asked this question before. The thing is, it cannot really be compared with something like driving a car or hosting people with Airbnb, because that is something that everybody can do. Translation is actually a pretty complex process, and managing it still requires humans. It requires project managers and so on. So far, we have failed to create platforms so that we can automate this management, but eventually it will come by consolidating technologies and making it easier to process translations. Eventually we will see will be from an Uber of translation, but where will it come from? Current attempts are more promising in the US, simply because they have managed to raise tens of millions of dollars. In the end, it will also be a question of funding.

Let’s talk a little bit about an industrial policy for the language technology sector. The EU is funding a number of R&D projects. That’s one way of bringing money into the system. But it is also potentially a very important client of such services. How could it organise its procurement in a smarter way?

That's right, the EU has been funding research in that area for decades, with some good results and some not so good results. But at the same time, the European Union is the biggest translation organisation in the world. Today, it spends over €1 billion per year financing that activity, so you can imagine how much money it could save by increasing productivity. And instead of funding research, it could also put out a call for tender and ask industry to deliver the kind of systems it would like. That would make industry more focused, and it could also lead to the delivery of better products in a quicker and more sustainable way.

So you would do away with the European officials that work as translators, and replace them with private sector service providers?

That’s not what I mean. Many of them work with very confidential documents, as civil servants, but they can only translate a fraction of what they are supposed to translate. It is a question of making them more productive and allowing them to focus on the interesting and creative part of their work, whilst automating the more repetitive tasks. A lot of gains could be made there. This would be a great way of advancing in the field of procurement, as the Americans are doing.

Finally, let’s turn to the private sector users. You have called for what you refer to as “language infrastructure”. What would it be and what would be its legal status?

It is a key aim of LT-Innovate to create something like a European language cloud, because when you develop language solutions, there are certain basic elements that you have to develop for each individual language. If you want to support ten languages you have to do them ten times. This is cost prohibitive for most companies, even big ones. So what we suggest is to have some sort of maintained language infrastructure that provides the basic language services, and on top of which companies can build their own solutions. As with other infrastructures, it should be run by a public or semi-public organisation, a non-profit organisation, which makes sure that it is accessible for every company and every user for a fair price and an optimal licencing scheme.

And it could be LT-Innovate itself that provides this?

Yes. As an organisation, LT-Innovate could operate such an infrastructure.

Why Machines Alone Cannot Solve the World's Translation Problem

Sixty years ago this week, scientists at Georgetown and IBM lauded their machine translation "brain," known as the 701 computer. The "brain" had successfully translated multiple sentences from Russian into English, leading the researchers to confidently claim that translation would be fully handled by machines in "the next few years."

Fast forward six decades, and MIT Technology Review makes a remarkably similar proclamation: "To translate one language into another, find the linear transformation that maps one to the other. Simple, say a team of Google engineers."

Simple? Not exactly.

Even in the 1950s, IBM acknowledged that to translate just one segment "necessitates two and a half times as many instructions to the computer as are required to simulate the flight of a guided missile." It's also highly doubtful that the scientists at Google see anything "simple" about their new method, which relies on vector space mathematics.

Granted, there is a beautiful simplicity in statistical machine translation, such as Google Translate. Essentially, the more data you have, the better the probability of a high-quality translation as an end result. But what do you do when you don't have enough data? Or in the case of Google, what do you do when the data might be out there somewhere, but it isn't part of the free and public web that you're designed to mine?

That's when you come up with new techniques, just as Google has done. Their new method -- one that is meant to complement, but not replace their statistical approach -- automatically creates dictionaries and phrase tables without help from humans. The new technique uses data mining in order to compare the structure of one language to another, and then generates phrase tables and dictionaries accordingly. This means that Google won't have to rely exclusively on documents available in two languages to improve its translation quality. It will have other methods, such as this new one, to add to the mix.

What does this mean? Even Google isn't satisfied that statistical machine translation will move things along quickly enough. That method has its limitations, just like all methods do.

What's fascinating is that every few months, starry-eyed and often misinformed journalists herald a new era for language translation, announcing a "groundbreaking milestone" related to a technology that has been around for 60 years.

And their claim is always the same: "The translation problem is solved!"

Unfortunately, equating such minor machine translation accomplishments with "solving the translation problem" is like assuming that because we've walked on the moon that we can all just pack up and move there. We can't, and we may never be able to. But that doesn't stop us from trying.

Machine translation, or computer-generated translation as we often call it at Smartling, is a technical marvel. It serves many important purposes, can be used properly in specific, limited cases, and is useful for a variety of tasks that are typically unrelated to the final output on the page. Personally, I'm a believer in making strides toward improving machine translation. For that reason, I profiled the work of Franz Och, the brain behind Google Translate, in Found in Translation.

However, machine translation is not going to replace professional human translators anytime soon. Here are six reasons why:

1. It's Tough to Get Good Translation, Even From Perfectly Bilingual Human Beings. 

One of the reasons that machine translation cannot replace professional human translation is the same reason that plain old bilingual laypeople, for many tasks, cannot replace professional human translation. For most translation jobs, the task of translation requires more than just knowledge of two languages. The idea that you can simply create one-to-one equivalencies across languages is false.Translators are not walking dictionaries. They recreate language. They craft beautiful phrases and sentences to make them have the same impact as the source. Often, they devise brand-new ways of saying things, and to do so, they draw upon a lifetime's worth of knowledge derived from living in two cultures. Machines cannot exactly do that.

2. Translation Quality is Highly Subjective.

Even if machines could approximate human translation quality, it's unclear which version of human quality they would emulate. Give a text to 100 human translators, and you'll get 100 different translations. Which one offers the best "quality?" In many ways, this is like asking someone which rendition of a song is best when sung by 100 different singers. Your choice will be subjective in many ways, even if you can argue that one artist hit a flat note while another had perfect pitch. While this diversity of human language expression makes things complicated, it's also a necessity. Machine translation tools, so far, present far more limited options with their output, which are generally too simplistic for the complex linguistic realities of most translation projects.

3. There Are Too Many Languages Out There.

Google Translate today supports 80 languages. There are between 6,000 and 7,000 languages alive today, of which about 2,000 are considered endangered. If we use a very conservative estimate and say there are only 1,000 languages of significant economic importance in the world today, that still leaves 920 languages yet to be developed. If Google were to add 10 languages per year, it would take 92 years for us to see even a fraction of the world's human languages addressed through machine translation. Most of us won't be around by then, meaning that machine translation -- even at the poorest levels of quality -- will not be a reality for the majority of the world's languages during our lifetime.

4. Most Languages Are Not Written.

The vast majority of the world's languages are spoken or signed. Online, much of our communication is migrating from text to a combination of text plus audio, and even more importantly, video, which encompasses audio as well and helps us leap past text. This means that written language need not be the barrier it once was for people whose languages lack written forms. It also means that translation has its limits. Spoken language suddenly takes on new importance as the internet travels to places far-flung. Smartphones and tablets with visual, tactile and audio inputs make text less important in our world. This doesn't mean that text translation won't be important. It might just mean it will increasingly take place behind the scenes, with audio or video output instead.

5. Context Ss Key.

In a language like English, a single word can have hundreds of different meanings, depending on the context (see "Clear Examples of Why Context Matters"). In fact, the Oxford English Dictionary's lexicographer for the letter "R," Peter Gilliver, claimed that the verb-form alone of "run" has no less than 645 distinct meanings. Can a machine learn each of these meanings for every word in not just one language, but two? This isn't an easy question to answer. In fact, the OED explains that even the very nature of what constitutes a word is up for debate:

"It's impossible to count the number of words in a language, because it's so hard to decide what actually counts as a word. Is dog one word, or two (a noun meaning "a kind of animal," and a verb meaning "to follow persistently")? If we count it as two, then do we count inflections separately too (e.g. dogs = plural noun, dogs = present tense of the verb). Is dog-tired a word, or just two other words joined together? Is hot dog really two words, since it might also be written as hot-dog or even hotdog?"

Not only that, but word-for-word translation is impossible, so instead of thinking about words, when humans use context to figure out meaning, we think not just of single words, but how those words interact with the ones around them. Those combinations are constantly changing and multiplying, limited only by human creativity. Machines can hardly keep up.

6. Language Is Simply Too Important.

How important are the words your company uses to describe its products or services? They are critical. For many companies -- including, perhaps ironically, Google -- the voice of the brand all centers around word choice. How human beings make choices about the products they buy and the services they use relates directly to the words that are used to market and sell them. Perhaps when machines are the ones doing the buying, they'll be less picky about language. For now, humans are still the ones opening their wallets, and humans are a strange bunch, with very real and emotional reactions to language. Our taste or distaste for a particular term often relates to our upbringing, our culture and even our past experiences. Humans cannot accurately predict which words will annoy or confuse even the people we know best. How can we expect a machine to fare any better?

So, if machine translation can't fix all our language woes anytime soon, why does the world keep celebrating each and every related milestone as if it were a major achievement? Well, because it really would be nice if cross-language issues could be simplified. In fact, humans would love it if communication matters in general -- even monolingual ones -- were less complex.

The bottom line is this: Computers will never fully solve the translation problem, and even to make micro-strides toward that audacious goal, they will need significant help from humans. The question isn't, "Will we get there?" but rather, "How far will we get, and how fast?" In the meantime, the utopia of computer-generated translation is a dream worth having, albeit a recurring one.

________________________________

http://www.huffingtonpost.com/nataly-kelly/why-machines-alone-cannot-translation_b_4570018.html?ir=Technology

Machine translation Conquering Babel

Simultaneous translation by computer is getting closer

In “STAR TREK”, a television series of the 1960s, no matter how far across the universe the Starship Enterprise travelled, any aliens it encountered would converse in fluent Californian English. It was explained that Captain Kirk and his crew wore tiny, computerised Universal Translators that could scan alien brainwaves and simultaneously convert their concepts into appropriate English words.

Science fiction, of course. But the best sci-fi has a habit of presaging fact. Many believe the flip-open communicators also seen in that first “Star Trek” series inspired the design of clamshell mobile phones. And, on a more sinister note, several armies and military-equipment firms are working on high-energy laser weapons that bear a striking resemblance to phasers. How long, then, before automatic simultaneous translation becomes the norm, and all those tedious language lessons at school are declared redundant?

Not, perhaps, as long as language teachers, interpreters and others who make their living from mutual incomprehension might like. A series of announcements over the past few months from sources as varied as mighty Microsoft and string-and-sealing-wax private inventors suggest that workable, if not yet perfect, simultaneous-translation devices are now close at hand.

Over the summer, Will Powell, an inventor in London, demonstrated a system that translates both sides of a conversation between English and Spanish speakers—if they are patient, and speak slowly. Each interlocutor wears a hands-free headset linked to a mobile phone, and sports special goggles that display the translated text like subtitles in a foreign film.

In November, NTT DoCoMo, the largest mobile-phone operator in Japan, introduced a service that translates phone calls between Japanese and English, Chinese or Korean. Each party speaks consecutively, with the firm’s computers eavesdropping and translating his words in a matter of seconds. The result is then spoken in a man’s or woman’s voice, as appropriate.

Microsoft’s contribution is perhaps the most beguiling. When Rick Rashid, the firm’s chief research officer, spoke in English at a conference in Tianjin in October, his peroration was translated live into Mandarin, appearing first as subtitles on overhead video screens, and then as a computer-generated voice. Remarkably, the Chinese version of Mr Rashid’s speech shared the characteristic tones and inflections of his own voice.

Que?

Though the three systems are quite different, each faces the same problems. The first challenge is to recognise and digitise speech. In the past, speech-recognition software has parsed what is being said into its constituent sounds, known as phonemes. There are around 25 of these in Mandarin, 40 in English and over 100 in some African languages. Statistical speech models and a probabilistic technique called Gaussian mixture modelling are then used to identify each phoneme, before reconstructing the original word. This is the technology most commonly found in the irritating voice-mail jails of companies’ telephone-answering systems. It works acceptably with a restricted vocabulary, but try anything more free-range and it mistakes at least one word in four.

The translator Mr Rashid demonstrated employs several improvements. For a start, it aims to identify not single phonemes but sequential triplets of them, known as senones. English has more than 9,000 of these. If they can be recognised, though, working out which words they are part of is far easier than would be the case starting with phonemes alone.

Microsoft’s senone identifier relies on deep neural networks, a mathematical technique inspired by the human brain. Such artificial networks are pieces of software composed of virtual neurons. Each neuron weighs the strengths of incoming signals from its neighbours and send outputs based on those to other neighbours, which then do the same thing. Such a network can be trained to match an input to an output by varying the strengths of the links between its component neurons.

One thing known for sure about real brains is that their neurons are arranged in layers. A deep neural network copies this arrangement. Microsoft’s has nine layers. The bottom one learns features of the processed sound waves of speech. The next layer learns combinations of those features, and so on up the stack, with more sophisticated correlations gradually emerging. The top layer makes a guess about which senone it thinks the system has heard. By using recorded libraries of speech with each senone tagged, the correct result can be fed back into the network, in order to improve its performance.

Microsoft’s researchers claim that their deep-neural-network translator makes at least a third fewer errors than traditional systems and in some cases mistakes as few as one word in eight. Google has also started using deep neural networks for speech recognition (although not yet translation) on its Android smartphones, and claims they have reduced errors by over 20%. Nuance, another provider of speech-recognition services, reports similar improvements. Deep neural networks can be computationally demanding, so most speech-recognition and translation software (including that from Microsoft, Google and Nuance) runs in the cloud, on powerful online servers accessible in turn by smartphones or home computers.

Quoi?

Recognising speech is, however, only the first part of translation. Just as important is converting what has been learned not only into foreign words (hard enough, given the ambiguities of meaning which all languages display, and the fact that some concepts are simply untranslatable), but into foreign sentences. These often have different grammatical rules, and thus different conventional word orders. So even when the English words in a sentence are known for certain, computerised language services may produce stilted or humorously inaccurate translations.

Google’s solution for its Translate smartphone app and web service is crowd-sourcing. It compares the text to be translated with millions of sentences that have passed through its software, and selects the most appropriate. Jibbigo, whose translator app for travellers was spun out from research at Carnegie Mellon University, works in a similar way but also pays users in developing countries to correct their mother-tongue translations. Even so, the ultimate elusiveness of language can cause machine-translation specialists to feel a touch of Weltschmerz.

For example, although the NTT DoCoMo phone-call translator is fast and easy to use, it struggles—even though it, too, uses a neural network—with anything more demanding than pleasantries. Sentences must be kept short to maintain accuracy, and even so words often get jumbled.

Microsoft is betting that listeners will be more forgiving of such errors when dialogue is delivered in the speaker’s own voice. Its new system can encode the distinctive timbre of this by analysing about an hour’s worth of recordings. It then generates synthesised speech with a similar spread of frequencies. The system worked well in China, where Mr Rashid’s computerised (and occasionally erroneous) Mandarin was met with enthusiastic applause.

A universal translator that works only in conference halls, however, would be of limited use to travellers, whether intergalactic or merely intercontinental. Mr Powell’s conversation translator will work anywhere that there is a mobile-phone signal. Speech picked up by the headsets is fed into speech-recognition software on a nearby laptop, and the resulting text is sent over the mobile-phone network to Microsoft’s translation engine online.

One big difficulty when translating conversations is determining who is speaking at any moment. Mr Powell’s system does this not by attempting to recognise voices directly, but rather by running all the speech it hears through two translation engines simultaneously: English to Spanish, and Spanish to English. Since only one of the outputs is likely to make any sense, the system can thus decide who is speaking. That done, it displays the translation in the other person’s goggles.

At the moment, the need for the headsets, cloud services and intervening laptop means Mr Powell’s simultaneous system is still very much a prototype. Consecutive, single-speaker translation is more advanced. The most sophisticated technology currently belongs to Jibbigo, which has managed to squeeze speech recognition and a 40,000-word vocabulary for ten languages into an app that runs on today’s smartphones without needing an internet connection at all.

Nani?

Some problems remain. In the real world, people talk over one another, use slang or chat on noisy streets, all of which can foil even the best translation system. But though it may be a few more years before “Star Trek” style conversations become commonplace, universal translators still look set to beat phasers, transporter beams and warp drives in moving from science fiction into reality.

http://www.economist.com

Linguistics Researcher Develops New System to Help Computers ‘Learn’ Natural Language

Linguists, computer scientists use supercomputers to improve natural language processing

Newswise — AUSTIN, Texas - For more than 50 years, linguists and computer scientists have tried to get computers to understand human language by programming semantics as software. Now, a University of Texas at Austin linguistics researcher, Katrin Erk, is using supercomputers to develop a new method for helping computers learn natural language.

Instead of hard-coding human logic or deciphering dictionaries to try to teach computers language, Erk decided to try a different tactic: feed computers a vast body of texts (which are a reflection of human knowledge) and use the implicit connections between the words to create a map of relationships.

“An intuition for me was that you could visualize the different meanings of a word as points in space,” says Erk, a professor of linguistics who is conducting her research at the Texas Advanced Computing Center. “You could think of them as sometimes far apart, like a battery charge and criminal charges, and sometimes close together, like criminal charges and accusations (“the newspaper published charges…”). The meaning of a word in a particular context is a point in this space. Then we don’t have to say how many senses a word has. Instead we say: ‘This use of the word is close to this usage in another sentence, but far away from the third use.’ ”

To create a model that can accurately recreate the intuitive ability to distinguish word meaning requires a lot of text and a lot of analytical horsepower.

“The lower end for this kind of a research is a text collection of 100 million words,” she explains. “If you can give me a few billion words, I’d be much happier. But how can we process all of that information? That’s where supercomputers come in.”

http://www.newswise.com

Machine translation: Babel or Babble?

EVER since God confounded the people of Babel, we have been left with imperfect solutions to communicating across borders. One of those has been the lingua franca, a commonly known second language in which different nationalities converse. That trick sufficed for millennia, but it could be reaching the end of its lifespan, according to Nicholas Ostler, author of "The Last Lingua Franca" (which The Economist reviewed in 2010). Machine translation software may become so advanced as to render second-language learning useless.

Mr Ostler renewed his claims that machine translation does away with the need for a lingua franca at the Hay Literary Festival last week. Many linguists disagree with him, including David Crystal, who has forecast that English may “find itself in the service of the world community forever.” But we once thought the same about Greek, Persian, Latin and French, all of which became obsolete.

“Evidently, automatic systems replacing a real lingua franca is likely to be a bit different” than one lingua franca replacing another, Mr Ostler told your correspondent. “The process will run faster for mainstream languages than for the peripheral smaller fry. But I think the development will be unmistakable within one generation—say by 2050.” 

Yet the quality of machine translation is still uneven at best. Google Translate is probably the best of the bunch. It has 200m users a month by the last count, and translates in a single day the equivalent of the whole annual output of the globe's professional translator corps. While there's no doubt that it is a powerful and useful tool, it can produce embarrassing errors. There are fewer mistakes than there were in 2001 when Google launched it, but there still seems a long way to go until sci-fi technology becomes real.

It's feasible that machine translation could replace human translators for written texts. Text is easier to translate than conversation, and is better suited to the technology, which is "trained" by huge corpora of human-translated texts. But these tools are only as good as the corpora themselves. Google Translate draws on a variety of texts, including documents from the European Union and United Nations. These are chock-full of legalese, and rarely represent the everyday language of man. Finding more representative texts without raising the hackles of copyright enforcers could prove a stumbling block. 

There is also a remaining reliance on English as an intermediary language for translation. Precious few Galician novels will have been translated into Welsh nor Welsh novels into Galician, but both make their way into English. Though humans may not use English as a lingua franca in the future, necessity dictates that the machine translators still will (as Asya Pereltsvaig seems to have busted Google Translate doing). What machine translation will allow, says Mr Ostler, is those “minority language speakers to intervene—at last—directly in wider issues to a wider audience” as their niche languages become easily translatable by gadgets. That does, of course, discourage people from learning the language in the first place.

Spoken language is too quick and fragmented for machine translation today. Simple dictation software must be carefully trained, never mind the extra step of translation.  Ordinary conversation is full of false-starts and errors that speakers and listeners barely notice, but which computers are baffled by. And of course tone of voice, cultural references, idiom and humour multiply the challenges. The true-to-life Babel fish is some ways away.

Idealist and pessimist tribes gather around the future of machine translation. Your correspondent is a pessimist, but would dearly like to be proved wrong, because Mr Ostler's vision of the future is an exciting one. “Increasingly, electronic tools will be there for people to get what they need from documents and recordings in foreign languages,” he explains. “And real-time gizmos will support people's needs in face-to-face interaction too.” Verb conjugation tables and vocabulary lists may be consigned to history. Add schoolchildren to the many others who would rejoice in a lingua-franca-free future.

http://www.economist.com