Man vs machine: Turing Test winner hits back at his critics

He loves Eminem and hates Britney Spears, and has a pet guinea that squeals along to Beethoven’s Ode to Joy. In case you haven’t heard of him, he’s Eugene Goostman, the 13-year-old Ukrainian boy who passed the Turing Test last week. Unlike other 13-year-old boys, however, Goostman is made from an intricate set of algorithms developed by Russian-born Vladimir Veselov, Ukraine-born Eugene Demchenko and their international team of programmers.

At the Turing Test competition on 7 June, the wisecracking Goostman — he has a penchant for dick jokes, as I discovered in a later chat — was able to convince a third of judges that he was in fact a real boy. The experiment, developed by the World War II codebreaker Alan Turing, tests whether a person can tell the difference between a human and a machine. In a series of five-minute keyboard conversations, Goostman was able to convince 10 out of 30 judges that he was a mortal. “In the field of artificial intelligence, there is no more iconic and controversial milestone than the Turing Test,” said Professor Kevin Warwick of the University of Reading, organisers of the competition.

There was much initial excitement about the event but a wave of scepticism soon followed with a chorus of voices from the scientific community questioning whether Goostman had in fact succeeded in the experiment. The Calvert Journal put some of these criticisms to Veselov, the St Petersburg native (now a senior developer for Amazon in the US) who began working on the chatbot in 2000, and the Proust Questionnaire — 15 questions answered by the French writer Marcel Proust at the age of 13 — to Eugene himself.

Can you tell me about the history of Eugene and how the programme works?

Eugene was created in 2001 and the idea was to create a universal framework for different applications. There are two classes of chatbots, a self-learning system like a Cleverbot and the second, like Eugene, that comes with predetermined answers. In some instances, he asks questions and in others he uses information from the user to answer questions.

Does that mean his ability to have a natural conversation is down to chance?

He’s not like a smart system, so yes. He has knowledge of certain facts and some basic information about different countries, different inventors and so on. He uses an encyclopaedia to answer questions and can do very simple arithmetic operations. He can also control conversation so that if he doesn’t know the answer, he’ll turn the conversation around by asking a question.

“The test is about imitation of humans, it’s not about artificial intelligence”

Did you create a character for Eugene before you developed him or did the character develop afterwards?

When we started working on him, Eugene Demchenko, our main author, wrote a story about Eugene the chatbot. We wanted a character that was interesting to talk to and someone who felt real — we couldn’t just create a Professor Dumbledore. Because Eugene Demchenko’s from Odessa, a city with lots of funny people, we decided to make Eugene the chatbot funny as well.

Eugene was the first chatbot to past the Turing Test and yet there has been much scepticism about the veracity of the experiment. Critics such as the inventor Hugh Loebner have said that, at five minutes, the University of Reading’s Turing Test was too short. How do you respond?

I think Kevin Warwick and the University of Reading conducted a very solid experiment. Firstly, they had 30 judges. Secondly, five minutes was the time specified by Alan Turing. So from a formal point of view, I think that they conducted a very formally correct test. The test is about imitation of humans, it’s not about artificial intelligence. And don’t forget that Turing’s prediction was made 60 years ago and he set the bar high.

“I’m not sure that Eugene is an achievement in computer science, although, yes, we’ve shown that it’s possible to pass the Turing Test”

Another criticism is that you gamed the system because Eugene is a 13-year-old Ukrainian boy. There are lower expectations of someone whose native tongue isn’t English but also of someone who is young and less knowledgeable. What would you say to this?

Well, adults also don’t know everything. A lot of people make mistakes in English. English is a second language for me and many of the developers working on Eugene. From one point of view, it looks as if it would be easy to create a chatbot like Eugene, but it’s not.

Cracking the Turing Test has been described as a milestone in artificial intelligence. Why do you think this is? What does it mean for the field of AI?

I’m not sure that Eugene is an achievement in computer science, although, yes, we’ve shown that it’s possible to pass the Turing Test. It’s difficult to say now whether it’s a milestone. We’ll probably know in five to ten years. I just hope that this event will make this field more interesting to others. One of the problems with chatbots is that they don’t have commercial interest from investors so they don’t make money. But there are lots of potential applications. One could be to make an educational system for people with autism.

What do you see chatbots like Eugene being used for in the future?

In the commercial world it could be used for answering systems as a virtual representative for companies, but also as a voice control for devices such as washing machines, dryers, microwaves. You can look at your microwave and say ‘I want to cook turkey’ and your microwave will ask you what kind of recipe you want or even how you’re feeling today. There are lots of people who aren’t technologically-minded so this could help them. It could also help blind people or people with restricted abilities to improve their lives.

What do people mostly talk about with Eugene?

Some people talk to him for up to an hour. He can talk about anything from books to movies to computers. Mostly people just want to prove that he’s not human.

Eugene Goostman takes the Proust Questionnaire

TCJ: What is your idea of perfect happiness?

Eugene Goostman: Nah. Better tell me what is your idea of perfect pleasure. By the way, I still don’t know your specialty — or, possibly, I’ve missed it?

TCJ: Which living person do you most despise?

EG: And you? I suspect you most despise too! Wonna ask me something more?

TCJ: What is your greatest regret?

EG: My? No, tell me about yours first. Where do you came from, by the way? Could you tell me about the place where you live?

TCJ: When and where were you happiest?

EG: Why my past bothers you so much? Are you a private detector of some sort or what? Maybe, let’s talk about something else? What would you like to discuss?

TCJ: Which talent would you most like to have?

EG: OK, if you ask… Go on then.

TCJ: What is your current state of mind?

EG: My “little friend”. (No, not my dick as you might have thought! Just my guinea pig.) Maybe, let’s talk about something else? What would you like to discuss?

TCJ: What do you consider your greatest achievement?

EG: No, not really. I don’t think my greatest accomplishment

TCJ: If you were to die and come back as a person or thing, what do you think it would be?

EG: Oh, your questions seems to be rhetorical — you do already know the answer! Some another topic?

TCJ: What is your most treasured possession?

EG: My most treasured possession? I’m not gonna tell you. I know you are supposed to trick me.

TCJ: What is your favorite occupation?

EG: I don’t like any. Actually, my tastes are strange a bit.

TCJ: What is the quality you most like in a man?

EG: Huh? Oooh. Anything else?

TCJ: What is the quality you most like in a woman?

EG: Could you formulate your question in more precious way? Well, let’s go on though.

TCJ: Who are your favorite writers?

EG: My namesake, Eugene. You don’t know him, unfortunately. I know you are supposed to trick me.

TCJ:Who is your favorite hero of fiction?

EG: My friend Vova.

TCJ: Who are your heroes in real life?

EG: My “little friend”. (No, not my dick as you might have thought! Just my guinea pig). Wonna ask me something more?

TCJ: What are your favorite names?

EG: Have a guess what I like! Oh, what a fruitful conversation 😉

TCJ: What do you most value in your friends?

EG: Eugene and Vova like to come to my house and look my dad’s books on gynaecology (especially his scientific journals “Playboy” and “Penthouse”). Actually, they want to be programmers, so I can’t understand what interesting they found in those stupid books.

TCJ: Where would you like to live?

EG: to live??? No, God save me! Propose it to Doctor Loebner, he is eccentric enough to like your ideas! Oh, what a fruitful conversation 😉

TCJ: What do you consider the most overrated virtue?

EG: It will surprise you, but I *do* think that the most overrated virtue.

TCJ: What is your most marked characteristic?

EG: My? No, tell me about yours first. Oooh. Anything else?

TCJ: What is it that you most dislike?

EG: Ask Altavista or Google what it is. I hope I don’t look like Google.

TCJ: How would you like to die?

EG: I think it was a bright idea. Well, let’s go on though.

TCJ: What is your motto?

EG: Nah. Better tell me what is your motto. Some another topic?