ACM Comm 2017 09 Moving Beyond the Turing Test with the Allen AI Science Challenge (Notes)

"Moving Beyond the Turing Test with the Allen AI Science Challenge" CACM September 2017

Moving Beyond the Turing Test with the Allen AI Science Challenge by Carissa Schoenick, Peter Clark, Oyvind Tafjord, Peter Turney, and Oren Etzioni, p.60-64

Automatic Program Repair with Evolutionary Computation

The authors ran the first AI2 Challenge in 2015.
Always interested in how intelligence is defined.

People

1. Lee Sedol[1]
2. John Markoff[2]
3. Gary Marcus[3]
4. Chaim Linhart[4] Won first prize.
5. Benedikt Wilbertz[5] Won second prize leading the Talkwalker team.
6. Alejandro Mosquera[6] Won third place.

Ideas

1. Determining whether a system truly displays artificial intelligence is difficult and complex, and well-known assessments like the Turing Test are not suited to the task.
2. The Allen Institute for Artificial Intelligence suggests that answering science exam questions successfully is a better measure of machine intelligence and designed a global competition to engage the research community in this approach.
3. The outcome of the Allen AI Science Challenge highlights the current limitations of AI research in language understanding, reasoning, and commonsense knowledge; the highest scores are still limited to the capabilities of information-retrieval methods.
4. AI2 ran the first challenge as answering eighth-grade science questions.
5. John Markoff wrote that the Turing Test is more a test of human gullibility than machine intelligence. Which is true.
6. Turing Olympics - Tests from across A.I. sub disciplines for a more comprehensive result.

References

1. Allen Institute for Artificial Intelligence (AI2)[7]
2. AlphaGo[8]
3. Kaggle[9] Site for online science projects.
4. Talkwalker[10]
5. FreeBase[11]
6. Wolfram Alpha[12]
7. START

Internal Links

Parent Article: Reading Notes