View source for Turing test

{{Short description|Test of a machine's ability to imitate human intelligence}}
{{Hatnote group|
{{Redirect|Imitation game|the film|The Imitation Game}}
{{Other uses}}
}}
{{Use British English|date=February 2024}}
{{Use dmy dates|date=January 2025}}
[[File:Turing test diagram.png|thumb|The "standard interpretation" of the Turing test, in which player C, the interrogator, is given the task of trying to determine which player&nbsp;– A or B&nbsp;– is a computer and which is a human. The interrogator is limited to using the responses to written questions to make the determination.<ref>Image adapted from {{harvnb|Saygin|2000}}</ref>]]
{{Artificial intelligence|Philosophy}}

The '''Turing test''', originally called the '''imitation game''' by [[Alan Turing]] in 1949,<ref name="Turing-1950">{{Harv|Turing|1950}}. Turing wrote about the ‘imitation game’ centrally and extensively throughout his 1950 text, but apparently retired the term thereafter. He referred to ‘[his] test’ four times—three times in pp. 446–447 and once on p. 454. He also referred to it as an ‘experiment’—once on p. 436, twice on p. 455, and twice again on p. 457—and used the term ‘viva voce’ (p. 446). See also [[#Versions]], below. Turing gives a more precise version of the question later in the paper: "[T]hese questions [are] equivalent to this, 'Let us fix our attention on one particular digital computer C. Is it true that by modifying this computer to have an adequate storage, suitably increasing its speed of action, and providing it with an appropriate programme, C can be made to play satisfactorily the part of A in the imitation game, the part of B being taken by a man?{{' "}} {{Harv|Turing|1950|p=442}}</ref> is a test of a machine's ability to [[artificial intelligence|exhibit intelligent behaviour]] equivalent to that of a human. In the test, a human evaluator judges a text transcript of a [[Natural language|natural-language]] conversation between a human and a machine. The evaluator tries to identify the machine, and the machine passes if the evaluator cannot reliably tell them apart. The results would not depend on the machine's ability to [[question answering|answer questions correctly]], only on how closely its answers resembled those of a human. Since the Turing test is a test of indistinguishability in performance capacity, the verbal version generalizes naturally to all of human performance capacity, verbal as well as nonverbal (robotic).<ref name="The Turing Test">Oppy, Graham & Dowe, David (2011) [http://www.illc.uva.nl/~seop/entries/turing-test/ The Turing Test] {{Webarchive|url=https://web.archive.org/web/20120320101053/http://www.illc.uva.nl/~seop/entries/turing-test/ |date=20 March 2012 }}. ''Stanford Encyclopedia of Philosophy''.</ref>

The test was introduced by Turing in his 1950 paper "[[Computing Machinery and Intelligence]]" while working at the [[University of Manchester]].<ref>{{cite web |url=http://www.turing.org.uk/scrapbook/test.html |title=The Turing Test, 1950 |series=The Alan Turing Internet Scrapbook |website=turing.org.uk |access-date=23 April 2015 |archive-date=3 April 2019 |archive-url=https://web.archive.org/web/20190403170058/https://www.turing.org.uk/scrapbook/test.html |url-status=live}}</ref> It opens with the words: "I propose to consider the question, 'Can machines think?{{' "}} Because "thinking" is difficult to define, Turing chooses to "replace the question by another, which is closely related to it and is expressed in relatively unambiguous words".{{sfn|Turing|1950|p=433}} Turing describes the new form of the problem in terms of a three-person [[party game]] called the "imitation game", in which an interrogator asks questions of a man and a woman in another room in order to determine the correct sex of the two players. Turing's new question is: "Are there imaginable digital computers which would do well in the ''imitation game''?"<ref name="Turing-1950"/> This question, Turing believed, was one that could actually be answered. In the remainder of the paper, he argued against the major objections to the proposition that "machines can think".<ref name="Turing's nine objections">{{Harvnb|Turing|1950|pp=442–454}} and see {{Harvtxt|Russell|Norvig|2003|p=948}}, where they comment, "Turing examined a wide variety of possible objections to the idea of intelligent machines, including virtually all of those that have been raised in the half century since his paper appeared."</ref>

Since Turing introduced his test, it has been highly influential in the [[philosophy of artificial intelligence]], resulting in substantial discussion and controversy, as well as criticism from philosophers like [[John Searle]], who argue against the test's ability to detect [[consciousness]].{{sfn|Saygin|2000}}{{sfn|Russell|Norvig|2003|pp=2–3, 948}} <!--The various imitation tests that Turing presented between 1948 and 1952 were his response to a public controversy over the future capabilities of digital computers, as documented by Turing scholar Bernardo Gonçalves.{{Sfn|Gonçalves|2024a}}{{sfn|Gonçalves|2023c}}-->

Since the mid [[2020s]], several [[large language model]]s such as [[ChatGPT]] have passed modern, rigorous variants of the Turing test.<ref name="humsci.stanford.edu">{{cite web |last1=Scott |first1=Cameron |title=Study finds ChatGPT's latest bot behaves like humans, only better {{!}} Stanford School of Humanities and Sciences |url=https://humsci.stanford.edu/feature/study-finds-chatgpts-latest-bot-behaves-humans-only-better |url-status=live |archive-url=https://web.archive.org/web/20240326182336/https://humsci.stanford.edu/feature/study-finds-chatgpts-latest-bot-behaves-humans-only-better |archive-date=26 March 2024 |access-date=26 March 2024 |website=humsci.stanford.edu |language=en}}</ref><ref name="A Turing test of whether AI chatbot">{{cite journal |last1=Mei |first1=Qiaozhu |last2=Xie |first2=Yutong |last3=Yuan |first3=Walter |last4=Jackson |first4=Matthew O. |date=27 February 2024 |title=A Turing test of whether AI chatbots are behaviorally similar to humans |journal=Proceedings of the National Academy of Sciences |language=en |volume=121 |issue=9 |pages=e2313925121 |bibcode=2024PNAS..12113925M |doi=10.1073/pnas.2313925121 |issn=0027-8424 |pmc=10907317 |pmid=38386710}}</ref><ref name="Jones2025">{{Citation |last1=Jones |first1=Cameron R. |title=Large Language Models Pass the Turing Test |date=2025-03-31 |arxiv=2503.23674 |last2=Bergen |first2=Benjamin K.}}</ref>

== History ==
=== Philosophical background ===
The question of whether it is possible for machines to think has a long history, which is firmly entrenched in the distinction between [[dualism (philosophy of mind)|dualist]] and [[materialism|materialist]] views of the mind. [[René Descartes]] prefigures aspects of the Turing test in his 1637 ''[[Discourse on the Method]]'' when he writes: {{blockquote|[H]ow many different automata or moving machines could be made by the industry of man&nbsp;... For we can easily understand a machine's being constituted so that it can utter words, and even emit some responses to action on it of a corporeal kind, which brings about a change in its organs; for instance, if touched in a particular part it may ask what we wish to say to it; if in another part it may exclaim that it is being hurt, and so on. But it never happens that it arranges its speech in various ways, in order to reply appropriately to everything that may be said in its presence, as even the lowest type of man can do.{{sfn|Descartes|1996|pp=[https://archive.org/details/discourseonmetho0000desc_i4z8/page/34 34]–35}}}}
Here Descartes notes that [[automata]] are capable of responding to human interactions but argues that such automata cannot respond appropriately to things said in their presence in the way that any human can. Descartes therefore prefigures the Turing test by defining the insufficiency of appropriate linguistic response as that which separates the human from the automaton. Descartes fails to consider the possibility that future automata might be able to overcome such insufficiency, and so does not propose the Turing test as such, even if he prefigures its conceptual framework and criterion.

[[Denis Diderot]] formulates in his 1746 book ''[[Philosophical Thoughts|Pensées philosophiques]]'' a Turing-test criterion, though with the important implicit limiting assumption maintained, of the participants being natural living beings, rather than considering created artifacts:

{{blockquote|If they find a parrot who could answer to everything, I would claim it to be an intelligent being without hesitation.}}

This does not mean he agrees with this, but that it was already a common argument of [[materialists]] at that time.

According to dualism, the [[mind]] is [[non-physical entity|non-physical]] (or, at the very least, has [[property dualism|non-physical properties]])<ref>For an example of property dualism, see [[Qualia]].</ref> and, therefore, cannot be explained in purely physical terms. According to materialism, the mind can be explained physically, which leaves open the possibility of minds that are produced artificially.<ref>Noting that materialism does not ''necessitate'' the possibility of artificial minds (for example, [[Roger Penrose#Consciousness|Roger Penrose]]), any more than dualism necessarily ''precludes'' the possibility. (See, for example, [[Property dualism]].)</ref>

In 1936, philosopher [[Alfred Ayer]] considered the standard philosophical question of [[other minds problem|other minds]]: how do we know that other people have the same conscious experiences that we do? In his book, ''[[Language, Truth and Logic]]'', Ayer suggested a protocol to distinguish between a conscious man and an unconscious machine: "The only ground I can have for asserting that an object which appears to be conscious is not really a conscious being, but only a dummy or a machine, is that it fails to satisfy one of the empirical tests by which the presence or absence of consciousness is determined".<ref>{{Citation |last=Ayer |first=A.&nbsp;J. |author-link=A. J. Ayer |title=Language, Truth and Logic |page=140 |publisher=[[Penguin Books|Penguin]] |year=2001 |isbn=978-0-334-04122-1 |bibcode=1936Natur.138..823G |volume=138 |journal=Nature |doi=10.1038/138823a0 |issue=3498 |s2cid=4121089 |doi-access=free}}{{clarify|date=January 2021|reason+This cite appears to be a "hybrid". Most attributes appear to be for a journal article published in 1936. Page and ISBN appear to be a booi. I do not know where the publisher and year came from.}}</ref> (This suggestion is very similar to the Turing test, but it is not certain that Ayer's popular philosophical classic was familiar to Turing.) In other words, a thing is not conscious if it fails the consciousness test.

=== Cultural background ===
A rudimentary idea of the Turing test appears in the 1726 novel ''[[Gulliver's Travels]]'' by [[Jonathan Swift]].<ref>Rapaport, W.J. (2003). [https://cse.buffalo.edu/~rapaport/Papers/TURING How to Pass a Turing Test] {{Webarchive|url=https://web.archive.org/web/20240613183326/https://cse.buffalo.edu/~rapaport/Papers/TURING |date=13 June 2024 }}. In: Moor, J.H. (eds) The Turing Test. Studies in Cognitive Systems, vol 30. Springer, Dordrecht. <nowiki>https://doi.org/10.1007/978-94-010-0105-2_9</nowiki></ref><ref>{{Cite web |last=Amini |first=Majid |date=1 May 2020 |title=Cognition as Computation: From Swift to Turing. {{!}} Humanities Bulletin {{!}} EBSCOhost |url=https://openurl.ebsco.com/contentitem/gcd:144758985?sid=ebsco:plink:crawler&id=ebsco:gcd:144758985 |access-date=13 June 2024 |website=openurl.ebsco.com |language=en |archive-date=13 June 2024 |archive-url=https://web.archive.org/web/20240613183323/https://openurl.ebsco.com/contentitem/gcd:144758985?sid=ebsco:plink:crawler&id=ebsco:gcd:144758985 |url-status=live}}</ref> When Gulliver is brought before the king of [[Brobdingnag]], the king thinks at first that Gulliver might be a "a piece of clock-work (which is in that country arrived to a very great perfection) contrived by some ingenious artist". Even when he hears Gulliver speaking, the king still doubts whether Gulliver was taught "a set of words" to make him "sell at a better price". Gulliver tells that only after "he put several other questions to me, and still received rational answers" the king became satisfied that Gulliver was not a machine.<ref>{{Cite web |last=Swift |first=Jonathan |date=1726 |title=A Voyage to Brobdingnag. Chapter 3 |url=https://en.wikisource.org/wiki/The_Works_of_the_Rev._Jonathan_Swift/Volume_6/A_Voyage_to_Brobdingnag/Chapter_3 |access-date=13 June 2024 |website=en.wikisource.org |language=en}}</ref>

Tests where a human judges whether a computer or an alien is intelligent were an established convention in science fiction by the 1940s, and it is likely that Turing would have been aware of these.<ref name=":3">{{Cite journal |last=Svilpis |first=Janis |date=2008 |title=The Science-Fiction Prehistory of the Turing Test |journal=Science Fiction Studies |volume=35 |issue=3 |pages=430–449 |doi=10.1525/sfs.35.3.0430 |jstor=25475177 |issn=0091-7729}}</ref> [[Stanley G. Weinbaum]]'s "[[A Martian Odyssey]]" (1934) provides an example of how nuanced such tests could be.<ref name=":3" />

Earlier examples of machines or automatons attempting to pass as human include the [[Ancient Greek]] myth of [[Pygmalion (mythology)|Pygmalion]] who creates a sculpture of a woman that is animated by [[Aphrodite]], [[Carlo Collodi]]'s novel ''[[The Adventures of Pinocchio]]'', about a puppet who wants to become a real boy, and [[E. T. A. Hoffmann]]'s 1816 story "[[The Sandman (short story)|The Sandman]]," where the protagonist falls in love with an automaton. In all these examples, people are fooled by artificial beings that - up to a point - pass as human.<ref>{{Cite book |last=Wansbrough |first=Aleks |title=Capitalism and the enchanted screen: myths and allegories in the digital age |date=2021 |isbn=978-1-5013-5639-1 |location=New York |publisher=Bloomsbury Academic |page=114 |oclc=1202731640}}</ref>

=== Alan Turing and the Imitation Game ===

Researchers in the United Kingdom had been exploring "machine intelligence" for up to ten years prior to the founding of the field of artificial intelligence ([[Artificial intelligence|AI]]) research in 1956.<ref>The [[Dartmouth conference]]s of 1956 are widely considered the "birth of AI". {{Harv|Crevier|1993|p=49}}</ref> It was a common topic among the members of the [[Ratio Club]], an informal group of British [[cybernetics]] and [[electronics]] researchers that included Alan Turing.{{sfn|McCorduck|2004|p=95}}

Turing, in particular, had been running the notion of machine intelligence since at least 1941{{sfn|Copeland|2003|p=1}} and one of the earliest-known mentions of "computer intelligence" was made by him in 1947.{{sfn|Copeland|2003|p=2}} In Turing's report, "Intelligent Machinery,"<ref>[[#{{harvid|Turing|1948}}|"Intelligent Machinery" (1948)]] was not published by Turing, and did not see publication until 1968 in:

* {{Citation |last1=Evans |first1=A. D. J. |title=Cybernetics: Key Papers |year=1968 |publisher=University Park Press |last2=Robertson}}

</ref> he investigated "the question of whether or not it is possible for machinery to show intelligent behaviour"{{sfn|Turing|1948|p=412}} and, as part of that investigation, proposed what may be considered the forerunner to his later tests:
<blockquote>It is not difficult to devise a paper machine which will play a not very bad game of chess.<ref>In 1948, working with his former undergraduate colleague, [[DG Champernowne]], Turing began writing a chess program for a computer that did not yet exist and, in 1952, lacking a computer powerful enough to execute the program, played a game in which he simulated it, taking about half an hour over each move. The game was recorded, and the program lost to Turing's colleague [[Alick Glennie]], although it is said that it won a game against Champernowne's wife.</ref> Now get three men A, B and C as subjects for the experiment. A and C are to be rather poor chess players, B is the operator who works the paper machine. ... Two rooms are used with some arrangement for communicating moves, and a game is played between C and either A or the paper machine. C may find it quite difficult to tell which he is playing.{{sfn|Turing|1948|p={{page needed|date=May 2011}}}}</blockquote>

"[[Computing Machinery and Intelligence]]" ([[#{{harvid|Turing|1950}}|1950]]) was the first published paper by Turing to focus exclusively on machine intelligence. Turing begins the 1950 paper with the claim, "I propose to consider the question 'Can machines think?{{' "}}{{sfn|Turing|1950|p=433}} As he highlights, the traditional approach to such a question is to start with [[definition]]s, defining both the terms "machine" and "think". Turing chooses not to do so; instead, he replaces the question with a new one, "which is closely related to it and is expressed in relatively unambiguous words".{{sfn|Turing|1950|p=433}} In essence he proposes to change the question from "Can machines think?" to "Can machines do what we (as thinking entities) can do?"{{sfn|Harnad|2004|p=1}} The advantage of the new question, Turing argues, is that it draws "a fairly sharp line between the physical and intellectual capacities of a man".{{sfn|Turing|1950|p=434}}

To demonstrate this approach Turing proposes a test inspired by a [[party game]], known as the "imitation game", in which a man and a woman go into separate rooms and guests try to tell them apart by writing a series of questions and reading the typewritten answers sent back. In this game, both the man and the woman aim to convince the guests that they are the other. (Huma Shah argues that this two-human version of the game was presented by Turing only to introduce the reader to the machine-human question-answer test.{{sfn|Shah|Warwick|2010a}}) Turing described his new version of the game as follows:

<blockquote>We now ask the question, "What will happen when a machine takes the part of A in this game?" Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman? These questions replace our original, "Can machines think?"{{sfn|Turing|1950|p=434}}</blockquote>

Later in the paper, Turing suggests an "equivalent" alternative formulation involving a judge conversing only with a computer and a man.{{sfn|Turing|1950|p=446}} While neither of these formulations precisely matches the version of the Turing test that is more generally known today, he proposed a third in 1952. In this version, which Turing discussed in a [[BBC]] radio broadcast, a jury asks questions of a computer and the role of the computer is to make a significant proportion of the jury believe that it is really a man.<ref>{{Harvnb|Turing|1952|pp=524–525}}. Turing does not seem to distinguish between "man" as a gender and "man" as a human. In the former case, this formulation would be closer to the imitation game, whereas in the latter it would be closer to current depictions of the test.</ref>

Turing's paper considered nine putative objections, which include some of the major arguments against [[artificial intelligence]] that have been raised in the years since the paper was published (see "[[Computing Machinery and Intelligence]]").<ref name="Turing's nine objections"/>

=== The Chinese room ===
{{Main|Chinese room}}

[[John Searle]]'s 1980 paper ''[[Minds, Brains, and Programs]]'' proposed the "[[Chinese room]]" thought experiment and argued that the Turing test could not be used to determine if a machine could think. Searle noted that software (such as ELIZA) could pass the Turing test simply by manipulating symbols of which they had no understanding. Without understanding, they could not be described as "thinking" in the same sense people did. Therefore, Searle concluded, the Turing test could not prove that machines could think.{{sfn|Searle|1980}} Much like the Turing test itself, Searle's argument has been both widely criticised<ref>There are a large number of arguments against Searle's [[Chinese room]]. A few are:

* {{Citation |last=Hauser |first=Larry |author-link=Larry Hauser |year=1997 |title=Searle's Chinese Box: Debunking the Chinese Room Argument |journal=Minds and Machines |volume=7 |pages=199–226 |doi=10.1023/A:1008255830248 |issue=2 |s2cid=32153206}}.
* {{Citation |last=Rehman |first=Warren. |title=Argument against the Chinese Room Argument |date=19 July 2009 |url=http://knol.google.com/k/warren-rehman/argument-against-the-chinese-room/19dflpeo1gadq/2 |archive-url=https://web.archive.org/web/20100719002718/http://knol.google.com/k/warren-rehman/argument-against-the-chinese-room/19dflpeo1gadq/2 |archive-date=19 July 2010}}.
* {{Citation |last=Thornley |first=David H. |title=Why the Chinese Room Doesn't Work |year=1997 |url=http://www.visi.com/~thornley/david/philosophy/searle.html |archive-url=https://web.archive.org/web/20090426055512/http://www.visi.com/~thornley/david/philosophy/searle.html |archive-date=26 April 2009}}
</ref> and endorsed.<ref>M. Bishop & J. Preston (eds.) (2001) Essays on Searle's Chinese Room Argument. Oxford University Press.</ref>

Arguments such as Searle's and others working on the [[philosophy of mind]] sparked off a more intense debate about the nature of intelligence, the possibility of machines with a conscious mind and the value of the Turing test that continued through the 1980s and 1990s.{{sfn|Saygin|2000|p=479}}

=== Loebner Prize ===
{{Main|Loebner Prize}}

The Loebner Prize, now reported as defunct,<ref>{{Cite news |date=2020-11-01 |title=Robot Bores: AI-powered awkward first date |url=https://www.bbc.co.uk/news/technology-54718671 |access-date=2025-01-12 |work=BBC News |language=en-GB}}</ref> provided an annual platform for practical Turing tests with the first competition held in November 1991.{{sfn|Sundman|2003}} It was underwritten by [[Hugh Loebner]]. The Cambridge Center for Behavioral Studies in [[Massachusetts]], United States, organised the prizes up to and including the 2003 contest. As Loebner described it, one reason the competition was created is to advance the state of AI research, at least in part, because no one had taken steps to implement the Turing test despite 40 years of discussing it.{{sfn|Loebner|1994}}

The first Loebner Prize competition in 1991 led to a renewed discussion of the viability of the Turing test and the value of pursuing it, in both the popular press<ref name="Artificial Stupidity">{{cite magazine |title=Artificial Stupidity |date=1 August 1992 |newspaper=[[The Economist]] |volume=324 |issue=7770 |page=14}}</ref> and academia.<ref name=SHAPIRO_SHIEBER>{{harvnb|Shapiro|1992|page=10–11}} and {{harvnb|Shieber|1994}}, amongst others.</ref> The first contest was won by a mindless program with no identifiable intelligence that managed to fool naïve interrogators into making the wrong identification. This highlighted several of the shortcomings of the Turing test (discussed [[#Weaknesses|below]]): The winner won, at least in part, because it was able to "imitate human typing errors";<ref name="Artificial Stupidity"/> the unsophisticated interrogators were easily fooled;<ref name=SHAPIRO_SHIEBER/> and some researchers in AI have been led to feel that the test is merely a distraction from more fruitful research.{{sfn|Shieber|1994|page=77}}

The silver (text only) and gold (audio and visual) prizes have never been won. However, the competition has awarded the bronze medal every year for the computer system that, in the judges' opinions, demonstrates the "most human" conversational behaviour among that year's entries. [[Artificial Linguistic Internet Computer Entity]] (A.L.I.C.E.) has won the bronze award on three occasions in recent times (2000, 2001, 2004). Learning AI [[Jabberwacky]] won in 2005 and 2006.

The Loebner Prize tested conversational intelligence; winners were typically [[chatterbot]] programs, or [[Artificial Conversational Entity (ACE)|Artificial Conversational Entities (ACE)s]]. Early Loebner Prize rules restricted conversations: Each entry and hidden-human conversed on a single topic,<ref name=saf>{{Scientific American Frontiers |4|3}}</ref> thus the interrogators were restricted to one line of questioning per entity interaction. The restricted conversation rule was lifted for the 1995 Loebner Prize. Interaction duration between judge and entity has varied in Loebner Prizes. In Loebner 2003, at the University of Surrey, each interrogator was allowed five minutes to interact with an entity, machine or hidden-human. Between 2004 and 2007, the interaction time allowed in Loebner Prizes was more than twenty minutes. The final competition was in 2019, due to a lack of funding for the prize following Loebner's death in 2016.<ref>{{Cite news |date=2019-09-13 |title=The hobbyists competing to make AI human |url=https://www.bbc.co.uk/news/technology-49578503 |access-date=2025-01-12 |work=BBC News |language=en-GB}}</ref>

=== CAPTCHA ===
{{Main|CAPTCHA}}
[[CAPTCHA]] (Completely Automated Public Turing test to tell Computers and Humans Apart) is one of the oldest concepts for artificial intelligence. The CAPTCHA system is commonly used online to tell humans and bots apart on the internet. It is based on the Turing test. Displaying distorted letters and numbers, it asks the user to identify the letters and numbers and type them into a field, which bots struggle to do.<ref name=":0" /><ref>{{Cite web |title=How CAPTCHAs work {{!}} What does CAPTCHA mean? {{!}} Cloudflare |url=https://www.cloudflare.com/en-in/learning/bots/how-captchas-work/ |access-date=27 September 2024}}</ref>

The [[ReCAPTCHA|reCaptcha]] is a CAPTCHA system owned by [[Google]]. The reCaptcha v1 and v2 both used to operate by asking the user to match distorted pictures or identify distorted letters and numbers. The reCaptcha v3 is designed to not interrupt users and run automatically when pages are loaded or buttons are clicked. This "invisible" CAPTCHA verification happens in the background and no challenges appear, which filters out most basic bots.<ref>{{Cite web |title=reCAPTCHA |url=https://www.google.com/recaptcha/about/ |access-date=27 September 2024 |website=Google}}</ref><ref>{{Cite web |title=How does reCAPTCHA work? How it is triggered & bypassed |url=https://datadome.co/guides/captcha/how-does-recaptcha-work/ |access-date=27 September 2024}}</ref>

== Attempts ==
Several early [[Symbolic artificial intelligence|symbolic AI programs]] were controversially claimed to pass the Turing test, either by limiting themselves to scripted situations or by presenting "excuses" for poor reasoning and conversational abilities, such as [[mental illness]] or a poor grasp of English.<ref>{{Cite web |last=Masnick |first=Mike |date=9 June 2014 |title=No, A 'Supercomputer' Did NOT Pass The Turing Test for the First Time And Everyone Should Know Better |url=https://www.techdirt.com/2014/06/09/no-supercomputer-did-not-pass-turing-test-first-time-everyone-should-know-better/ |access-date=26 September 2024}}</ref>{{sfn|Weizenbaum|1966|p=42}}{{sfn|Thomas|1995|p=112}}

In 1966, [[Joseph Weizenbaum]] created a program called [[ELIZA]], which mimicked a [[person-centered psychotherapy|Rogerian psychotherapist]]. The program would search the user's sentence for keywords before [[Reflective listening|repeating them back to the user]], providing the impression of a program listening and paying attention.{{sfn|Weizenbaum|1966|p=37}} Weizenbaum thus succeeded by designing a context where a chatbot could mimic a person despite "knowing almost nothing of the real world".{{sfn|Weizenbaum|1966|p=42}} Weizenbaum's program was able to fool some people into believing that they were talking to a real person.{{sfn|Weizenbaum|1966|p=42}}

[[Kenneth Colby]] created [[PARRY]] in 1972, a program modeled after the behaviour of [[Paranoid schizophrenia|paranoid schizophrenics]].{{sfn|Boden|2006|p=370}} Psychiatrists asked to compare transcripts of conversations generated by the program to those of conversations by actual schizophrenics could only identify about 52 percent of cases correctly (a figure consistent with random guessing).{{sfn|Colby|Hilf|Weber|Kraemer|1972|p=220}}

In 2001, three programmers developed [[Eugene Goostman]], a chatbot portraying itself as a 13-year-old boy from [[Odesa]] who spoke [[English as a second or foreign language|English as a second language]]. This background was intentionally chosen so judges would forgive mistakes by the program. In a competition, 33% of judges thought Goostman was human.<ref name=":0">{{Cite book |last1=Parsons |first1=Paul |title=50 Ideas You Really Need to Know: Science |last2=Dixon |first2=Gail |publisher=[[Quercus]] |year=2016 |isbn=978-1-78429-614-8 |location=London |pages=65 |language=en}}</ref><ref>{{Cite web |title=Computer chatbot ;Eugene Goostman; passes the Turing test {{!}} ZDNET |website=[[ZDNet]] |url=https://www.zdnet.com/article/computer-chatbot-eugene-goostman-passes-the-turing-test/ |access-date=26 September 2024}}</ref><ref>{{Cite web |last=Masnick |first=Mike |title=No, A 'Supercomputer' Did NOT Pass The Turing Test for the First Time And Everyone Should Know Better |date=9 June 2014 |url=https://www.techdirt.com/2014/06/09/no-supercomputer-did-not-pass-turing-test-first-time-everyone-should-know-better/ |access-date=26 September 2024}}</ref>

=== Large language models ===
{{Main|Large language model}}
{{Expand section|info on LLMs passing more-rigorous/modern Turing test variants|date=December 2024|small=no}}

==== Google LaMDA ====
{{main|LaMDA}}

In June 2022, [[Google AI|Google]]'s [[LaMDA]] model received widespread coverage after claims about it having achieved sentience. Initially in an article in ''[[The Economist]]'' Google Research Fellow Blaise Agüera y Arcas said the chatbot had demonstrated a degree of understanding of social relationships.<ref name="economist lambda">{{cite news |author=Dan Williams |date=9 June 2022 |title=Artificial neural networks are making strides towards consciousness, according to Blaise Agüera y Arcas |url=https://www.economist.com/by-invitation/2022/06/09/artificial-neural-networks-are-making-strides-towards-consciousness-according-to-blaise-aguera-y-arcas |url-status=live |archive-url=https://web.archive.org/web/20220609093229/https://www.economist.com/by-invitation/2022/06/09/artificial-neural-networks-are-making-strides-towards-consciousness-according-to-blaise-aguera-y-arcas |archive-date=9 June 2022 |access-date=13 June 2022 |newspaper=The Economist}}</ref> Several days later, Google engineer Blake Lemoine claimed in an interview with the ''[[Washington Post]]'' that LaMDA had achieved sentience. Lemoine had been placed on leave by Google for internal assertions to this effect. Google had investigated the claims but dismissed them.<ref name="washington post lambda">{{cite news |author=Nitasha Tiku |date=11 June 2022 |title=The Google engineer who thinks the company's AI has come to life |url=https://www.washingtonpost.com/technology/2022/06/11/google-ai-lamda-blake-lemoine/ |url-status=live |archive-url=https://web.archive.org/web/20220611124805/https://www.washingtonpost.com/technology/2022/06/11/google-ai-lamda-blake-lemoine/ |archive-date=11 June 2022 |access-date=13 June 2022 |newspaper=Washington Post}}</ref><ref name="fortune lambda">{{cite news |author=Jeremy Kahn |date=13 June 2022 |title=A.I. experts say the Google researcher's claim that his chatbot became 'sentient' is ridiculous—but also highlights big problems in the field |url=https://fortune.com/2022/06/13/google-ai-researchers-sentient-chatbot-claims-ridiculed-by-experts/ |url-status=live |archive-url=https://web.archive.org/web/20220613132958/https://fortune.com/2022/06/13/google-ai-researchers-sentient-chatbot-claims-ridiculed-by-experts/ |archive-date=13 June 2022 |access-date=13 June 2022 |work=Fortune}}</ref>

==== ChatGPT ====
{{Main|ChatGPT}}

[[OpenAI]]'s chatbot, ChatGPT, released in November 2022, is based on [[GPT-3.5]] and [[GPT-4]] [[large language model]]s. Celeste Biever wrote in a ''Nature'' article that "ChatGPT broke the Turing test".<ref name="nature.com">{{cite journal |last1=Biever |first1=Celeste |date=25 July 2023 |title=ChatGPT broke the Turing test — the race is on for new ways to assess AI |url=https://www.nature.com/articles/d41586-023-02361-7 |url-status=live |journal=Nature |language=en |volume=619 |issue=7971 |pages=686–689 |bibcode=2023Natur.619..686B |doi=10.1038/d41586-023-02361-7 |pmid=37491395 |archive-url=https://web.archive.org/web/20230726211022/https://www.nature.com/articles/d41586-023-02361-7 |archive-date=26 July 2023 |access-date=26 March 2024|url-access=subscription }}</ref> Stanford researchers reported that ChatGPT passes the test; they found that ChatGPT-4 "passes a rigorous Turing test, diverging from average human behavior chiefly to be more cooperative",<ref name="humsci.stanford.edu"/> making it the first computer program to successfully do so.<ref name="A Turing test of whether AI chatbot"/>

In late March 2025, a study evaluated four systems (ELIZA, GPT-4o, LLaMa-3.1-405B, and GPT-4.5) in two randomized, controlled, and pre-registered Turing tests with independent participant groups. Participants engaged in simultaneous 5-minute conversations with another human participant and one of these systems, then judged which conversational partner they believed to be human. When instructed to adopt a humanlike persona, GPT-4.5 was identified as the human 73% of the time—significantly more often than the actual human participants. LLaMa-3.1, under the same conditions, was judged to be human 56% of the time, not significantly more or less often than the humans they were compared to. Baseline models (ELIZA and GPT-4o) achieved win rates significantly below chance (23% and 21%, respectively). These results provide the first empirical evidence that any artificial system passes a standard three-party Turing test. The findings have implications for debates about the nature of intelligence exhibited by Large Language Models (LLMs) and the social and economic impacts these systems are likely to have.<ref name="Jones2025"/>

==Versions==<!-- This title is linked to by the article Computing Machinery and Intelligence -->
[[File:The Imitation Game.svg|thumb|The imitation game, as described by Alan Turing in "Computing Machinery and Intelligence". Player C, through a series of written questions, attempts to determine which of the other two players is a man, and which of the two is the woman. Player A, the man, tries to trick player C into making the wrong decision, while player B tries to help player C. Figure adapted from Saygin, 2000.{{sfn|Saygin|2000}}]]

Saul Traiger argues that there are at least three primary versions of the Turing test, two of which are offered in "Computing Machinery and Intelligence" and one that he describes as the "Standard Interpretation".{{sfn|Traiger|2000}} While there is some debate regarding whether the "Standard Interpretation" is that described by Turing or, instead, based on a misreading of his paper, these three versions are not regarded as equivalent,{{sfn|Traiger|2000}} and their strengths and weaknesses are distinct.{{sfn|Saygin|Roberts|Beber|2008}}

Turing's original article describes a simple party game involving three players. Player A is a man, player B is a woman and player C (who plays the role of the interrogator) is of either gender. In the imitation game, player C is unable to see either player A or player B, and can communicate with them only through written notes. By asking questions of player A and player B, player C tries to determine which of the two is the man and which is the woman. Player A's role is to trick the interrogator into making the wrong decision, while player B attempts to assist the interrogator in making the right one.{{sfn|Saygin|2000}}

Turing then asks:

<blockquote>"What will happen when a machine takes the part of A in this game? Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman?" These questions replace our original, "Can machines think?"{{sfn|Turing|1950|p=434}}</blockquote>

[[File:Turing Test Version 1.svg|thumb|The original imitation game test, in which the player A is replaced with a computer. The computer is now charged with the role of the man, while player B continues to attempt to assist the interrogator. Figure adapted from Saygin, 2000.{{sfn|Saygin|2000}}]]

The second version appeared later in Turing's 1950 paper. Similar to the original imitation game test, the role of player A is performed by a computer. However, the role of player B is performed by a man rather than a woman.

<blockquote>Let us fix our attention on one particular digital computer ''C.'' Is it true that by modifying this computer to have an adequate storage, suitably increasing its speed of action, and providing it with an appropriate programme, ''C'' can be made to play satisfactorily the part of A in the imitation game, the part of B being taken by a man?{{sfn|Turing|1950|p=434}}</blockquote>

In this version, both player A (the computer) and player B are trying to trick the interrogator into making an incorrect decision.

The standard interpretation is not included in the original paper, but is both accepted and debated.
Common understanding has it that the purpose of the Turing test is not specifically to determine whether a computer is able to fool an interrogator into believing that it is a human, but rather whether a computer could ''imitate'' a human.{{sfn|Saygin|2000}} While there is some dispute whether this interpretation was intended by Turing, Sterrett believes that it was{{sfn|Moor|2003}} and thus conflates the second version with this one, while others, such as Traiger, do not{{sfn|Traiger|2000}}&nbsp;– this has nevertheless led to what can be viewed as the "standard interpretation". In this version, player A is a computer and player B a person of either sex. The role of the interrogator is not to determine which is male and which is female, but which is a computer and which is a human.{{sfn|Traiger|2000|p=99}} The fundamental issue with the standard interpretation is that the interrogator cannot differentiate which responder is human, and which is machine. There are issues about duration, but the standard interpretation generally considers this limitation as something that should be reasonable.

==Interpretations==

Controversy has arisen over which of the alternative formulations of the test Turing intended.{{sfn|Moor|2003}} Sterrett argues that two distinct tests can be extracted from his 1950 paper and that, ''[[Pace (Latin)|pace]]'' Turing's remark, they are not equivalent. The test that employs the party game and compares frequencies of success is referred to as the "Original Imitation Game Test", whereas the test consisting of a human judge conversing with a human and a machine is referred to as the "Standard Turing Test", noting that Sterrett equates this with the "standard interpretation" rather than the second version of the imitation game. Sterrett agrees that the standard Turing test (STT) has the problems that its critics cite but feels that, in contrast, the original imitation game test (OIG test) so defined is immune to many of them, due to a crucial difference: Unlike the STT, it does not make similarity to human performance the criterion, even though it employs human performance in setting a criterion for machine intelligence. A man can fail the OIG test, but it is argued that it is a virtue of a test of intelligence that failure indicates a lack of resourcefulness: The OIG test requires the resourcefulness associated with intelligence and not merely "simulation of human conversational behaviour". The general structure of the OIG test could even be used with non-verbal versions of imitation games.{{sfn|Sterrett|2000}}

According to Huma Shah, Turing himself was concerned with whether a machine could think and was providing a simple method to examine this: through human-machine question-answer sessions.{{sfn|Shah|2011}} Shah argues the imitation game which Turing described could be practicalized in two different ways: a) one-to-one interrogator-machine test, and b) simultaneous comparison of a machine with a human, both questioned in parallel by an interrogator.{{sfn|Shah|Warwick|2010a}}

Still other writers<ref>{{Harvnb|Genova|1994}}, {{Harvnb|Hayes|Ford|1995}}, {{Harvnb|Heil|1998}}, {{Harvnb|Dreyfus|1979}}</ref> have interpreted Turing as proposing that the imitation game itself is the test, without specifying how to take into account Turing's statement that the test that he proposed using the party version of the imitation game is based upon a criterion of comparative frequency of success in that imitation game, rather than a capacity to succeed at one round of the game.

Some writers argue that the imitation game is best understood by its social aspects. In his 1948 paper, Turing refers to intelligence as an "emotional concept," and notes that <blockquote>The extent to which we regard something as behaving in an intelligent manner is determined as much by our own state of mind and training as by the properties of the object under consideration. If we are able to explain and predict its behaviour or if there seems to be little underlying plan, we have little temptation to imagine intelligence. With the same object therefore it is possible that one man would consider it as intelligent and another would not; the second man would have found out the rules of its behaviour.{{sfn|Turing|1948|p=431}}</blockquote>Following this remark and similar ones scattered throughout Turing's publications, Diane Proudfoot{{sfn|Proudfoot|2013|p=398}} claims that Turing held a ''response-dependence'' approach to intelligence, according to which an intelligent (or thinking) entity is one that ''appears'' intelligent to an average interrogator. Shlomo Danziger{{sfn|Danziger|2022}} promotes a socio-technological interpretation, according to which Turing saw the imitation game not as an intelligence test but as a technological aspiration - one whose realization would likely involve a change in society's attitude toward machines. According to this reading, Turing's celebrated 50-year prediction - that by the end of the 20th century his test will be passed by some machine - actually consists of two distinguishable predictions. The first is a technological prediction:<blockquote>I believe that in about fifty years' time it will be possible to programme computers ... to make them play the imitation game so well that an average interrogator will not have more than 70% chance of making the right identification after five minutes of questioning.{{sfn|Turing|1950|p=442}}</blockquote>The second prediction Turing makes is a sociological one:<blockquote>I believe that at the end of the century the use of words and general educated opinion will have altered so much that one will be able to speak of machines thinking without expecting to be contradicted.{{sfn|Turing|1950|p=442}}</blockquote>Danziger claims further that for Turing, alteration of society's attitude towards machinery is a prerequisite for the existence of intelligent machines: Only when the term "intelligent machine" is no longer seen as an oxymoron the existence of intelligent machines would become ''logically'' possible.

Saygin has suggested that maybe the original game is a way of proposing a less biased experimental design as it hides the participation of the computer.<ref>R. Epstein, G. Roberts, G. Poland, (eds.) Parsing the Turing Test: Philosophical and Methodological Issues in the Quest for the Thinking Computer. Springer: Dordrecht, Netherlands</ref> The imitation game also includes a "social hack" not found in the standard interpretation, as in the game both computer and male human are required to play as pretending to be someone they are not.<ref>{{cite news |first=Clive |last=Thompson |title=The Other Turing Test |date=July 2005 |publisher=[[WIRED]] magazine |url=https://www.wired.com/wired/archive/13.07/posts.html?pg=5 |work=Issue 13.07 |access-date=10 September 2011 |quote=As a gay man who spent nearly his whole life in the closet, Turing must have been keenly aware of the social difficulty of constantly faking your real identity. And there's a delicious irony in the fact that decades of AI scientists have chosen to ignore Turing's gender-twisting test&nbsp;– only to have it seized upon by three college-age women |archive-date=19 August 2011 |archive-url=https://web.archive.org/web/20110819081652/http://www.wired.com/wired/archive/13.07/posts.html?pg=5 |url-status=live}}. ([http://www.collisiondetection.net/mt/archives/2005/07/research_subjec.php Full version] {{Webarchive|url=https://web.archive.org/web/20190323061743/http://www.collisiondetection.net/mt/archives/2005/07/research_subjec.php |date=23 March 2019 }}).</ref>

=== Should the interrogator know about the computer? ===

A crucial piece of any laboratory test is that there should be a control. Turing never makes clear whether the interrogator in his tests is aware that one of the participants is a computer. He states only that player A is to be replaced with a machine, not that player C is to be made aware of this replacement.{{sfn|Turing|1950|p=434}} When Colby, FD Hilf, S Weber and AD Kramer tested PARRY, they did so by assuming that the interrogators did not need to know that one or more of those being interviewed was a computer during the interrogation.{{sfn|Colby|Hilf|Weber|Kraemer|1972}} As Ayse Saygin, Peter Swirski,{{sfn|Swirski|2000}} and others have highlighted, this makes a big difference to the implementation and outcome of the test.{{sfn|Saygin|2000}} An experimental study looking at [[Paul Grice#Conversational Maxims|Gricean maxim violations]] using transcripts of Loebner's one-to-one (interrogator-hidden interlocutor) Prize for AI contests between 1994 and 1999, Ayse Saygin found significant differences between the responses of participants who knew and did not know about computers being involved.{{sfn|Saygin|Cicekli|2002}}

== Strengths ==

=== Tractability and simplicity ===

The power and appeal of the Turing test derives from its simplicity. The [[philosophy of mind]], [[psychology]], and modern [[neuroscience]] have been unable to provide definitions of "intelligence" and "thinking" that are sufficiently precise and general to be applied to machines. Without such definitions, the central questions of the [[philosophy of artificial intelligence]] cannot be answered. The Turing test, even if imperfect, at least provides something that can actually be measured. As such, it is a pragmatic attempt to answer a difficult philosophical question.

=== Breadth of subject matter ===

The format of the test allows the interrogator to give the machine a wide variety of intellectual tasks. Turing wrote that "the question and answer method seems to be suitable for introducing almost any one of the fields of human endeavour that we wish to include".{{sfn|Turing|1950|loc=under "Critique of the New Problem"}} [[John Haugeland]] adds that "understanding the words is not enough; you have to understand the ''topic'' as well".{{sfn|Haugeland|1985|p=8}}

To pass a well-designed Turing test, the machine must use [[natural language processing|natural language]], [[commonsense reasoning|reason]], have [[knowledge representation|knowledge]] and [[machine learning|learn]]. The test can be extended to include video input, as well as a "hatch" through which objects can be passed: this would force the machine to demonstrate skilled use of well designed [[computer vision|vision]] and [[robotics]] as well. Together, these represent almost all of the major problems that artificial intelligence research would like to solve.<ref>"These six disciplines," write [[Stuart J. Russell]] and [[Peter Norvig]], "represent most of AI." {{Harvnb|Russell|Norvig|2003|p=3}}</ref>

The [[Feigenbaum test]] is designed to take advantage of the broad range of topics available to a Turing test. It is a limited form of Turing's question-answer game which compares the machine against the abilities of experts in specific fields such as literature or [[chemistry]].

=== Emphasis on emotional and aesthetic intelligence ===

As a Cambridge honours graduate in mathematics, Turing might have been expected to propose a test of computer intelligence requiring expert knowledge in some highly technical field, and thus anticipating [[Subject-matter expert Turing test|a more recent approach to the subject]]. Instead, as already noted, the test which he described in his seminal 1950 paper requires the computer to be able to compete successfully in a common party game, and this by performing as well as the typical man in answering a series of questions so as to pretend convincingly to be the woman contestant.

Given the status of human sexual dimorphism as [[Creation myths|one of the most ancient of subjects]], it is thus implicit in the above scenario that the questions to be answered will involve neither specialised factual knowledge nor information processing technique. The challenge for the computer, rather, will be to demonstrate empathy for the role of the female, and to demonstrate as well a characteristic aesthetic sensibility—both of which qualities are on display in this snippet of dialogue which Turing has imagined:

: Interrogator: Will X please tell me the length of his or her hair?

: Contestant: My hair is shingled, and the longest strands are about nine inches long.

When Turing does introduce some specialised knowledge into one of his imagined dialogues, the subject is not maths or electronics, but poetry:

: Interrogator: In the first line of your sonnet which reads, "Shall I compare thee to a summer's day," would not "a spring day" do as well or better?

: Witness: It wouldn't [[scansion|scan]].

: Interrogator: How about "a winter's day". That would scan all right.

: Witness: Yes, but nobody wants to be compared to a winter's day.

Turing thus once again demonstrates his interest in empathy and aesthetic sensitivity as components of an artificial intelligence; and in light of an increasing awareness of the threat from an AI run amok,<ref>{{cite web |url=http://waitbutwhy.com/2015/01/artificial-intelligence-revolution-2.html |author=Urban, Tim |date=February 2015 |title=The AI Revolution: Our Immortality or Extinction |publisher=Wait But Why |access-date=5 April 2015 |archive-date=23 March 2019 |archive-url=https://web.archive.org/web/20190323061749/https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-2.html |url-status=live}}</ref> it has been suggested<ref>{{cite web |url=http://artent.net/2015/03/27/art-and-artificial-intelligence-by-g-w-smith/ |author=Smith, G. W. |date=27 March 2015 |title=Art and Artificial Intelligence |publisher=ArtEnt |access-date=27 March 2015 |url-status=live |archive-url=https://web.archive.org/web/20170625075845/http://artent.net/2015/03/27/art-and-artificial-intelligence-by-g-w-smith/ |archive-date=25 June 2017}}</ref> that this focus perhaps represents a critical intuition on Turing's part, i.e., that emotional and aesthetic intelligence will play a key role in the creation of a "[[friendly AI]]". It is further noted, however, that whatever inspiration Turing might be able to lend in this direction depends upon the preservation of his original vision, which is to say, further, that the promulgation of a "standard interpretation" of the Turing test—i.e., one which focuses on a discursive intelligence only—must be regarded with some caution.

== Weaknesses ==

Turing did not explicitly state that the Turing test could be used as a measure of "[[intelligence]]", or any other human quality. He wanted to provide a clear and understandable alternative to the word "think", which he could then use to reply to criticisms of the possibility of "thinking machines" and to suggest ways that research might move forward.

Nevertheless, the Turing test has been proposed as a measure of a machine's "ability to think" or its "intelligence". This proposal has received criticism from both philosophers and computer scientists. The interpretation makes the assumption that an interrogator can determine if a machine is "thinking" by comparing its behaviour with human behaviour. Every element of this assumption has been questioned: the reliability of the interrogator's judgement, the value of comparing the machine with a human, and the value of comparing only behaviour. Because of these and other considerations, some AI researchers have questioned the relevance of the test to their field.

=== Naïveté of interrogators ===

In practice, the test's results can easily be dominated not by the computer's intelligence, but by the attitudes, skill, or naïveté of the questioner. Numerous experts in the field, including cognitive scientist [[Gary Marcus]], insist that the Turing test only shows how easy it is to fool humans and is not an indication of machine intelligence.<ref>{{Cite magazine |last=Marcus |first=Gary |date=9 June 2014 |title=What Comes After the Turing Test? |url=https://www.newyorker.com/tech/annals-of-technology/what-comes-after-the-turing-test |url-status=live |access-date=16 December 2021 |magazine=The New Yorker |language=en-US |archive-date=1 January 2022 |archive-url=https://web.archive.org/web/20220101024118/https://www.newyorker.com/tech/annals-of-technology/what-comes-after-the-turing-test}}</ref>

Turing doesn't specify the precise skills and knowledge required by the interrogator in his description of the test, but he did use the term "average interrogator": "[the] average interrogator would not have more than 70 per cent chance of making the right identification after five minutes of questioning".{{sfn|Turing|1950|p=442}}

Chatterbot programs such as ELIZA have repeatedly fooled unsuspecting people into believing that they are communicating with human beings. In these cases, the "interrogators" are not even aware of the possibility that they are interacting with computers. To successfully appear human, there is no need for the machine to have any intelligence whatsoever and only a superficial resemblance to human behaviour is required.

Early Loebner Prize competitions used "unsophisticated" interrogators who were easily fooled by the machines.<ref name=SHAPIRO_SHIEBER/> Since 2004, the Loebner Prize organisers have deployed philosophers, computer scientists, and journalists among the interrogators. Nonetheless, some of these experts have been deceived by the machines.{{sfn|Shah|Warwick|2010j}}

One interesting feature of the Turing test is the frequency of the [[confederate effect]], when the confederate (tested) humans are misidentified by the interrogators as machines. It has been suggested that what interrogators expect as human responses is not necessarily typical of humans. As a result, some individuals can be categorised as machines. This can therefore work in favour of a competing machine. The humans are instructed to "act themselves", but sometimes their answers are more like what the interrogator expects a machine to say.<ref>{{cite journal |title=Human Misidentification in Turing Tests |journal=Journal of Experimental and Theoretical Artificial Intelligence |date=Jun 2014 |doi=10.1080/0952813X.2014.921734 |author1=Kevin Warwick |author2=Huma Shah |volume=27 |issue=2 |pages=123–135 |s2cid=45773196}}</ref> This raises the question of how to ensure that the humans are motivated to "act human".

=== Human intelligence vs. intelligence in general ===

[[File:Weakness of Turing test 1.svg|250px|thumb]]
The Turing test does not directly test whether the computer behaves intelligently. It tests only whether the computer behaves like a human being. Since human behaviour and intelligent behaviour are not exactly the same thing, the test can fail to accurately measure intelligence in two ways:

; Some human behaviour is unintelligent: The Turing test requires that the machine be able to execute ''all'' human behaviours, regardless of whether they are intelligent. It even tests for behaviours that may not be considered intelligent at all, such as the susceptibility to insults,{{sfn|Saygin|Cicekli|2002|pp=227–258}} the temptation to [[lie]] or, simply, a high frequency of [[typographical error|typing mistakes]]. If a machine cannot imitate these unintelligent behaviours in detail it fails the test.

: This objection was raised by ''[[The Economist]],'' in an article entitled "[[artificial stupidity]]" published shortly after the first Loebner Prize competition in 1992. The article noted that the first Loebner winner's victory was due, at least in part, to its ability to "imitate human typing errors".<ref name="Artificial Stupidity"/> Turing himself had suggested that programs add errors into their output, so as to be better "players" of the game.{{sfn|Turing|1950|p=448}}

; Some intelligent behaviour is inhuman: The Turing test does not test for highly intelligent behaviours, such as the ability to solve difficult problems or come up with original insights. In fact, it specifically requires deception on the part of the machine: if the machine is ''more'' intelligent than a human being it must deliberately avoid appearing too intelligent. If it were to solve a computational problem that is practically impossible for a human to solve, then the interrogator would know the program is not human, and the machine would fail the test.

: Because it cannot measure intelligence that is beyond the ability of humans, the test cannot be used to build or evaluate systems that are more intelligent than humans. Because of this, several test alternatives that would be able to evaluate super-intelligent systems have been proposed.<ref>Several alternatives to the Turing test, designed to evaluate machines more intelligent than humans:
* {{Citation |title=Beyond the Turing Test |journal=Journal of Logic, Language and Information |author=Jose Hernandez-Orallo |year=2000 |pages=447–466 |volume=9 |issue=4 |doi=10.1023/A:1008367325700 |postscript=. |citeseerx=10.1.1.44.8943 |s2cid=14481982}}
* {{Citation |title=A computational extension to the Turing Test |journal=Proceedings of the 4th Conference of the Australasian Cognitive Science Society |author1=D L Dowe |author2=A R Hajek |name-list-style=amp |url=http://www.csse.monash.edu.au/publications/1997/tr-cs97-322-abs.html |access-date=21 July 2009 |year=1997 |postscript=. |archive-url=https://web.archive.org/web/20110628194905/http://www.csse.monash.edu.au/publications/1997/tr-cs97-322-abs.html |archive-date=28 June 2011}}
* {{Citation |title=Universal Intelligence: A Definition of Machine Intelligence |journal=Minds and Machines |author1=Shane Legg |author2=Marcus Hutter |name-list-style=amp |url=http://www.vetta.org/documents/UniversalIntelligence.pdf |access-date=21 July 2009 |year=2007 |pages=391–444 |issue=4 |volume=17 |doi=10.1007/s11023-007-9079-x |postscript=. |arxiv=0712.3329 |archive-url=https://web.archive.org/web/20090618152351/http://www.vetta.org/documents/UniversalIntelligence.pdf |archive-date=18 June 2009 |bibcode=2007arXiv0712.3329L |hdl=1885/14995 |s2cid=847021}}
* {{Citation |title=Measuring Universal Intelligence: Towards an Anytime Intelligence Test |journal=Artificial Intelligence |first1=J |last1=Hernandez-Orallo |first2=D L |last2=Dowe |volume=174 |issue=18 |pages=1508–1539 |year=2010 |doi=10.1016/j.artint.2010.09.006 |postscript=. |doi-access=free}}
</ref>

=== Consciousness vs. the simulation of consciousness ===
{{Main|Chinese room}}
{{See also|Synthetic intelligence}}
The Turing test is concerned strictly with how the subject ''acts''&nbsp;– the external behaviour of the machine. In this regard, it takes a [[behaviourist]] or [[Functionalism (philosophy of mind)|functionalist]] approach to the study of the mind. The example of [[ELIZA]] suggests that a machine passing the test may be able to simulate human conversational behaviour by following a simple (but large) list of mechanical rules, without thinking or having a mind at all.

[[John Searle]] has argued that external behaviour cannot be used to determine if a machine is "actually" thinking or merely "simulating thinking".{{sfn|Searle|1980}} His [[Chinese room]] argument is intended to show that, even if the Turing test is a good operational definition of intelligence, it may not indicate that the machine has a [[mind]], [[consciousness]], or [[intentionality]]. (Intentionality is a philosophical term for the power of thoughts to be "about" something.)

Turing anticipated this line of criticism in his original paper,<ref>{{Harvtxt|Russell|Norvig|2003|pp=958–960}} identify Searle's argument with the one Turing answers.</ref> writing: {{blockquote|I do not wish to give the impression that I think there is no mystery about consciousness. There is, for instance, something of a paradox connected with any attempt to localise it. But I do not think these mysteries necessarily need to be solved before we can answer the question with which we are concerned in this paper.{{sfn|Turing|1950}} }}

=== Impracticality and irrelevance: the Turing test and AI research ===

[[File:Sam Harris takes the Turing Test (51169748740).png|thumb|GPT-3 talkbot attempt]]

Mainstream AI researchers argue that trying to pass the Turing test is merely a distraction from more fruitful research.{{sfn|Shieber|1994|p=77}} Indeed, the Turing test is not an active focus of much academic or commercial effort—as [[Stuart J. Russell|Stuart Russell]] and [[Peter Norvig]] write: "AI researchers have devoted little attention to passing the Turing test".{{sfn|Russell|Norvig|2003|p=3}} There are several reasons.

First, there are easier ways to test their programs. Most current research in AI-related fields is aimed at modest and specific goals, such as [[object recognition]] or [[logistics]]. To test the intelligence of the programs that solve these problems, AI researchers simply give them the task directly. Stuart Russell and Peter Norvig suggest an analogy with the [[history of flight]]: Planes are tested by how well they fly, not by comparing them to birds. "[[Aeronautics|Aeronautical engineering]] texts," they write, "do not define the goal of their field as 'making machines that fly so exactly like [[pigeon]]s that they can fool other pigeons.{{' "}}{{sfn|Russell|Norvig|2003|p=3}}

Second, creating lifelike simulations of human beings is a difficult problem on its own that does not need to be solved to achieve the basic goals of AI research. Believable human characters may be interesting in a work of art, a [[video game|game]], or a sophisticated [[user interface]], but they are not part of the science of creating intelligent machines, that is, machines that solve problems using intelligence.

Turing did not intend for his idea to be used to test the intelligence of programs&mdash;he wanted to provide a clear and understandable example to aid in the discussion of the [[philosophy of artificial intelligence]].<ref>{{Harvnb|Turing|1950}}, under the heading "The Imitation Game," where he writes, "Instead of attempting such a definition I shall replace the question by another, which is closely related to it and is expressed in relatively unambiguous words."</ref> [[John McCarthy (computer scientist)|John McCarthy]] argues that we should not be surprised that a philosophical idea turns out to be useless for practical applications. He observes that the philosophy of AI is "unlikely to have any more effect on the practice of AI research than philosophy of science generally has on the practice of science".<ref>{{Citation |last=McCarthy |first=John |author-link=John McCarthy (computer scientist) |title=The Philosophy of Artificial Intelligence |work=What has AI in Common with Philosophy? |year=1996 |url=http://www-formal.stanford.edu/jmc/aiphil/node2.html#SECTION00020000000000000000 |access-date=26 February 2009 |archive-date=5 April 2019 |archive-url=https://web.archive.org/web/20190405214706/http://www-formal.stanford.edu/jmc/aiphil/node2.html#SECTION00020000000000000000 |url-status=live}}</ref><ref>{{Cite journal |last=Brynjolfsson |first=Erik |date=1 May 2022 |title=The Turing Trap: The Promise & Peril of Human-Like Artificial Intelligence |journal=Daedalus |volume=151 |issue=2 |pages=272–287 |doi=10.1162/daed_a_01915 |doi-access=free}}</ref>

=== The language-centric objection ===

Another well known objection raised towards the Turing test concerns its exclusive focus on linguistic behaviour (i.e. it is only a "language-based" experiment, while all the other cognitive faculties are not tested). This drawback downsizes the role of other modality-specific "intelligent abilities" concerning human beings that the psychologist Howard Gardner, in his "[[Theory of multiple intelligences|multiple intelligence theory]]", proposes to consider (verbal-linguistic abilities are only one of those).<ref>Gardner, H. (2011). Frames of mind: The theory of multiple intelligences. Hachette Uk</ref>

=== Silence ===

A critical aspect of the Turing test is that a machine must give itself away as being a machine by its utterances. An interrogator must then make the "right identification" by correctly identifying the machine as being just that. If, however, a machine remains silent during a conversation, then it is not possible for an interrogator to accurately identify the machine other than by means of a calculated guess.<ref>{{Cite journal |last1=Warwick |first1=Kevin |last2=Shah |first2=Huma |date=4 March 2017 |title=Taking the fifth amendment in Turing's imitation game |url=https://curve.coventry.ac.uk/open/items/9490ae0b-63a1-4427-871e-017cfaa2fe7e/1/Taking+the+Fifth+-+complete.pdf |journal=Journal of Experimental & Theoretical Artificial Intelligence |language=en |volume=29 |issue=2 |pages=287–297 |doi=10.1080/0952813X.2015.1132273 |bibcode=2017JETAI..29..287W |s2cid=205634569 |issn=0952-813X}}{{Dead link|date=September 2023 |bot=InternetArchiveBot |fix-attempted=yes }}</ref>
Even taking into account a parallel/hidden human as part of the test may not help the situation as humans can often be misidentified as being a machine.<ref>{{Cite journal |last1=Warwick |first1=Kevin |last2=Shah |first2=Huma |date=4 March 2015 |title=Human misidentification in Turing tests |journal=Journal of Experimental & Theoretical Artificial Intelligence |language=en |volume=27 |issue=2 |pages=123–135 |doi=10.1080/0952813X.2014.921734 |s2cid=45773196 |issn=0952-813X}}</ref>

=== The Turing Trap ===
By focusing on ''imitating'' humans, rather than augmenting or extending human capabilities, the Turing Test risks directing research and implementation toward technologies that substitute for humans and thereby drive down wages and income for workers. As they lose economic power, these workers may also lose political power, making it more difficult for them to change the allocation of wealth and income. This can trap them in a bad equilibrium. Erik Brynjolfsson has called this "The Turing Trap"<ref>{{Cite web|url=https://digitaleconomy.stanford.edu/news/the-turing-trap-the-promise-peril-of-human-like-artificial-intelligence/|title=The Turing Trap: The Promise & Peril of Human-Like Artificial Intelligence|website=Stanford Digital Economy Lab}}</ref> and argued that there are currently excess incentives for creating machines that imitate rather than augment humans.

== Variations ==

Numerous other versions of the Turing test, including those expounded above, have been raised through the years.

=== Reverse Turing test and CAPTCHA ===
{{Main|Reverse Turing test|CAPTCHA}}
A modification of the Turing test wherein the objective of one or more of the roles have been reversed between machines and humans is termed a reverse Turing test. An example is implied in the work of psychoanalyst [[Wilfred Bion]],{{sfn|Bion|1979}} who was particularly fascinated by the "storm" that resulted from the encounter of one mind by another. In his 2000 book,{{sfn|Swirski|2000}} among several other original points with regard to the Turing test, literary scholar [[Peter Swirski]] discussed in detail the idea of what he termed the Swirski test—essentially the reverse Turing test. He pointed out that it overcomes most if not all standard objections levelled at the standard version.

Carrying this idea forward, [[R. D. Hinshelwood]]{{sfn|Hinshelwood|2001}} described the mind as a "mind recognizing apparatus". The challenge would be for the computer to be able to determine if it were interacting with a human or another computer. This is an extension of the original question that Turing attempted to answer but would, perhaps, offer a high enough standard to define a machine that could "think" in a way that we typically define as characteristically human.

[[CAPTCHA]] is a form of reverse Turing test. Before being allowed to perform some action on a website, the user is presented with alphanumerical characters in a distorted graphic image and asked to type them out. This is intended to prevent automated systems from being used to abuse the site. The rationale is that software sufficiently sophisticated to read and reproduce the distorted image accurately does not exist (or is not available to the average user), so any system able to do so is likely to be a human.

Software that could reverse CAPTCHA with some accuracy by analysing patterns in the generating engine started being developed soon after the creation of CAPTCHA.<ref>{{Citation |url=http://www.cs.sfu.ca/~mori/research/gimpy/ |title=Breaking a Visual CAPTCHA |last1=Malik |first1=Jitendra |author-link=Jitendra Malik |last2=Mori |first2=Greg |access-date=21 November 2009 |archive-date=23 March 2019 |archive-url=https://web.archive.org/web/20190323061739/http://www.cs.sfu.ca/~mori/research/gimpy/ |url-status=live}}</ref>
In 2013, researchers at [[Vicarious (company)|Vicarious]] announced that they had developed a system to solve CAPTCHA challenges from [[Google]], [[Yahoo!]], and [[PayPal]] up to 90% of the time.<ref>{{Citation |url=http://mashable.com/2013/10/28/captcha-defeated/#QyX1ZwnE9sqr |title=Captcha FAIL: Researchers Crack the Web's Most Popular Turing Test |last=Pachal |first=Pete |access-date=31 December 2015 |archive-date=3 December 2018 |archive-url=https://web.archive.org/web/20181203140616/https://mashable.com/2013/10/28/captcha-defeated/#QyX1ZwnE9sqr |url-status=live}}</ref>
In 2014, Google engineers demonstrated a system that could defeat CAPTCHA challenges with 99.8% accuracy.<ref>{{Citation |url=https://www.zdnet.com/article/google-algorithm-busts-captcha-with-99-8-percent-accuracy/ |title=Google algorithm busts CAPTCHA with 99.8 percent accuracy |last=Tung |first=Liam |access-date=31 December 2015 |archive-date=23 March 2019 |archive-url=https://web.archive.org/web/20190323061748/https://www.zdnet.com/article/google-algorithm-busts-captcha-with-99-8-percent-accuracy/ |url-status=live}}</ref>
In 2015, [[Shuman Ghosemajumder]], former [[click fraud]] czar of Google, stated that there were [[Cybercrime|cybercriminal]] sites that would defeat CAPTCHA challenges for a fee, to enable various forms of fraud.<ref>{{Citation |url=http://www.infoq.com/presentations/ai-security |title=The Imitation Game: The New Frontline of Security |last=Ghosemajumder |first=Shuman |access-date=31 December 2015 |archive-date=23 March 2019 |archive-url=https://web.archive.org/web/20190323061742/https://www.infoq.com/presentations/ai-security |url-status=live}}</ref>

=== Distinguishing accurate use of language from actual understanding ===

A further variation is motivated by the concern that modern Natural Language Processing prove to be highly successful in generating text on the basis of a huge text corpus and could eventually pass the Turing test simply by manipulating words and sentences that have been used in the initial training of the model. Since the interrogator has no precise understanding of the training data, the model might simply be returning sentences that exist in similar fashion in the enormous amount of training data. For this reason, [[Arthur Schwaninger]] proposes a variation of the Turing test that can distinguish between systems that are only capable of ''using'' language and systems that ''understand'' language. He proposes a test in which the machine is confronted with philosophical questions that do not depend on any prior knowledge and yet require self-reflection to be answered appropriately.<ref>{{Citation |last=Schwaninger |first=Arthur C. |author-link=Arthur C. Schwaninger |title=The Philosophising Machine – a Specification of the Turing Test |journal=Philosophia |year=2022 |volume=50 |issue=3 |pages=1437–1453 |doi=10.1007/s11406-022-00480-5 |s2cid=247282718 |doi-access=free}}</ref>

=== Subject matter expert Turing test ===
{{Main|Subject-matter expert Turing test}}
Another variation is described as the [[subject-matter expert]] Turing test, where a machine's response cannot be distinguished from an expert in a given field. This is also known as a "Feigenbaum test" and was proposed by [[Edward Feigenbaum]] in a 2003 paper.<ref>{{Harvnb|McCorduck|2004|pp=503–505}}, {{Harvnb|Feigenbaum|2003}}. The subject matter expert test is also mentioned in {{Harvtxt|Kurzweil|2005}}</ref>

=== "Low-level" cognition test ===

[[Robert M. French|Robert French]] (1990) makes the case that an interrogator can distinguish human and non-human interlocutors by posing questions that reveal the low-level (i.e., unconscious) processes of human cognition, as studied by [[cognitive science]]. Such questions reveal the precise details of the human embodiment of thought and can unmask a computer unless it experiences the world as humans do.<ref>{{Citation |last=French |first=Robert M. |author-link=Robert M. French |title=Subcognition and the Limits of the Turing Test |journal=Mind |volume=99 |issue=393 |pages=53–65}}</ref>

=== Total Turing test ===

The "Total Turing test"<ref name="The Turing Test"/> variation of the Turing test, proposed by cognitive scientist [[Stevan Harnad]],<ref>{{citation |last=Gent |first=Edd |year=2014 |title=The Turing Test: brain-inspired computing's multiple-path approach |url=https://eandt.theiet.org/content/articles/2014/11/the-turing-test-brain-inspired-computings-multiple-path-approach/ |access-date=18 October 2018 |archive-date=23 March 2019 |archive-url=https://web.archive.org/web/20190323061739/https://eandt.theiet.org/content/articles/2014/11/the-turing-test-brain-inspired-computings-multiple-path-approach/ |url-status=live}}</ref> adds two further requirements to the traditional Turing test. The interrogator can also test the perceptual abilities of the subject (requiring [[computer vision]]) and the subject's ability to manipulate objects (requiring [[robotics]]).{{sfn|Russell|Norvig|2010|p=3}}

=== Electronic health records ===

A letter published in ''Communications of the ACM''<ref>{{Cite journal |doi=10.1145/3168260 |title=A leap from artificial to intelligence |journal=Communications of the ACM |volume=61 |pages=10–11 |year=2017 |last1=Cacm Staff}}</ref> describes the concept of generating a synthetic patient population and proposes a variation of Turing test to assess the difference between synthetic and real patients. The letter states: "In the EHR context, though a human physician can readily distinguish between synthetically generated and real live human patients, could a machine be given the intelligence to make such a determination on its own?" and further the letter states: "Before synthetic patient identities become a public health problem, the legitimate EHR market might benefit from applying Turing Test-like techniques to ensure greater data reliability and diagnostic value. Any new techniques must thus consider patients' heterogeneity and are likely to have greater complexity than the Allen eighth-grade-science-test is able to grade".

=== Minimum intelligent signal test ===
{{Main|Minimum intelligent signal test}}
The minimum intelligent signal test was proposed by [[Chris McKinstry]] as "the maximum abstraction of the Turing test",<ref>{{cite web |url=http://tech.groups.yahoo.com/group/arcondev/message/337 |title=Arcondev : Message: Re: &#91;arcondev&#93; MIST = fog? |access-date=28 December 2023 |archive-date=30 June 2013 |archive-url=https://archive.today/20130630171858/http://tech.groups.yahoo.com/group/arcondev/message/337 |url-status=dead}}</ref> in which only binary responses (true/false or yes/no) are permitted, to focus only on the capacity for thought. It eliminates text chat problems like [[#Naïveté of interrogators|anthropomorphism bias]], and does not require emulation of [[#Human intelligence vs. intelligence in general|unintelligent human behaviour]], allowing for systems that exceed human intelligence. The questions must each stand on their own, however, making it more like an [[IQ test]] than an interrogation. It is typically used to gather statistical data against which the performance of artificial intelligence programs may be measured.<ref>{{citation |last=McKinstry |first=Chris |author-link=Chris McKinstry |year=1997 |title=Minimum Intelligent Signal Test: An Alternative Turing Test |journal=Canadian Artificial Intelligence |issue=41 |url=http://hps.elte.hu/~gk/Loebner/kcm9512.htm |access-date=4 May 2011 |archive-date=31 March 2019 |archive-url=https://web.archive.org/web/20190331081116/http://hps.elte.hu/~gk/Loebner/kcm9512.htm |url-status=live}}</ref>

=== Hutter Prize ===

The organisers of the [[Hutter Prize]] believe that compressing natural language text is a hard AI problem, equivalent to passing the Turing test. The data compression test has some advantages over most versions and variations of a Turing test, including:{{citation needed|date=November 2024}}

* It gives a single number that can be directly used to compare which of two machines is "more intelligent".
* It does not require the computer to lie to the judge

The main disadvantages of using data compression as a test are:

* It is not possible to test humans this way.
* It is unknown what particular "score" on this test—if any—is equivalent to passing a human-level Turing test.

=== Other tests based on compression or Kolmogorov complexity ===

A related approach to Hutter's prize which appeared much earlier in the late 1990s is the inclusion of compression problems in an extended Turing test.<ref>{{Citation |title=A computational extension to the Turing Test |journal=Proceedings of the 4th Conference of the Australasian Cognitive Science Society |author1=D L Dowe |author2=A R Hajek |name-list-style=amp |url=http://www.csse.monash.edu.au/publications/1997/tr-cs97-322-abs.html |access-date=21 July 2009 |year=1997 |postscript=. |archive-url=https://web.archive.org/web/20110628194905/http://www.csse.monash.edu.au/publications/1997/tr-cs97-322-abs.html |archive-date=28 June 2011}}</ref> or by tests which are completely derived from [[Kolmogorov complexity]].<ref>{{Citation |title=Beyond the Turing Test |journal=Journal of Logic, Language and Information |author=Jose Hernandez-Orallo |year=2000 |pages=447–466 |volume=9 |issue=4 |doi=10.1023/A:1008367325700 |citeseerx=10.1.1.44.8943 |s2cid=14481982}}</ref>
Other related tests in this line are presented by Hernandez-Orallo and Dowe.{{sfn|Hernandez-Orallo|Dowe|2010}}

Algorithmic IQ, or AIQ for short, is an attempt to convert the theoretical Universal Intelligence Measure from Legg and Hutter (based on [[Solomonoff's theory of inductive inference|Solomonoff's inductive inference]]) into a working practical test of machine intelligence.<ref>An Approximation of the Universal Intelligence Measure, Shane Legg and Joel Veness, 2011 Solomonoff Memorial Conference</ref>

Two major advantages of some of these tests are their applicability to nonhuman intelligences and their absence of a requirement for human testers.

=== Ebert test ===

The Turing test inspired the [[Ebert test]] proposed in 2011 by film critic [[Roger Ebert]] which is a test whether a computer-based [[Speech synthesis|synthesised voice]] has sufficient skill in terms of intonations, inflections, timing and so forth, to make people laugh.<ref name=twsL37>{{cite news |author=Alex_Pasternack |title=A MacBook May Have Given Roger Ebert His Voice, But An iPod Saved His Life (Video) |publisher=Motherboard |quote=He calls it the "Ebert Test," after Turing's AI standard... |date=18 April 2011 |url=http://www.motherboard.tv/2011/4/18/a-macbook-may-have-given-roger-ebert-his-voice-but-an-ipod-saved-his-life-video |access-date=12 September 2011 |archive-url=https://web.archive.org/web/20110906063605/http://motherboard.tv/2011/4/18/a-macbook-may-have-given-roger-ebert-his-voice-but-an-ipod-saved-his-life-video |archive-date=6 September 2011}}</ref>

=== Social Turing game ===

Taking advantage of [[large language model]]s, in 2023 the research company [[AI21 Labs]] created an online social experiment titled "Human or Not?"<ref>{{Cite web |last=Key |first=Alys |date=21 April 2023 |title=Could you tell if someone was human or AI? |url=https://www.standard.co.uk/tech/human-or-not-ai-game-robot-turing-test-b1075894.html |access-date=2 August 2023 |website=Evening Standard |language=en |archive-date=2 August 2023 |archive-url=https://web.archive.org/web/20230802143702/https://www.standard.co.uk/tech/human-or-not-ai-game-robot-turing-test-b1075894.html |url-status=live}}</ref><ref>{{Cite web |last= |title=Massive Turing test shows we can only just tell AIs apart from humans |url=https://www.newscientist.com/article/2376899-massive-turing-test-shows-we-can-only-just-tell-ais-apart-from-humans/ |access-date=2 August 2023 |website=New Scientist |language=en-US |archive-date=22 July 2024 |archive-url=https://web.archive.org/web/20240722040929/https://www.newscientist.com/article/2376899-massive-turing-test-shows-we-can-only-just-tell-ais-apart-from-humans/ |url-status=live}}</ref>  It was played more than 10 million times by more than 2 million people.<ref>{{Cite journal |last=Biever |first=Celeste |date=25 July 2023 |title=ChatGPT broke the Turing test — the race is on for new ways to assess AI |journal=Nature |language=en |volume=619 |issue=7971 |pages=686–689 |doi=10.1038/d41586-023-02361-7 |pmid=37491395 |bibcode=2023Natur.619..686B |doi-access=free}}</ref> It is the biggest Turing-style experiment to that date. The results showed that 32% of people could not distinguish between humans and machines.<ref>{{Cite web |title=Can you distinguish people from AI bots? 'Human or not' online game reveals results |url=https://www.zdnet.com/article/can-you-distinguish-people-from-ai-bots-human-or-not-online-game-reveals-results/ |access-date=2 August 2023 |website=ZDNET |language=en |archive-date=6 May 2024 |archive-url=https://web.archive.org/web/20240506000924/https://www.zdnet.com/article/can-you-distinguish-people-from-ai-bots-human-or-not-online-game-reveals-results/ |url-status=live}}</ref><ref>{{Cite web |last=Press |first=Gil |title=Is It An AI Chatbot Or A Human? 32% Can't Tell |url=https://www.forbes.com/sites/gilpress/2023/06/01/is-it-an-ai-chatbot-or-a-human-32-cant-tell/ |access-date=2 August 2023 |website=Forbes |language=en |archive-date=9 July 2024 |archive-url=https://web.archive.org/web/20240709171615/https://www.forbes.com/sites/gilpress/2023/06/01/is-it-an-ai-chatbot-or-a-human-32-cant-tell/ |url-status=live}}</ref>

== Conferences ==

=== Turing Colloquium ===

1990 marked the fortieth anniversary of the first publication of Turing's "Computing Machinery and Intelligence" paper, and saw renewed interest in the test. Two significant events occurred in that year: the first was the Turing Colloquium, which was held at the [[University of Sussex]] in April, and brought together academics and researchers from a wide variety of disciplines to discuss the Turing test in terms of its past, present, and future; the second was the formation of the annual [[Loebner Prize]] competition.

[[Blay Whitby]] lists four major turning points in the history of the Turing test&nbsp;– the publication of "Computing Machinery and Intelligence" in 1950, the announcement of [[Joseph Weizenbaum]]'s [[ELIZA]] in 1966, [[Kenneth Colby]]'s creation of [[PARRY]], which was first described in 1972, and the Turing Colloquium in 1990.{{sfn|Whitby|1996|p=53}}

=== 2008 AISB Symposium ===

In parallel to the 2008 [[Loebner Prize]] held at the [[University of Reading]],<ref>
{{citation |url=http://www.reading.ac.uk/cirg/loebner/cirg-loebner-main.asp |title=Loebner Prize 2008 |publisher=University of Reading |access-date=29 March 2009}}{{Dead link|date=July 2018 |bot=InternetArchiveBot |fix-attempted=no }}
</ref>
the [[Society for the Study of Artificial Intelligence and the Simulation of Behaviour]] (AISB), hosted a one-day symposium to discuss the Turing test, organised by [[John Barnden]], [[Mark Bishop]], [[Huma Shah]] and [[Kevin Warwick]].<ref>
{{citation |url=http://www.aisb.org.uk/events/turingevent.shtml |title=AISB 2008 Symposium on the Turing Test |publisher=Society for the Study of Artificial Intelligence and the Simulation of Behaviour |access-date=29 March 2009 |archive-url=https://web.archive.org/web/20090318025939/http://aisb.org.uk/events/turingevent.shtml |archive-date=18 March 2009}}
</ref>
The speakers included the Royal Institution's Director [[Susan Greenfield, Baroness Greenfield|Baroness Susan Greenfield]], [[Selmer Bringsjord]], Turing's biographer [[Andrew Hodges]], and consciousness scientist [[Owen Holland (academic)|Owen Holland]]. No agreement emerged for a canonical Turing test, though Bringsjord expressed that a sizeable prize would result in the Turing test being passed sooner.

== See also ==

{{Div col|colwidth=20em}}
* [[Artificial intelligence in fiction]]
* [[Chatbot]]
* [[ChatGPT]]
* [[Computer game bot Turing Test]]
* [[Explanation]]
* [[Explanatory gap]]
* [[Functionalism (philosophy of mind)|Functionalism]]
* [[Graphics Turing Test]]
* [[Hard problem of consciousness]]
* [[List of things named after Alan Turing]]
* [[Mark V. Shaney]] (Usenet bot)
* [[Mind-body problem]]
* [[Mirror neuron]]
* [[Natural language processing]]
* [[Philosophical zombie]]
* [[Problem of other minds]]
* [[Sentience]]
* [[Social bot]]
* [[Technological singularity]]
* [[Theory of mind]]
* [[Voight-Kampff machine]] (fictitious Turing test from ''Blade Runner'')
* [[Winograd Schema Challenge]]
{{Div col end}}

== Notes ==

{{reflist|23em}}

== References ==

{{Refbegin|30em}}
* {{Citation |last=Bion |first=W.S. |year=1979 |chapter=Making the best of a bad job |title=Clinical Seminars and Four Papers |publisher=Abingdon: Fleetwood Press.}}
* {{Citation |last=Boden |first=Margaret A. |year=2006 |author-link=Margaret A. Boden |title=Mind As Machine: A History of Cognitive Science |publisher=[[Oxford University Press]] |isbn=978-0-19-924144-6 |url-access=registration |url=https://archive.org/details/mindasmachinehis0001bode}}
* {{Citation |last1=Colby |first1=K. M. |last2=Hilf |first2=F. D. |last3=Weber |first3=S. |last4=Kraemer |year=1972 |title=Turing-like indistinguishability tests for the validation of a computer simulation of paranoid processes |journal=[[Artificial Intelligence (journal)|Artificial Intelligence]] |volume=3 |pages=199–221 |doi=10.1016/0004-3702(72)90049-5 |first4=H.}}
* {{Citation |last=Copeland |first=Jack |author-link=Jack Copeland |year=2003 |title=The Turing Test |work=The Turing Test: The Elusive Standard of Artificial Intelligence |editor-last=Moor |editor-first=James |publisher=Springer |isbn=978-1-4020-1205-1}}
* {{Citation |last=Crevier |first=Daniel |author-link=Daniel Crevier |year=1993 |title=AI: The Tumultuous Search for Artificial Intelligence |location=New York, NY |publisher=BasicBooks |isbn=978-0-465-02997-6}}
* {{Citation |last=Danziger |first=Shlomo |year=2022 |title=Intelligence as a Social Concept: a Socio-Technological Interpretation of the Turing Test |journal=Philosophy & Technology |volume=35 |issue=3 |page=68 |doi=10.1007/s13347-022-00561-z |s2cid=251000575}}
* {{cite book |last=Descartes |first=René |year=1996 |title=Discourse on Method and Meditations on First Philosophy |publisher=Yale University Press |location=New Haven & London |isbn=978-0-300-06772-9 |url-access=registration |url=https://archive.org/details/discourseonmetho0000desc_i4z8}}
* {{Citation |last=Diderot |first=D. |author-link=Denis Diderot |year=2007 |title=Pensees Philosophiques, Addition aux Pensees Philosophiques |publisher=[Flammarion] |isbn=978-2-0807-1249-3}}
* {{Citation |last=Dreyfus |first=Hubert |title=What Computers ''Still'' Can't Do |year=1979 |publisher=MIT Press |location=New York |author-link=Hubert Dreyfus |isbn=978-0-06-090613-9 |url=https://archive.org/details/whatcomputerscan00hube}}
* {{Citation |last=Feigenbaum |first=Edward A. |author-link=Edward Feigenbaum |year=2003 |title=Some challenges and grand challenges for computational intelligence |journal=Journal of the ACM |volume=50 |issue=1 |pages=32–40 |doi=10.1145/602382.602400 |s2cid=15379263}}
* {{Citation |last=French |first=Robert M. |author-link=Robert M. French |year=1990 |title=Subcognition and the Limits of the Turing Test |journal=Mind |volume=99 |pages=53–65 |issue=393 |doi=10.1093/mind/xcix.393.53 |s2cid=38063853}}
* {{Citation |last=Genova |first=J. |year=1994 |title=Turing's Sexual Guessing Game |journal=Social Epistemology |volume=8 |pages=314–326 |doi=10.1080/02691729408578758 |issue=4}}
* {{Citation |last=Harnad |first=Stevan |author-link=Stevan Harnad |year=2004 |contribution=The Annotation Game: On Turing (1950) on Computing, Machinery, and Intelligence |url=http://cogprints.org/3322/ |editor-last=Epstein |editor-first=Robert |editor2-last=Peters |editor2-first=Grace |title=The Turing Test Sourcebook: Philosophical and Methodological Issues in the Quest for the Thinking Computer |publisher=Klewer |access-date=17 December 2005 |archive-date=6 July 2011 |archive-url=https://web.archive.org/web/20110706102356/http://cogprints.org/3322/ |url-status=live}}
* {{Citation |last=Haugeland |first=John |year=1985 |title=Artificial Intelligence: The Very Idea |publisher=MIT Press |author-link=John Haugeland |location=Cambridge, Massachusetts}}.
* {{Citation |last1=Hayes |first1=Patrick |last2=Ford |first2=Kenneth |year=1995 |title=Turing Test Considered Harmful |journal=Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence (IJCAI95-1), Montreal, Quebec, Canada. |pages=972–997}}
* {{Citation |last=Heil |first=John |year=1998 |title=Philosophy of Mind: A Contemporary Introduction |location=London and New York |publisher=Routledge |isbn=978-0-415-13060-8}}
* {{Citation |last=Hinshelwood |first=R.D. |year=2001 |title=Group Mentality and Having a Mind: Reflections on Bion's work on groups and on psychosis}}
* {{Citation |last=Kurzweil |first=Ray |author-link=Ray Kurzweil |year=1990 |title=The Age of Intelligent Machines |isbn=978-0-262-61079-7 |publisher=MIT Press |location=Cambridge, Massachusetts}}
* {{Citation |last=Kurzweil |first=Ray |year=2005 |title=The Singularity is Near |publisher=Penguin Books |isbn=978-0-670-03384-3 |url=https://archive.org/details/singularityisnea00kurz}}
* {{Citation |last=Loebner |first=Hugh Gene |author-link=Hugh Loebner |title=In response |year=1994 |journal=[[Communications of the ACM]] |volume=37 |issue=6 |pages=79–82 |url=http://loebner.net/Prizef/In-response.html |access-date=22 March 2008 |doi=10.1145/175208.175218 |s2cid=38428377 |archive-date=14 March 2008 |archive-url=https://web.archive.org/web/20080314154148/http://loebner.net/Prizef/In-response.html}}
* {{McCorduck 2004}}
* {{Citation |editor-last=Moor |editor-first=James |year=2003 |title=The Turing Test: The Elusive Standard of Artificial Intelligence |isbn=978-1-4020-1205-1 |publisher=Kluwer Academic Publishers |location=Dordrecht}}
* {{Citation |last=Penrose |first=Roger |author-link=Roger Penrose |year=1989 |title=The Emperor's New Mind: Concerning Computers, Minds, and The Laws of Physics |publisher=Oxford University Press |isbn=978-0-14-014534-2 |title-link=The Emperor's New Mind|bibcode=1989esnm.book.....P }}
* {{Citation |last=Proudfoot |first=Diane |title=Rethinking Turing's Test |journal=[[The Journal of Philosophy]] |volume=110 |issue=7 |pages=391–411 |date=July 2013 |doi=10.5840/jphil2013110722 |jstor=43820781}}
* {{Russell Norvig 2003}}
* {{Citation |last1=Russell |first1=Stuart J. |last2=Norvig |first2=Peter |year=2010 |title=Artificial Intelligence: A Modern Approach |edition=3rd |location=Upper Saddle River, NJ |publisher=Prentice Hall |isbn=978-0-13-604259-4 |title-link=Artificial Intelligence: A Modern Approach}}
* {{citation |ref={{harvid|Saygin|2000}} |first1=A. P. |last1=Saygin |first2=I. |last2=Cicekli |first3=V. |last3=Akman |year=2000 |title=Turing Test: 50 Years Later |journal=Minds and Machines |volume=10 |issue=4 |pages=463–518 |url=http://crl.ucsd.edu/~saygin/papers/MMTT.pdf |doi=10.1023/A:1011288000451 |hdl=11693/24987 |s2cid=990084 |hdl-access=free |access-date=7 January 2004 |archive-date=9 April 2011 |archive-url=https://web.archive.org/web/20110409073501/http://crl.ucsd.edu/~saygin/papers/MMTT.pdf}}. Reprinted in {{harvtxt|Moor|2003|pp=23–78}}.
* {{citation |first1=A. P. |last1=Saygin |first2=I. |last2=Cicekli |year=2002 |title=Pragmatics in human-computer conversation |journal=Journal of Pragmatics |volume=34 |issue=3 |pages=227–258 |doi=10.1016/S0378-2166(02)80001-7 |citeseerx=10.1.1.12.7834}}.
* {{citation |last1=Saygin |first1=A.P. |year=2008 |contribution=Comments on "Computing Machinery and Intelligence" by Alan Turing |editor1-first=R. |editor1-last=Epstein |editor2-first=G. |editor2-last=Roberts |editor3-first=G. |editor3-last=Poland |publisher=Springer |location=Dordrecht, Netherlands |bibcode=2009pttt.book.....E |last2=Roberts |first2=Gary |last3=Beber |first3=Grace |title=Parsing the Turing Test: Philosophical and Methodological Issues in the Quest for the Thinking Computer |doi=10.1007/978-1-4020-6710-5 |isbn=978-1-4020-9624-2 |s2cid=60070108}}
* {{Citation |doi=10.1017/S0140525X00005756 |last=Searle |first=John |author-link=John Searle |year=1980 |url=http://members.aol.com/NeoNoetics/MindsBrainsPrograms.html |title=Minds, Brains and Programs |journal=Behavioral and Brain Sciences |volume=3 |issue=3 |pages=417–457 |s2cid=55303721 |access-date=19 March 2008 |archive-date=23 August 2000 |archive-url=https://web.archive.org/web/20000823030455/http://members.aol.com/NeoNoetics/MindsBrainsPrograms.html}}. ''Page numbers above refer to a standard [[pdf]] print of the article. See also Searle's [https://web.archive.org/web/20010221025515/http://www.bbsonline.org/Preprints/OldArchive/bbs.searle2.html original draft].''
* {{Citation |last1=Shah |first1=Huma |last2=Warwick |first2=Kevin |author2-link=Kevin Warwick |year=2009a |contribution=Emotion in the Turing Test: A Downward Trend for Machines in Recent Loebner Prizes |title=Handbook of Research on Synthetic Emotions and Sociable Robotics: New Applications in Affective Computing and Artificial Intelligence |editor1-last=Vallverdú |editor-first=Jordi |editor2-last=Casacuberta |editor2-first=David |publisher=Information Science, IGI |isbn=978-1-60566-354-8}}
* {{Citation |last1=Shah |first1=Huma |last2=Warwick |first2=Kevin |author2-link=Kevin Warwick |title=Testing Turing's five minutes, parallel-paired imitation game |journal=Kybernetes |volume=4 |issue=3 |pages=449–465 |date=April 2010a |doi=10.1108/03684921011036178}}
* {{Citation |last1=Shah |first1=Huma |last2=Warwick |first2=Kevin |author2-link=Kevin Warwick |title=Hidden Interlocutor Misidentification in Practical Turing Tests |journal=[[Minds and Machines]] |volume=20 |issue=3 |pages=441–454 |date=June 2010j |doi=10.1007/s11023-010-9219-6 |s2cid=34076187}}
* {{Citation |last=Shah |first=Huma |date=5 April 2011 |title=Turing's misunderstood imitation game and IBM's Watson success |url=https://www.academia.edu/474617 |url-access=registration |access-date=20 December 2017 |archive-date=10 February 2023 |archive-url=https://web.archive.org/web/20230210114350/https://www.academia.edu/474617 |url-status=live}}
* {{Citation |last=Shapiro |first=Stuart C. |title=The Turing Test and the economist |year=1992 |journal=ACM SIGART Bulletin |volume=3 |issue=4 |pages=10–11 |doi=10.1145/141420.141423 |s2cid=27079507 |doi-access=free}}
* {{Citation |last=Shieber |first=Stuart M. |title=Lessons from a Restricted Turing Test |year=1994 |journal=[[Communications of the ACM]] |volume=37 |issue=6 |pages=70–78 |url=http://www.eecs.harvard.edu/shieber/Biblio/Papers/loebner-rev-html/loebner-rev-html.html |access-date=25 March 2008 |doi=10.1145/175208.175217 |arxiv=cmp-lg/9404002 |citeseerx=10.1.1.54.3277 |bibcode=1994cmp.lg....4002S |s2cid=215823854 |archive-date=17 March 2008 |archive-url=https://web.archive.org/web/20080317130231/http://www.eecs.harvard.edu/shieber/Biblio/Papers/loebner-rev-html/loebner-rev-html.html |url-status=live}}
* {{Citation |last=Sterrett |first=S. G. |year=2000 |title=Turing's Two Test of Intelligence |journal=Minds and Machines |volume=10 |doi=10.1023/A:1011242120015 |page=541 |issue=4 |hdl=10057/10701 |s2cid=9600264 |hdl-access=free}} (reprinted in The Turing Test: The Elusive Standard of Artificial Intelligence edited by James H. Moor, Kluwer Academic 2003) {{ISBN|1-4020-1205-5}}
* {{Citation |last=Sundman |first=John |title=Artificial stupidity |journal=[[Salon.com]] |date=26 February 2003 |url=http://dir.salon.com/story/tech/feature/2003/02/26/loebner_part_one/index.html |access-date=22 March 2008 |archive-url=https://web.archive.org/web/20080307105346/http://dir.salon.com/story/tech/feature/2003/02/26/loebner_part_one/index.html |archive-date=7 March 2008}}
* {{Citation |last=Thomas |first=Peter J. |year=1995 |title=The Social and Interactional Dimensions of Human-Computer Interfaces |publisher=[[Cambridge University Press]] |isbn=978-0-521-45302-8 |url-access=registration |url=https://archive.org/details/socialinteractio0000unse_b0z8}}
* {{Citation |last=Swirski |first=Peter |year=2000 |title=Between Literature and Science: Poe, Lem, and Explorations in Aesthetics, Cognitive Science, and Literary Knowledge |publisher=[[McGill-Queen's University Press]] |isbn=978-0-7735-2078-3}}
* {{Citation |last=Traiger |first=Saul |year=2000 |title=Making the Right Identification in the Turing Test |journal=Minds and Machines |volume=10 |doi=10.1023/A:1011254505902 |page=561 |issue=4 |s2cid=2302024}} (reprinted in The Turing Test: The Elusive Standard of Artificial Intelligence edited by James H. Moor, Kluwer Academic 2003) {{ISBN|1-4020-1205-5}}
* {{Citation |last=Turing |first=Alan |author-link=Alan Turing |year=1948 |chapter=Machine Intelligence |title=The Essential Turing: The ideas that gave birth to the computer age |editor=Copeland, B. Jack |isbn=978-0-19-825080-7 |publisher=Oxford University Press |location=Oxford}}
* {{Turing 1950}}
* {{Citation |last=Turing |first=Alan |author-link=Alan Turing |year=1952 |chapter=Can Automatic Calculating Machines be Said to Think? |title=The Essential Turing: The ideas that gave birth to the computer age |editor=Copeland, B. Jack |isbn=978-0-19-825080-7 |publisher=Oxford University Press |location=Oxford}}
* {{Citation |last=Weizenbaum |first=Joseph |author-link=Joseph Weizenbaum |title=ELIZA – A Computer Program For the Study of Natural Language Communication Between Man And Machine |journal=[[Communications of the ACM]] |volume=9 |issue=1 |date=January 1966 |pages=36–45 |doi=10.1145/365153.365168 |s2cid=1896290 |doi-access=free}}
* {{Citation |last=Whitby |first=Blay |year=1996 |contribution=The Turing Test: AI's Biggest Blind Alley? |title=Machines and Thought: The Legacy of Alan Turing |volume=1 |editor=Millican, Peter |editor2=Clark, Andy |publisher=[[Oxford University Press]] |pages=53–62 |isbn=978-0-19-823876-8}}
* {{Citation |last1=Zylberberg |first1=A. |last2=Calot |first2=E. |year=2007 |title=Optimizing Lies in State Oriented Domains based on Genetic Algorithms |journal=Proceedings VI Ibero-American Symposium on Software Engineering |pages=11–18 |isbn=978-9972-2885-1-7}}
{{Refend}}

== Further reading ==

{{Refbegin}}
* {{citation |first=Paul R. |last=Cohen |year=2006 |url=http://www.aaai.org/ojs/index.php/aimagazine/article/download/1849/1747 |title='If Not Turing's Test, Then What? |journal=[[AI Magazine]] |volume=26 |issue=4 |access-date=17 June 2016 |archive-date=15 February 2017 |archive-url=https://web.archive.org/web/20170215021119/http://www.aaai.org/ojs/index.php/aimagazine/article/download/1849/1747 |url-status=live}}.
* [[Gary Marcus|Marcus, Gary]], "Am I Human?: Researchers need new ways to distinguish artificial intelligence from the natural kind", ''[[Scientific American]]'', vol. 316, no. 3 (March 2017), pp.&nbsp;58–63. ''Multiple'' tests of artificial-intelligence efficacy are needed because, "just as there is no single test of [[Athletics (physical culture)|athletic]] prowess, there cannot be one ultimate test of [[intelligence]]." One such test, a "Construction Challenge", would test perception and physical action—"two important elements of intelligent behavior that were entirely absent from the original Turing test." Another proposal has been to give machines the same standardized tests of science and other disciplines that schoolchildren take. A so far insuperable stumbling block to artificial intelligence is an incapacity for reliable [[disambiguation]]. "[V]irtually every sentence [that people generate] is [[ambiguity|ambiguous]], often in multiple ways." A prominent example is known as the "pronoun disambiguation problem": a machine has no way of determining to whom or what a [[pronoun]] in a sentence—such as "he", "she" or "it"—refers.
* {{Citation |last=Moor |first=James H. |year=2001 |title=The Status and Future of the Turing Test |journal=Minds and Machines |volume=11 |issue=1 |pages=77–93 |issn=0924-6495 |doi=10.1023/A:1011218925467 |s2cid=35233851 |url=http://portal.acm.org/citation.cfm?id=596908 |postscript=.}}
* [[Kevin Warwick|Warwick, Kevin]] and Shah, Huma (2016), "Turing's Imitation Game: Conversations with the Unknown", Cambridge University Press.
{{Refend}}

== External links ==

{{Commons category}}

* [http://www.turingtestopera.com The Turing Test – an Opera by Julian Wagstaff]
* [http://www.rmcybernetics.com/science/cybernetics/ai_turing_test.htm The Turing Test] – How accurate could the Turing test really be?
* {{cite SEP |url-id=turing-test |title=The Turing test}}
* [https://web.archive.org/web/20050512232753/http://crl.ucsd.edu/~saygin/papers/MMTT.pdf Turing Test: 50 Years Later] reviews a half-century of work on the Turing Test, from the vantage point of 2000.
* [http://www.longbets.org/1 Bet between Kapor and Kurzweil], including detailed justifications of their respective positions.
* [http://www.sussex.ac.uk/Users/blayw/tt.html Why The Turing Test is AI's Biggest Blind Alley] by Blay Witby
* [http://www.jabberwacky.com/ Jabberwacky.com] {{Webarchive|url=https://web.archive.org/web/20050411014336/http://jabberwacky.com/ |date=11 April 2005 }} An AI [[chatterbot]] that learns from and imitates humans
* New York Times essays on machine intelligence [https://web.archive.org/web/20040816143151/http://www.rci.rutgers.edu/~cfs/472_html/Intro/NYT_Intro/History/MachineIntelligence2.html part 1] and [https://web.archive.org/web/20040816143151/http://www.rci.rutgers.edu/~cfs/472_html/Intro/NYT_Intro/History/MachineIntelligence2.html part 2]
* {{Scientific American Frontiers | 2| 5|"The first ever (restricted) Turing test"}}
* [https://web.archive.org/web/20091018043414/http://csunplugged.org/turing-test Computer Science Unplugged teaching activity] for the Turing test.
* [[n:Computer professionals celebrate 10th birthday of A.L.I.C.E.|Wiki News:]] "Talk:Computer professionals celebrate 10th birthday of A.L.I.C.E".

{{Authority control}}

{{Alan Turing}}

{{DEFAULTSORT:Turing Test}}
[[Category:1950 in computing]]
[[Category:Alan Turing]]
[[Category:Computer-related introductions in 1950]]
[[Category:History of artificial intelligence]]
[[Category:Human–computer interaction]]
[[Category:Philosophy of artificial intelligence]]
[[Category:Turing tests| ]]