Philosophy of AI and Machine Intelligence

Before asking whether a machine can be intelligent, philosophy of AI has to confront the fact that 'intelligence' itself has no single agreed definition. Russell and Norvig organize historical definitions of AI into four camps that ask whether a system should think like a human, act like a human, think rationally (correctly, by the laws of logic), or act rationally (choosing the action expected to best achieve its goals given what it knows) — and argue that the field has increasingly converged on the last of these, rationality, as the most tractable and least anthropocentric standard (Russell & Norvig, AIMA, ch. 1). The Stanford Encyclopedia of Philosophy's entry on artificial intelligence frames this as a genuinely philosophical dispute rather than a settled technical matter: whether 'intelligence' should be judged by internal processes resembling human cognition, or purely by external, functional success at tasks, has direct consequences for how we evaluate any given AI system's claim to being intelligent (SEP, 'Artificial Intelligence').

Alan Turing sidestepped the definitional dispute entirely in his 1950 paper 'Computing Machinery and Intelligence' by replacing the question 'can machines think?' with an operational substitute he called the imitation game: if a machine's conversational responses cannot be reliably distinguished from a human's by a human judge, Turing argued, we should be willing to say the machine thinks, regardless of what is happening mechanically inside it (Turing, 1950). This reframing was philosophically significant precisely because it made intelligence a matter of observable behavior rather than of introspectible inner experience — a move the Stanford Encyclopedia's entry on the Turing Test situates within a longer debate over whether behavioral indistinguishability is sufficient evidence for mental properties, or merely evidence of very good mimicry (SEP, 'The Turing Test'). That same tension — behavior as evidence versus behavior as mere performance — resurfaces throughout this lesson and the next, and it is the reason philosophers distinguish sharply between claims about what a system can do and claims about what a system is.

Even a system that behaved perfectly rationally would still be bound by the mathematical limits of computation itself. The Church-Turing thesis holds that any function computable by an effective, mechanical procedure can be computed by a Turing machine, and — read in the other direction — that whatever a Turing machine provably cannot compute lies beyond the reach of any algorithm whatsoever, including any future AI system built on conventional computing hardware (SEP, 'The Church-Turing Thesis'). This is not a practical limitation to be engineered away with more data or compute; it establishes, for problems like the Entscheidungsproblem, permanent boundaries on what any algorithmic reasoner — human-built or otherwise — could in principle decide, and it is the reason philosophers of AI distinguish 'we haven't solved this yet' from 'this is provably unsolvable by any computational process.' Rationality and computability are therefore two independent constraints on any AI system: an agent can be perfectly rational relative to its knowledge and still be running on a substrate that is provably incapable of deciding certain classes of problem, no matter how it is designed.

A stronger claim than 'machines can behave intelligently' is that mental processes literally are computations — the computational theory of mind (CTM), which the Stanford Encyclopedia describes as holding that the mind is a computational system similar in important respects to a Turing machine, with core mental processes constituting computations over symbolic representations rather than merely being usefully modeled by them (SEP, 'The Computational Theory of Mind'). CTM has been foundational to cognitive science and AI alike: it licenses building formal, mechanistic models of reasoning, perception, and language on the assumption that doing so is not just a useful analogy but potentially a description of what minds, biological or artificial, actually are. If CTM is correct, then in principle nothing about being made of carbon rather than silicon is essential to having a mind — a conclusion the next two arguments in this lesson test from opposite directions, one attacking it and one extending it.

John Searle's 1980 Chinese Room argument was designed specifically to attack the strongest version of this claim. Searle imagines himself locked in a room, manipulating Chinese symbols purely by following an English rule book, producing responses indistinguishable from a fluent Chinese speaker's without understanding a word of Chinese; he concludes that syntactic symbol manipulation, however sophisticated, cannot by itself produce genuine semantic understanding, and therefore that a program passing the Turing Test need not actually think (SEP, 'The Chinese Room Argument'). The argument remains genuinely contested rather than resolved: the 'Systems Reply' holds that while Searle-in-the-room does not understand Chinese, the entire system (Searle plus rule book plus notebooks) might; the 'Robot Reply' holds that grounding the system in real sensorimotor interaction with the world could produce genuine understanding that a disembodied symbol-manipulator lacks. This debate has direct contemporary stakes, since large language models raise the identical question in a new form.

Functionalism offers a different route to the same destination. On this view, what makes something a mental state of a given type is not its physical substrate but the causal role it plays — the pattern of inputs, outputs, and relations to other mental states it participates in — which implies that a silicon system could in principle instantiate genuine mental states if it reproduced the right functional organization, independent of being made of neurons (SEP, 'Functionalism'). Whether functional equivalence is sufficient for consciousness, however, remains unsettled: the 'hard problem of consciousness' asks why any physical or computational process should be accompanied by subjective experience at all, and the Stanford Encyclopedia notes ongoing, unresolved disagreement between physicalists who treat this as a temporary explanatory gap and dualists who consider it insurmountable in principle (SEP, 'Consciousness'). This module treats machine consciousness as exactly that: an open question, not a settled finding — in contrast to the largely uncontested engineering fact that modern systems can produce fluent, task-competent behavior.