Creative Minds Like Ours? Large Language Models and the Creative Aspect of Language Use

2.1 The Cartesian Observations

Chomsky’s (1966, 2009a) most extensive treatment of CALU is in his Cartesian Linguistics. In it, he traces the observations bound up in CALU to Descartes who argued that a feature of human behavior distinguishing them from non-human animals and machines is how the former use their language. This distinction has roots in what Descartes proposed as an exception to the “mechanical philosophy”—the idea that all natural phenomena could be explained in reference to their constituent parts and physical contact (Cohen, 1985, pp. 153–154). Language use escapes, Descartes argues, mechanical explanation, making it a useful test for the existence of a “mind like ours” (Chomsky, 1988, p. 5). In his Discourse on Method, Descartes lays out this “language test”² to be applied to subjects that appear human:

Descartes’ test proposes that a machine possesses a mind like ours if it: utters words in an intelligible manner (“competent to us”); expresses a thought through variously arranged words of its own volition (“to declare” them); and expresses such thoughts through words in a manner that is appropriate to the remarks it has heard from others (“appositely to reply to what is said in its presence”).³ This behavior is exhibited by humans of the “lowest grade of intellect”—it is a common human ability.

Cordemoy (1668, pp. 4–6) extends Descartes’ formulation of the language test, arguing that the distinction between humans and mechanical beings lies in the relationship between words and thought; specifically, the “facilness of pronouncing Words” (Cordemoy, 1668, p. 13) is reflective of thought (Cordemoy, 1668, pp. 9–13). The key, then, is not the mere production of words but how words and other signs are used:

It is not unreasonable to believe that a machine possesses a mind like our own if it makes signs that do not arise in necessary connection with its environment; it makes signs that complement and correspond with the signs that the human interlocutor uses to express their thoughts; it communicates a novel idea that nonetheless corresponds with the human’s existing thoughts; and the signs build upon those of the human.

2.2 The Cartesian Observations in Biolinguistics

Chomsky’s approach to the Cartesian observations is to incorporate them in a biolinguistic framework in which human creativity, evidenced most strikingly in ordinary language use, is both enabled and constrained by rule-based properties of the mind conceived as an organic phenomenon (Smith, 2004, pp. 184–185). CALU’s role in the biolinguistic framework is thus different than its Cartesian function: rather than a test for the existence of other minds, CALU is an exercise of species-specific intellectual freedom (den Ouden, 1975, p. 15; see also, Land, 1974, pp. 16–17).⁴ From the Cartesian writings, three features of ordinary human language use are extracted:

2.3 The Scientific Intractability of CALU

As noted, CALU was split by generative linguists for the sake of scientific tractability. CALU appears to fall outside the bounds of determinacy and randomness, which mark the ends of a spectrum of possible scientific reasoning. CALU is a behavior that is not determined (stimulus-free), can take an undefined form (unbounded), yet it corresponds with the mental states of others who may have no direct knowledge of the speaker’s experience (appropriate); it is neither determined nor random yet it is appropriate. CALU is thus not typically considered amenable to scientific explanation in biolinguistics (see Chomsky, 2009b, p. 200). The implication is that humans are forced to simply “stare in puzzlement at…expression of thought that is coherent and appropriate but uncaused…” (Chomsky, 1995, pp. 38–39), perhaps indefinitely.⁶ CALU, taken in its entirety, is scientifically intractable.

The reasoning typically employed is premised on scientific inquiry as an exercise of human cognition, with some problems remaining incomprehensible owing to “our peculiar cognitive makeup…it does not indicate any objective profundity or divine design” (McGinn, 1993, p. 87). If the human mind is unable to make sense of this tripartite behavior—unable to offer some account of the phenomenon that deepens our understanding of it—then any science the human mind constructs will be similarly limited. The generative approach remains consistent with this disposition towards science up to the present, in which the starting assumption of any scientific discipline is “that the world can be understood” (Fox & Katzir, 2024, p. 72).

This is a guiding point—generative linguists did not, armed with the tools of post-Turing computability theory, aim to resolve the question of why individuals use their language in a particular way in any arbitrary instance. Rather, the characterization of a computational system that enables the free use of language became the target of inquiry, in this way segmenting the underlying mechanisms of language off from the use of these mechanisms in ordinary life. This point guides our analysis because the scientific intractability of CALU establishes a route to an internalist orientation and a competence-performance distinction in the study of language, among other crucial theoretical maneuvers including the postulation of a generative language faculty. LLM-driven arguments simply neglect CALU, the distinctive challenge it poses to a science of language, and its implications for theory-construction.

2.4 CALU’s Theoretical Significance and LLMs

CALU is “the central fact to which any significant linguistic theory must address itself” (Chomsky, 1964, p. 7). I argue this significance is demonstrated in theory-construction in three ways:

In contrast, LLMs fail to deal with CALU on each count:

With that, we turn to the status of LLMs in linguistics.

3.1 What Is a Large Language Model?

It is worthwhile to clarify what we mean by “Large Language Models.” The most prominent types of LLMs are “Generative Pre-Trained Transformers,” or GPTs, built from the Transformer architecture (Vaswani et al., 2017). The increasing successes of GPT-1 (Radford et al., 2018), GPT-2 (Radford et al., 2019), and GPT-3 led to an effort to build a system that can generalize beyond its training data by scaling up its internal capacity and training dataset (Brown et al., 2020, pp. 3–10). The effectiveness of this technique compelled researchers to build dialog agents—models that can engage in specialized conversational tasks, like Google’s LaMDA family of Transformer-based dialog models (Thoppilan et al., 2022). In November 2022, OpenAI (2022) released ChatGPT: a conversational agent underpinned by an LLM (a modified version of GPT-3). ChatGPT-4, a significantly larger model, was released in March 2023 (OpenAI, 2023).

The process of building an LLM begins with the construction of a training dataset consisting primarily of text-based data. This text-data is inputted into the models via “tokens,” which represent words or parts of words—the model “reads” words as tokens. During “pretraining,” LLMs are given an objective: often, it is to predict the next token based on a specific number of previous tokens. The model then attempts to match the predicted token against occurrences in its training data and sends this feedback through the model to update its internal parameters—a process called “backpropagation” (see generally, Mahowald et al., 2024, p. 522). Importantly, LLMs are statistical models in that they are models of the “statistical distribution of tokens in the vast public corpus of human-generated text” (Shanahan, 2023, p. 2). Asking an LLM a question is effectively the same as saying, “Here’s a fragment of text. Tell me how this fragment might go on. According to your model of the statistics of human language, what words are likely to come next” (Shanahan, 2023, p. 2)?

An LLM is not necessarily a chatbot. Moreover, when a user interacts with a chatbot like ChatGPT, they are not interacting with the LLM. In conversational agents, the LLM is “embedded in a larger system to manage the turn-taking in the dialogue” (Shanahan, 2023, p. 4). Moreover, the system must also “be coaxed into producing conversation-like behavior. Recall that an LLM simply generates sequences of words that are statistically likely follow-ons from a given prompt” (Shanahan, 2023, p. 4). LLMs are the base of a more complex system. The conversational behavior of systems like ChatGPT is the result of a technique for aligning language models’ output with human expectations called Reinforcement Learning from Human Feedback (RLHF) (Ouyang et al., 2022).⁷

For our purposes, the following terminology will be employed to make a simple distinction: when referring to LLMs embedded in conversational systems, the term “LLM-powered chatbot” will be used; otherwise, the term “LLM” will be used.

3.2 LLMs in Linguistic Theory

The use of deep neural networks to intervene in linguistic debates both pre- and post-dates the ChatGPT enthusiasm (e.g., Lakretz et al., 2021; Warstadt et al., 2019; Wilcox et al., 2024). What follows is a survey of more recent research debating the proper role of LLMs in linguistics. The works selected are relevant to our subsequent discussion of CALU and LLMs.

Contreras Kallens et al. (2023) argue that Transformer-based LLMs fulfill the promise of earlier connectionist models: they provide an “existence proof that the ability to produce grammatical language can be learned from exposure alone without language-specific computations or representations” (Contreras Kallens et al., 2023, p. 2). On this view, LLMs demonstrate that only domain-general mechanisms are required for human-like language acquisition, contra nativists who posit domain-specific mechanisms. They qualify their view, importantly, to note that they are not claiming LLMs are language users nor that they understand language—they are not users because they lack cognitive capacities supporting social interaction. Their central claim is that LLMs’ grammatical output is theoretically significant for cognitive science (Contreras Kallens et al., 2023, p. 3). The authors frame LLMs as a “usage-based answer to the [Poverty of Stimulus] argument” in which humans ‘memorize, abstract, and generalize’ linguistic data in language acquisition and its use (Contreras Kallens et al., 2023, p. 3).

More wide-ranging research by Mahowald et al. (2024) argues for a distinction between formal linguistic competence and functional linguistic competence—the former refers to “the knowledge of rules and statistical regularities of language” whereas the latter refers to “the ability to use language in real-world situations” (Mahowald et al., 2024, p. 518). They argue that LLMs do acquire the abstract rules of human language to a significant extent (e.g., hierarchical structure and abstraction)—thereby serving as viable tools in the study of human language acquisition and processing. The authors further argue that LLMs do not achieve functional linguistic competence as this is done “in tandem with non-language-specific capacities in real-world circumstances” (Mahowald et al., 2024, p. 519).

Importantly, the formal-functional distinction is justified because “language is robustly dissociated from the rest of high-level cognition” (Mahowald et al., 2024, p. 520), and a ‘good at language -> good at thought’ and ‘bad at thought -> bad at language’ fallacy leads the study of LLMs’ linguistic (and non-linguistic) abilities astray (Mahowald et al., 2024, pp. 517–519). Thus, formal linguistic competence is not sufficient for real-world usage, and modular architectures that integrate language with additional systems are required, much like the human brain/mind (Mahowald et al., 2024, pp. 522–535).

Moreover, Piantadosi (2023) argues “that language models should be treated as bona fide linguistic theories” (Piantadosi, 2023, p. 7).⁸ These theories “develop representations of key structures and dependencies” (Piantadosi, 2023, p. 7) and successfully integrate syntax and (some) semantics “without incorporating any of Chomsky’s key methodological claims, like…competence vs. performance, respect “minimality” or “perfection,” and avoid relying on the statistical patterns of unanalyzed data” (Piantadosi, 2023, p. 15). Piantadosi (2023, pp. 26–28) also argues that the success of LLMs at capturing human-like linguistic output refutes (in their capacity as theories) the “Galilean method” employed by Chomsky; specifically, they refute the search for underlying principles at the expense of broad data coverage.

A flurry of responses emerged to Piantadosi. Katzir (2023) argues that ChatGPT-4 generates different acceptability judgments about sentences in English, fails to distinguish between linguistic competence and performance, and lacks a distinction between likelihood and grammaticality, among other disanalogies that together indicate LLMs are “not suitably biased” to acquire human grammatical knowledge (Katzir, 2023, p. 3). Milway echoes this by arguing that LLMs’ learning is less constrained than that of humans (Milway, 2023). This point about (un)constrained learning is taken up in greater detail by Kodner et al. (2023) who argue that Piantadosi’s use of LLMs to explain human language acquisition neglects the original purpose of the poverty of stimulus argument: namely, “children generalize from their limited input in specific ways, navigating a constrained space of possible natural language grammars” (Kodner et al., 2023, p. 2).

Dupre (2024) argues computational modeling idealizes human language learning too much. The theoretical usefulness of LLMs depends on them being sufficiently analogous to human learners. For proponents of computational modeling, the generation of strings is (or assumed to be) generated by an underlying rule or constraint system (i.e., a grammar) (Dupre, 2024, pp. 1–3). But, in generative linguistics, a grammar “describes not the set of public symbols produced or producible by competent speakers, but the laws governing the language-specialized mental faculty underlying such usage” (Dupre, 2024, p. 3). The expression of a hierarchically structured mental representation recruits greater cognitive activity than just the language faculty (Dupre, 2024, pp. 3–4)—meaning the adequacy of computational modeling depends on ‘factoring out’ the myriad causal sources that exist in addition to language that comprise a learner’s dataset (e.g., communicative goals, externalization systems, etc.).

This highlights the extent to which computational models are perhaps too idealized—that a grammar can be learned by these models from data generated by the grammar is not tantamount to showing that it can be learned from the data of myriad systems (Dupre, 2024, p. 5). The “disunified” process of human language acquisition—contra Piantadosi’s singular, data-driven process—indicates the following:

The implication is that computational models, like LLMs, may not be adequately modeling human language acquisition.

Beyond responses to Piantadosi, additional work systematically tests three GPTs’ response stability regarding the grammaticality of prompts, finding that all models exhibit “marginal overall above-chance accuracy and absence of response stability” (Dentella et al., 2023, p. 6). These results are interpreted to indicate that the models simultaneously appear to master the form of language but do not produce the level of accuracy and stability in grammaticality judgments that should result from this mastery (Dentella et al., 2023, pp. 6–8). Hu et al. (2024) responded with re-evaluations, arguing that Dentella et al.’s tasks differed between models and humans, thereby negatively affecting the former’s results (Hu et al., 2024, p. 3). Leivada, Dentella, and Günther (2024, pp. 2–3) responded, arguing that in the absence of systematic testing that captures the underlying reasoning used in specific tasks by humans and LLMs, an inference from an LLM’s human-level performance to its possession of a human-like competence is invalid.

Finally, Moro et al. (2023, p. 84) argue that LLMs can acquire “impossible languages” and “possible languages” with equal facility, a sharp disanalogy with humans who can only deal with the former as a puzzle rather than as a grammatical structure (Chomsky & Moro, 2022, pp. 21–22). To be sure, the claim of equal facility may be exaggerated, but the disanalogy remains real. Kallini et al. (2024) trained GPT-2 models to test whether they can acquire impossible languages. They find that models trained on possible languages learn more efficiently; the models “prefer natural grammar rules”;⁹ and models develop human-like solutions to non-human grammatical patterns (Kallini et al., 2024, p. 2). The authors conclude that the models “do not master our set of synthetic impossible languages as well as natural ones…” (Kallini et al., 2024, p. 9). While the authors frame these results favorably for the use of LLMs as models of human language, that GPT-2 models can acquire impossible languages, albeit less efficiently than possible ones, is a disanalogy with human linguistic competence.

An explicit discussion of CALU is conspicuously absent in this literature (for a brief exception, see Moro et al., 2023, p. 83). Yet, the absence of CALU in the invocation of LLMs in linguistic theory is odd, indicating that a full account of what constitutes ordinary human language use is missing. This itself has significant downstream effects on explanatory theory-construction, to which we now turn.

4.1 Stimulus-Controlled

LLMs are controlled by a combination of internal and external stimuli in predictable ways; they are not stimulus-free (Moro et al., 2023, p. 83). A Transformer-based LLM is (typically) designed to predict the next token based on a specified number of previous tokens. When provided with an input, the model predictably provides a likely continuation based on the statistical probabilities associated with said tokens. While the model is stochastic (Bender et al., 2021, pp. 616–617), this does not imply stimulus-freedom as its operations are determined. Three facts hold steady: (1) The model will generate an output value; (2) The output value generated will be a likely continuation of the input value; (3) The model will not decide, absent internal or external instructions to the contrary, to not generate an output value, to alter the process of token generation, or to initiate its own conversations.

Of course, an LLM can be provided, for example, with internal programming instructions that do lead the model to not generate an output based strictly on the most likely continuation of an input. The model can be instructed to sometimes select the second- or third-likeliest next-token. The basic point remains: the model is bound to stimuli.

In LLM-powered chatbots, the situation is fundamentally the same: while the base LLM is not directly queried by end-users, the model responsible for filtering inputs to the base LLM is controlled by the programming instructions set forth by humans while the base LLM is bound by the inputs (external stimuli) it is fed. Stimulus-freedom does not emerge either from the size of the base LLM or the complexity of a conversational system.

LLMs and LLM-powered chatbots are stimulus-controlled in highly specific and predictable ways; they are impelled to act in certain ways, but never inclined.

4.2 The Turing Test Objection

One objection is that LLMs are language users because they pass Turing’s (1950) “Imitation Game” in which a human must judge whether their anonymous interlocutor is a human or a machine. Should the machine be judged a human (a sufficient number of times), then it passes the test. Indeed, there is evidence that Turing was influenced by Descartes’ language test¹⁰ in the Imitation Game’s design (see Abramson, 2011), at which point one might suggest passing Turing’s test amounts to passing Descartes’ test. Recent experimental results indicate the former is a live possibility (see Jones & Bergen, 2024).

Even if we grant that GPT-4 and comparable LLMs (or LLM-powered chatbots) passed the Turing Test, this is not tantamount to passing the Cartesian test. Pulman (2018) argues that the Turing Test is considerably more limited than the Cartesian language test, referencing in part the questioning-answering format typical of the Imitation Game.¹¹ This format’s limitations go deeper, not least of which is that the anonymous machine has no choice but to participate in the exchange (i.e., it is stimulus-controlled from start to finish).

McGilvray argues that even if a machine does pass Turing’s test, “no fact of the matter has been determined; no scientific issues resolved. The test offers no evidence in favor of a specific science of mind, and it does not show that the mind works the way the computer that passes the test does” (McGilvray, 2009, p. 114). Passage of the test simply offers one reason why one may begin to start using the language of ‘thinking machines’ (McGilvray, 2009, p. 114). Chomsky (2009c) observes, tying these strands together, that the steady progression of science following Newton and the collapse of the mechanical philosophy meant that Turing’s Imitation Game was formulated within “an entirely different framework” than the Cartesian tests for other minds (Chomsky, 2009c, p. 105). Turing’s ambitions were lower than the Cartesians’, not aiming at scientific objectivity about possession of a human-like mind (or intelligence).

The lowered expectations of Turing’s test bring to light fundamental problems in testing for CALU. The description of stimulus-free, unbounded, yet appropriate behavior is the result of observations made first and foremost of human beings; human behavior is the baseline. It is not that the basis for such an ability is, in principle, not replicable on silicon substrates. The problem, in its most abstract formulation, is that any human invention that is routinely subject to the stage-setting familiar of contemporary Turing tests or bounded by the direction of a human to produce the outputs in which we search for Cartesian creativity makes such a search nearly self-defeating. It is not self-defeating in principle, as it is possible to imagine a machine that, once developed, acts autonomously in ways that do not require the prompting familiar to LLMs or programming instructions and manual patching to keep its unbounded outputs tethered to human reality.¹² But the fact that human understanding does not appear capable of penetrating CALU contributes to the dilemmas in testing for it.¹³

Yet, nothing stated here contradicts our foregoing analysis of LLMs and their chatbots in relation to CALU’s three components. Passing the Turing Test, as it is contemporaneously conceived, would not necessarily indicate that they exhibit Cartesian creativity. Our conclusion that LLMs are not language users remains.

4.3 Internalism and the Competence-Performance Distinction

Humans specifically are language users. Being clear that a “language user” exhibits CALU is therefore fundamental to a scientific account of human language; this description informs subsequent inquiry. This owes to the importance of identifying a mind capable of species-specific intellectual freedom. Yet, neglected in the observation that LLMs are not language users by Contreras Kallens et al. (2023)—and the superimposition of a usage-based approach—is that the characteristics of human language use justify an internalist orientation and a competence-performance distinction in its study.

The post-Turing split of the Cartesian problem of language use into a matter of characterizing a computational system and the use of this computational system—with the latter relegated to the domain of near-mystery—should not give the misleading impression that CALU is no longer operative in theory-construction. For Chomsky, and implicit in the foundations of the biolinguistic enterprise, the “creative aspect observations, along with the poverty of stimulus observations, offer a set of facts with which his and—he holds—any science of language must contend” (McGilvray, 2001, p. 5). CALU, recall, refers to language use that is neither determined nor random yet it is appropriate; uncaused but appropriate expression of thought. The way these facts are dealt with is not to explain actual language use, as this is “a scientifically intractable aspect of the world” (Asoulin, 2013, p. 235). Any theory that attempts to explain uncaused but appropriate language use yields “no scientifically interesting regularities” (Asoulin, 2013, p. 240).

Rather, it is precisely in the formation of an explanatory theory that aims to make part of this problem scientifically tractable. Thus, an internalist science of language that aims at the mechanisms that make creative language use possible—but not the use itself—is justified (Asoulin, 2013, pp. 241–242; see also McGilvray, 2009, p. 2). From this, a distinction between linguistic competence (the mechanisms) and linguistic performance (the use) is derived. This distinction only makes sense in a scientific framework if one can meaningfully engage with the idea that a finite biological system (i.e., the human brain) can yield infinite outputs; that “[i]t is possible to invent a single machine which can be used to compute any computable sequence” (Turing, 1937, p. 241). As a naturalistic inquiry, the biolinguistic approach forgoes Descartes’ attribution of the capacity for infinite linguistic generativity to the possession of a “spiritual entity” (see Riskin, 2016, p. 63).

The superimposition of a usage-based approach onto LLMs to signify their theoretical importance is sharply limited if it fails to establish this baseline definition of what counts as a language user. An internalist orientation and a competence-performance distinction are justified in reference to the phenomenon that does exhibit CALU—humans—and it is unclear why LLMs’ output (however grammatical it appears to humans) should change this. That said, if LLMs did reproduce CALU, this would mean that they are the only other (known) phenomenon that is capable of this behavior and it would buttress their use in the study of human language. However, this would not settle the matter of how to conduct a science of human language (see Section 4.1), not least of all because humans are biological phenomena whose creative language use may be enabled by a different set of underlying capacities.

5.1 LLMs-As-Theories and Data Coverage

CALU offers a set of facts that justify an internalist orientation in the study of language and a competence-performance distinction. Implicit in this reasoning is a lowering of expectations for what a science of language can accomplish in the face of CALU’s intractability. As a result, biolinguists are inclined to abstract away from ordinary language use and idealize “the” language faculty—a disposition embedded in the “Galilean” method. We thus turn to Piantadosi’s argument that LLMs are theories of human language that serve as a “refutation of the “Galilean” method” (Piantadosi, 2023, p. 27) without reference to methodological moves like the competence-performance distinction.

Piantadosi’s argument is this: many natural systems exhibit high-level behaviors that may be surprising if one only extrapolates from the underlying rules of the systems—this phenomenon is known as emergence (e.g., individual traders in a stock market follow simple rules, yet the market-level effects “are the emergent result of millions of aggregate decisions”, Piantadosi, 2023, p. 27). Coupled with the fact that some systems are massively complex, the only viable way to test principles that explain their behavior is through simulation—and computational tools are best suited to this task. LLMs are simulations of human language, indicating that their grammatical and semantically coherent outputs are emergent results of explanatory value. Such simulations show the futility of the pursuit of underlying principles at the expense of data coverage (a pursuit characterized as the Galilean method, Piantadosi, 2023, pp. 26–28).

Piantadosi neglects a critical relationship between CALU and the Galilean method to which we now turn.

5.2 CALU and The Galilean Method

Chomsky’s earliest extended invocation of the ‘Galilean style’ is in his 1980 Rules and Representations. It is preceded by Chomsky’s (1980, pp. 6–7) articulation of the distinction between “problems” and “mysteries”—those issues where the human intellect can make meaningful progress and those that exceed its grasp, respectively (see Collins, 2021 on this distinction). In tandem, Chomsky articulates the Cartesian view “that we can profitably study motivation, contingencies that guide action, drives and instinct…But…the freedom to choose remains, and remains inexplicable in these (or any) terms” (Chomsky, 1980, p. 8). Chomsky’s view is that CALU—which falls into the “freedom to choose”—is closer to a “mystery” than a “problem.”

Directly after these remarks Chomsky invokes the “Galilean style,” citing theoretical physicist Steven Weinberg, deeming the style to be a “narrower version” of the Cartesian thesis that human action cannot be meaningfully studied, but its influences and underlying mechanisms can be studied (Chomsky, 1980, p. 8). This analogy bears on the roles of abstraction and lowered expectations in science. Weinberg’s (1976) description of the Galilean style emphasizes the abstraction of ordinary phenomena, in the process distancing oneself from commonsense ideas about them (as is routine in physics). Chomsky is thus analogizing the Cartesian distinction between body and soul (i.e., mind)—only the “body” can be understood—and a science of physics that lowers its expectations for understanding natural phenomena.¹⁴ He suggests that physics may be “a remarkable historical accident resulting from chance convergence of biological properties of the human mind with some aspect of the real world” (Chomsky, 1980, p. 9).

As Section 4.2 argues, the intractability of CALU places limits on a science of language, in this way complementing the Galilean lowering of expectations for scientific understanding. In the biolinguistic framework, ordinary language use is segmented off from its underlying competence out of a recognition that biological properties of the human mind do not appear to attain a ‘chance convergence’ with this aspect of reality. This is why Chomsky deems the Galilean style to be a “narrower version” of the Cartesian thesis on free human action (Chomsky, 1980, p. 8)—it is a complementary base of inquiry from which a competence-performance distinction is derived, though never losing sight of the internalist orientation of linguistic science.¹⁵

Free linguistic behavior can be made partly scientifically tractable by studying its underlying mechanisms, dependent on theoretical accommodations including competence-performance and a willingness to abstract away from linguistic performance to an idealized competence. As Collins explains, Galileo sought to ‘decompose’ natural phenomena to diminish the influence of interaction effects “in the search of formally precise invariances” (Collins, 2023, p. 5). Many factors contribute to language use; the Galilean disposition toward idealization “is a way of isolating an invariant phenomenon from general factors…” (Collins, 2023, p. 6). Adhering to this, Chomsky abstracts away from the brain’s performance systems (Rey, 2020, p. 20), thereby conceiving of language as it would be in the absence of interaction effects. This reflects not an attitude towards data per se, but an attitude towards abstraction which informs data-selection (Collins, 2023, pp. 2–3). Piantadosi, in conceiving of LLMs-as-theories that refute the search for underlying principles at the expense of broad data coverage, misses this step and the analogy with CALU.

5.3 LLMs-As-Theories Re-Evaluated

LLMs are not language users because they do not replicate CALU. As a result, unless one imposes theoretical tenets onto the model, LLMs do not by themselves identify, explain, or shed useful theoretical light on CALU in humans. If LLMs are theories of human language, on what basis do they not deal with its creative uses? On what basis do they eschew a competence-performance distinction and the abstraction to language’s enabling mechanisms? On what basis is language use scientifically tractable from the standpoint of broad data coverage? It is not clear that the statistical association of broad data offers any explanatory insight into CALU or aids the construction of a theory that makes it partly tractable.

This does not rule out the use of LLMs in linguistics, albeit in more limited respects. LLMs may demonstrate that there are more regularities in natural language use than our intuitions would lead us to expect. Yet, LLMs cannot lay claim to the fundamental facts about creative language use that can and should inform theory-construction—and already do within the biolinguistic approach.

One objection is that we are promoting a Galilean disposition toward idealization while also having accepted above, as Dupre (2024) argues, that computational modeling idealizes too much. The objection points us in a fruitful direction: language use pairs sound (or sign) with meaning and thus interfaces with other cognitive systems. Idealization, then, cannot ignore this non-linguistic cognitive activity. This raises questions about cognitive architecture, to which we now turn.

6.1 Unboundedness and the Language Faculty

The biolinguistic conception of a generative language faculty has roots in the intertwined history of modern Universal Grammar (UG) and computability theory. The language faculty is assumed to exist under UG, reflecting the history in which UG was “revived without awareness in the generative enterprise” with the new conceptual tools provided by computability theory (Chomsky, 2021, p. 9). Language, under UG, is a natural object characterized by recursive enumerability which yields digital infinity (Mendívil-Giró, 2018, p. 861). The postulation of a computational system that recursively produces grammatical sentences over an indefinite range is made on the basis that a brain with finite memory cannot list all possible grammatical sentences (Mendívil-Giró, 2018, pp. 873–874). The term recursion in modern UG’s early days was directly inspired by mathematical logic, making “recursive…equivalent to computable” (Mendívil-Giró, 2018, p. 874). Indeed, the Minimalist Program’s “Merge” operation preserves recursion as a property of a computational system (Mendívil-Giró, 2018, pp. 874–875).

Huybregts (2019) echoes this reasoning and relates it to the derivation of the competence-performance distinction, where competence is conceived as a computational system:

Thus, a distinction between linguistic competence and performance is the null hypothesis, not vice versa (Huybregts, 2019, p. 4). Huybregts’ broader argument—that infinite generativity cannot be reached via a stepwise approach—is relevant to the postulation of a competence characterized by a computational system, though one made only after the Cartesian problem is reformulated in the biolinguistic framework. The generative language faculty enables the unboundedness of human language use.

6.2 Stimulus-Freedom and the Language Faculty

Stimulus-freedom raises its own problem: How does language ‘regulate’ the generation of form-meaning pairs in the absence of identifiable stimuli? This problem goes beyond simplistic input-output processing: it is the problem of ‘free access’ to language in which linguistic resources can be recruited effectively at will (Collins, 2004, pp. 519–520). Crucially, the biolinguistic framework does not attempt to solve this problem, but to identify the cognitive mechanism(s) that—at least in part—enable this free access.

Based on our foregoing analysis, the answer is internal—the enabling mechanism will not be found in the individual’s environment, but rather ‘inside the head.’ Language’s cognitive basis must possess a degree of operational autonomy (McGilvray, 2005, p. 222). Specifically, whatever enables stimulus-freedom must fulfill two responsibilities: (1) the production of structured expressions with a discrete set of cognitive resources; and (2) interfacing with other, relevant cognitive systems. McGilvray explains that a modular language faculty is required:

The emphasis is on the modularity of a generative language faculty; a system in possession of “conceptual resources” that can make these resources “legible” to others in the mind (McGilvray, 2005, p. 218). The modularity of the language faculty merely indicates that this system can be studied independently of others for the sake of explanatory theory, in this case with its own domain-specific resources (see McGilvray, 2014, p. 235), though “as part of a broader investigation of its interactions with other such systems…” (Chomsky, 2013, p. 35).

One could eschew the role of a domain-specific language faculty in favor of, say, the interactivity of domain-general mechanisms. However, if CALU depended merely on domain-general mechanisms, then one should expect to find this phenomenon in the purposeful behaviors of non-human animals. Yet, we do not, making this conclusion dubious (Baker, 2007, pp. 239–240).

6.3 Appropriateness, Abduction, and Modularity

To get at its cognitive basis, Baker (2007) poses CALU as a poverty of stimulus problem. To acquire CALU, a child would have to learn that its parents are not automata because they use their language in a stimulus-free, unbounded, and appropriate manner. The child would then have to learn that they too can use language this way. The third step is where the effort at empirical analysis appears to collapse: the child would then have to learn “how not to be an automaton—how to develop the capacity to use language in this way” (Baker, 2007, p. 241). This cannot even be framed properly, as “we have no precise algorithmic way to specify the knowledge that this capacity depends on or the processes that it involves” and therefore “we cannot estimate the amount of information that is involved” (Baker, 2007, p. 241). Baker (2007, pp. 241–245) goes on to argue that the more basic problem of determining whether those around the child are automata is not learnable, either. CALU, then, is plausibly innate.

Baker’s (2007, pp. 239–240) (tentative) conclusion about its cognitive basis is notable: that CALU is a module of the human mind¹⁸—though, not localized in a particular brain region (Baker, 2011, pp. 90–91). Baker comes to a parallel conclusion: that the CALU module cannot be characterized computationally. This conclusion turns on the observation that “the whole notion of “appropriate” is an abductive one. We judge that what someone says to us is appropriate not at all on the basis of the syntactic structure of what is said, but entirely on the semantic properties of what is said” (Baker, 2007, p. 253). Baker’s (2007, pp. 252–253) implication, drawing from Fodor (2000), is that it makes sense CALU has not (as of writing) been replicated via a computer program, as abductive reasoning seems beyond the scope of computation.

6.4 LLMs, CALU, and Architecture

Using LLMs as models of human language with parallels to cognitive architecture runs into two problems. First, attempts to leverage LLMs as models of human language risk “[reversing] the nature of the argument” (Leivada, Dentella, & Günther, 2024, p. 3). Any scientific account of human language must deal with CALU. Yet, arguing that LLMs are useful models (or theories) of human linguistic competence without incorporating the Cartesian observations is to start from an incomplete point of inquiry. Biolinguistics recognizes the Cartesian problem, splits it for the sake of scientific tractability, and uses the tools that enable this split—in part, computability theory—to postulate an innate computational system that makes possible creative language use. To conceive of LLMs as mastering the abstract structure of natural language is entirely premature as they offer no conception of the problem whatsoever. Even for Mahowald et al. (2024) to ground their formal-functional distinction in neuroscientific data about their respective mechanisms ignores the insight that “[d]ata only have meaning within a theoretical framework…If the framework is deficient, the interpretation of the data will be insufficient, too” (Mahlmann, 2023, p. 346).

Second, the attempt to attain functional competence in LLMs (or future AI systems) with comparison to human language use is similarly deficient in its foundations. As Section 5.2 illustrates, there is a deliberate lowering of expectations in the biolinguistic approach out of a recognition of CALU’s scientific intractability. Discussions of LLMs’ functional competence understate the seriousness of the problem of language use, hardly recognizing—if at all—its creative character. As a result, science is given full steam to study problems it may be unable to penetrate, or seriously unlikely barring theoretical accommodations that LLMs may not permit or indicate (e.g., competence-performance, idealization of the language faculty, etc.).

Consider how Transformer-based LLMs do not replicate CALU—they do not replicate its stimulus-freedom, appropriateness, and the strong interpretation of unboundedness. This plausibly rules out the Transformer architecture in the pursuit of functional competence, as achieving these would require changes of a fundamental nature. In principle, however, the mechanisms that enable, at least, stimulus-freedom and unboundedness are computable and can thus be reproduced via a different architecture. This would likely require architectural modularity—explicit building of modules.

Appropriateness is a trickier matter. In Section 2.2, we identified that appropriateness is not merely functional and the stimulus-controlled nature of LLM-powered chatbots, fine-tuned via RLHF, is purely functional. This could plausibly be resolved, though not by a system with a “conception” of appropriateness provided via RLHF, as this is premised on a functional system/end-user dynamic. How a system could be designed to achieve this is a more difficult question. RLHF is a data-driven process, relying on structured and explicit examples of human-generated data of what counts as (in)appropriate outputs—and this is not even sufficient. Yet, the only way to scientifically make sense of CALU is internalism; the cognitive mechanisms that enable it. We have no theory that provides a list of all the circumstances and associated utterances that are appropriate, as CALU indicates such a theory may be a fool’s errand (see McGilvray, 2001, pp. 8–18).

We are forced back into the mind. Baker (2007) argued that interpreting and producing appropriate expressions appears to be an abductive process and modular at that. Research on representing abduction computationally has progressed. However, computational tractability—an efficient algorithmic solution—remains a roadblock (see van Rooij et al., 2019, ch. 12). Human brains possess limited computational power, indicating that brute force is not the key to tractability (see Blokpoel et al., 2018, p. 2). A solution, then, is likely to require explicit building of modules into a system whose architecture is radically different than that of a Transformer-based system.

All this assumes that computation alone supports CALU. This assumption has its doubters (e.g., Baker, 2007, pp. 252–253). It is not clear how computation can direct itself, so to speak, in a manner that escapes the bounds of the input it receives. Here, we get to the heart of the biolinguistic approach to CALU, the value of splitting the problem for the sake of scientific tractability, and the usefulness of lowering scientific expectations. We have been exploring each component of CALU individually; stimulus-freedom, unboundedness, and appropriateness. But CALU does not exist in this carved-up fashion—language use simply is free. When we break it down in this way we are doing so as part of an exercise to theoretically accommodate it. It is not clear if—and how—the mechanisms for each component being computable would collectively give rise to CALU in a computational model (in this way making it, presumably, something more than just a model). The split of the Cartesian problem is a theoretical accommodation; the cognitive mechanisms that make CALU possible are a necessary but perhaps not sufficient condition for CALU to arise. CALU arises under some configuration of these three components, but there is no logical certainty that they are sufficient (see Chomsky, 1982, p. 433). This raises doubts about the possibility of replicating CALU on a computational device, architectural or emergent modularity notwithstanding.

All this illustrates the extent to which Mahowald et al. (2024) overstate the success of LLMs and what they forebode. The dissociation of language and thought has significant negative effects on explanatory theory-construction, chiefly in the deficient characterization of ordinary human language use, the lack of construction of a theoretical framework to make CALU partly tractable, and a failure to lay claim to a cognitive architecture that enables—but does not explain—free language use. Whereas some criticize biolinguistics from the standpoint of computational modeling for exhibiting a ‘contempt for applications’ of theoretical work (Pullum, 2009, p. 17), the approach recognizes the difference between explanation and application; it recognizes the scale and distinctiveness of the problem of CALU and the need to make part of it scientifically tractable.