Origins of Alignment: Problems Before the Name

What did researchers understand about directing intelligent machines before “AI alignment” became a research agenda? Turing, Wiener and McCarthy addressed different problems: recognizing intelligence, communicating purposes and knowing the limits of a task. Parallax’s argument is that these distinctions help explain why capable performance alone cannot establish dependable delegation. The historical record supports particular arguments and some explicit research connections; the comparison across them is this Note’s interpretation.

On this page: Teaching and purpose · Limits of expertise · Objectives and guarantees · Modern research · Historical lesson

Teaching, performance and purpose

In Computing Machinery and Intelligence (1950), Alan Turing replaces the broad question of whether machines think with an imitation game involving written answers and an interrogator. Section 7 turns to educating a child-machine. Reward and punishment can change the likelihood of behavior, but Turing argues that they convey too little information to constitute the whole teaching process. Other communication channels could help a machine learn to follow symbolic orders. His reported teaching experiments were limited, and he did not regard them as successful.^[1]

For alignment, the useful distinction is between evidence of intelligent performance and the means of directing learned behavior. Turing’s teaching proposal makes communication central; it does not demonstrate that the resulting learner shares a teacher’s purposes. The comparison with Alignment Feedback is thematic, rather than evidence that contemporary preference learning descends directly from this passage.

Norbert Wiener’s Some Moral and Technical Consequences of Automation (1960) asks what happens when a machine acts on the purpose communicated to it. His warning concerns incomplete communication between human and mechanical agencies, especially when action is too fast or irreversible for a person to intervene. He also considers the difficulty of reassessing the human ends served by scientific development over much longer timescales. These are arguments about automation and control, supported partly through examples and literary analogies, rather than measured failure rates for modern AI.^[2]

Wiener’s distinction between the purpose supplied and the outcome desired resembles today’s Outer Alignment problem. His intervention argument also illuminates Meaningful Human Control: a nominal opportunity to intervene is of little use if information or time is missing. Those are Parallax’s comparisons. Their usefulness does not establish a continuous line of influence from cybernetics to every later safety method.

What an expert system leaves out

John McCarthy approaches delegation through representation. In Some Expert Systems Need Common Sense (1984), he examines MYCIN, an adviser for treating bacterial infections. Its vocabulary represents bacteria, tests and treatments while leaving out much of the surrounding medical world. McCarthy’s cholera illustration shows why eliminating an infection can be an inadequate treatment recommendation; it is an example in his argument, not a documented clinical injury.^[3]

The contrary point matters: McCarthy says useful systems can operate without general common sense when a competent user supplies the missing context. He leaves unresolved whether MYCIN itself needs common sense. Dependable delegation might therefore require a richer system model, a capable physician who understands its limits, or both. Narrow competence is neither automatically useless nor sufficient for autonomous responsibility.^[3]

Here there is a documented connection within a research program: McCarthy explicitly relates his 1984 position to his earlier common-sense work. His Making Robots Conscious of Their Mental States (1995) again invokes that program and proposes representing what a robot knows, including when it needs external information. It presents preliminary logical machinery and says systems with the proposed abilities do not yet exist. This functional account of self-knowledge establishes neither subjective experience nor deployment safety.^[3][4]

Making objectives mathematically explicit

The 1996 Reinforcement Learning: A Survey describes learning through interaction with an environment and a scalar reward signal, with different criteria for evaluating long-term performance. It formalizes a learning problem; the reward criterion does not by itself determine what people ought to want.^[5]

The 1999 Policy invariance under reward transformations: Theory and application to reward shaping makes a narrower question precise: when can added rewards help guide learning while preserving optimal policies? Its potential-based shaping result depends on the stated Markov decision-process and discounting conditions. Preserving the original objective is different from establishing that it represents the intended outcome.^[6] This distinction gives a technical form to the separation between effective optimization and desirable purposes; it is not evidence that the theorem was derived from Wiener’s warning.

Bill Joy’s 2000 essay Why the Future Doesn’t Need Us instead argues for relinquishing dangerous capabilities in robotics, genetics and nanotechnology, with particular concern about self-replication.^[7] Whether to develop or delegate a capability is a public choice as well as an engineering question. Joy’s proposed response and forecasts remain an argument to assess, rather than an experimentally established consequence of learning theory.

From broad concerns to research methods

Research Priorities for Robust and Beneficial Artificial Intelligence (2015) distinguishes verification of formal requirements, validity of the requirements and resulting behavior, security against manipulation, and control after operation begins.^[8] Concrete Problems in AI Safety (2016) then organizes accident risks around wrong objectives, expensive supervision and failures during learning, including exploration and distributional shift. Its practical research agenda does not depend on invoking an extreme future scenario.^[9] Together these entries show how different concerns can become distinct research questions, rather than one undifferentiated demand for safety.

Capability and supervision also require different evidence. Attention Is All You Need (2017) proposes the Transformer to reduce sequential computation and evaluates translation and parsing performance. Its eight authors’ architecture contribution does not establish aligned goals.^[10] Deep reinforcement learning from human preferences (2017) instead learns a reward from human comparisons and optimizes it in Atari and simulated locomotion tasks. Its introduction explicitly cites the 2016 accident-risk agenda when motivating objective misspecification: that is an identifiable research connection.^[11]

Learning to summarize from human feedback (2020) cites the 2017 preference-learning work and related language-model studies while adapting reward learning to summarization. It reports improvements by selected human evaluators, alongside limits of optimizing a learned proxy.^[12] These citations document connections between particular studies. They support a more specific history than inferring that today’s systems implement Turing’s child-machine or McCarthy’s logical introspection. Key Papers: A Reading Guide develops the modern research routes; Alignment Feedback examines whose judgments supervision captures.

What carries forward

Technical supervision leaves a question about institutional authority. The June 4, 2024 Right to Warn letter proposes reporting channels and protections against retaliation for people raising concerns about advanced AI.^[13] Its proposal addresses who can surface risk information, rather than validating a model-control technique. In Parallax’s interpretation, the older questions about purposes and intervention now have to be asked of organizations as well as machines.

The historical lesson is a set of separations that remains useful: intelligent performance from desirable action; a communicated reward from an intended purpose; expertise within a model from awareness of what the model omits; a formal guarantee from a reason to delegate. Remedies changed from proposed education and logical representation to reward learning, evaluations and reporting arrangements. Each still needs evidence appropriate to its claim. History helps identify the question a method answers—and the decisions it leaves to people.