An Information for Industry, Inc. Publication VISTAS IN INFORMATION HANDLING Volume I THE AUGMENTATION OF MAN’S INTELLECT BY MACHINE General Editor PAUL W. HOWERTON Vice-President, Information for Industry, Inc. Associate Editor DAVID C. WEEKS Assistant General Manager, Communication Sciences Division Information for Industry, Inc. 1963 Washington, D. C. SPARTAN BOOKS London CLEAVER-HUME PRESS Also published in Computer Supported Cooperative Work: A Book of Readings, Irene Greif [Editor], Morgan Kaufmann Publishers, Inc., San Mateo, CA, 1988, pp. 107-126. Also published in Organization and Groupware, T. Nishigaki [Editor], NtT Publishing, 1992._ Copyright © 1963 by Information for Industry, Inc. Library of Congress Catalog Card No. 63-14865 Printed in the United States of America. All rights reserved. Sole distributors in Great Britain, the British Commonwealth and the Continent of Europe: CLEAVER-HUME PRESS 10-15 St. Martins Street London W. C. 2 11 PREFACE The high and formal discussions of learned men end oftentimes m drsputes about Ji n«n«, with which (.ccordihg to the »» .nd wisdom »f ^ maticians) it would be more prudent to begin, and so by means of definitions reduce them to order.—Sir Francis Bacon, Aphorisms. There are many fine treatises on the various applications of the new machine tools for the extension of man’s ability to digest and understand great volumes of data issuing from the research efforts of t^ world. It is the purpose of this new series, Vistas in Information Handling, to provide a forum for discussion of advanced thinking on the augmentation of man’s intellect by these machines. As editors of this series, we shall strive to report not only this advanced thinking but also the application of older techniques to new purposes. If differences of scientific opinion need to be aired, let these differences be explicated by their proponents in full debate. We welcome such ^"^Tb employ Francis Bacon’s phrase quoted above, much “high and formal discussion’’ frequently involves one or more of the transcendent subdisciplines of the communication sciences. We have, therefore, intro¬ duced our series with a collection of reports which start with a concept for augmenting man’s intellect; we then examine natural language the means by which this augmentation is symbolized. , , #• We have invited discussion of associative memory, of mo e s or structural analysis of natural language, and of empirical results o * language data processing. Because illustration is always useful m under- * < staLng advanced concepts, we have included the subject of literature processing in a discipline (chemistry), including solutions to problems inherent in Japanese scientific and technical literature. Finally, in the belief that application of known methods of information handling to new purposes is a principal objective of science, we have included papers on the development of critical scientific data (to try to seek an answer to the question: “How critical is critical? ), and on t e integration of engineering data processing with the automatization of ^%ur plan is to turn out two volumes a year, one dealing with “software and advanced intellectual concept, the other with advanced design o equipment. iii IV PREFACE With these goals in mind, we believe the Vistas in Inf ot mat ion Handling will extend beyond the frontiers of present operative experi¬ ence, while at the same time maintaining a pragmatic balance between concept and reality. The authors of the nine papers in this book are too well known in their respective fields to require restatement of their qualifications. Their thoughts and works speak for themselves. We gratefully acknowledge the encouragement of our colleagues in Information for Industry, Incorporated. Special mention should be made of the faith and perspicacity of Mr. Robert Teitler of Spartan Books, who conceived the idea for this series. February, 1963 Paul W. Howerton, General Editor David C. Weeks, Associate Editor CONTENTS 1 A Conceptual Framework for the Augmentation of Man’s ^ Intellect by Douglas C. Engelbart, Stanford Research Insti¬ tute, Menlo Park, California. Man is confronted by increasingly complex problem situations which require use of great stores of data and experience together with artifacts of simplest to most sophisticated electronic aids. His intellect, then, must be extended so that comprehension is better and more quickly achieved with concomitant reaching of solutions to these complex problem situations. This opening chapter provides a conceptual framework for the augmentation of man’s intellect. 2 Linear Associative Information Retrieval by Vincent E. Giuliano and Paul E. Jones, Arthur D. Little, Inc., Cambridge, Massachusetts. The authors are concerned here with the recognition and exploitation of word-term associations for the retrieval of documentary materials to support the research or decision-making function of an organism. A general theory of association and associative retrieval is based on the use of linear transformations, both for establishing associations among terms and for discriminating among documents. The theory is illustrated by a simple experimental device. 3 Dimensioning in an Associative Memory by Benjamin F. Cheydleur, The Philco Corporation, Computer Division, Willow Grove, Pennsylvania. 55 A construct for associative memory multi-linkaging that is feasible for ordinary computer memory permits progressively precise entry into the region. A maximum of nine memory cycles (nine micro¬ seconds) is required for lexemes involving fewer than four polygrams with one microsecond per polygram per item needed for establishment of the association of concept. The general adaptability of any such construct is an inverse function of the degree of redundancy in the natural computer coding of items and descriptors. V VI contents 4 Structural Models for Linguistic Automation by Win fred P. Lehmann and Eugene Pendergraft, Linguistics Research Center, The University of Texas, Austin, Texas. ^ Because written communication between humans involves combi¬ nations of symbols which may be words or ideographs, the logical interaction of these copibinations must convey a concept. Since all known languages have underlying similarities, it is entirely reasonable to assume that structural models of modular form can be developed to explicate the linguistic laws. Efforts have been made to accomplish this purpose since antiquity, but only recently have the tools been available to make full-scale comparisons between language structures. 5 Automatic Translation and Language Data Processing by Leon E. Dostert, Georgetown University, Washington, D. C.. . 92 The fact, which must be accepted whether palatable or not, that “perfect” translation of writings in one natural lanpage to another is not humanly or mechanically achievable. Work in the automatic translation of such materials has been underway for ten years. We are now ready for a critical review by one of the real pioneers in the field of the state-of-the art to determine courses of action for further research. Standards of acceptance vary widely, but the outputs by operating systems are useful. Implications of automatic translation research in information handling are becoming more evident, as semantic and syntactic laws are validated. 6 Homographs: Their Classification and Identification by Milos Pacak, Georgetown University, Washington, D. C. Ill Word-forms which are spelled alike but have entirely different meanings and may even have different syntactic functions are called homographs. Classification of word-forms in terms of traditional grammar does not seem to be suitable for the purposes of automatic language data processing. With exemplars from Russian-English machine translation experience, a system of classification and identi¬ fication for machine handling of homographs is described. 7 Problems of the Chemical Literature and Some Solutions bv Herman Skolnik, Hercules Powder Company, Wilmington, Delaware. For the augmentation of man’s intellect new professional classifica¬ tions have come into being in the scientific disciplines—the literature scientist. The chemical profession has been the most vigorous in developing such skills and training. The extension of the chemist’s capability to comprehend the growing literature of a discipline which is becoming more and more fragmental can have application in other sciences. CONTENTS Vll 8 Problems Inherent in the Handling of Scientific and Technical Information in Japan by Toshikazu Takayanagi, Japan Information Center of Science and Technology, Tokyo, Japan. Language symbolism which has come down over the centuries can of its.inherent nature create problem areas for augmenting man’s use of the compilations of data and experience to solve his complex situations. The Japanese language is a case-in-point. How the Japan Information Center of Science and Technology has met and resolved some of these difficulties can be useful in analogous situations elsewhere. 9 The Production of Critical Scientific Data by Robert J. Howerton, Lawrence Radiation Laboratory, the University of California, Livermore, California'.. . 192 The limitations of measuring devices and the decisions concerning the best possible values for physical data have been the burden of those scientists who develop critical data. Critical data are those data to which critical evaluation has been applied by qualified com¬ pilers. But “how critical is critical?” The author suggests a means of curtailing the volume of scientific publication by establishing stores of the data-type reporting to which experimentalists may turn with certainty of getting best possible values. 10 Toward Integration of Engineering Data Processing and Automatization of Design by Borje Langefors, Svenska Aeroplan Aktiebolaget (SAAB) Linkoping, Sweden. 201 All of the efforts to augment man’s intellect are of little practical value unless from time to time the systems resulting from these efforts can be applied. The work in Scandanavia toward using automatic data-processing systems as an extension of engineering design is significant. Although design is a highly creative work, the designer must have means of reducing established principles and data to his application. This aspect of design effort is well suited to automation. CHAPTER 1 A CONCEPTUAL FRAMEWORK FOR THE augmentation of MAN’S INTELLECT* Douglas C. Engelbart Stanford Research Institute, Menlo Park, Calif. INTRODUCTION By “augmenting man’s intellect” we mean increasing the capability of a man to approach a complex problem situation, gain comprehension to suit his particular needs, and to derive solutions to problems. Increased capability in this respect is taken to mean a mixture of the following: that comprehension can be gained more quickly; that better compre¬ hension can be gained; that a useful degree of comprehension can be gained where previously the situation was too complex; that solutions can be produced more quickly; that better solutions can be produced; that solutions can be found where previously the human could find none. And by “complex situations” we include the professional problems of diplomats, executives, social scientists, life scientists, physical scientists, attorneys, designers—whether the problem situation exists for twenty minutes or twenty years. We do not speak of isolated clever tricks that help in particular situations. We refer to a way of life in an integrated domain where hunches, cut-and-try, intangibles, and the human “feel for a situation” usefully coexist with powerful concepts, streamlined terminology and notation, sophisticated methods, and high-powered electronic aids. This paper covers the first phase of a program aimed at developing means to augment the human intellect. These methods or devices can include many things, all of which appear to be but extensions of those developed and used in the past to help man apply his native sensory, mental, and motor capabilities. We consider the total system of a human * Partial support of this work was received from the Air Force Office of Scientific Research, Directorate of Information Sciences, under Contract AF 49(638)-!024. This paper is an excerpt from a summary report (AFOSR-3223) which also includes a discussion of background material, a hypothetical description of a computer-based augmentation system, and research recommendations. The paper could never have been written without the considerable effort of Mrs. Rowena Swanson of the AFOSR, who extracted and organized the material upon which this chapter is based. 1 2 VISTAS IN INFORMATION HANDLING plus his augmentation devices and techniques as a proper field of search for practical possibilities. This field constitutes a very important system in our society; like most systems its performance can best be improved by considering the whole as a set of interacting elements rather than a number of isolated components. This kind of system approach to human intellectual effectiveness does not find a ready-made conceptual framework such as exists for estab¬ lished disciplines. Before a research program can pursue such an approach intelligently, so as to derive practical benefits within a reasonable time in addition to results of long-range significance, a conceptual framework must be searched out—a framework that provides orientation as to the important factors of the system, the relationships among these factors, the types of change among the system factors that offer likely improve¬ ments in performance, and the kind of research goals and methodology that seem promising. Man’s population and gross product are increasing at a considerable rate, but the complexity of his problems grows even faster. And the urgency with which solutions must be found becomes steadily greater in response to the increased rate of activity and the increasingly global nature of that activity. Augmenting man’s intellect, in the sense defined above, would warrant the full-time efforts of an enlightened society if its leaders could be shown a reasonable approach and some plausible benefits. OBJECTIVE OF THE STUDY The objective of this study is to develop a conceptual framework for a coordinated research and development program whose goals would be the following: (1) to find the factors that limit the effectiveness of the individual’s basic information-handling capabilities in meeting the various needs of society for problem solving in its most general sense; and (2) to develop new techniques, procedures, and systems that will better adapt these basic capabilities to the needs, problems, and progress of society. We have established the following specifications for this frame¬ work : (1) It must provide perspective for both long-range basic research and research that will yield immediate practical results. (2) It must indicate what this augmentation will actually involve in the way of changes in working environment, thinking, skills, and methods of work. (3) It must be a basis for evaluating and assimilating the possibly relevant work and knowledge of existing fields. (4) It must reveal areas where research is possible and indicate ways to assess the research; must be a basis for choosing starting points and developing appropriate methodologies for the needed research. framework for the augmentation of man’s intellect 3 Two points need emphasis here. First, although a conceptual frame¬ work has been constructed, it is still rudimentary. Further search and actual research are needed for the evolution of the framework. Second, even with a basic framework, an apparently small modification can significantly alter the results of the framework. The framework must therefore be viewed as tentative, and not considered as a detailed prediction or a collection of factual statements. CONCEPTUAL FRAMEWORK A. GENERAL The conceptual framework we seek must orient us toward the real possibilities and problems associated with using modern technology to give direct aid to an individual in comprehending complex situations, isolating the significant factors, and solving problems. To gain this orientation, we examine how individuals achieve their present level of effectiveness, and expect that this examination will reveal possibilities for improvement. The entire effect of an individual on the world stems essentially from what he can communicate to the world through his limited motor channels. This communication, in turn, is based on information received from the outside world through his limited sensory channels; on infor¬ mation, drives, and needs generated within him; and on his processing of that information. His processing is of two kinds: that which he is generally conscious of (recognizing patterns, remembering, visualizing, abstracting, deducing, inducing, etc.), and that involving self-generated information, unconscious processing and mediating of received infor¬ mation, and mediating of conscious processing itself. The individual does not use this information or processing to grapple directly with the sort of complex situation in which we seek to give him help. He uses his innate capabilities in a rather indirect fashion, since the situation is generally too complex to yield directly to his motor actions, and always too complex to yield comprehensions and solutions from direct sensory inspection and use of basic cognitive capabilities. For instance, an aborigine who possesses all of our basic sensoty-mental- motor capabilities but does not possess our background of indirect knowledge and procedure cannot organize the proper direct actions necessary to drive a car through traffic, request a book from the library, call a committee meeting to discuss a tentative plan, call someone on the telephone, or compose a letter on the typewriter. Our culture has evolved means for us to organize and utilize our basic capabilities so that we can comprehend truly complex situations and accomplish the processes of devising and implementing problem solutions. VISTAS IN INFORMATION HANDLING The ways in which human capabilities are thus extended are here called augmentation means, and we define four basic classes of them: 1. ^rt//acfs—physical objects designed to provide for human comfort, the manipulation of things or materials, and the manipu¬ lation of symbols. 2. Language—the way in which the individual classifies the picture of his world into the concepts that his mind uses to model that world, and the symbols that he attaches to those concepts and uses in consciously manipulating the concepts (“thinking”). 3. Methodology the methods, procedures, and strategies with which an individual organizes his goal-centeted (problem-solving) activity. 4. Training—the conditioning needed by the individual to bring his skills in using augmentation means 1, 2, and 3 to the point where they are operationally effective. The system we wish to improve can thus be visualized as comprising a trained human being together with his artifacts, language, and method¬ ology. The explicit new system we contemplate will involve as artifacts computers and computer-controlled information-storage, information¬ handling, and information-display devices. The aspects of the conceptual framework that are discussed here are primarily those relating to the individual’s ability to make significant use of such equipment in an integrated system. Pervading all of the augmentation means is a particular structure or organization. While an untrained aborigine cannot drive a car through traffic because he cannot leap the gap between his cultural background and the kind of world that contains cars and traffic, it is possible for him to move step by step through an organized training program that will enable him to drive effectively and safely. In other words, the human mind neither learns nor acts by large leaps, but by a series of small steps so organized or structured that each one depends upon previous steps. Although the size of the step a human being can take in comprehension, innovation, or execution is small in comparison to the over-all size of the step needed to solve a complex problem, human beings nevertheless do solve complex problems. It is the augmentation means that serve to subdivide a large problem in such a way that the human being can walk through It m little steps. The structure or organization of these little steps or actions we designate as process hierarchies. Every thought process or action is composed of subprocesses. Such subprocesses include making a pencil stroke, writing a memo, or devising a plan. An appreciable number of discrete muscle movements must be coordinated to make a pencil stroke. Similarly, making particular pencil framework for the augmentation of man’s INTELLECT 5 strokes and composing a memo are complex processes in themselves which are subprocesses to the over-all writing of the memo. Although every subprocess is a process in its own right in that it consists of further subprocesses, there is no advantage here in isolating the ultimate “bottom” of the process-hierarchical structure. There may be no way of determining whether the apparent “bottom” (processes that cannot be further subdivided) exist in the physical world or in the i limitations of human understanding. In any case, it is not necessary to begin from the “bottom” in discussing particular. process hierarchies. No person uses a completely unique process every time he performs a new task. Instead, he begins from a group of basic, sensory-mental-motor process capabilities, and adds to these certain of the process capabilities of his artifacts. There are only a finite number of such basic human and artifact capabilities from which to draw. Moreover, even quite different higher-order processes may have in common relatively high-order subprocesses. When a person writes a memo (a reasonably high-order process), he makes use of many processes as subprocesses that are common to other high-order processes. For example, he makes use of planning, composing, dictating. The process of writing a memo is utilized as a subprocess within many different processes of a still higher order, such as organizing a committee, changing a policy, and so on. It is likely that each individual develops a certain repertory of process capabilities from which he selects and adapts those that will compose the processes that he executes. This repertory is like a tool kit. Just as the mechanic must know what his tools can do and how to use them, so the intellectual worker must know the capabilities of his tools and have suitable methods, strategies, and rules of thumb for making use of them. All of the process capabilities in the individual’s repertory rest ultimately on basic capabilities within him or his artifacts, and the entire repertory represents an integrated, hierarchical structure (which we often call the repertory hierarchy). We find three general categories of process capabilities within a typical individual’s repertory: (1) those executed completely within the human integument, which we call explicit-human process capabilities; (2) those possessed by artifacts for executing processes without human intervention, which we call explicit-artifact process capabilities; and (3) those we call the composite process capabilities, which are derived from hierarchies containing both of the other kinds. We assume that it is our H-LAM/T system (Human using Language, Artifacts, and Methodology, in which he is Trained) that performs a process in any instance of use of this repertory. Let us consider the process of issuing a memorandum. There is a particular concept associ¬ ated with this process—that of putting information into a formal package 6 VISTAS IN INFORMATION HANDLING 7 and distributing it to a set of people for a certain kind of consideration. That the type of information package associated with this concept has been giv^ the special name of memorandum shows the denominating effect of this process on the system language. The memo-writing process may be executed by using a set of process capabilities (intermixed or repetitive form) such as planning, developing subject matter, composing text, producing hard copy, and distributing. There is a definite way in which these subprocesses are organized that represents part of the system methodology. Each of these subprocesses represents a functional concept that must be a part of the system language if it is to be organized effectively into the human's way of doing things, and the symbolic portrayal of each concept must be such that the human can work with it and remember it. If the memo is short and simple, the first three processes may be of the explicit-human type (i.e., the memo may be planned, developed and composed within the mind), and the last two of the composite type. If it is complex, involving a good deal of careful planning and development, then all of the subprocesses may be of the composite type (at least including the use of pencil and paper artifacts), and there may be many different applications of some of the process capabilities within the total process (successive drafts, revised plans). Executing the above-listed set of subprocesses in proper sequence represents an execution of the memo-writing process. However, the very process of organizing and supervising the utilization of these subprocess capabilities is itself a most important subprocess of the memo-writing process. Hence the subprocess capabilities as listed would not be complete without the addition of a seventh, which we call the execuiive capability. This is the capability stemming from habit, strategy, rules of thumb, prejudice, learned method, intuition, unconscious dictates, or combina¬ tions thereof, to utilize the appropriate subprocess capabilities in a particular sequence and timing. An executive process (i.e., the exercise of an executive capability) involves such subprocesses as planning, selecting, and supervising; it is within the executive processes that the methodology in the H-LAM/T system is embodied. To illustrate the capability-hierarchy features of our conceptual framework, let the reader consider an artifact innovation appearing directly within the relatively low-order capability for composing and modifying written text, and see how this can afifect his hierarchy of capabilities. Suppose you had a new writing machine^—a high-speed electric typewriter with some very special features. You can operate its keyboard to cause it to write text much as with a conventional type¬ writer. But the printing mechanism is more complicated; besides printing a visible character at every stroke, it adds special encoding features by means of invisible selective components in the ink and special shaping of framework for the AUGMENTATION OF MAN’S INTELLECT As an auxiliary device, there is a gadget that is held like a pencil and, instead of a point, has a special sensing mechanism which can be moved along a line of the special printing from your writing machine (or one like it). The signals which this reading stylus sends through the flexible connecting wire to the writing machine are used to determine which characters are being sensed, thus causing the automatic typing of a duplicate string of characters. An information-storage mechanism in the writing machine permits you to sweep the reading stylus over the characters much faster than the writer can type; the writer will catch up with you when you stop to think about what word or string of words should be duplicated next, or while you reposition the straightedge guide along which you run the stylus. This hypothetical writing machine thus permits you to use a new process of composing text. For instance, trial drafts can rapidly be composed from rearranged excerpts of old drafts, together with new words or passages which you insert by hand typing. Your first draft may represent a free outpouring of thoughts in any order, with the inspection of foregoing thoughts continuously stimulating new considerations and ideas to be entered. If the tangle of thoughts represented by the draft becomes too complex, you can compile a reordered draft quickly. It would be practical for you to accommodate more complexity in the trails of thought you might build in search of the path that suits your needs. You can integrate your new ideas more easily, and thus harness your creativity more continuously, if you can quickly and flexibly change your working record. If it is easier to update any part of your working record to accommodate new developments in thought or circumstance, you will find it easier to incorporate more complex procedures in your way of doing things. This will probably allow you, for example, to accommodate the extra burden associated with keeping and using special files whose contents are both contributed to and utilized by any current work in a flexible manner—which in turn enables you to devise and use even more complex procedures to better harness your talents in your particular working situation. The important thing to appreciate here is that a direct new innovation in one particular capability can have far-reaching effects throughout the rest of your capability hierarchy. A change can propagate up through the capability hierarchy, higher-order capabilities that can utilize the initially changed capability can now reorganize to take special advantage of this change and of the intermediate higher-capability changes. A change can propagate down through the hierarchy as a result of new capabilities at the high level and modification possibilities latent in lower levels. These latent capabilities may have been previously unusable in the hierarchy and become usable because of the new capability at the higher level. _ _i_ 8 VISTAS IN INFORMATION HANDLING The writing machine and its flexible copying capability would occupy you for a long time if you tried to exhaust the reverberating chain of associated possibilities for making useful innovations within your capability hierarchy. This one innovation could trigger a rather extensive redesign of this hierarchy; your method of accomplishing many of your tasks would change considerably. Indeed, this process characterizes the sort of evolution that our intellect-augmentation means have been undergoing since the first human brain appeared. For our objective of deriving orientation about possibilities for actively pursuing an increase in human intellectual effectiveness, it is important to realize that we must be prepared to pursue such new-possibility chains throughout the entire capability hierarchy (calling for a “system^' approach). It is also important to realize that we must be oriented to the synthesis of new capabilities from reorganization of other capabilities, both old and new, that exist throughout the hierarchy (a “system¬ engineering” approach). B. THE BASIC PERSPECTIVE Individuals who operate effectively in our culture have already been considerably “augmented.” Basic human capabilities for sensing stimuli, performing numerous mental operations, and communicating with the outside world are put to work in our society within a system—an H- LAM/T system—the individual augmented by the language, artifacts, and methodology in which he is trained. Furthermore, we suspect that improving the effectiveness of the individual as he operates in our society should be approached as a system-engineering problem—that is, the H-LAM/T system should be studied as an interacting whole from a synthesis-oriented approach. This view of the system as an interacting whole is strongly bolstered by considering the repertory hierarchy of process capabilities that is structured from the basic ingredients within the H-LAM/T system. The realization that any potential change in language, artifact, or method¬ ology has importance only relative to its use within a process, and that a new process capability appearing anywhere within that hierarchy can make practical a new consideration of latent change possibilities in many other parts of the hierarchy—possibilities in either language, artifacts, or methodology—brings out the strong interrelationship of these three augmentation means. Increasing the effectiveness of the individual’s use of his basic capa¬ bilities is a problem in redesigning the changeable parts of a system. The system is actively engaged in the continuous processes (among others) of developing comprehension within the individual and of solving problems; both processes are subject to human motivation, purpose, and will. Redesigning the system’s capability for performing these processes means FRAMEWORK FOR THE AUGMENTATION OF MAN’S INTELLECT 9 redesigning all or part of the repertory hierarchy. To redesign a structure we must learn as much as we can about the basic materials and com¬ ponents as they are utilized within the structure; beyond that, we must learn how to view, measure, analyze, and evaluate in terms of the functional whole and its purpose. In this particular case, no existing analytic theory is by itself adequate for the purpose of analyzing and evaluating over-all system performance; pursuit of an improved system thus demands the use of experimental methods. It need not be solely the sophisticated or formal process capabilities that are added or modified in the redesign. Even so apparently minor an advance as artifacts for rapid mechanical duplication and rearrangement of text during the course of creative thought process could yield changes in an individual’s repertory hierarchy that would represent a great increase in over-all effectiveness. Normally we might expect such equip¬ ment to appear slowly on the market; changes from old procedures would be small, and only gradually would the accumulated changes create markets for more radical versions of the equipment. Such an evolutionary process has been typical of the way our repertory hierarchies have formed and grown. But an active research effort, aimed at exploring and evaluating possible integrated changes throughout the repertory hierarchy, could greatly accelerate this evolutionary process. The research effort could guide the product development of new artifacts toward taking long-range meaningful steps; simultaneously, competitively minded individuals who would respond to demonstrated methods for achieving greater personal effectiveness would create a market for the more radical equipment innovations. The guided evolutionary process could be expected to be considerably more rapid than the traditional one. The category of “more radical innovations” includes the digital computer as a tool for the personal use of an individual. Here there is not only promise of great flexibility in the composing and rearranging of text and diagrams before the individual’s eyes, but also promise of many other process capabilities that can be integrated into the H-LAM/T system’s repertoire hierarchy. C. DETAILS OF THE H-LAM/T SYSTEM 1. Synergism* as the Source of Intelligence If we ask ourselves where human intelligence is embodied, our present state of knowledge forces us to concede that it appears to be elusively distributed throughout a hierarchy of functional processes—a hierarchy * Synergism, is a term used by biologists and physiologists to designate (from Iyobs^er’s New International Dictionary, 2d ed.) the . . cooperative action of ♦discrete agencies such that the total effect is greater than the sum of the two effects 'aken independently . . .” 10 VISTAS IN INFORMATION HANDLING whose foundation extends into natural processes beyond the level of present definition. Intelligence, however, seems primarily to be associated with organization. All of the social, biological, and physical phenomena we observe about us seem to derive from a supporting hierarchy of organized functions (or processes), in which the principle of synergism applies to give increased phenomenological sophistication to each succeedingly higher level of organization. In particular, the intelligence of a human being, which appears to be derived ultimately from the signal- response characteristics of individual nerve cells, is a synergistic phe¬ nomenon. 2. Intelligence Amplification During the course of this study, we had originally rejected the term intelligence ampHBcation, (initially used by W, Ross Ashby* 0 to characterize our objectives. Instead, we characterized them as the attempt to make a better match between existing human intelligence and problems to be solved. But we have come to accept the foregoing term in a special sense that does not imply any attempt to increase native human intelligence. Intelligence amplification seems applicable to our goal (of augmenting the human intellect) in that the entity to be pro¬ duced will exhibit more of what can be called intelligence than an unaided human could demonstrate. That which possesses the amplified intelli¬ gence is the resulting H-LAM/T system, in which the LAM/T augmen¬ tation means represent the amplifier of the individuars intelligence. In amplifying human intelligence we are applying the principle of synergistic structuring that pertains in the natural evolution of basic human capabilities. What our culture has done in the development of our means of augmentation is to construct a superstructure that is a synthetic extension of the biologically derived sensory-mental-motor structure on which it is built. In a very real sense, the development of ‘‘artificial intelligence'* has been going on for centuries, 3. Two-Domain System The human together with his artifacts comprise the only physical components in the H-LAM/T system. It is upon their combined capa¬ bilities that the ultimate capability of the system will depend. This conclusion was implied in the earlier statement that every composite process of the system decomposes ultimately into explicit-human and explicit-artifact processes. There are thus two separate domains of activity within the H-LAM/T system: that represented by the human, in which all explicit-human processes occur, and that represented by the artifacts, in which all explicit-artifact processes occur. In any composite process there is cooperative interaction between the two domains, requiring interchange of energy (much of it for information exchange FRAMEWORK FOR THE AUGMENTATION OF MAN’S INTELLECT 11 Outside world Fig. 1.1. Representation of the two active domains within the H-LAM/T System. purposes only). Figure 1.1 depicts this two-domain concept and embodies other concepts discussed below. Where a complex machine represents the principal artifact with which a human being cooperates, the term man-machine interface has been used for some years to represent the boundary across which energy is exchanged between the two domains. However, the man-artifact interface has existed for centuries, ever since humans began using artifacts and executing composite processes. Exchange across this “interface” occurs when an explicit-human process is coupled to an explicit-artifact process. Quite often these coupled processes are designed for just this exchange purpose, to provide a functional match between other explicit-human and explicit-artifact processes buried within their respective domains that do the more significant things. For instance, the finger and hand motions (explicit- human processes) activate key-linkage motions in the typewriter (coupled to explicit-artifact processes). But these are .only part of the matching processes between the deeper human processes that direct a given word to be typed and the more involved artifact processes that actually imprint the ink marks on the paper. The outside world interacts with our H-LAM/T system by the exchange of energy with either the individual or his artifact. Again, 12 VISTAS IN INFORMATION HANDLING special processes are often designed to accommodate this exchange. However, the direct concern of our present study lies within the system, with the internal processes that are and can be significantly involved in the effectiveness of the system in developing the human^s comprehension and pursuing the human’s goals. 4. Concepts, Symbols, and a Hypothesis Before we pursue further direct discussion of the H-LAM/T system, let us examine some background material. There is a certain progression in the development of our intellectual capabilities—not necessarily historical—that can shed light on the human part of the system: 4.1. Concept Manipulation. Humans have the biological capa¬ bility for developing abstractions and concepts. They can mentally manipulate these concepts to a certain extent, and **think” about situations in the abstract. Their mental capabilities allow them to develop general concepts from specific instances, predict specific instances from general concepts, associate concepts, remember them, etc. We speak here of concepts in their raw, unverbalized form. For example, a person letting a door swing shut behind him suddenly visualizes a person behind him carrying a cup of hot coffee and some sticky pastries. Of all the aspects of the impending event, the spilling of the coffee and the squashing of the pastry somehow are abstracted immediately and associated with a concept of personal responsibility combined with a fear of the consequences. But a solution comes to mind immediately as an image of a quick stop and an arm extended back toward the door, with motion and timing that could prevent the collision, and the solution is accepted and enacted. With only nonverbal concept manipulation, we could probably build primitive shelter, evolve strategies of war, hunt, play games, and make practical jokes. But further powers of intellectual effectiveness are implicit in this stage of biological evolution (the same stage we are in today). 4.2. Symbol Manipulation. Humans made another great step forward when they learned to represent particular concepts in their minds with specific symbols. Here we temporarily disregard communi¬ cative speech and writing and consider only the direct value to the individual of being able to do his heavy thinking by mentally manipu¬ lating symbols instead of the more unwieldy concepts which they represent. Consider, for instance, the mental difficulty involved in herding twenty-seven sheep if, instead of remembering one cardinal number and occasionally counting, we had to remember what each sheep looked like, so that if the flock seemed too small we could visualize each one and check whether or not it was there. 4.3. Manual, External, Symbol Manipulation, Another signifi¬ cant step toward harnessing the biologically evolved mental capabilities framework for the augmentation of man’s intellect 13 in pursuit of comprehension and problem solutions came with the development of the means for externalizing some of the symbol-manipu¬ lation activity—particularly in graphic representation, which supple¬ ments the individual’s memory and ability to visualize. (We are not concerned here with the value derived from human cooperation made possible by speech and writing, both forms of external symbol manipu¬ lation, but with the manual means of making graphic representations of symbols—a stick and sand, pencil and paper and eraser, straightedge or compass, and so on.) It is principally this kind of means for external symbol manipulation that has been associated with the evolution of the individual’s present way of manipulating his concepts (thinking). It is undoubtedly true that concepts which people found useful became incorporated as symbols in their language. However, Korzybski® and Whorf'* (among others) have argued that the language we use affects our thinking to a considerable extent. They say that a lack of words for some types of concepts makes it difficult to express those concepts, and thus decreases the likelihood that we will learn much about them. If this is so, once a language has begun to grow and be used it would seem reason¬ able to suspect that the language also affects the evolution of the new concepts to be expressed in that language. Apparently there are counter-arguments to this; e.g., if a concept needs to be used often but its expression is difficult, then the language will evolve to ease the situation. However, the studies of the past decade into what are called self-organizing systems seem to reveal that subtle relationships among interacting elements can significantly influence the course of evolution of such a system. If this is true, and if language is (as it seems to be) a part of a self-organizing system, then it appears probable that the state of a language at a given time strongly affects its own evolution to a succeeding state. For our conceptual framework, we tend to favor the view that a language does exert a force in its own evolution. We observe that the shift over the last few centuries in matters that are of daily concern to the individual has necessarily been forced into the framework of the language existing at the time, with alterations generally limited to new uses for old words, or to the coining of new words. The English language since Shakespeare has undergone no alteration comparable to the alteration in the cultural environment; if it had, Shakespeare would no longer be accessible to us. Under such evolutionary conditions, it would seem unlikely that the language we now use provides the best possible service to our minds in pursuing comprehension and solving problems. It seems very likely that a more useful language form can be devised. The Whorfian hypothesis states that “the world view of a culture is limited by the structure of the language which the culture uses.” But there seems to be another factor to consider in the evolution of language 14 VISTAS IN INFORMATION HANDLING and human reasoning ability. We offer the following hypothesis, which is related to the Whorfian hypothesis: Both the language used by a culture, and the capability for effective intellectual activity, are directly affected during their evolution by the means by which individuals control the external manipulation of symbols. (For identification, we will refer to this later on as the Neo-Whorfian hypothesis.) If the Neo-Whorfian hypothesis could be proved readily, and if we could see how our means of externally manipulating symbols influence both our language and our way of thinking, then we would have a valuable instrument for studying human-augmentation possibilities. For the sake of discussion, let us assume the Neo-Whorfian hypothesis to be true, and see what relevant deductions can be made. If the means evolved for an individual's external manipulation of his thinking-aid symbols indeed directly affect the way in which he thinks, then the original Whorfian hypothesis would offer an added effect. The direct effect of the external-symbol-manipulation means on language would produce an indirect effect on the way of thinking via the Whorfian- hypothesis linkage. There would then be two ways for our external symbol manipulation to affect our thinking. One way of viewing the H-LAM/T system changes that we contem¬ plate—specifically, integrating the capabilities of a digital computer into the intellectual activity of humans—is that we are introducing new and extremely advanced means for externally manipulating symbols. We then want to determine the useful modifications in the language and in the way of thinking that could result. This suggests a fourth stage to the evolution of our human intellectual capability. 4.4. Automated External Symbol Manipulation. In this stage, the symbols with which the human represents the concepts he is manip¬ ulating can be arranged before his eyes, moved, stored, recalled, operated upon according to extremely complex rules—all in very rapid response to a minimum amount of information supplied by the human, by means of special cooperative technological devices. In the limit of what we might now imagine, this could be a computer, with which individuals could communicate rapidly and easily, coupled to a three-dimensional color display within which extremely sophisticated images could be constructed, the computer being able to execute a wide variety of processes on parts or all of these images in automatic response to human direction. The displays and processes could provide helpful services and could involve concepts not hitherto imagined (e.g., the pregraphic thinker would have been unable to predict the bar graph, the process of long division, or card file system). In searching for some simple ways to determine what the Neo-Whorfian hypothesis might imply, we might imagine some relatively straight- framework for the augmentation of man’s intellect 15 forward means of increasing our external symbol-manipulation capability and try to picture the consequent changes that could evolve in our language and methods of thinking. For instance, imagine that our budding technology of a few generations ago had developed an artifact that was essentially a high-speed, semiautomatic table-lookup device, cheap enough for almost everyone to afford and small and light enough to be carried on the person. Assume that individual cartridges sold by manufacturers (publishers) contained the lookup information, that one cartridge could hold the equivalent of an unabridged dictionary, and that a one-paragraph definition could always be located and displayed on the face of the device by the average practiced individual in less than three seconds. What changes in language and methodology might not result? If it were so easy to look things up, how would our vocabulary develop, how would our habits of exploring the intellectual domains of others shift, how might the sophistication of practical organization mature (if each person could so quickly and easily look up applicable rules), how would our education system change to take advantage of this new external symbol-manipulation capability of students and teachers and administrators? The significance to our study of this discussion lies in the perspective it gives to the ways in which human intellectual effectiveness can be altered by the particular device used by individuals for their external symbol manipulation. These hypotheses imply great richness in the new evolutionary spaces opened by progressing from Stage 3 to Stage 4. We would like to study the hypotheses further, examining their possible manifestations in our experience, ways of demonstrating their validity, and possible deductions relative to going to Stage 4. 5. Capability Repertory Hierarchy I The concept of our H-LAM/T system possessing a repertory of I capabilities that is structured in the form of a hierarchy is most useful j in our study. We shall use it below to tie together a number of considera- j tions and concepts. j There are two points of focus in considering the design of new repertory i hierarchies: the materials with which we have to work, and the principles I by which new capability is constructed from these basic materials. [ 5.1. Basic Capabilities. Materials in this context are those capa- \ bilities in the human and artifact domains from which all other capa¬ bilities in the repertory hierarchy must be constructed. Each such basic capability represents a type of functional component with which the I system can be built. Thorough redesigning of the system requires making I an inventory of the basic capabilities available. Because we are exploring I for perspective, and are not yet recommending research activities, we I are free to discuss and define in more detail what we mean by ‘^basic 16 VISTAS IN INFORMATION HANDLING framework for the AUGMENTATION OF MAN’S INTELLECT 17 capability” without regard to the amount of research involved in making an actual inventory. The two domains, human and artifact, can be explored separately for their basic capabilities. In each we can isolate two classes of basic capability—those classes distinguished according to whether or not the capability has been put to use within our augmentation means. The first class (those in use) can be found in a methodical manner by analyzing present capability hierarchies. For example, select a given capability at any level in the hierarchy and determine whether it can be usefully changed by any means that can be given consideration in the augmen¬ tation research contemplated. If it can, then it is not basic but can be decomposed into an eventual set of basic capabilities. Proceed through the hierarchy; capabilities encountered which cannot be usefully changed compose the basic capability inventory. Ultimately, for every such recursive decomposition of a given capability in the hierarchy, every one of the branching paths will terminate in basic capabilities. Many of the branching paths in the decomposition of a given higher-order capability will terminate in the same basic capability, since a given basic capability will often be used within many different higher-order capabilities. Determining the class of basic capabilities not already utilized within existing augmentation systems requires a different exploration method. Examples of this method occur in technological research, where analyt¬ ically oriented researchers search for new understandings of phenomena that can add to the research engineer’s list of things to be used in the synthesis of better artifacts. Before this inventorying task can be pursued in any specific instance, some criteria must be established as to what possible changes within the H-LAM/T system can be given serious consideration. For instance, some research situations might have to disallow changes which require exten¬ sive retraining, or which require undignified behavior by the human. Other situations might admit changes requiring years of special training, very expensive equipment, or the use of special drugs. The capability for performing a certain finger action, for example, may not be basic in our sense of the word. Being able to extend the finger a certain distance would be basic, but the strength and speed of a particular finger motion and its coordination with higher actions generally are usefully changeable and therefore do not represent basic capabilities. What would be basic in this case would perhaps be the processes whereby strength could be increased and coordinated movement patterns learned, as well as the basic movement range established by the mechanical-limit loci of the muscle-tendon-bone system. Similar capability breakdowns will occur for sensory and cognitive capabilities. 5.2. Structure Types. The fundamental principle used in building sophisticated capabilities from basic capabilities is structuring—the special type of structuring (which we have termed synergistic) in which the organization of a group of elements produces an effect greater than the mere addition of their individual effects. Perhaps purposeful structuring (or organization) would best express the need, but how the structuring concept must mature is uncertain. We are developing growing awareness of the significant and pervasive nature of structuring within every physical and conceptual element we inspect, where the hierarchical form seems almost universally present as stemming from successive levels of organization. The fundamental entity which is being structured in each and every case seems to be what we could call a process, where the most basic of physical processes (involving fields, charges, and moments associated with the dynamics of fundamental particles) appear in every case as the hierarchical base. Dynamic electro-optical-mechanical processes associ¬ ated with the function of our artifacts, and metabolic, sensory, motor, and cognitive processes of the human, which we view as relatively fundamental components within the structure of our H-LAM/T system, each seems to be ultimately based (to our degree of understanding) on the above-mentioned basic physical processes. The elements that are organized to give fixed structural form to our physical objects (e.g., the “element” of tensile strength of a material) are also derived from what we could call synergistic structuring of the most basic physical processes. At the level of the capability hierarchy where we wish to work, it seems useful to distinguish several different types of structuring, even though each type is fundamentally a structuring of the basic physical processes. Tentatively we have isolated five such types, although we are not sure how many we shall ultimately want to use in considering the problem of augmenting the human intellect, nor how we might divide and subdivide these different manifestations of physical-process struc¬ turing. We use the terms mental structuring, concept structuring, symbol structuring, process structuring, and physical structuring. 5.2.1. Mental Structuring, Mental structuring we apply to the internal organization of conscious and unconscious mental images, associations, or concepts which somehow manage to provide the human with understanding and the basis for judgment, intuition, inference, and meaningful action with respect to his environment. (The psychologist’s “cognitive structure” may be very near to what we need in our concept of mental structure). We do not now try to specify the fundamental mental “things” being structured, nor the mechanisms that accomplish the structuring or the use of that which has been structured. We feel reasonably safe in assuming that learning involves some kind of meaningful organization within the brain, and that whatever is so organized or structured represents the 18 VISTAS IN INFORMATION HANDLING FRAMEWORK FOR THE AUGMENTATION OF MAN’S INTELLECT 19 operating model of the individual’s universe to the mental mechanisms that derive his behavior. Further, our assumption is that when the human in our H-LAM/T system makes the key decision or action that leads to the solution of a complex problem, this action will stem from the state of his mental structure at that time; the basic purpose of most of the system’s activity on that problem up to that point was in developing his mental structure to permit the mental mechanisms to derive a solution from it. We don’t know whether a structure is developed in a manner analogous to (a) the development of a garden, where one provides a good environ¬ ment, plants the seeds, keeps competing weeds and injurious pests out, but otherwise lets natural processes take their course, or (b) the develop¬ ment of a basketball team, where much exercise of skills, patterns, and strategies must be provided so that natural processes can slowly knit together an integration, or (c) the development of a machine, where carefully formed elements are assembled in a precise, planned manner so that natural phenomena can immediately yield planned function. We do not know the processes, but we can develop and have developed em¬ pirical relationships between the experiences given a human and the associated manifestations of developing comprehension and capability; we see the near-future course of the research toward augmenting the human intellect as depending entirely on empirical findings (past and future) for the development of better means to serve the development and use of mental structuring in the human. We do not mean to imply by this that we renounce theories of mental processes. What we mean to emphasize is that the pursuit of our objective does not have to wait on understanding the mental processes that accomplish what we call mental structuring and that derive behavior therefrom. Not to make the fullest use of any theory, that provided a working explanation for a group of empirical data would be to ignore the emphases of our own conceptual framework. 5.2.2. Concept Structuring, Within our framework we have developed the working assumption that the manner in which formal experiences favor the development of mental structures is based largely on concepts as ‘‘media of exchange.” We view a concept to be a tool that can be grasped and used by the mental mechanisms, that can be composed, interpreted, and used by natural mental substances and processes. The grasping and processing done by these mechanisms can often be accomplished more easily if the concept is explicitly represented by a S 3 ^mbol. Somehow the mental mechanisms can learn to manipulate images (or something) of symbols in a meaningful way and remain calmly confident that the associated concepts are within call. Concepts seem to be structurable in that a new concept can be com¬ posed of an organization of established concepts. For present purposes we can view a concept structure as something which we might try to develop on paper for ourselves or work with by conscious thought processes, or as something which we try to communicate to one another in serious discussion. We assume that for a given unit of comprehension to be imparted there is a concept structure (which can be consciously developed and displayed) that can be presented to an individual in such a way that it is mapped into a corresponding mental structure which provides the basis for that individual’s “comprehending” behavior. Our working assumption also considers that some concept structures would be better for this purpose than others, in that they would be more easily mapped by the individual into workable mental structures, or in that the resulting mental structures enable a higher degree of comprehension and better solutions to problems, or both. A concept structure often grows as part of a cultural evolution—either on a large scale within a large segment of society, or on a small scale within the activity domain of an individual. But it is also something that can be directly designed or modified, and a basic hypothesis of our study is that better concept structures can be developed—structures that when mapped into a human’s mental structure will significantly improve his capability to comprehend and to find solutions within his complex- problem situations. A natural language provides its user with a ready¬ made structure of concepts that establishes a basic mental structure, and that allows relatively flexible, general-purpose concept structuring. Our concept of “language” as one of the basic means for augmenting the human intellect embraces all of the concept structuring which the human may make use of. 5.2.3. Symbol Structuring. The other important part of our “language” concerns the way in which concepts are represented—the symbols and symbol structures: by means of which words as structured into phrases, sentences, paragraphs, monographs, or charts, lists, dia¬ grams, and tables. A given structure of concepts can be represented by any one of an infinite number of different symbol structures, some of which would be much better than others for enabling the human per¬ ceptual and cognitive apparatus to search out and comprehend the conceptual matter of significance and/or interest. A concept structure involving many numerical data, for example, would generally be better represented with Arabic than Roman numerals; quite likely, a graphic structure would be better than a tabular structure. In our special framework, it is worth noting that a given concept structure can be represented with a symbol structure that is completely compatible with the way a computer handles symbols. Such structuring has immensely greater potential for accurately mapping a complex concept structure than does the structure which an individual might practically construct and use on paper. A computer can transform back 20 VISTAS IN INFORMATION HANDLING 21 and forth between some limited view of the total structure as represented by a two-dimensional portrayal on a screen, and an aspect of the n- dimensional internal image that represents this “view/’ If the human adds to or modifies such a “view,” the computer integrates the change into the internal-image symbol structure (in terms of the computer’s favored symbols and structuring), and thereby can automatically detect a certain proportion of his possible conceptual inconsistencies. The human need no longer work on rigid and limited symbol structures, where much of the conceptual content can only be implicitly designated in an indirect and distributed fashion. Many radical new ways of matching the dynamics of our symbol structuring to those of our concept structuring are basically available with today’s technology. Their exploration would be most stimulating, and potentially very rewarding. 5.2.4. Process Structuring. Essentially everything that goes on within the H-LAM/T system (in relation to our direct interest here) involves the manipulation of concept and symbol structures in service to the human’s mental structure. Therefore the processes within the H-LAM/T system that we are most interested in developing are those that provide for the manipulation of all three types of structure. This brings us to the fourth category of structuring, namely, process structuring. As we currently use it, the term process structuring includes the organization, study, modification, and execution of processes and process structures. Whereas concept structuring and symbol structuring together represent the language component of our augmentation means, process structuring represents primarily the methodology component. Many of the process structures are applied to the task of organizing, executing, supervising, and evaluating other process structures. Others are applied to the formation and manipulation of symbol structures (the purpose of which will often be to support the conceptual labor involved in process structuring). 5.2.5. Physical Structuring. Physical structuring, the last of the five types which we currently use in our conceptual framework, is nearly self-explanatory. It represents the artifact component of our augmenta¬ tion means, insofar as the actual manifestation and organization of the physical devices are concerned. 5.2.6. Interdependence and Regeneration. An important feature to be noted from the foregoing discussion is the interdependence among the various types of structuring which are involved in the H-LAM/T system, where the capability for doing each type of structuring is depen¬ dent upon the capability of achieving one or more of the other types of structuring. (Assuming that the physical structuring of the system FRAMEWORK FOR THE AGUMENTATION OF MAN’S INTELLECT remains basically unchanged during the system’s operation, we exclude its dependence on other factors in this discussion.) This interdependence has a cyclic, regenerative nature which is very significant. A good portion of the capability for mental structuring is finally dependent on the process structuring (human, artifact, composite) that enables symbol- structure manipulation. But it also is evident that this process structuring is dependent not only on basic human and artifact process capabilities but also on the ability of the human to learn how to execute processes and—no less important—on the ability of the human to select, organize, and modify processes from his repertory to structure a higher-order process that he can execute. Thus capability for structuring and executing processes is partially dependent on the human’s mental structuring, which, in turn, is partially dependent on his process structuring (through concept and symbol structuring), which is partially dependent on his mental structuring, etc. This means that a significant improvement in symbol-structure manipulation through better process structuring (initially perhaps through much better artifacts) should enable us to develop improvements in concept and mental-structure manipulations that can in turn enable us to organize and execute symbol-manipulation processes of increased power. When considering the possibilities of computerlike devices for aug¬ menting human capabilities, often only the one-pass improvement is visualized. This presents a relatively barren picture in comparison with that which- emerges on consideration of regenerative interaction. 5.3. Roles .\nd Levels. In the repertory hierarchy of capabilities possessed by the H-LAM/T system, the human contributes many types of capability that represent a wide variety of roles. At one time or another he will be the policy maker, the goal setter, the performance supervisor, the work scheduler, the professional specialist, the clerk, the janitor, the entrepreneur and the proprietor (or at least a major stock¬ holder) of the system. In the midst of some complex process, in fact, he may well be in several roles concurrently—or at least have the responsi¬ bility of the roles. For example, usually he must be aware of his progress toward a goal (supervisor), he must be alert to the possibilities for changing the goal (policy maker, planner), and he must keep records for these ami other roles (clerk). A given capability at some level in the repertory hierarchy seems to include standard grouping of lower-order capabilities which can be viewed as existing in two classes—an executive class and a direct- cdntributive class. In the executive class are capabilities for compre¬ hending, planning, and executing the process. In the direct-contributive class are the capabilities organized by the executive class toward the direct realization of the higher-order capability. For example, when the 22 23 VISTAS IN INFORMATION HANDLING telephone rings, direct-contributive processes are picking up the receiver and saying '‘hello/' The executive processes comprehended the situation, directed a lower-order executive-process that the receiver be picked up and, when the receiver was in place (first process accomplished), directed the next process—the saying of “hello/' This represents the composition of the capability for answering the telephone. At a little higher level of capability, more of the conscious conceptual and executive capabilities become involved. To telephone someone, there must be conscious comprehension of the need for this process and how it can be executed. At a still higher level of capability, the executive capabilities must have a degree of power that cannot be provided by unaided mental capabilities. In such a case, a sequence of steps might be drafted and checked off as each is executed. For an even more complex process, comprehending the situation in which the process is to be executed— before even beginning to plan the execution—may take months of labor and a very complex organization of the system's capabilities. At any particular moment the H-LAM/T system is usually in the middle of executing a great number of processes. For example, the human in the process of making a telephone call may be in the middle of the process of calling a committee meeting which could be a sub-process of the process of estimating manpower needs, and so on. Not only does the human need to play various roles (sometimes concurrently) in the execution of any given process, but he is playing these roles for the many concurrent processes that are being executed at different levels. This situation is typical for individuals engaged in reasonably demanding types of professional pursuits, and yet they have never received explicit training in optimum ways of performing any but a very few of the roles at a very few of the levels. A well-designed H- LAM/T system would provide explicit and effective concepts, terms, equipment, and methods for all these roles, and for their dynamic coordination. 5.4. Model of Executive Superstructure. It is the repertory hierarchy of process capabilities upon which the ultimate capability of the H-LAM/T system rests. This repertory hierarchy is rather like a mountain of white-collar talent that sits atop and controls the talents of the “workers." We can illustrate executive superstructure by considering it as though it were a network of contractors and subcontractors in which each capability in the repertory hierarchy is represented by an inde¬ pendent contractor whose mode of operation is to do the planning, make up specifications, subcontract the actual work, and supervise the per¬ formance of his subcontractors. This means that each subcontractor does the same thing in his turn. At the bottom of this heirarchy are those independent contractors who do actual “production work." framework for the augmentation of man's intellect If by some magical process the production workers could still know just what to do and when to do it even though the superstructure of contractors was removed from above them, no one would know the difference. The executive superstructure is there because humans do not operate by magic, but even a necessary superstructure is a burden. We can readily recognize that there are many ways to organize and manage such a superstructure, resulting in vastly different degrees of efficiency in the application of the workers' talents. Suppose that the applicable talent available to the total system is limited. The problem is one of distributing that talent between super¬ structure and workers for maximum total production and efficiency. This situation has close parallel to the H-LAM/T system in its pursuit of comprehension and problem solutions. Closer parallel exists by postulating for the contractor model that the thinking, planning, super¬ vising, record keeping, etc., for each contractor is done by a single individual who time-shares his attention and talents over the various tasks of the entire superstructure. Today's individual does not have special training for many of the roles he plays, and he is likely to learn them by cut-and-try and indirect imitation processes. The H-LAM/T system also often executes a complex process in multipass fashion (i.e., cut-and-try). This approach permits freedom of action which is important to the effectiveness of the system with respect to the outside world. We could expect significant gains from automating the H-LAM/T system if a computer did no more than increase the effectiveness of the executive processes. More human time, energy, and productive thought could be allocated to direct-contributive processes, which could be coordinated in a more sophisticated, flexible and efficient manner. But there is every reason to believe that the possibilities for much-improved process structuring that would stem from this automation could in turn provide significant improvements in both the executive and direct-contributive processes in the system. 5.5. Symbol Structures. The executive superstructure is a neces¬ sary component in the H-LAM/T system, and there is finite human capability which must be divided between executive and direct-contrib¬ utive activities. An important aspect of the multirole activity of the human in the system is the development and manipulation of the symbol structures associated with both his direct-contributive roles and his executive roles. When the system encounters a complex situation in which compre¬ hension and problem solutions are being pursued, the direct-contributive roles require the development of symbol structures that portray the concepts involved within the situation. But executive roles in a complex problem situation also require conceptual activity—e.g,, comprehension, selection, supervision—that can benefit from well-designed symbol 24 25 VISTAS IN INFORMATION HANDLING structures and fast, flexible means for manipulating and displaying them. For complex processes, the executive problem posed to the human (of gaining the necessary comprehension and making a good plan) may be more difficult intellectually than the problem faced in the role of direct- contributive worker. If the flexibility desired for the process hierarchies (to make room for human cut-and*try methods) is not to be degraded or abandoned, the executive activity will have to be provided with fast and flexible symbol structuring techniques. The means available to humans today for developing and manipulating symbol structures are both laborious and inflexible. To develop an initial structure of diagrams and text is difficult, but because the cost of frequent changes is often prohibitive, one settles for inflexibility. Also, the flexi¬ bility that would be truly helpful requires added symbol structuring just to keep track of the trials, branches, and reasoning thereto that are involved in the development of the subject structure. Present symbol- manipulation means would soon bog down completely among the complexities that are involved in being more than just a little bit flexible. In H-LAM/T systems, individuals work essentially continuously within a symbol structure of some sort, shifting their attention from one structure to another as they guide and execute the processes that ulti¬ mately provide them with the comprehension and the problem solutions they seek. This view emphasizes the essential importance of the basic capability of composing and modifying efficient symbol structures. Such a capability depends heavily on the particular concepts isolated and manipulated as entities, on the symbology used to represent them, on the artifacts that help to manipulate and display the symbols, and on the methodology for developing and using symbol structures. In other words, this capability depends heavily on proper language, artifacts, and methodology, our basic augmentation means. The course of action which must respond to new comprehension, new insights, and new intuitive flashes of possible explanations or solutions is not an orderly process. Existing means of composing and working with symbol structures penalize disorderly processes heavily. It is part of the real promise of the automated H-LAM/T systems of tomorrow that the human can have the freedom and power of disorderly processes. 5.6. Compound Effects. Since processes in many levels of the hierarchy are involved in the execution of a single higher-level process of the system, any factor that influences process execution in general will have a highly compounded total effect on the system’s performance. There are several such factors that merit special attention. Basic human cognitive powers, such as memory, intelligence, or pattern perception can have such a compounded effect. The augmentation means employed today have generally evolved among large statistical framework for the augmentation of man’s intellect populations, and no attempt has been made to fit them to individual needs and abilities. Each individual tends to evolve his own variations, but there is not enough mutation and selection activity, nor enough selection feedback, to permit very significant changes. A good, automated H-LAM/T system should provide the opportunity for a significant adaptation of the augmentation means to individual characteristics. The compounding effect of fundamental human cognitive powers suggests further that systems designed for maximum effectiveness would require that these powers be developed as fully as possible—by training, special mental tricks, improved language, new methodology. In the automated system contemplated here, the human should be able to draw on the explicit-artifact process capability at many levels in the repertory hierarchy. Today, artifacts are involved explicitly in only the lower-order capabilities. In future systems it should be possible for computer processes to provide direct manipulative service in the execu¬ tive symbol structures at all the higher levels, which promises a com¬ pounding of the effect a computer may have. Another factor capable of exerting a compound effect on over-all system performance is the human’s unconscious processes. Clinical psychology seems to provide clear evidence that a large proportion of a human’s everyday activity is significantly mediated or basically prompted by unconscious mental processes that, although “natural” in a functional sense, are not rational. Observable mechanisms of these processes (observable by a trained person) include an individual’s masking of the irrationality of his actions, and the construction of self-satisfying rationales for any action that could be challenged. Anything that might have so general an effect on our mental actions as implied here is a candidate for ultimate consideration in the continuing development of our intellectual effectiveness. It may be that the first stages of research on augmenting the human intellect will have to proceed without coping with this problem except to accommodate to it as well as possible. This may be one of the significant problems whose solution awaits our develop¬ ment of increased intellectual effectiveness. OTHER RELATED THOUGHT AND WORK When we began our search, we found much literature of general significance to our objective—frankly, one is tempted to say too much. Without a conceptual framework we could not efficiently filter out the significant kernels of fact and concept from the huge mass which we initially collected as a “natural first step” in our search. We feel rather unscholarly not to have buttressed our conceptual framework with plentiful reference to supporting work, but in truth it was too difficult to do. Developing the conceptual structure represented a sweeping 26 VISTAS IN INFORMATION HANDLING synthesis job full of personal constructs from smatterings picked up in many places. Under these conditions, giving reference to a back-up source would usually entail qualifying footnotes reflecting an unusual interpretation or exonerating the cited author from the implications we derived from his work. We look forward to a stronger, more compre¬ hensive, and more scholarly presentation evolving out of future work. However, we do want to acknowledge thoughts and work we have come across that bear most directly upon the possibilities of using a computer in real-time working association with a human to improve his working effectiveness. These findings fall into two categories. The first, which would include the present report, offers speculations and possi¬ bilities but does not include reporting of significant experimental results. Of these, Bush® is the earliest and one of the most directly stimulating. Licklider,® who provides the most general clear case for the modern computer, coined the expression man-computeT symbiosis to refer to the close interaction relationship between the man and computer in mutually beneficial cooperation. Ulam^ has specifically recommended close man-computer interaction in a chapter entitled ‘‘Synergesis,*’ where he points out in considerable detail the types of mathematical work which could be aided. Good‘S includes some conjecture about the possi¬ bilities of intellectual aid to the human by close cooperation with a computer in a rather general way, and also presents a few interesting thoughts about a network model for structuring the conceptual kernels of information to facilitate a sort of self-organizing retrieval system. Ramo has given a number of talks dealing with the future possibilities of computers for ‘‘extending man^s intellect,” and wrote several articles.^His projections seem slanted more toward larger bodies of humans interacting with computers, in less of an intimate personal sense than the above papers or than our initial goal. Fein,^^ in making a comprehensive projection of the growth and dynamic interrelatedness of “computer-related sciences,” includes specific mention of the enhance¬ ment of human intellect by cooperative activity of men, mechanisms, and automata. Fein coined the term synnoetics as applicable generally to the cooperative interaction of people, mechanisms, plant or animal organisms, and automata into a system whose mental power is greater than that of its components; he presents a good picture of the integrated way in which many currently separate disciplines should be developed and taught in the future to do justice to their mutual roles in the im¬ portant discipline defined as “synnoetics.” In the second category, there have been a few papers published recently describing actual work that bears directly upon our topic. Licklider and Clark, 12 and Culler and Huff,i^ at the 1962 Spring Joint Computer Conference, gave what are essentially progress reports of work going on now in exactly this sort of thing—a human with a computer-backed FRAMEWORK FOR THE AUGMENTATION OF MAN’S INTELLECT 27 display getting minute-by-minute help in solving problems. Teageri'*-*® reports on the plans and current development of a large time-sharing system at M.I.T., which is planned to provide direct computer access for a number of outlying stations located in scientists’ offices, giving each of these users a chance for real-time utilization of the computer. There are several efforts we have heard about but for which there are either no publications or for which none have been discovered by us. Just before the deadline date, we have received two publications from the M.I.T. Electronic Systems Laboratory; an Interim Engineering Report (of work done over two years ago), “Investigations In Computer-Aided Design,” appears to contain much detailed analysis of applied work in close man-computer cooperation. A Technical Memorandum, “Method for Computer Visualization,” by A. F. Smith, apparently elaborates on Chapter VII of the Interim Report. These documents seem extremely relevant. Mr. Douglas Ross, of the M.I.T. Electronic Systems Labora¬ tory has, we have recently learned, been thinking and working on real-time man-machine interaction problems for some years. In addition, an M.I.T. graduate student, Glenn Randa,^® developed the design of a remote display console under Ross for his graduate thesis project. We understand that another M.I.T. graduate student, Ivan Sutherland, is currently using the display-computer facility on the TX-2 computer at Lincoln Laboratory to develop cooperative techniques for engineering design problems. At the RAND Corporation, Cliff Shaw, Tom I llis, and Keith Uncapher have been involved in implementing a multistation time-sharing system built around their JOHNNIAC computer. Termed the JOHNNIAC Open-Shop System (JOSS for short), it apparently is near completion, and will use remote typewriter stations. Undoubtedly there are other efforts falling into either or both cate¬ gories that have been overlooked. Such oversight has not been intentional, and it is hoped that these researchers will make their pertinent work known to the present writer. SUMMARY AND RECOMMENDATIONS This paper states the hypothesis that the intellectual effectiveness of an individual is dependent on factors which are subject to direct redesign in pursuit of an increase in that effectiveness. A conceptual framework is offered to help in giving consideration to this hypothesis. The framework in part derives from recognition that human intellect is already “aug¬ mented,” and incorporates the following attributes: 1. As principal elements, the language, artifacts, and methodology which man has learned to use. 2. Dynamic interdependence of the elements within an operating system. VISTAS IN INFORMATION HANDLING 3. A hierarchical system structure, best considered a hierarchy of process capabilities whose primitive components are the basic human capabilities and the functional capabilities of the artifacts, organized into increasingly sophisticated capabilities. 4. As capabilities of primary interest, those associated with manipu¬ lating symbols and concepts in support of organizing and executing processes from which are ultimately derived human comprehension and problem solutions. The framework also pictures the development of automated symbol manipulation to accommodate minute-by-minute mental processes as a significant means of increasing intellectual capability. This can be a logical next step in the cultural evolution of the means by which humans can match their mental capabilities against their problems. This approach pertains to any problem area in which the human does his thinking with concepts that he can express in words, charts, or any other explicit symbol forms. If the hypothesis and extrapolations discussed here and elsewhere (AFOSR 3223) are substantiated in future developments, the conse¬ quences will be exciting and assumedly beneficial to a problem-laden world. What is needed now is a test of this hypothesis and a calibration on the gains, if any, that might be realized by giving total-system design attention to human intellectual effectiveness. If the test and calibration prove favorable, then better and better augmentation systems could be developed for our problem solvers. In this light, a research program is recommended aimed at (a) testing the hypothesis, (b) developing the tools and techniques for designing better augrnentation systems, and (c) providing feal-world augmentation systems. These goals are idealized, but results in these directions are nonetheless valuable. The apporach should be on an empirical, total- system basis, i.e., coordinated study and innovation among all the factors admitted to the problem in conjunction with experiments that provide realistic action and interplay among the variables. The recom¬ mended environment for this approach is a laboratory with a computer- backed display and communication system. The experimental work of deriving, testing, and integrating innovations into a growing system of augmentation means is helped by having a specific type of human task on which to operate. From a long-range research-program point of view, characteristics of the task of computer programming make it particularly attractive as the initial such specific task. In our view, we do not have to suspend such research until we learn how human mental processes work. We do not have to wait until we learn how to make computers more '‘intelligent.” We can begin develop- powerful and economically feasible augmentation systems on the framework for the augmentation of man's intellect 29 basis of what we now know and have. We will want to integrate further basic knowledge and improved machines into existing augmentation systems. However, getting started now will provide not only orientation and stimulation for these pursuits, but also better problem-solving effectiveness with which to carry out such pursuits. REFERENCES 1. Ashby, W, Ross, Design for a Brain, 2d ed., John Wiley fis Sons, Inc,, New York, 1960. 2 ^ -^ Design for an Intelligence Amplified, Automatic Studies, C. E. Shannon and J. McCarthy, Princeton Univ. Press, Princeton, N.J., 1956, pp. 215-234. 3. Korzybski, A., Science and Sanity, International Non-Aristotelian Library Publishing Company, Lancaster, Pa., 1933. 4. Whorf, B. L., Language, Thought, and Reality, M.I.T. and John Wiley fit Sons, Inc., New York, 1956. 5. Bush, Vannevar, “As We May Think,” The Atlantic Monthly July, 1945. 6. Licklider, J. C. R.. “Man-Computer Symbiosis,” IRE Transactions on Human Factors in Electronics, March, 1960. 7. Ulam, S. M., A Collection of Mathematical Problems, Interscience Publishers, Inc., New York, 1960, p. 135, 8. Good, I. J., “How Much Science Can You Have at Your Fingertips?” IBM Journal of Research and Development, October, 1958. 9. Ramo, Simon, “A New Technique of Education,” IRE Transactions on Education, June, 1958. 10 -^ “The Scientific Extension of the Human Intellect,” Computers and Automation, February, 1961. 11. Fein, Louis, “The Computer-Related Science (Synnoetics) at a University in the Year 1975,” unpublished paper, December, 1960. 12. Licklider, J. C. R., and W. E. Clark, “On-Line Man-Computer Communica¬ tion,” Proceedings Spring Joint Computer Conference, National Press, Palo Alto, Calif., May, 1962. 13. Culler, G. J., and R. W. Huff, “Solution of Non-Linear Integral Equations Using On-Line Computer Control,” paper for presentation at S.J.C.C., San Francisco, Ramo-Wooldridge, Canoga Park, Calif., May, 1962, 14. Teager, H. M., “Real-Time, Time-Shared Computer Project,” report, M.I.T. Contract # Nonr-184l(69) DSR #8644, July 1, 1961. 15 ^ -^ “Systems Considerations in Real-Time Computer Usage,” paper presented at ONR Symposium on Automated Teaching, Oct. 12, 1961. 16. Randa, Glenn C., “Design of a Remote Display Console,” Report ESL-R-132, M.I.T. Cambridge, Mass., February, 1962, available through ASTIA.