The reputation of the computer for impersonality and inhuman efficiency is due, in part, to the publicity given the computer as a poet, a chess-player, and a translator of exotic languages. "Machine intelligence" is a subject with fascinating technical and philosophical aspects. To date, however, there is no evidence that a computer capable of "taking over" anything it was not specifically programmed to. take over is attainable. Indeed, as pointed out earlier, programming a computer to handle anything complicated is usually a very difficult and expensive job, requiring generous amounts of money, expertise, and management capability.
It seems safe to predict that economic and organizational constraints on the uses of computers . will not change: radically during the next few years. Although computing power and data-storage capability are steadily becoming cheaper, and problemoriented programming is being improved, no dramatic breakthroughs are in sight. This prediction, however, cuts two ways. If we can comfortably assume that computers will not take control of anything on their own volition, we may still feel some disappointment that the application of computers will tend to remain in the hands of trained specialists whose competence is primarily in data processing rather than in the fields that data processing serves. Some would say that this circumstance results from an abdication by managers of their proper role, but whatever the reason, the effect can easily be to insulate the record-keeping functions of an organization from the pressures of both consumers and suppliers of data.
The presence of a specialized group of data-processing professionals in an organization can create a constituency within the organization whose interests are served by any increase in data use, without much regard for the intrinsic value of the increased use. The point is underlined by an experience common to many organizations. Some unit is already operating a computer facility for accounting, processing scientific or engineering data, or for some other straightforward application to which the technology is well-adapted. Because the facility has extra computer time available, it is soon discovered that attractive software packages can be purchased to enable the computer to enlarge its scope and become a "management information system."
Such systems are founded on the proposition that efficient decision making requires that managers have available to them a greater or more timely supply of relevant information than they have been getting. As commonly observed, however, most managers do not need more of relevant information nearly as badly as they need less of irrelevant raw data.8 Thus, until the theory of management itself has progressed to a stage where the necessary data content of management-oriented systems can be predicted, their users are likely to find them disappointing.
Another, potentially more serious, consequence of putting record keeping in the hands of a new class of data-processing specialists is that questions of record-keeping practice which involve issues of social policy are sometimes treated as if they were nothing more than questions of efficient technique. The pressure for establishing a simple, identification scheme for locating records in computer-based systems is a case in point.
The technical argument for having a standard universal identifier for records about individuals focuses on increasing the efficiency of record keeping and record usage. Proponents argue that if every item of data entered into an automated system could be associated with an identifier unique to the individual to whom the data pertain, updating, merging, and linking operations would be greatly. simplified and far less error-prone than they are today. Moreover, records could be used more intensively; administrative records indexed by Social Security number, for example, could also be used for certain types of research which require matching data on individuals from several different record systems.
To reap the full technical advantages of a standard identiflication scheme, it is necessary for each individual to supply the identifier assigned to him every time he has contact with a record-keeping organization using it. This practice is already familiar to the clients of banks, credit-card services, and many other organizations that have developed their own standard schemes. What worries people is that the inconvenience to record-keeping organizations of having to devise their own numbering arrangements will encourage the adoption of a single universal scheme for use in all computer-based personal data systems. If this happens, organizations that share an interest in monitoring and controlling the behavior of some portion of the population will acquire an enlarged capacity to do so, since they will all be able to know when an individual has contact with any one of them. Fingerprints, for example, are the standard method used by the police to identify persons arrested for crimes. Fingerprinting assures accurate identification and may seem a reasonable way of dealing with criminal offenders, but it is a dubious model for other types of record-keeping organizations to follow.
It is, of course, a long step from having each individual identified in the same way in every data system to creating a giant national data bank of dossiers constructed from fragments of records on citizens in widely dispersed data systems. There would have to be some strong incentive for "putting it all together," and as we noted earlier, it is doubtful that even the dollar cost of doing so could be justified on any reasonable grounds. However, it is not necessary to build a giant national data bank to experience some of the effects of having one. There are already systems in operation which have some of the control capabilities that such a centralized dossier system would create.
One computer-based personal data system that came to our attention was a comprehensive health information system developed and maintained by an agency of the Department of Health, Education, and Welfare on an Indian reservation in the Southwest. Approximately 10,000 Indians living in the area have records in the system and another 4,000 have records in it but, for one: reason or another, are not part of the active patient population. These 14,000 record subjects are, by and large, an economically dependent population with very serious health problems. Within the confines of the geographic area covered by the system-about the size of Connecticut-they are also a highly mobile population, with each individual going by any one of several different names depending on circumstances.
The health facility consists of a combination of in-patient, out-patient, and field-clinic services. The purpose of its cornputer-based record-keeping system is to develop a complete, cradle-to-grave, medical dossier on each individual eligible to use the facility, so that all can benefit from a comprehensive diagnostic and treatment program that aims to control illness by preventing its occurrence, or by taking preemptive steps at the, first sign of a medical problem.
The record-keeping system has three basic components: (1) an administrative one that notes and describes every contact each patient has with any segment of the health facility, including the "interdisciplinary" teams of doctors, nurses, and social workers who travel about administering tests and providing ambulatory health services; (2) a statistical-reporting one that attempts to observe fluctuations in the incidence of certain types of ailments and to pinpoint "high risk" groups needing special preventive attention; and (3 ) a "surveillance" one that consists of the recorded results of medical tests administered according to a schedule established by the health facility. The system is a little more than three years old. By the summer of 1972 it contained about 50 million characters of data, or approximately 3,500 characters per patient-record. It accommodates data in narrative as well as standard computer-accessible form.
The system is an elegant tool for addressing a complex set of social problems. It would be hard to argue that the patient population being cared for would be better off without the services the system makes possible: It is also apparent that knowing who an individual is, and the details of his medical history, can be of vital importance in treating patients, but the system has certain social control capabilities that should be noted nonetheless.
The surveillance component, for example, has the primary purpose of discovering incipient medical problems in individual patients. To do this effectively, each patient must be induced to comply with the health facility's testing schedule, and the health data system can be used to encourage compliance. As long as a patient has no need for medical treatment, he can avoid the testing program. However, once he becomes a patient, for whatever reason, his record will be there at the doctor's fingertips showing all tests he has not had but should be persuaded to have before he leaves the field clinic or wherever it is that he has come to the medical facility's attention. In discussing a system serving such, patently humane purposes, words like "control" and "coercion" may have an objectionable ring, but the coercive potential of the surveillance component, especially in some other area of application, is evident.9
In another environment, the statistical-reporting component of the system could also have potentially unsavory consequences for individuals. It is characteristic of modern organizations to single out "high risk" categories of people to whom the normal standards and rules do not apply. Often these high risk groups are identified from statistical studies of populations that use the services an organization offers. The consequences for any given individual exhibiting the characteristics of the high risk group may range from total exclusion (uninsurability) to being made eligible for special treatment (remedial education, free medical care). Although there is nothing intrinsically harmful in such practices, in dealing with human populations it is essential not to assume that any single member of a statistically defined group will necessarily behave in the way predicted for the group as a whole. Theoretically, the adverse consequences of "statistical stereotyping" can be avoided by permitting an individual to know that he has been labelled a risk and to contest the label as applied to him. However, depending on the circumstances-and particularly on the stake that an organization may have in being able to predict the behavior of each individual in its clientele-a lone individual could have considerable difficulty making his case.
Even the administrative record-keeping component of a comprehensive data system can have coercive effects. When the; administrative part of the health data system was described to the Committee, repeated reference was made to the advantages of knowing that a patient has previously been treated for an emotional disorder when he shows up at a clinic claiming that he has accidentally scratched his wrist on a rusty nail. One hopes, that his chances of being discharged after some bandaging and a tetanus shot are about the same as his chances of being committed for treatment as a potential suicide. But are they? Should they be? In some other record-keeping environment, could an individual depend on having someone equivalent to a trained medical practitioner available to make such a judgment?
Finally, it is important to note that the health data system has grown very rapidly, that elements like the "high risk" categorization were not present in the beginning, and that the health facility is now trying to improve its method of identifying patients for the purpose of updating and retrieving the information it maintains about them. In this particular situation, the Social Security number happens to be considered a poor identification device because many patients are thought to have more than one; but the patients also tend to have several different names, so the managers of the data system are trying to develop their own unique numbering scheme cross-referenced with all known "aliases" for each patient.
Scheduling, labelling, monitoring, improved methods of identifying records about individuals-these are being discussed in some quarters today as if they were mere tools for delivering services to people efficiently. In the health data system just described, the surveillance component is regarded as a way of providing preventive health care; of taking preemptive steps to halt the natural development of illnesses and conditions conducive to illness. It is hard to quarrel with those objectives, or for that matter with the objectives of a great many data systems now in operation or being planned. Should a national credit card service be prohibited from using a sophisticated personal data system to prevent its card holders from going on irresponsible spending sprees? Should school districts be forbidden to use personal data systems to help prevent children from becoming delinquents?
These are difficult questions to answer. Often the immediate costs of not using systems to take preemptive action against individuals can be estimated (in both dollars and predictable social disruption), while the long-term costs of increasing the capacity of organizations to anticipate, and thus to control, the. behavior of individuals can be discussed only speculatively. One fact seems clear, however; systems with preemptive potential are typically developed by organizations, and groups of organizations, who see them primarily as attractive technological solutions to complex social problems. The individuals that the systems ultimately affect, the people about whom notations are made, the people who are being labelled and numbered, have, by comparison,, a very weak role in determining whether many of these systems should exist, what data they should contain, and how they should be used.