MIT/Brown Vannevar Bush Symposium - Fifty Years After 'As We May Think' (Part 3/5)

[MUSIC PLAYING]

VAN DAM: Our last speaker today is Michael Lesk. And to tell you a bit about why he is here, as you know, memex was a personal information organization and retrieval mechanism. And whereas Michael is probably known to you as one of the original Unix hackers-- for example, Lex and Yacc and UUCP-- the real reason for his being here is that he's a deep scholar equally at home in the humanities and in computer science where he's been specializing in information retrieval.

He's the chief research scientist at Bellcore. He's also visiting professor at the University College in London where he simultaneously holds appointments in computer science and librarianship. And he's had an abiding interest in books and libraries. Michael.

LESK: Thank you very much. Of all the undeserved praise, I will in particular correct that I did not write Yacc. I'm very flattered to be here. And I thank you. The title of this talk is the Seven Ages of Information Retrieval. If we could have the first slide, Shakespeare wrote about the seven ages of man. And information retrieval, we think of it as being born in 1945.

I think it's going to work out very well to view it as a human life. That, i.e. by the time that it would be, say, 70 in 2015, we will be done. We will have achieved what Bush set out to. We will have a library of a million books fitting into your desk, at least virtually. To show you how far along we are on this, everyone in the audience think. The last time you needed to know something that you had to look up, you didn't remember it and the guy in the next office didn't know, did you look it up on a screen or on paper?

How many on a screen? Raise your hand. And how many on paper? Raise your hand. Last time, okay. It was about 75% to 80% on screens. Hands down. That is how far we have come. And I can easily believe that in the next 20 years, we'll get the rest of the way. So I'm going to talk about IR as a life with its seven stages. This works particularly well for me because I was born in 1945.

And so I think of this, right, you know, when IR was 20 years old and should have been in, you know, sort of college student, change-the-universe mode, well, you know, I was working for Gerry Salton. And we were publishing the first paper about SMART and CACM, trying to introduce free-text indexing. So it works well enough.

The other comment I use as a thread through the talk is there's been a tension throughout this life. We've heard a lot about Bush. Now, in addition to Professor Bush, there was another MIT man named Warren Weaver who was active at the same time. Weaver wrote an article in 1949 recommending machine translation. Bush, remember, is alluding in his article to the work of the physicists, the people who built the nuclear bomb and the microwave radio.

Weaver, who was another MIT man, another propeller head going out from Cambridge to rule the world, was thinking in terms of the cryptographers, the people who had broken codes with computers during the war. He thought of, well, we could use that technology on language. He said, quote, "it is very tempting to say that a book written in Chinese is simply a book written in English, which was coded into the Chinese code. If we have useful methods for solving almost any cryptographic problem, may it not be that with proper interpretation we already have useful methods for translation?"

So Bush started hypertext and information retrieval. Weaver started machine translation. But there's also a tension here. Bush thought of information organization as, people will make trails analogous to the manual indexers who had been working in libraries for generations. Weaver is talking about statistics. And throughout the life of information retrieval, there has been this tension between, are we going to do intellectual analysis-- whether it be manual or automatic-- or are we going to sort of do word counting?

It's exactly the same tension that exists in chess between the people who say, we'll try all legal moves as far ahead as we can. And those who say no, no, no, there must be a role for intelligent selection of which moves to evaluate. Add in the same way, we've had this tension in IR for its entire history. So, okay, we'll go through this. As I say, in 1945 the field starts. Shakespeare's next stage is the schoolboy, which would have been the '60s, then adulthood, experience.

Shakespeare actually goes on to things like soldier and justice and comic character on the stage. I won't do those, so we've changed in a little bit. Bush's predictions are rather interesting. If we could have the next slide, please. This is a list of some of the things Bush said we were going to have. And it will not be unfamiliar to everybody in the business that most of the technology predictions were achieved at some point.

We have instant photography. In fact, we had it very soon after he wrote. We have motor-driven cameras and we have automatic exposure cameras. We have computers that have card and film control and select their own data. What don't we have? Well, we haven't actually achieved his storage goals. We can't put a million volumes in a desk yet, although Mead Data Center has the equivalent of a million volumes.

They have 2.6 terabytes. But that's a very selected set of stuff. We don't have speech recognition. We don't have really working OCR. And some of the things that were done turned out not to be worth doing. Stereo photography has been achieved, but it's sort of a toy to sell to tourists. And the ultramicrofiche that was held up earlier today, again was achieved.

But if you're a librarian and let's say out of each $100 you budget, you spend $25 on the building and somebody tells you that regular microfiche would reduce that cost to $5 and you don't do it, it's unlikely that telling you that ultra microfiche will reduce the cost to $2 will make enough of a difference. If reducing 125 down to 105 doesn't cut it, down to 102 isn't going to cut it either. So again, we've done most of the hardware stuff. We're still waiting on some of the software stuff.

We haven't got automatic typing from dictation. We haven't got OCR. We also, by the way, haven't got the stuff about plugging your nerves directly into electronic systems. Although, I did see a story on that in the newspaper fairly recently. What I found more interesting was that Bush talked a lot about individualized systems. The way Bush envisioned his interface, each person would have their own personalized information space.

For most of the history of information retrieval, that isn't what's been done. What's been done is to have systems that look the same to everybody who uses them and which therefore provide the best access to the information that is sort of impersonal, the journal articles. But again, that's something we'll come back to later. So let's go on. We still have this infancy stage.

Now, in 1957 the Soviet Union put up the first artificial earth satellite. And that produced a wide array of fears that the United States was falling behind in science and a realization that the United States didn't even know much about what was going on in Russian science. So there was some funding of Russian language studies and machine translation. And there was a lot of funding of information transfer, to say, well, the answer will be that we'll improve our knowledge by building systems to distribute information.

There were widespread urban legends of some company that had spent either $100,000 or $250,000 to reproduce a result that had been readily available in literature, but they had not found. I once eventually saw a paper in which someone claimed that they had run the story into the ground, but it was false. But it was widely believed at the time, so people set to work.

The first thing they did was they built quick indexes. I don't know how many of you remember keyword-in-context indexes, a man named HP Luhn. If you think that's primitive, you should remember what the competition was. How many people here remember something called edge notch punch cards? A reasonable number. You know, I thought about bringing one. And then I said, where am I going to find an edge notch card today? So I made some.

And the idea for those of you who haven't seeing them is you have these cards. And there's a row of holes and you can hold them up with something through the holes. And let's say this is a bibliography, so this card has the citation for Moby Dick. And this card has the citation for Principles of Computer Graphics. And you take each hole and you assign it.

So let's say that we decide that this hole is going to be books written by Andy van Dam. So for this card, we tear out that notch. And let's say the next hole is books written by Al Aho. We haven't got one of those. And let's say the next one is going to be books written by Jim Foley. So again we'll make a notch. And let's say that somewhere we get to Herman Melville and we make another.

And now you see we have this batch of cards. And again, you can put something through the slots. And suppose you want all books, all your references for van Dam you pick his hole. You put something through. Now traditionally, you used a darning needle. But these holes are so big in the demonstration which I'm going to emphasize that this is a digital process by using my finger. And you shake the cards and the right ones fall out.

This is the retrieval technology of the 1950s. So this is what we got replaced. Now as I said, because of things like Sputnik, it was a boom time for retrieval in the '50s and '60s. More people attended SIGIR-- Special Interest Group on Information Retrieval Conferences-- in the early '60s with the predecessor conferences, actually, than attended them a couple of years ago. It's really remarkable.

The first experiments were systems that used indexing. Because things were being keyed in and they were painful to do. What happened in the '60s when we've gone into the experimental schoolboy stage was people started doing experiments on free-text retrieving, most particularly Gerry Salton, who died recently, unfortunately, of cancer a few weeks ago. Gerry was the first person to try to do large scale experiments to say, yes, free-text indexing can be compared and really work.

Gerry worked a lot with a man named Cyril Cleverdon. Cyril Cleverdon was the inventor of the recall and precision measures. For the first time, we had an attempt to make this part of computer science an experimental science in which people would run experiments. This is not typical in computer science. You know, people do not do evaluated experiments. When I wrote UUCP, we didn't say, oh, well, now let's take 20 MIT undergraduates and tell 10 of them to get a message to Brown with UUCP and 10 of them to try Amtrak or Greyhound and see which ones get there first.

We just sort of write it and put it out and say here. Well, Gerry and Cyril had the idea that, no, information retrieval was going to be evaluated experiments. Because there was this history of there is a lot of stuff out there. And we need an effective way to find it. There had been a long tradition of, well, the way to do this is with standardized nomenclature and whatnot. But nobody knew whether they would really work.

And so a lot of experiments were done. The test collections, in fact, that Gerry and Cyril prepared in the 1960s were basically still in use until about 1992 as the standard retrieval techniques. Now, this stuff was all basic keyword retrieval. As this work was going on, some new techniques that were also sort of statistical went on like relevance feedback-- the idea that, well, we'll find one relevant document and we'll use that as a set of search terms to retrieve more relevant documents.

But all of this was, again, statistical. Now, at the same time, I said there's this other thread. The other thread is intellectual analysis. And in the IR context, that was AI. That was artificial intelligence. And what we had was that people like Terry Winograd and Daniel began looking at programs that would do linguistic analysis of queries, and perhaps some documents, and attempt to match them and attempt to retrieve answers on that basis.

Now, these two groups didn't get along very well. The AI people were in the computer science tradition of we're going to build something that will make enough examples to fill the back of a thesis, and that's it. And the IR people were down this trend of, we're going to run the same 1,400-document text-collection over and over again until the experimenters memorized all the articles on aeronautics. And all this was going on, but the tension is still there.

The tension fortunately is only at the research level. Because at this point, we're still in the schoolboy phase and not that much is actually getting done. Now, then we get into the 1970s. In the 1970s, we're now an adult. The field is now, you know, in the 20s. And what's happening? The main thing that's happening is computer type setting and word processing.

You know, I appreciated Ted Nelson's videotape of real cut-and-paste. I remember that. I haven't seen anybody do that for 20 years. Once we had computer typesetting, and I hadn't realized that Bush had worked-- I mean, one of the other things I've learned from Andy's talk was that Bush had worked on typesetting and that he had worked on management theory. So that he was not only the progenitor of information retrieval, but he was also sort of the antecedent of Monotype and Linotype modern typesetting machines and of Dilbert.

But we now had, because of computer typesetting and word processing, we had large volumes of material in free text. We also had online time sharing. All of the experiments Gerry Salton did in the '60s were done in a batch [INAUDIBLE]. You wrote down your queries and sent them into a system, you got it back later. And there was a whole lot of talk about selective dissemination of information. People would write down lists of queries and leave them in the background to be run.

And this all was blown away when time sharing came in. All of a sudden, you could put up these real systems. You also had some early examples of real cooperation. OCLC, for example, this was an organization now called the-- it was originally called the Ohio College Library Center, that being an insufficiently expansive name. It is now the online computer library center.

But OCLC was founded to distribute catalog cards in libraries. And there was a cooperative effort. If a library got a book which hadn't been cataloged in the file, they would catalog it and enter it in the file and other people would then retrieve that record. And if you entered a record, you got the next 10 records free or something as an incentive to do that. These were all very limited search systems. But they were very popular. A lot of promises were made. Why don't we look at the next slide.

People believe that, you know, there was going to be a change in libraries, last chance before everything goes on microfilm. And now if we could look at the top half of the next slide please. There are earlier statements about microfilm that will remind you of some of the statements that have been made over the years by the more extreme proponents of artificial intelligence and hypertext in which people say that microfilm, micro photography is one of the most important developments in the transmission of the printed word since Gutenberg. And so what I have to say from this is that hypertext did not only not invent text as they would have you believe, they didn't even invent hype.

[LAUGHING]

[APPLAUSE]

So what happened in the '70s in the research arena? Well, again, we've still got this tension. We still have the people doing statistical processing. And now, in fact, they get a new weapon. Keith van Rijsbergen shows up with probabilistic information retrieval and introduces even more statistics to an area. I should confess that I'm not terribly sympathetic to things like statistics, you know. I was taught as a chemist that statistics are what people do when they can't get their apparatus in good enough adjustment to get convincing answers.

But we went on. And the AI people were getting into trouble now. They had made lots of promises about what they were going to do with machine translation and computational linguistics. And to get away from these, they had started making promises about speech recognition. And the thing they did they got into information retrieval was they invented expert systems. Now, the easiest way to think about expert systems in the late '70s and early '80s is that they occupied the same buzzword niche that intelligent agents do today.

And people wrote, for example, "the 1980s are very probably going to be the era of the expert system. By the end of the decade, it is possible that each of us will telephone an expert system whenever we need to obtain advice and information on a range of technical, legal, or medical topics." In fact, when I went out to get quotes like that for some talk a few years ago, it was shooting fish in a barrel. They were all over the place.

People have wildly different views about the importance of this. You know, my name is Lesk. It's a very rare name, and it's not an English word or a word in any other language that I know. When I do a search on "Lesk" in the database, if I don't find myself, it's because I found my brother or my second cousin. That's it. Stu Card is in the room. I suspect Stu Card does not feel the same way about dumb searching of ordinary strings.

There was a problem, though, that none of the AI systems really seemed to generalize very well. Roger Schank was perhaps the best proponent of AI systems in the '70s, that we're going to introduce higher-level language processing. And their idea was that every document could be mapped into standard frameworks. For example, there's a very large number of medical articles that boil down to, we have a batch of rats. They're all suffering from such and such a disease.

We gave them such and such. And some of them got better and some of them died. This is how many in each category. And Schank's group would try to construct such schemas for many areas. And then they tried to fill them out. They produced a lot of argument about whether this was for real or not. There were other systems like Bill Woods' or LUNAR System or Stan Petrick's Transformational question-answering that were evaluated, but they tended not to do off ordinary text.

So we weren't getting very far on this. But we still had this tension between people who said, no, all you need is statistics. And now intellectual analysis will help. But everyone was going away from manual indexing. They all said, well, we can't afford to do manual indexing. So if we're going to do intellectual analysis, it's going to have to be done by machines. And we've got to get machines that are smart enough to do that.

So now we get on to the 1980s. And a couple of things go on in the 1980s. We have a steady increase in word processing to the extent that it becomes impossible to buy hot-lead typesetting machines. And they only exist as sort of craft devices. And the price of disk space goes down. So everybody starts to think of, ah, these are going to be the new alternatives. There's a little data on the next few slides. Go to the next slide, please. [INAUDIBLE] any data.

This is a Berke Breathed "Outland" cartoon in which one of the characters comes over and says, you know, could I borrow-- Oh, no, he said, we're on the brink of a gleaming, digital upheaval. 500 cable channels, so many sitcoms, so little time. TVs, telephones, computers merged in one, our lives awash with instant visual input. And well, what do you want? Well, I actually wanted to curl up with your copy of Winnie the Pooh. And you can see in the last frame that the guy is here staring at a compact disk. And that's what--

[? NELSON: ?] Hey, Lesk.

LESK: What?

[? NELSON: ?] These are the slides I was gonna close with.

LESK: Sorry, well, we'll go onto the next one then, the next slide. And the next slide is why the libraries are starting to chase this so hard, why there's a business here. This is what university library budgets look like. Your typical university spends 3% to 4% on its library. Unless it's the place up the river that you all are afraid of which spends about 6% or 7% on its library. Actually, there was a nasty comment about Harvard made earlier. I'll make a nasty one back.

All right, I'm talking about the seven ages of man in Shakespeare. Everybody who knows which play it comes from, raise your hand? Got you. The answer is as you like it. But it reminds me of the story about the time the bus fare in Cambridge went from $0.50 to $0.60 and some kid gets on and he pays the old fare and walks past without realizing. The bus driver calls out and says, hey, you, are you from MIT and you can't read or from Harvard and you can't count?

Anyway. Most universities-- 3% to 4% on the library. And it's not going up. Where does the library budget go? About a third of it goes for purchases, about half of it goes for salaries, and the rest goes for other. This is not quite fair. Most universities don't monetize space. About another third would have to be added if the library were charged the fair rental value of the buildings it's cluttering up.

Of what's spent on buying a book, where does that go? The right-hand column is the breakdown of the prices of books. Retail markup is like 40%. Distributor gets 15%. Printer gets 15%. Publishing office gets 20%. Author gets 10%.

What that means is, if you say, suppose we blew away this system, suppose, you know, what Ted Nelson wants happened. We were getting stuff directly from the author to the reader. And we didn't have to bother with paying for the printer or paying for the librarian to put it on the shelf. You know, relatively little of that money is needed. So there's a lot of potential economic gain in this system if we can do it.

Can we do it? The next slide is the price of disk space over time. And this is a chart of how much disk space-- And you see it's going down very nicely. And now I'll tell you what the people in the back can't see, which is it's a log scale. Between 1971 and now, we've had 100,000 fold decrease in the price of disk space. So that's why these things are becoming possible.

Next slide is, as a result, the increase in the number of databases. But an interesting thing, the top green is the number of online commercial databases. The red line near the bottom that's going up much faster is CD-ROM. And then the mag tape is the blue line that's actually going down. So what we're seeing is a switch to CD-ROMs, which are becoming increasingly popular. CD-ROM is one of the big inventions of the '80s.

Let's go on-- well, let me just talk for a minute about the '80s. Okay, by the '80s, we're up to the 40 year mark, right? So the world is now getting mature. And what that means is, yes, these systems are commercially available. Lots of people have personal accounts on DIALOG and CompuServe and they're looking things up. And people are actually getting used to OPACs. Ordinary people walk into libraries now and they get met with computer terminals, not with card catalogs.

Some of them write nasty articles for The New Yorker magazine and make trouble for us. But basically, most people are pretty happy. What is annoying people is that the research community has been cranking along all these years, right? And they've invented probabilistic retrieval and relevancy [INAUDIBLE]. And none of it's in use. All these commercial services are doing dumb free-text searching. They don't think they need either intellectual analysis or even good statistical analysis.

The AI community, meanwhile, is down on this expert system and knowledge representation language stuff that imagines translations into unbelievably sophisticated and complicated languages. I mean, Feigenbaum got the Turing Award last year. I think he should have gotten a special award from the Department of Commerce for sending the Japanese down the fifth-generation rat hole. And the enthusiasm, in the early part of the '80s, this stuff was riding high.

In the middle of the '80s, we got what is called AI winter. And we have now gone into a world in which instead of people believing that it was possible-- I should say, the idea that you can take natural language descriptions of subjects and turn them into a single artificial symbolism predates artificial intelligence. There's probably one other person in the room who remembers the name JJ Faraday, put your hand up.

Dick? Where's Dick Marcus? You must know Faraday. He's the only one. Faraday was this British information retrieval guy who came up with this amazing nine operator language with special symbols for each operator, you know, and he believed that if he encoded something in this language, he would get 100% precision in recall. Everything would be perfect.

Well, it's the same thing as translating into [INAUDIBLE] or some language like that. Most people now are sort of almost back. There was a famous linguist named Benjamin Lee Whorf. Whorf had a theory that, in fact, language constrains thought. What you will think depends on the language you use to express it. And many of us are going back to believe things like that. Okay, so now we've made it through the '80s, things are going fine.

Basically, the intellectual analysis people are in full retreat. The statistical people are riding high. All the systems that are running are dumb. And we get into the 1990s. And the world is now, you know, in the late '40s, and it's time for the midlife crisis. What is happening? Well, what happens is the internet. Even one year ago, there were people at Bellcore saying to me, only 15% of the computers in people's homes even have modems. Who cares about the internet?

Nobody says that anymore. I now see charts suggesting that by 2003, the entire population of the world will be on the net. Now, what is remarkable is that everybody is providing it and everybody is classifying it themselves. You know, it's the revenge of the Bush people over the Weaver people. The Weaver people have been winning for 40 years. They've almost won. And now all of a sudden the Bush people make a comeback. And there's all these people, you know, organizing their own stuff.

Now, admittedly, there is a lot of problem with quality on this. You know, I used to be taught as an example of probability, the statement that if you took a million monkeys and sat them in a million typewriters and gave them long enough, they'd write all the works of Shakespeare. The internet has proven that this is not true.

[LAUGHING]

[APPLAUSE]

Sorry? There's this vast amount. And the question is, how do you sort through this? I mean, the Lycos people now think there are 10 million pages on the net or more. You know, everybody now thinks, you know, in the '80s, I first met with the expectation that everyone is supposed to have a fax machine. And now you meet with the expectation everyone is supposed to have a home page.

The world's hard disk industry will ship over a terabyte of disks this year-- two megabytes for every person in the world. I can remember not too long ago when the people at Bell Labs, not a stingy institution, were bugging me to keep my disk space below 200 kilobytes. The next thing that we'll talk about is, well, what are we going to do to get some of the quality valued information into the libraries.

And one of the answers is scanning. One of the big things that happened in the '90s, is the rise of scanning. So the next slide shows an example. I figure I have to sooner or later get to my own research. This is a sample of the core information retrieval system which we built at Cornell. This is based on scanned chemical journals. This page is an image of a real page. This is also an image.

You're looking at a picture of the page. Somebody keyed the page in in Columbus, Ohio, took the computer typesetting tape, printed it on paper. I took the paper, fed it back in through a scanning machine to get that. You may say, that's dumb. The next slide is the alternative. It is an ASCII thing, in which in this case, it's a different Bellcore retrieval system.

But in this case, this stuff is regenerated from ASCII. We did some experiments-- or actually "we" is Dennis Egan who's sitting in the back somewhere, people can read both of these about equally quickly, so they both work. And they both work if you have to search for something a lot better than paper. So you now see a lot of things coming out in image form, a lot of libraries doing things.

Some more examples of how you get things in. The next slide shows another alternative of how you might get things into the digital library. Mozart writing a digital version of Symphony number 38. This is not what you do. Let's see, what's the next? Put up the next slide. The next slide is an example. I was helping some people who wanted to digitize Charles Sanders Peirce.

So he wrote 100,000 pages of manuscripts such as that at the bottom. And you could scan it in, and the scholars could read it. I should say the sample here is the page when he was applying for money to the Carnegie Foundation to publish his works in 1900. They turned him down. In 1992 about, the people who wanted to scan all his manuscripts applied to the Carnegie Foundation for money, and they turned down again. So scholars never learn.

Let's skip the next slide, go on to the one after that, which is an example. Once you get into images, you can do lots of other things. This is a British Library 1771 map of New England. The black is the original map. The red is an overlay of where the boundaries really are, assuming that the geological survey today knows what it's doing. And if you look at this, you'll see that the latitude is reasonably good, but the longitude is somewhat mucked up, which is what you'd expect.

The next slide is another scanned map. And this slide is one of the reasons why you're not seeing these, you know, from some-- I'm sorry, rotate it 90 degrees, please. Right. This is Manhattan Island in 1775. And the reason that I'm not really doing this on a SparkBook-- the image from which this comes is 50 megabytes. And it takes a little bit long to put up. And the view graph goes up very quickly.

Simply, as a matter of attractiveness, we'll put up the next slide, which is another map, Lord Jeffery Amherst's map of New York in 1770. The next slide is to move on to still other media. There's lots of talk about other media. I like to listen to the radio. So I've got this system where I've got a radio plugged into my workstation permanently tuned to the public radio station in New York.

And every day it digitizes "All Things Considered" and "Morning Edition." And I can listen to them at my convenience later. And I can actually listen to them faster than normal. I can also clip out interesting things and save them. And I can do things like segment by looking for silences.

The top bit is that I also like to listen to the BBC. Well, you know, they don't really broadcast a strong enough signal. So a colleague in London has a radio plugged into her workstation. And it's tuned to BBC Radio 4. And if you ever wonder why the response on the net isn't even good at 4:30 AM, it's because there are people like me dragging over the BBC from London so they can listen to it when they get to work late.

Let's see, other things. Bush talked about OCR. The next slide is some random attempt to evaluate OCR. And the top is a newspaper story which would be perfectly readable to you if you could see it. It got 80% of the words right. Better printing you get 90%. But if you're counting by words, we still don't have that. We need to get there.

The next slide is another example of why you would like to have images. And the problem, "cat," C-A-T, three bytes. That little frame maker's sketch was 1,000 bytes in frame maker. The picture on the right is 12,000 bytes. So a picture is worth a thousand words, but it costs it. I said the price of disk space had gone down about a factor of 100,000. The problem is the difference between video and Ascii is about 50,000. So nearly all of the progress has been given back.

What does that mean? In the 1960s and early '70s, all sorts of scholars had key punched one thing, they had their copy of Paradise Lost or something that they had keyed in that they were doing research on. And right now it's the same way. I meet people who have their 45 seconds of digital video that they're experimenting with. So, all right, based on past history, another five to 10 years we're going to have really large digital collections which will be video collections, which we will have no idea how to search.

So, all right, what is happening with that? Well, research. Interesting thing with research-- there is suddenly a text retrieval industry. All of a sudden-- well, actually, let me back off. I want to talk a little bit about the libraries again. And they're getting all this stuff. And next slide please. Here's another quote for the future, you know. "Libraries for books will cease to exist." To be true in 1984, predicted in 1964.

No, you know, it didn't happen. Today, it's another story. What are the relative costs? Scanning a book-- well, let's put up the next slide, please. Scanning a book costs about $30, between $30 and $40. And you need another $10 to pay for the disk drive to hold it. To build a space to put it on the shelf, Cornell-- $20 for a book in the newest book stack they've built. Berkeley is building a stack at $30 per book.

The top building there, the British Library in London, will come in at $75 per book. And the French National Library is $100 per space for a book on the shelf. So the libraries are now in a situation where within a few years, it will be cheaper for them to scan and not put up the space to put the book. And since many of them are under strong pressure not to put up any more buildings for other reasons, you're going to see an awful lot more of conversion.

It's already economical if any libraries could get together. But they can't do it right. Let's see, we also have a text retrieval industry. Next slide is the online search industry. We're at a couple of billion dollars a year now among the different vendors in which LexisNexis is biggest and Dialog is next. The next slide is the software sales to support that industry.

This is from 1990 to '96. The rate of growth in selling programs like Personal Librarian and DynaText and things like that. So this is a thriving business now. And by god, all of the research technologies that have been ignored for so long are now in use. Bruce Croft's Center for Intelligent Information Retrieval, which has developed a lot of intelligent text processing algorithms, provided the software for the THOMAS system at the Library of Congress. Gerry Salton's software was used to make a CD-ROM encyclopedia.

The Waze system has relevance feedback in it. So we have an active industry. And by god, it's finally using the research results. Politically, we have a big thing. Vise president Gore, as people probably know, decided that since his father had started the interstate highway bill through Congress, he would do the internet and the national information infrastructure as the analogous thing. And he constantly talks about his school child in his hometown of Carthage, Tennessee being able to turn on her computer and plug into the Library of Congress.

And so the feds have started a major digital library program. Do me a favor, skip the next two slides and put up the one after that. The black spots here are the federally DLI groups at Stanford, Santa Barbara, Berkeley, Michigan. Some other projects are in green. For example, the JSTOR project in Michigan is the Mellon Foundation's attempt to see whether it really is true that libraries can scan as a replacement for shelf space.

And what Mellon is doing is paying to scan 10 major economics and history journals back to the beginning of time, of their time, and seeing whether these libraries can use these instead. We also got a return to evaluation. I said that for 30 years, everybody had been using the same collections for evaluation. And Donna Harman started running something called the TREC Conference in which people sent in hundreds of queries were run against a gigabyte of text. And you suddenly got some realistic numbers.

And what we learned, not surprisingly, was that there is still enormous scatter in the performance of these systems. The reason we can't agree on what a good system is is that it depends very much on the query and the user. You could do enormously better with best hindsight. If you could say for each query, on this query, we'll use that search system, on the other query, we'll use the other search system. We don't know how to do that yet, but there's clearly gains to be made.

So as I said, the midlife crisis is, all of a sudden this is succeeding, but it's succeeding with manual content analysis rather than just the statistical retrieval. They're actually both working together, sort of the Bush disciples have made a comeback. All right, so what happens next? The next decade is the 2000s. IR is going to be 55 to 65. So this is fulfillment, right? We should be [INAUDIBLE] away the money for our retirement.

And our problem is going to be, how are we going to do the image? We can get all these images, video, it's bad enough indexing the text. At least we have free text retrieval. What are we going to do with all the images of the video? We've got to have manual analysis for that at the moment. I don't know how we do it. That's going to be the big issue.

But it's going to have to rely on people doing it, at the moment. And that's going to require us to learn how to manage manual indexing after many years in which librarians were sort of viewed as, you know, well, librarians are sort of the equivalent of Quill pen makers in the typesetting era. Suddenly, we need these people. We need people who know how to organize information and make use of it. Okay, but I think we can get through that.

Finally-- 2010. This is-- I've labeled it retirement on my view graph. Put back the original view graph, please. Shakespeare had it as senility. I prefer to be more optimistic. At this point, I think that, basically, the conversion job has been done, that by 30 years, you know-- well, by this time, we will have had enough of the scanning and enough of the keying available and we will have a market in which people can buy this stuff that we will basically have the material we need online.

We will need to know how to find it. But we will have it there. And we can all go off to study biotech or something. The printed books remain sort of in warehouses somewhere. The library buildings on campuses have been turned over to more bureaucrats and administrators and something else.

What might go wrong with this? I'm very optimistic, but what are the problems going to be? One of them is internationalism. We're doing just fine now on the internet on the assumption that everybody in the world speaks English. At some point, that's going to break down. And it's going to be political as well as technical. There's going to need to be more new kinds of research.

I am a little worried that 10 years from now, there will be library departments giving PhDs in arcane details of probabilistic indexing, carrying out the arguments that Keith van Rijsbergen had with Gerry Salton in the 1970s. You know, we've got to get onto some of these things. There will be a few of us old fogies who even remember that we were promised automatic language parsing and question answering. We're still waiting for it.

More seriously, I'm worried about some of the social effects. CB radio, anybody remember CB radio, right? You know, lots of trash. How do we keep the internet from being overwhelmed with that stuff? How do we get a role for the people who will sort out what is reasonable and what is not and let us find that stuff? That is going to require some form of charging. Bob Kahn already said, we need to go to service charging.

We need some leadership in how to do this right. My problem is that, you know, it's hard to look to economics for leadership. Even the Economist magazine once said that an economist was somebody who was good with numbers but lacks the personal charisma to be an accountant. And there is a problem of where do we get the leadership to build a system, such as the one that Ted Nelson has envisioned, where people actually do get paid and it works.

At the moment, the copyright law is a real problem. You know, somebody made some comment about, well, we needed more legal stuff. I mean, I have been going to computer conferences since 1969. So I remember the days before there were lawyers in computing. I never heard at one of those conferences anybody suggest that we had a problem in computing because there weren't enough lawyers paying attention to it. There is a story that--

Okay, oh, wait, no, that's right. IBM did prepare a CD-ROM to commemorate the 50th anniversary of the Columbus voyage to America. There is a story that they paid over a million dollars to clear the rights for the material used on that CD-ROM. That's okay, but only 10,000 of it was paid to the rights holders. All the rest went into the administrative costs. We have to get somewhere.

There is a real possibility that the world of the future is everything published after about 1990 is available because the publishers have it and they're making it available. Everything before 1920 which is out of copyright is available, and everything in between is falling into a black hole in which every once in a while, someone tries hard to find the people who have control and get the data.

Finally, as I alleged in an earlier question, I'm worried about political problems. There are technologies which looked as if they were going to be successful and have been stopped by political or legal liability issues. As I say, nuclear power, childhood vaccination. We need to try to design our world so that doesn't happen to us. All right, but I don't want to end on an unhappy note. You know, it is possible that we will all, you know, be drowned in information water or something.

But I think that it will actually work. I think given the rates of progress, Bush's dream will be achieved in one lifetime, in the lifetime of this profession. Now Bush actually talked about the information organization. He said there will be a new profession. There will be trailblazers, people who make trials on request. Now, part of the problem is that today people think of librarians as someone who alphabetizes books and puts them on a shelf. The function that is known in a library as "mark and park." You label the book and stuff it on the shelf.

We need to get sort of a higher status for this, whether it's called information trailblazer. I'm not as good as some people at making up new words for things. I'm optimistic about that. Once upon a time, accountants had to be good in arithmatic. Then computers came along and made skill in doing arithmetic totally irrelevant to the real world.

Did that mean that accountants became uninteresting minimum wage people? No, accountants took over all the major corporations. Bean counters now run the world. So if computers go in there and say, all right, alphabetizing is no longer interesting, what happens to librarians?

And I hope that, you know, information becomes more valuable. Somebody earlier talked about information as a sea. All right. The purpose of librarians and trailblazers, whatever, the purpose of those people in the future is now going to be to navigate. It is not going to be to provide the water. That's it. Thank you.