MIT MechE Symposium: Mechanical Engineering and the Information Age - Douglas Hart, C. Forbes Dewey Jr., Peter T.C. So and Sanjay Sarma

[MUSIC PLAYING]

HART: Anyway, [INAUDIBLE], who had some very exciting things to talk about from then rising star Ron Adrian, who was at University of Illinois. And that's how I-- it was that talk. And in the back of the room, I heard-- Hans Liedman, who happened to be sitting in the back of the talk with me, commented about picture physics. And I later found out that it was a term coined by von Karman, who detested any kind of imaging and viewed it as pseudoscience.

And so now I display it quite proudly. With the invention of CCD cameras and computers, we've gone from picture physics as a term of disdain to one of-- well, a badge of honor. The very heart of all machine vision applications, as I'll talk to you, is founded in correlation technology. And correlation technology, that is-- it's the purest form of image processing to some extent, because it's based entirely on the statistics of the intensity profile.

And so everything from video compression, where one is trying to detect the motion of an object in a plane of view, to pattern recognition, where one's trying to measure sizes of objects in production and so forth are based on correlation. It's not used very often. And the reason it's not used is it's very computationally costly. And it's slow. And it has some other-- and because of that, people abandon it and try to search for other ways to accomplish the same process.

And one of the things we discovered-- or I should say, discovered early on-- was that in-- if one's trying to do local correlation for whatever reason, pattern recognition, sizing, there's a lot of redundant information. And one need not resort to standard techniques, which are FFT spectral correlation, to do this. FFT spectral correlation was always believed to be, by far, the most computationally efficient method of doing this.

What I stumbled into was the fact-- the idea that, well, there's a lot of redundant information. And the reason you use that redundant information is to develop a very high statistical probability of finding the correct solution in a local correlation. And if one goes back into information theory and says, well gee, the information content of a single pixel is really the probability-- has to do with the probability that that pixel, or that intensity, exists within some region.

And so pixels with low probability have very high information content. And pixels with high probability have low information content. For instance, if you had a black image with a few white specks, the white specks contain all the information, and the black specks have very little information. So with this idea, I said, well, why not compress the image and correlate only on the high-probability component of the image? And one can, in fact, do that. And the trick is, how do you do that? And I won't go into the details here.

But typical images can be compressed tremendously in this way. 100 to 1 is not uncommon. You don't recognize what the image looks like anymore, but the image can correlate quite accurately. And what you find out is that you get about 1,000 times faster than FFT processing if you do that.

It's even better than that. And the reason it's better than that is because it involves integer mathematics that are very, very well-tailored to hardware. Right now, on a standard PC, we can get about 10,000 vectors a second over a 32-by-32 block. And we have a hardware device that we're actually putting together right now which we're assuming a million vectors a second. So this is extremely fast by today's standards of being able to process.

So what can you do with it? Well, we didn't stop there. And the reason we didn't stop there was we said, well, there's got-- one of the problems with correlation is it's a statistical process, and therefore there's a certain probability that you'll find the wrong answer. And that became quite awkward for us. So we developed yet another technology.

And that is, if you take two adjacent regions, and you correlate those regions, and then you multiply the correlation table element by element-- you're taking a zeroth-order correlation of a correlation. If you will, it's a second-order correlation-- what you find is that all the noise is eliminated. If this is a correlation table and this is the correlation peak, perhaps and you-- anything that doesn't correlate within both areas then gets eliminated. And you end up with a very, very clean signal, a very high signal to noise ratio.

One of the very interesting things, it turns out, is that not only does it give you a very high signal to noise ratio, but it has a very wonderful property in that, if these are two regions I'm going to correlate and I have some sort of gradient and displacement occurring between the regions, then the correlation table yields a gradient that's in the direction of the gradient-- or sorry, crosswise to the gradient-- of the direction of the displacement. Then if you multiply the correlation tables together, then what happens is this part doesn't match with this part. And this part doesn't match with this part. And it forces a beautiful Gaussian-- that's not Gaussian, but I can't draw-- Gaussian profile that occurs at exactly the interface.

So not only do you have this advantage where you've boosted your signal to noise ratio-- it actually goes as an exponential function-- but you've also managed to narrow the exact location in which that correlation occurs. So you know precisely in the image where that is. So now you have a method in which you can have-- you have a very, very high probability of being able to locally correlate. And you have a very fast algorithm.

What can you do with all this? Well, I said I was a fluid dynamicist, and that's how I got involved in this. And indeed, what you're looking at, if it were in a little bit better focus perhaps, is an image taken from a standard off-the-shelf CCD camera-- nothing fancy. And these are 60,000 independent velocity vectors of a swirling flow undergoing a sudden expansion.

This is resolved down to the limits of what we can image. And I think if you look at just one little, tiny section, you can see just how fine the resolution is. We can pick up individual vortices and so forth. And we can do this-- you know, this was 6 seconds of processing to do. Our new system, we're hoping to do this in real time-- video rate being real time.

So the technology is quite exciting. With this kind of accuracy, this kind of resolution, one is able to resolve flow structures, in this case, to amazing levels. We're not stuck in 2D. We can do it in 3D.

There's a holographic cube. It was done by Joe Katz. It was processed by us. There's over 10 million vectors in this cube, just to give you an example of the type of stuff we're doing.

So that was-- I'll get back to this technology in a second. The other thing we became interested in was a technology of laser-induced fluorescence, which, if you look at the information content within an image, laser-induced fluorescence is very hard to beat. It has-- holography and laser-induced fluorescence, or LIF, have the highest information content one can achieve with a camera. But the problem you have when you try to implement this in real life is that it requires some sort of light source. And that light source always becomes distorted. And so it's very hard to use as a quantifiable measure other than to track some velocity, or displacement, or such.

Well, the biologists loaned us an interesting technology for measuring calcium. And we adopted this. What we did was we went in and we put two different dyes into the oil. And the dyes have this interesting characteristic. If you look at-- and here we were using it on oil. So that's why it says "film thickness." Although it doesn't-- it could be-- it could be intensity of the laser, if you will. And here's the emission. And the dye goes up. And then it saturates at some point, either with the intensity or, in this case, film thickness.

It turns out that one can use the ratio between these two dyes within the thing to eliminate the effects-- or non-uniformity effects-- of the imaging properties. And in this case, we're using the non-linearity part of the two. And if one takes a ratio between the two, this ratio becomes a very strong function of film thickness.

And we built a camera system. This is a dual camera system. It's two thermally-cooled 12-bit CCDs. They're coupled through what's called a multi-image module, which is made primarily for microscopes. We added a lens on the outside of this. And we have a dichroic mirror on it. And we can separate the two colors between the two cameras.

And when you do this-- just to give you an example, here's a case where we have some varying film thickness. And you can see the rings that we caused from our optics of our laser system. And when you take the ratio of this, it completely flattens out the profile. And one is able to measure the relative film thickness simply from this ratio very accurately.

One can also-- well, let me-- before I get on, let me show you another-- an application of this, one of the things we did just for fun. But it actually turned out quite interesting. We put a window in the side of a diesel engine. We fired a laser into the diesel engine as it was running. We imaged it with our camera system. And we were interested in the oil transport through the ring pack as the engine ran.

And we were able to measure oil film thicknesses accurately down to the half-micron level as the engine's running. This is actually a movie. I don't have it with me. But there's a very interesting phenomena. It became a best seller in Detroit, I heard.

It's so accurate, in fact, that you can measure the hook on the ring, which is-- it's about, oh, maybe 200 microns or so wide here. And you can detect the oil film thickness in the half-micron range within that ring. We weren't stuck just measuring oil film thickness. We can do the same thing, and we can measure temperature.

Here we have characteristics of two dyes. We have one that's thermally stable with temperature and one whose fluorescence drops with temperature. You can mix these and take a ratio in much the same way we did oil film thickness. And if one does this-- here we have oil film thickness. And we have a fixed temperature. And you can see the gradient due to the film thickness changed. But you can also see the laser rings. And when you take the ratio, you get a beautiful, flat profile that shows constant temperature.

And in fact, we calibrated this. There are a number of problems in doing this technology for temperature, not the least of which is nobody has any information on temperature effects on dyes. So it requires-- but we've been able to calibrate this relatively accurately. You see this is from 0 to 200 microns, oil film thickness. We're using this in seals as our primary application, looking at seals and bearings.

So once we had this technology, we went back to the correlation technology and said, gee, why are we stuck doing it this way? Why not take and manufacture small spherical particles that have two dyes in them? And then presumably, we'd be able to measure the temperature of a fluid at the same time we can measure the velocity of the fluid.

And so these are pearl polymerization of two dyes, pyrromethane and rhodamine. And these are about-- they range-- I guess they range 3 to 20 microns in diameter. And they're made using an ultrasound process.

And lo and behold, this is a-- we imaged it with a stereo camera pair. This is a hot-- or cold fluid entering a hot tank. And we got the three-dimensional velocity profile of the jet, and we get the temperature profile as well.

Just to show you something a little more dramatic, there is the swirling flow coming in, sudden expansion. And the hot fluid is swirling and coming up into a colder fluid. And you can see how it mixes. So we can image that.

Well, this was all well and good, but fluids was a very limited application for all this technology. So we got sidetracked. And we came up with another device. And that was, why couldn't we use this correlation technology at such high speeds and do 4D imaging, basically 3D imaging in video rates?

We came up with a unique device. We have a camera system that has a rotating aperture within that camera. And we project a speckle pattern onto an object. We have this rotating aperture that rotates. And then as we-- if you look at what happens in this system, it's a wonderful way to image. And the reason is if you have the aperture on one side, you get one view of the image. If you have the aperture on the other side, you get another view of the image. If you're at the focal point of the-- sorry, focal plane-- of the camera, both images from this aperture position and this aperture position end up exactly in the center, same spot.

But if you're out of focus, they are displaced by some distance. And you can correlate these two-- the images taken from the two aperture positions and measure this distance through correlation. And one can extract the x,y,z component of the image.

The beauty of this system is, if you're looking at a stereoscopic system, you have optical occlusion. If you're looking at a system like this, the precision in the alignment occurs entirely due to the grinding of the optics, which is highly precise. We're using off-the-shelf lenses now, Nikon lenses. And we simply attach a rotating aperture on the back end of these cameras. And so it's very low-cost. I'll show you one. We have a three-dimensional camera that's under $100 that I have sitting here.

This is a fancier version, and then the one I'll show you. This is our latest lens that we're trying to get down to the diffraction limit of the optics so that we can get very, very high-accurate imaging. Our little $100 camera, which I'll show you, is capable of doing this at video rates. This is a very crude, cheap camera-- nothing fancy. This was our very first attempt ever at doing this.

And you can image a face. We actually have human faces. I didn't want to put mine up here. Actually, the funny thing is I'm doing this with a group that's in Taiwan. And they imaged my face. And the problem was, as they put it, I'm too prominent, meaning my nose was stuck out too far. And they had trouble with the correlation algorithm that we had.

But anyway, we can image this. You can rotate it three dimensions. The applications of this range from-- what they're interested in and where the money, I hate to say-- video games all the way up to heart surgery, and putting it in endoscopes, being able to go into probes. We can do this extremely low-cost. We could put this on top of a PC. We've done this many times just as a demonstration model, as I'll show here. But there's any number of applications.

I'm interested in this technology. Because once you image in this way, things become no longer an image in a computer, but an object in a computer. So you treat the world as objects, not images. And that's a very powerful idea. Computers are wonderful things at image processing. But now you have something where everything in it is a solid model of something where it can be rotated, manipulated, transformed, so forth.

Let me-- because I'm running out of time, I don't want to run out-- over and get you back on schedule. I don't know whether this is going to work or not. I have a-- I have-- this was the camera that took that image. With lens, it's $100. It's a very cheap device.

And all we did was take this-- you can see it's a very small aperture too. So it's not particularly accurate. A good aperture lens-- we're now using f/1.485-millimeter lenses, which have far better accuracy. So this tiny, little lens, in the middle of it, we've stuck an off-axis iris. And we've just connected it so that we can rotate it. And if I can set it down here, perhaps I can image something that one can look at and see how it displaces.

Focus in. I can just rotate it. But if I rotate the aperture, you see how the image is shifting? It shifts in a circular pattern. And you notice the background is shifting in the opposite direction. I don't know whether you can see it through my shaking. Not sure whether that was apparent.

But what's nice about this technology is it's 30 hertz. So we can image and get 3D. But if you sit there, the aperture is rotating. So it's taking different information each time it takes an image. And so as you set it there, the accuracy of your image becomes greater, and greater, and greater. So it just sort of fades in.

You get a fairly good image with just two. And then you get three, four, five, six images, and the accuracy improves with each image. And it almost goes down to whatever the distortion in your optics are and your ability to calibrate out that distortion. George will show a really fascinating device where he's eliminated a lens completely. Our objective here is to develop a device that's robust, fast, low-cost camera for the masses. Anyway I'll take any questions if I still have a few minutes.

MODERATOR: Thank you very much.

[APPLAUSE]

We do have time for a couple questions, so ask away.

AUDIENCE: [INAUDIBLE] for sale--

HART: No.

AUDIENCE: --in the market?

HART: --but I'll sell you one. No, the company-- or actually, the group I'm working with want to sell it. Currently, it's interesting, they got cheap on us. And they only bought the patent rights to it for Taiwan. And we still-- MIT still holds the rights to the United States, and Europe, and Japan, which is interesting. Because all the-- they paid for the development of it. And all the users are in-- are outside their realm. So yeah, we're in a good shape on this one.

MODERATOR: More questions?

AUDIENCE: What kind of-- in a microscope context, what is the resolution in the z-axis?

HART: The resol-- it depends greatly on your optical setup, is really what it amounts to.

AUDIENCE: [INAUDIBLE] diffraction limit.

HART: The diffraction limit is-- the z direction is the diffraction limit in the same as the x, y. So you're about 10 microns. That's if you push the technology to the extreme. In a practical system like we're talking about here with a really low-cost imaging camera, nowhere near diffraction limited of course, for the size of a head, our z direction is the same as the x, y. So if you have 512 pixels across the face, you have essentially 512 pixels in depth as well, equivalent to. Yeah?

AUDIENCE: Camera scans at 30 hertz?

HART: Yeah, this camera scans at 30 hertz.

AUDIENCE: How fast is the aperture moving?

HART: The aperture moves roughly 90 degrees every image. So it rotates. We don't worry about blurring, because it correlates out. It's blurred in the same direction. So we just take the blurring and live with it.

We have plans for our very high-speed application where we take a burst of data. And there we use LCD high-speed apertures. And instead of rotating, we just fix them in two spots and just go click, click, click, click, click, click. And hopefully we can get high-speed motion that way.

MODERATOR: OK, well in the interest of keeping on schedule, thank you very much. [INAUDIBLE].

[APPLAUSE]

I think you can see from this why optics is so important in mechanical engineering instrumentation and applications. For getting information out of a system, you really can't beat visible light. I mean, also, it's amazing, the kinds of applications it suggests once you get going down this road.

So our next speaker is Forbes Dewey. Forbes is professor of mechanical engineering, professor of bioengineering at the bioengineering division at MIT. And he's going to talk to us about-- [INAUDIBLE]-- IT and bioinformatics.

DEWEY: The opportunity to speak to you today was something that I really grabbed onto. Because I think it's an opportunity to really pay tribute to an investment that Nam Suh made in the department a number of years ago. It was a real risk when he went out and encouraged everybody to get involved in the information technologies, and in particular, invested in a lot of young faculty coming into the department who had information technology as one of their key tools. And in my opinion, that investment has paid off marvelously.

And I think it's hard now, looking backwards, to realize what a risk it was at the time. So this is the first opportunity I've had to really pay tribute to what I think is one of the outstanding decisions in the department over the last 20 years. To me, information technology is absolutely essential to deal with the enormous amount of data that we have streaming in from every different corner of our technological universe. And what I'd like to do today is to talk to you a little bit about bioinformatics and some of the work we've been doing to handle all of the biological and medical information.

I want to go through just a couple of quick slides to tell you a little background. This is the-- oh. This is the team-- I lost my-- this is my team of students. Ngon Dao is my lead information architect. And Pat McCormick was previously.

Pat is now out with a startup in Silicon Valley called Tellme, which will be going public in about six months or less. So you should watch for that. He was the seventh employee out there. And I have already been dunning him about contributions to the Institute when that happens. He was involved in the 50k competition at MIT.

And Ngon Dao, in fact, his team won the 50k competition at MIT this past year. And as a comment to some of the ideas that Alex d'Arbeloff was talking about at lunch, I think that that's a very innovative program where, in fact, the students, in order to compete, have to go out to the marketplace, and understand the needs of the marketplace, and formulate solutions. Because the prize is they get $50,000 from a series of judges who come out of the venture capital community.

And in fact, they have an opportunity and an encouragement from that community to go out and commercialize the ideas that come up with. This whole 50k competition has been-- is now being replicated by many, many other universities. And I think it's a marvelous incentive for students to go out and, in fact, make that connection between market need and technology.

I'll just skip a couple of slides here. The International Consortium for Medical Imaging Technology is something we started in 1990. And it has been one of the things that has sponsored a lot of the research I want to talk about today. I'd like to go through, with you, about what the needs are in the bioengineering, and in particular, in the large data sets that you get in medicine and biology, describe those information objects, talk about the technology that we can use to organize that information and query it, talk about several solutions that are possible, and then just quickly, run through a series of projects that we're working on at the present time.

Now why do we need information technology? Well, first of all, the amount of data that is coming out of the medical and biological communities is just enormous. No longer do you take a whole week to do a single experiment. With a chain chip, you can do 10,000 experiments in one afternoon. So the stakes have just changed by orders of magnitude. And our ability to deal with all of that information in a rational way has not.

Second thing is that the information you need to make an intelligent decision-- for example, in trying to do a rational drug design-- requires that you pull together pieces from many different databases, many different models, different experiences. And in order to do that, you need an architecture that will support that kind of query. Finally, you can't do these systems one-off and expect them to be robust, and secure, and complete. They're going to break the minute-- in an academic setting, the minute-- the graduate student leaves. It will never get rebuilt.

And in an industrial setting, the minute that the person that built it gets pirated away, the company struggles in patches for the next five years until somebody redesigns a new system. You can't afford to do that. What you really need is an architecture that will survive.

Just to give you some examples, an MR image of 256 by 256, by 2 bytes per pixel, by 40 pictures, 40 slices is about 5 megabytes. If your medical record contains 1,000 pages, 400 words per page, and 10 bytes per word, you've got 4 megabytes. So one MR image is basically worth all the written information about you in the medical record in terms of space, in terms of the ability to query, and so forth.

To give you an example, Beth Israel hospital has saved, for the last 16 years, all of the patient encounters of all the patients that have gone through in terms of the textual information that's been involved with it. That total, which, a few years ago, looked huge, is 18 gigabytes. That's smaller than the disk drive that was just installed in my-- in the computer in my lab, actually, yesterday. So these things are changing very rapidly.

Now let me tell you why this is a multimedia problem. And here's a simple example out of Neil Pappalardo's area. This is an X-ray. This is just a schematic of it. And here are the annotations about that X-ray that were put in there by an expert. And this is all useful information. This is the X-ray. Somebody else can pick up the two.

But those two pieces of information are disconnected. And it's a very simple matter to try to key that textural information to the image itself. This increases the information content. It allows you to distribute and use that information in a much more open system. So this is just a very simple example. You can get much more complicated when the image analysis becomes more difficult.

So just to summarize, we've got-- in the whole bioengineering and medical community, we have clinical information and hospital records that may be important. We have engineering data on materials. We have engineering models of the cardiovascular system. We have chemical models, biochemical models of reaction rates and so forth. And we have various experimental results where we've gone out into the laboratory with gene chips and otherwise to get data.

There is also the entire biological sphere, which includes, as I mentioned, the gene chip arrays and the human genome database, which is now being completed. There's just an enormous variety of data here. And our challenge that we've been looking at is how to connect all of these different ones.

Let me give you another example that is a case that we're working on locally with Brigham and Women's Hospital. This is a drug discovery and drug testing protocol where we have a hundred patients. Each patient gets measured 20 times. All of these patients have multiple sclerosis. And we're trying to look to see how the lesions that you can pick up with appropriate image analysis change over a function of time.

The data mounts very rapidly for this, to tens of gigabytes. We also have an equivalent amount of information after the images have been analyzed. And what we're trying to pick out from that is, in fact, whether the frequency of the lesions, and the size of the lesions, and so forth will change with the use of one drug or another, and then to correlate that with information in a genetic database.

Now I have a little movie here. And you have to watch very closely down in this lower right-hand corner. Because unfortunately, it goes very fast. I've not figured out a way to slow down my computer. Normally you like a fast computer. But in this case, it's too fast. Let's see if we can-- uhp, there we go.

What you were looking at in that was, in fact, 20 different time points for a single patient, all of the images re-registered exactly by use of image analysis techniques. And then each time point took about 10 hours on a large Sun machine to do the segmentation to get the individual lesions. Nonetheless, that's the kind of data that are going to be used to effectively do drug trials and drug discovery.

So how are we going to go about developing common component software for this environment and bringing these different information pieces together? Well, I have three things that I would propose. One is to use open software tools to the extent that you can, things that will operate on multiple platforms. The second thing is to try to get international standards. And there are some that are in existence, and more coming. And finally, to try to take the best view you can of leading technology and see where you expect the field to be in five years so that you don't design to a standard that's either going to go away or become less important.

Well, you've got a lot of choices today for the kind of tying together of these information environments. And I've just given you some eye candy here to try to suggest what all of those various choices are. I'm sure you've heard of Java. I'm sure you've heard of XML, which is an extension of the HTML language that was-- is used in all of the browser technology and so forth.

CORBA means Common Object Request Broker Architecture. DICOM is a medical imaging standard. HL7 is a medical informatics standard. TCP/IP is the connection protocols that have been used since the mid '80s. All of these things are available. And you have various proponents of them. And the question is, how do you architect something that uses these in an intelligent way?

Well, they're just-- just briefly, there are three classes of software you have to develop. One is user access software. This is really the client part of it. You need databases for the storage systems. And finally, the third piece is the intermediate piece, which is the exchange software. And that's the one I want to emphasize here.

Here are all the clients up here. You're trying to run various models. You're trying to do analyses of data. You're querying about which patients are in trouble and so forth. Down here you have all sorts of different databases and varied information. And how do you put together the software layer that pulls all that together?

Well, first thing you want to do is you want to start with object-oriented technology. And I simply say that it used to be you could just put things in a file system, and name the file, and remember where it was. But there's too much data now. You can't do that. You can't exchange it. Ordinary relational databases can't deal with the multimedia. Object-oriented databases have a real problem, because there are no good standards for those. The object-relational is pretty good, but it's not absolutely necessary to this process.

But the one thing you do get if you use an object-oriented system is that all of this multimedia stuff is transparent. You can describe, up here, in an object-oriented program, exactly what's in the database. So the programming paradigms become very simple. If you don't use an object-relational system, then you get-- or an object-oriented system-- then you get into a real problem that you have to make all sorts of compromises in the relational system in order to get this object-oriented view from the outside.

One of the things we did as an example was to take the DICOM medical imaging standard, which is an international standard, and make a completely object-relational database out of it. You can't read all of the details here. But it was incredibly successful at allowing you to go in and query the image on its characteristics in a very enlightened way such that the image itself, as well as all of the derived information, was all available within the database with a common query format.

Here's another example. We have more than just databases. We have models. Here's a cardiovascular model. Roger Kamm, in our department, has developed this.

And what we're doing now is to put this on the web so that the model itself can query a database automatically for the information it needs. An outside user on the web can come in and set these various parameters. And the model will then run and produce information about what the various pressures are, and the flows are, around the cardiovascular system. This will be very nice for teaching. But you can imagine that these kinds of models are going to get more and more important for playing what if games in a clinical environment and for drug discovery.

Now the key technologies here reside in this interface. Here's the typical client. Here's the database. A request goes to the database. Results come back. But the database, in fact, could be fairly complex.

It could be a model which goes to another database, right? So there's a cascading effect that happens over here. And the client doesn't necessarily need to know anything about that. All the client needs to know is what kind of request has to be put back and how to interpret what comes the other way.

So we've developed a number of concepts to deal with that. One of the ones that's been very successful is to use something called a class mapper to produce an object-oriented view of the database independent of whether the database is object-oriented or not. Then what we do is we wrap that in what's called an XML wrapper. We then send that through a Common Object Request Broker environment that picks up the transport protocols, exchanges the XML with the client. The client then knows how to unwrap the XML and to display the results. So in a nutshell, that's what we do.

This is just another way to look at it. Here's the user up here, putting input in and getting display back. The request is interpreted. You can look on this as the transport layer. Here's, now, the database. This is the class mapper here, that represents the interface to the database.

I have an example that I can show somebody later. I guess I'm out of time, right?

MODERATOR: No, you have-- you actually have four more minutes.

DEWEY: Four more minutes, all right.

MODERATOR: [INAUDIBLE] and questions.

DEWEY: OK, well, anybody that's interested in how one applies this to the new gene chips that are coming out from Affymetrics and so forth, I'd be happy to talk to you about it later. Just qualitatively, what we have here is a basic chip schema that has been published by this genetic analysis technology consortium, which is basically Affymetrix and a couple of their friends. This is very probe-centric in the sense that it just defines what's on the chip. But there's no way to connect that with something like the human genome database, for example.

And furthermore, you don't know where that chip came from, or what the conditions were under which it was taken, what kind of subject information. It's not connected, right? So the next thing you have to do is you have to add all the information about how the sample was prepared and what the sample represents. So that's part of your database as well.

Then, the next thing you have to do is you have to connect that with the public gene databases so that you can now begin to make sense out at the entire system. And this is an ongoing project that we have right now that I think is very exciting. Because we probably-- I mean, unless we stumble, we're probably going to be the first people to actually be able to connect the new human genome databases with the Affymetrix chips in a way that you can go in and query at a very high level.

Like, give me all of the patients who have the following genetic profile, and see if you can match the following genes. Or here are six genes on these various people. I want you to take those six genes two at a time. And I want you to find the correlation between those two genes for the following people and then correlate that against the following diseases that they may have. I mean, these are the kind of high-level things you want to do in an object-oriented environment. But you can't do that unless all of the machinery works appropriately underneath it.

Well, I've mentioned most of the systems that we've been working on except the integrated health care. Since 1990, we've been working on some integrated patient record systems. They've been pretty well-adopted now in the UK. And they provide the clinical physician with a very good multimedia environment.

So I'd just like to close by saying that what we're trying to do is to get better diagnostics and therapy through technology. Thank you very much.

[APPLAUSE]

MODERATOR: Peter, why don't you come and start getting set up? [INAUDIBLE] with some questions.

DEWEY: Yes

AUDIENCE: How have you dealt with the problem of not having a standard for an object-oriented database?

DEWEY: What we have done in most of our work is to use the Informix object-relational database, which is quite similar to the Db2 that IBM has. The other big one is the Informix database, which is not as developed. But we just simply use this class mapper idea to handle that.

There is also an issue about object-oriented queries, which you can solve with SQL3. But SQL3 is still in the standards development phase. And you have to make a few compromises there. But those have not been nearly as difficult as working through the whole issue that you have to force everything to [INAUDIBLE]. And then everything else is easy. Yes.

AUDIENCE: It's interesting to see that this format be done by somebody in mechanical [INAUDIBLE] problem rather than computer science. So my question is, is there some kind of advantage that somebody in mechanical engineering can be-- can have over somebody in computer science in doing this kind of problem?

DEWEY: Well, I'm not a computer scientist by training. My background, like Doug Hart's, is fluid mechanics. I got into this through the back door by doing a lot of image analysis and getting involved in using optics for lots and lots of experiments in the laboratory. Then I realized that if we're really successful at this, as Doug, and George, and lots of others are, we're just going to be generating data that are just coming out of our ears. What do you do with it?

And all of a sudden, it was very obvious to me that this was a wonderful challenge. And I knew the marketplace. As Alex said, I knew the marketplace. That was my big, unfair advantage over anybody in computer science. So then it just went out and got it.

MODERATOR: Yeah, I mean, you know images. And would you prefer to have somebody who knows images and knows the marketplace doing the development to someone who only knows how to program computers, right? Exactly the two questions that Alex brought up-- do you have the person who knows the technology do the development, or the person who knows the-- and the person who knows the marketplace do the development as opposed to someone who just knows how to program? Yeah, one more question.

AUDIENCE: Actually, I'd like to second or third what Forbes was saying. It's a common route that many of us take-- I guess, like Forbes was saying, when we do optics, and imaging, and we try to develop this technology, a very common question is, what do you do with the data. So some of us take the computer science approach. For some others, the approach is more cognitive. So we ask the question, well, how does the brain do this? So you know, then, again, it doesn't matter who's department you belong, right? As long as you're having fun with it, it's OK.

MODERATOR: OK. Let's thank Forbes again.

[APPLAUSE]

Our next speaker is Peter So. I discovered he has an even longer title than mine. I lied. He is the Esther and Harold Edgerton Career Development Assistant Professor of Mechanical Engineering.

AUDIENCE: He just became associate professor.

MODERATOR: Oh, associate professor of mechanical engineering, excuse me. So Peter will also talk to us about mechanical engineering innovation in IT bioengineering. Thank you, Peter.

SO: Thank you, Seth, for the invitation to come here. It sounds like a really exciting conference. And the previous speaker actually set up what my talk is going to be very well. When Seth first asked me to give a talk in the IT conference, I said, well, I mostly think of myself as a hardware builder. So I was saying, well, what am I doing with information technology?

Then I start going back to look at what I do in my lab and start thinking, well, actually, almost everything that I do have some information technology content in there. And some of them are actually quite exciting. So this is a little bit of the work that what I am doing that I'm going to tell you about here.

As I look at it, biological system are probably some of the most complex systems, as Forbes has showed you, for a little while. Traditional biology and medicine is extremely descriptive. If you look at a X-ray, let's say a radiology slide, a very good doctor can extract very useful information, whereas a not so good doctor properly don't extrapolate useful information. But you wouldn't know who is good and who is bad.

And today, with the information technology that we have-- and it is quickly advancing-- we are at a position where we can really start to untangle and quantify the complex matrix of information that one could get in a biological system, and particularly, biological imaging, which I think I will focus on. And in some way, the quantification of a biological system, which is, in particularly, including ourself, is critical to control how we function, and which, of course, related to medical treatment as well as the possibility of developing new products.

Today my talk is going to be in three parts. I'm going to talk a little bit about what advances in information technology has allowed me to do today which I don't think it's able to do about maybe five years ago in terms of how to use information technology to impact some types of biological imaging applications. And then I'm going to tell you a little bit about how does some type of biomedical imaging problem pushes the frontier of algorithm development in terms of how to process information such that you can extract useful information from really complex biological data. Finally, I'm going back one step and say, well, as the information technology developed, we are able to do new things.

And we are developing new algorithms. But with new instrument development, we are actually generating more and more data. And actually, we are generating data in a probably exponential rate today, which, of course, would give you further demand in the advance of information technology.

I'm going to focus on the first problem, what information technology has allowed us to do today that we cannot do before. One of the central questions in terms of cancer biology as well as the treatment of cancer is how does mutation occur. What are the rate of mutations? Mutation, being one of the main cause of carcinogenesis, the initiation of cancer in the body, can you quantify the rate of mutations for a normal person as well as when a person is exposed to various toxins? So the question is, can one develop a system such that you can look at one type of recombination which we are interested in, which is called-- one type of mutation, which is called a recombination, that we are interested in? This is some work that we are doing in collaboration with Bevin Engelward in bioengineering and health science division.

So how do you quantify mutation? As it turns out, there are wonderful things in biology that you can actually put a gene in almost any normal organism that you can manipulate, such as mice, to make them glow green. This is a protein called GFP. If you put the protein in, you can make a mouse go green.

But you can also-- OK, now you might ask, why do you want to make things go green? Now, of course, it makes a extremely cool picture, number one. Another reason is, now, if you want to measure mutation rate, you can put in a defective copy of the gene into the animal. But when there is a mutation, the defective copy would combine with a second copy that is complete. When that happens, a cell goes green.

So now you can have a fluorescent marker in the animal that you can watch how many cell go green, where they go green. If they go green-- let's say it's a mutation-- does this particular clone expand? Does it become a big tissue that is green, which, some way, is called clonal expansion and is very important for cancer?

So there are a couple things that you have to do. Number one is you need a genetic engineer that can make the mouse for you. Number two is you need some technology such that you can look into tissue to count the green cells and look at the distribution of the green cells relative to the rest of the tissues in the animal. So the technology that I have available in my lab is so-called two-photon microscopy. I'm not going to go into the detail here.