MIT MechE Symposium: Mechanical Engineering and the Information Age - David R. Wallace, Anthony Patera, Seth Lloyd & Panel

Search transcript...


MODERATOR: As Alex d'Arbeloff mentioned in his lunch time talk. One of the main issues that we're looking at here is, okay, in some sense, it's too late for us to do anything about the fact that information technology is important for mechanical systems, right? That's going to happen whatever we do. Nobody's going to take the microprocessors out of cars because we asked them to.

But we're the people here who are educating students going to live in this world. And so what are we going to do about it? I mean, pretty clearly, the kinds of things that we've been doing up until now, though of course, they're very good, don't necessarily address these issues. So how are we going to address these issues? That's one of the questions. How are we going to include information technology in education for engineers?

And secondly, because we always do it both ways, what does information technology have to do for education? How can we use information technology in order to make education more efficient for engineers? All right.

So our first speaker here, who is actually an expert on this field, David Wallace, who is Associate Professor of Mechanical Engineering, will talk to us about new media education. All right. Here we go. All of that being said.

WALLACE: So what I'm going to talk about is really the new media possibilities that are enabled by the use of information technology in engineering education. To provide a little background about myself, I'm the Associate Director of the MIT CAD Lab. I do my research in the area of computer-aided design. And my teaching is in the area of product design, both at the graduate and undergraduate levels.

So what am I going to do? I'm first going to provide a little bit of motivation, which is at least the angle at which I am going to be coming from in this presentation, provide a few facts of things that we've learned and that are going on, talk a little bit about some experiments that we've done in the department at MIT, and then move on to some thoughts about teaching and learning models in this new information technology enabled environment.

So to me, this represents the space in which we teach. And we're certainly used to, in universities, the notion of presence and traditional media, where traditional media may be classrooms, lectures, traditional labs, tutorials, and things like that. And of course, now into the fray we have the possibilities of using new media and incorporating tele-presence as part of our educational programs.

And so really, ideally we'd like to be able to work pretty seamlessly throughout this space. And some of the key questions are, how do we design our curricula in an informed way so that we can understand the difference in when we might want to use traditional media, or new media, or presence and tele-presence? And it's an important question, because often the tele-presence can compete or at least offer some of the similar-sounding educational opportunities at a lower cost than a residential education.

So moving on to motivation, what I think we need to be doing and are interested in doing is trying to develop and test new strategies for teaching when we have this combined media environment that can be applicable to a broad set of curricula. And we can understand when we might want to use presence or tele-presence to really have the most effective educational opportunities possible.

And what I mean by that is, first of all, we need high quality mass education. And we also need very high quality residential education. So we can clearly understand and show the value of the on-campus educational experience. So that's the motivation.

And now I just want to move on to some facts, or some I would like to pose as facts. And there's still often a lot of debate about presence, or tele-presence, or correspondents, in slightly more dated form of the same thing. And this is, in fact, the "No Significant Difference" web site. And what it has is a list of over 300 review journal and conference articles that cite no significant difference based on location, i.e. presence or tele-presence alone. It's also available in book form.

So in some sense, I think the debate about presence is really open. It's pretty clear you can do a good job with presence, and you can do a good job with tele-presence. But presence in a sense has no inherent advantage. What really matters is what learning and motivational opportunities you provide in the educational experience. So it's kind of like what do you do with the time? How well do you use the situation to provide an educational experience?

We'll also just note that there is also a growing significant difference site, where most of the papers are finding that remote education has an advantage. And one might speculate that this may be because some of those programs are fairly carefully thought out, maybe more than some of the traditional classes.

So the message really is, how things are implemented is very important no matter what you do. And it's not just whether you're doing it with information technology or with people together or remotely that's really going to be the most important factor. So you have to think carefully about what your teaching strategy is.

I think another observation is that we all generally feel that we could be doing a better job. And this is just an example from Ben Linder, who was a doctoral student in the department under Woody Flowers. But basically, the idea is that he surveyed the senior students at six major universities, asking them how much energy was in a 9-volt transistor battery. And that's a log scale on the bottom, and you see a pretty widely varying response.


And I mean, it's a fair debate. You could say, well, should we expect them to really be able to answer this question or not? And some people might say, yes, some people might say, no. But on the other hand, I think we can say we would like all of our graduating students to be able to look at the answer they're proposing and decide whether it's ridiculous or not. Right?

And there's all kinds of other stories about things happening. And in some ways, the students in some sense aren't that much different than they were. Maybe different experiences, but they're people just like everyone else was historically. And so we need to think about what we can do to improve the way we're teaching them.

Some other facts. Sticking our lecture notes online is something that's fairly easy to do, and it certainly has administrative advantages. But in some ways, it really has a modest educational impact. And this is probably the most common applications but it really doesn't go to the core of trying to use the technology and new opportunities to teach better.

So it's not surprising, because we need to design materials to take advantage of the media that we're using. And we need strategies and pedagogical reasons for choosing the different media. And simply stated, we design a lecture in a certain way if we're going to stand up and talk, just like I'm talking to you now.

If we were going to be doing in an electronic format with multimedia capabilities, we would design it and structure it very differently. So we shouldn't expect to be able to do just a direct port of materials from one media to the other and get big advantages.

Another area of discussion or debate is often about what is the best way to teach. And if you look at a lot of different materials on education-- and teaching engineering is one, and there's a number of books on the subject-- in some ways, there really is no singular best way to teach.

So of course, there are different types of people that like to learn using different types of formats of presentation of the material. They like to learn things in different order, like with theory first, examples first, motivation first, applications first.

And there are different use patterns. Some people like to hear a story, follow it along. Other people really just want to go in, grab the salient facts. So those are all different possibilities that are really contingent upon the people themselves in different learning styles.

There's also different types of learning, whether we're trying to do training, or whether we're trying to really teach deep theory or general concepts. And there's different types of content. And we should expect that there may be different ways to address some of these issues differently.

So it's not a question of what is the best approach. But how do we have a framework so that we can apply a lot of different tools and approaches in an effective manner? And this is one hypothesis that I certainly believe in. I'll show some experiments pertaining to in a bit that at least the new media materials certainly can be designed so that they can accommodate some of the different learning style issues.

Okay, and with that, we're now going to move on to talk a little bit about some experiments. And this is a new media teaching model that is a result of some experiments. I'll just give it to you upfront.

But the gist of it is to have students study codified material using lectures and quotes, in a sense, that are online. And then following that up with experiential activity or mentoring with someone that has experience or wisdom, if you will, that's hard to codify. And then providing the opportunity for in-practice application to really consolidate learning in a deep way.

And that particular model was based on some experiments where we first tried to sit down and design a framework that we could provide the codified materials online-- in a sense, in lieu of a lecture. And really motivating that structure around the different learning styles idea. And then once we wanted to do that, really evaluate it. How well does it compare to a regular classroom?

And if we can use these materials in lieu of a class, how can we spend time differently with students, versus the traditional lecture? So I'm just going to move on.

So we developed a general template for providing lectures. And this was for, actually, an implementation of it, but for a traditional 90-minute lecture. And to just want to note down the right side is what would be the equivalent of the lecture divided up into 94 different sub-topics.

And if you follow it through, it's more or less going to read like a story. If you go point to point, it provides you very parallel access into just about every part of the lecture. Right now, we're in a section actually related to prototyping and actually how to prime things. And in that, the content is divided up.

And it's hard for you to read. But essentially, there's a bin, which is for why do I want to know this. If there's any theory behind it, that's where the theory is provided. Procedural information, which is how you do the application-- it's really an application. An example, so the students can very quickly or conveniently, if you will, tab between different types of learning information.

And then finally, within the subject, the students-- so we're looking at how to-- so that's the procedural part of priming-- they can play a video which is annotated, so they can watch and listen. They can listen to audio and look at the pictures. Or they can go through a very traditional read and look at the pictures, just like an online textbook. And we've implemented four lectures in this way, two for product design and two related to classical control.

Once we had, in particular, the prototyping lecture implemented, we wanted to compare this to our regular classroom teaching. So we had half of the class take the web lecture and the other half have a traditional lecture on the same content. And they all had access to the same textbook. But you either had a class or you didn't.

And then they had a project, and an outside group evaluated the project. And what we could do is then track a lot of information, track how they did on the projects according to the jury, and also do focus groups to try to learn a little bit about what we're doing.

So just to compare, first of all, the jury's scoring of the prototyping project. And what this is on the left is, if we divide the class group and the web group, but we look at all of the assignments they did over the term, where they were taught in the class and the same thing. And what we can show is that they are indeed expected to perform the same.

And when we look at how they did on the assignment, in fact, there was about a 7.3% difference. So in other words, the people who only had the web-based experience did a little bit better than the people who had the class. And statistically, for these data, it's about a 1 in 100 chance of seeing that difference in the mean where they actually really were performing the same. So there's a reasonable statistical evidence to say that they performed differently.

Let's think a little bit about why. Here are some of the things we can design the online material to do. It's a classroom lecture, 90 minutes. Most lectures are either one hour or 90 minutes. Both are longer than people's attention spans. And if we look at how many [INAUDIBLE] whacks at it the students online took to cover the entire lecture, the average was about three, which corresponds to 33 minutes per session, which is very close, if you look at some of the data, to what people's attention spans are.

This little snapshot down at the bottom is essentially the number of people studying as a function of time of day. And the only slot that's quiet is between 4 AM and 8 AM. The spike is actually at lunchtime. And we also found through the focus groups that they like being able to do the studying by themselves, because they didn't feel peer pressure. They felt they could spend a lot of time where they didn't understand things, and not feel like they were not as smart as the next person in the class, basically.

And we also have data where we had these parallel delivery paths. And we can show that there were groups that were movie centric, there were groups that were text and image centric, and there were groups that would switch back and forth, probably based on the content of the section, and whether they thought a different medium might have different types of information.

So what we can't say is that, looking at these differences, one of the other nice features that when we try experiments with new media materials is that we can actually track and understand a lot of things, and learn about how to do better.

But given the notion that a lecture like that online can work, the question then becomes, how do we use the class time? And so to follow up what we did is, of course, in the next year, the entire class uses the web material. And we ask them to prepare beforehand by studying this.

But the class is divided up into two groups and taught differently in person. One group, I call it the active classroom, so it's a lecture where we're following active learning type of guidelines. But the gist of it is, we give a complete second coverage of the material, which is potentially a positive thing.

For the other group, we actually make no attempt to cover all of the material, but we just go and work with them and do some things with them that cover part of the material. Then we follow the same prototypes where they all do the same sort of project, and an external review of how they do.

And in this case, again, we shouldn't show, we would expect them to perform the same based on their other assignments. And we see a difference of about over 10%. And in this case, from a statistical standpoint, there is a chance of about 1 in 1,000 of seeing that big of a difference in mean when the two groups were actually the same. So it seems that combination works pretty well.

And again, we might think, well, why? And earlier on, I said a lot of education is about motivation, and preparation, and studying, which is important. On the left is in a survey how the students rated their preparation beforehand. And so in blue are the people that essentially thought they were coming to a lecture. And the red is the people who thought that they were going to do some things.

And of course, you can see that the people in red were much more motivated to prepare ahead. These are the actual data, because they were all studying the web lecture. And what we can see is that the 60th percentile person coming to a classroom was less prepared than the least prepared person that was going to come and do an exercise.

So there were some strong motivational differences. And in fact, the only differences in the instructions between the two groups is, one said, you're strongly encouraged to review the complete lecture before class, and the other one, we'll assume that you've covered it and we'll be working on things in our class period.

So you can use the mix of information technology and traditional approaches to provide strong motivational forces. And when we asked students about-- and then, this is in a survey, if I could have only had one of the forms of instruction. If we look at the people who had the class and the web, 60% said, well, I'd take the web only.

If we look at the people that had the experience, essentially 80% said they would take the experience only. And that was in the fact that they really wouldn't-- we didn't cover all of the information there. So if they took the experience only, they would have missed the huge part of what they really needed to learn. But they placed a very high value on that experience.

And so in some sense, one of the messages is, when we're working with the students, we should be trying to do things with them, and things that are not readily codified in other forms.

So where might this type of model apply? And so this model which we were trying to support was studying codified materials using appropriate IT-based approaches, experiential activities, and then in-practice application. And what I want to add is, each lecture took about 500 hours to produce for about an hour of lecture. So there's some overhead and preparing them.

But certainly, this seems like an appropriate form where this knowledge base is stable. There's a lot of students, or a lot of customers, If you want to think of it that way. And they may or may not be super interested in learning the material, because we can bring in very high production value.

We also actually have used this model at the Ford Design Institute for their technical employee training. And what it's probably not appropriate for is emerging knowledge, where there's a small customer base and very strong interest and motivation to learn on their own. And you don't need, essentially, some of the production value.

So now with that through, what I just want to talk and close up with are some thoughts about teaching in this new media environment. And this is based on work done by a committee with several different disciplines involved in the Department of Mechanical Engineering. How am I time-wise?

MODERATOR: You have 4 minutes.

WALLACE: Okay. Should be just about right. So when we worked on this, and what we really wanted to do is try to find a way to think about all the possibilities and how we might apply them in some different teaching strategies. And these are some methods that have really been around a long time, one which we call the Socratic method, which is independent study guided by mentoring, which is more or less what I've described.

Just-in-time, which is the notion that you're working on an application, and you learn when you need to, because you've encountered a problem. And finally, a classical scientific discovery mode of learning through observation, study and hypothesis, and experiments.

And so really, what we think needs to be done is to identify the combinations of material delivery, hard facilities, and interactions for these different types of learning models, and trying to understand what ones suit different types of materials and teaching goals.

What we envision for pretty well all of these is, one thing is that most students will have laptops. And for the Socratic, we think that we'll have these types of codified lecture modules that support different learning styles. And then we can combine them with mentoring experiences.

And these mentoring experiences may be mini lectures. They may be some form of a classical lecture. It could be personal mentoring. It could be electronic forums. It could be online quizzes with feedback. It could be a studio environment. And there's activities in all of those areas and we'd like to think about how we would use them.

And one thing I can say that I feel pretty confident about, may be a bit controversial, but at least for me, I find that 15 minutes of direct contact with two or three students is easily worth a couple hours of me standing and lecturing. So that's kind of what's driving that model.

For the just-in-time model of education, what we think is needed is essentially a marketplace of learning modules, so that students or employees are working on projects, and when they encounter problems, they can find the resources to learn the bits that they need at the time.

And the reason we think there's going to be some sort of a module marketplace is that the projects are often very different and diverse. And so it's hard to predict upfront what types of codified material you would have. And it would be, I think, too great a task for any one institute to produce all of the materials that are needed in an appropriate way.

And you essentially need highly customizable curricula. And this is one of the challenges in 2009, which is a privately designed course that we teach where different teams of students are all working on different projects. And so they run into different problems as they're working throughout the term.

And finally, the scientific discovery model. We envision-- this is a picture of the high school chemistry lab concept. But really, the classroom and the laboratory are integrated. And there's a lot of interesting issues related to the use of physical space versus virtual labs-- haptic devices, for example, to virtual simulations, or physical toolboxes that need to support this type of environment. So there's a lot of potential combinations of traditional and information technology approaches that need to be thought about.

So in summary, I think there's quite a bit to be optimistic about, as there's a lot of new teaching opportunities that are arising. And I think what we need to be doing is thinking about how we can use them effectively. The one thing I will say, though, is that things are moving very fast.

There are a number of open universities, virtual universities, companies or organizations offering courses at very low cost. And so I think now is really the time that we need to explore and develop a strategy for teaching in this environment. So that's the end of my talk.


MODERATOR: So questions for David? Tony, maybe we could get you to try to work on you technical project while David's answering questions.

AUDIENCE: You say the direct contact is maybe 8 times better than the lecture. How did you arrive at that metric?

WALLACE: That particular metric, that's one that's not as derived as most of the data that I put up there or as careful. And that's, to be honest, my gut feeling. And to provide some substance to that in the graduate course that I'm teaching, I now lecture-- a number of the lecturers are using this approach. And then we work on activities.

I do some traditional lectures. But just about every third week, I don't lecture. And I have 15-minute meetings with two students at a time, see what they're doing, and I work with them. And I feel that the results are better. So that's so much more qualitative statement, but that's what I base that on.

And I find that 15 minutes, to be practical, you could spend more time and it would be great. But that's the chunk of time that it becomes a manageable proposition for me as an individual doing this. So yeah.

AUDIENCE: To follow up to that, with what you describe with your technology and integrating it into the teaching, and then thinking about distance learning as an area that is growing, then back to your 8 to 1 contact. Where do you see things perhaps going? And is the distance learning going to replace residential education or what?

WALLACE: Sure. I'll offer some thoughts on that. It's, of course, speculation, just like if anyone else is answering the question. My own opinion is that first of all, I think the line between distance learning and residential learning will become more blurred. And that I think even in a residential university, some things will be done almost in a distance-learning style. Such as the very codified materials.

Examples of that-- I could look the references up. But I know of lots of cases where they've begun to offer courses on campus both internet-based and traditional. And the internet-based clearly win out, in terms of what the students want to do. So the lines will become blurred.

But I think that what the residential universities really need to do is focus on the unique part that you can't really do at a distance, so that there is still a very strong motivation for wanting to have a residential education. Because you can work essentially with your peers, and also with people that have wisdom to impart that can't be readily codified and as easily disseminated.

Now, in the distance-learning mode, you can still try to do things to help impart that. So for example, you might set up a scenario where maybe you're broadcasting a type of learning experience, where you have some students who are up and actually working and interacting, almost in a mentoring way, with the faculty. And then there's observers at remote sites. And that can help bring some of the flavor of the one-on-one experience.

But it's still, from a motivational standpoint, very different than the knowledge of the students that they're going to have to go and talk, and do these things, and be seen by others. So I don't think there'll be a complete replacement. But there may be a shift in scale and what's done in different ways.

MODERATOR: One more question, and then we'll move on to the next talk. Yeah, go ahead.

AUDIENCE: It seems easy to embrace these kinds of technologies when you're trying to present theory or topical knowledge like the knowledge part. What about the know how, like how to solve problems? Have you thought a little bit about it?

WALLACE: Yes. I think, again, actually, the point you make, I think, is a good one. And it goes back to, I think, that things that can be readily codified are very appropriate for this type of approach. Things that can't be is where the value added in the experience is. And that, in a sense, the know how that's procedural, which is, in fact, lots of times what you see in employee training programs at companies, that's just as codifiable as anything else.

The kind of wisdom of actually doing an experience with experiments, say, with someone who has a lot of depth in that field is something that you're going to have a much more difficult time packaging, if you will, in this way. And that's where I think that a big part of the residential education's value should be.

MODERATOR: I know some of you have more questions. But we should really get on, because we're behind schedule, and we may have technical difficulties. So David's going to be on the education panel coming right up. So please grill him some more.


PATERA: I thought David and I had agreed that there wasn't any substance in our presentations, and now he's gone.


PATERA: All right. [INAUDIBLE] should not worry about the sound and maybe get me the video anyhow.

MODERATOR: Okay, this, by the way, is Tony Patera. He's [INAUDIBLE] mechanical engineering. He's the deputy director for the Singapore Alliance. And he's going to talk to us today about the MIT-Singapore Alliance for Education.

PATERA: So I will start with what was going to be an audio-video, and now is going to be a video with your own personal voiceover. And not even a video at this point.


All right. So SMA is the Singapore-MIT Alliance. And this is a little video that we made for promotional purposes. But it also gives you a sense of the program. This was launched last year. The first class will be graduating this summer.

And there are two programs initially-- in advanced materials and in high-performance computation. And as you can see, there's a very significant distance component, that I'll be describing a little more in a moment. The program begins by the students in Singapore with about a month of various teaching activities. They then come to here at MIT.

This here is indicating what they do while in Singapore, which they recognize as MIT. The beginning of the program-- the two days is focused on the spirit of technopreneurship within MIT, with examples from the MIT faculty. Here's John Vance. As well as a continuation of the more formal activities that they began in Singapore related to coursework and project work.

There's also the usual social activities that have been arranged for the students here. There we're teaching them how to abandon healthy Southeast Asia [INAUDIBLE].


Here we're putting on one of the posts that went to a bottom of a lake in Arkansas.


AUDIENCE: Can you turn your mic on please?

PATERA: And here, the students are saying how much in particular they enjoyed my lectures during their time at MIT.


When they return from MIT back to Singapore, they continue on with about a month of classes at Singapore. And then begins the distance aspect of it. And a key component of the distance aspect is that you saw MIT students in the classroom there. And now, in a moment, you will be seeing the Singaporean students sitting in Singapore.

We send two streams to Singapore. One is the video stream, which is all the usual talking head business. The other stream is the data stream which is essentially uncompressed PowerPoint slides and other data aspects that therefore arrive uncorrupted on the Singapore side.

The facilities that you're seeing here are the ones at MIT. There is a mirror site at Singapore. And in fact, in some of the courses, we even beamed internally to MIT. You're seeing it now from the Singaporean-- the SMA student's perspective, where you saw the two large screens at the front of the room.

The left is the video stream, the right is the data stream. And in fact, these are the kinds of PowerPoint animations that are used in some of the lectures. And I'll be talking about that a bit subsequently, as well. This again is from the Singaporean side-- Jacob White on the left. You see him now in the inset here. And on the right was the data stream that he sends over, which you see here, as well.

And again, that's from the SMA student perspective. What was being used on the left is a document camera, which is simply a higher data integrity version of a blackboard. There is a significant asynchronous component, in which all of the various lecture materials-- digitized videos, lecture notes, slides-- are posted on websites and integrated using, effectively, a system related to the command system used at MIT.

Here you see the Singaporean students viewing Professor [INAUDIBLE]. The SQL, all right? Having seen him one slide, they now see him through the digitized videos, which we found to be a very effective way for them to be able to review material. There's also other various, by this point, the standard mechanisms for them to search through the material and to pick the various pieces that they're looking for.

In addition to the distance interactions, many of the students spend time at MIT. In particular, all the PhD students. And here, you see them commingled with MIT students on campus, and also talking about research projects, here with Professor Suresh.

Research is a significant component of the SMA program. And I'll talk about that a bit more later. Here, it's appropriate to indicate that the third program is housed primarily, but not exclusively, in mechanical engineering. You see Professor Chen and Professor Hart and their counterparts in Singapore designing the manufacturing program, which is the third of the three academic programs. There are also major players from the Sloan School in that program, as there are, in fact, in several of the programs that are currently launched.

And so I think that is about it. They're just saying how great everything is. And I think at this point I can conclude. It seems to have concluded for me. And so what I'd like to do now is make a few-- there are a few uplifting moments at the end that I spared you.

But what I'd like to do now is use that as data and make a few observations, most of which I think are quite obvious. First, it's an experiment in teaching and in research. And I think you saw that I put, "at a distance." But that's only really part of what all of this is about.

And so I'd like to first make some general remarks. The key one is that SMA is not really typical of distance efforts. The first is, there is a focus on an elite graduate cohort. The students that are admitted essentially pass the same kind of muster that we require of all our own students here at MIT, in terms of GRE's and, of course, letters of recommendation.

So again, the focus is on elite, as opposed to many other distance efforts, which are more for the unwashed masses. Both teaching and research components figure prominently. Again, many distance efforts are focused on information transfer, much more of the formal teaching, as opposed to the more labor intensive research side.

There are both distance and face-to-face elements, as I indicated. The students come here, and the faculty go there. All right? So there's a significant component of what we consider more classical modalities that are not facilitated by technology in any significant way.

There's a very high faculty-to-student ratio. Many distance education efforts are stimulated by the desire to be able to much more inexpensively educate a much larger group of students. In this case, we seem to have almost maximized the inverse of that ratio, in terms of the resources that are being devoted to these students.

There's a strong emphasis on interaction. All right? This is not a case where there's a lot of tools that are developed, put online, and they become a one-on-one activity between the student and the tool. There's a lot of interaction between the faculty and the students. There are strong links with MIT residential initiatives. Many of the programs have, for example, stimulated M-Eng degrees here on campus, based on material developed within the Singapore-MIT Alliance rubric.

There's also a very strong partnership emphasis with Singapore. There are Singaporean faculty involved, so that students, even when they're in a distance mode relative to MIT faculty, have Singaporean faculty from NUS and NTU that serve as local face-to-face faculty in the usual sense. So there's always this component of a more traditional environment.

There are very advanced facilities and technical support. This is not the kind of thing one can do in one's own garage yet. And finally, there is a quasi-antipodal geographic framework that was apparently maximized for frequent flyer miles rather than convenience by our dear provost, Bob Brown, when he set up this program.

And finally, there's, because of all that, a rather nontrivial price tag. Again, distance education sometimes is seen as the economic panacea for educating a larger and larger group of people. Certainly they should not emulate the Singapore-MIT Alliance if that is their objective.

So let me then make a few more specific observations, first related to teaching, and then just a few on research. The first on teaching is that technology is no replacement for, and can significantly detract from, scholarly, organized, complete, and effective development, by which I mean preparation and assimilation of educational material.

And to give you one example of that, this is very anecdotal, obviously. We had one course in which there was a great emphasis on more dynamic presentations, and there was not too much background material available. We had another course which had much more detailed lecture notes, but much more static and less stimulating, in some sense, lectures.

And we found that the students, although they certainly enjoyed the dynamic presentations, would much prefer that we spent our time on making sure there was always adequate codified material for them to be able to appropriate and subsequently apply the material.

This is a bit peculiar, perhaps, to the SMA program, in the sense that we're developing new graduate curricula. So existing textbooks are not readily available, and in as much, background material is more imperative developed for each course. But nevertheless, I think there is a more general message to be taken away.

Ah, I guess that is not the right button. I'd like to go back to the asterisk, which says, it can significantly detract from, but need not. We saw a number of examples that Dave Wallace showed where obviously content technology was enhancing it. I've chosen to take this more negative view, because I think there's a lot more material out there that's not exciting. And the kinds of things that Dave is doing is the exception, not the rule.

This really refers to, as I just described, content technology, and in some sense, using technology to enhance the way the content is presented. All right, there's another aspect of technology which is obviously crucial to the SMA effort, which is transmission technology. You saw there some of the live beaming sessions that are done.

We have the classroom here at MIT. And there's the MIT faculty member. There's the MIT students present. That is live, simulcast over internet to the Singapore site. Obviously having all those pieces work is absolutely critical to the success of the program. And I think the summary at this point is that advanced video conferencing technology is reasonably effective and reasonably reliable, or reasonably reliable and reasonably effective.

Issues remain. Audio and video clarity. You saw the audio and video clarity. We've actually had at least as much trouble with the audio as the video, which is a bit surprising. Bandwidth and compression are an issue. We have MIT faculty that are extremely animated. In Singapore, that comes out as kind of a throwback to Claymation after they've been sent through this low-class filter.


Apart from sedating our faculty, there doesn't appear to be a near-term solution to that problem. And it's something that has to be addressed. Cueing and flow is also critical. We all take for granted in an environment like this how easily it is to pick up on peripheral cues in order to know where the attention has to be focused. At a distance, a lot of that cueing goes away, partially because of clarity, partially because of camera angles, partially because of one's ability to see the remote site on a screen which tends to be rather small and at the back of the classroom.

So for all those reasons, things are not quite where they should be. But they're not bad. And the feedback is generally good, based on over 100 live beamed sessions and also on student performance.

MODERATOR: Maybe use this one.

PATERA: Yes. Is this not working?

MODERATOR: Oh, the battery's dead.

PATERA: Battery's dead. So keep the other one also?

MODERATOR: Yeah. That one's fine.

PATERA: I'm gonna get electrocuted by the end of this.


In terms of student performance, we did comparisons not quite as scientific as Dave's, in terms of the MIT cohort and the SMA cohort. And we found that they were statistically indistinguishable. In fact, so much so that the data looked rigged. But I think that the takeaway is not so much the data as it is the fact that the students in Singapore were able to learn the material at a rate similar and an ability similar to that of the MIT students here on campus.

Observation three is that lectures without student participation are a real waste of expensive estimating resources. And, as opposed to alternatives, such as lectures with much interaction, or recitations and tutorials. And the reason is, if we pay facility charges, we have it in a very expensive room. Technician charges, we have two or three technicians typically on each side making sure that the event transpires smoothly.

Internet 2 and ISDN access charges-- all these live simulcasts are done over I-2 or ISDN. Off-hour extra compensation, because of the fact that Singapore is 13 hours different from us. The lectures take off typically between 7:00 and 9:00 in the morning and 8:00 and 10:00 in the evening.

If you add all that up, and this is all being done for real-time sessions, and all you have is a faculty member that spends 99% of the time talking, that's not really a very good use of those resources. And in some sense, this has only made more clear what's already clear at MIT, and that's why I put the SMA in parentheses.

At MIT, there are lots of charges and costs associated with giving a lecture, as well. They're not as easily attributed to things like technicians and all the rest. But they're clearly there. And once again, you might ask, is that really a very good use of resources? And that's precisely, I think, one of the points that Dave Wallace was making in his earlier talk.

But of course, what enables all this is web archiving and integration, so that you can get the material before class, and then use the class for more interesting, interactive maneuvers. And in fact, that has been used very successfully so far at SMA. We archive the digitized lecture videos, the slides, and the lecture notes. And this provides for a very rich environment.

And when I go to Singapore, I often will circulate through the cubicles where the students sit. And in fact, you see them very often going back to these videos of the lectures to see what were the key points. And they also have access to the lecture notes, which develop things in more detail. And they seem to, that way, in some sense it acts as a replacement for the more constant and redundant interactions that would occur if they were not at a distance.

On research, I have just two very obvious observations. The first one is that most MIT faculty already engage in some form of distance research-- back to MIT while on travel, or to institutions of students of colleagues. The data for that is that it's certainly difficult to find an MIT faculty member at MIT. They must be somewhere. And wherever they are, they must be interacting with their students back at home in order to continue making progress on the research front.


A slightly less obvious observation is that distance research can be enhanced by really now standard technology. One-to-one video conferencing, which is just a euphemism for net meeting and variants thereof. Slightly more sophisticated-- shared collaborative applications. It's now very easy to take scribbled notes, as you would present to a student sitting across from you, put it through a scanner, have that integrated into a shared and collaborative whiteboard, so that you can annotate it, and the student at the other side-- in this case, in Singapore-- can annotate it simultaneously.

And again, that kind of a session is almost as good as what you might be able to construct with a student sitting on the other side of the desk. And finally, there are various hardware devices, tablets, and other input systems that make this shared, collaborative application usage a little easier.

And in fact, we've integrated all this together into little individual distance kits that we give to the faculty participating in the SMA program, so that they have all the access to both the software and the hardware that they need, in terms of interacting with their students.

And finally, then, by way of conclusions-- even at 12,000 miles, teaching and research remain-- well, they remain teaching and research. And that's more or less by construction. Where a lot of this program is, in some sense, not revolutionary, it's reactionary-- trying to replicate some of the very good things about MIT at a distance.

Obviously some of the things are a little more revolutionary. But we're trying to keep a balance between the two. Good transmission technology is absolutely essential. And I would say, it's not quite there, but it's almost there. As you could tell from my earlier comments, I'm not quite as enthusiastic about content technology, unless it's very carefully applied.

Finally, some regular face-to-face interactions are still crucial, in particular, for research, I think, in terms of establishing a rapport with the research student, which most of us have now done. In terms of launching the research aspect of SMA, that would be clearly impossible if there were only net interactions that were available.

That's all I have prepared. And I'd be glad to take any questions that you might have.


AUDIENCE: Tony, I've had the experience in a couple of instances of setting up a research program, for example, with Imperial College in London, where we started and completed the project basically by email. And in the beginning, the people that were involved only knew each other vicariously, in the sense that they only knew the two project directors, who in turn knew each other.

And it was very interesting to me that over the period of the three or four months when those intense projects occurred, there was a real camaraderie and sharing that went on between the people that were working on the project, even though they never saw each other in person, even with video conferencing.

I'm just wondering if there was any of this personal infrastructure building that occurred here between the students?

PATERA: Probably to a certain extent. I think that the students come in, and they're immediately blasted with sort of a standard MIT graduate program. And so there not much time left, actually, for the more cohort building activities. We've actually lightened up some of the programs, precisely because we felt that, given the distance, that there was not enough time left on the side for those kinds of relationships.

So I think the answer is that it's probably a little different, in terms of research group to research group, and faculty to students, in terms of priorities as to what they have on their plate and what they have to get done first. But I certainly agree with you that, in the cases where it did happen, it was clear that there was a way to develop some rapport electronically. Yes?

AUDIENCE: Do you see an inherent benefit in remote coordination of research, due to the fact that there has to be some attention given to documenting results in order to facilitate the communication? Is that something you've seen as a benefit?

PATERA: Well, it's something I've certainly seen before, in terms of when, again, when faculty travel, sometimes they have a better sense of exactly what's going on than when they're here. And to a certain extent, that's also happened in the SMA program, in the sense that there's a more conscious effort to make sure that both sides know what's going on. And therefore, communications tend to be a little better.

The one thing actually your comment reminds me of is, there's also this usual wisdom about progress being better, because if you're separated by 12 hours, of course, there's one side that's working, right? They pass the ball to the other side.

The fallacy in that argument, of course, is that graduate students already are precessed by 12 hours.


And so, in fact, everybody's up at the same time. So that has not panned out quite the way we anticipated.

AUDIENCE: What I saw in the video was 15, 20 students. It would be much cheaper for they team up with government to send them here. And they adopted usually to study in the US [INAUDIBLE]. What the benefit compared to all of the expense of the citizens-- millions of dollars, probably, it would cost them.

PATERA: Yeah. A few million. Yeah. Well, there's several answers to that. There's several aspects to the answer to that. The first is that the same way as MIT sees this as an investment and an experiment, in terms of understanding how these kinds of technologies can be applied. So does Singapore.

And obviously, Singapore would like to position itself, and is, in fact, positioning itself, as a hub for education in Southeast Asia. And in as much, knowing how these technologies can be applied is obviously going to be crucial. So there's a clear sense from both the MIT and from the Singapore side that understanding how to use these technologies is very important.

The other thing, of course, it's not only 25 students. What you saw were various snapshots. The steady state population of students will be more like 50 times 5, or 250 students. And as you know, an MIT education, in fact, is not so cheap either. So if you do the multiplication, it's not such a [INAUDIBLE].

But the key point is that this is really seen as a way of exploring what can be done within the context and confines of what we consider to be quality top-level education, using some of these new modalities in a creative way. So it that. Yes.

AUDIENCE: Even if this is above market, could you give some idea of the cost for one course?

PATERA: For one course? Yeah. No, I won't. No. But the reason I won't is that, in fact, I mentioned at the beginning, there's both research and teaching components. So in fact, by giving you-- by dividing the total number by the number of courses, I'm applying a research overhead on each of those courses, which is not very--


PATERA: By the end of five years, including cost both on the MIT side and the Singapore side, it will be well over $100 million.

AUDIENCE: The group in Singapore, is it affiliated to any school there?

PATERA: Yes, of course. I'm sorry.

AUDIENCE: Which school is that?

PATERA: What you would have heard on there, is that there are two-- actually, there's more than two now. There are two primary universities in Singapore-- National University of Singapore and Nanyang Technological University. Both are involved.

AUDIENCE: How many students have completed their degrees under this program? [INAUDIBLE]

PATERA: No. The program was just launched last year. So as of now, the answer is simple. Exactly zero.

AUDIENCE: [INAUDIBLE] being spent in the last five years getting ready for this [INAUDIBLE] the next five years.

PATERA: Oh. Over the next five years? Yeah, as I said, there'll be roughly 35 times 5-- about 150 students graduating each year from the Professional Master's degree. And in addition, over the course of the program, I guess there will be about maybe 50 or 60 M-Eng and PhD research candidates that are also graduated.

MODERATOR: All right. Let's thank our speaker again.


The next item on our agenda is the panel to discuss education. The panel participants are E. Dan Hirleman, who is the professor and the department chair at Purdue University, P.K. Raju from Auburn, Professor Nam Suh from MIT, and Dave Wallace, also from MIT.

And so, the format is, each panelist will say a few words, depending on what they feel like saying. Then we'll have discussion of that. And then we'll have a group discussion of everything all together. Okay? Yeah. But then I think we can put these panelists up here, so that they can sit at this table and look exalted.


SUH: Okay. Since we have distinguished colleagues who will be speaking on their programs, I'll be very brief. Some of the things that you've heard, in terms of how we are trying to conduct education and whatnot has to be considered in the context of a lot of other things we are doing in the department of Mechanical Engineering.

So let me just say a few words, and then I'll ask our colleagues here to speak on what they have in mind. At MIT in our department, we created a complete new undergraduate curriculum. And we teach in a very different fashion. We graduated our first class last year under this new curriculum. So this curriculum is, simply put, an integrated way of teaching to provide better context for learning.

So for example, in the area of fluids, thermodynamics, and heat transfer, we have one sequence. So we do not have a separate subjects for thermodynamics, as such. But we teach all three combined, so that we can provide better context for learning. We have four of these sequences.

And we are writing new books. And new books will be published by Oxford University Press under this special series known as MIT-Pappalardo Series of Mechanical Engineering books. So thanks to Neil Pappalardo and Jane Pappalardo's support, we are able to create an endowment fund for book writing. And hopefully, from now into perpetuity, we'll have lots of books coming out.

And the first series of undergraduate core textbooks will be coming out sometime, I hope, next year or so. So we have invested a great deal of money supporting faculty effort to write new textbook, developing new teaching materials, and new ways of teaching, and so forth.

In addition, we will be spending-- we just got support from this MIT-Microsoft alliance, I guess, whereby we will be developing new ways of teaching. Dave Wallace talked about three different modes of learning. And to do that, we'll be actually developing all the details as to how those things can be implemented.

So one of the things we are doing is we are modifying our lecture halls, so we can indeed teach these two different modes of learning, namely scientific weight, scientific discovery, and Socrates way mode of learning will be tried in a new lecture hall. So we are building new lecture hall. And so that's ongoing, as well.

So we will be tearing apart our existing lecture halls starting this January. And then, on top of that, to implement this hands-on kind of teaching, we have renovated all of our teaching labs. And some of you may see it tomorrow. And so it has taken greater investment, in terms of both the time of the faculty and financial resource, to make it happen. And so we are continuing to put greater investment in these areas.

And some of my colleagues say that, gee, we're investing too much money in undergraduate education. But then, when you actually look at what we have done, most of our investment in education has gone into graduate education. And I won't belabor the point. I won't give you the details.

But most of our faculty members teach three hours a week, and the rest of the time do research and supervising graduate students. So in proportion to the investment made, in terms of faculty, salary, what have you, clearly graduate education has been getting most of our attention. In recent years, we've tried to make sure that our undergraduates get the best education any institution anywhere can provide. And that's the thing we have been doing.

And we have been helped a great deal by alumni with a vision as to what it is that MIT ought to do to make MIT the place where students should come and learn. So we have these many concurrent activities taking place. And I think in that context, you have to understand some of these things taking place.

So as part of now the MIT Cambridge program, there will be something very similar to MIT Singapore program, which will be launched sometime very soon. The money will come out of British government and the UK government. And we'll be, again, launching somewhat similar, but very different in content, and so forth, with the University of Cambridge in England. Again, with the hope of changing Cambridge to be more like MIT. That's sort of what we are told.

And this department would be the first department to be engaged in that experiment. So we will be accepting six students from Singapore. And we'll be sending six of our students to Cambridge. And one of the things we have to work out is to make sure that, when our students go over to Cambridge, indeed they can get all the credit hours that they need to satisfy, so they can graduate in the same time period.

So many of these things are going on. And indeed, there are lots of exciting things happening here at MIT. And I just don't have enough time to go into all of this. But perhaps during this evening and tomorrow, if you have any questions, we'll be very happy to answer anything you have in mind.

With that, I'd like to introduce our colleagues. Dan Hirleman from Purdue. And he will be talking to us about what's going on at Purdue. Dan?

HIRLEMAN: Thank you. Could you go ahead and try to turn on the projector, please? Well, it's my pleasure to be here with you today. I've really enjoyed the workshop. And I thought what I'd do is just briefly-- and just take a couple minutes. I think we're all going to just take five or so minutes, and then open it up to the group.

But I just touched on a few points of things going on at Purdue, some of which might be unique, some of which you might see going elsewhere. And the four topics in IT and education that I highlighted are virtual labs, distance design teams, innovation realization lab, and then web-based course-ware. And those are some topics that I believe other folks are working on, as well.

Why don't I start with, web-based course-ware. We have done some experiments with courses, call it web enhanced or web based. And our most recent example is a graduate course, a combustion course. And the components that come to mind in that, we have the lectures on streaming videos. So basically, the students can, anytime through the semester, tap into the lectures. Very similar to the last presentation.

We have then, in addition to that, PowerPoint notes in parallel. I think that the strategy that's been used in that is the notes are not complete. The notes have some blanks to encourage this active haptic participation, I guess you'd say, in note taking, and try to engage the students a bit more.

Yeah, could you please advanced the-- yeah, I'm sorry. You'll have to [INAUDIBLE]. So those are the six areas. And here's the topics I was just talking about. We have the on-demand viewing of lectures. And an uncertified stat is this Professor Gore in combustion this semester, the latest one. 10 hits per student per lecture. We think that's the students in the class trying to re-look at the lectures.

Downloadable notes with blanks. One of the interesting things which a number of you may have, as well, is the frequently asked questions bulletin board, whereas students begin to ask questions, and consistencies begin to develop. Then the faculty member works diligently to create a very nice answer to that and posts it.

There's a chat room involved. And then is divided the class up into distance study groups, and assigned students to be leaders in those study groups. I think those are about five students or so. The grade sheet's online, and then that's the web address for that particular course.

The next thing I'll talk about is, just briefly, distance design teams. We heard some of that. This involves interdisciplinary, multi-university design teams. And there's a number of experiments going on. I know Alan Pennington's here from Leeds. There's some experiments with Purdue and A&M-- the one that Alan's involved with is Arizona State. Leeds, Boeing, and Rolls Royce, if I recall.

So this as an undergraduate design team working across those boundaries. Industry advisors, as well. And then internet and white board capabilities enable that. If I could have the next slide, as well.

There's a project that's called the Innovation Realization Lab. It impacts on something we heard at lunch. This is an NSF project between Krannert, which is the management and business school at Purdue, and the engineering schools. This is where we pair an MBA student with a PhD student in the last two years of his or her PhD.

And the goal is then to get the-- I'm sorry, there's a typo there. That's PhD. The goal is to commercialize the PhD, the technology coming out of the thesis. So the two students work together and there's some network distance collaboration involved with that. So that gets to some of the comments at lunch.

Then the very last thing I'll briefly talk about is some virtual lab experiments. Some of our faculty use animation in classes. I'm sure that's common with you. And then one thing that reaches down to the high school level is something called RemoteScope. And if I could have the next slide, just briefly.

This is where at Purdue we're trying to work with the high school, in terms of education, inquiry-based science. And so there's a set of microscopes manned by experts at the University. And then in principle, the high school students are able to remotely access a set of slides. The next.

There's multiple samples you can see, multiple magnifications. And the students can control those few parameters. The very last slide gives you some overview of that. And in principle, it's a network of multiple types of microscope scanning probes-- optical, near-focal scanning. In principle, SEMs as well. And then, a set of high schools. Thank you.

SUH: Thank you, Dan.


I think we'll hear from all speakers first, and then take questions. And the next speaker will be Professor Raju from Auburn University.

RAJU: Yeah. I will go there.

SUH: Okay.

RAJU: Good morning. Thank you for inviting me to be on the panel. This morning, we have heard two distinguished speakers talking about bringing the technology marketplace interface into an undergraduate community classroom. So I want to share with you the methodology we're trying at Auburn. And we have developed some course modules which can be used at the senior level, at the sophomore level, and we're now trying to go to the freshman level.

So I get the others to bring real world issues into the classrooms. And that's collected [INAUDIBLE]. We have a group of faculty from engineering and business. And have a team of [INAUDIBLE] college of education.


And students from business and engineering work on these projects. The outline of my presentation. I'll make it very brief. Just want to share with you the goals, and instructional objectives, and some of the case studies we've developed, and the dissemination efforts.

The goals and objectives are to show the relevance of SMET education. SMET stands for science, math, engineering, and technology education and solving real world engineering problems. The educational objectives are-- connect SMET courses to real world problems, provide excitement of discovery, and motivate active learning.

And the second goal is to improve high-level cognitive skills of the students. The students should be able to identify criteria, analyze alternatives given multiple criteria, make a choice and defend the choice persuasively, be actively involved in learning situations. And the third goal is to integrate information technologies into the course material. That is, enhance synchronous learning opportunities and enhance asynchronous learning opportunities.

This includes learning off [INAUDIBLE] are provided through a CD-ROM, or even the web-based material, where the students can learn from that material, discuss with your peers. And the [INAUDIBLE] learning opportunities are provided for them where they come to the classroom and do the role playing that has been taken advantage of the people involved in the classroom in their problem.

So I challenge you to some of the examples of the problems we gathered. The methodology to achieve these goals and objectives-- shaping the future of engineering education. We have heard many people talk here. National organizations saying that saying the same thing. Industry is going to say the same thing. They want to integrate theory, design, and practice.

So what we did here was to develop the case study. And this case study, we have a pick in partnership with an industry. Identify a real problem there, and bring the real problem into the classroom.

That is the interview, right from the CEO to the technician who was involved in the problem. And tape record the conversation, document it, take pictures, videos, and put it all on the CD-ROM together. And the students will have access to books, everything [INAUDIBLE] on the baseline, as well as the CD-ROM. So we have a web-based material involved, the video involved, the CD-ROM involved. And we do case study.

The second object you have for us was to show the relevance of SMET education to real world problems. MIT may not have this problem, but some of our institutions have this problem. That is, we have a high dropout rate of students from freshman to sophomore year. Not because they can't understand the courses. Not because they have not basically already seen this course.

But what we found out was, they drop out from engineering because they don't see the relevance of the science and math taught in freshman level to the engineering process of what engineers do later in life. So most of them transfer to business schools.

So what we thought was to show them the relevance of science, math principles to the real world problems. And that we think we can do it by case study, using work are not familiar, as well as web-based case studies in the classroom.

Some examples of the case studies. This is a Della Steam Plant case study. I think I should briefly describe this to you. What happened here is, after regular maintenance of a turbine generator set, they started the turbine generator. And the system started vibrating. And the vibrations were so heavy that the building started shaking, and people ran helter skelter.

So we documented it and got the problem to the classroom, interviewing right from the plant manager to the technician, to show how decision making is done in the real world, and what vibrations are, and why causing vibrations is necessary. And how the vibration principles are related-- physics, math, et cetera. So mechanical engineering, vibration, safety, and instrumentation was the topics we looked at.

And we developed another case study with NASA-- design of the solid rocket booster field joint to consider ethical, safety, reliability, risk, schedule, and cost factors. Ethical considerations in engineering design, statistical methods. And this is a joint effort with NASA Marshall.

And then, learning from failure. We have another case study that is the Challenger episode, to teach the students what happens if you make a wrong decision, and how wrong decisions can be made in engineering designs.

And we worked with AUCNET USA. It's a Japanese company. They do real time online auto auctions. To show students the difference between satellite and web based systems.

And this is an interesting case study. We worked with Chick-fil-A. And I was surprised to know that the operating system you select for the point of sales terminal, where you have the employee selling you the sandwich, makes a difference in the delivery time of your sandwich.

Say for example, if they select Windows CE operating system versus NT system, you can get the sandwich five minutes earlier. And five minutes makes a difference. And I thought this is an excellent case to show to the students why they need to know about these operating systems. And we had support from the CEO of Chick-fil-A. And we brought this into the classroom.

Another case study we did was with a nuclear power plant to show thermodynamics, heat transfer, and nuclear power. And this is the design of a cooling tower. This is also an interesting case study. This is planning a maintenance outage. We were given access into the power plant when they were doing a maintenance outage. It's a $2 million project. We were there with our cameras and tape recorders, taped the whole thing, and developed the case study.

The idea here is to show how a huge project is handled in the real world, and what the surprises will be. For example, here they replaced a rotor in the maintenance. And in the end, they got a new rotor, and they were fitting it in. The rotor length was shorter than the required one. And that was a challenge for the plant manager. And that's one of the aspects we brought into the classroom apart from the other technical details.

Now I used a set of these modules as a course in my sophomore level class, teaching concepts in engineering design. We had two groups-- one using these case studies, one using the lecture mode of teaching the subject. And I'm not going into the details of the evaluation. But what we found was, a majority of students found this case study methodology to be interesting, exciting, and relevant, and useful.

And the students realized the connection of theory and real-world practice, perceived that the method assisted them in enhancing the learning process. And multi-media materials helped improve learning-driven factors.

We have also developed a model to study the differences between a multi-media lecture and written notes-- how the students perceive it. And we have a publication out of it. And again, the learning different factors, like learning interest, challenging, self-reported learning, and learn from others, are the important aspects of using this multi-media.

We have tried this in different courses. Already about 710 students went through this. We tried it in business, about 880 students. We tried it in Alabama A&M and other minority institutions. And they found it very useful.

Now, I want to make a sales pitch here. As a part of our dissemination effort, we are organizing a workshop from May 11 to 13. And I have some brochures here. I would encourage you to ask your colleague or colleagues to attend this workshop. The idea here is, we want to provide them hands-on experience of these case studies so that they can use the modules in their classes. Thank you very much.


SUH: The last panelist here is Professor Dave Wallace of our department. And he made a presentation. Dave, would you like to add some more?

WALLACE: I'm on a short tether. Anyway, I'll make this very brief, since I've already said a fair bit about the subject. What I thought I would do is just briefly describe the undergraduate course that I teach. It's a core course of 120 students per year. And it's a fourth-year course, which is the last in the design and manufacturing series. And it's a course really about the developing products and the engineering process involved in that.

And what we do is, we really motivate the entire learning experience by putting the students in the position to develop substantial products, which they really have the ownership of the idea. And they work in large teams to do this. And they also work with real budgets of $6,000 per team that they can use for supplies. And there's also support and other materials that we provide as part of the course.

And so the goal is really to build a real working alpha prototype. And to start the course, what we do is we provide them with a technology theme, and really, that's about it. So for example, last year we had our theme to be that we are interested in developing products that assist the active elderly population. And our technology theme is, we develop electromechanical devices. Pretty broad.

And so the students really take it from there and go through the entire process to design and develop these alpha prototypes. Examples of the products designed this last year include what's called a garbage porter-- a device that picks up garbage pails, and is almost like a powered walker. That allows elderly people to take their garbage out to the curb. Which actually is a large issue. It's an issue for my mother, in fact. But in fact, a large number of people have that problem.

Another product which you can imagine is like the standing La-Z-Boy. But what's unique about it is it also can be driven around like a wheelchair. So it provides mobility inside the house while in the chair that helps them also get up and down from the standing position.

Other products include a powered wagon, because gardening is one of most popular activities for elderly people. But it's a powered wagon with a very nice force feedback handle, so that it actually has no controls, but just feels like a very light wagon. No matter what you're doing, it just walks along with you, based on your pulling on the handle.

So that's what we do. And to do that, we try to take an approach where the faculty work very directly with the students. So the teams that are developing each product-- there's 15 students for each team. There are two faculty members involved that are really down working in the shop with them throughout the process. And we're really following the just-in-time paradigm for doing this.

And from a teaching standpoint, this is a very fun and challenging thing for the instructors, because you don't know what problem you're going to be solving any given day. And you get used to the notion of saying, well, I don't know how to solve that. But I think we can figure it out. And that's a very real part of engineering and research, for that matter.

Also, in addition to the faculty, we have alumni mentors that work with the teams-- typically two to four mentors. Some of them in the area, some of them at a distance to provide additional expertise. And to support that, we're working on, of course, the development of the web-based modules, like I was talking about, both on the process of product development, and also on different topics.

And we're also trying to work on access to expertise through alumni and other interested patrons, really. And so there's a lot left to do, I think, on the latter part of those support resources. But I really do think that this is, in some way, is the way to teach product design in a unique way that can really only be offered through the facilities that a residential campus can provide.

SUH: Thank you, David. I guess it's time for you to raise issues, questions, comments, what have you. So the floor is open. Yes?

AUDIENCE: Dave, I'm curious. Is your group or your work the only work going on at the Institute in regard to residential training and web-based training?

WALLACE: No. There's a lot of--

AUDIENCE: Could you repeat the question?

WALLACE: The question was, are there other activities going on on campus. And the answer to that is, yes. There are quite a few. There's CAES, which is the Center for Advanced Engineering Studies. It does a lot. There's the MIT-Singapore. There's work going on with Cambridge.

There's also the Microsoft iCampus Initiative, which is funding a number of different proposals to work on different aspects of use of technology on campus. There's the d'Arbeloff funding, which is also related to that. So there's a lot of different activities.

AUDIENCE: Yeah. The d'Arbeloff funding, what's it, $10 million?


AUDIENCE: Specifically for enhancing residence based learning using information technology.

WALLACE: And I believe that's being focused on that first year. Yes?

AUDIENCE: [INAUDIBLE] as we've talked today, with this new way of teaching, it's going to require much more enormous effort from the faculty side in the new way of teaching. When it comes to [INAUDIBLE] promotion where only research publications are given high priority for teaching in large universities, what is MIT doing about that? Do they get credit? They end up professors when they get to that stage?

SUH: Well, all of us would have somewhat different answer to it. So my personal answer would be that I think we are able to attract top-notch faculty members who can do extremely good job in both. It's not the case of doing well in one and doing poorly in the other. And it turns out that our faculty members in this department are excellent teachers. And also, they have an international reputation in their field.

And so I'm not sure it's matter of a trade-off. It's really doing both jobs extremely well. Just to give you an example, I'm very pleased this year that two of our colleagues received what MIT call MacVicar faculty fellowships. What MIT does is select few people each year. And they will have up to 50 people. And they give them a special allowance for being a good teacher.

It turns out that this year, MIT selected six out of-- that's MIT-wide, that we have 1,000 professors-- and MIT selected six. And out of six, two of them are professors of mechanical engineering. And in fact, one of the professors clearly got the recognition because of his involvement in the undergraduate teaching.

14 of our professors are what we call designate professors. These are the professors who are implementing our new undergraduate curriculum. They are the ones writing books and they are devoting great deal of time to make it happen. Some of them are not tenured. And indeed, we look at teaching record very, very carefully. And if someone is not a good teacher, probably he or she will not be allowed to stay at MIT as a tenured faculty member.

So often we have discussion on this teaching versus research. But I say, one ought to be able to do a good job in both. Yes. Jim?

AUDIENCE: Professor Suh, would you talk about the new undergraduate curriculum?

SUH: Right. This, as I said, we graduated the first class in 1999. And we actually discussed this need to revamp our undergraduate curriculum for many years. Finally, we decided to implement it. And we had a great deal of disagreement as to when is the right time to improve the new curriculum.

Some people want to see all the details worked out before we implement. And some of us thought that you can not really develop all the details, because it takes enormous amount of effort to do that. So we decide to do this. And essentially, the idea is that we wanted to make sure that we look at learning from the student's point of view and provide right context.

So we felt that by combining some of these subjects in a single sequence, we can provide the context. As I said earlier, thermodynamics, heat transfer, fluid mechanics is in one sequence. And at the beginning, there was a great deal of controversy, because lots of professors complained about the way the person in charge was teaching the subject.

But now there is no controversy, in fact, because students like it. You can think of the way we teach in some of these subjects like peeling onion. We teach many of these principles at the beginning. And then we solve a series of more and more difficult problems. And through that, students begin to understand the context. They begin to understand the theories in the context of how real problems are solved.

And I think it's working out well. And I can give you some of the stories. Some of the professors in our department didn't like the way that that particular subject-- and I can use some other sequence, and I can give you the same story-- did not like the way it was presented.

And it turned out that we just hired one of the senior professors from our side in the fluid mechanics area. And he taught the subject in two different ways. And then he came to the conclusion the new way is better ways of teaching it. And that stopped any additional discussion on the subject, because one person who was not biased one way or the other came into teaching of this subject, and then decided that the new way of teaching is better.

Normally, when we teach it this way, students complain a lot during the first term. In this subject, it's a two-term sequence. First time they complain a lot, because they are given all these basic principles all at once. The ones here are thermodynamics, heat transfer, and fluid mechanics.

But at the end of the second term, students really feel good about it. If you look at student evaluation, that's exactly how it comes out. And so that professor who did that was selected as the MacVicar teaching faculty fellow this year.

And you may think that I'm boasting. But in some ways, I guess I am. But one of our young faculty members was so great. He is untenured faculty. Anyway, he was elected to be this chaired full professor in the department of Applied Mathematics and Physics at the Cambridge University. And I am trying hard not to lose him, but it's a very strong statement that indeed, we have very strong young faculty to be chosen as one of the few senior faculty members at the University of Cambridge. And he's not even tenured here.

So anyway, we have very strong faculty. Yes?

AUDIENCE: This morning we saw how information technology has been integrated into research projects to make new intelligent machines more innovative products. But this afternoon in the education session, we saw information technology used only as a tool to deliver the education to the students. Has information technology been integrated in the curriculum, so that the graduating engineers will be able to develop intelligent products?

SUH: Well, Dave Wallace's presentation really dealt with that issue. And a number of our professors are incorporating technology in many of their subjects. They use lots of simulations in their teaching. But I must confess, I guess we're seeing the early phase of introducing information technology in our courses. Yes.

AUDIENCE: I do not mean as a tool. But I mean in the subject content, like the mechanical engineer is learning information technology subjects to integrate them into their mechanical engineering training.

AUDIENCE: I can talk about that, actually. We have, over the past five years, a fairly large number of courses have been developed, in fact. For instance, [? Sunny ?] Suh is teaching a graduate course on the internet. And you can imagine that he's part of the--

SUH: [INAUDIBLE] registered.

AUDIENCE: For both graduate and undergraduate. And an undergraduate professional sophomore course based for the internet. I believe you're also on a committee to look into the possibility of having a computer programming requirement-- is that?

Yeah. I have developed both graduate and undergraduate courses in mechanical engineering, specifically on the subject of information. And together with Paul Penfield, who just stepped down as head of Electrical Engineering Computer Science, we're teaching a freshman course on what information is-- so to introduce freshmen to the subject. It's got 70 people in it. It's doing pretty well.

We have courses on mechatronics. We have graduate courses on information, adding information, and control. Actually, information is really quite well integrated at the moment into our curriculum. We don't have any required. It's not required in the part of the undergraduate core. But there are many options for undergraduates in Mechanical Engineering to learn about information technology.

SUH: Yeah. We are very serious about the incorporating information technology in all phases about undergraduate and graduate education. So I think you'll see more of this happening in the years and decades to come. Yes?

AUDIENCE: And I actually found a further on that. That one of the things that I've often heard all the professors in the department complain about here--

SUH: You mean senior professors.


AUDIENCE: Senior professors. Is that the incoming students have never used a retro screwdriver. And actually, junior professors complain about this, too. And it's true. All right? People used to take apart cars and take apart mechanical devices. They don't any longer, because when you take them apart and put them back together, they don't work. All right? So you can't do it.

Now, this is seen as a big problem in teaching mechanical engineering. How do you teach mechanical engineering to mechanical illiterates? However, there is a flip side to this, which is that the students who are coming in right now are often much better off in terms of knowing about information technologies than both junior and senior professors from the department.

How often have you asked a graduate student to make your disk drive work, right? I've certainly done that a bunch of times. And these students, actually, they come in, in some sense, they don't know about fixing machines. But they know about fixing bits. Right? They're primed to know about bits.

In our freshman course that I'm teaching with Paul Penfield, it's amazing how much those students know about information and information technologies. It makes it really easy for us to teach the course. Because it's as if they were taking apart cars. Except they're not. They're taking apart computers, installing disk drives, installing programs. They know about information technology.

So what we're teaching them reflects what they actually come to us already possessing.

SUH: In fact, just to augment what Seth said that we are teaching now. We just created for the first time this term freshman subject where students are asked to take things apart and assemble it, understand how mechanical things really work. And also, at the same time, we try to teach them a little bit about history of technology. So this is a freshman elective subject. And we think that will do them a lot of good.

AUDIENCE: I think one of the most important information technology incursions into mechanical engineering is in the whole area of measurement. The society is becoming ubiquitous with measuring devices. And I think it's absolutely important that our students know the ramifications of that, and how to deal with the information that those instruments deliver.

I think it could be a really important distinguishing characteristic of the next generation of students that they feel as comfortable using A-to-D and D-to-A devices to control, and manipulate, and design things as they do using wheels, and gears, and pulleys.

Because that's where it's at. And that is, in fact, information technology. They need to know how to deal with that information in an economical way and know the pieces that go into it. I have a graduate course on advanced instrument design, which is basically focused on that issue.

SUH: Yeah. Just to add to what [INAUDIBLE] has said. We have about 60 professors. And you can see that you saw roughly maybe one sixth of our department here today. And people in other fields, fields like manufacturing, by instrumentation, and so on, and so forth. They are doing a great deal of research, as well as trying to create educational paradigms involving IT. So you haven't seen that aspect.

In fact, many of our manufacturing colleagues are working on the things that require information technology, as well. But you haven't seen that. Just too many people. Yes.

AUDIENCE: I have a question for Dave Wallace. He quoted the sobering number of 500 hours of preparation time for one of lecture material on the web. Do you have any suggestions for dealing with that seeming roadblock?

WALLACE: Sure. Well, I guess the way to think about those materials-- the way I would think about it is, people are used to spending a long time writing a book. And there is that very large figure of preparing the materials. To be honest with you, that same lecture, for me to do it live in class, required about 50 hours of prep, because of a lot of the props were actually the same.

But some of that time was production time. Some of that time was intellectual content development. The pure intellectual content development was on the order of the 50 to 70 hours. The rest of it was some form of production or other. And so one of the clear messages is, there needs to be an infrastructure in place that makes it a tenable proposition, so that, if you will, the different people in the chain of producing the material contribute in the area that is their most valued competency.

And so that's one of the things that would chain. Because obviously, that's not something every faculty member is going to do for every lecture. So there needs to be the appropriate chain. And then there needs to be the delivery mechanism, so we can choose the subjects, and do that for the ones where there is the audience to support it.

And it actually, I think, can fairly quickly end up being a very economical way to do things.

SUH: Dave can do this in about five days or so. He puts in over 120 hours a week.


So 100 hours of effort is not that much. Any other comments? Yes.

AUDIENCE: I sense a beginning a revolution in mechanical engineering education here. And it's a rude awakening that Seth mentioned over here, that there is a group of incoming mechanical engineering students that do not know gears and these traditional mechanical elements.

And we try to redefine what mechanical engineering means, or do you foresee some renaming of mechanical engineering to incorporate these new interdisciplinary aspects of mechanical engineering? What's happening if other departments in the nation follow MIT, would we be looking at renaming mechanical engineering to fit what is really happening here? Or are we redefining mechanical engineering?

SUH: I'm not really sure how to answer that. But we are really broadening what it means to be a mechanical engineer, so that our graduates can deal with the tasks they will be facing. In industry, even in academia, it's no longer good enough to have a Mechanical Engineering department solely made up of thermodynamicists and applied mechanicians. And still a large number of schools do that.

And indeed, that's one of the motivations for having this conference is that, if we continue to teach only mechanics, and thermodynamics, and what have you, then I don't think they can become a leader. They can only deal with a subset of the overall engineering tasks.

Because nowadays, you cannot deal with overall engineering tasks without knowing software, without knowing control, without knowing how to deal with information technology, how to manage people, and so on and so forth. And then at least by exposing our students to what's out there, by creating a big window through which they can look out, at least knowing that there are such sub-branches of engineering discipline, they can become more of a engineer who can become leaders who can shape the future of technology and industry. So that's the goal.

AUDIENCE: But if industry out there has a very traditional mindset on mechanical engineering. So when they hire some people for mechanical engineering, they have a certain mindset as to what they want from these engineers. Maybe industry should be involved in all of this process, as well, so mechanical engineers are really trained with other interdisciplinary IT-based education other than just--

AUDIENCE: I don't think you have to worry about involve the business. [INAUDIBLE]

AUDIENCE: I'd like to ask another two professors from the other schools whether you buy this argument or don't buy this argument?

HIRLEMAN: Well, thanks. I've made a couple of notes relative to the hands-on--

AUDIENCE: [INAUDIBLE] of the discipline across there's no such gearbox is always purely mechanical in some way terminal. The things we do today are much [INAUDIBLE].

HIRLEMAN: I think we don't believe we want to lose the some of the hands-on aspects of mechanical engineering. So we offer something we call a hands-on short course, where they actually take apart an engine and put it back together. And it has to run or they don't pass.

And in fact, it's probably the most exciting we see the kids is when they fire-- I think we get lawnmower engines from one of our sponsors. And when they fire up that engine, and it actually comes up, that's one of the highlights of their sophomore year, it turns out.

In terms of information technology in the curriculum, I think we offer measurements at the undergraduate level. And we factor in-- I mean, I even taught assembly code to mechanical engineers for a while, because I believe you have to know what a computer is in the architecture to succeed. Now, not everyone agrees with that approach.

And then, in terms of IT, I think we're just starting to move down to the undergraduate. And we have a course that's our senior design and manufacturing. It has rapid prototyping, CAD/CAM type things. We offered that in conjunction. So that's undergrads elective. We offered it in conjunction with a smart machines course. So we had teams of grads and undergrads who had to build a device with an embedded micro-system. So it actually had to carry something out, so an electromechanical system. So I think we're having it both ways.

RAJU: I agree with my colleague here. But some of the restrictions our schools have, and probably MIT doesn't have is, we have to satisfy ABET criteria. MIT probably doesn't have to. So we are restricted, to a certain extent, to cater to the industry needs of our region.

Say, for example, I'm coming from Auburn Alabama. So we have to have our industry input into our curricula, which is the basis of ABET 2000 assessment. And so that's the reason why, for example, my presentation concentrated on developing modules to help our professors in getting the ABET 2000 criteria satisfied.

But MIT probably has the luxury of not doing it. But that doesn't mean we are not taking the cue from MIT and the discussions we have here. For example, information technology is integrated into some of our courses in the form of mechatronics, instrumentation, and other things. So that's where the difference is.

AUDIENCE: We mentioned there is no question about the importance of information technology in mechanical engineering. But it may be unique in Berkeley, but we try and improve real [INAUDIBLE] application coming from high school. There is a huge concentration in computer science area. So [INAUDIBLE] so simplify the equation that information technology equals computer science among the high school kids.

So I think it's not [INAUDIBLE], I mean redefining mechanical engineering. But related to the whole question, unless this aspect may be publicized strongly to the high school kids, I think we are near a problem. Because a fate of our school the how much enrollment of each department may be a basic mode of [INAUDIBLE] our income. [INAUDIBLE] so I don't count on MIT is experiencing that kind of enrollment, where you are seeing that as an issue.

SUH: Well, obviously, everyone watches student enrollment, where there go, and so on, and so forth. But basically, my personal view is that we have to do what is right for our students. And if you don't do it, the kind of things we are talking about, you can not even mat the control engines nowadays, right? Engine management system controls engine as well as transmission. And without this integrating tool, which is information technology, we cannot do all this.

One of the things that was kind of interesting, amusing, and I was going to tell the visiting committee on Monday, and you might find it interesting. Our mechanical engineering enrollment at MIT is still quite high. We have 300-some students-- sophomore, juniors, and seniors. And I think in terms of student enrollment, we are the second-largest department.

And at MIT, four departments teach about 70% of all our undergraduates. And we have 20-some departments, but four departments teach somewhere around 70% of our undergraduates. And we are the second largest in terms of undergraduate enrollment.

And during the last five years, our undergrad enrollment went down from 365 to 315. And we're wondering why. And in preparation for the visiting committee, I was looking at some of these numbers very carefully. Guess what? And since I look like an Irishman, I can say this, eh?


It's the Asian-Americans not coming into ME. We had about 115 Asian-Americans five years ago. Today we have 65. So that drop is all in Asian-Americans. And I blame Confucius' teaching. Because--


Because Confucius' teaching always put these people who don't do any work and who used their hands and want to compose poems as being the highest class. Right? Anyone who had to use hands were at the low end of it, just one notch above the merchants, eh?

And I think that culture must permeate with all Asian-Americans, because their parents must tell them something. But it's interesting statistics. Number of woman students did not decrease. Number of the Caucasian students did not decrease. It's the Asian-Americans that decreased in mechanical engineering.

So that the problem that you're alluding to may have less to do with the all student population leaning toward computer science, or anything like that. But certain segments have certain cultural bias, or whatever the case might be. This is something that I'm trying to understand. Yes?

AUDIENCE: I counted the number of students that graduated in computer science in Concordia. And 85% of them are Asian. In terms of the numbers through the Computer Science department. So that's [INAUDIBLE] that phase that draw towards that direction. But I have a question involving what happens. I think the expectation with society for engineering professional and also with students.

We seem to expect our student to be able to do hardware, also within the software, be able to do management, good [INAUDIBLE] skill, good social aspect, and so on. But in terms of the program, we have a four-year program. I don't know what to explain to MIT, how can you fit all of that into the curriculum and let student finish in four years?

SUH: That debate has been going on ever since engineering education got started about 150 years ago. But if you just look at just a row of the data, the answer to that is relatively simple. And certainly my colleagues in electrical engineering and computer science would differ from me, because they had went different ways.

The knowledge that all engineering students had to acquire over these years has increased all the time. In fact, people like to think that the knowledge required is increasing at an exponential rate. And I'm sure 50 years ago, 70 years ago, people said the same thing. And yet, we don't live that long.

I mean our life expectancy went up a little bit. Longevity went up a little bit. But it hasn't gone up that much. And as a result, we have to codify what we know. We've got to be able to somehow improve efficacy of education process so that we can, indeed, convey more knowledge in the same period of time.

At the same time, some of the things we used to teach our students are no longer needed, because we have better technology. And indeed, the things-- I still use my logarithmic table at home. I have on tablet. And whenever I don't have the calculator, whatever, I use that to calculate it.

But our young students, I don't think they would know how to use it, right? But they don't have to use it, because at least I was taught in class how to use that table, right? But our students don't have to do that. So I think there is a certain degree of efficiency, efficacy, and whatnot you can bring into teaching by using technology. In that sense, IT is going to play a major role. David?

WALLACE: I just wanted to add one thing to that, which is, I think that with information technology and all of that, of course, the increased access to information modules for learning, I think also we can really expand in the area of lifelong learning, where you view the on-campus educational experience as where you need to get the core and the backbones. But that you aren't really going out there unless you know everything, prepared for everything you know means a much better job of providing a mechanism that you can continue to learn as you need to.

SUH: Yeah. [INAUDIBLE] just-in-time is going to very important. So people can just go into a module, and learn what they need to know so they can do the job. Yes. Would you identify yourself first? I should have asked here. So that you are the first one to identify yourself.


AUDIENCE: Bill [? Predlock ?] from Michigan Tech. I was just at a conference at the end of March. It was [INAUDIBLE] by SME group for the Mechanical Engineering department heads on trying to get change in mechanical engineering education-- motivated change. So it was roughly related to what we're talking about.

One of the speakers was a common speaker. Coincidentally, he was from MIT, and coincidentally he was from the Mechanical Engineering department. And I wanted the panel to respond to this. One of the things he said was stop lecturing, stop writing books. So you probably know who he is.


I wanted to know your response. Is that the future?

HIRLEMAN: I guess I'll give it a try. I think it may get back to what Dave said about the learning styles. And I think some students respond to lectures, and some don't. So I think at Purdue, we are trying a number of different teaching styles to address a number of different learning styles.

So we have faculty that don't lecture.

SUH: In other words, you cannot create the kind of thing that Dave Wallace created on a video without thinking through the entire process, okay? And so in order to be effective in teaching, you have to codify. So that our students, when they leave MIT, when they squeeze what they learn, there will be something remaining in their hand, in terms of knowledge.

And so I think we will go through a transition, and probably the people who said that books are needed starting over 100 years ago, practice is more important. And so forth. We went through this whole argument, that design should be taught by letting students actually just get experience, rather than teaching them any theories.

And that discussion, by the way, went on in other fields, as well. But what made the engineering education so effective, so efficient, is because of codification. So that instead of teaching how something should be done, we came up with a control theory. By teaching control theory, we have been able to not deal with controlled non-linear systems, and so forth.

And that would not have been possible without scholarly work. And writing a book-- codifying it-- is scholarly work. And so we can not diminish the importance of scholarly work in education. Otherwise, we will just have a bunch of arm waving people. So we just cannot do that. Yes?

AUDIENCE: I believe the distinguished speaker last week mentioned that the engineering degree is only the new liberal arts degree for this century, and should be recommended message to the high school students by just more. Where is it all on mechanical engineering? Since mechanical engineering is the broadest of all engineering degrees, will it become a new liberal arts degree? Any comments on that?

SUH: Anyone care to comment on that?


WALLACE: I can say at least the belief that certainly related to product development, increasingly, in a sense, the mechanical engineers are the ones that get it to work. And so what that means is, which is, I think, very consistent with a lot of things we're talking about, they have to be pretty well-rounded people.

Now, if that's what you're implying, yes, I think the mechanical engineers, they need to have the course. There's a core of the discipline. And you need, in some ways, I think, to be deep in something. But at the same time, will be increasingly versed to deal with a lot of these different situations.

AUDIENCE: Recommend that to the high school students and let them know that [INAUDIBLE].

SUH: I was told by the chair that we've got to wrap it up. So I have to follow his order. At the same time, since I'm standing here, let me just say this. I think most important part of engineering education is teaching students how to look at facts, how to look at basic laws of nature, basic ways of synthesizing things, and then think logically.

And the power of logical thinking is what we are teaching in engineering. And that can be applied to a number of other areas. In many other areas-- it's interesting. Sir John Sununu said this at one of our distinguished alumni segment, and they're quite true. In many other fields, they draw conclusions first. And they rationalize, justify it by bringing all sorts of quotations saying, so-and-so said this. This is same as what I said. But ignoring all other things that person said.

But in engineering, we don't do that. And I think that power of rational thinking is important in all areas. And I personally like to think that some of our undergraduates will go into other fields and influence other fields, as well. So if you interpret that as meaning that is the meaning of liberal education, so be it. Okay. And let's have the dinner. Okay? Thank you.