Language Once Was Key—Now It's Design
Grady Booch, Rational Software's chief scientist and a pioneer in software modeling, tells you when you do—and don't—need to get into serious design, how to deal with the impedance mismatch between SQL and OOP, and more.
by Lee Thé

February 2003 Issue

Grady Booch, one of the fathers of OOP, has an impressive resume in the programming world. He's Rational Software's chief scientist, co-developer of the Unified Modeling Language (UML) and the Rational Rose analysis and development tool, and author of books such as the UML User Guide (with Ivar Jacobson and James Rumbaugh) and Object-Oriented Analysis and Design with Applications. He belongs to the Association for Computing Machinery (ACM), the Institute of Electrical and Electronics Engineers (IEEE), the American Association for the Advancement of Science (AAAS), and Computer Professionals for Social Responsibility (CPSR). Equally versed in .NET, Java, and Open Source, Grady is more than qualified to comment on .NET and enterprise development from a global perspective.

Q: It seems development platforms such as .NET and Java are going to dominate the world of programming for the foreseeable future. How this will influence (impact) the world of programming? Which are in your opinion the key benefits and limitations?

Booch: Let me step back. In the role I play at Rational I worry about the next three to five years. I have my eye on about seven or so technologies relative to the art and science of building software systems. One of the key ones on that list is the rise of platforms—both .NET and J2EE. Those are relevant because they raise the level of abstraction for developers. And indeed you can characterize the whole history of software engineering as that rise in level of abstraction.

As for .NET and Java "dominating" the world of programming … that's an emotionally laden word. Actually those platforms are irrelevant to lots of things in the programming space—mainly those that have no GUI, such as embedded applications and issues of integrating legacy headless applications. But .NET and Java are going to be dominant elsewhere, especially in the Web-centric space, which is the fastest growing and perhaps the most important one. The .NET and Java platforms are going to be dominant there because they address the key pain point of raising the level of abstraction. Companies such as Microsoft are encouraged by the whole marketplace and their business model to keep raising the tide for what its platform covers.

.NET and Java effectively mean you're no longer building on top of a bare operating system. They have codified a lot of the mechanisms you used to have to build, thereby decreasing the distance between the platforms and the actual application I have to deliver against.

Q: Do you see any limits to this evolutionary process?

Booch: Just the issue of complexity. .NET isn't easy to absorb. But then we're working in a domain where applications are intrinsically concurrent and distributed, and must be secure. That is not a simple domain in and of itself. Not to be disrespectful of the average code warrior, but building distributed concurrent secure apps isn't a core competency of such people. They may make some good local decisions for those three issues; however, building a Web-based system that is extensible, scalable, and secure requires good global policies as well. One of the advantages of platforms such as .NET is that they codify a lot of those hard things so that global decisions can be made intelligently, then carried out locally by the individual developers.

Q: Semantically VB.NET is nearly identical to C# or Java. The limits of VB6 are gone, but are VB developers ready to master the power of real object orientation? Is there a risk of us seeing "object-oriented spaghetti code?"

Booch: I used to be a language bigot; I thought ADA was the way the world was intended to program. But we're talking a couple of decades ago. It predated even VB I think. Alan Cooper had done his thing around that time. And at the time I really thought that the choice of language was one of the most important decisions you could make on the development team. But then I encountered this wonderful language called Smalltalk, and C with classes which because C++, and then all the scripting languages and eventually Java. As I go around the world working with lots of projects, the choice of language is important because it brings to the table lots of specific skills, lots of specific tools, lots of specific patterns and idioms for use. And there certain things I can do better in some languages than others.

But by and large interesting software systems tend to involve many different languages. And the difficult decisions are the ones made above the choice of the language itself. Still, in the presence of any new language there's always the issue of moving up to that new language. I have certainly seen that moving from previous versions of Visual Basic to VB.NET is disconcerting for some because there are enough impedance mismatches to require you to make changes to your code base.

In some cases you can conclude that there's not a compelling business reason to move to the latest and greatest. On the other hand, if I continue to build systems that I never turn off, that are extremely long lived, then you always have to deal with that kind of absorption of new technology. You find systems with many moving parts that all change simultaneously. And the introduction of a new language is one bit of that.

The choice of a language is somewhat orthogonal to how one really designs. I can write beautiful code in VB.NET, but I can write really bad code in it as well. People come to me saying "We have legacy Java problems" because they've absorbed Java from day one; they've written all this stuff, and it's really ugly. It has nothing to do with the language but everything to do with how they architect with the language and with the practices they use to do it. The language becomes an enabling feature, and I can fail just as easily in VB.NET as I can in Java.

More and more this is why I say the choice is becoming so important. Over time it renders the choice of language far less interesting. And of course Microsoft has taken the strategy "one platform many languages." That's especially interesting for organizations that already have a multilingual situation.

In the presence of Web services the choice of languages becomes even less important. Because you're going to use little snippets of scripts and lots of different languages on the Web user interface side, such as VB, maybe Java, lots of things. Again, it's raising the level of abstraction. Languages now help us glue those little pieces together and therefore fit in the interstitial spaces of our systems.

Q: Is software development an art, a science, or a branch of engineering?

Booch: Yes. [laughs] I'm encouraged to see the growth of our industry as it moves towards being an engineering discipline. Calling yourself an engineer is actually illegal in most U.S. states if you aren't board certified. In Texas the state Board of Professional Engineers has taken on the responsibility of suggesting engineers be licensed. It's a good thing. There's a similar effort to license software engineers in Canada.

In the era of the dotcom world a lot of bad code was written. And in retrospect I think you can claim that one of the reasons certain dotcoms failed is because they built profoundly bad infrastructures for their businesses. These were companies that relied on their ability to execute in software … of course some of them also had profoundly stupid business models. But even with the best business model if you can't execute in software in a software-intensive business you're going to fail.

Bjarne Stroustroup, the inventor of C++, says that our civilization runs on software. And as civilization relies more and more upon software, then I believe people in the software profession have an ethical/moral responsibility to do well in the profession. As a nation we do not tolerate it if you build bridges that fall down. Yet we allow people to do that in software on a daily basis. That's a sign of the immaturity of our industry. On the other hand you can find groups in our industry recognizing that perhaps there is a body of knowledge defining what good software engineering is and isn't. For example there's the IEEE's effort on what's called the "sweet box of software body of knowledge." And the licensing efforts I talked about in the various states and countries. Those are good signs.

Q: Does Microsoft's certification program help?

Booch: Sort of. Most of the certification I see from the various vendors deals with "Do I know how to really deal with technology X?" But being a software engineer (or any other kind of engineer) involves fundamentals that transcend any particular point technology: Do I know how to abstract? Do I know how to work with a team? Do I know how to manage stakeholders in building collaborative products?

I have a huge book from the American Institute of Architects called the Architect's Handbook of Professional Practices. It deals with the licensing of civil architects, and in it you'll find things in the technology you need to know about, including certain stresses and physical materials and their properties. That's the foundation of the profession, and most of the software vendor certificates deal with that part of it. But we've missed a lot in terms of the science of engineering that says "How do you truly engineer systems?" "How do you architect these?"

You see these debates going on in all the engineering disciplines, including "Are we art or are we science?" Because in our field, as in all the engineering fields, we're not after the discovery of deep scientific facts. As engineers we build things that are useful. And that building requires hard science. You want to build things that won't fall down upon you. In the software sense we want to build things that are reliable, sustainable and so forth. But it's also the case that one has to be creative. That's the artistic part of it.

Q: Alan Cooper has talked a great deal about what he feels are efforts by software engineers to usurp the field of software architecture by presenting engineering standards as architectural standards. He claims that they're different.

Booch: I agree with him. I'm staring right now at the IEEE standard 1471-2000—"The IEEE Recommended Practice for Architectural Description for Software-Intensive Systems." It's a reasonable practice. I have to say I'm a little biased because much of the deep thinking in it comes from Philippe Kruchten of Rational and his insights about multiple views of architecture. Many of the architects I encounter in the world do get this notion that you have to get multiple views. But what the industry lacks—and I think this is what Alan is seeing—is that we just don't know what the right architectures are for many classes of systems. So, to standardize upon some of these now is woefully premature.

Take the Web space, for example. This is a genre of systems that has been around for a couple of decades in one incarnation or another. At the beginning we as an industry simply didn't know the right way to build Web-centric systems. So a lot of experimentation happened. In effect we had new materials. People tried things. Some didn't work, some worked well. Those that did work well become codified in standards such as what the W3C is doing, what the Internet Engineering Taskforce is doing, and even in platforms such as .NET and J2EE.

As we discover those patterns they get codified in tangible products. I'm a great believer in the open market, in capitalism. So I'm delighted to see that there's not just .NET but also J2EE and Open Source and, yes, I'll say that even in a Microsoft forum. It keeps every one of those players honest. It forces them to keep improving what they're doing. That's great for end users over time.

Q: I've heard Microsoft criticized for having a total monopoly with Microsoft Office and that the tools are not evolving properly because of that.

Booch: I'm not going to criticize Microsoft, but I believe choice is a good thing.

Q: Do you agree that there is a huge mismatch between relational SQL databases and OOP modeling? If so, how much has UML helped to bridge the gap? Why is UML better than other approaches?

Booch: That mismatch happens on both a technical and a social level. A technical level in the sense that relational databases are a different universe than object-oriented databases. Socially in the sense that database people grew up in a different culture from the programming culture. The two communities are full of their own notation and language and idioms. So it's difficult to bolt them together. Nonetheless, for many classes of systems one has to do it.

If I'm an application developer I tend to see my world in terms of objects, because that's what my end users manipulate; that's what messages look like. For example, if I'm streaming out XML, XML schemas in particular look object-oriented in nature. If I've got views of something, those are also object-oriented in nature. Well, how do I bolt that to a database? A dozen years ago a whole set of companies coughed up what were called object-oriented databases. I don't think any of them have survived to any tangible degree. I'm sure I'll get some nasty e-mail after this comes out because somebody will say, "Oh but we're surviving."

Q: Except in small niches.

Booch: Exactly. They're niches, including what Oracle has done with providing the object to relational blurring. But one of the most common patterns I see people building in enterprise systems is reducing the impedance mismatch between their object (and XML) schemas and the relational database itself. I believe they'll continue to do so because of the intrinsic social and technical mismatch between the two.

That said, over time the direction Tim Berners-Lee is headed with the Semantic Web makes the whole Web look like one big object-oriented database. So there are a number of technology trends that continue to push the object-oriented database model, and I think it's a great thing. Yet a number of pragmatic trends continue to foster relational databases.

Another reason for the mismatch has to do with how you use the data. If I have applications that need to navigate semantic links, that mandates an object-oriented view of the world. But if I want reports that do simple searches and things like that, a relational database makes more sense—especially if I have large volumes of data. However, some organizations have to meet both needs. A regional company managing millions of products and millions of customers is going to need relational databases in its back end. And yet those companies have to expose access to those databases from many different views and many different stakeholders in an object-oriented way. .NET doesn't say anything about this, J2EE doesn't say anything about this. It becomes an architectural decision.

Q: Will that change in the future?

Booch: I don't see any startling state changes in that space. Again, the community tried to do that with OODBMSs, with Oracle's efforts, with object relational databases. It's a fundamental state difference between the two—a wicked problem. We're seeing applications with XML databases that are in effect creating XML-centric, DBMS-like things, and I think certain classes of applications will tend to migrate toward that.

Still, there will always be big enterprise applications that must bridge that gap between the relational and the object oriented. A whole class of tools have arisen that deal with small Web sites and modest databases. But the moment I step outside the architectural framework a given tool embodies, then it becomes painful and unnatural. That's why it's difficult for national and global enterprises to do this stuff; you're not going to find your average Web-centric tool to address the huge richness and scope of the things that must be built.

Don't get me wrong here. If there's any common theme, it's that software development has been, is, and will remain not only a critically important business but also a difficult one indeed. Building complex software is not simple.

Q: Some people see UML diagrams as just a different notation for what they already do with high-level languages such as Java or C#. They say that UML adds nice pictures but little value to their jobs.

Booch: People have said the same thing about structured analysis. There are classes of applications for which modeling doesn't make sense. You don't need it if you're building disposable software, if you're building small applications on top of a larger framework, and you can really internalize all the key decisions in your head or inside the heads of a small set of people.

Modeling exists to help you manage, reason about, understand, and construct complex systems. The problem is that the complexity curve keeps growing. As even James Gosling [the creator of Java] has said, "for certain Java applications there are things that I just can't reason about or understand if I stare at the code itself." The same is true for C++, for any language. But certain patterns, structures, and behaviors manifest themselves best if I start drawing pictures about them. The most commonly used tool of any developer is not Emacs, it's a good old whiteboard where people sketch out things saying, "I think this code should do X." In effect they're creating models which are abstractions of their reality that transcend the code itself.

Bad uses of UML are where people over-engineer—or simply use it for documentation. Documentation is not what models are useful for. Rather, they should be living, breathing things that help visualize and reason and construct a system. This is going to sound strange coming from me, but the most important product of any development organization is executable code. Everything else is secondary or tertiary. That's not to say they're not important, but that that they're important to the extent that I need those other things to help build that executable code.

I'm surprised and humbled by how many places UML has popped up in the world. It has a broader reach than I ever imagined it would have, and it's growing beyond my wildest dreams. I do see people using it in really stupid ways, and there will still be classes of applications for which it doesn't make sense. But in the presence of growing complexity, modeling is just sound engineering practice.

Q: Can you attach any sort of metrics to the scale/scope/complexity of a project that requires modeling? Any kind of rules of thumb to pass along? You know how much engineers love numbers—even when they're vague guesstimates.

Booch: I'm going to tell you what's in my gut, but this has no scientific basis. And I'm sure people are going to read it and some manager's going to say, "Oh we're below X number of classes, therefore we don't need to do any modeling." It's going to happen and I'll just have to ignore that. So here it goes. In terms of team size, if you've got one person that person's not going to generate sufficient volume of code to require a lot of modeling. I generate Web sites for a variety of uses—for church, some other folks. And I actually do use UML because I need some aspects, but I throw away most of the diagrams afterwards because the code I'm writing tends to be disposable.

Once you start building an application with the following characteristics, then modeling does begin to make sense: First, it uses more than a few dozen classes or abstractions. At that point I'm starting to have to stick in my head more things than evolution enables me to handle at once. So visualizing those things helps.

Second, it starts involving the need to communicate with stakeholders beyond my small team of three or so—especially if those stakeholders are non-developers such as end users who are business analysts and we need to communicate about workflow. Or I need to communicate with my network folks and say, "Here's the topology upon which we're going to build."

Third, is a matter of the intrinsic complexity of the application. Does it have a lot of use cases? Is concurrency/distribution/security stuff involved? Once you need to start bolting those things together you get this semantic mismatch of technologies as well as the sociology. And modeling helps you throw away the details and focus on the essence of what you're trying to build.

Now there's another growing—and to me surprising—use of UML, which we see especially manifested in the presence of XDE, now inside Visual Studio .NET. In my first visions of modeling, found in the Booch method and later in my work with Jim [Rumbaugh] and Ivar [Jacobson] and others to form UML, I thought modeling is a great way to do forward engineering and maybe a little bit of reverse engineering. Recently, though, I'm also seeing people using XDE and modeling who know nothing about UML. They'll start writing some code and realize, "Oh, I need to navigate this and understand these bits." So they'll query the model and produce diagrams that help them navigate the code base in Visual Studio .NET. Then they'll throw away those models.

Q: How does all this feed into the upcoming UML 2.0?

Booch: The direction of UML 2.0 will be for supporting model-driven development. Imagine that you're looking at a widely distributed system. You know that debugging concurrent systems is just plain wicked. So wouldn't it be interesting if you could look at debugging from the level of UML itself? This becomes possible in UML 2.0. Once you have the modeling tools integrated into the IDE, you've lowered a major barrier for developers to do modeling. And developers, by and large, follow the path of least resistance. That's actually a good thing, by the way. To some degree you even see this with UML 1.x today, because Rational has XDE inside Visual Studio .NET and (can I mention the E-word here?) Eclipse as well. So the two most important IDEs—some might take exception to that but it's still true—have modeling in them.

Q: Some people see Rational tools as an "all-or-nothing" proposition adding tremendously to both purchase and especially training costs. What are they missing? Is it possible that the payoff really occurs after the initial application rolls out, during the inevitable maintenance and modifications that occur through an application's lifecycle?

Booch: We do have a suite, and it's somewhat of an all or nothing proposition, but we sell the point products as well. Every organization has its own culture and pet points of pain that they tend to address. For some, embracing the whole set of tools makes economic and technical sense for them. For others the point products make sense. We believe there's greater value in integration, and embracing this does make the most sense if you look at the life cycle. A continuum of things go on, not a lot of discrete items.

When you approach development like that, one of the greatest hooks has nothing to do with the tools; it has to do with the Rational Unified Process. Organizations struggling with building software systems usually have a process problem, not a tool problem. What is the culture for development? Once you have a reasonable process in place, then and only then can you apply the right kind of tools. Applying a tool to solve a problem you've not yet identified is more likely to annoy people, because you're thrusting that tool upon them.

The healthiest organizations we encounter do development iteratively and incrementally; testing continuously. They have good configuration management policies. A lot of organizations still don't have this. An astounding number of development teams don't use any configuration management tools. This reflects the fact that a lot of software development teams are going about it in an ad-hoc fashion. That's a problem of process. And until they get their process right no tool you throw at them will add value.

That said, when we go into an organization first we have to ask what their point of pain is. It's not an all or nothing proposition. If your pain lies in building releases, maybe your problem is configuration management. Or maybe you have a problem with requirements management. Well, that's another set of things. You look at an organization and say what hurts the most? You give them what they need for that and keep improving upon the process.

Q: Civil engineering employs accepted standards for specifying a building's design in standardized blueprint form, so you can agree on a contract, say "go build it," and be fairly assured that it will be done on time and on budget. Do you see your products or some other methodology providing standardized blueprints for software projects?

Booch: You can look at that in two ways. There are standard blueprints for the things being built and also ones for the process of building. My wife and I spent three years designing and building a house. We used lot of blueprints. And the key lesson there is that developing those blueprints was an iterative and incremental process, with a living document in play through the entire lifecycle of building the house. Furthermore, there were multiple views of it. This goes back to the views thing I was talking about earlier. The other cool thing is that we could go to our builder and say, "I'd like something that looks like this, in that perspective—French provincial or whatever." And he would immediately know what we meant.

Well that's the cool thing about patterns in software engineering. And I would claim that over the last decade the three most interesting things that have happened in software engineering have been the advent of UML, of the Rational Unified Process (RUP) (of course I'm a little biased on those two) and of patterns, in the sense of what the Gang of Four—Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides—have done and described in their book Design Patterns, and other books in that genre.

The cool thing about patterns is that using them enables you to name structures that appear over and over. A structure such as model view controller (MVC) is a common one. It tells me some ways in which I should provide specified degrees of separation between the parts of a Web-centric system. It's an old Smalltalk pattern.

And as it happens there are dozens and dozens of patterns people have named and used already. Those become mini-blueprints. By the way it's no surprise that some architects have influenced the whole pattern community. There's an architectural theorist, Christopher Alexander, who wrote some delightful books: The Timeless Way of Building and A Pattern Language. He was influential in the thinking of the patterns community and in software engineering. So we can now name these kinds of things. In the Java space, for example, Sun has created a number of patterns on top of J2EE, because you've got J2EE or .NET—I'm moving my hand at waist level here—and if I wave my hand over my head that's the application level; you've got in effect a semantic gap between the application and the platform. Patterns can help you fill that gap. So from the technology side yes we can name some of those patterns.

Where the interesting work is going on, especially in the enterprise, is with some of the big financials—insurance companies and the like. They have made a major effort to name, codify, and define the architectural patterns that shape their entire enterprise. And there is tremendous benefit from doing so. You produce an architectural framework from which you can build many instances of an interesting system.

Now from a project perspective of architecting, that's where tools like the Rational Unified Process come into being. You're building subsystems. Today we know the best way to do them: incrementally and iteratively by growing the architecture. That's sort of the essence of it.

If you go to civil engineering practice, people spend years learning the process of building buildings, and they get lots of on the job training. Well, we as a software industry haven't quite gotten to the point where we can codify all those things; where I can say, "Here is a picture of an architecture. Fill in the little details," and boom you have an application.

For certain limited classes of applications you can do that now. And I think development is headed towards get us even closer to that, helped by thinkers such as Charles Simonyi and Gregor Kiczales. Kisczales was at Xerox Parc; Charles of course was at Microsoft in the area called aspect-oriented programming (AOP). Charles left Microsoft recently with Bill's blessing to form a company called Intentional Software, which is working on solving the problem of having patterns defined by different stakeholders conflicting with each other. Technically, it's a challenging problem and an important one. For any interesting system you have multiple stakeholders, each with their own views of the world, their own patterns. Today we have to weave those patterns together fairly mechanically. The promise of AOP and model-driven development is to replace a lot of that manual effort with automation.

Q: Do you believe we will we ever be able to turn software development into something as predictable and boring as building a highway?

Booch: For the most part we know the engineering forces acting upon highways. We know the shape and size of them, the variations in them, and lots of local concerns affecting them. Unfortunately, with most software systems, we aren't quite there yet. We haven't been able to codify domains and cultures enough to be able to say, "I know all the forces around this, such that I can define all the details and automate it." And so many changes push upon the software development team that it's hard to name those patterns. The average development team has to deal with cost, schedule, functionality, compatibility, performance, scalability, plus reliability, failsafe and fault tolerance issues. And every domain and culture is a little different.

On top of that you've got technology churn. Those folks in Redmond keep pushing out new releases of things, as do the folks in upstate New York and down in Cupertino. So, technology does not stand still. And your customers are beating up on you as well. You've got to build resilient systems, which is a tougher engineering problem than what you find in long-lived industries such as highway building. We're not changing the materials out from under you. We're not giving you the means of defining your own atoms and molecules. And yet we do that in software.

Q: Earlier you said development should be iterative and incremental. How does that differ from fixing bad code until it works?

Booch: There's a role for failure in design. One of my favorite authors is a guy by the name of Petrosky. He's written a delightful series of books, my favorite of which is To Engineer is Human, where he talks about how we learn from our engineering failures. In software it's not quite as direct, however. People fail and you get into this "code and fix" cycle, which looks seductively correct. But from an economic return on investment viewpoint it becomes draining. You get into death marches. And a code and fix development method simply is not a sustainable process.

But you tend to be led into it in the face of all these forces that weigh upon you. So software developers will often approach a problem thinking, "I will program my way out of it." However, many problems turn out not to be programming problems. That's where the issues of process and architecture start raising their heads. When you're building enterprise applications that matter—by which I mean they have a deep economic value to the organization that creates them—those problems have something to do with technology but a lot to do with architecture.

Q: Why is Rational advocating OO and inheritance, instead of COM-like component model?

Booch: We don't. Actually inheritance is relatively uninteresting. Lots of other issues dominate. We tend to focus upon good architectural practice. Working at this level of abstraction, OO falls out of that. Inheritance sometimes falls out of that as well. And if you look at the way people build systems today it's not just all COM out there. There's a lot more to gluing systems together than building clients on top of COM.

Q: Why are there few or no tools to help with security, concurrency and synchronization issues and the like?

Booch: We as an industry just don't know the right ways to codify what we're doing in the security space. At Rational we have a chief architect and we also have a chief security officer. The problem of building secure systems is so hard that we have a person to ascertain those policies, which also get pushed down to individual developers. Building secure systems is just a plain wicked topic and insofar as we don't know the patterns with which to follow them that's why it's difficult to have tools.

Q: What is the future of Web services and GXA in conjunction with traditional development?

Booch: I'm pretty jazzed with the direction of Web services. We look at organizations that are already effective at building component-based systems. Meaning they know how to componentize their stuff and deploy them that way. For them the move to Web services is actually a small step. Plus it's not a difficult step, because it represents a different packaging for the things they're already exposing. But there's value in it because now it allows them to transcend a particular platform.

I'm working with a group I think I can mention publicly: MedBiquitous, a project at Johns Hopkins led by Dr. Peter Green. The essence of the project is to try to tie together the various medical research agencies and data across a variety of different specializations in medicine, so that you break down the barriers of specialization. That's a gross simplification of what they're doing but it's pretty cool stuff. The essence of their architecture is based upon Web services, which enables them to make key architectural decisions that transcend particular platforms and technologies.