I believe that early, effective big picture diagrams are key to application development project success. According to the old saw, no project succeeds without a catchy acronym. Maybe so, but I'd say no project succeeds without a good big picture diagram. The question: what constitutes a good one? To me good high-level diagrams have four key characteristics: they are simple, precise, expressive, and correct.
The late Eberhardt Rechtin, writing with collaborator Mark Maier, tells us that there's a different skillset needed for high level system definition. As indicated by the titles of his books, he sees the architecture role in complex systems development as equal parts technology and art, especially in early definition. The architect builds the big picture based on deep knowledge of the field, understanding of the problem domain, and experience-based heuristics, or guidelines learned in the school of hard knocks. Finally, the authors admonish the architect to "discard the nonessentials. Model (abstract) the system at as high a level as possible, then progressively reduce the level of abstraction. In short, Simplify!" (their emphasis).
The reason such diagrams are key to a successful project is that they are meaningful to all participants, and unify all of the different stakeholders in their efforts. The larger and more complex the system, the more critical that the efforts of the large and diverse team working on the project share a common understanding of the goal through a well executed high level diagram.
So the quality of a high-level diagram depends on the architect's depth of technique, perception of an idea to communicate, and lessons learned by experience in putting the two together. To Rechtin's point, those same prerequisites also apply to a sketch, a haiku, or a work of abstract art.
Here are four examples of big picture concepts, two from IT and two from art, that illustrate four key qualities of effective high-level diagrams:
Simplicity: Thelonious Monk's Evidence and Bright Mississippi
Jazz man Thelonious Monk is considered one of the most original music innovators of the 20th century. In addition to his quirky and surprisingly original works, he often played his own takes on pop songs of his day. There are two cases where he took pop standards and stripped them to their essence, turning them into something quite different that reveals and enhances previously unnoticed elements of the original. In one case, Just You Just Me becomes Evidence, revealing the angular rhythms and open space Monk heard in the bright little ditty. In another, he took the old standard Sweet Georgia Brown (popularly known as the Harlem Globetrotters theme) and turned it into proto-funk masterpiece Bright Mississippi. By driving to simplicity, Monk revealed fascinating rhythmic and harmonic characteristics. Similarly, the architect drives to simplicity to reveal linchpin elements of a complex design.
Precision: The ISO/OSI Model
The International Standards Organization Open Systems Interconnection model is the standard conceptual view of communications between networked components. Originated by Hubert Zimmerman around 1980 to address the need for "standards which would allow constitution of heterogeneous computer networks", the model enables developers to conceptualize communications between two network nodes at any of its seven layers of abstraction. Mr Zimmerman recognized a core problem in network component development was that every developer had to worry about all potential technical issues when designing a point to point solution. The OSI model enables developers to hold constant the characteristics at other levels and focus on a single layer at a time, hugely increasing development efficiency.
Zimmerman's simple model precisely zeros in on a core key problem for massive benefit. His diagram precisely illustrates the concerns that a developer does and does not manage when developing a communications component.
Expressiveness: War and Peace, 1952
The Picasso line drawing shown above illustrates how a few simple lines can convey meaning. One quick look reveals the woman's security and contentment. At a second look one is astounded by the diagram's simplicity relative to the emotion the few lines communicate.
Of course architects creating a high level diagram won't be Picassos, and they won't convey emotion, but like Picasso their diagrams will be abstracted to convey as much information as possible in just a few strokes of the pen.
Correctness: Bill Inmon's Data Warehouse Model
As reported in this previous post, in the early 1980s, Bill Inmon, then a database administrator, discovered that reporting and operational systems have very different usage patterns, and benefits of duplicating operational data in a separate reporting database far outweigh the redundancy cost.
Mr. Inmon developed this revolutionary insight into a four-sector diagram that divides a reporting system into four distinct sectors: operational data sources; an "atomic" database, later called the data warehouse, a "departmental" datamart sector, and a presentation layer that includes reporting and analysis. (The staging area shown in the diagram above was a later addition).
Like the ISO/OSI model, Inmon's diagram brings to the forefront the distinctions between components in each sector grounded in their vastly different usage patterns and processing requirements. Because this high level diagram is grounded in technical specifics, architects can guide development teams by defining technically specific interface and design ground rules for components in each sector.
It may be daunting for most of us to compare efforts with those of great artists and technical innovators like Monk, Zimmerman, Picasso, and Inmon, but if architects deeply understand their core design at the outset, they can build a vision for the new system with the simplicity, precision, expressiveness, and correctness to help unify all participants' efforts as the project begins.