Simple Architectures for Complex Enterprises: A Case Study in Complexity

Let’s take a look at a real-life case study of a complex system. There are three important lessons to be learned here. The first is how complexity creeps into a project, even one with the benefit of extensive planning. The second is how this unchecked complexity leads to project failure, even one with seemingly unlimited resources. The third is how Simple Iterative Partitions (SIP) might have saved this project, even when it was well into failure mode.

The case study I’ll discuss is one of the largest and most complex systems yet tackled by any government organization. It is the National Program for Information Technology (NPfIT), a program run by the British Government’s National Health Service (NHS). Sometimes NPfIT is referred to simply as the National Program, or, as they say in Britain, the National Programme. Remember these acronyms—NPfIT and NHS—you will be seeing a lot of them in this chapter.

Overview of NPfIT

NPfIT was launched in June 2002. The basic goal of NPfIT was, and continues to be, to automate and centralize the massive recordkeeping that is the backbone of its national health care system run by the NHS. Health care in Britain is mostly nationalized, unlike the United States where health care is mostly ad hoc. This centralized system creates a unique opportunity to standardize the recordkeeping of a very large number of patients and health care providers. NPfIT is promising the following capabilities when completed:

• Automation of all patient care information.

• Access to any patient record by any authorized health care professional in the UK

• Ability for primary health care staff to book appointments for patients with any other health care worker in any health care facility in the UK

• Automation of prescription services

The NHS describes NPfIT systems as follows:

• A key aim of the National Program [NPfIT] is to give healthcare professionals access to patient information safely, securely and easily, whenever and wherever it is needed. The National Program is an essential element in delivering The NHS Plan. It is creating a multi-billion pound infrastructure which will improve patient care by enabling clinicians and other NHS staff to increase their efficiency and effectiveness.³⁴

In a nutshell, the NPfIT promises an integrated system connecting every patient, physician, laboratory, pharmacy, and health care facility in the UK. NPfIT functionality can be loosely divided into three main categories: regional clinical information systems (CIS), infrastructure systems, and shared applications. The NPfIT architecture is shown in Figure 6-1.

Figure 6-1. NPfIT architecture.

Regional clinical information systems connect health care providers (for example, hospitals, clinics, and physician offices) within a geographic area and provide their main point of contact for NPfIT. These are shown as the “hairy spheres” hanging off the central sphere shown in Figure 6-1.

By my estimates, the regional clinical information systems account for approximately 79 percent of the total initial budget for NPfIT, approximately $9.8 billion. Keep in mind that NPfIT expenses are given in British pounds, and I have converted these numbers to U.S. dollars. These costing numbers are based on conflicting source data and a fluctuating exchange rate, so take these estimates as educated guesses.

Infrastructure systems will provide connectivity, security, and directory services to the NPfIT. These infrastructure systems include the New National Network (N3), which provides the network facilities, and the spine, which includes shared software facilities such as directory services. Care Records Service (CRS), the shared patient records, is sometimes shown as part of the spine and sometimes as separate shared applications.

After the regional clinical information systems, the infrastructure is the second largest part of the NPfIT budget, accounting for approximately 18 percent of the initial NPfIT budget, or $2.3 billion.

Shared coordinated activity across the entire NPfIT system appears to make up a relatively small part of the overall NPfIT budget, less than 5 percent, or about $300 million. The most important of these shared applications include the following:

• Choose and Book. A system that allows an appointment to be booked for any patient at any facility in the system

• Electronic Transfer of Prescriptions. A system that allows prescriptions to be entered for any patient in the system and filled by any pharmacy in the system

• Picture Archiving and Communications Service. A system that allows the central storage and retrieval of picture data, especially x-rays

The amount of data that must be coordinated is immense. According to the NHS³⁵, in a typical week the NHS processes

• Six million patient visits to general practitioners

• Over 64,500 emergency calls by NHS ambulances

• 360,000 patient x-rays

• 13.7 million NHS prescriptions

The NHS estimates that NPfIt will need to coordinate about 3 million critical processes and 30 million transactions per day.

The NPfIT geography is split into five clusters, or regional groups of patients and health care providers. The NPfIT budget is almost $2 billion per cluster. The clusters are arranged as follows:

• North East (which includes Tees Valley, Northumberland, South Yorkshire, West Yorkshire)

• North West and West Midland (which includes Greater Manchester, Cheshire)

• Eastern (which includes Essex, Trent)

• London

• Southern (which includes Avon, Dorset, Thames Valley)

The initial budget was allocated in 2004 among many different vendors. The highly lucrative regional cluster contracts each had a primary vendor, a CIS vendor, and other miscellaneous secondary vendors. The primary and CIS vendors awarded to each region is shown in Figure 6-2.

Figure 6-2. Primary and CIS vendors by regional cluster.

So NPfIT is a multibillion-dollar project split between at least a dozen vendors spread over a geographic territory of close to 100,000 square miles; it offers services to 60 million people and is expected to process over 300 transactions per second. I would call this project highly complex.

Now that you have a basic overview of NPfIT, let’s see how well this project did using traditional architectural approaches. Perhaps you will recognize ghosts of your own projects in this description.