The essence of architecture

Architecture in the world of information management is becoming an accepted phenomenon. However, from time to time discussions still pop up about what architecture really is. Most explanations are complex, but in fact I think it is quit simple.

As far as I am concerned, the essence of architecture is the grouping of elements (or parts) into structures (or clusters) based on predefined criteria or principles.  For example: responsibilities can be grouped to organizational units, functional parts to processes, parts of information to knowledge bases or databases and parts of automated functionality to applications. Build on this definition the following principles can be derived:

1. In an architectural study, elements should be determined in order to group these into clusters.
2. An organization can be analyzed form different viewpoints, e.g. from a business point of view, from an informational point of view or from an IT point of view. For each viewpoint, the elements that are relevant and needs to be taken into account differ by nature. The way of grouping also differ since each viewpoint has it’s own problem definition. So, different points of view result in different architectural solutions. Therefore, it should be clear which point of view (or aspect area) is contemplated when an architecture is designed.  This results in different kind of architectures like business-, information-, product-, process- and Information systems architectures.
3. Each architecture should be integrated and therefore consistent with each other.
4. Designing an architecture (grouping elements into structures) should always be based on grouping criteria (or principles). Without these criteria, each grouping is valid. The criteria determine the architecture and should be derived from the strategy of the organization.
5. The architectural design based on the criteria is an ideal solution. In practice, however, there are various constraints that force the “feasible” solution to be different. The ideal solution and the feasible solution should be distinguished.

These simple principles always help me to explain what architecture is all about and I used these principles in assessments of architectural frameworks.

Business IT Development

We currently begin to accept that there is one stable basis in the field of business and IT: That is the fact that things are changing and that things will keep on changing. We live in a world that is changing continuously and we have to cope with the dynamics of our environment.

The environment of firms is changing, consumers demand for more variety of products and services, more customization, and for a good price and proven quality. Products, services and even the production have to be compliant to changing legal regulations and social responsibilities. The company should be adaptive: processes and organizational structures should be adaptable and management should have the capacity to change these processes and structures. The IT should not be the limiting factor for these changes, but should support these changes instead.

The IT itself is also changing. Not only the hardcore IT is changing (growing speed and capacity) but also architectural aspects (communication, webservices) and maybe most important the usability. The use of internet and social media tools has huge impact on society.

So, the challenge is to support the developing business with IT that is also developing continuously. This is what I refer to as Business IT Development. The IT should support the changes in business processes and organizational changes without the need for a re-design. New technology trends should be adopted by the business easily.

The story above does make sense, but the challenge is how to define, design and implement a flexible IT environment that supports an adaptive business.

Oelan, the company that I will join on September 1, 2010 will meet this challenge. The mission of Oelan is to support organizations with the transformation to an adaptive organization with flexible IT. A dedicated business unit called “Business IT Development” is started which will provide services to design adaptive organizations and flexible IT. These services include Business and Information Architecture, Business Process Management and IT Architecture. The basic design principle is flexibility. The starting point is to describe the essence of the business in a modular way, resulting in an overview of the essential tasks of the business instead of the current processes and operations.  Based on this model of the business, the flexible IT environment will be designed according the service-oriented principles.

In the coming months we will define a service offering as described above which will help our customers in the ever changing world.

IT services do not deliver business agility

Different major IT company’s promise business agility through service orientation. Reading all the article and blogs about this topic it seems to be a tacit assumption that service orientation delivers business agility. First of all, service orientation can be applied in several ambition levels. But even if the highest level of ambition regarding to serviced orientation is applied, I argue that this still doesn’t guarantee business agility. The following story of increasing SOA ambition level will illustrate this.

• The first step towards a service oriented architecture is to expose current functionality as a set of web-services. All functionality which is embedded in ERP, CRM and all other kind of databases and systems will be available,  with the help of adapters if needed,  as web-services. The services can even be exposed on the internet through a gateway. But, this will not deliver business agility.  
• The services probably have different protocols, semantics, formats, security and management capabilities. Therefore using these services is unambiguous.  So, we publish the services on an enterprise service bus and list the services in a registry. This will still not deliver business agility.
• The services doesn’t support the processes of the consuming applications. We compose the  elementary services in such a way that the new set of services is relativily small and recognizable by the business and directly applicable for the consuming applications. We are getting close, but this will still not deliver business agility.
• The next step is to be capable to change the services quickly in order to support new or adapted processes. Although a BPM-tool might be in place to reconfigure the composed services, the most important thing is that the elementary services have the right granularity. It should be possible to compose the elementary services to any composed services needed by the consuming application. Now, we are talking, but still we talk about IT agility. This is still not business agility
• Real business agility is achieved if the organization has the capacity to adapt in respect to environmental changes . Many things change in the environment of an organization. One of the most important one is the (real) consumers need for new or adapted products or services (these services are different than the ones we discussed so far). In most cases business processes and even organizational structures have to change in order to produce and deliver these new products and services efficiently. The IT should be flexible, but business agility surely requires an appropriate culture, management capabilities, flexible resources (of all kind) and a flexible organizational structure.

A last note to think about: Another major environmental change is the changing Technology (IT in particular). Why do we try to design and build flexible IT systems to be future proof if we already know that we will use new technology in the future anyway?

The IS field and the narrow view of the CIO

Traditionally, the view of the CIO is focused on Information Technology. Main concerns are related to the IT systems that should be implemented. Which system to buy, what kind of system to build, how to do that efficiently and how to govern and maintain systems at low costs are keeping the CIO busy. But if we consider the total field of Information Systems (IS) or Information Management (IM), we notice that there is more to manage than just the IT systems. For instance: Information. What type of information is relevant for the organization. How do people gather, perceive, manage, and use information? It’s nowadays interesting to see where the information is coming from since more and more information is coming from outside the organization by using social media networks .  Another interesting point is to see how employees use different open source tools and configure them in such a way that they can do their job (at least partly). A remarkable phenomenon is that the tools employees use at home commonly better suit their needs than the tools they use at their work do.

Issues about the information and about how employees and organizations behave seems to be more important than the IT itself. Or to put it more radical as Carr in 2003 stated: IT doesn’t matter. He argued that “As information Technology’s power and ubiquity have grown, it’s strategic importance has diminished. The way you approach IT investment and management will need to change dramatically”

In the architectural scene, they are already convinced. Most architectural frameworks distinguish different aspects like “business”, “information” and “technology”. However, in many practical architectural studies, “business” and “information” architectures are still mingled with technology and organizations struggle in making a good distinction (Separate business and IT to get Business-IT alignment).

This not only occurs in the commercial world but also the academic field noticed the poorly-defined management discipline. Meas (1999) considered the IS field through a generic framework. He proposed to use 9 areas and their interrelationships, where each area is the combination of “business”, “information/communication” and “technology” on one hand and “strategy”, “structure” and “operations” on the other hand. The focus of the CIO is focused on the technology column, mainly on the operational area.

Lyytinen (2010) discussed the subject matter of the IS-field by plotting them in three area’s: “organized human enterprise”, “information” and “technology”.  The focus of the CIO is traditionally on “technology” and on the “technology”- “organized human enterprise” relation.  Besides the IT infrastructure itself, the main concerns are how IT can improve productivity, support process execution, management and coordination.

I argue that in the coming years, the focus of the CIO should shift to other area’s than technology: Information and organized human enterprise. The following subjects will matter: 

Information management

More and more Information is available from outside the organization. The issue is how to manage to use the information. People, even (or may be especially) within an organization, perceive information differently. It should be clear what kind of information is needed. Getting the right information on the right time on the right place is the challenge.

The impact of technology to humans and to the organization.  

How should an organization deal with new tools like the social media tools, collaboration tools. Not only the current processes are supported in a different way, but the processes will also differ due to these tools.

The picture helps to get position what initiatives and activities are currently done in an organization and to see what other kind of initiatives should be started. The focus of the CIO will move from Technology to Information and organized human enterprise.

At the end of the day, it is not about IT, but it is about what we do with it.

The business story of SOA

Defining the right level of granularity is one of the most challenging tasks to do in the design of a Service Oriented IT Architecture. There are no unambiguous design rules and most architects seem to define the services based on experience and a set of rules of thumb. It is important that the design of services support the desired goals, which in most cases is business agility (according the literature and my own experience). Services with a high granularity (i.e.: coarse grained) obviously don’t lead up to flexibility as we have seen in the last decades by implementing monolithic applications. Services with a low granularity (i.e.: fine grained) results in a very complex systems with many relations between (re-usable) services, which is hardly manageable and adaptable and therefore not flexible either. So the optimal granularity, which provide business agility,  is somewhere in the middle. In order to define an effective Service Oriented IT Architecture one should focus on business agility.

Business agility is achieved when a business is able to adapt its organization (including IT) to response to environmental changes. A tacit assumption is that (among others) modularity leads to agility for two main reasons: (1) the ability to modify a local unique contingency without affecting the whole system and (2) it enables its components to be recombined in different ways, often to serve different functions. Service Oriented IT Architecture aims to achieve agility and therefore the architecture should reflect the modular business structure. So, in order to find the right granularity of services, I think it is good for IT people to have the concept of modularity related to business agility in mind. But, what is this business agility all about? An overview of the most significant changes in the history of business changes may help.

Until the mid 20th century, enterprises were commonly organized based on the “Mass Production” concept. Standardization of products and services resulted in economies of scale. A necessary condition, however, is that efficiency of the delivery process must be maintained at all times. This requires most of all stability in the business environment. There is no flexibility for customer specific delivery. Process efficiency was further achieved by specialization of labour, standardization of internal processes and even standardization of the delivery of products and services such as customer communication.   

The major reversal in the business environment regarding to change was noticed in 1954 by Drucker: “It is the customer who determines what a business is”. The consequence was that businesses had to respond to the customers’ variety of demands while still being efficient. This ‘’reverse thinking’’ resulted in a new paradigm as defined by Davis (1987) as “Mass customization” which attained widespread popularity with Pine’s book in 1993.  One of the most clear definitions of mass-customization is given by Hart (1996): “the use of flexible processes and organizational structures to produce varied and often individually customized products and services at the price of standardized, mass-produced alternatives”.

Some traditional examples of company’s who were able to deliver customized products due to a modular production process are Nike and Dell. Both firms implemented the essence of the modular concept by designing, developing, and producing those parts which can be combined in the maximum number of ways. The structure of the modular products were mirrored to the structure of the organization.

As an evolution, Mass Customization” is followed up by “mashup”, “virtual” or “boundaryless” organizations. These organizations have a ultimate modular structure that supports highly temporal structures and the creation of processes on the fly. The need for Service Oriented IT Architecture that supports and even enables these organizations is increasing.

The granularity of services in a Service Oriented IT Architecture should be based on the modular business architecture. Company’s implementing Services Oriented IT Architecture should have a decent business architecture in place as well as a clear strategy to transform the modular business structure to a Service Oriented Architecture. Distinguish business architecture and IT architecture is a prerequisite.

2010: Bazaar or the Cathedral?

This period of year is the time to look back and especially look ahead. Therefore I challenge you to think about 2010: Will it be the year of the cathedral or the bazaar?

The title is borrowed from Eric Raymond’s¹ famous essay about the contrasting concepts of a cathedral – long in detailed planning, fully conceived of to a detailed extent, and fully constructed prior to opening; with the bazaar – a constant babble of contending interests and agendas, open to all and sundry.  

The concept of the bazaar is the fundament for the Open Source community. Trends like ‘mass collaboration’ and ‘commons-based peer production’ even changed the World Wide Web from being primarily about the storage and retrieval of information into a fully-fledged basis for communication, social interaction and participation.

The new features of the digital age or network society really do amount to a basis for a fundamental rethinking about social interaction, communities, and collaboration. considering these features from a business point of view results in some interesting ideas. Let’s consider some.

After the great economic crisis of last year, many people lost their faith in the financial institutions. After the formal nationalization of all those banks which have been rescued as part of the general bail-out, there is a public demand to organize this in a completely different way. Tony Briant² referred to the mutuals, which are ‘friendly’, ‘co-operative’, ‘benevolent’ or ‘mutual’ societies, particularly the investment and finance organizations. The idea of mutuals, in the form of building societies, dates from the late 18th century when groups of people got together to pool assets so that between them they could eventually buy the land, materials and other resources needed to build a house for each of them. One of their key roles was to act as mortgage providers, balancing the funds they held as savings against the funds they provided to lenders financing their house purchases.

Combination of the technology and ideas of mutuals has been taken up by the financiers which results in the new form of internet-based mutuality. For the financiers it has proved to be the platform upon which a truly global, supra-national financial system can be established – able to evade any local or national control or other forms of regulation, simultaneously offering high volumes of near-instantaneous transactions at negligible cost. Imagine how this will change the financial system!

The basis of this new form of collaboration is its anarchic character, allowing a babble of different agendas, free from central control – particularly government and other quasi-governmental bodies. It represents the alternative basis for cooperation and solidarity. Think about the health care system and even the government itself!

People might think that this has nothing to do with them for the coming years, except in case of some nice initiatives e.g.: c,mm,n.org, Nabuur.org). But think about your own organization. I guess this is like a cathedral, an hierarchical institution managed from the top. But is it possible to achieve the same results in a bazaar-like context?

I wish you all a great 2010 with a lot of wisdom. And if you have some moments of reflection during the holidays, think about what your role will be in the bazaar next year.

[1] http://catb.org/~esr/writings/cathedral-bazaar/

[2] http://www.opendemocracy.net/article/email/mutuality-2-0-open-source-the-financial-crisis

What did SOA actually bring us?

The main premise about SOA is that it provides flexibility, business-it alignment and reusability. In a truly service oriented architecture, I honestly believe that this will be the case. However, what I notices in some practical cases is completely the opposite. This highly depends on the context of the IT landscape in which SOA is implemented. If we ignore the use of services just to integrate applications, we can distinguish two different scenarios of implementing service oriented architectures. The first one is the scenario where the application landscape consists of services and human interaction applications (scenario A). This is a pretty ideal situation in the land of SOA. The second scenario is one that occurs more often. The application landscape is a mix of services and legacy systems (scenario B). The services are commonly used as an abstraction layer above the legacy systems.

In both scenarios it is common practice to apply structured layers where at least a distinction is made between the presentation (exposure of the functionality to the users), the functional layer (services based on the functional/business point of view) and the technical layer (not directly related to business functionality and technically supporting services). The lowest layer yields the world of legacy systems, connectors, data mapping and transformation services etc.

Generic SOA business cases are commonly based on scenario A while scenario B seems be more common practice. Let’s consider the three benefits based on the two implementation scenarios.

Flexibility:

Two types of flexibility is obtained by the principles of modularity: (1) configurability: enabling its components to be recombined in different ways, often to serve different functions and (2) adaptability: enabling to modify a local unique contingency without affecting the whole system. Configurability and adaptability are characteristics of scenario 1. For scenario 2, this is different. Configure the flow of services may be possible but probably the legacy systems will limit this ability. Adapting the functionality will in most cases also require a modification of the legacy systems. So there is not much to win here either.

 

Business-IT alignment:

Assuming that the services of the functional layer are designed from a business point of view and therefore represent and mirror the elementary business tasks, the services architecture will mirror the business and therefore there is business-IT alignment on an operational level in both scenarios. In case flexibility is part of the business strategy, scenario 2 is not aligned on a strategic level, since scenario 2 is not flexible as argued above.   

Reusability:

Services on the functional level are likely to be reusable in a different context (channel or business process). In scenario 2 however, this depends on the context-independency of the legacy systems.

On the technical level, the reusability is low apparently. The costs of designing and running relatively small reusable services is high compared to the use of duplicated services.

Summary:

	Scenario 1	Scenario 2
Flexibility – configurability Flexibility – adaptability	+ +	- -
BIT alignment operational BIT alignment strategic	+ +	+ -
Reusability functional Reusability technical	+ -	+ -

To wrap this up, I argue that the benefits of SOA are highly depended on the circumstances of the IT landscape in which the services are implemented.

A well forgotten one: The paradigm of control

One of the comments on the previous post about well forgotten ideas posted by Michael Poulin immediately reminded me to the paradigm of control of de Leeuw (1982). This is one of my favorite paradigms because I found it very useful in practical situations. From the moment a colleague of mine explained this paradigm, I used it in many situations and it helped me defining solutions and analyzing different kind of problems. Therefore, I want to share this with those who don’t know the paradigm or just forgot  it. 

The paradigm of control of de Leeuw is based on the theory of systems. It first describes the main concepts of the system theory of control: the so-called controlling organ (CO) and target system (TS). The TS is the system that has to be controlled in one way or another. This can be almost  anything but in the organizational world it commonly is a group of workers performing activities, e.g.: an organization, a program, a project or a project team. These systems are controlled by a controlling organ e.g.: director, manager, steering committee or team lead. The controlling organ can physically also be part of the target system, but we distinguish the concepts. Within this framework, de Leeuw describes 5 topics that are required in order to manage a target system effectively.

First off all, the TS should have a well defined goal (1). In IT projects I worked in, this was not always clear. In some cases it was thought by project management that the goal was to deliver a system as specified. At the end of the project, the customer wasn’t happy with the result because the IT system didn’t support the users in fulfilling their tasks in the new business processes. Specifications may not be sufficient, but the goal of the project was to deliver a system that supported the users in their work.    

Secondly, in order to control the TS, information (2) about the target system should be available for the CO. All relevant information should be communicated to the CO. Defining the relevant information can be quite complex and the processes of the TS should be generate this information. Information about the progress of an IT project is in most projects hard to get for project managers. It is for instance not easy to determine whether a project is ready for 50, 60 or 70%. The classical quote of an implementer is:”I’m almost ready with this coding”. It is not rare that at the end of the day it took far more time than expected. Iterative development is a real benefit for this. Functionality that is implemented and tested gives a good indication of the progress.   

Thirdly, the CO should be capable to process (3) the information. High loads of information is typically processed by using management information systems. For IT projects, the load of information is not that high. However, this point also include the right interpretation of the received data. Project management should be capable to discern the relevant information from the irrelevant information. In practice, this means that good managers should also recognize relevant non-verbal communication and act upon it.

Fourthly, a model of the TS (4) should be available. The effect of modifications on the TS should be predictable. Notice that the model can also be conceptual, but this will only be possible in case of small and relative simple controlling organs and target systems. A Development Case used in RUP-projects is a kind of a model of the TS.  

At last, but not least, control measures (5) are required. One can imagine that changes will not be implemented properly if management doesn’t have the trust of the people in the TS.

In the past years I have done some formal and informal audits on IT projects. The five requirements for effective coordination was always the starting point for these audits. In most cases, I was able to point out the problem based on the 5 requirements. And in all cases, the problem could be appointed to one of the 5 requirements at the end of the day.

SOA, a business or IT issue?

In several SOA projects in which I was engaged, it was quit hard to convince managers of the business architects and business analysts group that SOA had impact on their work (in this context SOA was understood as an architectural design of software). Of course, they understood the statement that SOA would deliver a flexible IT environment and they were happy about that. But their final statement was: “SOA is an IT issue, it has nothing to do with business modeling or business architecture”.

The question is: “are these managers right?”. Is SOA just a IT concern, is it an IT solution in order to get IT better aligned with the business or is it also a business concern. Does it have impact on the way the business is modeled?
I explained the situation by considering business-IT alignment. According to the famous article of Venkatraman, business-IT alignment of two different types has to be considered. The first one is strategic integration of business and IT while the second one is operational integration of business and IT. Strategic integration is the link between business strategy and IT strategy, it deals with the IT capability of IT functionality to both shape and support business strategy. Operational integration means that the infrastructure and processes of business and IT are aligned: the organizational requirements and expectations should be coherent with the delivery capability of the IT functions. This means that tasks executed in business processes should be supported by appropriate IT services, which is the common way to define IT services by the way.

From an operational integration point of view, this means that the required functionality by the business should be delivered by the IT function. Users of applications should be supported in executing their tasks. It doesn’t matter how these applications are designed and implemented (e.g. service oriented, component based, object oriented). From this point of view the manager is right. It doesn’t matter how the IT function support the business, as long as it does support it properly. The IT function may be wise to design and build according a Service Oriented paradigm, but then it is still an IT issue.

However, we have a different story if we consider this from a strategic point of view. A strategic business requirement may be flexibility. A tacit assumption is that modularity leads to organizational flexibility. Therefore, the business should be defined in terms of business modules. These business modules are commonly referred to as business services. Defining business services is a business issue and depends on the desired flexibility. The IT structure should be aligned to this business structure. The IT structure should have at least the same level of modularity as the business structure. In other words, the software services may be more fine grained but may not be more coarse grained than the business services. If flexibility is required as it is in this case, it is obvious that the business services have to be defined before the proper software services can be defined.

The conclusion is that SOA is supports business-IT alignment (operational), but on the other hand SOA requires business-IT alignment (strategic). In order to define the right software services, the business services should be defined.  In practice, businesses are commonly not structured in terms of services because there seems to be no need from a business point of view. However, there is a need to model business services to get truly (strategically) business-IT alignment: Time for business analysts and architects to come up with the right business services.

Separate business and IT to get Business-IT alignment

Almost sounds like a paradox.  In the last years, I encountered similar issues while working in business and IT transformation programs in large companies. The underlying principle in these projects was to increase business-IT alignment. In all projects people struggle with this issue and it was fuzzy what the tasks of business architects and analysts (BAA) were and what the task of the IT architects and analysts (ITAA) were. In theory it was quite simple: BAA were responsible for everything that has to do with the business and ITAA were responsible for everything that has to do with IT. In practice, however, BAA concern with IT things while ITAA concern about business things. At the end of the day business is IT for a significant part and IT is business, right?

The separation between business and IT was not clear, not on a conceptual level and also not on an organizational level. How to align things that are not clear? How can it be that something apparently simple can cause that much issues?

Use Cases seem to be a great instrument to clearly define the boundary between business and IT. The business describes what functionality it expects from IT and the requirements for the IT systems are determined at the same time. The use case fulfils the contract between the business user and the IT provider and it is obvious that a contract is agreed upon by both parties. Of course, this only works straight if the functionality is not mixed up with technology related requirements. And this is exactly what is happening in practice. The required functionality is often described in terms of IT components, other IT systems, software services or even databases.

With the introduction of SOA it even became worse. Service types like business services, information services, software services and all kind of mixed types like business aligned software services are introduced. The definition of these types are described but what these things actual are is perceived very differently by the employees of an organization.

In order to achieve business-IT alignment, first the distinction between business and IT on a conceptual level must be clarified. The next step is to clarify this on an organizational level. It must be clear who is responsible for which artifact (type of service). 

Traditional “system theory thinking” helps to distinguish between business and IT.  A business can be considered as a system. A part of the business is a subsystem. If a hotel is the system, than the “checkin” part is a subsystem. This is the part of the business where guests provide their ID and preferences and the clerk assigns a room based on the availability. All information is registered and the key is handed over to the guest. This subsystem yields a lot of things, like obtaining the appropriate information, determine the appropriate room, registering information and providing the key to the guest. All these things can be done by the clerk or being automated. In a little French chambre d’Hotel, the clerk which usually is the owner will do everything personal. In a big motel company, everything will be automated, including reading the passport and dropping of the key. So, let’s consider this subsystem as a service. Should we now call this a business service or an IT service? We can be both. The subsystem “Checkin” contains both business and IT. From a business point of view, one can describe the subsystem without taking IT things into account. This is what is referred to in the classical system theory thinking as an aspect system. Aspect systems are always a specific view on a real thing. The business service is a view of the real “checkin” service. One could also describe the IT aspects of the service (IT service) or just the software aspect of the service (software service).

So all the different types of services (e.g.: business, software service) are a view on the same real thing (the “checkin” service). Business and IT views describe different aspects of the same thing.

In the context of service orientation, Business-IT alignment is achieved if the business services and IT services describe different aspect of the same thing (“checkin”). BAA should define and describe the business services. ITAA should define and describe the IT services based on the business services. And at the end of the day, the business service and software service are implemented as the same physical thing.