Avoid common mistakes on your manuscript.
The phenomenon of so-called “black swans” is gaining more and more attention within the context of IT projects. This increase in popularity was amongst others fueled by a study published by the University of Oxford, which revealed that IT projects often fail because of extremely rare, but definitely harmful events (Flyvbjerg and Budzier 2011).
The former investment banker Nassim Nicholas Taleb coined the term “black swans” and described these as highly improbable events, which are characterized by the following three attributes: unpredictability, high impact, and rare occurrence (Taleb 2008). The term “black swan” itself goes back to the presumption that all swans must be white, because only white swans had been observed in Europe up to that point in time. The first discovery of a black swan in Australia was a shock, because it contradicted all existing observations and apparently prevailing laws. This realization coined the metaphoric use of the term “black swan” for an event erroneously assumed to be impossible.
“Black swans” can have devastating impact within the context of IT projects. For example, they cause one of six IT projects to outrun the budget by about 200% (Flyvbjerg and Budzier 2011). A more precise analysis of “black swans” in IT projects revealed that a major reason for their occurrence can be found in project planning (Kendrick 2008). In this field, especially risks regarding the valuation of IT projects as well as regarding the consideration of interdependencies should be mentioned:
-
Valuation of IT projects: One of the major reasons for the occurrence of “black swans” lies in the fact that risk and return of IT projects are estimated primarily on the basis of employees’ experience. Several studies reveal that project managers frequently evaluate their projects too optimistically or overestimate their operating efficiency. Here, the adequate identification as well as the correct valuation of essential influencing factors is a huge challenge, additionally amplified by the uniqueness of IT projects. Furthermore, false valuations can lead to significant losses especially considering the increasing number of IT projects, which inevitably results in growing complexity. In spite of these challenges, as of today – if at all – only ex-ante valuations are made, i.e., one-time-only valuations before the start of the project, in order to reach an (open loop) decision about the investment without considering (closed loop) later possibilities to react. This initial valuation is frequently followed by a “blind flight” phase until major problems can no longer be ignored and therefore restructuring decisions become unavoidable. This approach neglects the fact that IT projects are continuously subject to change, e.g., because of new requirements they have to fulfill. Especially long-term IT projects, which are significantly affected by dynamic circumstances, cannot be adequately assessed by one-time-only ex-ante valuations. Thus we conclude that one-time-only ex-ante valuations of IT projects are not sufficient to evaluate and control complex and dynamic IT projects.
-
Interdependencies of IT projects: The complexity mentioned above primarily derives from interdependencies between IT projects or other IT assets. Thus delays of IT projects, for example, are often caused by delays of interconnected IT projects. At the same time, dependencies on existing IT assets, e.g., specific hardware or software, can also be considered as risk factors if their availability is crucial for the success of the IT project. Therefore, interdependencies bear the risk that the success of a single IT project is affected not only by the success of the project itself, but rather by the success of dependent IT projects or by the availability of IT assets. At the same time this means that “black swans” might also affect all dependent IT projects. Thus, the identification and consideration of interdependencies within project planning is a crucial success factor.
In order to demonstrate the impact of “black swans” more clearly, two real IT projects will be described in the following, which turned out to be “black swans”.
-
The (partly private, partly public) IT project “eGK – electronic health card” in Germany aims at modernizing processes within the health care sector and making them more efficient. In order to reach that goal, about 110,000 medical doctors, 2,000 hospitals, and 300 health insurance companies needed to be linked to one another. This dimension already demonstrates the huge complexity. This complexity is mirrored in the published cost-benefit-analysis of the project: In 2001, the initial investment was estimated to amount to 1 billion Deutsche Mark (equals 511 million euros), in addition the running costs were estimated at 70 million euros per annum. In contrast, the estimated benefit accumulated to 275 million euros per annum. Three years later, the estimated costs had increased to 1.4 billion euros whereas the expected benefit increased to almost 516 million euros per annum. Additional estimations about the payback period ranged from 31 to 108 months. In 2006, the estimation of the total costs was eventually revised upwards to about 3 billion euros (which equals 6-times the initial estimate!), whereas the benefits did not even double their amount – with corresponding consequences for the profitability of the project. As far as is known today, these estimates will still not remain stable. One of the major reasons for the extreme adjustments to both costs and benefits was the huge complexity as well as the fact that numerous stakeholders repeatedly changed the IT project’s requirements during its runtime. This example shows that ex-ante valuations of benefits and costs might reflect the current level of expectations, which however can be quite independent from the real future development of the IT project. Therefore, in the context of large and complex IT projects, it is not sufficient to solely conduct ex-ante valuations and hope for their realization (cf. Mertens 2009).
-
In 2003, the American clothing company Levi Strauss & Company decided to replace its entire historically grown IT environment by amongst others implementing a new ERP system. In the beginning, the IT project performed fairly well, so that the company could start the launch of the new system in the United States in early 2008. Since the launch was considered a very risky step within the IT project, the company decided to pre-ship many orders to their wholesalers before the launch in order to avoid backlogs resulting from unforeseeable problems during the launch. But reality outperformed all expectations: Technical difficulties during the integration and connection of legacy systems caused all three US distribution centers to be offline for an entire week. This meant that customer orders and therefore the whole sales department came to a standstill. This standstill, which resulted from interdependencies between legacy systems and the new ERP system as well as the distribution system, reduced the quarterly earnings by 98% compared to the previous year. This equals a monetary loss of nearly 200 million US dollars (cf. Krigsman 2008).
Considering these examples, the question arises why such investment decisions were made in spite of the existence of numerous investment valuation methods or why these kinds of events, which occurred long after the beginning of a project, were not considered in advance.
A major reason for this certainly is the application of today’s methods of decision theory. There is a large number of theoretic models, which aim at considering random events within investment decisions. Therefore, risk is mostly seen as random deviation from an expected value. But especially in the context of IT projects, it is not necessarily a random event that causes the failure of an IT project. The course (and thus eventually the success) of IT projects is rather influenced by conscious or unconscious actions, omissions (where the principle of hope results in wrongfully ignoring issues), or decisions that were made. Thus, we can conclude that limiting risk to a random deviation, as it is usually done in today’s investment valuation, is not sufficient in the case of IT projects. This difficulty is compounded by the fact that interdependencies among IT projects, which are caused by the interconnectedness of IT projects, can significantly increase the impact of occurring risks. Due to this interconnectedness, risks can reach systemic dimensions which in the worst case result in a domino and thus cascading effect.
Therefore, decision makers should proceed to obtain a sufficiently precise understanding of the possible reasons for the occurrence of “black swans”, in order to avoid them before and during the runtime of an IT project. At this point, Business and Information Systems Engineering (BISE) as an interfacing discipline can and has to build a bridge between the requirements of information technology and the application of theoretic models of decision science, in order to support decision makers who arrive at this type of investment decisions. It is a central task for BISE to provide all relevant information throughout all project phases that is needed for a value-oriented decision support. It further should provide assistance regarding the identification and gathering of appropriate information, develop concepts of how this information needs to be processed, and provide methods that derive recommendations based on this information for project management. The goal is to integrate information gathering and processing into the practice of project valuation and steering. The following pages describe how BISE can support the avoidance of “black swans” throughout different project phases (before, during, after):
-
Before the start of an IT project, the focus lies on its valuation and therefore on the comparability of the various influencing factors. Therefore, a business case is calculated based on monetary figures. There are already some concepts that valuate the total costs of an IT project (e.g., “total costs of ownership (TCO)” or “constructive cost model (CoCoMo)”). While the project management tries to precisely estimate the costs of IT projects, the potential benefits are often neglected. This is due to the fact that benefits usually occur after the successful completion of a project and are very difficult to forecast. But also benefits that can be assigned to decisions which are made during the runtime of a project can play a crucial role for the investment decision. The starting point for the benefit-cost-integrated valuation of those project decisions is the assessment of the corresponding cash flows. The amount of cash inflows results from both “hard benefits”, which can be measured directly or indirectly in monetary units (e.g., cost savings), and “soft benefits”, which can hardly be allocated and only with difficulty measured in monetary units (e.g., quality improvements). Project-induced cash outflows can also be divided into directly or indirectly measurable cash flows (e.g., licensing costs) and hardly assignable and therefore hardly measurable cash flows (e.g., transaction costs). Since these cash flows do not occur with certainty due to the fact that they are rather influenced by dynamic developments throughout the project, it is important to include their variability, which can be interpreted as risk, in the investment decision. Furthermore, existing interdependencies with other IT projects or processes within the company should be analyzed. Frequently the same resources, such as the IT infrastructure or specifically trained personnel, are used by multiple IT projects, which can cause serious problems. To avoid these problems, the anticipated interdependencies should be considered even before the project is started. However, all influencing factors described above (benefits, costs, risks, and interdependencies) are based on subjective estimates conducted by project staff, which inevitably leads to misjudgments. Therefore, it is also a task of BISE to develop concepts which reduce subjective misjudgments with the help of debiasing methods. In order to be able to evaluate an IT project under consideration of all mentioned influencing factors ex-ante, a decision rule, which accounts for all or at least the most relevant aspects, should be used for project valuation. All in all we conclude that BISE can and must contribute more to support an integrated valuation of IT projects.
Furthermore, the complexity of the valuation and steering of IT projects can be reduced by means of higher transparency resulting from the implementation of an integrated and standardized assessment method, which is another positive aspect of the approach described above. Standardized valuation templates for example can simplify the valuation and steering of IT projects and provide consistency in project assessment. Through the conceptual development of those templates on the one hand, and through the definition of a corresponding decision rule, BISE can make a significant contribution to improve ex-ante project valuations.
-
During the runtime of an IT project, continuous and closed loop project controlling is inevitable, since internal factors (e.g., too optimistic estimates) and external factors (e.g., technological development) might necessitate reallocation of resources. Within the ex-ante business case, the factors functionality, time, and budget are determined. This determination relies on estimations which are affected by changing conditions as well as by actions and decisions made within a dynamic environment during the runtime of a project. Thus, there is a trade-off among those three factors, which will be described in the following: If the focus of a project is placed on time (i.e., on the implementation of the project in time), after the identification of changing conditions which negatively affect the course of the project the factors budget and/or functionality will suffer. The reallocation of resources, as for example reducing functionality or exceeding the anticipated budget, requires a new valuation of the business case, because these kinds of changes can negatively affect the benefits of the IT project. The project manager therefore needs information about the impact of intentional changes of these factors in order to initiate necessary corrective steps and counteract adequately. Thus continuous project controlling enables an early identification of changes of the predicted cash flows during the course of an IT project. Considering that projects run within complex contexts and have diverse dependencies on other projects and processes, the identification of changes and the corresponding active steering are of major importance. Even if the described project runs smoothly, bottlenecks might, for example, arise through adding a new project to the IT project portfolio. Furthermore, unexpected changes within interdependent projects can lead to the shortage of resources, which is why a continuous update of the factors used for project valuation is inevitable. By doing so, decisions taken prior to the start of the project can be adjusted based on available information. Moreover, this approach leads to a more accurate determination of the project value over the course of the project, since more accurate information about the actual realization of estimated factors becomes available over time. Due to this, the project manager knows whether risks, which were considered within the ex-ante valuation, are materializing over the course of the project or not. Whereas the tracking of cash outflows is already included in project controlling, the tracking of benefits (i.e., which factors affect the achievement of anticipated benefits of a project) and thus of cash inflows has been neglected so far. In order to monitor and steer the benefit realization over the course of the project as well as for an early identification of the need for action, the controlling of benefits becomes crucial. Therefore, BISE should contribute to a value-oriented allocation of resources within the scope of a continuous project steering by developing best practices, which, e.g., focus on the process of a continuous IT-controlling and provide corresponding valuation and steering templates.
-
After the completion of a project, a checkup becomes necessary to identify whether the prospected goals have been achieved – and last not least to reap the benefits of learning from the mistakes that were made. Often the obtainable benefits of an IT project realize long time after its successful completion. Therefore, it might become necessary to take subsequent actions (e.g., tracking of user acceptance or direct cash inflows) to assure the realization of the benefits predicted. Up to now, this aspect has been left widely unconsidered in practice, which leads to a waste of value added. Thus a sound ex-post controlling of the benefits of an IT project is essential for assuring the achievement of objectives and should therefore be integrated in the valuation and steering process of an IT project or IT project portfolio.
The use of standardized valuation and steering templates in the course of a project, as mentioned above, enables the documentation of the developments of influencing factors. On the one hand such a record facilitates the comprehensibility and transparency of past decisions, on the other it delivers valuable experiences to other comparable projects and therefore provides a basis for organizational learning.
In summary it can be said that the discipline of BISE is able to and even has to contribute far more to the success of IT projects instead of solely applying rigid investment evaluation methods one-time-only and before the start of an IT project. Moreover, it is the responsibility of BISE to enable the application of an integrated method for the valuation and steering, while including benefits, costs, risks, and interdependencies across the entire lifecycle of an IT project. However, the examples mentioned above show that the described set of measures still significantly lacks application in practice. Therefore, there remains a long way to go until the collaboration of science and practice is able to generate methods which are academically established as well as practically evaluated, to facilitate an early identification of “black swans” as well as to trigger readjustments and reallocations of resources against the background of value creation. The measures mentioned above should not be considered as final solution – they should rather make researchers think about their future work at the interface between theory and practice. Only in this manner, means and possibilities can be developed to reduce the probability of “black swans” or at least to diminish their devastating impacts and thus to sustainably enhance the value added by IT projects.
References
Flyvbjerg B, Budzier A (2011) Double whammy – how ICT projects are fooled by randomness and screwed by political intent. http://users.ox.ac.uk/~mast2876/WP_2011_08_15.pdf. Accessed 2012-01-23
Kendrick T (2008) Avoiding black swans: managing risks using the PERIL database. http://www.failureproofprojects.com/BlackSwan2008.pdf. Accessed 2011-12-14
Krigsman M (2008) Levi Strauss: SAP rollout ‘substantially’ hurt quarter. http://www.zdnet.com/blog/projectfailures/levi-strauss-sap-rollout-substantially-hurt-quarter/917. Accessed 2011-12-14
Mertens P (2009) Fehlschläge bei IT-Großprojekten der Öffentlichen Verwaltung. Arbeitspapier, Fachbereich Wirtschaftswissenschaften, Universität Erlangen-Nürnberg
Taleb NN (2008) Der Schwarze Schwan: Die Macht höchst unwahrscheinlicher Ereignisse, 2nd edn. Carl Hanser, München
Author information
Authors and Affiliations
Corresponding author
Additional information
This article is also available in German in print and via http://www.wirtschaftsinformatik.de: Buhl HU, (2012) Der Beitrag der Wirtschaftsinformatik zur Früherkennung und Vermeidung von “Black Swans” bei IT-Projekten. WIRTSCHAFTSINFORMATIK. doi: 10.1007/s11576-012-0313-7.
Rights and permissions
About this article
Cite this article
Buhl, H.U. The Contribution of Business and Information Systems Engineering to the Early Recognition and Avoidance of “Black Swans” in IT Projects. Bus Inf Syst Eng 4, 55–59 (2012). https://doi.org/10.1007/s12599-012-0206-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12599-012-0206-8