Case Study: Performance of IT products

Do you know the performance of your IT?

We have already reported on the advantages of knowing the performance of your IT products. Because only those who really know the strengths and weaknesses of their IT know where there is potential for optimization and can consciously work on this to increase value. The instrument of choice here is IT benchmarking.
In this article, we use a case study from our practice to show how IT benchmarking can look in practice.

Please notice, that we define the term ‘performance’ somewhat more broadly. By this we mean not only the general performance of IT products but also the sum of development effort, productivity in terms of development speed and
costs and the efficiency of IT products. Before we go into the case study, here is a brief definition of what we mean by IT benchmarking.

What is IT benchmarking?

IT benchmarking is a strategic method in which the performance of a company’s IT services is compared with those of other companies. Of course, only the comparison with the best in class is worthwhile. The BAMAC GROUP focuses on financial evaluation with the aim of achieving an economic advantage in terms of value added for its customers.

Determining the performance of IT products

The performance of IT products is to be tested for an industrial company. The method used to determine performance is the method of the International Function Point User Group (IFPUG) in the current version. Five representative applications were selected for the analysis. These applications have already been developed, so that information about the content (range of functions), effort and costs is available. The aim is to find answers to the following questions.

• What productivity is available in the applications already developed for Change (development) and Run (maintenance)?
• Are the determined productivities in line with the market (benchmark)?

The special feature of this case study was the combination of function point analysis and IT benchmarking in order to determine the performance of the IT products.

Procedure and dependencies

Initialization

The first step is to record customer expectations. The resulting target image is the basis for the scope of the IT benchmarking analysis and the planning to be drawn up.
In preparation for the kick-off meeting, we asked our client to provide various documents, including technical and IT concepts and change requests, which were required for the analysis. In addition, contact persons, e.g. project manager, specialist, IT architect, were identified for the individual applications, who should be available during the analysis phase for queries regarding functional and technical functions.
We receive the data on actual expenditure in person days (PT) and EUR (internal and external hourly rates) from the Project Management Office (PMO). Checking the availability of resources and data as well as integration into the customer organization are important steps in this phase.

Kick-Off

All project stakeholders meet at the kick-off. The planning and the expected result types are discussed here. If this has not yet happened, the dates for the next steps are set at the kick-off.

Analysis

In this case study, function point analysis was initially used as a method. In this phase, the data provided by the company determines not only the quality of the results, but also the duration of the process. The better the documentation and data situation, the faster we can determine results.
If no data is available from the company, it is collected during workshops and interviews. In this step, we also resolve comprehension issues and check the data for benchmark capability. This was followed by an IT benchmarking analysis in which the development speed (FP/PM[Change]) and development costs (EUR/FP) as well as the maintenance costs were determined.
Using the tailoring of the COCOMO model, it was possible to determine a phase distribution for the projects based on the percentage distribution.

QA workshop

This was followed by a quality assurance workshop with the specialist and technical contacts to review the assumptions for the individual IT products. In the course of this, we clarify open questions and, if necessary, correct some assumptions.

Results on the performance of IT products

The five applications were small and medium-sized applications (143 – 487 FPs). The function point method according to IFPUG could be applied to all applications, as these were management information systems. The benchmarking of the applications revealed that one application was in the top quartile in terms of development time and development costs (change).

However, it was noticeable that the maintenance costs were unusually high. We suspect that this is due to problems being shifted from application development to maintenance and operation.

The productivity determined for the development period (FP/PM) showed above-average values. The productivity for the development costs (EUR/FP) made it clear that there is potential for cost savings. The values determined indicated 20% higher costs than selected reference projects (benchmarking).

The diagram illustrates the process.

Important for the process

In the example described, the documents provided were ideally suited for the analysis. We assume that this was due to the fact that it was a post-calculation – the documents had already been quality-assured and approved. A look at the rough concept also made it clear that the formulated requirements, which were available for planning and commissioning at the start of the project, were extremely suitable for our analysis.

Benefits for the industrial company

The gain in this investigation was obvious. On the one hand, it was confirmed that the combined procedure of function point analysis and IT benchmarking provides the company with a suitable procedure for carrying out estimates in a transparent and comprehensible manner. The information on productivity and the benchmark carried out in parallel revealed potential savings, which were used as the basis for discussions with three sourcing partners. In addition, it became apparent that the management quickly accepted the methodology and recognized the logic of it as comprehensible and therefore trustworthy approach. The next step was to incorporate the procedure into the organization, which was done in another measure.

This example shows that agility and customer satisfaction are our daily business. We always adapt our evaluation strategies and tools to the needs and questions of our clients in order to determine the most accurate key figures. In our blog, you can find a detailed case study on the possibilities of IT benchmarking. If you would like to talk about specific challenges and problems you are currently facing, please feel free to contact us.