Assessment of user-centred design processes as a basis for improvement action : an experimental study in industrial settings

Abstract

Many software-intensive products and systems on the market today reveal a poor level of usability despite the availability of knowledge about how to develop good usability, namely user-centred design. Improving the status of user-centred design has proved to be a challenge in product development. An effective step to start out on the road to such improvements is to carry out a current state analysis. Our research problem was to learn how to perform an effective current state analysis, i.e. an assessment, of user-centred design processes to provide a basis for improvement action. As the main part of our research, we carried out five experimental assessments in industrial settings during the years 2000 and 2001.

The main result of the research is a novel assessment approach. The objective of the approach is to provide a basis for improvement of the performance of user-centred design in individual product development projects. The particular features of the approach are a new user-centred design process model, a three-dimensional process performance model and implementation of the assessment in the form of a workshop with extensive participation by the project members. The user-centred design process model is method-independent, and consists of six processes that are defined through outcomes. The performance of processes is assessed from the viewpoints of quantity, quality, and integration.

Our theoretical examinations show that different assessment categories can be identified depending on the different foci and purposes of an assessment. Moreover, it may even be appropriate to have different assessment approaches for different categories. In the category of our focus and purpose, i.e. ‘assessment of user-centred design processes for performance improvement’, the focus of an assessment should be in the substance rather than in the management of UCD processes. An assessment approach should be regarded as an artefact and it should be subject to ongoing refinement. An assessment should be an intervention that provides both an effective training session and a meaningful experience to the participants. Clarity in both the concepts and the results seems to take precedence over having standard reference models.

In addition, we created a preliminary theory of usability capability. It identifies three dimensions of usability capability: user-centred design infrastructure, performance of user-centred design in product development projects and usability in business strategy. We also propose some new assessment artefacts for other assessment categories and contrast our research experience with established research frameworks.