Experimental ergonomic evaluation with user trials : EEE product development procedures

Abstract

<div lang="en" class="abs"><p class="abs_header">Abstract</p><p>The main difficulty in the usability evaluation of a product concept or a prototype is that it is very difficult to take reliably into account many different characteristics or attributes, which cannot be measured with the same unit, scale or instrument. Secondly, evaluation always involves some uncertainty. One can never be really sure that all the essential aspects that affect the final user emphasis are taken into consideration. This uncertainty can be decreased by involving diverse people in the evaluation process during R&D or, before the final decision, by elaborating the best ideas to a level at which they can be really used on a pilot scale in the field. Most often, though, the latter takes too much time and would result in many other problems. Evaluation can possibly be best enhanced by using enough involved people, i.e. various experts and especially end-users, who need or use the product in question. A key ingredient to the success of product development, in addition to ergonomic knowledge, is often thought to be active involvement of the intended product users by (1) measuring user-product interaction and (2) participation in design decisions. When a potential end-user experiments with the product, both quantitative and qualitative evaluation methods can be used. This thesis shows some methodological possibilities of evaluation, especially through simulation. It also describes in detail the practical phases of the experiments. For example, a lot of development was needed to find out how to communicate product alternatives and their concepts to (elderly) users. And most importantly, this thesis aims to give evidence of how the procedure called experimental ergonomic evaluation (EEE) should be feasibly implemented and statistically confirmed for significance and consistency.</p><p>A special focus in the experiments was placed on elderly end-users. Since the number of elderly citizens is increasing, there is a need for products to help the elderly live independently at their homes. Studying and understanding how users accomplish their tasks helps to identify their needs and to formulate implications for the design of technology to satisfy those needs. Thus, user studies conducted before beginning to design a new technology provide a proactive way of involving users in the design process. The first prototypes then enable usability studies, such as user trials. With an emphasis on usability engineering, trials can be developed into more feasible EEE procedures for industrial companies.</p><p>All the developed and applied EEE procedures were based on a user-centred approach with different user trial types (N = 15). The users as subjects (N = 264) performed as real tasks as possible and, based on their perceptions during the trials, gave their preferences or scored certain variables. The subjects were also observed and measured by the researcher. The products or other technologies in the trials comprised a total of 9 cases, ranging from “low-tech” steps and chairs to “high-tech” information and communication technology (ICT) applications. The perceived preference and observed performance measures were then combined. Different methods are needed simultaneously to make the results more accurate. The present EEE procedures proved to be cost-effective, efficient and sufficiently valid at least in a research context.</p><p>The EEE procedures ranged from subjective estimations, such as rating and ranking, to more complex multi-criteria methods that can be used to facilitate decision-making, such as conjoint analysis, Mitchell’s paired comparison and use-value analysis. Objective evaluation was also used, including measurement of products and users as well various observations. Both experts and end-users (subjects) had their own important roles in the experiment. Based on this study, EEE procedures are easy to implement in industry for routine usability testing in the course of product development. EEE with its wide coverage yields more universal and absolute usability values, not only ones based on direct benchmarking.</p></div>