Building an image- and video processing application on Apple iOS platform using a parallel programming model

Abstract

Today powerful parallel computer architectures empower numerous application areas in personal computing and consumer electronics and parallel computation is an established mainstay in personal mobile devices (PMD). During last ten years PMDs have been equipped with increasingly powerful parallel computation architectures (CPU+GPU) enabling rich gaming, photography and multimedia experiences and general purpose parallel computation through application programming interfaces such as OpenGL ES and Apple Metal.

Using a narrative literature review this study viewed into current status of parallel computing and parallel programming and specifically its application and practices of digital image processing applied in the domain of Mobile Systems (MS) and Personal Mobile Devices (PMD). While the research on the context is an emerging topic, there still is a limited amount of research available on the topic. As acknowledged in the literature and in the practice, the OpenGL ES programming model for computing tasks can be a challenging environment for many programmers. With OpenGL ES, the paradigm shift from serial- to parallel programming, in addition to changes and challenges in used programming language and the tools supporting the development, can be a barrier for many programmers.

In this thesis a Design Science Research (DSR) approach was applied to build an artefact — an image- and video processing application on Apple iOS software platform using OpenGL ES parallel programming model. An Open Source Software (OSS) parallel computing library GPUImage was applied in the implementation of the artefact filtering- and effects functionality. Using the library, the process of applying the parallel programming model was efficient and productive. The used library structures and functionality were effectively suppressing the complexity of OpenGL ES setup- and management programming and provided efficient filter structures for implementing image- and video filters and effects. The application filtering performance was measured in real-time- and post-processing cases and was perceived as good, alongside the feedback collected from demonstration sessions and end-users.

However, designing new custom cinematic filters using OpenGL ES Shading Language is a challenging task and requires a great deal of specific knowledge of technical aspects of the OpenGL ES domain. The used programming language (OpenGL ES Shading Language) and tools supporting the work process of design, implementation and debugging of the GPU program algorithms were not optimal in terms of applicability and productivity. Findings note, that more generic and applicable language would benefit the development of parallel computation applications on PMD platforms.