Reflections

This week was all about design research and the use of prototypes. The intension of the two lectures was to give an idea on how design technologies, prototypes and artifacts can contribute to research. In particular, it was stressed how important the idea creation and the use of prototypes is, especially for a technical driven research. Therefore, the week was helpful for me to get to know the process of designing research and developing prototypes, topics I have not had knowledge about before.

The questions we had to ask ourselves are how we actually have to come up with ideas, how we then filter and validate them, and how we communicate ideas in the end. The most important step in coming up with a high-value idea is to focus on the problem. In particular, there is a difference between a problem and the real problem of a situation. Our teacher exemplified this by setting two people in the woods, when suddenly a bear comes into the scene. Since the bear conveniently is hungry and certainly a faster runner, the two have a problem now. As the lecturer pointed out though, the real problem is not who can run faster, but who can outrun the other person.
Transferred to the design process, it is more important to define the actual problem than to follow the first thought of solution that comes into your mind. Ninety per cent of our time should therefore be taken to find the real problem while ten per cent of the time should be focused on finding a solution.

What I deducted from this particularly is the connection between the real problem and the idea. A great idea actually works, because it complements the real problem with or without knowing it.
So, how do you come up with a potentially great idea? You have to find a real pain point first, meaning a real problem that needs solving. If the market and the timing is right in addition, you have to check if the idea is able too stand strong when further pursued. I compared these criteria on inventions, which we clearly still benefit of today and which were designed out of genius ideas: the TV remote control (1955), the Microwave oven (1955), the Jet Airliner (1958), the Industrial Robot (1961), the Communications Satellite (1962) or the LED (1962), just to name a few (Hutchinson, 2005). Although market situations tend to change quickly, it is apparent that these inventions are still profitable and the products used on a daily basis.

In order to prove that an idea or certain method works, you have to demonstrate its feasibility with the proof of concept, which mostly includes the development of a prototype. As I have already formulated the roles of prototypes in my first theme post, I will just shortly focus on their most important qualities: prototypes are a "first, typical or preliminary model of something, especially a machine, from which other forms are developed or copied" (Oxford Dictionaries, 2015). They typically present different stages during a design development, always representing the current perspective of the designer. Since prototypes continuously question the design decisions, they provoke to gain more knowledge and with that are able to direct the research process.

Hutchinson, A., 2005. The Top 50 Inventions of the Past 50 Years [online]. Popular Mechanics. Available: http://www.popularmechanics.com/technology/gadgets/a341/2078467/ [Accessed at 11 Oct. 2015].

Oxford Dictionaries, 2015. prototype: Definition von prototype im Oxford Dictionary (Amerikanisches Englisch) [online]. Available: http://www.oxforddictionaries.com/de/definition/englisch_usa/prototype [Accessed at 11 Oct. 2015]. 

1. Blog Comment (Link):

As always I enjoyed reading your blog posts since you sincerely recap on what you have learnt throughout the week and structure your thoughts in an easy and clear manner. We probably all have gone through qualitative and quantitative methods throughout our school or university life, still, you highlighted two important aspects I haven't thought about anymore. In our seminar we started to discuss our bachelor theses, which also shows how different bachelor programs produce diverse methods in approach. Reading through your pre theme post, it is obvious that you analyzed the research paper on FoodLog in-depth, in order to detect its strengths and weaknesses. Great job!

2. Blog Comment (Link):

You did a great job reflecting on the past week by extracting the most important things you have learnt. In my blog post I also put together the key elements when setting up a questionnaire, that's why I found your inside information on this particularly interesting. I share the opinion that even seemingly unimportant things like the layout or the number of questions can already influence the data results. Therefore, questionnaires are one very defining part in the design research of quantitative research studies and strongly shape the quality and magnitude of the outcome.

3. Blog Comment (Link):

Thank you for your reflections on this week's theme Quantitative Methods. I agree with you that this topic neither gave away a lot of new information nor provided much room for a discussion similar to our first themes. However, it is very interesting to read about your group discussion and in particular about the categorization procedure of a questionnaire. Moreover, your clear structure succeeds in guiding the reader through the text and communicates your knowledge well. Well done!

4. Blog Comment (Link):

It was interesting to read about your thoughts and experiences during the past theme. Each week we have passed, it gets harder to remember what we had actually read for the week, so I agree with you and admit to always re-read my own postings as well! In our seminar, we ended up discussing the methods of our bachelor theses, but in the end that just proved how diverse quantitative methods could be. Reading your pre-bog post, I think your revision of methodology is very helpful. Also the division into the areas of design, sampling, data collection and data analysis was a great way to answer the given questions enhanced by the clear structure. You did a great job preparing and reflecting on this week and I enjoyed reading your postings!

5. Blog Comment (Link): 

I really enjoyed reading about the research paper chosen in your pre-post and the insight scoop on its chosen variables and the outcome of the study. In your reflections you also demonstrate your knowledge about the qualitative and quantitative methods while detecting their benefits and limitations. I was particularly interesting to read about your bachelor thesis topic! Great job!

6. Blog Comment (Link):

Thank you for your well-rounded reflections on this week's theme quantitative research. It feels like you have a deep knowledge of the subject and can appoint easily when this method should be used. In addition, your explanation on its benefits by using your research paper achieves to emphasize the accuracy and scope of this method. Furthermore, I think you're text is very well written and nicely structured. I especially appreciate the text highlights since they really emphasize the most important messages.

7. Blog Comment (Link):

You did a great job reflecting on what you and your seminar group have learnt and discussed this week. I especially appreciate your comment on the gender perspective of the article Drumming in Immersive Virtual Reality, since I haven't thought about this before. You also summarized perfectly how categorization in quantitative research can also influence the quality of the outcome by its limited response scale. However, there's probably no perfect solution since one is dependent on classification with a growing data extent, but when using a more individual method of gathering data, the research is very likely smaller and not generalizable at all.

8. Blog Comment (Link):

You really demonstrate an usual and distinct approach in your reflections, which made it very interesting to read! I haven't though about the levels of subjectivity and objectivity in quantitative method in-depth like you did, but now with your structure, well-written thoughts and examples you really explained the difference and the underlying problems perfectly.
"Although abuse of data is much less found in research papers than in magazines, newspapers, polls and TV, it is important to realize that statistics is not always “objective” or reliable." How often do we hear 'based on the study of...', 'following new numbers of...' in nowadays media. Your statement here really zones in on the problem of quantitative methods: that the word study might actually have a bad connotation since it is misused in our everyday life and that we are actually confronted with a lot of wrong data on a daily basis!

9. Blog Comment (Link):

Your reflection really summarizes all the important messages of this week's theme quantitative methods. It is clear that you already had a good understanding of these techniques before, but I still appreciate your concise discussion of their benefits and limitations. I especially like that you put your own thoughts in this and tried to get more out of the topic, although you were very familiar with it. You found a good example to explain the combined use of qualitative and quantitative methods - this really visualizes how research can increase in value by blending both together. Enjoyed reading your posts!

10. Blog Comment (Link):

Thank you for your well-written and crisp reflection! You succeed in guiding the reader through your text making it easy to follow your thoughts. It is interesting that the lecture and seminar could change your perspective on objectivity and how we are often judging before assessing. Your explained the connection between the design of the study and the researcher's perception in a very understandable manner. The visualization with candy really helps to clarify your point in favor of a combination of the two methods: "this would shed light on our perception of quantity" – Well said!

Qualitative and case study research

1. Select a media technology research paper that is using qualitative methods. The paper should have been published in a high quality journal, with an “impact factor” of 1.0 or above. The following are examples of questions to discuss in your blog posting:

Which qualitative method or methods are used in the paper? Which are the benefits and limitations of using these methods?

Following the given instructions, I chose a high quality research paper called Cultural Similarities and Differences in the Design of University Web sites by Ewa Callahan, which was published in the Journal of Computer-Mediated Communication (Impact factor: 3.117). It "examines cultural differences and similarities in design of university Web sites using Hofstede’s model of cultural dimensions" (Callahan, 2005: 239).

Graphical elements on the university websites in eight different countries were compared to each other using the content analysis method. In particular, there was a focus on the visual content, differencing between the criteria of organization and graphical design of the homepage. On the other hand, the method is beneficial since it enables the establishment of individual categories applied on the websites and can fully adept to the researcher's considerations without any restrictions. On the other hand, there is a big chance that not all possible graphical elements on each site are considered. In addition, the study is limited to the genre of university homepages, which means for instance, that the pictures used are very likely to show a higher quantity of younger than older people. Since qualitative methods usually have to compromise between individuality and data size, this study also was limited within the selection of countries.

What did you learn about qualitative methods from reading the paper?

The study shines light on a university's opportunity to adapt to future students and possibly how to rise its attraction level. Although the qualitative method proves to be very extensive and tries to include various categorizations, I learned that it can never really attest to reality or include all possibilities. Especially, cultural differences are hard to compare specifically, but still, it is interesting to see how to undertake a qualitative design process by using Hofstede's cultural model, how his indexes correlated with the abstracted data and how this is further on interpreted. Additionally, the study method succeeded in detecting cultural markers typical of specific cultures.

Which are the main methodological problems of the study? How could the use of the qualitative method or methods have been improved?

One methodological problem might be that the graphical design is not just influenced by the culture in the country of the university, but also depends on the intension, background and motivations of the designer. Often the homepage of one specific country is replicating the organizational structure of homepages in other countries and are therefore visualizing more than one cultural perspective. Another issue is that cultural aspects are strongly influenced by the language and discourse styles, which were not analysed in this study. Therefore, the method could have been improved by including language as a variable. It would have also been interesting to see, if cultural differences in web design influences the usability or attraction level for different types of students. Moreover, since just a few universities in a country were tested, it should be considered to extend the scale of research data in terms of the number of universities in a country or the number of countries tested itself.

Callahan, E., 2005. Cultural Similarities and Differences in the Design of University Web sites. Journal of Computer-Mediated Communication, Vol. 11, No. 1, p. 239-273.

2. Read the following article:

Eisenhardt, K. M. (1989). Building Theories from Case Study Research. Academy of Management Review, 14(4), 532-550.

Select a media technology research paper that is using the case study research method. The paper should have been published in a high quality journal, with an “impact factor” of 1.0 or above. Your tasks are the following:

Briefly explain to a first year university student what a case study is.

By using single or multiple cases on various levels of analysis, a case study tries to understand the dynamics present within single settings. They typically use different data collection methods that can be qualitative or quantitative (e.g. archives, interviews, questionnaires, observations). The intension of case studies is to postulate explanations, and test or generate theories.

Use the "Process of Building Theory from Case Study Research" (Eisenhardt, summarized in Table 1) to analyze the strengths and weaknesses of your selected paper.

In the following, I will analyze the paper with the help of Eisenhardt's table summary as a roadmap for how to execute this kind of research. The paper in question deals with Revealing the Vulnerability of People in Places: A Case Study of Georgetown County, South Carolina, published in the Annals of the Association of American Geographers (2014 Impact Factor: 2.291).

Getting Started
The paper successfully defines the research question after explaining beforehand the necessity of the research itself. By using a conceptual model of vulnerability that implements both biophysical and social indicators to assess hazards, the design is trying to understand the complexities of vulnerability and its attributes and effects in a real-world setting.

Selecting Cases
There were no theories and hypotheses brought up, but the study examined a specific population in Georgetown County, South Carolina. The location was chosen since the researchers were familiar with the area, the area experiences a lot of different hazards and has a broad sociodemographic profile.

Crafting Instruments and Protocols
The research uses qualitative and quantitative data, in order to investigate the study area. For instance, the data of hazard events and the rate of occurrence had to be collected by using historical data. Further, the hazard zones had to be detected and spatial impact areas were defined. The research uses for instance the model of the National Hurricane Center for likely affected areas.

Entering the Field
Going through various studies and research data that investigated natural hazards, the researchers describe the context of the case study and the underlying social and biophysical elements in-depth.

Analysing Data
The extensive data was analysed by putting the different data layers into a geographic information system. To extract the information, all layers were combined into a single composite of intersecting polygons that had different biophysical hazard scores assigned. That way, the geography of individual hazard zone as well as 'vulnerable' areas could be examined.

Shaping Hypotheses
The research logically derives evidence from the researched data and explains extensively the causes and in-between relationships. Wile the factors are very hard to define, terms like social vulnerability or the vulnerability of places were assessed in detail.

Enfolding Literature
Throughout the research article, literature is compared to the study and the provided information, especially at the beginning to cover the ground of geography as human ecology. However, there is not a specific chapter in the closing parts, which would enfold literature.

Reaching Closure
The case study presents a template for future integrated place-based hazard studies. The public policy ground still leaves a lot of space uncovered, and this research proves that researchers can significantly contribute.


Cutter, S., Mitchell, J. and Scott, M., 2000. Revealing the Vulnerability of People and Places: A Case Study of Georgetown County, South Carolina. Annals of the Association of American Geographers, Vol. 90, No. 4, p. 713-737.

Eisenhardt, K., 1989, Building Theories from Case Study Research. The Academy of Management Review, Vol. 14, No. 4, p. 532-550.

Reflections

In last week's lecture and seminar, we heard about the main character traits of the quantitative method in respect to the research paper Drumming in Immersive Virtual Reality, as well as recapped the benefits and limitations of this type of research, which has been additionally covered in our pre theme.

One thing that stayed in my mind is that the lecturer stressed particularly the importance of the design process. Quantitative research is following a fixed design, which is already predetermined before the data collection starts. On the one hand, this enables a clear structure, which focuses on the research objectives; on the other hand, the flexibility and freedom during the measurements are very low. Since this type of research in particular depends on numerical measurements, questionnaires are often used.

When our lecturer mentioned that the procedure to the final questionnaire could actually take years, I got interested how a questionnaire is actually set-up. In the following, I will therefore reflect on the approach and line of action.

A questionnaire in a quantitative study is usually formally standardized to try to test explicit hypotheses that have been formerly generated. The questions are prescribed in their wording and order, their affiliated explanation, and their response format enabling an easy completion.

The starting point is to articulate which objectives you intend to meet with the questionnaire. Therefore, it has to be clear which information is actually required, which should be also comprised in the research proposal. Similar to marketing research, the next step is to decide which respondents are targeted. It must be taken into account that the samples of interviewees are reacting in terms of their age, education, etc.
Moreover, the method of questioning has to be established (personal singular/group interviews, via mail/telephone) since the construct of the questions has to change according to the medium.
On the one hand, the content has to comprise the most critically important questions concerning the research objectives. Therefore, the questions should be tested in their contribution factor or redundancy. On the other hand, some questions might specifically be neutral to distract the examinees from the study purpose.
The design of the questions can be formed in three different ways (closed, open-ended, open-response); the wording via the closed-form provides the most exact answer, and easy evaluation and specification afterwards. In order to keep the attention of the interviewees up, you have to guide them through your questionnaire with some opening questions, a natural question flow, a good variety also keeping the length and time in check. In the end of the progress, the appearance should be focussed on as well, since the layout has a significant effect on how much and what quality data is collected; there a pre-test could also help to improve the design.


Fao.org, 2015. Chapter 4: Questionnaire Design [online]. Available at: http://www.fao.org/docrep/w3241e/w3241e05.htm [Accessed 5 Oct. 2015].

1. Blog Comment (Link):

The research article you chose seems very interesting, especially in our age of digitalization. I wonder how the technology has actually changed the behaviour of nowadays children, therefore, I will read through this article for sure.
It is interesting to read about your seminar discussion and that you used the example of the pen to explain the difference between a theory and a hypothesis. I think that a there is no such a thing as a wrong theory, but just a hypothesis that prove out to be wrong. To make sure that it is not just an experimental error like you described, it is probably always advised to compare the hypothesis to practise more than once.

2. Blog Comment (Link):

It is obvious that you put a lot of time and effort into your two posts this week. You did a great job by structuring the topics and focussing on the main points of your research paper, the lecture and the seminar. I think the most interesting thing of the comments this week is to read the research articles chosen by every student. The role of Twitter in the Political Campaign seems very contemporary and the integration of social media a development of new media that will just rise in the future. Interesting read!

3. Blog Comment (Link):

Thank you for your thoughts this week. Your clear structure of your reflections makes it very easy to get into the topic. Interesting how you worked with the research article and the questions you chose to focus on. I agree with you, I also found the third's week topic harder to get into and to follow. However, you did a great job at summarizing the theme's tasks and describing what you have learnt, in particular that you have to put the validity of theories also in context of the academic discipline. Well done!

4. Blog Comment (Link):

Thanks for sharing your insights on this week's theme. I agree with you that social media has developed to a very one-sided tool of profiling. People usually want to get validation constantly. Jumping back to Adorno and Horkheimer for a second and their take on a slow developing superstructure, I wonder what affect this current social media behaviour will have had for instance on the values within our society in a few decades. Let's hope therefore, that the given positive sides of this type of interaction prevail. Regarding your post-reflection, you described the meaning of theory clearly and easily to comprehend. Great job!

5. Blog Comment (Link):

Your research paper sounds very interesting. I wonder what results it presented in the end, if they are significant personal and social factors to be representative as a gamer. You take on the research shows that you analysed it in-depth and thought about possible improvements. In general, it is easy to follow your thoughts thanks to your clear writing, emphasized remarks and the visual images you put in your text!

6. Blog Comment (Link):

I agree with you and your remarks on this week's topic. Since I generally always put more focus on the practical sides on media, it made it hard for me to get into the texts about theory. However, our group discussion proved to be very interesting – although it is quite hard to pinpoint the term theory that everybody would agree with. Your quote is indeed chosen quite well, since it reflects the meaning of hypothesis in a very crisp way and nevertheless leaves room for reverberation.

7. Blog Comment (Link):

Thanks for sharing your well-rounded thoughts on theme 3. Your take on theory and its connection to our previous topics made me think about them once more. Likewise your emphasis on why theory can never be stated as true. A theory has always again been matched by practical experience due to direct and indirect changes surrounding it. Your samples are great to shed light on this 'theoretical' subject and I enjoyed reading your reflections on it. Great job!

8. Blog Comment (Link):

Your reflections are as insightful as ever, therefore it was a joy to read them! I felt the same way about this week's theme and the difficulty of defining theory. Your research paper seems to break ground on an uncovered topic, but indeed one that will grow in importance in the following years. Social media has obviously a lot of advantages, however it is interesting to see it from an objective news perspective. Since I also belong to the category of 'news selector', I wonder how this shapes my perspective in the long run. Realistically, I probably miss out on news coverage I would have stumbled on by reading a newspaper, which intends to cover a range of topics for a wider audience. It will be interesting to see how digital journalism will develop further on and moreover, how journalists will keep their integrity.

9. Blog Comment (Link):

Thank you for your reflections! You succeeded in explaining the week's main achievements with the help of a clear structure shaping your thoughts. The seminar has evidently accomplished to give more complexity to the subject, for example by touching on the difference between theory and hypothesis.
One statement I found especially fascinating in your chosen research paper is that politicians have yet failed to fulfil the potential of social media. It will be interesting to observe how campaign management will be perfected in the future and which risks lay in social media's power of steering voters to one side. What are your thoughts about this?

10. Blog Comment (Link):

I really enjoyed reading your thoughts about this week's theme. Your well-structured texts make it easy to follow and you have plenty of interesting remarks on our subject 'theory'. It seems even difficult for scientists to really specify the term itself. Here, it is interesting to read your reflection on the seminar and how you categorized theory into good and bad.
The topic you chose as research paper is quite interesting. I was not that aware about local news stations being forced to change due to the alteration in broadcasting and how this transformation affects the profession as a journalist. I especially like your sentence "more news does not mean more democracy or a better informed public", since it is really thought provoking. Well done!

Design research

To answer the following questions I read the given research paper, written by Haibo Li with colleagues 'Turn Your Mobile Into the Ball: Rendering Live Football Game Using Vibration' and also read the chapter 'Prototyping tools and techniques' of the book The human-computer interaction handbook.

How can media technologies be evaluated?

Suggested by the given paper, media technology can be tested by experiments, by evaluating its usability or by trainability, which helps in improving the system. As we are in the age of digitalization and a lot of technologies depend on the user surface, I would put in particular a lot of focus on evaluating usability. Following the guidelines ISO, "usability is defined as the effectiveness, efficiency and satisfaction with which specified users can achieve specified goals in particular environments" (ur Rehman et al., 2008: 1026).

What role will prototypes play in research?

Prototypes describe the thoughts of the designer at specific stages in the design process, and always try to represent what is the best way to usefully contribute answering the research question. Since prototypes take on different purposes throughout the design process, they also take on different forms. In general, there are two different sorts of prototypes: the Off-Line prototype (paper prototypes) and the On-line prototypes (software prototypes).
The first ones are non-computer based, usually drafted by hand very quickly at the early stage of the design and are destroyed as soon as their purpose is fulfilled. The latter, On-line prototypes, require a computer, have usually a higher cost, but can contribute to research variously by showing animations, interactive video presentations, or contain written programs or applications. These are likely to contribute to research, since they give users the opportunity to explore the system actively. Moreover, "(p)rototypes increase creativity, allow early evaluation of design ideas, help designers think through and solve design problems, and support communication within multi-disciplinary design teams" (Beaudouin-Lafon and Mackay, 2003: 1102).

Why could it be necessary to develop a proof of concept prototype?

Since prototypes are an essential part of an interactive design process and therefore also the resulting data, the concept has to be thought through in-depth. Throughout the design process, they represent the progress of the research, implementing the ideas of the research designers and showing how the system could look and feel like. The concept prototype has to follow certain principles set up by the researchers in order to assure a representative outcome of the research that could be used as basis further on.

What are characteristics and limitations of prototypes?

Prototypes vary in their possible forms and characteristics widely depending on their intention. They could be rough sketches or detailed computer software. However, they always intend to give the designers, users and developers an early glimpse of a possible future product. They are concrete tools to explore the design space, continuously question the design decisions and can therefore steer the research process, because they "highlight the areas that must be refined or require additional ideas" (Beaudouin-Lafon and Mackay, 2003: 1029). The advantage, for instance in the given research paper is that the prototype functioned as test object for both researchers and mobile-users presenting an un-known system and enabling experiments in order to develop it further on and test the user responds to vibrotactile rendering of live game information.

The limitations of prototypes are their confined test coverage being only tested on a few examples, not build for other cases. Therefore, prototypes will most likely not work (or crash) on other cases since they are too specific and built for something else. Moreover, prototypes can do early test rounds on users and usability, but since its still just a model, you do not get the final user assessment as you would for the finished product.

How can design research be communicated/presented?

Design research evaluates the test results of the design process like any other research paper. Evaluations should either be based on the quantitative measures or on the qualitative measure like user impressions. Since the results are very specific and are likely to vary within other designs, the researcher has to consider other possible factors, like product features that would evolve in a non-test environment. 

Beaudouin-Lafon, M. and Mackay, W., 2003. Prototyping tools and techniques. The human-computer interaction handbook. J. Jacko and A. Sears, p. 1006-1031. 

ur Rehman, S., Sun, J., Liu, L. and Li, H., 2008. Turn Your Mobile Into the Ball: Rendering Live Football Game Using Vibration. IEEE Trans. Multimedia, Vol. 10, No. 6, p. 1022-1033.  

What is the 'empirical data' in these two papers?

In the first research paper by Fernaeus and Tholander, the empirical data was derived from video recorded material. The data was then analysed with the help of four concepts specifically discussed in tangibles user interfaces. In the second research paper by Lundström, a series of interviews, conducted with experts or other selected partners, represent the empirical data.

Can practical design work in itself be considered a 'knowledge contribution'?

Both research papers prove that their design work generated some new knowledgeable information that has contributed in form of new topic related learning. This could be, for example, in form of challenges, warnings or training characteristics.

Are there any differences in design intentions within a research project, compared to design in general?

There might be a difference in their intention of usability. I think that in a research project usability is a very important goal while design in general can also focus on other outcomes more. Design, I argue, is of course also adapting to usability, but other factors might overrule it, e.g. its appearance or art value. During the design process, researchers might also value the problems that come up since this is information that has to be considered during the research; designers in general are more solution-oriented and could more easily change to other techniques in order to achieve the goal.

Is research in tech domains such as these ever replicable? How may we account for aspects such as time/historical setting, skills of the designers, available tools, etc?

Since research projects usually take a long time to develop and later on to analyse, it is hard to replicate research in general, not just within tech domains. If you are replicating the exact variables, repeating the exact research, with the same software, same participants and same researchers, you might come close to the same result, but I still think the results will always be different. Time and setting play an important role, but they are inflexible variables and not replicable at all. The human perception also changes, so it would be a rare case if the participants would react the same way twice. Moreover, the goal of research is to go with time and support change through continuously developing and evaluating new ideas and research approaches.

Are there any important differences with design driven research compared to other research practices?

I think since design research takes attributes of the field of design in general, it could perhaps be said that design tries more to be good instead of new, tries to focus more on the user instead of the research goal. Whilst other research practices focus more on the outcome itself, the design process in research design is more important, adjusting and reflecting ideas for example in regard to their usability. 

Fernaeus, Y. and Tholander, J., 2006. Finding design qualities in a tangible programming space. Proceedings of the SIGCHI conference on Human Factors in computing systems - CHI '06.

Lundström, A., 2014. Differentiated Driving Range. Proceedings of the 6th International Conference on Automotive User Interfaces and Interactive Vehicular Applications - AutomotiveUI '14.