Anders Berglund, Uppsala University, Sweden
anders.Berglund@it.uu.se
 

Berglund, A. (2006, July), Phenomenography as a Way to Research Learning in Computing. Bulletin of Applied Computing and Information Technology Vol. 4, Issue 1. ISSN 1176-4120. Retrieved from

ABSTRACT

In this article I argue that results of research in computer science education that use methodologies from pedagogy can enlighten us, as computer scientists, in our teaching - by demonstrating the kind of results that can be obtained by an empirical phenomenographic research approach with regard to students learning computing. I will discuss how these results can feed back into the practice of computing education.

Keywords

Computer science education, computing education, phenomenography

1. INTRODUCTION

Computing is hard to learn for many students. This is something we know as practitioners, and that is confirmed in a literature overview made by Ben-Ari (2001). Particularly, programming at an elementary level is a subject that some students find exceedingly difficult.

A way for our community to tackle these difficulties is to learn more about what and how the students understand and learn about computing. With solid results as a basis, we are better equipped to offer our students the possibility to obtain a good foundation in computing and prepare them for lifelong learning.

A researcher who studies the learning of computing is herself or himself a part of the research project. She or he influences the project both by selecting a particular theoretical stand on learning and by deciding on how the project should be performed. Decisions on these issues, that I will call research approach in this paper, must thus be conscious and explicit. In Berglund, Daniels & Pears (2006, p. 27), we argue that “different approaches offer varying perspectives on the research questions and serve to lead the researcher down different roads”.

This article focuses on the use of a particular research approach, phenomenography (Marton & Booth, 1997), to study computing education. Phenomenography is an empirically based, qualitative approach having its roots in pedagogical research. In a phenomenographic research project, the researcher analyses and describes the different ways in which a phenomenon (or a set of phenomena) are understood, experienced, perceived or learned about from the learner’s point of view. That is, the outcome of a research project is a set of descriptions of the various ways in which the phenomena under scrutiny appear to the learner.

The rest of this article consists of three main parts: An overview of phenomenographic results in computing education, a brief introduction to phenomenography and a case study which provides an example of a project course in computing education, where phenomenography is used to inform teachers and course designers about the students’ experience of being graded.

2. PHENOMENOGRAPHIC RESULTS IN COMPUTING EDUCATION

Phenomenography has proved successful in research on higher education and has served as a theoretical framework for several studies in computing education. The approach has been used to uncover how students understand the computing concepts that are taught in a course. The rationale for such studies is to increase a teacher’s awareness of her or his students and their relationships with the objects of their learning. In this way, she or he gets insights that can serve as tools to improve her or his teaching. Booth’s pioneering work (1992) describes how students understand concepts such as recursion, within the course. Continuing in this tradition is the work by Cope (2000) about students' understanding of information systems, Berglund (2003) about students’ understanding of network protocols, and the work by Eckerdal & Thuné (2005) on students' understanding of objects and class. Recently, Doyle and Lister (in press) have explored how students understand and learn Unix. In all these cases, a limited set of ways in which the concepts are understood have been revealed.

Not only what the students learn, but also how they go about learning and what objectives they have can be explored in a phenomenographic research project. For example, in Berglund & Eckerdal (accepted for publication) we discuss what students majoring in computer science strive to learn. By tackling this question from the students’ perspective, we reveal that the students might have different aims in taking a course: academic learning, learning of project work, or development of social competence. In turn these aims can be experienced in different ways by students. A student who strives for academic learning can for example perceive this aim as being focussed on the grade, on learning something that can be used for solving assignments, on learning computer science or on learning something new.

How students approach their studies is also interesting to explore from the perspective of the students. First Booth (1992) and later Bruce et al. (2004) have described how students in introductory programming courses approach programming and how they go about learning to program. Berglund and Wiggberg (in press) have analysed how advanced students in computer science go about learning their subject area. The three studies reveal similar outcomes. Some ways of tackling the learning tasks are less advanced, focusing on “the signs” such as the syntax, examples or “standard solutions”, whiles others, more advanced, describe ways of handling learning where “the signed”, or the content of the studies, come to the fore. The variation between the studies lies in specific issues that are related to the content of the learning, the students, the settings, etc.

Phenomenographic results can be further analyzed using other interpretative frameworks in order to reach implications for teaching. In Eckerdal & Berglund (2005), we use the concepts of process - object duality (Hazzan, 2003; Sfard, 1991) to advance the discussion of object and class. According to our interpretation of the ideas introduced by Hazzan and Sfard a process understanding is the most elementary understanding, since it focuses on the process of writing code. The student who shows an object understanding not only writes code, but  also reasons with the abstract concepts that are represented by the constructs. We argue that some students do not even reach a process understanding, and thus face great difficulties in tackling programming problems.

The focus on particular phenomena in phenomenographic research normally results in surrounding concepts, or the context of these phenomena, being left unexplored. This particular focus on the phenomenon is often a strength, but may sometimes prevent the researcher from seeing relationships between different phenomena, or between the phenomena and the context in which they belong. In Berglund (2005), I have used elements of another research approach, activity theory (Engeström, 1987), to analyse how phenomenographic results concerning how students learn computer networks are related to the students’ perceptions of their learning and the learning environment. There is a complex relationship between the students learning of the subject area, their experience of their learning environment, how they go about studying and what they strive to learn.  By understanding this relationship a teacher gets a handle on the situation, which she or he can use to change her or his teaching.

Vartiainen (2005)  moves the perspective from that of the students towards that of the teachers. He discusses the ethical problems that are encountered by the teachers of an advanced project based course in computer science and finds that ethical problems appear in the relationship between the teacher and the situation. The experiential character of phenomenography, with its focus on how something is experienced, understood or perceived, makes the approach useful for tackling ethical issues.  Lister, Box, Morrison, Tenenberg, & Westbrook (2004) also take the perspective of the teacher in that they discuss data structures from the perspective of the computer science community. In this way, they argue, the different ways of understanding data structures that exist become visible, and thus possible to debate.

For reasons such as these, there is an interest in the emerging computing education research community to further explore the strength of phenomenography. This is manifested in on-going research projects in computing education in several countries, events such as the recent Phenomenography In Computing Education, (PhICER) workshops ( http://www-staff.it.uts.edu.au/~raymond/phicer/  , http://www.it.uu.se/research/group/upcerg/PhICER ), and the Researching Phenomena Symposium: Three perspectives; three cases at Auckland University of Technology, NZ, in 2006.

3. WHAT IS PHENOMENOGRAPHY?

Phenomenography is simply an attempt to capture critical differences in how we experience the world and how we learn to experience the world. Nothing more and nothing else. (Marton, 1995, p. 180)

These are the concluding words in a rather complex article debating ontological questions in phenomenography. Earlier in the same article Marton states:

The focus of phenomenography is on variation. (ibid., p. 176)

These two quotes summarise what phenomenography is about: experience and variation. They also suggest the nature of the results: a set of categories, each of which describes a certain way in which a phenomenon is understood.

Learning is, from a phenomenographic perspective, a change in a student’s understanding (or experience) of that, which she or he intends to learn. Thus, learning is shaped both by the learner and the object of her or his learning. For example, since the two network protocols TCP and UDP are different, they are different entities in the phenomenographic relationship of learning and shape this relationship in different ways. Similarly, the learner changes when she or he learns something, and becomes “new” or “different” when she or he understands something in a new way. This relationship is illustrated by arrow 1 in Figure 1 below.

Figure 1. The perspective taken in phenomenographic research

The phenomenographic researcher, who is interested in how students learn computer network, studies the relationship between the learners and the object of their learning and describes the variation in how the object of learning is experienced (or understood) in a cohort. Arrow 2 in Figure 1 shows this relationship.

The results of a phenomenographic research project do not describe individuals or individual learning. Neither does phenomenographic research attempt to quantify the findings. Instead, the results are valid for a collective; the results simply summarise the different experiences of something that can be found in a student cohort.

A phenomenographic research project is data-driven, in the sense that the results grow in the researcher’s interaction with data. The researcher avoids the use of pre-set categories or the allowing of her or his preunderstanding of the students’ experience to colour the results. Instead she or he brackets her or his preconceptions, and goes in a dialogue with the students and their ideas, first during interviews (which is the most common method for data collection in phenomenographic research projects) and later by reading and rereading the transcribed interviews.

As a result of this dialogue between researcher and data, a set of categories, often rather small, emerge. Each category then comes to describe a particular way in which the phenomenon under investigation is perceived. Together, the set of categories describes the variation in how the phenomenon is experienced. Since the categories describe different perceived aspects of the same phenomenon, they are logically related to each other, often in an inclusive structure. Were they not related, they would be categories of different phenomena.

3.1 Doing Phenomenography

The careful planning and formulation of well articulated research questions is the key to a successful project. The purpose must be defined and the researcher must select a research approach - phenomenography or another way of performing research - so that the research question and the approach go hand in hand.

The outcome of a study can never be better than the data it builds upon. The data collection - in most cases in: the interviews - must therefore be planned so that the students tell the interviewer about their various experiences of the investigated phenomena. In order to maximize the variation in the data, the interviewed students have to be selected to represent a broad set of backgrounds, interests and aims for their studies etc. Each of the interviews should also be performed with the intention of encouraging the interviewee to illuminate different aspects of the phenomena. For example, could the students be stimulated to verbalize different aspects of TCP, by changing the context in which the network protocol is discussed, from seeing it in school-book terms, talking about the TCP labs,  considering larger projects with the applications of TCP, to reasoning about exam questions (Adawi, Berglund, Booth, & Ingerman, 2001)?

A phenomenographic analysis is a slow and complex process, full of hesitation and doubt, when the researcher reads and rereads the transcribed statements of the students, now as parts of the interviews, then as parts of the growing categories. The analysis can be visualised as a kind of sorting: First the interviews are cut into separate quotes, and then these quotes are sorted into piles. The piles are not predetermined, but change and develop during the process, as a result of the researcher's interpretation of the data. When ready, the researcher has a rather small set of piles (often two to seven), where each pile describes a certain way in which the phenomenon can be experienced, where the researcher can explain in simple words what the pile means, and where there is a clear structure between the piles. This structure confirms that the piles illuminate different aspects of one phenomenon. If no structure could be found, the categories would be independent and would describe (particular aspects of) two (or more) different phenomena.

When the analysis has come to an end, is documented and written about, the phenomenographic analysis is done, and the pure phenomenographic part of the project is finished. Still, the project is not ready. The outcome should be studied and further discussed in the light of other research projects (phenomenographic or not), be made available for the communities involved (such as teachers, students, computer scientists or educationalists), and - most importantly - be brought back to education as a tool for developing and improving teaching and learning.

3.2 Applying the Results

The outcome of some of the phenomenographic projects is straightforward to apply. For example, the fact that some students do not even reach the process understanding of programming (Eckerdal & Berglund, 2005), mentioned above, points towards the need to offer some procedures to solve standard problems. With such procedures, referred to by Hazzan (2003) as canonical procedures, as a basis, a student has a foundation to develop a richer understanding both of how to write code and how to reason with more abstract entities.

Studies have shown that if a teacher knows in which ways a phenomenon can be understood by the students, and meets the students by focusing on  these particular ways of seeing a phenomenon, the outcome of the teaching, in terms of how well the students understand the phenomenon in question, is considerably improved (Marton & Tsui, 2004; Pang, 2003; Runesson, 2001). This implies, for example, that the ways of understanding the network protocol TCP, that were revealed and described in the study of Berglund (2003), mentioned above, can be used to design teaching. The teacher should lecture, create projects or labs, that encourages students to explore TCP in terms of communication between two computers; a connection over a network; or a standard for communication.

However, the most important application of phenomenographic studies, I argue, is indirect. As a phenomenographic study takes the students’ perspective, it gives a voice to the students. The students talk, through the results, to teachers, programme designers and researchers about how they understand different concepts, how they go about studying, and why they handle their studies in the way they do. A teacher who knows such results, knows more about her or his students, and is better equipped to meet their needs in her or his teaching. In short, phenomenographic results can create awareness among teachers about the students and their situation.

3.3 Why Phenomenographic Results Can Be Trusted

The discussions above have illustrated how the researcher becomes a part of the outcome of a phenomenographic research project. She or he brings her or his competence as a researcher, her or his understanding of the subject area, her or his experience of the situation etc into the project, and uses these insights as tools in her or his efforts to interpret the students’ statements. Two different researchers would thus get different results if studying the same research question. Yes, even the same researcher would (most probably) show a richer understanding if she or he repeated the same study (Booth, 1992). In this way, a phenomenographic study does not share the tradition of the natural sciences, the positivistic tradition, where the outcomes of a project should be repeatable and the researcher should act as an observer (Cohen & Manion, 1994).

For these reasons, the issues of trustworthiness and generalizability in phenomenographic research are continuously debated within the phenomenographic community (Bowden & Green, 2005; Cope, 2004; Marton, 1994; Sandberg, 1997; Säljö, 1988; Åkerlind, 2005). A key question is whether tools, that normally are attributed to positivistic research approaches, such as validity and reliability, can be used to discuss the quality of phenomenographic research, or if the non-positivistic character of phenomenography makes these concepts useless for judging the outcome of a phenomenographic research project.

Although some nice attempts to judge trustworthiness of phenomenographic results in the positivistic tradition have been made, the non-positivistic character of phenomenography dominates. However, phenomenographic results are not “subjective”, in the sense that they are only with the researcher. Firstly, qualitative research has an apparatus of its own for judging quality of research, with concepts such as credibility, transferability, dependability and confirmability (Lincoln & Guba, 1985; Patton, 2004). Secondly, the results are anchored not only in the researcher, but also in her or his study object. Thirdly, the structural relationship between the categories serves as an indication of consistency between the categories. Fourthly, the importance of a misinterpretation by the researcher is diminished by the collective structure of the phenomenographic results. These ideas can and should be used to evaluate a phenomenographic project. Particularly, I want to stress the role of the reader. She or he must judge to what degree results from one project can be transferred to the situation in which she or he works.

4. CASE STUDY: UNDERSTANDING GRADING

To put some “flesh on the bones”, the rest of this paper will present a problem concerning grading in a project course. It will show how phenomenographic research can be used, together with other forms of analysis, to learn about the effects of how a course is organized.

The case study discusses a course in advanced computer systems, often referred to as the Runestone course, which was offered in collaboration between Uppsala University, Sweden and Grand Valley State University, MI, USA from 1998 to 2005 (Daniels, 1999; Last, Almstrum, Erickson, Klein, & Daniels, 2000).

In the course advanced students developed a software system to control a motorised toy, a Brio labyrinth (see Figure 2). The students worked in teams of six, with three team members at each of the two sites. Due to the complexity of the task, the full team had to collaborate in order to achieve the aim. As a result of the discussion concerning grading in this course, a research project was set up, to investigate the different ways in which the students experience being graded. What we learnt helped us to decide what to change in the course, and, as it turned out, more importantly, what not to change.

Figure 2. A Brio labyrinth in its original form to the left, and in the motorized version to the right

The issue of grading is complex in a project-based course, since the grade of the individual is related to the performance of the others. In the Runestone course, these matters are brought to a head, as the full team never meets face to face. Furthermore half the team members never physically meet their teacher, and are thus graded by a teacher who they do not “know” and who does not recognize their faces.

The course spanned two university systems, with different rules, grading systems, grading schemes, cultures and attitudes towards grading. While the American students were awarded letter grades (A - D as passing grades, and F for failure), the Swedish students received “pass” or “fail”. The course had its own “neutral” Runestone grading system, from which the local grades were derived. The grade in the Runestone system was formed by several components: the outcome of a team’s work, the team’s working process, individual achievement, and peer evaluation.

Since assessment is often thought of as the means by which a teacher leads students into a desired way of working, the possible effects of the different grading scales became the topic of a vivid e-mail discussion among the staff when the third offering of the course was being planned (Pears, Daniels, Berglund, & Erickson, 2001). Fears were expressed that Swedish students would have less incentive to work to the maximum of their ability, because of the Swedish pass/fail grading scheme. Statements from an email discussion included:

It’s not fair to the GVSU students to put them in teams with people who have different motivations because of the way grades are recorded.

”Perception is reality”…..If one group perceive that their partners are not contributing as much as they should because of a different grading, then this is their reality and it could poison the Runestone waters.

This coarser final grading means that in some sense Swedish students have less incentive to work to the maximum of their ability, and this is further reinforced by their exposure to the USA grading system through the collaboration process.

To tackle the situation, it was decided to investigate some aspects of the grading: the teachers’ distribution of grades, the peer evaluation, the students’ experience of being graded in this course, and their attitudes towards the grading (Berglund, 2005).

The teachers’ distribution of the grades over the two countries is presented in Table 1.  A pattern can be found, in that the students in Sweden are awarded higher grades by the teachers than their American colleagues. The pattern identified among the teachers was of a different character: While the Swedish teacher valued the working process higher, the American teacher put a larger emphasis on the product in terms of the programme code.

Table 1. Average grades awarded by the teachers to the students at the different sites

The peer evaluation was implemented by giving each student 120 virtual US dollars to distribute to the members of her or his team in relation to their performance. The analysed outcome is presented in Table 2 with regard to how the students at each university on average distributed the amount, and which amount the students received from their colleagues at the two different locations.

Table 2. The outcome of the peer evaluation, analyzed by country

These results confirm those of the distribution of the grades: The Swedish students did at least as well as the Americans. Taken together, this suggests that the initial fears concerning the work of the Swedish students were - just fears. They stemmed from the expectations of the staff, rather than from facts or from the students’ expectations.

However, new questions came to the fore from these results. The grade is not a strong motivating factor as assumed in the initial discussion.  How, then, do the students experience grading in this course? And, can another strong motivating factor be discerned?

The phenomenographic study proposes an answer to those questions by showing three qualitatively different ways of how students experience being graded. These are summarised in Table 3.

Table 3. The different ways of experiencing being graded in the Runestone course

The categories are, as indicated in section 3.1, based on quotes from the full set of interviews. The examples of interview extracts given here are intended to give the reader a feeling for what the students have said and do not give a full account of the data.

I guess um, everybody has concerns about the grades. That's something that has been brought up quite a bit in the class, you know, not just in my team, but in, among everybody that, um, if things, if the code doesn't work but we've worked really hard on it and had to get through all these team issues and stuff, are we still going to fail the class because our code didn't work. Um, that's something that there is a lot of concern about and I, you know, personally I'm not concerned about it myself, but I think that it is something that ought to be addressed in the future, you know (Cat. 1)

If my class were pass/fail here, and the rest of my classes weren't graded as they are, I would not put as much effort into it. I've already noticed a lot of my classes are lacking in grades because I have to put so much effort into this project. (Cat. 2)

[...]  But, you know, I feel that, you know, if you are a group you should work hard to the best of your ability whether you are being graded on it or not, or which style of grading. (Cat 3.)

No, OK, that's certainly how they feel, they're worried they won't bother because they're going to pass anyway. But it's generally not a problem at home, I think, because everyone wants to do a good job and not be ashamed when they present it. (Cat 3.)

The worries expressed among the staff that the different grading systems would result in different incentives to work hard in the course must be seen as ungrounded in the light of these results. Certainly, this could be the case for some individuals, but if Swedish students were commonly satisfied with a lower mark, the statistical data would have shown a different picture, and the qualitative analysis would have given different results. The mechanisms that motivate the students to do a good job are complex and diversified. All claims, that might previously have been recognised by teachers and students concerning “how the students think” about grading and results are thus oversimplifications. Reality has proved richer and much more complex.

The results also show that the factors that can overshadow the grading have a social dimension. These results are confirmed in other studies in this project (Berglund, submitted for peer review), and are also discussed by Coupland (2004) who argues that “learning together” is a strong motivational factor for the students.

This case study has shown how a research project can explore aspects of the students' learning that were previously not known to us as teachers. While the statistical analyses opened for a more thorough research question, the phenomenographic study answered a why-question. The teachers in this course draw the obvious conclusion: To keep the grading as it was. The grading was not, as they initially worried, a contra-productive factor.

As a final remark, I would like to point the reader’s attention to her or his own role. It is she or he who can decide if these findings are relevant in the research setting that she or he is working, and in that case, in which ways. The results themselves discuss the Runestone course, as it was given in the year 2000. Berglund (2005) describes the study in full detail, offering the reader the means of evaluating the results for herself or himself.

REFERENCES

Adawi, T., Berglund, A., Booth, S., & Ingerman, Å. (2001). On context in phenomenographic research on understanding heat and temperature. Presented at the 9th EARLI conference, Fribourg, Switzerland.

Åkerlind, G. S. (2005). Variation and commonality in phenomenographic research methods. Higher Education Research and Development, 24, 321-334.

Ben-Ari, M. (2001). Constructivism in Computer Science Education. Journal of Computers in Mathematics and Science Teaching, 20(1), 45 - 73.

Berglund, A. (2003). What is good teaching of computer networks? In the proceedings of the 33rd Annual Frontiers in Education (FIE) conference, Boulder, Co, USA. 2SD13 - 12SD19

Berglund, A. (2005). Learning computer systems in a distributed project course: The what, why, how and where. Uppsala Dissertations from the Faculty of Science and Technology,  62. Uppsala, Sweden: Acta Universitatis Upsaliensis.

Berglund, A. (submitted for peer review). The learning of computer systems in a distributed project-based course - seeing the full picture.

Berglund, A., Daniels, M., & Pears, A. (2006). Qualitative Research Projects in Computing Education Research: An Overview. Australian Computer Science Communications, 28(5), 25 - 34.

Berglund, A., & Eckerdal, A. (accepted for publication). What do CS Students Strive to Learn? Computer Science Education.

Berglund, A., & Wiggberg, M. (in press). Students learn CS in different ways. To appear in the Proceedings of ITiCSE 2006, Innovation and Technology in Computer Science Education, Bologna, Italy.

Booth, S. (1992). Learning to program. A phenomenographic perspective. Ph D.  Acta Universitatis Gothoburgensis, Göteborg.

Bowden, J., & Green, P. (Eds.). (2005). Doing Developmental Phenomenography. Melbourne, Australia: RMIT University Press.

Bruce, C., Buckingham L., Hynd, J., McMahon, C., Roggenkamp, M., Stoodly, I. (2004). Ways of experiencing the act of learning to program:  A phenomenographic study of introductory  programming students at university. Journal of Information Technology Education, 3, 143 - 160.

Cohen, L., & Manion, L. (1994). Research methods in education. London: Routledge.

Cope, C. (2000). Educationally critical aspects of the experience of learning about the concept of an information system.  La Trobe University, Bundoora, Victoria, Australia.

Cope, C. (2004). Ensuring validity and reliability in phenomenographic research using the analytical framework of a structure of awareness. Qualitative Research Journal,, 4(2), 5 - 18.

Coupland, M. (2004). Learning with new tools. Unpublished PhD thesis.  The University of Wollongong, Wollongong. Australian Digital Theses http://adt.caul.edu.au/.

Daniels, M. (1999). Runestone, an International Student Collaboration Project (No. 11). Linköping: NyIng report.

Doyle, B. J., & Lister, R. (in press). A Preliminary Phenomenographic Study Concerning Student Experiences of Unix. In the proceedings of the 19th Annual Conference of the National Advisory Committee on Computing Qualifications, NACCQ, Wellington, New Zealand.

Eckerdal, A., & Berglund, A. (2005). What Does It Take to Learn 'Programming Thinking'? In the proceedings of the 1st International Computing Education Research (ICER) Workshop, Seattle, WA, USA. 135 -143

Eckerdal, A., & Thuné, M. (2005). Novice Java Programmers´ Conceptions of "Object" and "Class", and Variation Theory. In the proceedings of the 10th Annual SIGCSE Conference on Innovation and Technology in Computer Science Education, Lisbon, Portugal. 89 - 93.

Engeström, Y. (1987). Learning by expanding. An activity-theoretical approach to developmental research. Helsinki, Finland.

Hazzan, O. (2003). How Students Attempt to Reduce Abstraction in the Learning of Computer Science. Computer Science Education, 13(2), 95-122.

Last, M., Almstrum, V., Erickson, C., Klein, B., & Daniels, M. (2000). An International Student/Faculty Collaboration: The Runestone Project. In the proceedings of the ITiCSE 2000, Helsinki, Finland. 128 - 131

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. London: SAGE publications.

Lister, R., Box, I., Morrison, B., Tenenberg, J., & Westbrook, D. S. (2004). The dimensions of variation in the teaching of data structures. In the Proceedings of the 9th Annual ACM SIGCSE Conference. ITiCSE: Innovation and Technology in Computer Science Education, Leeds, United Kingdom. 92-96

Marton, F. (1994). Phenomenograhy. In T. Husén & T. N. Postlethwaite (Eds.), The International Encyclopedia of Education, 2nd Edition. (pp. 4424 - 4429). Oxford, UK: Pergamon Press.

Marton, F. (1995). Cognosco ergo sum. Reflections on reflections. Nordisk Pedagogik, 15(3), 165 - 180.

Marton, F., & Booth, S. (1997). Learning and awareness. Mahwah, New Jersey: Lawrence Erlbaum Associates, Inc.

Marton, F., & Tsui, A. B. M. (2004). Classroom Discourse and the Space of Learning. Mahwah, New Jersey: Lawrence Erlbaum Ass.

Pang, M. F. (2003). Two Faces of Variation: on continuity in the phenomenographic movement [1]. Scandinavian Journal of Educational Research, 47(2).

Patton, M. Q. (2004). Qualitative research & evaluation methods. London, UK: SAGE.

Pears, A., Daniels, M., Berglund, A., & Erickson, C. (2001). Student Evaluation in an International Collaborative Project Course. In the proceedings of the Symposium on Applications and the Internet Workshops, IEEE, San Diego, CA, USA. 74 -79

Runesson, U. (2001). Learning and variation. Presented at the 9th EARLI conference, Fribourg.

Sandberg, J. (1997). Are Phenomenographic results Reliable? Higher Education & Development, 16(2), 203-212.

Sfard, A. (1991). On the dual nature of mathematical conceptions: Reflections on processes and objects as different sides of the same coin. Educational Studies in Mathematics, 22, 1-36.

Säljö, R. (1988). Learning in educational settings: Methods of inquiry. In P. Ramsden (Ed.), Improving Learning: New Perspectives (pp. 32 - 48). New York, NY, USA: Nichols Publishing.

Vartiainen, T. (2005). Moral conflicts as perceived by instructors of a project course. In the proceedings of the Koli Calling, the 5th Baltic Conference on Computer Science Education, Joensuu, Finland. 25 - 32

Copyright © 2006 Anders Berglund
The author(s) assign to NACCQ and educational non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The author(s) also grant a non-exclusive licence to NACCQ to publish this document in full on the World Wide Web (prime sites and mirrors) and in printed form within the Bulletin of Applied Computing and Information Technology. Authors retain their individual intellectual property rights.

Home | Issue Index | About BACIT

 Copyright © 2006 NACCQ. Krassie Petrova, Michael Verhaart & Donald Joyce (Eds.). An Open Access Journal, DOAJ # 11764120