Praveen Sadasivan
Auckland University of Technology, New Zealand
Prasad78@aut.ac.nz

Sadasivan, P. (2005, May), Efficacy of Pair Programming in Completing Programming Tasks. Bulletin of Applied Computing and Information Technology Vol. 3, Issue 1. ISSN 1176-4120. Retrieved from

ABSTRACT

This report documents an experiment conducted at Auckland University of Technology (AUT), involving eight students (3 pairs and 2 individuals) completing a Collaborative Computing module. Their assignment was to develop a Lotus Notes prototype. All participants were given a questionnaire at the end of the task. This report also touches on the personal experience of the researcher who was also a participant in the experiment. The results of this study are mostly in line with similar studies conducted in the past. The study revealed that the quality of work done by pairs was either equal to or better than that of the individuals, despite having spent only half the amount of time on the task as the individual programmers.

Keywords

Pair Programming, extreme programming

1. INTRODUCTION

Pair programming is one of the twelve practices of Extreme programming (XP), which is one of the most popular agile software development methodologies. Agile methods came into existence in order to address the problems of traditional software development. They are intended to support early and quick production of working code by structuring the development into small release cycles and focus on continual interaction between developers and customers. In spite of this intention there has not been much scientific research conducted in any of these areas (Abrahamsson, Warsta, Siponen & Ronkainen, 2003). eXtreme Programming (XP) is an agile method that is gaining lot of popularity and is the most prominent of the methods (Schneider & Johnston, 2003). In recent years, the growth of XP has brought considerable attention to collaborative programming. Developed over a fifteen year period by Kent Beck and his colleagues, Ron Jeffries and Ward Cunningham, XP is a computer software development approach that credits much of its success to the use of pair programming by all of their programmers, regardless of experience (Williams, Kessler, Cunningham & Jeffries, 2000).  The pair programming dimension of XP requires that teams of two programmers work simultaneously on the same design, algorithm, code, or test (Nosek, 1998). Sitting shoulder to shoulder at one computer, one member of the pair is the "designated driver," and controls the keyboard and mouse while actively creating code. The "non-driver" constantly reviews the keyed data in order to identify tactical and strategic deficiencies, including erroneous syntax and logic, misspelling, and implementations that don't map to the design. After a designated period of time, the partners reverse their roles, or work with other co-workers from the same team on another piece of code. Code produced by only one partner is discarded, or reviewed collaboratively before it is integrated into the design (Williams & Kessler, 2000).

The purpose of this report is manifold. It discusses the outcome of an experiment carried out involving eight students (3 pairs and 2 individuals) who used pair programming to complete their programming tasks This report also discusses the information that was gathered using questionnaires. There is also an ethnographic component to this study, as the researcher was personally involved in the experiment, identifying the benefits and issues related to pair programming. The experiment conducted for this study is referred to as ‘AUT Experiment’ throughout this report.

2. PAIR PROGRAMMING

Software Engineering is one of the most sought after specialisations worldwide. Though world events and an economic slump in several countries including the United States has influenced the demand and opportunities for Software Engineers, it is still perceived as a versatile area of work and study. Though programming is perceived to be a difficult area, there has never been a shortage of aspirants wanting to learn programming and eventually become Software Developers/Engineers. In spite of all good intentions and diligent work of computer science educators, students find introductory computer science courses very frustrating - so frustrating that typically one-quarter of the students drop out of the classes and many others perform poorly (Williams & Kessler, 2001). Despite the best efforts of educational institutions to teach effective software engineering, few students are properly prepared to be professionals in the area (Hunt & Thomas, 2000). This is just one part of the issue. The other part is related to the industry.

Software development organizations are generally staffed to meet anticipated need.  When planned projects exceed the available work force capability, more programmers are hired - hopefully with enough lead-time to train and prepare them to be productive when the time comes.  If the new programmers are not hired with enough lead-time to enable sufficient training, they may not be of much use when their contributions are required.  Under these circumstances the need is for some sound training/educating strategies.

Considering this, academic institutions and software development organizations have always made efforts to introduce different techniques of teaching/training and help students/staff to learn and perform in their programming modules/tasks. More often than not none of the strategies have produced the desired results, which make it imperative to unearth a methodology that can address this problem. It is essential that an effective learning curve accelerator strategy be identified and this points towards pair programming which has many advantages.

To date, there are not many relevant studies conducted in New Zealand.  Studies in pair programming are still in an exploratory stage and there is no definite indication of what it is sensitive to. Hence, there is more work to be done before it can be universally accepted.

The central  research question for this study is: Is Pair programming an effective and useful methodology that can be used by programming students?

The objectives of this study are:

1. To identify  the benefits of pairing students for their programming tasks
2. To determine if pair programming can be used as a teaching approach to increase the technical and environmental knowledge of programming students
3. To determine if pair programming can be an effective learning curve accelerator.

3. METHODOLOGY

The study involved students developing a Lotus notes application as part of their Collaborative Computing module assessment, at Auckland University of Technology. The study attempted to  evaluate and ascertain  the efficacy of pairing programming students. For this purpose,  a field experiment was conducted in which students who were doing their Collaborative Computing module were given the option of developing a Lotus Notes prototype in pairs or as individuals. The formation of pairs was the choice of the students. The prototype was expected to support the process of peer review of academic articles prior to publication, by enabling easy contribution, storage, retrieval and tracking of submissions and reviews.  Overall, the prototype was  expected to have the following  functional abilities:

• Ability to submit an article for review.
• Ability to submit a review for an article.
• Ability to manage versions of articles and associated reviews.
• Ability to store an article, its versions and  its reviews.
• Ability to retrieve an article’s  versions and reviews.
• Ability to submit instructions to authors.
• Ability to read instructions and change instructions.
• Ability to assign reviewers.
• Ability to track the status of an article.

During the time the pairs were working together, they were observed and the final prototype  were examined with respect to their functionality . After the completion of the task, a questionnaire was administered to each of the students, be it  a member of a pair or an individual. [The author was also  involved as a participant]. The process flow chart is given in Figure 1.

Figure 1. Process flowchart for the experiment and analysis

The data collection was followed by analysis of the results focusing on the results of the survey, where a questionnaire was used. In order to represent the responses by students who filled in the questionnaire, simple frequencies and percentages were calculated. This is not a detailed study involving a large sample size therefore the data analysis was not very intense.

4. DISCUSSION AND FINDINGS

For making the presentation of results more effective, the information gathered during the experiment, ethnography and administration of the questionnaire is presented together. “We thoroughly liked working in pairs, in fact working in pairs is what made us complete this task. If we had some more time available, we would have developed something better”, was the comment of one of the students who worked in pairs. This obviously indicates high satisfaction levels. It also points to a high possibility of this student developing something better, as the confidence level of this student is very high.

The students who worked in pairs appeared to be more relaxed and comfortable and with a high degree of knowledge about what had to be done. There were instances when individuals were found wanting for advice and unconsciously collaborated with others to clarify things, share and acquire knowledge. The unconscious collaboration is an interesting aspect which emerged from this study. The influence of pair programming appears to be so good that it can instigate individuals to adopt it without even knowing it. There needs to be more research done on this particular area treating pair programming as not only contributing to the pairs but also having a ripple effect which can impact others.

During  the experiment, students were given the choice of either doing their task in pairs or as individuals. Out of 8 students enrolled for the module i.e. Collaborative Computing, 6 students decided to work in pairs. Other than being observed, they were also requested to fill in an anonymous questionnaire. This questionnaire, designed for data collection, was sent to 8 students, seven of whom responded promptly, thereby making a response rate of 87.5 percent. The student who did not respond was part of a pair of which the other student did respond to the questionnaire. Of the seven students who responded, five were involved in pair programming and two programmed individually. Eighty percent of the students who programmed in pairs said that they liked working in pairs, mainly because it generates more ideas and it enhances interaction and provides more knowledge. One student, who was not sure if working in pairs was a good idea, said so, as it depends on the efficiency of the partner. All the students felt that working with a partner was beneficial and it improved their performance. The pairs were chosen by the students themselves, without any external intervention. Almost all of the students who worked in pairs noted that they had spent roughly 10 hours each on the task, whereas the individual programmers indicated that they had spent about one week, which could be around 40 or more hours (see Figure 2).

Figure 2. Hours spent

As some of the past studies indicated, the time taken by the pairs is nearly half of that required by the individuals to complete a task of the same intensity. Eighty percent of the pairs felt that marks should be awarded equally among pairs. This shows a high level of satisfaction and appreciation for equal effort and contribution. The same percentage of pairs felt that there was no wastage of time between them as they mostly stuck to the task at hand. All the pairs were willing to work with their partners again, which is a good sign and further consolidates the observation that students working in pairs had a high level of satisfaction and developed some kind of personal relationship. The complete summary is given in Table 1.

Table 1. A summary of all responses

In Table 1, respondents 1 to 5 worked in pairs, 6 and 7 worked individually and 8 is the student who did not respond to the survey. Table 2 can be used to interpret subsidiary responses. PP  is used to denote “Pair Programming”.

Table 2. Subsidiary responses

The  data in the tables highlights certain aspects of pair programming, which are discussed below.

4.1 Knowledge Growth

With pair programming a pair consisting of two individuals is equally responsible for any code written. This helps to decentralise knowledge and the growth of knowledge by sharing happens rapidly. In my personal experience, I felt the same as our understanding of the software increased tremendously fast. This fact is also acknowledged by 80 percent of the pairs, who expressed that by pairing, one has more ideas and it also enhances interaction and provides more knowledge, facilitating the learning process. This is in line with the finding of a study by Nosek (1998), which found that students who programmed in pairs outperformed those who worked alone and the learning process took place very fast.

4.2 Decreased  Dependency on Mentors

The experiment revealed that once in pairs, the tendency of students to depend on their mentor slowly fades away. Though 60 percent of students approached a mentor they either felt that it did not contribute much towards learning or the contact was at a bare minimum. The remaining 40 percent did not feel that it was required. This is again an interesting observation. Even among individual programmers, only one of them felt it necessary to approach a mentor. This could be because of a difficulty in accessing them. Decreasing dependency of pairs on mentors is not a new finding. Similar kinds of observations were made by Williams (2001), in an experiment.

A study conducted by Williams & Upchurch (2001) at North Carolina State University revealed that the pair programming model is beneficial for student programmers. Initial quantitative and qualitative results of the study demonstrated that the use of pair programming in the computer science classroom enhances student learning and satisfaction and reduces the frustration common among students. Additionally, the use of pair programming was found to be relieving the burden on educators because students did not view the teaching staff as their sole form of technical information. The number of cheating cases teachers needed to deal with came down. Pair programming cuts down on cheating because pair-pressure causes the students to start working on projects earlier and to budget their time more wisely. Additionally, the students have a peer to turn to for help, and therefore do not feel helpless. Naturally, though, pair programming requires the teaching staff to deal with obvious workload imbalances between the partners that they would not have to deal with if each worked individually.

Similar results were found in a study conducted by Dr. Laurie Williams (Williams, Shukla, & Anton, 2000), to test the law which states “adding manpower to a late software project makes it later”. The study indicated that the total time spent mentoring with pairing was less (26%) than without pairing (37%). The Mean assimilation time with pairing was less (12 workdays) compared to without pairing (27 workdays).

4.3 Cost Effectiveness  (Less Programmer/Hours)

The results of this study showed that the hours spent by pairs was less than half of that spent by individual programmers. The cumulative hours taken by pairs were roughly 50 percent of what was taken by the individuals. Also almost all pairs felt that there was no time loss what so ever. This again is one of the positives of pair programming. One school of thought says that pair programming can lead to ‘buddy talks’ and wastage of time. This need not be true if the teams are focused and goal oriented. In a study conducted by Cockburn & Williams (2001) using interviews and controlled experiments, the authors investigated the costs and benefits of pair programming. The purpose of this study was to re-examine the results of some earlier studies and to explain why pair programming is beneficial. They found that for a development-time cost of about 15%, pair programming improves design quality, reduces defects, reduces staffing risk, enhances technical skills, improves team communications and is considered more enjoyable at statistically significant levels. A look at the final output indicated that all the pairs had done good quality work in spite of devoting a smaller number of hours. The quality of output was either higher or similar to that of individuals who had devoted 50 percent more time.

The number of hours taken by programmers to complete their task has been a bit inconsistent throughout different studies. As in a study conducted by Nosek (1998), where 15 full-time experienced programmers were randomly assigned to either work as part of a two-member team or to work by themselves on a programming problem for 45 minutes. Teams were found to significantly outperform individual programmers in terms of functionality and readability, to report greater satisfaction with the problem-solving process, and to have greater confidence in their solutions. However, it should be noted that pair programming was found to take more total programmer time than traditional solo programming, although the elapsed time was less. Pairs required an average of 60 programmer-minutes to complete programming assignments compared to the 42 programmer-minutes used by solo programmers. It should not be concluded, however, that pair programming requires more time. Nosek did not include time spent debugging in his analysis and this debugging may be advanced in pairs. This point is particularly salient when code quality is considered; Nosek found that code produced by individuals is more error prone than code created by pairs.

The findings of the study indicate that there was no wastage of time due to wasteful talking etc. This goes well with the results of a study conducted by Dr. Laurie Williams (Williams, Shukla, & Anton, 2000), to test the law which states “adding manpower to a late software project makes it later”. This study revealed that pair programming reduced intercommunication time through informal, on-the-spot discussions.

4.4 Higher Confidence Levels and Satisfaction

The AUT Experiment revealed that all the students involved in pair programming were willing to work again in pairs. This indicates that their satisfaction levels were very high. An experimental study conducted by Williams and Kessler at the University of Utah provides us with further empirical evidence of the effectiveness of pair programming (Williams & Kessler, 2000). In this study, 41 upper level students enrolled in a course on web design were randomly assigned to complete four programming projects either independently or in pairs. During each programming cycle, the 13 solo programmers completed one program, while the 14 pairs completed two. In addition to producing more bug free code, pair programming appears to enhance the programmers' enjoyment and confidence. Students practicing collaborative programming, as well as professional pair programmers were anonymously surveyed. Over 90% reported enjoying their jobs more when working in pairs, and 95% reported feeling more confident in their solutions (Williams, 2000). This shows that the confidence, satisfaction and overall performance are high.

A study conducted at the University of Wales gave indications of a very similar kind. The study revealed that students in general appreciate pair programming and believe that it helps them achieve good solutions. Even students who were not very self-confident enjoyed and appreciated pair programming. The students who rated themselves as exceptionally good in programming did not enjoy pair programming but in all cases their final product and overall performance was appreciably better (Thomas, Ratcliffe & Robertson, 2003).

An experiment on students of an introductory programming course indicated that with success owing to pair programming, students who did not have intentions to major in computer science related fields had also taken up a computer science related course (McDowell, Werner, Bullock & Fernald, 2003). This indicates the increase in their level of confidence. Pair programming is found to be helpful in retention of more students in introductory computer science streams and it is definitely not a deterrent to student performance (Nagappan, Williams, Ferzli, Wiebe, Yang, Miller & Balik, 2003).

From my observation, when in pairs, students were found to be more joyful and brain storm more effectively. Individual programmers were often found gloomy and looking for someone they could talk to. “Wow, I think I know what we can do to get this working. Can we add a new user named Anonymous with read access and see” was one of the comments by a programmer to his partner when they were stuck with some access issues while developing the prototype. This indicates the amount of enjoyment they were having and the seriousness of participation.

4.5 “Pair” Pressure

In the AUT Experiment, nearly 80 percent of the pairs felt that they were ensuring that they did not let their partners down. This is the result of pair pressure. A detailed study was performed at the University of Utah in 1999 using senior software engineering students, which revealed that pair programmers were more consistent in completing assignments on time.  The programmers felt they worked harder because they didn’t want to let their partner down.  This is known as “pair pressure”, and appears to be an important aspect of successful pairing.  Not all the pairings produced superior results when compared to individuals, but the lower-performing pairs were also below the norm when given individual coding assignments.

Initially, the pairs required about the same percentage increase in man-hours to complete an assignment as reported in the Nosek experiment - around sixty percent.  However, in later assignments, this number dropped to as low as 15% additional cumulative hours.  When the code was exercised using automated test cases, codes produced by pairs passed 15% more tests.  The pairs also produced a lower Kilo Lines of Code (KLOC) count, indicating superior software design (Cockburn & Williams, 2001).

In the AUT Experiment, quality of pairing did was not an issue. The option of programmers choosing their own pairs worked well. This could be one of the areas for future research. The success of pairs in the AUT Experiment could be because of chance or a rare coincidence. Further research will help explore the issues with team dynamics much more deeply.

5. CONCLUSIONS

This report highlights the fact that pair programming is a valuable practice that can be adopted in educational institutions as well as software development organizations. The inherent virtues of pair programming make it applicable to all stages of software development. The AUT Experiment and survey along with personal observation confirm that this is a good methodology that can be used for teaching students as well as for making students do their programming tasks. The AUT Experiment proves again that pair programming is highly effective in terms of completing programming tasks. Therefore, it is a useful methodology that can be used by programming students.

Almost all studies conducted so far have reported that pair programming is highly efficient and contributes to efficient and speedy knowledge transfer. It produces fewer, defect-free lines of code, consumes less time in overall development as re-work will be negligible or none and the satisfaction levels of pairs are observed to be high.

The study revealed that pair programming is more productive, gives a high degree of satisfaction, is very cost effective and also the products developed are of good quality. Though the pairing was done as per the choice of the students, the results were appreciably better. This indicates that though pairing is an intricate issue, pairs can be really effective if the students are given the choice of selecting their own teams. This is a good option to consider in future. This could be a bit difficult if the students are new to the course and to each other. However, in light of the findings of this study, it can be concluded that all the lauded advantages of pair programming are valid. It is indeed a performance booster.

ACKNOWLEDGEMENTS

The author wishes to thank Catriona Carruthers for the thorough editing of the article in terms of style, language and grammar -- which greatly improved the overall presentation and readability.The author would also like to thank Dave Parry, Tony Clear, Diana Kassabova and all the students who were involved in the study, for their help and support.

REFERENCES

Abrahamsson, P., Warsta, J., Siponen, M. T., & Ronkainen, J.  (2003). New directions on agile methods: a comparative analysis, Proceedings of the 24th International Conference on Software Engineering, 2003, Portland, Oregon, p. 244-254.

Allen, E., Cartwright, R., & Reis, C. (2003). Production programming in the classroom, Proceedings of the 34th technical symposium on computer science education, 2003, Reno, Navada, USA, p. 89-93.

Anderson, A., Beattie, R., Beck, K. et al. (1998). Chrysler Goes to Extremes, Distributed Computing, p. 24-28.

Biederman, B., W., & Ward, P. (1997). Organising Genius: The secrets of creative collaboration, Addison-wesley Publishing Company, Inc.

Blotner, J., A. (2002). Agile techniques to avoid firegighting at a start-up, OOPSLA2002 Conference on Object-Oriented Programming Systems Languages and Applications, Seattle, Washington.

Cockburn, A., & Williams, L., A. (2001). The Costs and Benefits of Pair Programming, in Extreme Programming Examined, Addison Wesley-Longman.

Forum. (2001). Pair Programming Successes and Failures. Retrieved September 2, 2004 from World Wide Web: http://discuss.fogcreek.com/joelonsoftware/default.asp?cmd=show&ixPost= 2058 (The Joel on Software Forum).

Fullan, M. (1993). Change forces: Probing the depths of educational reform. Bristol, PA: The Falmer Press.

Hedin, G., Bendix, L., & Magnusson, B. (2003). Introducing software engineering by means of extreme programming, Proceedings of the 24th International Conference on Software Engineering, Portland, Oregon, p. 586-593.

Humphrey, W., S. (1995). A Discipline for Software Engineering. Reading, Massachusetts: Addison Wesley Longman, Inc, p. 114-123.

Hunt, A., & Thomas, D. (2000). The pragmatic programmer, Addison-Wesley.

McCaleb, S., P. (1994). Building communities of learners: A collaboration among teachers, students, families and community. New York: St Martin's Press.11.

McDowell, C., Werner, L., Bullock, H., & Fernald, J. (2002) The Effects of Pair programming on Performance in an Introductory Programming Course,  Presented at The 33rd ACM Technical Symposium on Computer Science Education, February 7-March 3, 2002.

McDowell, C., Werner, L., Bullock, H., & Fernald, J. (2003) The impact of pair programming on student performance, perception and persistence,  Proceedings of the 24th International Conference on Software Engineering, 2003, Portland, Oregon, p. 602-607.

Muller, M., M., & Tichy, W., F. (2001). Case study: extreme programming in a university environment, Proceedings of the 23rd International Conference on Software Engineering, Toronto, Ontario, Canada, p. 537-544.

Nagappan, N., Williams, L., A., Ferzli, M., Wiebe, E., Yang, K., Miller, C., & Balik, S. (2003). Improving the CS1 Experience with Pair Programming, Proceedings of SIGCSE 2003, February 19-22, 2003, p. 359-362.

Nawrocki, J., & Wojciechowski, A. (2001). Experimental Evaluation of Pair Programming, Proceedings of the 12th European Software Control and Metrics Conference, ESCOM 2001, 2-4 April 2001, London, p. 269-276.

Nosek, J., T. (1998). The Case for Collaborative Programming, Communications of the ACM, p. 105-108.

Schneider, J., & Johnston, L. (2003). eXtreme Programming at universities: an educational perspective, Proceedings of the 24th International Conference on Software Engineering, Portland, Oregon, p. 594-599.

Smith, S., & Stoecklin, S. (2001). What we can learn from extreme programming, The journal of computing in small colleges, 17 (2), 144-151.

Thomas, L., Ratcliffe, M., & Robertson, A. (2003). Code warriors and code-a-phobes: a study in attitude and pair programming, Proceedings of the 34th technical symposium on computer science education, Reno, Navada, USA, p. 363-367.

Williams, L, A. (2000a) Pair Programming Questionnaire, Retrieved on May 26, 2003 from http://collaboration.csc.ncsu.edu/questionnaire/questionnaire.htm.

Williams, L., A. (2000b). The Collaborative Software Process PhD Dissertation, in Department of Computer Science. Salt Lake City, UT: University of Utah.

Williams, L., A. (2001). Integrating Pair Programming into a Software Development Process. Retrieved September 2, 2004 from World Wide Web: http://ieeexplore.ieee.org/xpl/abs_free.jsp?arNumber=913816

Williams, L., A., & Kessler, R., R. (2000a). All I Really Need to Know about Pair Programming I Learned In Kindergarten, Communications of the ACM, May 2000, p. 108-114.

Williams, L., A., & Kessler, R.,R. (2000b). Experimenting with Industry's "Pair programming" Model in the Computer Science Classroom, Journal on Software Engineering Education.

Williams, L., A., & Kessler, R., R. (2000c). The Effects of `Pair-Pressure' and `Pair-Learning' on Software Engineering Education, Proceedings of Thirteenth Conference on Software Engineering Education and Training , p. 59-65.

Williams, L., A., & Kessler, R., R. (2002). Pair Programming Illuminated, Addison Wesley Professional.

Williams, L., A., Kessler, R., R., Cunningham, W., & Jeffries, R. (2000). Strengthening the Case for Pair programming, IEEE Software , July/Aug. 2000.

Williams, L., A., Shukla, A., & Anton, A., I. (2002). Pair Programming and the Factors Affecting Brooks’ Law, North Carolina State University.

Williams, L., A., & Upchurch, R., L. (2000). In Support of Student Pair programming retrieved on May 25, 2003 from http://pairprogramming.com/WilliamsUpchurch.pdf.

Copyright © 2005 Praveen Sadasivan
The author(s) assign to NACCQ and educational non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The author(s) also grant a non-exclusive licence to NACCQ to publish this document in full on the World Wide Web (prime sites and mirrors) and in printed form within the Bulletin of Applied Computing and Information Technology. Any other usage is prohibited without the express permission of the author(s).

 Copyright © 2005 NACCQ, Krassie Petrova, Michael Verhaart & Christo Potgieter. All rights reserved.
Individual articles remain the property of the authors.