Linda Way & Dr Noel Bridgeman
Western Institute of Technology at Taranaki, New Zealand
l.way@witt.ac.nz, n.bridgeman @witt.ac.nz

Way,L. & Bridgeman, N. (2004, June), SoDIS, scenarios and ‘project success’. Bulletin of Applied Computing and Information Technology Vol. 2, Issue 2. ISSN 1176-4120. Retrieved from

ABSTRACT

This paper reports on an investigation into the means required to incorporate the SoDIS methodology and software into the Western Institute of Technology at Taranaki (WITT) Information System’s project management curriculum. The first phase of the investigation consisted of examining the steps involved in both the ‘Project Success’ life cycle and the SoDIS methodology in order to determine the “best fit” for integration of the two processes. A Scenario-based simulation (SBS) is selected as the most appropriate teaching method, and the techniques of developing and implementing such simulated exercises were examined. The second phase involved research into Scenario Based Learning (SBL) methods to determine if it would support a team exercise, and aid in the development of practical skills in both project and risk management. Two trial scenarios were developed to cater for the two streams within the programme (development and support). These required the students’ to undertake part of the project management process, including use of the SoDIS process and software.

A staged approach was determined to be necessary, to effectively integrate and evaluate an appropriate simulation throughout the entire course. The first stage of the integration of both the simulation and SoDIS provided a successful learning experience for students, and a sound foundation for incremental improvement. The second stage (yet to be implemented) will involve the scenario based simulation becoming part of the course assessment rather than just an “add-on”.

Keywords

Project management, risk management, scenario based simulation, SoDIS.

1. INTRODUCTION

The Western Institute of Technology at Taranaki (WITT)'s Bacheleor of Applied Information Systems (BAppIS) degree has two compulsory courses, BIS6.201 “Management of a Project” and BIS7.301 “Project”. The 'level 600'course is designed to give students the necessary knowledge and skills to aid successful completion of the 'level 700' Capstone Project. Capstone Projects are often a student’s first interaction with Industry, with students acting in the role of a contractor. However, students cannot be expected to have the knowledge and skills of experienced Industry Contractors.

WITT’s staff had been involved in a number of workshops on the SoDIS risk management software package. This involvement led to recognition of the potential for SoDIS to be used by students as a risk assessment methodology and risk assessment tool to assist them in minimising the risks of their working with industry for the first time, plus improve their chances of successful completion of their Capstone project. It was therefore decided to investigate means of integrating SoDIS into the WITT project management process developed specifically for tertiary Information Systems student projects in Bridgeman and Bridgeman’s ‘Project Success’ (1993).

2. PROGRAM PHILOSOPHY AND OBJECTIVES

The overall goal of the BAppIS program is to prepare students for careers in information system development or support roles. The goal of the BIS6.201 project management paper is to prepare students in the BAppIS degree for the successful completion of their BIS7.301 capstone project.

Most projects undertaken by these students are completed individually rather than in a group situation, primarily due to the scope of projects that are available in New Plymouth. While there are advantages in this situation from an assessment perspective, students are potentially disadvantaged in that they are deprived of the opportunity to work in a team situation on a relatively large system development project. Introduction of a scenario-based exercise of reasonable scale in course BIS6.201 may provide students with a ‘trial run’, plus the opportunity to experience group dynamics and interaction.

3. THEORETICAL FRAMEWORK

Bridgeman and Bridgeman’s (2003) Project Life Cycle (PLC) consists of five phases: finding, defining, designing, constructing and presenting. Within these phases, a number of processes are to be followed, technical and academic outputs are to be created, and milestones are to be reached. The life cycle is designed to fit into one semester and to ‘evolve a coherent project process for the special niche of student practitioner’ (Bridgeman & Bridgeman, 2003).

Although it is a requirement of student project selection that projects may not be ‘mission critical’, there is always potential for a variety of risks to be involved. The fact that Project Management involving the development of any system is a risky business was observed and recorded by Machiavelli (1513) as follows:

‘It must be remembered that there is nothing more difficult to plan, more doubtful of success, nor more dangerous to manage than the creation of a new system. For the initiator has the animosity of all who would profit by the preservation of the old institution and merely lukewarm defenders in those who would gain by the new.

When the project involves an information system, the risks are even greater! According to a study by the National Institute of Standards and Technology (Davis, 2002), half of all U.S. software projects fail. Why? The answer is alluded to in the quote above. Many projects fail due to attitudes of the people who stand to gain or loose from the system, namely the stakeholders.

3.1 Project Success and SoDIS

Gotterbarn (1999) points out that software can be produced on time, within budget, and meet the client’s stated requirements, and still fail. He discusses the lack of effective measures for risk and stakeholder analysis in most System Development Life Cycles (SDLC). The real problem, according to Gotterbarn (1991), is the failure of these methods to identify relevant indirect stakeholders, and to consider the social, political and ethical issues that need to be underlined and addressed in order to mitigate the impacts of these issues on indirect stakeholders. The solution to this problem is the use of SoDIS.

The problem of where SoDIS best fits into a generic SDLC was also considered by Gotterbarn (1997). He recommends that it should be incorporated into the planning stages, and re-addressed throughout the life cycle. He divides software planning generally into three phases: Feasibility, Requirements, and Detailed Analysis Phase, and recommends a pre-audit of each phase. In fact, these are the names of the three SoDIS stages.

Gleason (2003) also suggests these three phases for large-scale projects, and “at least twice for smaller projects. The first thorough audit should begin once the initial project plan is in place, the last just before rollout.”

Even in student projects, some risks may go beyond the areas of time, cost and user’s requirements, and students may need to deal with social, political and ethical issues. Integration of SoDIS into the Bridgeman and Bridgeman’s (2003) PLC should assist students to better identify relevant stakeholders and ethical issues in order to mitigate risks. Only the actual placement of SoDIS within the PLC had to be determined.

3.2 Scenario-Based Learning Theory

Few would deny that reality is the ultimate learning experience. However, it is often difficult to provide students with the opportunity to undertake ‘real’ projects. It is also widely accepted that knowledge without application is worth very little (Kindley, 2002). Therefore, there is an apparent need to attempt to simulate reality in order to provide students with the opportunity to apply their knowledge. Scenario Based Learning is one method of doing this. It is important to understand how Scenario-Based Learning (SBL) differs from traditional approaches in order to identify where it is appropriate. Kindley (2002) provides a table of comparison between the traditional and scenario-based methods. The table considers such concerns as scope, focus, styles and objectives, design, subject types etc. It indicates that the traditional approach is most suited to cases where the focus is on the subject to be mastered, where answers are either right or wrong, and few choices are required. This method can cater to all learning styles, but caters least to kinesthetic learners. It best suits simple, well-structured and known topics, and is knowledge focused. By contrast, SBL is suitable for situations where the answers are not necessarily right or wrong, but the focus is on learner behavior, which can be successful or unsuccessful. It is best used for a problematic and complex topic, and in a realistic context, where performance and problem solving are the focus. This describes most student projects and caters particularly well to kinesthetic learners.

The ‘Penn State IST Learning Initiatives’ web site suggests the use of cases based on reality, that present technical, organizational, ethical and scientific problems as the most effective (scenario based learning).

Simulation-based learning is similar to scenario-based learning with the former allowing for a much larger number of decision points and success and failure paths for conveying key concepts. This paper suggests that it may be possible to think of a combination of the two types of learning, with scenario based simulations (SBS) consisting of a number of SBL occurrences, and that scenario based simulations could be used to approximate the student project process.

There are four common kinds of scenarios. Skills-based scenarios allow students to demonstrate certain abilities, skills and understandings. In problem-based scenarios, students are presented with a scenario which involves them being required to solve a problem or dilemma. The emphasis here is on the problem-solving process, not the solution. In issues-based scenarios, students investigate issues arising from a scenario, and make choices, or take a stand concerning these issues. Speculative-based scenarios provide information on which to speculate concerning behaviour, assumptions and outcomes of human actions.

A single scenario can combine many or all of these characteristics, and provide the means for students to undertake all of these learning activities.

3.3 Scenarios and Risk Management Theory

Shell is one large multinational that is a strong champion of the use of scenarios as a risk identification and assessment tool. While this use is not specifically in an information systems context, it provides further support for the choice of SBL as an appropriate technique for instruction in risk management.

According to the Shell web site, “scenarios are carefully crafted stories about the future embodying a wide variety of ideas and integrating them in a way that is communicable and useful.” The idea of incorporating scenarios as a basic tool of risk analysis is, according Kloman (2003) ‘the cornerstone of effective risk analysis’, and according to Grosse (1988), the ‘heart of identifying risk’. Scenarios actually synthesize the goals of risk management.

According to Davis (2002), there are types of risk that are neither objective nor easily comparable, that often involve other people’s fears and interests. He believes that this is where scenario plans can expand our understanding of the effects of multiple unexpected events on organizations and stakeholders. This risk is the type referred to by Machiavelli and addressed by the SoDIS methodology. According to Davis (2002), scenarios should be the primary tool for risk assessment for today’s risk manager. It seems, therefore, that the combination of SoDIS and scenario-based exercises would be very effective.

4. METHODOLOGY

4.1 Fitting SoDIS into “Project Success” SDLC

Three SoDIS audits were recommended for the first implementation of SoDIS. These phases coincided with those recommended by Gotterbarn (1997) and the SoDIS phases. These were linked with the Bridgeman and Bridgeman (2003) lifecycle to coincide with completion of Milestones three, four and five (Figure 1).

Figure 1. Position of SoDIS in SDLC

4.2 Learning Theory in Practice - Design of Scenarios

It appears that SBL is ideally suited to use in teaching PM and Risk Management knowledge and skills. However, Kindly (2002) advises that learning solutions should be designed with both approaches in mind, providing the knowledge first, and then practice (Figure 2).

Figure 2. Learning hunks from Kindly (2004)

The course was structured to impart the knowledge concerning processes and techniques through lectures. The class then completed the simulation exercises in their groups. This structure provided the opportunity for the information to be promptly actualized and reinforced.

It was decided to develop two different scenarios as the basis for the simulations. These were geared to the skill sets acquired in the support and development streams of the degree. The support stream undertook a project involving training staff of a large company in the use of SoDIS that required them to assess the user’s requirements and skill levels etc. The development stream teams were given a project involving the creation of a small database to store locations of Maori pa sites.

The situations combined the common kinds of scenario to encourage students to examine issues and make choices, demonstrate their problem-solving abilities and develop their skills in both project management and IS development. They were required to speculate concerning possible behavior, make assumptions and anticipate potential outcomes of human actions (particularly through the use of the SoDIS process).

4.3 Strategy for Integration of Scenarios with Risk Management

It was decided to begin the integration of a scenario and the use of SoDIS to assess risk within the project management process in the second semester of 2003. Due to the limited time available for preparation, the completion of the scenario–based simulated project was not incorporated into all of the assessments. However, students did use SoDIS to identify risks in their project. The results of their SoDIS analysis were required to be provided and discussed as part of their final assessment.

5. REFLECTION ON OUTCOMES

5.1 Gauging the Fit between SoDIS and the Project Success SDLC

Many of the students found that there was insufficient time available to complete more than the first SoDIS analysis. This was due to the simulation exercise being required in addition to three controlled assessments. From the experiences reported by the few students who did undertake all three analysis it was determined that the placement in Bridgeman and Bridgeman’s (2003) life cycle was not really the ‘best fit’. The three consecutive phases were too close together for many differences to be identified. It was determined that an analysis at an earlier phase would be beneficial, and that a post-implementation SoDIS review would be particularly enlightening.

5.2 Learning Practice through Scenarios

The following list of questions for evaluating the effectiveness of a learning experience is provided on ‘The Penn State IST Learning Initiatives’ website:

• Did it achieve what the author had hoped?
• Did it serve well to advance the goals of the course?
• Were there unanticipated pitfalls for the students?
• What did students say about the usefulness of the case

In this case, the answer to the first two questions is yes. There were no unanticipated pitfalls with either scenario, and most of the students claimed to have found the use of the scenario-based exercise very useful. Although some students felt it created extra work for them, they also enjoyed the opportunity to apply their knowledge in a practical manner. Some said the actual completion of the project (which time precluded in this first iteration) would have been more satisfactory. They would then have been able to assess the results of their SoDIS against the actual project outcomes. Students stated that the simulation provided the ‘next best’ learning experience to reality.

In the long term, the full scenario-based simulation exercise will be incorporated in the assessment schedule, to form basis of the assessments for whole course.

5.3 A Scenario and SoDIS - A Success!

The SoDIS process requires a Work Breakdown Structure as the basis for its risk analysis. Therefore some type of project case study, either real or fictitious, is required to carry out the process. Use of the scenario-based simulation worked well. Most students, however, only managed to complete one SoDIS analysis due to time available.

While no quantitative data was collected in this instance, the students were required to describe and evaluate their experiences with the use of the SoDIS process. All students claimed to have found the process worthwhile, several reported finding some stakeholders and associated risks that they would have overlooked, had they not followed the process.

Table 1: Results of the first scenario implementation

Using Scenario Based Simulations to integrate SoDIS into the Project Management Course enabled the students to readily identify the stakeholders. They identified potential risks and impacts that would potentially have resulted from the project. The students expressed appreciation of the SoDIS process!

It was interesting to note that the development stream (database) students were the hardest to convince to follow the processes (both project management and SoDIS). Ultimately, however, they were the most convinced of the benefits of the SoDIS process.

6. RECOMMENDATIONS FOR FUTURE IMPROVEMENTS

While the students and tutor felt the experience was a success in its first iteration, the course requires further development of many aspects. The second iteration is in progress, and will be implemented in the first semester of 2004.

Further investigation into the actual implementation of scenario-based learning is required, to ensure that the course takes full advantage of this methodology in future.

According to Kindley (2002), the outcome measures and performance behaviours of the scenario(s) should constitute the evaluative criteria for assessing performance in the scenario. Therefore it seems that the scenario should form the basis of assessment for most, or all, of the course. This would allow more time for the students to work on the scenario.

It is suggested that the use of SBS scenarios could provide the opportunity for students to choose their own team; to evaluate their choices of team members and then evaluate their team’s performance. This would provide students with the opportunity to experience work in a team environment. It is also recommended that the students be required to investigate and determine the system requirements of their “scenario” for themselves, after using appropriate information gathering techniques while interacting with their “client” - the tutor.

Most importantly, future iterations should aim to provide quantitative evidence of the value of using SoDIS analysis in identifying the inherent risks in a software development project.

7. REFERENCES

Bridgeman, N. & Bridgeman, E. (2003). Project Success. Best Way Books, New Plymouth

Gleason, D.H. (2003). Ethical Decision-Making: Critical Questions to Ask Information Ethics Boston, Ma, USA.

Grosse, V. (1988). Managing Risk: Systematic Loss Prevention for Executives, (2nd edition). Prentice Hall, New Jersey.

Kindley, R.W. (2002). Scenario-based E-Learning: A step beyond traditional E-Learning. Learning Circuits. Retrieved April 1, 2004, from http://www.learningcircuits.org/2002/may2002/kindley.html.

Kloman, F.H. (2003). Using scenarios. Risk Management , Vol 30. Retrieved from http://www.riskmanagement.com.au/RM/NEWS/2003-04-22RMREPORTSMAY.

Davis, G. (2002). Scenarios and Risk Management. The Conference Board. New York.

Gotterbarn, D. (1991). Computer ethics: Responsibility regained. National Forum, The Phi Kappa Phi Journal, 71(3).

Gotterbarn, D. (1997). Reducing software failures: addressing the ethical risks of the software development lifecycle, Australian Journal of Information Systems.

Gotterbarn, D. (1999). Promoting ethical responsibility in software development, Proceeding of the AICE Computer Ethics Conference.

Machiavelli, N. (1515). The Prince. Translated by WK Marriott. Rendered into HTML by Jon Roland of the Constitution Society. Retrieved April 1, 2004, from www.constitution.org/mac/prince00.htm.

Problem Based Learning: Writing Problem Scenarios , Penn State Learning Initiatives. Retrieved from http://students.ist.psu.edu/~dim302/scenario.htm.

Copyright © 2004 Linda Way & Noel Bridgeman
The author(s) assign to NACCQ and educational non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The author(s) also grant a non-exclusive licence to NACCQ to publish this document in full on the World Wide Web (prime sites and mirrors) and in printed form within the Bulletin of Applied Computing and Information Technology. Any other usage is prohibited without the express permission of the author(s).

 Copyright © 2004 NACCQ. All rights reserved.
Individual articles remain the property of the authors.