Chapter 4
User participation in curriculum materials development
Bill Sharp
Curriculum Research and Development
Technical Education Division, Western Australia
The research project
A persistent theme emerging from recent evaluation and implementation literature suggests that, if faithful replication of curriculum materials is desired, there needs to be a sense of ownership engendered in the potential users of the materials (Fullan & Pomfret, 1977). To achieve this sense of ownership, it has been suggested that potential users should participate in the curriculum development process. Harlen's (1979) proposal that "... teachers should share in the development process and not just be receivers of the product" (p.190) seems to advocate this approach. Eden and Tamir (1979) also support user involvement when they suggest that "... shared understanding between planners and teachers appears to be a prerequisite for sound curriculum implementation" (p.454).
A study of curriculum implementation in TAFE in Western Australia (Kennedy et al, 1984a), in comparing two separate curriculum interventions, claimed that the attitude of lecturing staff towards an innovation which involved them in the development process was significantly better than for an innovation that did not. One of the recommendations from this study was that the decision making aspect of the curriculum development processes should be designed so that they give opportunity for decision making to the ultimate users therefore developing a sense of ownership over the innovation by those users.
Similarly the evaluative study by the British Further Education Curriculum Review and Development Unit (1981) suggests that user involvement not only improves the quality of instruction of innovative curricula, but is also an effective form of teacher development.
Although there is considerable weight of evidence in support of the participative model, there is little documentation in the literature involving either specific management process guidelines; the views of users as participants in the development process; or the views of curriculum designers and development programme coordinators as change agents, This chapter is based on a documented case study of a curriculum materials development project (Sharp, 1987) which employed "user participation" as a strategy.
The case study focused on an examination of the project's development process as it unfolded; the designers' views on the strategy of user involvement as part of the development process; and the users' views of their involvement in the curriculum development process. The assumptions were that, by adopting user participation as a strategy of development, the quality of the curriculum materials would be enhanced and that, by involving the users in the design decision making process, problems of implementation would be largely overcome.
To assist the reader with some of the terms used in the description, the following is offered.
The vehicle for the case study on which this chapter is based was a
TAFE sponsored curriculum materials development project, The materials
were a number of Industrial Simulation Instruction Packages (ISIP) for
the fitting and machining study area (FMSA). The ISIP were to be designed
by three lecturers (the designers) drawn from the FMSA staff, who were
to act as principal designers and also as team leaders during user participation
design and evaluation workshops.
The project management rationale
The predetermined management process game plan, strategies and tactics were described by the writer prior to the commencement of the project and are portrayed in graphical flow chart form (Figure 1). A post hoc description (Table 1) documents the deviations from the predetermined game plan, strategies and tactics by showing the causal effect of the aggregation of incidents into themes and new tactics.
The predetermined game-plan (Figure 1) depicts two main strategies and associated tactics. Firstly, the strategy of user involvement in the design development process was to consist of user design workshops for those fitting and machining staff who volunteered to participate in developing the materials decided on by the three designers. The strategy was intended to give the designers a pool of accessible expert design information and to engender a sense of ownership amongst participating lecturing staff. Staff development time was to be utilized for these design workshops. They were to be spaced at strategic points in the development process and led by the designers.
The second main strategy involved the division of labour and resources between the three college-based designers. This strategy required a means of maintaining uniform interpretation of the design concepts during the development process and was to be implemented by means of strategically spaced designers' planning meetings. These meetings were to be arranged by the writer who would adopt the roles of project coordinator and evaluator. The evaluator role was to follow a style proposed by Tawney (1976), who views the evaluator as someone who takes part in formulating the product and also acts as
a data interpreter for the decision makers (data, information, decision) a group motivator and, when necessary, devil's advocate and guide a researcher who will concentrate on how the new curriculum will be eventually accepted at user level, both administratively and practically, by principals and teachers alike (p.28).
The designers' planning meetings were the main element of an ongoing formative evaluation process for the ISIP materials design and development. Apart from bringing the three designers together for the exchange of ideas and uniform decision making, the writer, in the role of evaluator, used the meetings as a means of ensuring that the designers were kept aware of the agreed design criteria, as well as the instructional design concept.
Figure 1 Pre-determined game plan for ISIP
Strategy - Project Development divided between three TAFE colleges
(1, 2 & 3)
Strategy - User involvement in design decision making process
The development process description (Table 1) was put together from base data collected for the ISIP management process and represents the documentary essence of the curriculum project's progress through all of its development phases.
The data were gathered by retrospective analysis of minuted accounts of meetings, workshops and the writer's notes and observations of development and informal contact with the designers and participating lecturing staff. The information gleaned from this data base was compared with the predetermined strategies and tactics. Incident caused deviations from the components of the predetermined game plan are recorded by way of explanatory comment in Table 1.
As an example of an incident's effect, the first documented in the study is reproduced here to help the reader appreciate the significance of subsequent incidents. This will also serve to indicate the influence that an incident might have on a predetermined game plan's strategies and tactics.
Part of the strategy for the first meetings was to be the inclusion of certain key people including a member of the college's fitting and machining senior staff, the designer domiciled at the college, the college technician and members of the college's fitting and machining lecturing staff who wished to participate. This proposed mix did not eventuate at any of the three colleges. Neither college lecturing staff, other than those directly involved in the project, nor the college technician attended.
The absence of the lecturing staff was due largely to teaching commitments and as there were plans for them to become involved during the design process, their absence was not crucial. However, this was not the case with the college technicians who were to construct the mechanical parts of the curriculum package. There had been a specific request made at the policy planning meeting in November 1984 for the technicians to attend all meetings. This was rejected by FMSA senior staff at their first meeting in December 1984.
Effectively this meant that the technicians' time was not under the control of the principal designers and this created planning problems at all the colleges. Also, as the technicians did not attend planning meetings, but received instructions on what was required of them through the designers, their sense of ownership was somewhat diminished. At one college, a combination of this diminished ownership and poor communication with the designer, strained working relationships and retarded the progress of the project.
The contention is that this incident had a significant effect on the anticipated developmental process and required compensating action on the part of all the project participants.
The development process
The specific purpose of the study was to document the management process of the development project and evaluate aspects of user participation. As can be seen in Table 1, documentation of the project commenced during the policy decision making stage. However, the management process as such did not commence until the designers were involved, therefore the documentation of observations of the planning and coordination of the management process started with the entry of the designers.
The three designers had been nominated by the fitting and machining senior staff at the meeting held in December 1984. As these nominations were not confirmed until February 1985, the designers participated neither in policy-making nor the preparation of the brief for the instructional design. Their first contact with the project, in terms of actual involvement, was at the initial planning meetings held at their respective colleges in March 1985. Prior to these meetings, the designers had access to the curriculum materials general design brief which explained the underlying concepts for the instructional design.
The initial meetings were to serve a number of purposes. The first purpose was to disseminate information on the broad design criteria for the ISIP to each college's fitting and machining staff. The second purpose was to outline the parameters within which each designer would work (ie, college 1 - transmissions; college 2 - gearing; and college 3 - bearings). Thirdly, another purpose was to explain the financial and human resource limitations for each segment of the project. Finally, by way of brainstorming sessions, it was intended to convey to the group the instructional material development possibilities inherent in the design criteria.
Table 1 The ISIP development process
| Date | Development process | Analysis of Interventions |
| July '84 | Proposal. | |
| Oct '84 | Funding granted. Management plan formulated by author in consultation with Fitting and Machining Study Area Leader and Senior Education Officer CR&D.
|
In terms of the taxonomy this is the policy decision area and took place prior to the case study. Decisions made here set the scene for the subsequent game plan, strategies and tactics. |
| Dec '84 | FMSA senior staff meeting confirms college and principal designer selection. Management plan changed to comply with FMSA senior staff, as follows:
|
These are the outcomes from negotiation with the Fitting and Machining senior staff. There were two major deviations from the management policy plan. Item 1 proved to be a formality and made no significant difference to the plan. However, item 2 proved to be an incident which did influence progress. |
| Date | Development process | Analysis of Interventions |
| Mar '85 | Initial design planning meeting at colleges 1, 2 & 3. | Strategy of ISIP development at colleges 1, 2 and 3 operationalized by tactic of these meetings. |
| Mar/ Apr |
Design briefs refined, costed and monies allocated for material and equipment purchase. Informal contact between coordinator and designers. Main concerns were materials supply and administration problems. | |
| Apr '85 | First designers' planning meeting as a group chaired by coordinator. Coordinator adopts role of formative evaluator. |
Implementation of formative evaluation tactic. |
| Apr/ May |
Coordinator plans initial user design decision making workshop. Designers prepare preliminary sketches and gather resource information in preparation for workshop. |
|
| May '85 | First user design workshop consisted of 8 hours of 15 user lecturers' staff development time. The agenda for the workshop was an introduction to the project then division into groups with the three principal designers as leaders. | User involvement strategy operationalized by tactic of using principal designers as a group leaders. |
| May/ June |
Principal designers incorporate ideas stemming from workshops into physical features of ISIP and also edit written information coming various sources. Noticeable variation in approach to design problem. Two principal designers attempt to develop both paper-based and mechanical materials in parallel as agreed. The third designer concentrates efforts on the mechanical features. | The incident noted here, with one designer's deviation from plan, develops into a theme, However effect on development strategy was neutral, therefore new tactic did not form. |
| June/ Aug |
Second designers' planning meeting consisted mainly of general exchange of ideas and progress reporting. Informal contact between designers and their college peers on design matters meets with limited success. Coordinator maintains role as formative evaluator and perceived some resentment to this function. In this time period basic mechanical design and construction methods well established. Construction at various stages of development at the three colleges. Second user design workshop planned. Two of three designers' express keen interest - third lukewarm, however participants willingly. | The effect of incidents become more evident. evident. For example, using College colleagues as sources of advice between formal workshops (although dysfunctional according to the pre-determined strategy of user involvement), was balanced by the need for student and peer evaluation trials. However, difficulties with paper-based materials and principal designers' inability to agree on format etc., develop into a theme which influences and finally changes the game plan. |
| Second user design workshop. Format as for first workshop with a report on progress of mechanical features. College 1, 2 and 3 designers as leaders. Greater emphasis was to be placed on paper-based instructional materials. | ||
| Aug/ Dec |
There were two more meetings of principal designers chaired by coordinator in this time period. The outcomes from these meetings when coupled with informal discussions, indicated that the designers were finding it difficult to reach agreement on format and uniform presentation style for paper-based materials. Individual designers tested their own versions of the paper-based materials and assessment procedures. Good progress with mechanical features construction, as evaluation of trials of prototypes by selected students and designers' colleagues show. |
|
| Date | Development process | Analysis of Interventions |
| Feb '86 | Project time extended by negotiation with Curriculum Research and Development and Study Area Manager. | |
| Feb '86 | Third designers' planning meeting. Major agenda item: the need to resolve problems associated with paper-based instructional materials. | Predetermined strategy of ISIP development at three colleges continues to be influenced by aggregation of incidents stated above. The aggregation of incidents eventually forms new tactic of single college development for the latter stages of the process. The theme becomes new strategy and has a major place in the post-hoc description of this game plan. |
| Mar '86 | Mechanical features complete and ready for final student trials. Difficulties persist on format design for paper based materials and assessment procedures, Coordinator arranges for designers to meet and work together for one day per week in an attempt to have these difficulties resolved. Some progress made. (Coordinator attended first of these meetings mainly to reaffirm initial instructional design concept of using student concrete experiences to assist concept development.) |
|
| Apr '86 | Designers' planning meeting. Decision made to move ISIP to College 3 for final student trials and preparation for user evaluation workshop. The issue of format, style and assessment not resolved. Decision made, through negotiation with designers, to rely on outcomes from next user workshop for solution to problem. Agenda for third user design workshop written and evaluation checklist designed. |
Decision taken at April '86 meeting was first formal recognition by the designers that a new intervention strategy was necessary if ISIP project was to be completed. Change theme continues to gather momentum. Decision made during informal contact and discussion between coordinator and designers. |
| May '86 | Third user design workshop. Introduction of materials by each designer. User lecturers work through ISIP as students, using checklist and their experience as instructors for qualitative evaluation. Principal designers analyse evaluation checklists and take heed of verbal comments from third user design workshop participants. Findings confirm need for uniformity of design for all paper-based materials. Preferred format and layout clearly indicated. |
This workshop becomes a formative evaluation of the ISIP by user lecturers. The outcomes from this offer an acceptable solution to the maintenance of the instructional concept stated in the design brief and add strength to the emerging theme. |
| June '86 | At the final designers' planning meeting decision made for total programme to be finished by College 3 designer. Decision was amicably reached with the three designers agreeing that this was best possible path. |
Theme formalised as new strategy with subsequent tactic changes to suit. Evidence suggests that although game plan can be stated at commencement of development process, a description should be made post-hoc. The changes made during the development process can be traced to the aggregation of incidents and a large part of this was brought about by the vigilance of the formative evaluation process. |
| July/ Aug '86 |
Student trials continue and the ISIPs refined according to results of these trials and the user evaluation workshop. Paper-based instructional materials made ready for publishing. | |
| Sept '86 | Questionnaire and interviews administered to users and principal designers respectively. | |
Implicit in the agenda for these meetings was the aim to create an awareness that the input of lecturing staff to the ISIP design process would be welcomed. Also it was hoped to ensure that the designers were given the opportunity to interpret, in the presence of the project coordinator and their college senior staff, the project's management conditions, as stated in the design brief. This, it was hoped, would help overcome any future administrative difficulties.
The designers' planning meetings followed on from these initial meetings. These meetings presented the operationalizing tactics for the project's management, the forum for the exchange of design ideas and a platform for the ongoing evaluation of the curriculum materials development. The agenda for the first meeting (Table 1) typifies the pattern of subsequent meetings. For example at this meeting the designers pooled ideas, exchanged information on materials needed for the mechanical features, agreed on standardisation for construction materials (eg, frame material size, shafting size, bearing type, etc.) and also discussed in general terms the predetermined instructional design criteria (Sharp, 1987, Appendices B and Fl). The writer's role at this meeting was as both facilitating chairman and evaluator in the Tawney (1976) mould.
Interspersed with the designers' planning meetings were the three user design workshops (Table 1). The agenda for the first two workshops went as planned with the designers acting as group facilitators and information sources for the users. However, due to difficulties associated with the integration of the paper-based portion and the mechanical features of the ISIP, the third workshop became an evaluation trial of the materials by the user lecturers (Table 1). The format for this workshop differed from the first two in that the designers were given time to introduce their portion of the completed ISIP, but their role thereafter was to be that of critical observers of the trial. To support their observations a checklist and questionnaire containing open ended questions specific to the integration of the ISIP were utilised.
Findings on participative strategy
A semi-structured interview schedule containing open-ended questions supported by probes (Sharp, 1987, Appendix E) was administered to the designers at their individual colleges in September 1986 (Table 1). The questions were framed in such a manner as to give the designers the opportunity to express their views on both the management process in general and user involvement in particular. The scheduled questions were not intended to seek comparison between the designers' responses but rather to gather illuminative information, in the manner proposed by Parlett and Hamilton (1977), on critical aspects of the development process. These interviews were recorded and the results of the recordings analysed.
When questioned on the effectiveness of the user participation design workshops, the designers were enthusiastic about the concept both as a means of getting valuable input from their peers, as well as giving them the opportunity to make their peers aware of what they were trying to achieve. They claimed that the ideas generated at the workshops made a significant difference to their approach to some of the problems associated with ISIP design.
The third user design workshop (Table 1) had become an evaluative trial of the curriculum materials by user lecturers, primarily because the designers were finding it difficult to reach decisions on the most effective format for the paper-based materials. Student trials of the mechanical features had been progressing well but it was becoming apparent to the designers that the students were not using the paper-based materials as intended and that their lecturing colleagues, who were assisting with the student trials, were treating the curriculum materials as a number of unrelated practical projects rather than a self contained integrated instructional system.
In response to interview questions on the third user design workshop, the three designers were unanimous in their praise of its value. To them this was the most successful of the three as it brought home the necessity for the creation of materials that were not only stamped with the designers' individuality, but also were acceptable as instructional materials to those people who were going to use them. The designers agreed that there were three main outcomes from this workshop.
One of the disappointments of the participative strategy was the perceived lack of success in the attempt to involve their college colleagues in the day-to- day ISIP design problems. The designers were of the view that the majority of lecturing staff were too involved in their own college workload to make any significant contribution. The designers compared this college participation with the user design workshops, claiming that with the workshops there was a sense of common purpose. In their view, this common purpose assisted the productive, creative design process. After the first workshop, the designers had developed a clear sense of ownership; it is possible that anything less than total commitment from their college peers could have been construed by them as disinterest. However, even though this aspect of participation was on the surface disappointing, as a result of the tactics employed the fitting and machining lecturers who worked at a design college were made aware of the instructional design concepts prior to actual implementation.
A Likert style attitude rating scale was administered to those lecturers who had participated in the user design workshops. Unlike the designers' interview schedule, this attitude rating scale had a single focus. That focus was to ascertain how they, as participants, felt about taking part in this particular curriculum development. The objective was achieved by posing questions on user participation as a general concept.
A frequency count was made for each item on the racing scale and the results of that count tabulated as percentages. The inference drawn from this tabulation was compared with the anticipated outcome suggested by the literature on user involvement. From this comparison, a qualitative assessment was made of the implications which user involvement had for this particular curriculum development and might have on subsequent development involving TAFE lecturers.
Thirteen of the 15 lecturers who attended the user design workshops responded to the attitude rating scale questionnaire. The items from the questionnaire and the percentage responses to the five point agreement scale are reproduced in Table 2. Given that attitude measures are difficult to validate (Hennerson et al, 1978), due to the situational factors and also, in this case, the relatively small sample, it is not intended to claim that these results can be generalised. However, the findings are an important adjunct to the development processes documentation and, as such, offer a possible path for further study of user participation as an implementation strategy.
Table 2 User response to attitude rating scale
Conclusions
The evidence suggests that the user participation model was readily adopted by the lecturers involved in the curriculum materials development. It would also seem that the designers were comfortable with the methodology, as they made favourable comments on the support which they received from their colleagues. The results from the attitude rating scale and the designers' interviews were generally in favour of user participation, thus confirming the assumption of the study.
There are clearly problems associated with the management of a participative curriculum development process and it has not been this writer' s intention to understate the difficulties. However, if a sound plan which can be adapted to suit changing circumstances is implemented, and the communication network is open in the egalitarian sense, difficulties should be minimised.
In respect to the curriculum development's management, the Intervention Taxonomy (Hall et al, 1979) was most useful, both as a structure for development planning and as a means of describing the process. In the planning stage, the Taxonomy's division of interventions into levels allows for planning to be described in unequivocal terms. During the development process, an awareness of the effects of incidents, and their aggregation into themes which might change the course of the predetermined gameplan, is of considerable value in the management of a project. The recording of incidents as events in the gathering of data for a retrospective account enables the project manager to either predict a possible outcome or take corrective action through a change of tactics if that possible outcome seems to be dysfunctional.
It is possible that the development cost of a project of this type would be lower if one person is given the sole task of design and development. However, it is the writer's considered opinion that any possible higher cost would be offset largely by a number of factors. Firstly, the quality of the curriculum materials design is enhanced not only by the creative input of participating individuals but also by the collective instructional wisdom which the participating users apply during formative evaluation. Secondly, there is the relative certainty that developed material will be implemented in ways embodying the design criteria, because of user involvement in the development process and the staff development which goes hand in hand with the process. Finally, any reference to the cost factor is probably best answered by a quotation from the Further Education Curriculum Review and Development Unit (1981) study's recommendations:
Sooner or later it must realised that attempting curriculum change on the cheap is a false economy. The price is eventually paid in terms of failure, expensive modification and nonproductive hostility. (p.8)
Although a form of recommendation is implied in the findings and conclusions of this paper, the intention has been to inform rather than define a specific methodology. It is hoped that the information gained from the case study, as applied here through the combination of management and evaluation techniques, will be of benefit to curriculum developers who might choose to use the participative model.
References
Eden, S. & Tamir, P. (1979). Curriculum implementation retrospect and prospect. In Curriculum implementation and its relationship to curriculum development in science. Science Teaching Centre, Hebrew University of Jerusalem: Israel Inc Press.
Fullan, M., & Pomfret, A. (1977). Research on curriculum and instruction implementation. Review of Educational Research, 47(2), 335-397.
Further Education Curriculum Review and Development Unit (FEU). (1981). Curriculum change: An evaluation of TEC programme development in colleges - The main findings. Stanmore Middlesex: Department of Education and Science.
Hall, G.E., Zigarmi, P. & Hord, S.M. (1979). A taxonomy of interventions: The prototype and initial testing procedures for adopting educational innovation program research and development. Paper presented at Annual Meeting of the American Educational Research Association, San Francisco, April, 1979.
Harlen, W. (1979). Implementation through involvement in development processes. In Curriculum implementation and its relationship to curriculum development in science. Science Teaching Centre, Hebrew University of Jerusalem: Israel Inc Press.
Hennerson, M.E., Morris, L.L. & Fitzgibbon, C.T. (1978). How to measure attitudes. Beverley Hills, CA.: Sage Publications.
Kennedy, K., Williamson, J. & Patterson, C. (1984a). Curriculum implementation in TAFE in Western Australia: Two case studies. Adelaide: TAFE National Centre for Research and Development.
Kennedy, K., Williamson, J. & Patterson, C. (1984b). Curriculum implementation: An annotated bibliography. Adelaide: TAFE National Centre for Research and Development.
Parlett, M. & Hamilton, D. (1977). Evaluation as illumination: A new approach to the study of Innovatory programmes. In M. Parlett and D. Hamilton (Eds.). Beyond the numbers game. London: McMillan Education.
Sharp, W.W. (1987). User involvement in a curriculum development project: Plans, strategies and tactics. Unpublished Masters project, Perth: Curtin University of Technology.
Tawney, D. (1976). Evaluation, information for decision-makers. In Curriculum evaluation today: Trends and implications. London: McMillan Education Ltd.
Cite as: Sharp, B. (1988). User participation in curriculum materials development. In C. McBeath, (Ed), Case studies in TAFE curriculum, p.27-37. Perth, WA: West Australian Social Science Education Consortium, Curtin University of Technology. http://www.clare-mcbeath.id.au/case-studies/chap4.html