Case study approaches in TAFE curriculum research:
Some reflections based on practice
Surveys and Evaluative Studies
New South Wales Department of Technical and Further Education
Curriculum research in TAFE
Curriculum research in TAFE has typically been characterised by a reliance on quantitative research methods - most often survey research. Such an emphasis has served us well in providing useful answers to curriculum problems and in collecting data to be used in the design, implementation and evaluation of curriculum products.
During the years of diversification and high growth in the TAFE sector following the Australian Council of Technical and Further Education Report of 1974 (Kangan), survey methods served as the basic tool for many of the new curriculum initiatives undertaken as a result of the directions heralded in that report. A wide range of new courses were designed to meet hitherto unmet community/industry needs and demands; statewide mechanisms were established to undertake the systematic evaluation of courses, curricula and materials; and representative or multi-disciplinary task forces or working parties were commonly established to undertake curriculum planning and developmental activities. In most States and Territories, centralised pools of curriculum research and development professionals were established (or increased in size) to meet the demands of these new curriculum initiatives.
That these initiatives relied heavily on survey methods for their research base was probably due to a number of factors. The established market responsiveness of TAFE to industry/community demands and governmental policies was served by the adoption of recognised and unsophisticated market research (survey or polling) strategies. As well, during the 1970s, TAFE curriculum researchers in Australia often chose to adopt occupational analysis techniques which had been demonstrated as useful by vocational and technical training developers within the armed forces of both Canada and the United States of America.
The research skills which many of the new TAFE curriculum professionals brought with them were those gained in the physical sciences and in psychological measurement. Moreover, many of the senior decision makers in TAFE were products of their training in engineering and/or the physical sciences. It was therefore both pragmatic and purposeful to employ research methods which TAFE had the capacity to use effectively. Further, such methods were understood amongst TAFE's advisory clientele, including industry training spokespersons. These methods most often resulted in quantifiable and comparative data which could be interpreted readily to make evaluation, planning and resource allocation decisions.
The first half of the 1980s has witnessed a number of changes in the curriculum research milieu in Australian TAFE. Amongst these has been the marked trend towards improving TAFE's responsiveness to community/industry demands - a trend which has led researchers to seek faster, more economical means for curriculum problem solving (Anderson & Jones, 1986). This trend has been further pronounced in the past two years by the increasing and more prescriptive accountability requirements of State and Commonwealth governments.
A more subtle change has occurred in the backgrounds and experiences which some recent curriculum professionals have brought to TAFE. The wider recognition of TAFE as a legitimate and dynamic educational enterprise has attracted educational professionals with more specialised curriculum research qualifications and more diverse research experiences. Of significance amongst these have been skills and experiences based in the social sciences, leading to a greater capacity and interest in employing enquiry methods from, for example, the sociological and anthropological domains.
Coupled with these changes has been an increasing awareness by TAFE researchers that use of quantitative enquiry methods alone sometimes fails to solve the curriculum problem in question or, at best, overlooks important elements of the evidence that might be available, This awareness has been accentuated by the increasing complexity of curriculum problems and the political environment in TAFE, and by the recent sharper focus on equitable allocation of scarce financial resources amongst a diversifying range of competing TAFE priorities.
Case study approaches
TAFE researchers therefore have sought and experimented with methods that are able to deal with the complexities and changes that have been outlined and are realistic in a TAFE environment. Qualitative methods have been employed, mostly in conjunction with the more established survey methods. In the area of needs, or occupational, analysis, for example, Anderson and Jones (1986) have identified a range of "group process methods" and techniques that have been successfully adapted from other research fields to TAFE curriculum. Included amongst these are the Search Conference, DACUM (Developing a Curriculum), and the Delphi group method.
These methods generally have substituted the representative or random sampling platform of survey research with the judgemental selection of a small number of stakeholder representatives. In using these methods, a small group of selected experts or stakeholders is "facilitated" by a skilled group leader who abides by a set of "rules" which vary depending on the theoretical basis of the method and "value orientations" of the facilitator. In most instances, however, the rules emphasise the importance of participants' behaviour being democratic, sensitive and communicative, and the data collection outcomes are acknowledged as group outcomes generated by consensus.
In other curriculum endeavours, particularly in evaluations of curriculum products and processes, researchers have employed a range of ethnographic or interpretative research methods (see Richards, Sharp, Ashurst and Funnell in this volume). Researchers have employed a range of structured and unstructured interview techniques and to a lesser extent have adopted action research and participant observation methods in certain circumstances.
Case study research methods1 have also been used, sometimes as a sole method of enquiry, but more often as a partial method. Interestingly, many of the earlier examples of case study work in TAFE are found amongst the research and evaluation literature relating to the Participation and Equity Program (PEP), or the Transition Education Program prior to 1984. Examples of these are found in the published reports of Hailstone and Pulsford in South Australia (1981), Bartlett in Queensland (1980), Victoria's Choice and Diversity Project (1984), Rustomji in NSW (1983a; 1983b), and researchers who participated in the series of reports which formed part of the Transition Education Case Study Project in Victoria, including Smith (1982). More recently this case study research tradition has been continued by Thorn and Chapman (1987) and Chapman, Lappan and Thorn (1987) in their innovative evaluative work on the New South Wales TAFE PEP, which has included the preparation of both student and community centred case studies. In some instances the research was undertaken by researchers from outside TAFE. In some others at least, the research methods were selected by researchers who at that time were newly recruited to TAFE, often as temporary or contract researchers employed to undertake a PEP or other specific research study.
Frequently the instances of curriculum case study research undertaken in TAFE have been retrospective - looking back on a curriculum process or activity, or building a picture of the perceived impact of a curriculum product or process after the event. In addition to some examples of retrospective case studies amongst the PEP researchers noted earlier, two of the studies contained in this volume are of this nature (McBeath, Furber). Others are found in the macro-evaluations of national TAFE programmes, and are evidenced in the works of Broderick (1981), Poole and Kuhl (1983) and Jones (1983).
Researchers who have employed case study methods most often justify their selection of method in terms of their wish to gather qualitative data - data which are subjective and judgemental and which afford a richness of understanding of the complexities of the curriculum problem which does not often emerge from survey research.2 This reason would appear to reflect the developments or adaptations to research approaches being made by researchers in response to the changed TAFE environment outlined earlier. Curriculum decision making in TAFE requires a better understanding of the complex educational, economic, political and social issues that might be involved, as well as of the perceptions of different stakeholder groups, than is generally possible from quantitative methods alone.
The writer has employed case study research method on several occasions for evaluating curriculum processes and products in a national TAFE context (Jones 1983; Anderson & Jones 1986; Jones & Krzemionka 1987). In these instances, case studies were supplemented by other research methods in varying degrees. The reflections on case study research methods and practice that follow draw largely from the experience gained in undertaking those national projects.
National Core Curricula case studies
The prime method of data collection for the project An evaluation of the development and implementation of National Core Curricula in Australian TAFE (Jones 1983) was case study. Case studies of seven national core curricula3 initiatives were undertaken during 1982. Three of the case studies were retrospective in that their subjects were curriculum initiatives that already had commenced and, in two instances, completed prior to 1982. Four of the case studies were concurrent in that they commenced at the same time that the curriculum initiative commenced in 1982. These four case studies were also participative, in that they were undertaken by the researcher who was simultaneously a participant in the curriculum initiative.
The retrospective case studies were constructed following interviews with key participants in the curriculum development process (including curriculum professionals, TAFE teachers and senior teachers/administrators) and with a small sample of industry representatives who were familiar with TAFE curricula in the subject area under study. The concurrent case studies were constructed using a diary method by the researcher, who was also participating in the curriculum development process as a curriculum adviser.
The seven case studies were undertaken to enable evaluative comparisons to be made in relation to the effectiveness and efficiency of the process of curriculum design, the quality and relevance of curriculum outcomes, the level of awareness and adoption of the curriculum outcomes by teachers and the effectiveness of implementation. The purpose of the evaluation was to design a model or models for the development and implementation of future national core curricula activities.
The three retrospective case studies required a more intensive research effort to secure data which would be as comprehensive as the data obtained for the concurrent studies. The concurrent studies were served effectively by the use of a diary of events, decisions and observations made by the researcher, both during and immediately after participation in the curriculum development process. For the three retrospective studies, however, effective data collection was dependent firstly on a complex process of tracking key people and documents across all of the State and Territory TAFE Authorities. Often this tracking process led the researcher down the wrong track! Once the right track was identified, there followed the exhaustive process of gathering the required data by interview, often up to seven hours long, and document examination. Over 100 people were interviewed for the three retrospective case studies. Collection of the required data for these was undertaken over a 12 month period and consumed approximately the equivalent of half a working year. By comparison, data collection strategies for the concurrent participative case studies were time efficient. They could be predetermined to a large extent; they were directed towards single data sources and locations; and they were task- and process-oriented rather than people-oriented. Data collection for each of these four case studies required approximately the equivalent of one week.
Reflection on the different qualities of the two case study approaches also leads to comparisons of research effectiveness and validity. The involvement of the researcher as a participant in the concurrent case studies raises questions concerning the validity of the observations made by the researcher, especially since the ultimate purpose of the observations was to be evaluative. To what extent, for example, was the researcher able to distance himself from the curriculum development initiative in order to make unbiased observations about the curriculum development process? And to what extent was the researcher able to perform his dual roles as researcher and curriculum development facilitator effectively? Would not effective participation in the curriculum development process preclude, or at least diminish, an effective research effort? Moreover, what possible confounding effect did the double role of the researcher have on the other participants in the curriculum development initiative? Indeed did (or should) they know of the double role?
The writer is not able to provide definitive answers to these questions. Suffice to say, the researcher was aware of the complexities and potential conflict of roles and, from the outset, resolved to place greater priority on participation in the cooperative curriculum development act rather than the research act, lest his recognition as a participating member of the group might not be secured. It was felt that failure to gain recognition in this way would have undermined both his purposes as a curriculum developer and as a researcher. This preference for engagement in the curriculum development act, therefore, probably diminished his effectiveness in the research act. Others will have to reflect upon the questions concerning research validity which arise from the objectivity/subjectivity of the researcher's position in the research environment. The other participants were aware of the researcher's dual role and did not manifest any concern about it.
Despite concerns for the researcher's participant position in the research environment in the concurrent case studies, it is likely that the qualitative nature of the research data collected was enhanced because of this position. It is felt that the participant position enabled the researcher to form a deeper understanding of the curriculum development process under study, leading to the identification of important process factors (such as the potency of States' rights in core curriculum decision making) that might otherwise have been overlooked. Also, as a participating member the researcher was able to gain additional insights arising from the dynamics of the group.
Whilst the concurrent case studies afforded greater research power in the examination of the curriculum development process, the retrospective studies appeared to afford greater power to the examination of curriculum outcomes. There was less opportunity to reflect on outcomes in the concurrent studies quite simply because there were fewer outcomes at that stage. This research advantage, however, also appeared to derive from the opportunity for introspective reflection by participants in the one-to-one interview situation. Often, observations about unintended outcomes, or unanticipated process factors, were made by interviewees who had been able to form an impression about the whole curriculum development process in the light of the outcomes and their experiences in implementing outcomes.
The concurrent case studies were necessarily formative in nature. By comparison the retrospective studies enhanced the researcher's capacity to make summative judgements about the curriculum process and outcomes. While no measures of case study impact were recorded, either during or after the studies, it appears to the writer some years after the event that the retrospective studies had a greater impact on determining future policies and processes for national core curriculum development than did the concurrent studies. It is possible this was due to the primary audience's (TAFE administrators) greater readiness to place weight on summative evidence rather than formative evidence. If so, this would have followed, at least in part, from the methodological difference between the two types of case study. It is also thought likely that the audience had a greater familiarity with a research method which, although not a survey method, did have a closer resemblance to the traditionally used quantitative research methods based on representative sampling strategies.
Case studies of curriculum research methods
In 1983/84 a national study of TAFE curriculum research methods was undertaken. The study resulted in the report TAFE curriculum research: A review of group process methods (Anderson & Jones, 1986). The purpose of the study was again evaluative - an evaluation of the relative merits of different curriculum research methods to the TAFE context in Australia. The study was particularly concerned with examining research methods which held promise for enabling TAFE systems to make speedy and cost effective curriculum development responses to changes in industry/community requirements. Case studies of curriculum research methods were an integral part of the design for the project.
As in the discussion arising from the national core curricula project above, the case studies that were undertaken displayed qualitative differences. These differences likewise followed from the different methodologies used to study each, which followed from the different circumstances of the research method being studied. Some case studies were generative in that the case study data were generated from live (and video-recorded) examples of the research method in practice in a TAFE context. Others were projective in that they relied substantially on methodologies (found in the literature) in order to project what a particular method would be like if used in a TAFE context. The generative case studies were also in part participative.
It would again be possible to make some methodological comparisons between these two kinds of case study (the generative and projective). The comparisons however, would appear to have less efficacy than in the discussion in the previous section because of the quite different circumstances and purposes of the case studies, and because of the different levels of data accessibility for each research method that was studied. Some useful reflections however, are possible.
In the writer's experience, there is no one case study method or approach. The use of case studies should not be constrained by past or model case study practices, although intending case study practitioners of course should learn from and take advantage of the experiences and reflections of other practitioners. Most frequently, however, research problems and circumstances have unique characteristics. As is the case with social enquiry methods in general, the researcher should not be encouraged to duplicate a case study method that has been used successfully in other circumstances. Rather the researcher should be encouraged to adapt and modify successful approaches depending on the circumstances for the proposed study. For example, the references to retrospective, concurrent, participative, generative and projective kinds of case study made earlier are not text book classifications of case study methods. They are quite simply stylistic descriptors for case studies that have been conducted, and they arose from the different circumstances and issues of the research problem at hand in each case.
Nor are case studies only suitable for serving a single kind of research purpose or agenda. Case studies can and should be used in varying research circumstances and for the resolution of different kinds of curriculum problems and issues. My own experience has shown me the value of case study enquiry methods in examining (evaluating and improving curriculum development and implementation processes and products (in relation to National Core Curricula for example). They also have proven valuable in the provision of the building blocks for the development of training models in both curriculum development and research methods (as in the writer's study of curriculum research methods). The work of the PEP researchers already mentioned, and reflections on the third study outlined below, provide evidence that case studies have been integral to strategies aimed at promoting and disseminating descriptive and instructional material and information on new curriculum initiatives.
Case studies of schools/TAFE cooperative programmes
In 1985/86 a national study of schools/TAFE cooperative programmes was undertaken. This study resulted in the report Schools TAFE cooperative programs: A review of Australian practices (Jones & Krzemionka, 1987). The purposes of the study were descriptive, evaluative and promotional. It sought to identify the range and nature of Schools/TAFE cooperative programmes in Australia, to identify curriculum and accreditation issues in the design and implementation of such programmes and to promote the development of programmes that were considered to have potential for enhancing the post-Year 10 study options and pathways of young people.
Case studies of 14 cooperative programmes were conducted. These case studies formed a major part of the research effort for this project. Because most of the 14 case studies were to be undertaken under commission by on-site practitioners, the research team designed a set of case study guidelines. The guidelines were intended to achieve comparability of ease study reporting across the 14 studies. They guided writers to report their case study in terms of the history of the cooperative programme, its design process, how the programme had been accommodated within the full range of school and TAFE college offerings, a description of 33 pre-determined programme characteristics, impact of the programme, and any evaluative impressions of the programme.
The remote control management of these case studies, compared to the direct control experienced in the studies reported in the sections above, enables some useful practical observations to be made.
The case study guidelines that were designed were detailed and structured. They did not require the case study writer to pursue a specific data collection methodology. Rather they required the writer to report the study around key headings noted above. Reporting control, rather than methodological control, was sought because it was known that some of the on-site practitioners necessarily would write their case study concurrently; others retrospectively. The guidelines were effective in achieving reporting control and, in so doing, made the subsequent analysis of case study findings, to be undertaken by the project team, fairly straight forward.
Some case study writers collected data by individual interview using a checklist of questions, whereas others used group interviews with a checklist. In some cases, writers relied on document searching and analysis and then validated their findings with key participants in the programme initiative. Methodologically these approaches were quite different but, as noted in relation to the case studies of curriculum research methods referred to earlier, the different methodologies arose from the different circumstances for each. In any case they served well the research purposes of the project.
Selection of the 14 cooperative programmes to be studied was made from over 200 such programmes that had been identified by the project team in the States and Territories. This selection was made according to 17 selection criteria which took account of programme quality factors (eg, was the programme considered to be exemplary?), locational factors (eg, was the programme geographically and financially accessible to case study?) and equity factors (eg did the sample of 14 programmes selected include one or two that were designed to meet the specific needs of targeted groups, such as females?). The 14 programmes selected for case study were thus not intended to be representative of the 200-odd programmes that existed in Australia. Rather they were selected purposefully, so that critical instances (of exemplary or interesting practices) were included in the sample to facilitate the collection of the required qualitative data. In this way the sample was neither random nor representative.
The project's methodological concerns for representativeness and validity were accounted for by a population survey that was also undertaken as part of the project. The survey part of the project provided evidence which enabled the project team to draw conclusions about the scope and nature of cooperative programmes in Australia. The case study part of the project enabled the provision of qualitative data about their design and implementation processes, as well as about the complex curriculum, organisational and industrial issues involved in their effective design and delivery.
Ten of the case studies were classified as major studies; four were classified as minor. This difference arose from the project team's judgement concerning each case study writer's circumstance, availability and capacity to undertake the study to the level of rigour that would be required to accept the study as a major case study. Writers who were unable to deliver a case study because of lack of time or expertise, for example, were encouraged to submit a minor study. The four minor case studies received were consequently less informative and comprehensive as research tools, but were nevertheless still important in providing research evidence since they focused on programmes that had characteristics unique to them. The classification of case studies into major and minor groups was therefore one that arose from the circumstances of the project. Minor studies were not discouraged because they lacked the apparent rigour and design of the major studies; rather they were encouraged because it was felt they would make a useful contribution to the research purpose at hand. Indeed the project's overall research design was adapted to permit the contribution of the minor studies. Indeed the project's overall research design was adapted to permit the contribution of the minor studies. In this way the case study guidelines did not serve to constrain the case study approaches used. To achieve data that were useful to the project's purpose, the case study requirements were administered and designed flexibly.
Case studies written by commissioned writers were negotiated in four different ways. Four of the major studies were undertaken as formal commissioned research. For these, direct negotiation between the writers and the project team produced a contract which included a fee-for-service arrangement, specifications about conditions of ownership of the case study and a time-frame for delivery of the report. Two others writers were contracted less formally to undertake their work on an honorarium basis - a nominal small fee was paid. Others were undertaken following the granting of formal approval by the writer's employing agency to undertake the work as part of their normal duties. For these no fee was paid. The last kind of arrangement adopted was quite informal. For these the writers undertook to complete the work gratis, as an unendorsed service to the project.
Experience with these different modes has shown that the informal and on-duty modes are clearly least costly to the project's budget. Successful case study outcomes by these modes rely substantially upon the writer's own level of commitment to and interest in the project. Their outcomes are, however, also less reliable in terms of both timeliness and comprehensiveness. Draft case studies provided by these modes were sometimes late and in some cases deficient in content. By contrast the fee-for-service and honorarium modes are more expensive and more reliable in delivery of outcomes.
In respect of quality of case study reporting, it was found that this was dependent mainly on the case study writer's level of research experience and expertise. Researchers who were engaged to write case studies generally provided higher quality drafts than did writers who were selected because of their participant position in the programme initiative (such as teachers). This experience would therefore seem to have suggested a trade-off between costs and standard of report delivery, which is a useful lesson for those contemplating large scale research projects requiring case study work.
The writer's experiences in employing case study approaches to research curriculum issues in TAFE in Australia have varied in terms of research purposes and designs, data collection strategies, reporting styles, case study management types, as well as modes (external and internal) of case study contribution. These experiences have resulted in the belief that case study approaches significantly enhance a researcher's capacity to undertake useful and informative research in TAFE, particularly where fine-grained research evidence is needed to help in the understanding and evaluation of curriculum processes and outcomes.
An important lesson for the writer has been an appreciation of the need for flexibility in designing case study methodologies. Case studies provide a powerful means for gaining precise insights into the process or product being studied. To reap the full potential by these means, it is important to design the case study methodology to suit the circumstances of the research object and the resources and capacities of the means for the research. Case study research should not be straight-jacketed by theory or past practice. It is an imaginative and powerful research strategy that will require the researcher to employ every trick in the research text book (and some outside) to be effective in delivering useful and credible research outcomes.
Anderson, A. & Jones, N. (1986). TAFE curriculum research: A review of group process methods. Adelaide: TAFE National Centre for Research and Development.
Bartlett, V.L. (1980). Quantification in case study research. In Youth, schooling and unemployment. Collected papers of AARE Conference, Sydney, 10-19.
Broderick, J. (1981). An investigation into the curriculum development processes in TAFE in Australia: A comparative analysis. Prepared by the South Australian Department of Technical and Further Education for the Technical and Further Education Council.
Chapman, G., Lappan, K. & Thorn, W. (1987). Life after PEP: A series of case studies of students from the Participation and Equity Program. Sydney: NSW Department of TAFE. Choice and Diversity Project. (1984). In Better schooling: A national project in action. Victoria: Choice and Diversity Project.
Hailstone, D. & Pulsford, T. (1981). South-east link course evaluation. Adelaide: South Australian Department of Further Education
Jones, N. (1983). National core curricula: Development and implementation. Centre Report 1, Adelaide: TAFE National Centre for Research and Development.
Jones, N. & Krzemionka, Z. (1987). Schools/TAFE cooperative programs: A review of Australian practices. Volume V of a report prepared for the Commonwealth Schools Commission and the TAFE Council of the Commonwealth Tertiary Education Commission, Adelaide: TAFE National Centre for Research and Development.
Kangan, M. (Chair) (1974). TAFE in Australia: Report on needs in technical and further education. Canberra: Australian Government Printing Services.
Poole, K. & Kuhl, D. (1983). Evaluation of the implementation of the nationally common curricular content and practices in Plumbing, Gas fitting and Draining. Adelaide: South Australia Department of Technical and Further Education.
Rustomji, L. (1983a). Project based training courses: A report on a transition Landscape Construction Course in Sydney. Sydney: Curriculum Services Division, NSW Department of TAFE.
Rustomji, L. (1983b). Office skills training: A report on a 1981 transition education work skills course in Sydney. Sydney: Curriculum Services Division, NSW Department of TAFE.
Smith, H. (1982). Whitehorse Transition project. Case study no. 16, Transition Education Case Study Project. Melbourne: Transition Education Advisory Committee.
Thorn, W. & Chapman, G. (1987). Employer perceptions of the youth labour market. Sydney: NSW Department of TAFE.
Cite as: Jones, N. (1988). Case study approaches in TAFE curriculum research: Some reflections based on practice. In C. McBeath, (Ed), Case studies in TAFE curriculum, p.6-13. Perth, WA: West Australian Social Science Education Consortium, Curtin University of Technology. http://www.clare-mcbeath.id.au/case-studies/chap2.html