THE PASSIONATE SKEPTIC
Observations on the AMES
Certificate in Spoken and Written English*
[*currently published by NSW AMES, Australia]
Note 1: This paper, and another entitled Assessment in the AMES CSWE, were circulated as memoranda in the Adult Migrant Education Service teaching centres, Victoria, in early 1993. Teaching staff at a large meeting in the Myer House headquarters of AMES at that time almost universally endorsed the sentiments expressed. However the trenchant critique of CBT (competency based training) as applied to language teaching was seen as a political threat by AMES management, and my contract was not renewed.
The memorandum which follows is a personal reaction to a couple of months of experience with the AMES Certificate in Spoken and Written English (henceforth called the Competencies Curriculum). Obviously, it is very preliminary. However, I don't think that it is premature. Last year Susan Chou Allender advised us in one meeting to choose our battles. The Curriculum is a battle that I would have preferred (as a casual teacher) not to buy into, but it has become intrusive enough to demand some kind of riposte now. Many of you will disagree with all or part of the viewpoint expressed below. Besides, teachers are professional optimists, so the devil's advocate is a disquieting figure. Disagree by all means. Some purpose will have been achieved if you are driven to articulate and defend the virtues of the Competencies Curriculum as you see it.
b) Metaphors for language acquisition
An early inservice on the AMES Competencies Curriculum compared it, in a rather troubling analogy, with study for a vehicle driving test . While this was useful for making a point about skills testing, I fear that it could only reinforce misconceptions about language acquisition amongst those teachers with little real understanding of psycholinguistics (the majority). Perhaps the matter can be made more explicit by invoking another analogy.
If everyone were forbidden to play touch football until they had sat down with a rule book and passed a formal test, well the game would be killed stone dead. Learning a language is a bit like learning a (very tricky) sort of touch football, not like learning to drive a car (where you can kill someone else stone dead). What you do is get out there on the field and start running in the same general direction as all the other mugs. Sooner or later you figure out that some things are OK and that other actions will get you into a slanging match. Now and again you get a swipe at the ball. After you've been at it for a while you have enough of a clue to ask a sharp question or two at critical moments, and things suddenly come into focus.
The touch football learning process has several constituents that are central to our concern as language teachers. Learners can stay on the field even when they don't know what they are doing, and it is actually by being a continuous player -- albeit a bad one -- rather than a mere intellectual spectator, that they gradually come to intuit what the game is all about.
Depending upon particular aptitudes, the novice may be able to short-cut some of the trial and error in learning if another player can give a coherent account of what is going on when the action becomes confusing. In the long run this ability to relate to explanation (as opposed to action) may not make for a better player, but it can save time. The language classroom analogue of this is the teacher who can give a coherent (and lively) account of grammar in action when students become puzzled. Such teachers are pretty rare. It is a game, and having a good time is more important than sticking to the letter of the law. As it happens, this greatly facilitates the learning process; (apropos of which, I have never seen a script so devoid of humour, fun or romance as the pseudo-academic babble in the AMES Certificate in Spoken and Written English).
The touch football learning process is not hierarchical. Neither is the language learning process. That is, you do not have to master one skill before you go onto the next one. Learning a language is emphatically not like learning the categorical rules of mathematics or so-called hard science. The fact is that you cannot learn a language hierarchically (although it is often taught this way). It is a process of approximation on many fronts simultaneously. This is crucial. It is at the heart of public (and student) misconceptions about language teaching and learning.
A less folksy analogy for language acquisition and use may be derived from recent research into memory. A few of you may have heard an interview by Jane Figis on the ABC's Education Report (26/2/93) which discussed some awesome properties of human memory. Experiments with chess grand-masters have shown that their strategic forward planning is no better than that of a good weekend player. The actual rules of chess are trivial, so mere knowledge of these confers little advantage. The grand-master's enormous advantage stems from an ability to memorise up to 50,000 chess layouts for whole chess boards, and the move which best follows from any of these given configurations.
Now the rules of natural language are not trivial, there are thousands of them (none fully understood), but knowledge of a fairly small number of productive rules does allow the generation, in theory, of a huge number of more or less grammatical sentences. However, any language learner is aware that knowing a few grammatical rules is not sufficient for fluent language generation, any more than knowing the rules of chess leads in itself to skilful playing. What the learner needs is extended exposure to language in a vast number of collocation patterns (similar in some sense to the chess board configurations). The actual form in which collocations are stored remains unclear. The nature of this archiving is central to my own doctoral research. Whatever their pattern of storage, it is known that collocations will be best recalled when the learner's perceived need at the point of encounter is great, or the emotional temperature is higher than usual; (hence one famous U.S. French course where eggs were broken over students' heads).
A central criterion which we have to apply to the AMES Competencies Curriculum therefore is whether it is a suitable vehicle for delivering collocation patterns to which students will be receptive, and whether teachers will be able to deliver it with the kind of emotional charge that breeds life into abstract prescriptions. My answer is probably not without some determined subversion. Let me try to justify my skepticism.
c) Interpretations of the Scripture
Like any religious document, the AMES Certificate in Spoken and Written English will be interpreted to fit the hopes, illusions and political needs of many constituencies. It is has been written in Public Service doublespeak, rather pathetically it seems to me, to provide gravitas in the credibility stakes for funding. Alas, this fog of jargon will further guarantee that teachers, politicians and the public use the word-tokens of "competencies" with entirely conflicting meanings.
For students, funding bodies and employers the claim to Competency X will denote a hierarchical achievement in the mastery of English, just like learning to multiply before you can, say, calculate the hypotenuse of a triangle. Further, for these constituencies a "competency" will lay claim to competence, as in native-speaker, native-culture competence to handle a defined situation. No amount of semantic shuffling by educators is going to shift this expectation. What will happen of course is that failure to demonstrate native-like competence by competency-certified students will eventually devalue the term itself in common usage, and so undermine the status of any certificate.
Educators attempting to interpret the Competency Curriculum in daily teaching practice are well aware that the context of each language competency is defined throughout the document, that the level of competence expected at each stage is set by certain conditions, and that teaching tasks occur more or less cyclically at each stage (as language teaching tasks must in any realistic program). What this means, in terms of the preceding paragraph, is that there will be a wide disjunction between what teachers think they are doing and what the rest of the world thinks they have done.
d) Short Guide to an Incompetent Curriculum
The rest of this memo will consider what teachers might think they are doing, qua the Competencies Curriculum, and what it has to do with language acquisition. One way to begin such an analysis is to take some segments of the curriculum verbatim and attempt to interpret them.
Stage 3: Vocational English
Competency 2: Can use a range of learning strategies relevant to employment contexts
Element: i) Can formulate future career options; ii) can demonstrate a range of monitoring/self-assessment/self-directing strategies; iii) can apply learning strategies to new tasks related to employment context.
Performance Criteria: a) future career plans are realistic and achievable; b) demonstrates development of confidence in risk taking; c) indicates transfer of learning.
Competency 3/2 is a so-called knowledge competency. It is not directly about language learning at all. AMES has a perfectly valid role in going beyond language teaching, as I have argued in an unpublished newspaper letter attached to the end of this memorandum. So what can Competency 3/2 contribute to the welfare of a new immigrant to Australia? That depends. It depends on the experience of the teacher and the nous of the student counsellor, if she is involved.
What is a "learning strategy relevant to employment contexts"? Reading a couple of CES pamphlets? I think not. I'm a thirty year veteran of almost forty jobs, ranging from accounts clerk , to building labourer, to newspaper copy boy, to public servant, to shipyard worker, to salesman, to liquor van driver, to university lecturer. A bloke picks up the odd insight into "employment contexts", and one of them is that the glossy publications, air conditioned offices, vapid interviews, and the obfuscating newspeak of the public service machine have damn all to do with getting a job.
With a knockabout background I can weave competency 3/2 through with some colourful stories, give a few English collocations the emotional charge to make them memorable, get some useful teaching into the scarce classroom time. Now what about your average teacher who went from high school to teachers college and has never known the bleak despair as day follows grinding day on the production line, or the corrosive self-doubt after each failure to get past the receptionist's desk? What sort of chord will her blithe patter strike? Have another pamphlet. But back to our competency.
"Can formulate future career options.." Heck, at 47 I'm still deciding what to do when I grow up. Well, I did make up a "career plan" form for my current class -- of my own career detailed -- not for the dears to emulate (god help them), but to show the sort of information and language that could go into such a document. One column for bouquets (can dismantle a car engine, run 10km a day), one column for self-doubt (quick temper, too many interests), a column for jobs I wouldn't touch with a barge pole, another for yakka to live by until the revolution, and a final column for the holy grail, if you know what it looks like. It went down pretty well (so what if they think I'm crazy..) and elicited some brave attempts at crystal ball gazing by the students themselves. The point is that another bloodless, blank, fill-in form for career planning would have been another quickly forgotten class exercise. In classes all over Australia students are going to fill in blank "career plans" with suitable anodyne phrasing, and be assessed as "being able to formulate future career options". Joke.
Well, what are we left with? ".. demonstrates development of confidence in risk taking". Even I haven't tried bungey jumping yet. What the hell does this criterion mean? Is it really part of my job to teach whatever it is? ".. indicates transfer of learning." Really? From what to what? From a CES pamphlet to the factory gate?
My treatment of competency 3/2 has been a bit flippant. I've long held the theory that if you lock any desperate man in a room with the Melbourne telephone directory he will be able, after several years of mental torture, to emerge and declaim it as a new testament of human wisdom. Teachers down the ages have achieved something similar with all manner of abortive curriculums. Maybe it comes with the territory.
e) The Daily Survival Trick
Now I'll put my own head on the block with an extract from my teaching log: one recent three hour session, and relate it back to the Curriculum document.
1. (10 minutes) O'Neil English in Situations, p. 87: -ing Vs -ed. Intense listening to a paragraph, then precise oral response according to the structure of questions. Purpose: oral/aural training, revision of grammatical patterns, fulfilling student expectations about a language class (greatly appreciated); daily routine while latecomers arrive. Relation to Competencies Curriculum: everything and nothing. It could be arbitrarily assigned, for example, to Competency 3/3 (Can understand an oral presentation relevant to workplace contexts).
2. Distribute copy of Cobild Grammar, Unit 42, as weekly homework; (later self-corrected from an answer sheet). Purpose: self-paced learning and revision of basic grammatical structures. Meets student perceptions of language learning needs and provides a sense of progression without interfering with classroom activities. Greatly appreciated. Relation to Competencies Curriculum: arbitrary.
3. Distribution of TM's personal career planning table + a blank table for students. Explain that I'm trading my dreams for theirs. Instruction to chew it over at home and come back with something believable in a week's time. Relation to Competencies Curriculum: Meets Competency 3/2 assessment task requirement.
4. List of jobs from Challenge to Think. Task instructions: a) Choose 3 jobs you could do and 3 jobs personally unacceptable/impossible; b) justify your choices in writing; c) assemble class preferences on the whiteboard and get students with incompatible selections to argue out their differences. Relation to Competencies Curriculum: Workplace Competency 3/11 (Can write procedural texts) ?? Not really; Study 3/11 (Can write short essays) ?? Maybe at a pinch; Competency 3/4 (Can negotiate complex spoken exchanges) ?? Yeah, maybe; Competency 3/5 (Can participate in group discussions)?? Well perhaps, but it's not formal.
5. Dialogues from Crosstalk: "John Travolta" & "Doctor's Surgery". Procedure: teacher narrates a dialogue with dramatic effect and flippancy, then student pairs retell the conversation to each other, improvising from memory. Purpose: a) light relief after the serious business of jobs; b) internalisation of useful scripts and collocation patterns in a congenial and therefore memorable atmosphere. Relation to Competencies Curriculum: god knows.. Competency 3/6 (Can participate in casual conversation ) ?? Not really. The language has been preconstructed. Better be careful ... will the Thought Police book us for illegal activity?
6. Flowers, Business Vocabulary, (a weekly activity); p.10,11 : word choices for cloze on common terms in a business letter. Purpose: Build special business register vocabulary in context to allow basic functioning by students in an industry environment. Note: recheck next week. Relation to Competencies Curriculum: accidental; vocabulary acquisition essentially ignored by the Curriculum.
From the point of view of students and teacher the preceding session was highly successful. Students had a clear sense of progression from earlier lessons, and a projection to future activity. They practised some essential skills, and there was enough variety to keep the atmosphere lively without losing focus. From the point of view of the Competencies Curriculum it was a madman's breakfast, entirely undisciplined, taking micro-bites from half a dozen competencies and mixing in ingredients that hadn't been sanctified by the Curriculum at all. Finally, a record for the Myer House teaching resources file? Do they really want this? OK, if you insist, but I'm on an hourly rate. Somebody pay me another half hour per lesson to pretty the stuff up and add a post hoc rationalisation, then maybe I'll think about it.
f) What Happens If I Follow The Prescription?
Is it possible to stick to the straight and narrow? Can an entire teaching session, then an entire course be constructed from the prescriptions of the Certificate in Spoken and Written English? In a thematic sense the answers are, of course, yes and yes. In a linguistic sense the answer is that the Curriculum is so linguistically vague that anything can be snuck in, but that much of the linguistic activity in the classroom will never be meaningfully sheeted home to an element in the Curriculum. In terms of evaluation the answer is that a classroom activity can be evaluated, but asserting its transference to a general language competence is fraudulent, especially in the form of a publicly issued certificate.
A central feature of the AMES Competencies Curriculum is that it claims a small package of classroom exercises can define something called a language competency with enough completeness to predict a performance transfer to a whole category of linguistic behaviour. Thus five elements are identified for Competency 3/6, (Can participate in casual conversation), elements such as can understand structure and idiom of casual conversation, and these are to be made manifest in classroom tasks such as exchange of personal information, and according to performance criteria such as demonstrates understanding of the function of casual conversation, i.e.. social maintenance by achieving a number of interactions with co-workers/teacher. Well well, is that all? Shall we invite a discourse on the structure and idiom of casual conversation from all participating teachers?
This is not guidance to teaching; it is an intellectual wank. It can be interpreted at any level according to taste by anyone from the most inarticulate labourer to a postdoctoral researcher in discourse structure. So our neophyte teacher chooses a couple of token exercises, induces her students to utter the requisite words, then solemnly proclaims to the world that her class can participate in casual conversation and further assures her co-ordinator that "conversation has been done". No doubt her lesson was lively, and students had a few useful hours of conversation practice. But what does her evaluation really mean? What transferable skills has she really taught? The fact is that with a phenomenon like casual conversation, one of the most complex of all human activities, the essential processes going on were beyond the conscious ken of both the teacher and her students. These processes will have impacted on the participating individuals in a whole variety of ways, with outcomes over which the teacher had no control. She is a kid with a meat cleaver pretending to play brain surgeon.
The fraudulence of the claims made for knowledge & skills transfer by the AMES Competencies Curriculum become most apparent when placed alongside the sort of foci that have been found historically in language teaching classrooms:
Element Identification Specific Grammatical Rule(e.g.. More or less categorically NVN word order) defined. Generalizable to an infinite number of contexts. Sociolinguistic Rule(e.g.. title, Socio-contextually defined. FN, LN) Generalizable within understood degrees of cultural freedom. Contexts often subtle and complex. Functional expression(e.g.. Socio-contextually and apology) syntactically defined. Generalizable within understood degrees of register variation and cultural freedom. Contexts often subtle and complex. AMES Language Competency(e.g.. Behaviourally defined. Can "can participate in casual incorporate a vast number of conversation") grammatical rules, sociolinguistic rules and functional expressions; an infinite number of possible collocations. Generalization from a random instance of the behavioural type is a nonsensical proposition.
I have already made a number of references to evaluation. It is a large topic needing a paper of its own. However we may briefly note that evaluation of some kind is conducted by all parties to a learning activity: by the teacher, by the learners, and by others who participate in the outcomes of learning such as employers. Of these the teacher is most likely to proceed formally, and with a range of possible objectives. Formal tests are a way of evaluating both teaching and learning, used properly they can motivate students, and they can be diagnostic. There may be a further requirement to satisfy the administrative demands of an organisation such as the AMES with its plan to issue a Certificate to exiting students.
Organisational evaluation within AMES over the last several years has turned upon an end-of-course ASLPR assessment for speaking, listening, reading and writing. The ASLPR points are useful navigation markers, but their assignment is notional and in practice varies a good deal at the margin. My impression is that objective in-course testing has been haphazard at best in most classrooms, and that this has contributed to a sense by many students that there is no method in the madness at all. By placing emphasis on continuously evaluating what has been taught the AMES Competencies Curriculum is imposing a very useful discipline on teaching practice. Personally I give my students a formal test once a week on elements of the previous week's work. "Test" is a word they understand and value. I wouldn't have the hide to tell them I was assessing a "language competency": a euphemism they wouldn't comprehend, nor believe if they did.
Note that the Community Program is presently (February 1993) teaching the Curriculum with no systematic assessment, or any clear conception of what it implies (from my enquiries to Community teacher colleagues). This makes the Community Curriculum a rather different beast from, say, the Myer House Curriculum, although both Centres say they will issue the Certificate.
The big problem with assessment in the Certificate in Spoken and
Written English is the claims that are made about what has been assessed.
Not content with stating that a teaching objective for a specific classroom
exercise has been met, the Curriculum asserts that mastery of the sample
exercise confers competency across a broad spectrum of language behaviour.
A certificate is to be issued to this effect. As I have suggested above,
such a claim is fraudulent. It can only bring the Curriculum, the Certificate,
and its promoters into disrepute.
h) The Money Tree
Every professional in his or her lifetime is likely to see successive tides of accepted wisdom, methodology and practice. On each occasion the sea change is heralded as an advance in human knowledge and resistance to change is seen as not merely reactionary but futile. The actual virtues of each change vary -- some are indeed an advance while others are two steps backwards -- but the engine driving change is something far more fundamental. Each generation needs a cause, a vision, a vehicle for learning and for promoting personal reputations, a tool for getting to the top of the slippery career pole.
The AMES Competencies Curriculum is one of these generational vehicles, and lying down in front of the tank, one is likely to be run over. Playing reactionary is a new experience for me. In 1990, as a lecturer in linguistics at the University of the South Pacific, I was run over by another kind of tank. For about twenty years children in the twelve South Pacific micro-states have been nurtured on a program called the Tate Oral English Course plus the accompanying wooden prose of the South Pacific Readers. Tate manages to synthesise the worst aspects of Australia's old Situational English program, with stern injunctions to teachers and students not to utter any sentence but those in the books. Every language error is an indelible contamination. In my lectures I used this stuff as a model of how not to teach a second language. Well USP brought the publishing rights to this ika bono (rotten fish) from the South Pacific Commission for $100,000. It's a cash cow you see -- most schools order nothing else -- and the professor who closed the deal also happened to be Fiji's Minister for Trade. The Vice Chancellor didn't have the balls to stop him. Suffer the little children.
Well, who are marked down as winners
from the Competencies Curriculum bandwagon? Short term, pretty well anyone
who joins the parade. Teachers will have endless topics for mini-projects
and conference papers, materials writers are gifted a decade of demand
for "relevant" teaching
materials, and publishers will sell thousands of slim volumes at $19.95
each. The Fiji Trade Minister is a minor player compared to this league.
There's nothing inherently wrong with all this. The world has ever been thus.
I'm just being a little black cloud at the party because, looking around
the linguistic horizon, it seems to me that this particular picnic is going
to be a wash-out sooner than most.
i) What is to be done?
Last week my pigeonhole contained a meeting notice with the following comment: "Since the beginning of term, you have been teaching the competencies in the order you see fit. This was the best way to get to know the Certificate .... However, if we continue this way, we will end up with newly formed classes next term consisting of different competencies achieved by different students ... [A] .. solution would be to agree on a common core of competencies to teach this term.."
In other words, the aim is to achieve a lock-step system. The thrust of my comments in this memorandum has been to imply that anything which looks like a lock-step progression in the Competencies Curriculum is nonsensical. The statement that competencies will be "achieved" by students is an abuse of language. Some classroom tasks will be performed by students. The spectrum of language behaviour labelled by each so-called competency is so broad that a team of teachers could work with the same competency for a year without overlapping their lessons. If, on the other hand, they work from a common pool of detailed lesson plans (as there is some pressure to do) many will be going through the movements without passion, especially on topics beyond their natural aptitudes or interests.
As an interim solution to working with the AMES Certificate in Spoken
and Written English, I will genuflect in its direction by treating
it as a rough thematic guide to the general pattern of my lesson plans.
I won't be fazed if a lively teaching session takes micro-bites from half
a dozen notional competencies. I will continue to test my students weekly
across a rotation of skills in speaking, listening, reading, writing and
background knowledge. I will always tell them in advance the criteria by
which they are being assessed. After it's over I'll swear under my breath
and hunt for a "competency" label to stick on the test results. I will
do my damnedest to avoid meetings where we pretend to colour co-ordinate
the competency of participating in casual conversation with the
competency of using a range of learning strategies relevant to employment
contexts. I don't know what those things mean, and life is too short
to find out.