One of the things that I’ve been involved with at work over the past few years has been course book selection. As part of the process, research was undertaken by myself and a few colleagues about how we might go about selecting a book in a considered manner. Given that I often speak at conferences about dealing with the problems of course books, this seems like a good topic for my comeback post after two weeks away! What follows is a suggested framework, based on everything I’ve read up on about this subject, for course book selection that I think is worth sharing with you all.
Many new ELT course books come out every year, making difficult for those evaluating each one to choose the right supplementary book for each level of study. There is a body of literature indicating that selection methods in our profession have often been unsatisfactory; therefore there is a need for a checklist as a way of choosing course books.
One all too frequent method is for those in a position of authority to obtain copies of a selection of the latest publications, which are then briefly looked over or flicked through by some of the more experienced teachers in the institution, after which a series of books is chosen on the basis of the subjective judgments and/or first impressions of those teachers.
Unfortunately, frameworks of evaluation have had little real influence on course book selection in the past. This happens probably because teachers are either; A) unaware of the existence of checklists; B) don’t have access to them; C) do not want to make the effort to use them, or D) are put off by their length and apparently complicated nature. Nevertheless, the application of a framework of appropriate selection criteria fro potential course books should be considered an effective way of ensuring that the needs and wants of learners are given careful consideration when choosing supplementary books.
In terms of evaluating course books, criteria should be what Peacock (1997) describes as ‘emphatically local‘. By local, Peacock doesn’t mean a framework that is country specific; rather it should be as specific as referring only to the particular institution using it. Peacock’s aim when developing a framework was to create one adaptable enough for use worldwide in a variety of contexts; he nevertheless recognized the need for ‘localization’ and attempted to overcome the issue of using a flexible, multi-purpose framework by including a scoring table with weightings that can be varied by users according to any given situation. This notion of streamlining a flexible checklist is evident in the list we have produced.
Preparing a framework for evaluation
The goal of this framework* is to allow as thorough an evaluation as possible to be made in the time normally allocated for course book assessment (usually not long enough!), enabling a comparative and objective evaluation to be made before a book is chosen for use in class.
The framework here should be considered both an adaptation and an expansion on the ideas of the authors mentioned, with the addition of new items inspired by their research and ideas. I hope that the use of this framework will result in you finding the most suitable course book for your learners.
Sheldon (1988) made a good point when stating that the aim of any checklist should be to make book selection a “more coherent, thoughtful enterprise than it is at present”. A framework has to be as short and simple as possible to encourage its use. Scoring tables, when included, are beneficial in that they enable a comparison to be more easily made between course books, as institutions normally evaluate more than one book at a time: it is easier to compare scores than long lists of comments. Additionally, it is much quicker to assign a score than to write comments, increasing the likelihood of the framework being of real practical use.
The first item in the framework is adapted from the notion of the ‘flick test’ recommended by Matthews (1985). This should basically be viewed as a ‘first impression counts’ assessment of the book. If the book fails this first element of the criteria, Matthews suggests, the evaluator may not feel the need not continue, though this is not stated in the checklist, as stopping there is a personal decision for the evaluator.
Peacock’s list (1997) is particularly relevant to items 7, 8, and 9 which are specifically included to alert decision makers as to the necessity of using culturally appropriate materials in classrooms, and to try to exclude coursebooks which may offend learners.
Using the framework
Before using this list, it is necessary that evaluators add the weightings appropriate for your local situation to the scoring table: place specific importance on those items which seem particularly pertinent. This may be decided collaboratively by those who will be in the position to choose the supplementary book. Without weighting, the 58-item list would offer a maximum score of 116.
Objective weighting of the framework’s criteria is a crucial part of the evaluation process as it is the only way to ensure that the final score for a coursebook corresponds as closely as possible to the particular needs of the learners who will use the book. Selectors first have to decide what weightings to use. The objectivity of this process can be increased by using as many and as experienced evaluators as possible, and by taking sufficient time to reach a decision.
Ideally, suggests Peacock (1997) four or more experienced evaluators should take at least two meetings to reach a decision:
‘Weightings will influence which book is chosen: in Korea item 25 on the checklist was given a weighting factor of 8, ensuring that coursebooks which had plenty of sections for listening and speaking skills were more likely to be selected. Another (hypothetical) example is giving item 41 a weighting factor of 10; this would make it much more likely that coursebooks containing plenty of learner guidance were chosen.’ Peackock (1997)
It is important to remember that weightings on scores may be altered according to what you prioritize; framework items may be altered, removed or even others added; course books may additionally be evaluated by other means, and results compared with those of this framework of criteria; Williams (1981) suggested that the chosen books may even be piloted in class and the results compared with those predicted by an evaluation using such criteria.
Course book evaluation framework criteria
This framework of criteria has been put together specifically to evaluate supplementary grammar books used for teaching English as a Foreign Language to basic to upper intermediate level learners. Therefore, I expect you might need to make at least tweaks here and there if you plan to use these criteria.
This framework, while somewhat adapted to a specific requirement, is designed to produce a score for any book you choose to evaluate. While the scores will not necessarily provide a definitive evaluation of any given book, they will offer you a quick and adequately considered comparison in the event of several books being evaluated at the same time.
The criteria may be rated numerically, on a scale from 0 to 2 for instance, in the blank space before each one, as follows:
2 = Good
1 = Satisfactory
0 = Poor
The scoring table is then given at the end of the checklist.
Let’s now look at how this can be divided into different sections. Hopefully, each of the sections is sufficiently self-explanatory:
Section 1: General Impression
- A brief flick through the book reveals the overall appearance to be attractive and appealing to learners.
- Overall, the book appears to be up-to-date.
- The book’s description of itself appears to match the contents.
Section 2: Technical Quality
- The book is durable, with a strong cover, and is printed on good quality paper.
- The printing and illustrations are of high quality and the book has an attractive layout, without densely cluttered pages. It has been well edited.
- Color is used but not to a distracting extent.
Section 3: Cultural Differences
- Any cultural bias in the book is restricted to a degree acceptable to our learners.
- Cultures other than Western European or American are also portrayed in the book.
- The cultural tone overall is appropriate for use in our setting.
Section 4: How Appropriate is it?
- The materials, language foci and activities are in general appropriate for our learners.
- The book will meet the long- and short-term goals specific to our learners.
- Learners are not asked to perform roles or activities unacceptable in our setting.
- The activities are adaptable to personal learning and teaching styles.
Section 5: Motivation and the Learner
- Situations /Contexts used in the book are authentic and contemporary to an acceptable degree.
- Materials used have intrinsic interest and will appear relevant and interesting to learners.
- Materials with variety and pace are used.
- Personal involvement of learners is encouraged.
- The book encourages, to a certain degree, learners to assume responsibility for their own learning.
- There is a problem-solving and competitive element.
- The book exploits the social nature of classrooms.
Section 6: Pedagogic Analysis
- Methodologically the book is in line with the institution’s approach to language learning.
- Methodologically the book is in line with contemporary worldwide theories and practices of language learning.
- The balance between listening, speaking, reading, and writing skills development in the book is appropriate to our particular learners and learning situation.
- Skills integration is given sufficient attention.
- The development of discourse and fluency skills is given sufficient attention.
- The book contains adequate formal learner achievement tests / opportunities for self-assessment.
- There are activities for communicative interaction and the development of communicative strategies.
- The balance between individual work, pair work, group work, and whole-class work in the book is appropriate for our particular learning situation.
- One goal of the book is enabling learners to use English outside the classroom situation.
- New structures are presented systematically and in a meaningful context.
- New items receive sufficient and varied practice.
- The meaning of new vocabulary is presented in context.
- The grading of new items is not too steep or to gentle for our learners.
- In general the activities in the book are neither too difficult nor too easy for our learners.
- The book is sufficiently challenging to learners.
- There are mechanisms for giving regular feedback to learners.
- Units are not based around a storyline which may force the teacher to use every unit in sequence.
- There is variety in the makeup / format of individual units.
- Useful guidance is given to learners on correct use of the book.
- The style of speech / texts / contextualization used is appropriate for our learners.
- There is provision for the book to be used for self-study.
- New items are reviewed and recycled throughout the book / series.
- The book matches the curriculum objectives to a sufficient extent.
Section 7: Finding Your Way around the Student’s Book
- There is an adequate contents page.
- There is a comprehensive index.
- There is a complete summary of functions.
- There is a summary of new and reviewed grammar.
- If audio materials are used, there is a transcript in the student’s and/or teacher’s book.
- Sufficient guidance is given for the needs of all teachers.
Section 8: Supplementary Materials
- A teacher’s book is available and it gives useful and complete guidance, along with alternative activities.
- A workbook is available and it contains appropriate supplementary activities.
- Audio recordings are of good quality construction.
- Sound quality of tapes is good with no hissing, distortion, background noise, or other problems.
- Recordings have a variety of voices and they are native speakers talking at normal speed.
- If the book is part of a series, other books in the series are also suitable for use in your school.
- The coursebook, teacher’s book, audio and workbook are not prohibitively expensive for our students / school.
Further considerations
Scoring table
There are 58 criteria in the framework, with 2 points possible for each item. Items may be weighted before using the framework to reflect their relative importance in our teaching situation.
Suggested scoring procedure:
Enter the desired weightings in the ‘weight’ column. Enter the score you give for each item. Multiply each score by its weighting factor. Add up the totals to get the final score.
Note: It is advised that the course book be reassessed periodically in the light of the results of learner assessment and learner and teacher feedback. Additionally, it may be compared to newly released titles in the future.
*This framework contains elements of the checklists of Matthews (1985), Williams, D. (1983), Williams, R. (1981), Cunningsworth (1984), Breen & Candlin (1987), Sheldon (1988), Roberts (1990) and Peacock (1997).
References to authors mentioned and suggested further reading
Breen, M.P., & Candlin, C.N. (1987). Which materials? A consumer’s and designers guide. ELT Document 126. London: British Council, 13-28.
Cunningsworth, A. (1984). Evaluating and selecting EFL teaching materials. London: Heinemann.
Matthews, A. (1985). Choosing the best available textbook. In Matthews, A., Spratt, M., and Dangerfield, L. (eds.) At the chalkface. London: Edward Arnold, pp 202-206.
Roberts, J.T. (1990). RESDOC 7. Unpublished manuscript, University of Essex, England.
Peacock, M. (1997) Choosing the right book for your class, University of Essex, England.
Sheldon, L.E. (1988). Evaluating ELT textbooks and materials. English Language Teaching Journal 42/4, pp. 237-246.
Williams, D. (1983). Developing criteria for textbook evaluation. English Language Teaching Journal 37/3, pp 252-255.
Williams, R. (1981). A procedure for ESP textbook analysis and evaluation on teacher education courses. English for Specific Purposes Journal 1/2, pp 155-162.