Category: Assessment (Fall 2011)
Dr. Steve Bailey was co-founder of Shalhevet High School (Los Angeles) and a researcher, author and consultant in Jewish education, specializing in moral education and curriculum projects. He is a professional development workshop presenter and Co-Director of Quality Education by Design, www.QEDworkshops.com [1]
[1]The assessment approach described in this article is taken from my chapter entitled, “Jewish Literacy by Design: A Case Study of Developing and Implementing a Jewish Literacy Curriculum” in the forthcoming book, Perspectives on Jewish Literacy in Education, to be published by the University Press of Maryland. That chapter describes the full Jewish Literacy project referred to in this article in greater detail.
Steve Bailey describes an assessment project that transformed the teaching of Jewish studies in one school.
Although educators agree that assessment is an essential and integral component of any curriculum plan, there is much ambiguity about the definition and goals of assessment for education. After searching different definitions, I’ve chosen the following definition of “assessment of student learning” from The Higher Learning Commission (A Commission of the North Central Association of Colleges and Schools):
“Assessment of student learning is a participatory, interactive process that:
- Provides data/information you need on your students’ learning;
- Engages you and others in analyzing and using this data/information to confirm and improve teaching and learning;
- Produces evidence that students are learning the outcomes you intended;
- Guides you in making educational and institutional improvements;
- Evaluates whether changes made improve/impact student learning, and documents the learning and your efforts.”
The focus of this definition expands the direct goal of assessment – data about student learning of curriculum goals – to the more broad goals of improving both classroom teaching and the educational institution by evaluating the success of the school’s overall educational aims. One of the important byproducts of these expanded goals of assessment is that a school that assesses its success at teaching and learning can demonstrate to its community, as well as its faculty, that it produces graduates who possess the basic knowledge and skills of an educated Jew. Not only is this outcome a validation of teachers’ success, but also such an outcome is a powerful marketing tool that serves to attract enrollment and to retain already enrolled students. Parents are more likely to be willing to spend the tens of thousands of dollars for Jewish education if they get assurances that their sacrifice produces an educated child who has a positive, successful experience of Jewish learning and a sense of pride and competency in his or her Jewish knowledge and skills.
A case study
This broad approach to assessment characterized a five-year school curriculum project I undertook at Moriah College, a large (2000 students, K-12) Jewish day school in Sydney, Australia. The overall goal of the project was “basic Jewish literacy” – that is, the successful acquisition of knowledge, skills and experiences needed for a “successful, Jewishly literate graduate,” as defined by the school. Although the overall project covered the development and implementation of a comprehensive Jewish studies curriculum, as well as its assessment and evaluation of outcomes, the subject of this article is the rationale and description of the more innovative assessment components of the Jewish studies curriculum. The assessment components, excerpted below from the project description, were directed towards demonstrating that the school’s graduates were “Jewishly literate.”
Mastery learning
The fundamental approach towards assessment was strongly influenced by the educational philosophy of “mastery learning” as originally described by Benjamin Bloom (1971). This means that the focus of teaching was on each child’s mastery of the “essential Jewish learning” presented in the units of classroom instruction, as evidenced by assessment. Mastery learning expects a "right-skewed” distribution of quiz and test scores (most of the class earning higher marks), not a bell-curve distribution. For every class, both the enrichment and mixed, we expected that 60% of the students would show mastery of at least 70% of the core material taught in the unit. If the mastery criterion was not reached, the students needed to re-learn and retake the test until mastery. Obviously the percentage expectation was higher in enrichment streams and lower for special learning streams, but for the mixed-ability classes, this was a realistic expectation. The method of mastery learning served to motivate the student to study for the unit quiz and Term Test the first time, as well as assure that the child achieved the basic skills to continue to the next level of learning. To monitor successful outcome, i.e., attainment of mastery criteria for the class, all teachers recorded class grades for Term Tests. The teachers’ data was analyzed in terms of number of students attaining 70% criterion in each class on the initial testing. If the class fell below these criteria, discussions were held with the teacher to develop more effective teaching strategies.
Notice that in this strategy, there is an inherent component of accountability – both for the student and the teacher. Teacher accountability for the successful acquisition of the basic curriculum material was one of the innovative components that characterized this approach. It should be noted that although there was initial resistance on part of teachers to be held responsible for the learning success of their students, the resistance diminished quickly when teachers experienced the positive feelings of competency on seeing their classes meet the target criterion.
Standardized quizzes and Term Tests
In order to gather information for measuring student achievement on the essential learning objectives, we needed various types of assessments. After mastery learning, the use of standardized quizzes and Term Tests was the second assessment strategy. Following the completion of each unit, there was a classroom quiz on the core information in the unit. About 60-70% of the questions on the Term Test would be taken from these quizzes. Teachers were instructed to review the answers in class after the quizzes were completed, so that students could correct their quizzes and use the correct answers for future review. There was one project (computer related) assigned each term from the material learned in the first half of term. Once again, this served as assessment data, but also allowed a student to pursue personal interests and develop research skills.
At the end of each term, we scheduled a test in the Jewish Studies subject areas (e.g., Tanakh, holidays, laws and customs, Jewish history) for all sections of a year level. The test was based solely on the material presented in the unit’s Student Booklet and accumulated from the quizzes. One or two classes before the Term Test, teachers reviewed the items on the quizzes to focus students on studying the essential materials from their unit’s Student Booklet. The idea was to give students as much guidance as possible to successfully earn a passing grade on the initial test, without specifically revealing the questions. The overall goal was to get most of the class to pass with 70% correct or higher – not to trick, prove a point or punish non-academic students. After the tests, the curriculum developer provided an answer key for the standardized Term Test so that marking would be consistent and fair. In order to ensure unbiased grading, teachers exchanged section tests and did not mark their own classes. Teachers who taught enrichment classes or taught other material than appeared in the Student Booklet, were encouraged to test their own students, in their particular classrooms, on any additional material.
It should also be noted that the unit quizzes and Term Tests were standardized (produced by me based on the core curriculum, not by the individual teacher) so that the curriculum goals were objectively met. Having standardized quizzes also allowed students to use the frequent quizzes as study guides for the Term Test.
A third innovative component was assessment of effective teaching through classroom observations and sharing creative methodologies. Throughout the development and implementation of the curriculum project, the curriculum developer observed all teachers in a few classrooms each term. The purpose of the observation was to note the particular creative, effective teaching methodologies that were used in transmitting the standardized curriculum material in the Student Booklets and Teachers’ Guides. The classroom observations were not designed to evaluate the teacher but rather solely to note innovative teaching methods. The summaries of the individual positive observations were shared with all staff so that they might learn effective, creative methods of transmitting the required material, from their colleagues.
Jewish literacy comprehensive exam for graduation
Probably the most innovative assessment component of the project was the Jewish Literacy Comprehensive Exam for Graduation required of all graduates. It was the means by which we could assure that the goal of essential Jewish literacy, covering later years of formal Jewish Studies, had been achieved. The Comprehensive Jewish Literacy examination required students to demonstrate their achievement in Jewish knowledge and skills that they had acquired during their Middle and High School years, from Grade 7 to graduation. The computer-based, comprehensive test (taken at individual computer stations in the lab, by appointment) covered topics in Tanakh, holidays, Jewish law and custom, oral Torah, Jewish history, prayer and life-cycle events – as presented in their formal curriculum and in Jewish life on campus. All questions were taken from Term Tests in all subjects throughout the five years and a student study guide was composed to help the student review material and prepare for the exam. Teachers devoted the last term (six weeks) of classes before graduation to a review of the material presented in a special preparatory study guide (much like one prepares for a driving exam from a handbook that covers all the material on the exam). Success on the test required an 85% correct rate for each area. Areas not passed would have to be retaken until passed.
The computer-based Jewish Literacy exam required students to correct their wrong answer immediately before going on to the next question. This served as a learning tool so that students learned the correct information, helping them pass on the re-test. The purpose of the test was to assure Jewish Literacy, so our goal was for students to acquire the knowledge, by preparation as well as by learning from their mistakes, and successfully pass the test. Students expressed satisfaction with the format and the fairness of the exam. Also, students without earlier day school education, who had transferred from other schools or students who were in special learning programs or had significant difficulty with Hebrew, were given a modified examination.
By successfully passing this exam, students showed themselves (and their parents) that they were literate, educated Jews. Statistics were gathered on the three years after the completion of the project. The average passing rate (after re-test, when necessary), out of an average graduating class of 135 students, was 89% in 2005, 87% in 2006 and 82% in 2007. (I do not have follow-up data on the years after I left the school.) About 10%-12% of the students were excused from the test because of special learning problems or for coming into the school late in their academic careers, with little Judaic background. The Jewish Studies Department acknowledged each graduate’s success at passing the exam, by issuing a Certificate of Jewish Literacy. The personalized certificate acknowledged graduates who achieved over 85% (on the first round of testing) by indicating passing with “honors”, cum laude or summa cum laude.
Evaluation of the strategy
When the project began, an area of concern for all teachers regarded the overall assessment strategy. They felt that they were being asked to “teach to the test.” They were concerned that structured booklets, focused quizzes, guided study for Term Tests, mastery learning and formal preparation for a final comprehensive literacy exam, all promoted students’ concentration on pre-defined knowledge and skills, at the expense of creativity, abstract problem solving and generalization to new learning. However, this proved to be an ephemeral fear. As they taught the curriculum units they saw that the questions and activities in the Student Booklet encouraged creative thinking, autonomous reasoning, generalization and challenging projects. What they came to appreciate was that, indeed, when it came to essential Jewish learning for literacy, the student was clearly guided to master the core curricular material successfully, but it was not at the expense of personal creativity, initiative and imaginative new ideas. It was a strategy of assuring the acquisition and preservation of a constant core of knowledge and skills, while encouraging students (and teachers) to grow and develop along their personal interests and talents.
No less importantly, surveys of students showed that an average of 86% of students across all grade levels enjoyed learning from the Student Booklets, found the quizzes and tests fair and approved of the comprehensive Jewish literacy exam as a requirement for graduation. Anecdotally, teachers reported that students respected their Judaic Studies more than in the past because of the organization and sophistication of the curriculum and assessment methods. Students also felt a sense of personal competence through the mastery learning outcomes, since this approach assured “success,” even if it took more than one try. Teachers reported that, overall, their students felt that Jewish studies was meaningful and worth expending the effort to do class work, homework and study for tests. With this outcome-oriented approach to innovative, focused and planned assessment strategies, both teacher and student satisfaction was high.
All schools will define “Jewish literacy” in their own way and the curriculum plan for each school needs to be customized for its particular educational goals, as outlined in its mission statement. However, the overall structure, methodology and dynamics of the assessment approach described above, regardless of differing definitions of basic Jewish literacy, allows a school to demonstrate to its faculty, parents and the community that it actually graduates knowledgeable Jews.
References
Bailey, S. “Jewish Literacy by Design: A Case Study of Developing and Implementing a Jewish Literacy Curriculum” in Perspectives on Jewish Literacy in Education, in press, University Press of Maryland.
Bloom, B. (1971). Mastery learning. New York: Holt, Rinehart, & Winston.