ollie_4-fall14: Building a Better Mousetrap - 4 views
-
rubric that tells students, as a typical example, that they will get an A for writing a 1000 word essay that “cites x number of sources and supports its thesis with at least three arguments” will lead students to perceive writing as a kind of “paint-by-number” endeavor (Mathews).
-
joycevermeer on 20 Sep 14Yes, by saying "how many" students naturally look at quantity more than quality. So best is probably that we speak of both when creating rubrics.
-
-
ubric that tells students, as a typical example, that they will get an A for writing a 1000 word essay that “cites x number of sources and supports its thesis with at least three arguments” will lead students to perceive writing as a kind of “paint-by-number” endeavor (Mathews).
-
ubric that tells students, as a typical example, that they will get an A for writing a 1000 word essay that “cites x number of sources and supports its thesis with at least three arguments” will lead students to perceive writing as a kind of “paint-by-number” endeavor (Mathews).
- ...79 more annotations...
-
-
es, I have seen this with my reflection logs in my classes. When we use quantity rather than quality indicators, we do seem to get minimum responses. I plan to respond to the logs frequently to encourage teachers to use them for self-reflection. That was not possible working face to face.
-
My students reported not knowing what was being assesed was their biggest asssessment fear. Actual models and discussion about criteria is absolutely necessary.
-
-
Barbara Moskal, in her article “Scoring Rubrics: What, When, and How?” insists that rubrics should be non-judgmental: “Each score category should be defined using description of the work rather than judgments about the work.” For example, “sentence structure follows current conventions” would be better than “sentence structure is good.”
-
-
I believe some qualities must be defined using qualitative terminalogy. This is where student input and comparison of models of high and low quality criteira would help students define or picture the quality indicators that are "judgemental". Scampie1
-
I agree. After being given a number I feel like they don't have a qualification equivalent. When I took a course much like this at Iowa State, our professor had a rubric based on the quality of the post, not how many. It was nice to be given credit for content instead of word count. It goes back to the days when we were forced to write 10 pages on pandas with at least 10 sources with only one of those being the WWW (which is how it was stated). I also don't like having such a high quantity because with so many students in the course it becomes overwhelming to read each one, let alone provide a meaningful comment or reflection. There isn't much text that left to highlight, back to "less is more".
-
-
Indeed, since rubrics allow for widespread assessment of higher-level thinking skills, performance-based assessment is replacing or complementing more traditional modes of testing; this in turn means that teachers are changing their instructional modes to prepare their students for these tests. Kenneth Volger, in his study, “The Impact of High-Stakes, State-Mandated Student Performance Assessment on Teacher’s Instructional Practices,” analyzes the survey responses of 257 10th grade English, math, and science teachers and concludes that, since the implementation of such tests in public schools, there has been “notable increases in the use of open-response questions, creative/critical thinking questions, problem-solving activities, […] writing assignments, and inquiry/investigation.”
-
Indeed, since rubrics allow for widespread assessment of higher-level thinking skills, performance-based assessment is replacing or complementing more traditional modes of testing; this in turn means that teachers are changing their instructional modes to prepare their students for these tests. Kenneth Volger, in his study, “The Impact of High-Stakes, State-Mandated Student Performance Assessment on Teacher’s Instructional Practices,” analyzes the survey responses of 257 10th grade English, math, and science teachers and concludes that, since the implementation of such tests in public schools, there has been “notable increases in the use of open-response questions, creative/critical thinking questions, problem-solving activities, […] writing assignments, and inquiry/investigation.”
-
Indeed, since rubrics allow for widespread assessment of higher-level thinking skills, performance-based assessment is replacing or complementing more traditional modes of testing; this in turn means that teachers are changing their instructional modes to prepare their students for these tests. Kenneth Volger, in his study, “The Impact of High-Stakes, State-Mandated Student Performance Assessment on Teacher’s Instructional Practices,” analyzes the survey responses of 257 10th grade English, math, and science teachers and concludes that, since the implementation of such tests in public schools, there has been “notable increases in the use of open-response questions, creative/critical thinking questions, problem-solving activities, […] writing assignments, and inquiry/investigation.”
-
-
A high school math teacher I know tried to add writing in math as part of the Writing Across Curriculum program. Instead of using appropriate prompts for technical writing in mathematics, the rubric forced her to use themes and literacy criteria. Writing did improve in her school on state assessments but she didn't feel math scores were affected.
-
Hopefully the students that took the math course that added writing were in a better position to be successful later in life by gaining more writing instruction overall, even if the math scores weren't impacted in the short term.
-
-
-
The instructor’s comments on papers and tests are done after rather than before the writing, so they cannot serve as guidelines, compromising the value of writing comments at all
-
This is key. The students must get the rubric before turning in their work to get the maximum value from it. Additionally, I have found that if students practice assessing samples using the rubric prior to turning in their own they understand the goal more fully.
-
Deb, I think you are right having the students complete an exercise where they can study and grade samples prior to turning in their own work can be a very powerful tool in helping students reach mastery of a task.
-
-
Perhaps the greatest potential value of classroom assessment is realized when we open the assessment process up and welcome students into that process as full partners” (qtd. in Skillings and Ferrell).
-
This sounds great, but I can't imagine how long it would take for students to help create assessments. I taught 3rd and 4th graders, and I don't think I'd truly try this. Sounds great in theory, but I'm skeptical.
-
Deb, I'm with you on this one. I would be very interested to see how an instructor set up a lesson to allow for student collaboration in developing a quality rubric. I'm not saying that it can't be done but it could be a very cumbersome task. I'm trying to imagine doing it with my 30+ sophomores in a World History class.
-
I am also in agreement with the both of you. I have thought about doing this with my students after I do a demonstration. Asking the students what the learning targets should be. I am sure that some of them could do this with ease and it might allow them to take more ownership in their learning, but there just isn't enough time for this. It might be better for a small claass
-
-
I once gave extra credit to a student who realized that without providing a shred of meaningful content she could meet all the requirements of a state writing rubric he posted in his classroom. As required she used the word “persuade” and two synonyms, composed a clear topic sentence and closing sentence, and made no spelling or grammatical errors. But she did it without saying anything coherent.
-
That's a scary thought, although I have to say I've felt that kids met the letter of the rubric but missed the intent. Writing a good rubric is challenging. I have to honestly say that I've never truly felt like I nailed what I wanted with a rubric. However, I've had good luck with rubrics when kids used them to score samples. Then they understood the intent or goal of the rubric.
-
I agree that "writing a good rubric is challenging". This is an area that will continue to be a work in progress as I continure to work towards a clearer message being shared.
-
I thought this was interesting and scary too. A student could meet all the requirements and not really say anything.
-
-
mitagate both teacher bias and the perception of teacher bias (
-
I'm not sure I'd agree with this. I think rubrics are subjective by their very nature.
-
My thought is that having a rubric generally makes for less subjectivity than not having a rubric.....though subjectivity can be present either way.
-
I agree with Joyce. Unless a teacher is using rubrics at inappropriate places, she is usually measuring something that is subjective. The rubric helps put it in a more objective manner, allowing for more consistency between different assessors.
-
-
extremely short scales make it difficult to identify small differences between students.
-
Analytical or holistic
-
Look at some actual examples of student work to see if you have omitted any important dimensions.
-
So true - how many times have I created a rubric that I thought was good - that is until I looked at what I got from students and realized I missed something that should have been included in the rubric. 2nd iterations are usually better.
-
I've done exactly the same and omitted a key point that should have been included. I agree with the second iterations being better.
-
-
we need a meta-rubric to assess our rubric
-
Moreover, rubrics can help the student with self-assessment; what is most important here is not the final product the students produce, but the habits of mind practiced in the act of self-assessment.
-
When students receive the rubric for assessment prior to beginning their work this eliminates excuses a teacher may hear as everything is clearly stated and noted. A rubric serves as an excellent way for students to self assess their work and determining whether the goals have been achieved and at what level.
-
Agree, showing students the rubric first allows them to know what to aim for, what standards are critical for a particular assignment, and gives students a goal.
-
-
he criteria must be made clear to them
-
When developing a rubric one must remember to clearly state what makes the final product exemplary, proficient and may need additional work. Students can then evaluate or self assess at a more accurate level.
-
Sometimes as educators we can use a lot of jargon. We need to remember to keep it student focused if we are going to give the rubric to the students. It doesn't do the student any good if they don't know the meaning of the different criteria.
-
-
“think more deeply about their learning.”
-
When possible I believe we need to include students in the assessment process. This gives them a voice and buy-in to their learning.
-
I can sure see how involving children in creating the rubric would require them to think more deeply. I wonder if students would suggest a harder or easier assignment than the teacher would. Sometimes just giving opportunity for input brings on motivation.
-
-
well-designed rubrics help instructors in all disciplines meaningfully assess the outcomes of the more complicated assignments that are the basis of the problem-solving, inquiry-based, student-centered pedagogy replacing the traditional lecture-based, teacher-centered approach in tertiary education.
-
This quote seems to answer the basic question of why we should worry about designing rubrics? It is one tool in assessing assignments that have the potential to be more meaningful for our students (problem-solving, inquiry-based, student-centered).
-
Additionally, with these types of assignments without a rubric there is the possibility for subjective judgements. A quality rubric will remove the subjective aspect to grading.
-
I think once we understand rubrics and why we use them we can better decide when and how to use them.
-
-
along with supporting models of work
-
A quality rubric is good for students but a quality rubric as well as supporting models or samples of student work can be much more powerful in helping students see what is actually expected from them. This is an aspect that I feel I could do a much better job of in my own instruction.
-
I completely agree! As a primary teacher, I found models to be very powerful in giving students a visual of what different levels of quality work would "look like." Depending on the task, I would either post different levels of quality work and have them identify the criteria that matches each level or I would have them help me create models of what each quality level would look like. I fully recognize that this process is much easier in the primary grades where the tasks are often much smaller. However, I did find students were much more reflective on their work when we went through this process.
-
-
from students is not only unfair and makes self-assessment more difficult,
-
Moreover, some teachers have noticed how students who were good writers become wooden when writing under the influence of a rubric.
-
It seems that some of these anecdotes show that the rubrics being used are missing categories to assess all the needed categories. The first example is missing a component on being cohesive and this one is missing the creative component. I have done this in the past, given a rubric and then realized it is missing something critical. This is why rubric building takes so much time.
-
-
rubrics, in effect, dehumanize the act of writing.
-
With writing assignments is it not best that students just write from their heart first, then see if anything needs to be tweeked to fit the rubric requirements? Teachers can encourage this, but once the rubric is there it does make it hard....especially for younger children. This article brings up many valid points.
-
-
build your own rubric from scratch
-
I'd probably take the build-it-from-scratch option. Sometimes it easier to come up with your own ideas than change somebody else's and you can then be more creative.
-
I agree. It is fine to build one or two at the beginning, but then I find myself cannibalizing them to fit for an assignment. But, this builds consistency for the students as they can see what I am looking for and know that I have looked for that in prior assignments.
-
-
-
“Meaningfully” here means both consistently and accurately
-
When working on my second grade team we often spent time trying to calibrate the consistency and accuracy of our rubrics. On district wide assessments we would determine the assessments and rubric and each score separately. We would continue to do this and refine. It was always very eye opening to see how different we would score the same writings. The process was very important to the refining of the rubric to the point where it was able to be used consistently and accurately with similar results from all in the group. It sounds like we will be doing a similar project this week for our course.
-
When I taught 2nd grade, our team did a similar process. I do believe that calibrating is a very important component to rubrics. Rubrics alone certainly do not guarantee that every member of a team will score a piece of writing the exact same way.
-
It is interesting you both had this on a 2nd grade team. I have never done this, other than when I was a principal and I led inservice to do it. I wonder if any secondary teachers have experience calibrating.
-
I struggle as a consultant to get the instructors to be sure when using a rubric, that they use quantifyable descriptors that have a quality spelled out specifically. It is either present, or not, or measured someway in between.
-
Calibration is so important when you look at it from a district level view. When we have students moving from school to school or not even moving, but comparing data from schools, if rubrics haven't been calibrated or pieces commonly scored, the data really doesn't tell you what you may think it does.
-
-
if they are shared with students prior to the completion of any given assignment.
-
In my teaching experience, I found that many teachers skipped the process of sharing the rubrics with students. I felt that this was a huge missed opportunity with the children in our classroom. However, thoughtout my teaching career this became more of a focus in our district and it was amazing to see the transformation in student ownership of work when they had the criteria as defined in the rubric shared with them prior to their work.
-
If we aren't sharing the rubrics with the students, they really can't take that ownership because they are guessing at what the teacher wants from them.
-
-
others worry that doing so will encourage formulaic writing
-
This is an interesting point, and one that I had never really thought about. I can say that in my 7 years in the 2nd and 3rd grade level classrooms that I did not find this to be much of a problem. However, I can see how some secondary level rubrics could put more of an emphasis on number of words, spacing, requirements for number of arguments, etc. could lead to more of a formulaic style of writing.
-
I can say this is a big issue in writing, especially "writing across the curriculum", where it is assigned by non-language arts teacher.
-
-
Feedback
-
Our team of teachers in both 2nd and 3rd grade found great benefit from conversations with students about the rubrics we were using. It became easy to see how certain words of phrases in our rubrics were unclear to students, and how simple changes could make them more applicable and meaningful for our students. I think the feedback process is vital in creating a quality rubric for students.
-
-
As both institutional enrollment needs and social pressures for access raise the need of remediation (Soliday), rubrics become increasingly important to student success. Gisselle O. Martin-Kneep, in “Standards, Feedback, and Diversified Assessment: Addressing Equity Issues at the Classroom Level,” reports that extensive use of rubrics can help minimize students’ educational disparities and bring fairness into assessment on numerous levels:
-
imilarly, Heidi Andrade, in her study, “The Effects of Rubrics on Learning to Write,” has found that, while rubrics increased her students’ knowledge of the grading criteria and helped most of her students (especially the young male students) do well on the state writing test, many of the young female students, who had been more expressive in previous writing assignments, wrote poorly when writing, as we might say, to the rubric.
-
Very interesting points about the negative side of rubrics for students who may be gifted in the area of writing. Because my career has focused on students who are struggling, it makes me aware of how "one size does not fit all." I guess the gifted student writer has to learn how to pass the State Assessment and then encouraged and mentored to grow in their gift of writing....interesting.
-
I thought this was very interesting too. Rubrics may stifle creativity especially when we want students to be creative thinkers and writers.
-
I thought this made interesting points as we are working so hard to create rubrics, we have to also think about those students whose work could be so much more but they are fitting into the constraints of the rubric.
-
-
Steps in developing a scoring rubric
-
How well is the rubric tied to instruction? That is, does the rubric use the same critical vocabulary used in our instruction?
-
i appreciate the author stating that another criteria is that the rubric is tied to instruction and uses the same critical vocabulary. High stakes state testing is here. But, I prefer to put my energy into increasing student learning so that all students can feel successful in school.....closing the achievement gap.....and not just measured by state tests.
-
This is a very valid point here- I find that when working with special education students, they tend to struggle when different vocabulary and/or sets of presentation materials are used in assessment.
-
-
rubrics can be either “general” or “specific
-
Pilot test your rubric or checklist on actual samples of student work
-
I like the idea of a pilot test. This could be done during a students first draft and would give the teacher an opportunity to make revisions prior to the final being assessed.
-
This could be another place where student involvement in developing the rubric could come into play. Having the students use a rubric to assess their own work but at the same time assessing the rubric to see if it could be more detailed or written in more student friendly language.
-
-
The issue of weighting may be another area in which you can enlist the help of students. At the beginning of the process, you could ask a student to select to select which aspect she values the most in her writing and weight that aspect when you assess her paper.
-
When instructors plan on grading student thinking and not just student knowledge, they should articulate the vital features that they are looking for and make these features known to the student.
-
It certainly only seems logical to share rubrics with students when assignments are given, but I know this isn't always practiced. It shouldn't be a secret what features will be graded. Rubrics provide students with a clear vision of what is expected of them and helps guide their work.
-
This does defeat the purpose, doesn't it? Go to all that work to set expectations, and then not give students those expectations.
-
When I was teaching, I used rubrics as part of my instruction. When clear expectations are set for students, they know what to aim for- and performance is usually higher.
-
-
understandable to the student but also be linked specifically to classroom instruction
-
In the primary grades, I think it can be a challenge to find "understandable" language. I do believe that incorporating the language from the rubric into classroom instruction will help bridge the gap between being student friendly and also specific enough for adults to grade accurately.
-
In the high school setting, I find that giving the students the rubric that I am going to use to grade them, when I give them the assignment, allows for transparency (I'm not playing "Gotcha") and allows for the students to ask specifically what I am looking for, which then allows them to gear their work to what I am looking for as opposed to just hoping that they got it correct.
-
-
The argument against using rubrics
-
First, you must decide whether you need a rubric. Consider what the purpose of assessment is.
-
The second step is deciding who your audience is going to be. If the rubric is primarily used for instruction and will be shared with your students, then it should be non-judgemental, free of educational jargon, and reflect the critical vocabulary that you use in your classroom.
-
While longer scales make it harder to get agreement among scorers (inter-rater reliability),
-
Both types of rubrics benefit the teacher and the student in varying degrees: the teacher who relies on a general rubric does not have to develop a new one for each assignment and the student grows to understand fundamental standards in writing—like form and coherence—exist across the board; meanwhile, the teacher that uses specific rubrics is always composing new descriptions of quality work, but their students have clearer directions for each assignment. Of course, a teacher could have the best of both worlds here, by designing a rubric on a PC that allows for the easy insertion of assignment specific traits.
-
assess outcomes in learning situations that require critical thinking and are multidimensional.
-
“on what students have actually learned rather than what they have been taught,”
-
contends that we ought to illicit student input when constructing rubrics:
-
According to Thomas Newkirk, an English professor at the University of New Hampshire, “rubrics promote ‘mechanical instruction in writing’ that bypasses ‘the human act of composing and the human gesture of response’” (Mathews). This overly scientific view of writing, Newkirk and others argue, stunts the learning process. Moreover, Judith Halden-Sullivan sees a disconnect between the learning goals of Writing Across the Curriculum programs and the rubrics often designed to assess that learning. Assessment of this sort seems at odds with such concepts as “deep learning,” which implies a kind of learning that is beyond measurement, an elusive hard to describe enlightenment, but identifiable in the same way good art is: teachers know deep learning when they see it. Rubrics, Halden-Sullivan contends, reduce “deep learning” to “checksheets.”
-
According to Thomas Newkirk, an English professor at the University of New Hampshire, “rubrics promote ‘mechanical instruction in writing’ that bypasses ‘the human act of composing and the human gesture of response’” (Mathews). This overly scientific view of writing, Newkirk and others argue, stunts the learning process. Moreover, Judith Halden-Sullivan sees a disconnect between the learning goals of Writing Across the Curriculum programs and the rubrics often designed to assess that learning. Assessment of this sort seems at odds with such concepts as “deep learning,” which implies a kind of learning that is beyond measurement, an elusive hard to describe enlightenment, but identifiable in the same way good art is: teachers know deep learning when they see it. Rubrics, Halden-Sullivan contends, reduce “deep learning” to “checksheets.”
-
According to Thomas Newkirk, an English professor at the University of New Hampshire, “rubrics promote ‘mechanical instruction in writing’ that bypasses ‘the human act of composing and the human gesture of response’” (Mathews). This overly scientific view of writing, Newkirk and others argue, stunts the learning process. Moreover, Judith Halden-Sullivan sees a disconnect between the learning goals of Writing Across the Curriculum programs and the rubrics often designed to assess that learning. Assessment of this sort seems at odds with such concepts as “deep learning,” which implies a kind of learning that is beyond measurement, an elusive hard to describe enlightenment, but identifiable in the same way good art is: teachers know deep learning when they see it. Rubrics, Halden-Sullivan contends, reduce “deep learning” to “checksheets.”
-
, “rubrics promote ‘mechanical instruction in writing’ that bypasses ‘the human act of composing and the human gesture of response’”
-
Does the rubric relate to the outcome(s) being measured? Does it address anything extraneous? […] Does it cover important dimensions of student performance? Do the criteria reflect current conceptions of excellence in the field? […] Are the dimensions and scales well defined? […] Is there a clear basis for assigning scores at each scale point? […] Can different scorers consistently apply the rubric? […] Can students and parents understand the rubric? […] Is the rubric developmentally appropriate? […] Can the rubric be applied to a variety of tasks? […] Is the rubric fair and free from bias? Does it reflect teachable skills or does it address variables over which students and educators have no control, such as the student’s culture, gender or home resources? […] Is the rubric useful, feasible, manageable and practical? […] Will it provide the kind of information you need and can use effectively?
-
-
using rubrics to establish “performance benchmarks” for the “behavioral objectives” appropriate to each year in the program.
-
Rubrics for performance benchmarks that applicable to each year's objectives shows that they are not just using rubrics to use them, but that they are specific and aimed at the performance of the students.
-
I have been working with my district on standards based grading. I think rubrics can shape the standards in relation to quantitative data.
-
-
accurately measuring the specific entity the instructor intends to measure consistently student after student
-
“Perhaps the greatest potential value of classroom assessment is realized when we open the assessment process up and welcome students into that process as full partners”
-
In any case, withholding assessment tools (whether they are rubrics or more nebulous modes of evaluation) from students is not only unfair and makes self-assessment more difficult, it maintains the traditional gap between what the teacher knows and what the student knows.
-
In any case, withholding assessment tools (whether they are rubrics or more nebulous modes of evaluation) from students is not only unfair and makes self-assessment more difficult, it maintains the traditional gap between what the teacher knows and what the student knows.
-
Holding back rubrics and the way teachers are grading from students is kind of like giving the "gotcha" at the end of the assignment. I know when I am not clear on my directions, I get 500 questions about it. When I am clear and students know what they will be graded on, it is better for both of us.
-
-
Be prepared to evaluate your rubric, using your meta-rubric and feedback—direct feedback from the students and indirect feedback from the quality of their work. Modify accordingly.
-
-
This was a key "a-ha" for me. I think that rubrics are over used. Not everything can or needs to be assessed via a rubric. The key question for teachers should be first "What am I assessing?" then "Will a rubric meet the needs of what I am assessing?" So many times a rubric is used and it doesn't match what needs to be assessed.
-