Streamline SBG Feedback with Gravic Remark OMR #sbar #edtech #physicsed

I’m going to try out Skills Based Grading (SBG) next year in my Regents Physics courses.  I’ve talked to lots of teachers using it, read Marzano’s “Formative Assessment and Standards-Based Grading: Classroom Strategies That Work,”image  many terrific blogs, tweets, etc., and I’m convinced that providing students quick and detailed feedback on exactly how they’re doing with respect to course standards will benefit us all.

But I’m also worried.  Worried about the hiccups, the unknowns, the corners I may drive myself into.  Worried about tracking, about keeping up, about consistency.  And I’m worried about my ability to provide and record all the detailed feedback necessary.

Without a doubt I’ve been one of the hardest-working teachers in the building… I’m usually in my room by 6:30 a.m., most afternoons I don’t leave until 4:30 or 5 p.m., one night a week I often spend working until 8 to 10 p.m., and I come in for half a day or so on weekends fairly regularly.

imageI enjoy what I do, and I don’t mind the time commitment.  But I don’t want it to increase, especially with a family at home that I adore (and my daughter now believes watching baseball with Daddy is more fun than Mickey Mouse Clubhouse!!!).   So I can’t allow SBG to take any more time from me during the school year.  But how do I provide 100+ students with detailed, by-skill feedback on the larger standardized-type assessments, with multiple reassessment opportunities?  (Yes, I know about the standardized assessments, but here in NY emphasis is being heightened on standardized testing, including up to 40% of a teacher’s performance evaluation).

I spent several months researching this problem, with potential solutions ranging from a multitude of “punch-out”-type answer keys for individual assessments, all the way to having students do multiple self-assessments and exam breakdowns.  Of course, the personalized assessments that pervade the SBG mentality still apply, but for larger standardized assessments, including mid-terms and end-of-year practice exams and final exams, spending day after day grading the same exam across multiple skills just doesn’t make sense.

Finally, with the help of some terrific support folks at Gravic, I decided to try out Gravic Remark OMR.image   Remark OMR is a software package that allows you to scan multiple choice bubble sheets in a standard sheet-fed scanner, and evaluate them against an answer key which can break down questions into individual skill scores.  Further, with multiple exams and versions of exams, you can bar code the exam answer sheets against the answer key to help prevent mis-scoring.

The software package comes with a built-in analysis package which makes breaking down scores by class, individual skills, demographics, or any other student input quick and easy.

Setup of answer keys is fairly straightforward — you can make your answer keys in Word or any PDF creation system, and print them out on a standard copier.


The downside – Remark OMR is expensive.  A single-use installation license runs $995, and support is free for only 30 days.  Getting up and running with the software takes a little bit of tinkering, but within a few days you can be creating exams, scoring keys, and grading 50+ MC question sets across 100 students in 10-15 minutes.

I wouldn’t recommend it for all courses, but in a course where standardized testing is emphasized, and you want to provide many students detailed score breakdowns on a repeated basis across many multi-skill assessments, Remark OMR has terrific potential.  I used it as part of our Regents Exam review process this year… we gave the students old Regents Exams, and scored them using Remark OMR, providing each student detailed feedback on areas of strength and weakness.  Then, students developed an individualized action plan to work on their greatest opportunities of improvement independently using each other, review books, course notes, and the APlusPhysics physics tutorials before sitting down for a reassessment.

This process was repeated several times, and student feedback has been tremendous – they love how their review work is tied directly to their performance, they appreciate being able to track their improvement as we get ready for their culminating exam, and they particularly love the immediate feedback facilitated by the quick scanning and scoring process.

Three Wishes for Standardized Exams in 2011

As we begin the new year, I have high hopes for several changes in the administration, timing, and implementation of standardized physics exams from both the College Board and the NY State Board of Regents. Although I believe the likelihood of all of these happening in 2011 is quite slim, I maintain that all three are reasonable and feasible.


#1 Finalize Plans for AP-B Physics

The College Board’s decision to redesign the AP-B course deeply effects course sequencing at Irondequoit High School.  We’ve heard talk of the split for several years now. A seminar describing the changes was presented at last summer’s national AP conference in Washington, D.C., where the presenter and College Board representative stated “it’s a done deal, the only question is timing.”  We were told that the changes would be implemented in the 2011-2012 school year or 2012-2013 school year.  That’s fast!  We were promised more detailed information by last fall.  And we’ve heard nothing beyond a New York Times article which mentions potential changes in the 2014-2015 school year.

The preliminary redesign information presented at the 2010 AP conference indicated the course would be split into AP-1 and AP-2, where AP-1 is designed as a first-year course, and AP-2 is the more detailed, deeper second-year course.  The courses could be taken concurrently, although this was strongly discouraged during the presentation.

Actual implementation will have profound consequences for our district.  First, our school currently offers three levels of physics.  Regents Physics for juniors or seniors (a college-prep course based on NY state standards, equivalent to a typical Honors Physics course); AP-B, which can be taken as a first-year course by advanced students or as a second-year course following Regents Physics; and AP-C Mechanics and E&M, which can be taken by seniors who took AP-B as a first-year course.

Our concerns center around what the added AP-1 and AP-2 offerings will do to our other programs.  As a NY state school, we are highly encouraged to offer Regents Physics, consistent with state standards and a formal state-administered final exam in June.  Splitting the AP-B course into a two-year sequence could potentially damage our AP-C course, unless we replaced our current AP-B offerings with a combined one-year AP-1 and AP-2 (which is making the problem of the AP-B course having too much information in too little time even worse!). Or, we could combine AP-1 and Regents Physics together although the defined curricula don’t make for a smooth overlap, and offer AP-2 in place of our current AP-B course. Unfortunately, this makes it difficult for our enterprising students to jump right into AP-2 prior to AP-C, which means we would likely need to add yet another physics course, AP-12, as a first-year course for those students who want to take AP-C as a second-year course.  As you can see, this gets complicated in a hurry.

The bottom line — this change is going to take some time and require an overhaul of our entire science program and sequencing.  College Board, we need a timeline, we need details, and we need sample exams.


#2 Eliminate the NY State Regents Physics Exam

As a teacher, I want as much information as I can get about my students.  I use assessment to plan instruction. I use assessment to grade. I use assessment to let me know what I’m doing that’s working and what needs refinement. The current physics Regents exam and curriculum, however, doesn’t meet my needs for a culminating final exam, nor do I feel it adequately assesses my students’ understanding of physics.

The exam is largely a test of how well you can use your formula sheet (known as the reference table). If you can write down "givens," "finds," and pick a formula, you can plug and chug your way to a fairly high score without demonstrating true understanding. Not only that, but typically this is the last exam given after a week of exams, and in some cases it actually falls on a date after our school’s graduation. Most of the students taking the course have already been accepted into college, don’t need to pass the course to graduate, and therefore have no vested interest in doing well on the exam. Yet our department goals focus on students scores on this exam.

Further, topics included in the curriculum are addressed at inconsistent depths. Mechanics coverage is adequate, but electricity and magnetism, the precursors to so many aspects of our daily lives in the 2000s, is quite inconsistent. Students learn basic electrostatics as well as series and parallel circuits, then move into fundamentals of magnetism and basic EM induction. However, past exams indicate VERY few magnetism questions… less than one question every two years! Waves are introduced, leading into optics, but optics is quite incomplete. Lenses are not addressed, but refraction and diffraction are (although only qualitatively).

Most disturbing, however, is the final unit of the course.  Where you would expect to introduce basic atomic / nuclear physics and applications, the curriculum dictates a study of the Standard Model.  Not only is this topic inconsistent with learning "fundamentals" first, but the level at which it can be taught with the students’ background to this point in the course leads to rote memorization of a few facts and learning to copy answers off the formula sheet.  Teaching for Understanding?  Not a chance.

My wish for 2011 would be to see the state eliminate the Regents Physics exam, a consideration that has been rumored in light of state budget issues. There are plenty of standardized exams already available if we see a need for comparing students across classes, districts, and regions.

Instead, allow us more freedom within our districts to differentiate to student needs and interests.  Of course, fundamental concepts need to be covered in an introductory course — mechanics, energy, E&M, waves, atomic physics, and so on — but within these core areas, give me the freedom and time to focus on student needs and interests appropriately. Are the students excited about projectile motion? Let’s take the time to go further, learning how to apply concepts to real-world situations, making predictions, verifying, and including real-world parameters such as wind and drag. Students want to know about relativity and special relativity? Take some time to explore time dilation, length contraction, space-time, and point of view. Students are excited about electronics — expand E&M to include more than just resistive circuits… introduce diodes, transistors, integrated circuits, even design and processing!

There are so many areas students are interested in. Let’s eliminate an unnecessary exam that creates excessive paperwork, wastes money, and provides minimal qualitr5fy information about students while simultaneously providing teachers the opportunity to differentiate while encouraging engagement and enthusiasm.  In addition, eliminating the exam would provide an inviting avenue to replace our school’s current Regents Physics course with AP-1 physics, which is being designed to allow time for deeper exploration of selected topics.


#3 Offer AP-C Mechanics Exam in Winter

Yet another wish for the College Board. I teach AP-C physics (both mechanics and E&M) as a year-long course. Roughly 80% of AP-C students in the country take only AP-C Mechanics. Therefore, they spend the year preparing for their single exam in the spring, which they take as soon as they complete the course while the material is fresh.

The 20 percent of AP-C students taking both mechanics and E&M exams take the exams back-to-back on the same day, with a couple minutes of breather between the tests. They are therefore at a disadvantage because their mechanics course ended several months earlier — the material isn’t as fresh.

I would love to see the College Board offer a winter AP-C Mechanics exam, allowing us to complete this exam while the material is fresh in students’ minds before moving into E&M. Further, this would benefit students who are on waiting lists to the most prestigious colleges… a 5 on the AP-C Mechanics exam could help set them apart from other applicants, and results could be available in time for colleges to use the information in their final decision-making process.

College Board, please consider offering the AP-C Mechanics Exam in the winter.  (Yes, I know this is not likely to happen due to the cost incurred in creating another exam and scoring it, especially given the small number of students who would take it, but I have to think there’s a way this could be offered in a digitized format to protect exam integrity and reduce costs.)


Will It Happen?

There you have it, three wishes for administrative physics exam changes in the year 2011. Are they likely? Some more than others. I believe we will see more information from the College Board about the AP-B redesign, but I’m not holding my breath for any promised dates. I don’t believe the College Board sees any issue in the timing of the AP-C Mechanics Exam, so the first step is to at least communicate this desire. As for elimination of the Regents Exam, If state budget funding does push this to fruition, I believe there’s a strong chance the AP-B course split may push this issue on its own, although, once again, timing is uncertain.

What do you think? What changes are you envisioning in the coming year?