Wednesday, March 27, 2013

Quixotic Grading Dreams

This semester I am teaching an upper level graduate course, but the enrollment is quite high as I hadn't taught it for several years. I have no TA or grader, and I am very, very grumpy.

In the school system that I went through, homework for a grade was basically nonexistent -- I stopped having homework assignments in 4th grade. After that, there were lectures (you needed to take good notes) and textbooks to learn from. For math and physics, there were books of problems and you were supposed to go through them and practice in order to do well on the exams. The same thing in undergrad: there were traditional lectures and discussion sections, the latter were where the TA would do typical problems with us; still, there was no homework, just books of problems, and it was up to us to study and practice in order to do well.

When I came to the US to grad school, I actually liked the fact that there was homework in graduate courses; turning in homework weekly or biweekly forced me to study the material at a regular pace (I couldn't goof off and then cram before the exam), which meant that preparing for exams was trivial and took no time at all. I also liked that there were term papers and projects, which I had never had before.

Then I became a professor and started having to grade.

What I dislike is that, unless homework is graded, students will not do it at all. Especially at the undergraduate level, students have beautiful, colorful textbooks, full of examples and problems, yet no student (at least none that I know of) would even think of doing anything other than the problems assigned for homework. This is totally mind-boggling to me. I had a student a couple of years ago come ask me what he could do to prepare for the next exam, as he hadn't done so well on the previous one. I said the minimum is to read the lecture notes, his own in-class notes (students here take very few notes, which is another thing I just cannot wrap my head around; most sit and listen with their arms across their chests), read the book, and go over all the homework problems in detail, asking himself what it is that each problem actually emphasizes. He said he wanted more, and I said sure, by all means, do all the problems in the textbook, but he didn't find this to be an appealing option; he wanted me to select the extra problems for him to do.

I love assigning term papers and projects, and I don't mind grading them.They are substantial chunks of work, requiring students to synthesize multiple concepts and deliver a meatier result, but they are of most use in upper-level undergrad or graduate courses. But the week-by-week homework is really just for the students' benefit. So many of them copy from each other, or from previous generations if you recycle any problems, that it's pointless to grade this work. Sure, you can assign very little value to each homework assignment, but someone still has to grade the whole thing. Either some poor graduate student spends many hours every week grading undergraduate homework assignments or I have to do it. No matter how little homework counts for, the time needed to grade t is the same.

This is what I would ideally like to see regarding grading (I am talking about physical science STEM fields here): I assign homework every week, there is a due date, right after the due date I post the solutions. Most homework assignments are not graded, but students take the responsibility for their education and do them anyway because it's good practice of their craft. Or, for those who just want the grade, they do the assigned problems because it's good practice for the exams, and the exams count for a lot. Most homework would fall under this category, i.e. the traditional HW where students work some problems on a piece of paper; for undergraduate courses, this would cover nearly all assignments, depending somewhat on the class. Then there would be a few assignments, e.g. involving programming or calculating something complicated, that would be turned in and graded (the percentage of these would be higher in upper level undergrad or grad courses). I could also assign some larger projects and/or term papers that I would collect and score, and they would count for a nontrivial percentage of the final grade.

The point is, I don't want someone to have to waste time on grading trivial practice problems that the students should do in order to gain a basic level of proficiency, problems that can be (and are) easily copied and thus count for little. I would like to see more student ownership of their own performance in the course, and them being ready to put in the time to practice, without necessarily every minute of their time actually counting for a grade. (I hate the "But I worked so hard! I put in so much time!" pleas for a higher grade. I want to see what you learned, not just that you put in some time.)

I actually tried doing what I wanted to do this semester in my large graduate course. I told the students that  most HW assignments would not be graded, but that they should do the problems anyway in order to be ready for the exam. I said there would also be small projects and a paper, and those would be graded. You know what happened? No one was doing the homework. Referring to examples from homework in class was pointless, because students hadn't looked at them. So I went back to having them turn everything in, which I then have to grade, because this is a high level graduate course and the department gives no TA's or graders at that level. I am so darn grouchy right now.

Maybe I should have stuck to my guns. If they all do poorly because they don't practice, so be it. I have tenure, teaching evaluations don't matter much any more.

Has anyone tried and succeeded to somehow reduce the amount of grading (regardless of who does the grading) without affecting student learning? Any tips on how to make students put more effort into acquiring proficiency without every picosecond of their invested time having to count towards a grade? Or am I just being the Don Quixote of grading?


Anonymous said...

I think as long as you have a mid-point exam early enough in the semester that basically shows them they cannot pass this class without doing the HW, they'll get the picture. You could even pull all the questions from the first month of HW to make a point. As long as the sum of subsequent exams and graded work can get them back to an A, I think it's a good plan if you don't like grading.

I tutor in a STEM field where most of low-level undergrad HW problems can be found online. The HW doesn't count for much (2% of their grade) in a huge class where the answers are submitted online (so no grading), and most kids just google the answers... Until they fail the first exam. And then they all show up to tutoring for help and will start studying and doing practice problems on their own. They do appreciate it when you flag extra practice problems that are particularly useful. At least that's been my experience at Giant State University.

Alethea said...

I had a math class in undergrad where HW didn't count. To emphasize that we REALLY needed to do it on our own, the professor gave 3 tests, instead of just a mid-term and a final. This way, we would see early on that doing the HW was a must.

TS Dibble said...

In teaching a junior level physical chemistry class, I assign ungraded Exercises and graded Problems. The Exercises are are relatively trivial, single concept (often plug and chug) questions. They are listed on the same handout as the Problems, which are meatier questions (mostly my own). The Problems reflect the level of difficulty of exam questions.
I tell the studens they need to get to the point of being able to do the Exercises automatically if they are to do well on the Problems or exam questions. This may work for some students, but is not working for many of them.
In an atmospheric chemistry course for seniors and first year graduate students I have a similar structure, but answering the Problems often explicitly or implicitly requires the answers to the Exercises. When students realize this, they at least do those Exercises.

inBetween said...

Oof, my field is so different that I just don't grade at the undergraduate level. And grad level, it's minimal.

I have been meaning to comment on your last blog post - thanks so much for writing about that book. I'm really, really happy to hear that the press representation wasn't the whole picture. I feel better about the author. She definitely comes across in an unflattering light in most of the press.

Anonymous said...

I teach undergraduate chemistry courses. Initially, I assigned homework from the book with the solutions manual readily available. I did not grade the homework, just told students they needed to be able to understand the concepts if they wanted to succeed on exams.

After 2 years, a student said that if I would just take up homework, then they would do it in a timely fashion. I felt really conflicted because I think the students need to learn personal responsibility, but I also wanted to motivate them. I also wanted them to keep their problems to study.

I started taking up one randomly chosen homework problem a day. I grade it for completion (the students must show work). The exam scores increased by an average of 10 points.

Anonymous said...

Since moving from the UK to the USA I have also noticed that students don't do any work unless it is worth points. I have several theories about this:
1) If every other class assigns work worth points and you don't, there is a lot of incentive for the students to spend more time on the other professors work and not do yours.
2) When I was an undergrad in the UK, I didn't work particularly hard on the (ungraded) problems to begin with. The only grade that counted was the end of year exams. Then I learnt through bad exam grades that I would have to work harder and by my final year I was taking a lot more responsibility for my learning. But in the USA, this message is much less clear. As many US profs count HW grades, there is no repeated lesson to the student that they need to do the work even if it doesn't count. They might do badly in an ungraded HW class one semester but then go back to graded HW classes the next and revert to doing only the bare minimum.
3) University exams in the USA (even at graduate level) are pretty easy. What incentive is there to work outside of class when your exams are multiple choice (or as I have heard in some classes, multiple choice take homes with 3 resubmissions and incorrect answers indicated each submission...)

Alyssa said...

I agree with the idea to put an exam earlier in the semester so that they learn that doing the homework will help them get a better grade on the exam.

We are hosting a post-secondary education conference this summer, with the theme of "less is more" - email me if you're interested, and I can also summarize some of the interesting techniques people are using!

Anonymous said...

I like the randomizing component -- one random problem out of 10 will be graded.

You could do this along with having students (randomly) grade each other's problem sets: choose the one problem to be graded, go over the correct solution in class, discuss where partial credit happens, have students grade each other's HW, pass the whole thing to you and you record the grades. Boom, done.

Anonymous said...

I like the random collection idea. As another idea, could you just collect the homeworks as you go along and not hand them back until the end? If homework only counts for a small fraction of your final grade, at the end of the semester you would only grade homework for the few students that might actually have their overall grade change which should be a small percentage.

andi said...

we did the randomising component once - one question in the homework would be marked, rest ignored (but answers posted).

We stated repeatedly the choice was random (and it mostly was) but still the students spent a lot of time trying to work out each week which question we would mark (hardest, most number of steps etc).

gasstationwithoutpumps said...

I've posted on the "just scoring points" problem:

My solution is different. I don't assign problems or give exams, but have substantial projects due every week (design reports in an electronics course, programs in a bioinformatics course, … ).

This doesn't make the grading load lighter—indeed it increases it enormously, but all of it is stuff that is worth my time to give feedback on, not trivial little exercises.

My belief is that the point of teaching at a research university is to provide intensive interaction between the students and the research faculty—something you don't get with mega-lectures, MOOCs, robograding, TA grading, and all the other "cost saving" mechanisms that destroy the point of the institution.

Cloud said...

This was a graduate class??? I don't think any of my homework in grad school really counted. I honestly cannot remember if it was graded- or even given. All that really mattered was passing the qualifying exam at the end of the second year.

But more importantly, when these people make it out to "the real world"- whether that is a job in industry or a position in academia, these crutches that nudge them towards smart time management are largely gone. If I have to check in weekly to make sure someone is doing his or her work, that person has very little chance of advancing up the ranks. If students aren't ready to learn to take ownership of their own work and learning in graduate school, then you're probably doing them a favor in the long run to show them the consequences of that.

Alex said...

One colleague has an interesting approach: He provides the homework AND the solutions, and he doesn't grade any of it. Instead, he (sometimes) checks for completeness, but he also gives a quiz. The quiz is basically "Here's a problem from the homework, show me how to solve it, and explain each step." Showing the solution might just mean that they memorized, but explaining each step requires that they understand.

From what I've heard the results weren't spectacular, but they weren't any worse than what he'd gotten with regular homework, and it reduced the grading burden. Less grading burden, no less learning? I'm tempted to try it...

I suspect that some of the more progressive teaching and learning experts might have other prescriptions. However, I don't think there's any gimmick that gets around the need for practice, and the fact that humans often need a push to practice.

Ms. Scientist, Ph.D. said...

How about a check, check +, check - system? Basically, you check for completion and give a check for basic completion, check + if they've really given a lot of thought or provided a lot of their work, check - if the assignment is not completed. No check, of course, if nothing is turned it! We tell students that the checks/-/+ will be used in their participation score, or to 'bump them up' a letter grade if they're right on the edge at the end of the semester. Students then would have to take the initiative to check their answers against posted solutions. And it requires just skimming the pages. Hope that's useful!

Anonymous said...

I am also teaching a graduate course, and I give homework every other week or sometime in 3 weeks, just because I hate sitting with those paper to grade. I have found that the results are pretty consistant, which means students who are good, they put effort in each homework and receive top grades, and students who ignore what is happening in the class, they consistently are in the bottom pile. More homework does not get them better, it only takes away precious time for you on which you could be doing something interesting.

Funny Researcher said...

I have a simpler solution; saves professor's time and gets the students to do the homework.

1) give a home work problem for students to work on.
2) Collect all home work solutions done by the students in the class
3)Randomly redistribute the submitted home work to the students such that each students gets a home work sheet that is not hir's
4)Make the solution available and make the students mark their colleagues homework
5) collect the marked home work and randomly check few of the checked home works.
6) Note the scores and give the sheets back to the students.

I don't know how well it would work in practice, but in theory seems to be a viable solution. I plan to experiment with it some day.

Arseny Khakhalin said...

You can also:
1) Make grading extremely simple (one point for being on time; one point for each correct answer).
2) Make them grade each other's homework
3) Set it all on Canvas, and have it graded automatically.

nicoleandmaggie said...

My husband has tried pretty much everything-- homework quizzes, just grading for completeness, grading a random problem, and so on. The only thing that seems to get them to take it seriously is grading the full thing for both completion and correctness. That, of course, takes a lot of time, so he doesn't do it. The other strategies seem to have them take it partially seriously, but homework tends to lower rather than increase their GPAs in those settings. (Ironic, isn't it?) So right now he's grading based on completeness and they're somewhat doing the homework.

In my math classes, I argue they need weekly homework that's graded and I need a grader. So far that's been working. In my econ classes I assign fewer problem sets and grade them myself. That kind of kills the joy out of a few weekends for me during the semester. One of my colleagues has hired a grader out of pocket.

fizzchick said...

No great solution for this. The online homework systems have varying degrees of usefulness, but they do reduce your grading load to zero (and the better ones make the students compare conceptual answers, because each student gets different numbers). Last semester I had a grader who did each problem on a 2 point scale: 0=not done, 1= partially done/correct, 2=mostly done/correct. This is faster but still not ideal. This semester I just check who has homework done (small class, so it's quick to check off as they're doing group work or before class). The incentive is weekly quizzes, which are faster to grade than HW would be. However, some students just won't do it, even after I showed them the scatter plot of course grade vs # HW checks.

Anonymous said...

What worked for me in a grad level class was to assign smaller amount of homework very often (every single class period) and have the students hand it in the very next class period. I then graded one or two problems carefully and the rest for completion. By the second half of the semester the students got into the rythm of doing their homework and grading for completion was enough. Also, keeping the amount of homework relatively small and the deadline just 2-4 days away increased the rate of the homework completion as well as student participation in class as they were keeping up with the material. In the past, I used to assign larger homework sets with the deadline 2-3 weeks away. Many students did not start working on it until the day before it was due, and it just did not work very well.

nicoleandmaggie said...

Thought of another one-- in high school math, the teacher cold-called on people to demonstrate homework problems on the board at the start of class. I imagine that in college that could decrease attendance, however.

GMP said...

N&M, that's how most of my grade school was (at least math, physics, and chemistry). Someone was always on the board, doing problems in front of the class. They weren't homework problems as we didn't have HW per se; this was basically for a grade for the person on the board and practice for the rest of us.

But you are right, I cannot imagine it flying in the university setting here.

Alex said...

I've seen very progressive pedagogy types actually roll their eyes at talk of having students do problems on the board. I'm not sure if they have an evidence-based argument against it, or it just isn't hip enough.

GMP said...

Re students on the board: I think that students would probably complain that they are being put on the spot and/or embarrassed, and I can envision that some would complain that they didn't pay tuition to do the professors' work for them.
I might be getting too cynical...

nicoleandmaggie said...

As a pedagogical tool, it has many benefits, assuming you don't cold-call, and instead ask for volunteers. The main benefit is that it shows students how difficult it is to actually get up and teach. In my math classes I will often do the standard "Theory, I show an example, you do an example at your desk" then have a student who got the right answer come up and demonstrate. (For a twist, if nobody volunteers, I sometimes cold pick someone in the class to volunteer another classmate to demonstrate. That way the onus of cold-calling goes on the student.) But that's generally not homework, and doesn't help get people to do the homework.

Alex said...

Then don't call it "coming up to the board." Call it "Peer-modeled conceptual exercises" or whatever. People seem to dig buzzwords.

The students will still complain, but your Dean will side with you because you have a buzzword for it. And you have a ready-made Broader Impact statements. "The PI will use examples from this research for peer-modeled conceptual exercises in her STEM courses."

I wish I was joking.

profacero said...

I have this problem all the time. Best solution so far is: homework is for a "completion" grade -- it is either done or not -- .

I am not sure about anything but grading must be reduced.

Anonymous said...

If you can create an atmosphere of trust and tell students, for instance, that if they're stuck on a problem they can tag-team someone else, then you can often get them to think that working at the board is a good thing. In a grad class I think you could swing this. We are having "problem days" every few weeks in a grad class I'm sitting in on as a postdoc, and they're working remarkably well. In my grad school days, I remember setting up problem-solving time with my peers for algebraic geometry and dragging the prof down when we got stuck.

geek?!? said...

Um, call me crazy, but I did homework and problem-sets whether it was graded or not.
Actually, I went as far as to sit in on two classes I wasn't signed up for.
In one of the classes, I did all of the class assignments and almost did the class presentation until my partner flaked out and the prof wrote to me and told me to not do the presentation (the class grades the presentation and gives comments/feedback as part of their workload) because it was ridiculous for me to bear the load of an irresponsible student who was signed up for the course just so that the class could get that portion of the grade (she averaged out the grades for the class minus that presentation).
In the second class, I did all the mini-assignments AND the major paper; the prof marked my work, but gave me no grades except smiley faces.
No grades whatsoever for both classes, as officially, I didn't exist!

Anonymous said...

I was in a similar situation. I ended up making up homework/quizzes online that could be graded online automatically. It did take me a bit of work/time to make them online, but less than grading them all myself and more importantly it DID force the students to practice and read the materials assigned. I think it worked out very well, and now I have them developed for future classes such that I should only need to make minor changes as I update/change the class from time to time.

GMP said...

Anon at 8:37, can you tell us more about what you used to create HW and what you feel the limitations are on the type of problems and/or the course level for online assignments?

Anonymous said...

I have the same problem, enhanced by the fact that even when homework represents a sizeable amount of the final grade (30%), about 40% of the students copy from classmates and/or don't submit the work. I teach introductory organic chemistry (not in the US, and I am sometimes puzzled by these attitudes since during my studies, I never did any homework for a grade after I finished primary school), and my life has improved a lot since I migrated some of the homework to moodle (there are other similar platforms, such as claroline, but usually you just have to use whatever your school has chosen). I spent about two months without sleeping between preparing the normal course materials, grading conventional homework and exams and creating online exercises and quizzes, but the result paid off. Drawing molecules with ad-hoc software, saving each as an independent image and uploading it all to the platform is very time consuming (when I did something similar for a physical chemistry course it was much easier, since moodle accepts LaTeX) but at the end, you can create a bank of questions that can be randomized in very interesting ways, classified according to subject, difficulty or whatever characteristic you want etc. so that at the end, you can create a homework set that can be different for each student with 10 questions of this set, 5 of this other one, etc. After the fact, the computer grades it (I specially love this part ;) ) and you can show nice statistics about where students fail, what kind of questions they don't understand...
And it seems that the students like this system better too, since I get better completion rates from the computer homework than for the problem sets that they have to give me :S .

Anon 8:37 said...

GMP - I think the online solution that worked for me may not work for everyone. But I really like it.

The specific class that I commented on above (Anon 8:37) was a mixture of grads and upper level undergrads and they just were not reading the book thoroughly and really preparing for the weekly quizzes. So I made them open book online quizzes that forced them to do the readings and *think about the material*. Overall it really worked well both for me and for them. It was just another way to get them to do what many of us would automatically do - read the material thoroughly and think about it. Essentially my open book quizzes were homework questions.

For things that require calculations, I set those up as multiple choice answers, but I often will list up to 7+ choices so guessing just isn't going to work. if you want to get it right you really do need to do the work. BTW I do these as multiple choice because of the system our university uses - it can automatically grade those easier than other ways to input the answers but depending on your system and how much time you want to take you can have them submit answers several ways.

For those that are calculations, I then can go over how they should have worked out the problem in class or by emailing the complete answer (e.g how to do it) to them (depends on how much time I have to work with).

I also do not give them the answers to the questions until the due date has passed. This way the early ones are not able to share the answers with the late ones.But I time these homework assignments to my lectures so they are getting feedback pretty quickly each time so that the work is useful to their learning (in these cases I'm usually assigning short amounts of work every few days so that they are basically always having to keep up with the material but not doing what I would consider too much (however, I'm told that my classes are the most difficult in my dept, but I'm also told by many, that I'm the best instructor). In contrast, in some of my introductory undergrad classes I have huge databases of questions set up and they are just random. In these cases they get feedback immediately.

Overall, my methods increase the amount of time they are working outside of class and I see the students learning more. I also think it increases my workload, but if it is a class that I will teach repeatedly then in the long run it decreases my work load while also doing a really good job of helping them learn.

I also ask for essay questions online. I find that I can read and grade them faster because they are typed and I just really like working with computers and not having all those papers to grade and lose. It also helps me input their grades faster and thus I save time on actually calculating their final grade.

I really like having homework and open book quizzes and other exercises online to add to my lectures. I use them for both graduate and undergraduate courses at all levels. I think it helps me have more one-on-one interaction with my students, I think it helps them learn better and in some cases forces them to do what really good students would normally do on their own, and I think it is faster for me than the old method of grading via paper submissions in class.

I teach at a research university and have a hugely successful research program, but I care very much about my teaching and put a lot of work into it. I am able to do this by trying very hard to do less service and probably working too much! :)

- Anon 8:37

Anonymous said...

I did my student teaching last fall in a school that was trying to migrate to "standards based grading" and one of the components was that homework wasn't supposed to be counted towards the student's grade. Unfortunately, teachers were having a heck of a time getting students to do the "practice." On one hand, teenagers don't necessarily have the long-term vision to see that doing homework *does* count indirectly because it will help them score better on exams. On another hand, they've also been trained from a young age that the grade is what matters so the work that directly goes towards the grade matters.

Anyway, my points are that a) it's not just you and b) this is a lesson that they'll benefit greatly by learning. Grad students are more than developmentally capable of learning this lesson fairly quickly. I second the recommendation of lessening *your* grading load and incorporating self-assessment opportunities relatively early in the semester for the purpose of helping them realize that homework *does* matter, even if it doesn't go directly into their grade calculation.

(And as a side note... I'm also baffled by this concept of graduate students being assigned homework. My undergrad professors didn't even do that. It was up to us to figure out what and how we needed to study!)


gasstationwithoutpumps said...

In my main grad course, I assign weekly homework: either a computer program or a paper to write each week.

The grading takes a while, because I feel the need to give detailed feedback—that is my job as a professor. Students learn most from detailed feedback on work they have put some effort into. "Points" are useless.

I don't see any reason to assign "busywork" homework to make the students do the reading and don't see any point to grading such homework if it was assigned. Make the homework be something you care about, and the students will care about it more.