Flipped Textbook

I'm planning the interface for the rubric system. A rubric helps assess exercise solutions. Suppose there's a book for a writing course. It has an exercise on writing a business letter. The rubric for that exercise might be:

• Business letter layout
• No spelling errors
• No grammar errors
• Proper greeting
• Formal tone
• Reason for the letter is clearly expressed
• Next actions are identified

Each of these things is a rubric item. So a rubric is a collection of rubric items.

An item is a three-part thing:

• Item name
• "Correct" message
• "Wrong" message

The name is the name of the concept, like "Spelling." The Correct message is a sentence saying that the student handled the concept correctly, like "You got the spelling right." The Wrong message is a sentence saying that the student made mistakes, e.g., "Work on your spelling."

Each book has its own "rubric item set," that is, all of the rubric items for all of the exercises. A rubric item can be assigned to more than one exercise. For example, the writing textbook could have another exercise, writing a personal letter to a friend. The exercise would have a rubric. It might be:

• Personal letter layout
• No spelling errors
• No grammar errors
• Informal tone

Two of the rubric items are the same.

It's important that rubric items be reused, rather than duplicated. Here's what I mean. Here are the two exercises, and the rubric item set:

"No spelling errors" and "zero spelling errors" mean the same thing. There are two rubric items for the same assessment criterion.

The problem is that computers are stupid. My server couldn't be sure that these two things are the same. It would have to treat them as different items.

But supposed you did this:

There's only one rubric item for the spelling assessment criterion. This means that the FlippedTextbook server could tell when a student was making the same error across exercises. It could email the student, and recommend some optional material on spelling checks. This is just one of the advantages of having no duplicate items.

Now that's out of the way, here's what I'm thinking. One approach is to make authors set up the rubric items first, and only then start assigning them to exercises. I think not.

Authors would prefer to assign rubrics as they're creating exercises, not as a separate process. The rubric item set would develop organically as authors write the book. Authors would manage the rubrics as they're creating exercises - add a new rubric item, reword another, merge two, etc. For a good workflow, authors shouldn't have to stop working on an exercise, switch to a different rubric interface, make changes, switch back to the exercise interface, and keep working. That would quickly become annoying.

As authors work, they want to quickly find the most relevant rubrics for an exercise. It would help if authors could group rubrics into categories. For a programming book, categories might be Variables, Control structures, Layout, Decomposition, and such. Want to add a rubric item for proper indenting? Go to the Layout category.

Each rubric item would be in a single category. Multi-parent hierarchies are confusing.

Authors would create the categories as they go. They can rename categories, add new ones, move items between categories, etc.

There might be, say, 100 rubric items for a book. So a search function would some in handy. Click a Search button, type in a word, and the rubric manager would highlight matching items. Maybe even with synonyms, if I can work that out.

Here's an overview of the interface:

The area at the top is for creating the exercise. Below that is the rubric for the exercise, that is, a list of items. Below that is the rubric manager.

The rubric manager might look like this:

The items for the book would be on the left, in an accordion widget. Here's the jQuery widget:

Each section would be a rubric item category. Click on a category, and it expands to show the items in the category.

Many commands (top right of the interface - this is just a sample) act on several rubric items at once. E.g., Merge (replace two equivalent items with one). The user would select each rubric s/he wants to change, and they would be collected in the basket. Clicking on the button for the command would act on all the rubrics in the basket. Clicking on an item in the basket would remove the item from the basket.

Some commands only act on a single rubric, e. g., changing the wording. That would be done in the accordion directly. There would be some small buttons on each item in the accordion. For example, a Reword button, and a Basket button.

To build a rubric for an exercise, click on each rubric item in the accordion. The items would move to an area below the exercise. Click on a current item to remove it.

This interface would be part of every exercise. It would also be available separately from any exercise, for quick changes an author might want to make.

What do you think?