Information literacy assessment in 25 easy steps.

Today I humble-bragged about our instruction department working together to assess over 230 samples of student work from 30 sections of first-year English composition. So, how many steps did it take to get to this moment? And what comes next? Let’s take a look. I’ll give a step-by-step explanation of our assessment process, leaving out details about the content of the lesson and focusing on what we did and how.

Full disclosure: planning, scheduling, teaching, and assessing at scale is definitely a team effort. I am not responsible for every step below, but I did most of the wrangling (and expediting) after the sections were taught. Oh, and I am a quick draw with a mail merge.

  1. Create a shared lesson plan for a 75-minute one-shot instruction session, complete with learning outcomes, a hands-on learning activity, and a built-in active learning assessment.
  2. Print extra copies of lesson plan, class handouts, and other required materials.
  3. Review lesson plan and practice lesson with instruction team as needed.
  4. Set up Google Form to collect student responses.
    • The Google Form asks students to answer questions related to the class activity.
    • The Google Form does not ask for any identifying student information (we don’t collect student names, instructor names, librarian names, section number, etc.).
  5. Teach! (I’m not going to go into details of the scheduling but I’ll say that everyone in our department teaches at least a couple of sections.)
  6. During hands-on learning activity, librarians direct students to answer prompts in a Google Form.
  7. Collect all responses from all students in every section using same Google Form.
  8. After all sections have been taught, export all student responses to an Excel spreadsheet.
  9. Clean data, removing duplicates and blank submissions.
  10. Finalize scoring rubric, determining clearly defined performance indicators for exemplary, satisfactory, and unsatisfactory performance for each of seven criteria.
  11. Set up Word template to create hard copy student work samples for scoring.
    • Ensure each sample is numbered.
    • Ensure template includes spaces for initials of three reviewers.
  12. Mail merge Excel data into Word template, creating individual student work samples.
  13. Edit individual documents, removing blank pages.
  14. Print 232 student work samples.
  15. Send meeting invites to librarians for scoring sessions.
    • Reserve five hours in the same week.
    • All librarians are asked to commit to at least two hours of scoring.
  16. Reserve conference room for scoring sessions.
  17. Set up Google Form to collect scores.
    • The Google Form collects the number of the work sample and scores for each criteria.
    • The Google Form does not collect any data about the librarian scoring the sample, course section, or student.
  18. Administer scoring sessions. As a group:
    • Review rubric.
    • Review scoring procedures.
    • Establish norms by scoring a few student work samples together as a group.
    • Allow librarians to work individually, entering scores in Google Form.
    • Librarians write their initials at the top of each student work sample to ensure to that three different reviewers provide scores.
    • Student work samples rotate among reviewers as needed.
    • Expediter controls workflow using designated baskets for completed forms.
  19. After all 232 samples are reviewed three times, data is cleaned.
  20. Determine consensus scores for each criteria (e.g., if two reviewers give a score of Satisfactory and one reviewer scores as Exemplary, the consensus score is Satisfactory).
  21. Any scores that are unclear are sent back for additional review.
  22. Use consensus scores to create an overall assessment report that details student performance on each criteria of the assessment.
  23. Use assessment results and departmental discussions and observations to drive curriculum and assessment improvements for next semester.
  24. Share assessment report with Department Head, internal administrators, and University administration as needed.
  25. Archive report using standard knowledge management practices to ensure its discoverability.

 

And that, my loves, is how you “do assessment” in the library world (or my library world, anyway).