Assessing student learning online | University at Albany

What to Consider When Assessing Student Learning in an Online Course

At a few key points in an online course, our students submit projects, papers, reports, or exams. As we review and grade their work, our reactions can range from pleasant surprise to concern and confusion. All too often, we are confronted with unexpected and uneven student work, some of which falls well below our hopes for our students’ learning. When our students’ work is disappointing, the best response is to look carefully at our own practices and ask ourselves how we have approached assessment.

Less productive approaches to online assessment

Approaches to assessment can take many forms. When we teach online, one typical approach is to have students create digital posters, video, or audio with the latest fun, digital tools. Sometimes we decide that students should engage in a variety of assessments (one poster, one video, one podcast) to keep things exciting for students. Another approach to assessment is using what we inherit from a previous instructor or using what comes with a textbook we have adopted. And sometimes we simply use our existing online tests or essay assignments because they’ve worked well enough in the past.

The research suggests that these approaches to choosing our assessments may actually be the reason that our students’ work falls short of the mark. Why? Because we are making assessment decisions without a coherent plan. Assessments that might feel fun for students, convenient for us, or simply easily available won’t necessarily be aligned with our goals for student learning. In other words, they may not provide the right experiences to really help students learn and develop in the ways that we’d hoped.

Principles of Productive Assessment Design

To design the right assessments for our online courses, we can use research-based principles and practices that lead to greater student success and a more satisfying teaching experience.

Assessment design is productive when we shift away from the notion of assessment of learning to assessment for learning.
Assessments in a course must be developed together to ensure that they work together to move students to the learning that is articulated in our course goals.
Assessments should be described in a transparent way to clarify to students (and to the instructor!) the purpose of the assessment, what knowledge and skills will be drawn on to complete the assessment, and what successful attempts look like.
Assessment in online classes requires clear, consistent, and redundant communications about assignments.

Putting these Principles into Action

When we think about assessment for learning rather than assessment of learning, we ask new questions: “What assessments will help my students learn?” and “How can I help students see the learning value of my assessments so that they engage as fully as possible with that learning experience?”

When we don’t ask these questions, problems often arise. Let’s take the example of an instructor who wants students to create an Instagram account and post a short video three times during the semester to demonstrate different key concepts in a course. This might initially seem creative or interesting, but these assignments may not help students learn and they may result in student confusion and frustration. What about the instructor who uses another instructor’s test bank questions? This may be a convenience for the instructor, but if those test questions don’t tap the growth and change in thinking that the instructor is hoping to bring about in her students, the exam will exasperate the students and result in disappointment for the instructor.

Research on assessment suggests that using a carefully crafted assessment plan and transparent assessment descriptions help instructors focus on the key questions of what it is they want students to learn through their assessments and how they can motivate students to engage in these assessments in transformative ways.

An Assessment Plan

An assessment plan allows us to consider what assignments will work together, iteratively, to help students attempt and practice the learning that our course objectives have targeted. A good assessment plan has two key qualities.

First, the assessments should create a coherent experience for students. Often a course is studded with assessments that were not designed to work together or that require our students to do very different kinds of work. When students are confronted with three or four different kinds of assessments, they don’t see how each assignment builds across the course and their response is a mixture of fear and resentment. This experience becomes even more frustrating in an online course: students who engage in very different kinds of work must continuously learn new skills through the online interface of our course, putting a greater burden them (and us!) to communicate new resources and supports as each assignment requires distinct skills and tools. When assessments differ from one another or don’t work together to guide students toward the development of key skills, students see assessments as hoops to jump through rather than meaningful learning experiences. On the other hand, when assessments allow students to attempt and reattempt similar or related skills, students put effort and energy into these assignments and can build skills across the online course. The research suggests that predictability is a key to an effective online course, and that includes predictable, coherent assessments.

The second quality of a good assessment plan is that each assessment aligns with our goals for student learning. This may seem simplistic or obvious, but it is not unusual for instructors to design assessments that, while meaningful or engaging, have little to do with the course goals for student learning.

Your assessment plan is more likely to have these qualities when you sketch out all your assessments at one time and look for alignment both between assessments and between assessments and your course goals.

An example of an assessment plan

(from the course syllabus of a Human Development course)

Our course is organized around four learning sequences that map onto four developmental stages: infancy, early childhood, middle childhood, and adolescence. In each sequence, we will use the research and theory on physical, cognitive, and social-emotional development to analyze real problems and situations.

At the end of each course sequence we will work on a case study that allows you to apply the work we’ve been doing to a complex case that can be approached in many ways. After discussing the case study on discussion board, you will write your own case study analysis of this case in a five-page paper.

These four case study analysis papers are weighted differently, with each subsequent paper being worth more weight than the one before it. I do this because I know you will be developing your analytic and writing skills across the semester.

I will also have you use the feedback you get from me to make a plan to improve the next case study analysis paper. Throughout the learning sequence leading up to the paper we will be doing activities and homework that will help you learn to apply course material to realistic situations: this way you will build the skills that you need to write a good case study analysis paper.

These papers are aligned with course goals 2, 3, and 5 which have to do with analysis of real-world situations using principles of human development, application of these principles to creating solutions for real problems, and articulating your thinking in writing.

A Transparent Assessment

A transparent assessment is a rich description of an assignment that makes explicit to students the purpose of the work you are asking them to do, the steps to success, and ways they can evaluate their work. You might worry that providing this kind of detailed description will make the work too easy for students because they won’t be trying to figure out how to complete the assessment. Alternately, you might worry that student creativity will be suppressed when you detail the kind of work you are looking for in an assignment. In fact, students do their best work and work hardest when they are not struggling to guess why they are doing an assignment, what that assignment entails, how to proceed, and what successful work on the assignment might look like. This is the case in any class, but in an online class where students have to take extra steps to communicate with you about an assignment, it is crucial.

When you take the time to fully articulate both the why and the how of an assignment, you can also start to see more clearly if the assignment should be part of your assessment plan. And you can start to see the skills students will need to develop to be successful on the assignment. This can help you plan online discussions and activities as well as homework assignments, meaning that a transparent assessment will motivate and support your students and will also support your own teaching online!

Below you will find the five key elements of an assignment description (purpose, skills and knowledge, steps for success, criteria for success, and submission details) that effectively guide students into and through the work of an assignment. After each element is explained, you will find an example (from a Human Development class) where appropriate.

Transparent Assessment Elements - Descriptions and Examples

Purpose section description

Students don’t always see the value of the assignments we give them. When students see the intrinsic value and purpose of an assignment, they approach the work thoughtfully rather than doing the work superficially as a hoop to jump through. This means we need to explain to students how the assignment will help them learn in our class, will help them be more developed college students, and will help them in their life outside of the university. When we, as instructors, ask ourselves how the assignment is connected to the work of the discipline and to learning in a broader way, it also helps us make explicit the kind of thinking and work that the assignment requires.

Purpose section example

This assignment asks you to draft an analysis of a case study of a child using two key frameworks in the field of human development. One of the goals of our course is not just to learn about theories in the discipline of human development but to use them to support the development of children. And whether you go on to work with children or just to live as a caring adult in a world full of children, this assignment gives you an opportunity to practice observing, analyzing, and responding to a real situation in which you can play a crucial role in a child’s life. This careful and thoughtful work will also help you hone your critical thinking skills, which will contribute to your overall success at the university and in life.

Skills and knowledge section description

Students don’t automatically see the connection between the preparatory work they have been doing in a course and the assignment that is meant to draw on that work. Make it clear to students what skills they have begun to learn in class that they will continue to develop through the assignment. Also point students clearly to the content they will be drawing on to complete the assignment: these would be specific concepts, frameworks, or models that they will use. Refer students to the specific activities they’ve done and feedback they’re received that they will draw on to be successful on the assignment. Students will be able to gather together previous work and thinking and apply it to the assignment when you help them make these connections.

Skills and knowledge section example

As you do this assignment, you will apply the theories we’ve been studying in the last 4 weeks. These theories are family system theories, Bronfenbrenner’s ecological systems theory, and Tatum’s theories of identity development.

In discussion board debates, you have been practicing applying these theories to scenarios. You’ve used the theories to analyze scenarios to identify problems and propose solutions. In this assignment you will analyze a longer and more complex case study of a child and her family.

You will draw on the skills you are developing to analyze real life situations using these different theories. You will also be practicing how to use two theories together to analyze a case.

It will be helpful for you to review both the individual feedback I’ve given you on your “Scenario Reaction” Brightspace journals as well as the global feedback videos I have made in response to discussion board debates: you can use that feedback to think about how to improve your analytic work for this assignment.

Steps for success section description

Much of what we do as experts is work we do automatically. We don’t always realize that the work we do and the work we ask students to do involves preparatory steps, action steps, and metacognitive steps. Make these steps explicit to students so that they can make a successful attempt at the thinking and the work you want them to practice and demonstrate. Specify mistakes that they should avoid. Rather than spoon feeding them, you are creating an even more rigorous and realistic assignment when you describe fully the steps required for success. And students will be more enthusiastic and productive when they know what the assignment actually entails.

Steps for success section example

Preparatory steps:

Read this assignment description carefully and note where you have questions so that I can answer them in Prepare for Case Study discussion board to help everyone understand the assignment fully.
Reread your writing about family system theories, Bronfenbrenner’s ecological systems theory, and Tatum’s theories of identity development and note where you drew connections between those theories and where you showed differences between the theories.
Reread my comments on your “Scenario Reactions” Brightspace journals and my suggestions for ways to improve your analyses. Choose one suggestion and commit to using that in this assignment.
Read the case study “Keisha” at least twice. The first time, read for the gist: learn who the “characters” are and to get a sense of the events and timeline. The second time, read analytically, identifying what you think are the key problems in the case and then considering how the theories listed above can help you more fully analyze those problems. Make notes to record these new ideas.

Action steps:

Focus and deepen your analysis by creating a two-part paper.

The first part is your analysis of the case. In this part, you will identify what you consider to be the major problems faced by the child or family featured in the case and then analyze those problems using human development theories.
Analysis here means using a theory to see more of the problem, to see particular factors of the problem, or to connect the problem to other contexts and issues. You’ll need to fully explain those theories as you use them to pick apart and more fully conceptualize those problems.
Please don’t cite lectures or PowerPoint presentations: I want you to use your writing and your thoughts to demonstrate your understanding of the course readings and how they apply to the case.
Use at least three details from the case as evidence that the theories you identify in your analysis are actually at work here. So, for example, let us say that you feel a central problem in the case is that the child is struggling to play productively with peers because she has different cultural concepts about play.
You might decide that a specific theory about cultural psychology can help us understand that struggle. Be sure you fully explain your understanding of that theory and then apply it to the case. Then you might use two short excerpts from the case to show how the child in the case does or does not do things that the theory predicts or suggests.
Don’t spend lots of time in this section retelling the case. Your job here is to show how events in the case can be more fully understood using development theories.
In the second part of your paper, you will suggest one or more solutions to the problems you have identified in the case.
These solutions should stem from your analysis of the problems and should also be shaped by your understanding of the theories you are using. Use three well-chosen pieces of evidence from the case to support your claim that these solutions will support the child in the case.
Also in this second part, don’t just talk generally about your solution: propose a detailed, concrete, plan for action. The key here is detail: could a parent, teacher, or social worker follow your plan, and would all her questions be answered if she read your case study analysis?
Again, your plan should be fully grounded in the theories you are using in your analysis. In this section, you will also discuss what the possible consequences of your solution/s might be: I want to hear both the things you hope to have happen and the things that might not work so well.
It may help you if you write to an adult in the case. This will help you remember to be analytic and structured in your writing. This fictitious reader will need you to spell out the problems and the solutions in a well-organized way.
She needs you to explain what is happening in the case using theories of human development in a clear way, and in such a way that she can really see how developmental principles are at work in this case. Your evidence should help her think (for example) “Oh yes I do see how cultural mismatch really is going on here!”
If you think that this will help your writing, why not start out your paper this way, by addressing the adult who you’d like to read your analysis? Your writing will still be formal, but you will be focused on explaining what you understand in clear and detailed language.

Criteria for success section description

Assignment descriptions should articulate for students (and for ourselves) the necessary parts of an assignment and the levels of performance or achievement that they should aim for with each of these parts. Including a grading rubric as part of your assignment helps you think fully about what you are asking students to demonstrate in an assignment and they can be used by students to monitor their work. Designing rubrics takes time and energy, but it is a worthwhile investment because it helps clarify the assignment for you and your students.

Submission details section description

It’s easy for students to spend too much time focusing on formatting and submission worries rather than using their energy to do the meaningful thinking an assignment requires, so be sure to contain that information for them at the very end of your assignment description. Explain the following kinds of details to students: the style guidelines they should follow; font size; margins; file type; submission dates for drafts, feedback, final drafts; and where and how to submit their work on Brightspace. Be specific and detailed regardless of the kind of assignment you are giving students (papers, presentations, videos, artwork, etc.). Having all these logistical questions answered will make them feel comfortable and help them focus on the thinking they need to do, not its packaging.

Submission details section example

Your case study paper should be 5 pages in length. Use APA format, double-space the document and use twelve-point font.

APA style is used by scholars who work in psychology, education, and other social science fields. You should use it when you cite others’ words or work in your writing. Details about APA style can be found on the form “Writing in APA Style” on Brightspace in the “Writing Tips” folder.

Type up your references in a bibliography at the end of your case study paper. Your assignment should be in Word. Submit your work through the assignment link for this case study paper in Brightspace by October 15 at 11:59 pm.

Clear Communication about Assessments

Communicating about assessments in an online class actually begins before the class starts! Students need to know what to expect when they engage in your online class. Sometimes students are not aware that they will be expected to fully participate in synchronous class meetings where they will do work that prepares them for assessments; sometimes they don’t realize that they will be doing homework and short weekly assignments in an asynchronous class.

Sending a welcome letter to students a week or two before the start of the semester can help them begin to prepare for the course and for your assessments. A welcome letter is a short, motivating invitation to your online course. It provides an overview of the most exciting and interactive aspects of the class and the meaningful assessments you have designed for them. A welcome letter also provides an opportunity to begin to create connections with students: you can ask students questions that help you get to know who they are and what they might need to succeed in your course. We encourage you to visit our resources on welcome letters for asynchronous courses or synchronous courses.

Communication about your assessments starts with your welcome letter and continues throughout the semester. Students need to encounter information about your assessments repeatedly in an online course. In a face-to-face course, students are with you for about three hours each week, so they have many opportunities to ask you questions about upcoming assessments. Recreate these opportunities for getting information and asking questions about assessments by using tools like email, announcements, short videos, weekly check-ins, and online office hours to supplement your detailed assignment descriptions.

Finally, plan time for students to read and work with your assignment descriptions by assigning them as homework. Students in an asynchronous class can share their ideas about an assignment on a discussion board or in a journal assignment: ask them to describe how the assignment will tap their strengths, how it may challenge them, and their plan to allot time and energy to the assessment. In a synchronous online class, students can discuss their ideas in breakout rooms and then share out some key ideas to the whole class.

Resources

Anderson, L. W., Krathwohl, D. R. , Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R. , Raths, J., & Wittrock, M. C. (2014). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives (International Edition). Pearson.
Darby, F. and Lang, J. M. (2019). Small teaching online: Applying learning science in online classes. Jossey-Bass.
Walvoord, B. E. (2010). Assessment clear and simple: A practical guide for institutions, departments, and general education. (2nd ed.). Jossey-Bass.
Winkelmas, M., Boye, A., & Tapp, S. (Eds.). (2019). Transparent design in higher education teaching and leadership: A guide to implementing the transparency framework institution-wide to improve learning and retention. Stylus.