Just in Time Data

Just in Time Data

Over the past few weeks, we focused on Data-Driven Instruction: how to use data gathered over an extended period of time to form a detailed picture of each student’s achievements and needs. I’ve gotten a lot of feedback from teachers curious about ways to “dip their toes” into data collection or how to incorporate data into a more traditional classroom structure, so let’s look at one strategy for using Just-in-time Data to make lectures more responsive and interactive.


When we deliver information to our classes via lecture, we often wonder how much they are ‘picking up what we’re putting down.’ Typically, we find out how well a student has understood (and retained) information when they complete a graded assessment. But when the assessment comes days or weeks later, it can be hard to tell whether a student’s difficulties come from attention, comprehension, or retention. Also, finding out days later that a student (or group of students) has missed something is not nearly as useful as knowing in-the-moment.

Experienced teachers can “get a feel for the room” to tell whether a lecture is having an impact. This helps when a large portion of the class is lost, but it is very difficult to identify that the student in the middle of the third row started wandering off 7 minutes into the period. Another challenge is what Jennifer Gonzalez refers to in her blog (Cult of Pedagogy) as “fisheye”: a form of bias, where we tend to focus in on the students who are actively engaged and estimate the engagement and comprehension of the entire class based on a few students.

Quick Assessments with Socrative

There are plenty of low-tech ways to make real-time assessments (“touch your left ear if you think the answer is 14”), but  using Socrative to deliver a ‘Quiz’ or ‘Quick Question’ can be a great way to gather objective data mid-lecture or mid-activity.  Socrative meets all my basic standards for an amazing teacher tool (Free, Intuitive, Stable, and Useful), so I would recommend visiting their site to look around and watch the two-and-a-half minute intro video.

Socrative – Results of a ‘Quick Question’

If you select “Teacher Login” from the main screen, sign-in with Google, and name your “room,” students can login simply by clicking “student login” and entering the room name. To respond to a quiz or survey, there are no passwords to remember, and students (or workshop participants) can be answering questions in seconds, even the first time they use it. There are some games and advanced features, but I mainly use the basic functions: straightforward open response and multiple choice questions.

Screen Shot 2016-02-22 at 9.59.26 AM
Socrative Pre-Assessment

You can ask subjective questions before a lesson to gauge the interests and background of your class, and use this information to guide the direction of the lesson. You can also stop periodically to check for understanding with objective questions. This practice tells us whether to speed up, slow down, or repeat something. To see an example of a quiz I recently used as a pre-survey for a Gamified Learning workshop, login to Socrative as a student, using Room Code “EdTechJeff.” As Socrative is designed for real-time assessments, though, you will only be able to access it until I (as the teacher) ‘end’ the quiz or start another assignment.

Offline Assessments: Google Forms

Google forms has some similarities to Socrative, but is designed to create assessments or surveys that can be completed any time. In  a previous post and screencast I covered how to create self-grading quizzes using Google Forms and Flubaroo, but Google Forms can be used for gathering and organizing any type of response data. I’ve used it for quizzes, surveys (of students, parents, or coworkers), exit tickets, class elections and more.

Screen Shot 2016-02-22 at 6.35.18 PM
GoogleForm Questionnaire

GoogleForms is an extremely versatile tool, and once you become proficient in its many features, you will likely discover many more tasks it can help you accomplish. Recently, while delivering a presentation on Gamified Learning at the Educating Boys Conference (sponsored by NYSAIS and IBSC), I asked participants to complete two Google Forms: one a template to help them create a Gamified Learning Implementation Plan, and the other a workshop evaluation.

Screen Shot 2016-02-22 at 6.38.06 PM
Data Visualization from GoogleForms

Both Socrative and Google Forms offer real-time visualizations that will help you get an overview of class understanding in-the-moment, and both will store data for you to look at in more detail after-the-fact. I find GoogleForms to provide more options in terms of layout, types of questions, and other features. Generally, if you want students to respond to something simultaneously in-class,  use Socrative. Any time you want people to answer on their own schedule, choose GoogleForms, which also offers more features and flexible ways to use the data you gather.

Can Formative Assessments Make Lectures More Effective?

I’d love to hear your success stories of incorporating formative assessments into lecture-based classes. Feel free to ask any questions you might have, or even share difficulties you’ve encountered with these tools or strategies by commenting below.

3 Steps to Implement Data-Driven Instruction (Part 3/3)

In parts 1 and 2, we looked how to get started collecting data for DDI and then how to use that data to plan for whole and small group lessons. In this third and final section, we’ll consider ways to use data and data-driven platforms to personalize instruction for individual students. Here I’ll focus on the cycle of using individual student results to generate data that will help us in personalizing our instruction and support.

Screen Shot 2016-02-15 at 5.36.55 PM
DDI Feedback Cycle

No Data, No DDI

In Part One, we considered that before we can use data to guide instruction, we must generate that data. Unless you are willing to enter student responses into a computer, that means students must complete problem sets on an online platform. Typically, the students who would benefit most from DDI are those who are having difficulty keeping up with the class. Unfortunately, these are also the students who are less likely to complete independent work. Here are some best practices for getting a data set that will help you gauge student understanding:

  • Start in class – completing work in class doesn’t automatically mean every student will be on-task, but it does make it easier for you to to encourage and support them
  • Grade students on time completed or questions answered – Students who have previously struggled in your subject may feel they are unable to succeed. Those who are advanced may feel they can succeed without effort. By grading on time spent, you let all students know that their effort is valued and expected.
Screen Shot 2016-02-15 at 5.53.38 PM
Vocabulary.com – Record of Time Spent by Student
  • Share a leaderboard – Obviously, we do not want to publish results from students who have not met our expectations, but celebrating those who have met or exceeded them can make a data-driven platform feel like a game, and students who have not made the leaderboard will often have a good incentive to make it the next week.
  • Screen Shot 2016-01-30 at 12.19.23 PM
    Khan Academy Leaderboard

The Right Help at the Right Time

Once you have gathered an initial set of data, you can start to identify areas where personalized instruction is needed. Once upon a time we had to spend the better part of a class period (or more) lecturing an entire class about plurals and possessives. Some of our students already knew the difference. Others were daydreaming and still couldn’t tell the difference after the lectures. Today, we can look at something like the image below before we start teaching. If you saw this, how might it affect the way you approach the subject with your class?

NoRedInk Student Detail
NoRedInk – Student Detail

I would see that Omar and Kima are ready for the next lesson without any direct instruction. Lester might need a nudge, because he just hasn’t completed the work. I can see that McNulty has gotten hung up on “Singular Possessives 2,” so I would give him a quick tutoring session before asking him to move on.

Unlocking Higher Levels

One reason that games are so exciting is the ability for players to advance to higher levels once a lower level is mastered. To me, the most powerful aspect of DDI is the ability for students to progress at their own pace. Adaptive platforms will automatically progress students to harder material once they have demonstrated mastery, but there are often cases where, as the teacher, you will have a different (often better) idea of where your student should go next. How could the screenshot below help you in guiding this student’s learning?

Screen Shot 2016-02-15 at 6.51.02 PM

I notice this student has completed 58% of 4th grade. I also notice they have mastered all of the “foundations” except categorizing quadrilaterals. If this were an advanced third grade student or a grade level 4th grade student, I might have a one-on-one session to discuss quadrilaterals.

If it were a 5th or 6th grade student, I might decide that a higher priority is to build on their “foundations” in areas they have shown promise, such as fractions. In that case, I would likely assign the student more challenging fraction work, and only circle back to geometry once the student were on grade level with their arithmetic. I would likely also consider the student’s confidence and areas of interest – when a student is struggling, allowing them to work on topics that fit their interest or build their confidence can sometimes trump the need for more content coverage.

Are You Ready to Get Started?

This three part post on Data-Driven Instruction is by no means a comprehensive guide to DDI. My goal was to share some highlights from my experiences with data, give some insight into why data is helpful, and offer a few tips to help you get started. I hope my experiences will allow some of you to take the first step of gathering data. Remember that the data you gather and the way you use it will depend on the needs of you and your students – the most important part is to get started, as you will likely discover uses for data that neither of us could ever have imagined.

I would love to hear back from educators who have recently begun using data in their classes or who are thinking of trying it out. Please share your thoughts in a comment box, or send me an e-mail with your questions or success stories.

3 Steps To Implement Data-Driven Instruction (2/3): Whole Group and Small Group Planning

In my last post, I described how to get a data-driven program off the ground by encouraging and enticing students to answer questions on an online platform that stores and organizes their responses. In this week’s post, we will look at some ways that data can help us plan for whole group or small group lessons. I tend to interpret my class data by starting with a broad overview and “zooming in” to understand more specifics. It’s important to note that almost all of the data we will use comes in the form of visualizations (graphs, charts, etc) – this is one of the main advantages of using a digital platform to store, organize, and represent data – using digital data is much more time efficient and user friendly than using data gathered by hand. Here are the three levels I tend to focus on:

  • Course/Unit Overview  – This is the broadest level – an overview of the class’s progress through a course or unit. Use this to make broad plans (yearly or monthly overviews).

    Screen Shot 2016-02-06 at 2.42.51 PM
    Khan Academy Class Overview – Bars represent number of students who have mastered given percent of course
  • By skill – Once I have an overall sense of my class’s progress, I’ll zoom in one level to see which skills to target with the whole group, and which I might address in leveled groupings.

    Screen Shot 2016-02-07 at 12.19.21 PM
    Khan Academy data organized by skill
  • By student – Zooming all the way in, by student, lets us know when one-on-one support is in order. This can be a tutoring session, a pep talk about homework, or a call to a parent. (More on this in next week’s post).

    Screen Shot 2016-02-07 at 12.37.03 PM
    NoRedInk – Data organized by student

The Response to Intervention model can help us to think about data-driven instruction as a process of targeting whole group needs first, then differentiating through small group activities, and finally targeting unique needs of outlying students through one-on-one support. While RTI was originally designed to identify and support at-risk students, I have also found it useful in identifying and supporting the individual needs of high-performing students, who are similarly underserved by a “teach-to-the-middle” approach.

Class Overview for Weekly or Unit Planning

We can use whole group data to understand the progress of our class as a whole over time.

Screen Shot 2016-02-06 at 2.31.45 PM
NoRedInk – Excerpt from a Class Overview

Looking at the screenshot above, how could this inform your teaching on a whole group level? I might choose to spend a class on ‘Commas’ or ‘Verb Tenses’, while saving ‘Adjectives’ for a small group activity. Notice that the green check-mark means the whole class has mastered ‘Identifying Parts of Speech’ – before we could have whole group lessons on ‘Plural v Possessive Nouns’ or ‘Verb Tense,’ I wanted to make sure everyone was familiar with the fundamental concepts and vocabulary around nouns, verbs, and adjectives.

Creating Leveled Groups

When I look at an overview of class progress, I usually notice that students tend to cluster around certain levels of understanding.

Screen Shot 2016-02-06 at 2.42.51 PM
Use clusters to create leveled groupings

I like to make leveled groupings fluid – sometimes the class will cluster in three groups, but a few weeks later, they may spread out to four or five, so we can change the groups as needed. Sometimes, students will have similar overall levels of progress, but some are farther along in decimals while others are farther along in geometry. Being able to look at data on so many levels helps us make the best decisions in assigning groups and to make adjustments when needed.

Seeing students move around from group to group also helps students develop a growth mindset – Knowing they can move into a more challenging group by making an effort (or to a less challenging group by not being diligent or focused) can be a very strong motivator. I see on a daily basis the excitement and relief that students feel when they are properly matched with a group and with learning activities that are just at their comfort level.

Of course, data is just one part of the equation – when a student’s results confirm my opinion about their understanding, great. When the data is at odds with my opinion, I need to take another look – when this happens, I’ll usually have a one-on-one chat to better assess their understanding. Sometimes, a student knows the math but is challenged by the interface; other times, a student may be getting “help” when completing their work, so the data makes it seem like they understand more than they actually do.

Assigning Leveled Work

Once you have determined the appropriate groupings for your students, the next step is to assign work for each group. This is one of the more-challenging aspects of leveled groupings, as there are a number of ways to select activities, each of which has it’s advantages and disadvantages. One way is to start with the basics all students need to understand and work our way up from there.

  • Find a “floor” – identify a foundational level of understanding you want all students to have, and find the students who are not yet at that level. In the image below, most students have completed “Rounding Whole Numbers 2” and “Comparing Multi-digit Numbers”, but a few have not – these would be good assignments for that group to work on.
Screen Shot 2016-02-08 at 3.49.21 PM
Khan Academy ‘Skill Progress’ View
  • Find the “middle” – For students that have the foundations, scan through your skills to find those that about half the class has completed. In the image below, I’ve clicked on “Identifying Factors and Multiples,” to ‘zoom in’ on that skill and see which students have completed it. Students listed as ‘needs practice’ can be assigned this skill, though I would not assign it to students who are still building foundations.
Screen Shot 2016-01-29 at 4.23.31 PM
Khan Academy – Student detail within ‘Skill Progress’ view
  • Find a “horizon” – In the above image, the students who have mastered “Identifying Factors and Multiples” will need something more advanced. I’ll look a grade level above to find skills no one in the class has mastered. The gray bar in the image below shows me that no one has attempted “Signs of sums,” so I could assign this to a group that is hungry for a challenge.
Screen Shot 2016-02-08 at 3.58.44 PM
Khan Academy 7th Grade Skills

Advocating for Change

I hear a lot of questions about how to square leveled group work with a course curriculum. Won’t students in the basics group miss out on key lessons? When you implement a new strategy into your classroom, you should be prepared for some questions from curious (and concerned) parties, such as parents and co-workers.

I have found that while a basics group may not cover the entire grade-level curriculum, if they were to be exposed to this material, they would not fully understand and retain it anyway. (“The Importance of Background Knowledge” Marzano, 2004) I have also found that there are typically 1 or 2 students in a class that face this issue, and they are usually best served by focusing on skills they missed in a previous grade. For example, students that start the year 2 years below grade level may finish the year a half year below grade level, so they are still mastering several years of content in one school year. If they don’t receive the support they need in foundations (literacy, numeracy, etc), they are likely to just fall farther behind as the material gets more difficult.

I have also found that, other than these few students, the rest of the class covers well more than one year of content. We typically “review” the previous grade level early in the year, and students that were slightly below grade level tend to cover the prior year, current year, and at least a bit of the following year’s content. Students who are on or above grade level can often advance one or more grade levels and even have time to complete enrichment activities if they get too far ahead. So keep in mind, while the learning process may not look the same (“on Tuesday we are covering pages 15-18”), the measurable results will usually be significantly better than if you were to only teach-to-the-middle. An NYU study by Gureckis and Markant explores the advantages of self-directed learning.

In a future post, I’ll go into more detail about best practices for supporting students in the transition to small-group, Inquiry-based learning.

Are You Ready for some DDI?

This is just one way to choose work to assign leveled groups, and should be enough to get started. Depending on your platform, class size, and personal interests, you may want to use other methods – by digging in to the data and trying it out, you will likely discover a method that is better for you. If you need more help finding leveled assignments for your class, leave your question in the comments below or send me an e-mail. I’ll be happy to offer suggestions if I can. I would also love to hear about your experiences using DDI with your class!