Research points to intensive daily tutoring as one of the most effective ways to help academically struggling children catch up. There have been a hundred randomized control trials, but one of the most cited is of a tutoring program in Chicago high schools, where ninth and 10th graders learned an extra year or two of math from a daily dose of tutoring. That’s the kind of result that could offset pandemic learning losses, which have remained devastating and stubborn nearly four years after Covid first erupted, and it’s why the Biden Administration  has recommended that schools use their $190 billion in federal recovery funds on tutoring.

This tutoring evidence, however, was generated before the pandemic, and I was curious about what post-pandemic research says about how tutoring is going now that almost 40 percent of U.S. public schools say they’re offering high-dosage tutoring and more than one out of 10 students (11 percent) are receiving it this 2023-24 school year. Here are four lessons. 

1. Why timing matters

Scheduling tutoring time during normal school hours and finding classroom space to conduct it are huge challenges for school leaders. The schedule is already packed with other classes and there aren’t enough empty classrooms. The easiest option is to tack tutoring on to the end of the school day as an after-school program.

New Mexico did just that and offered high school students free 45-minute online video sessions three times a week in the evenings and weekends. The tutors were from Saga Education, the same tutoring organization that had produced spectacular results in Chicago. Only about 500 students signed up out of more than 34,000 who were eligible, according to a June 2023 report from MDRC, an outside research organization. Researchers concluded that after-school tutoring wasn’t a “viable solution for making a sizable and lasting impact.” The state has since switched to scheduling tutoring during the school day.

Attendance is spotty too. Many after-school tutoring programs around the country report that even students who sign up don’t attend regularly.

2. A hiring dilemma 

The job of tutor is now the fastest-growing position in the K–12 sector, but 40 percent of schools say they’re struggling to hire tutors. That’s not surprising in a red-hot job market, where many companies say it’s tough to find employees. 

Researchers at MDRC in a December 2023 report wrote about different hiring strategies that schools around the country are using. I was flabbergasted to read that New Mexico was paying online tutors $50 an hour to tutor from their homes. Hourly rates of $20 to $30 are fairly common in my reporting. But at least the state was able to offer tutoring to students in remote, rural areas where it would otherwise be impossible to find qualified tutors.

Tutoring companies are a booming business. Schools are using them because they take away the burden of hiring, training and supervising tutors. However, Fulton County, Georgia, which includes Atlanta, found that a tutoring company’s curriculum might have nothing to do with what children are learning in their classrooms and that there’s too little communication between tutors and classroom teachers. Tutors were quitting at high rates and replaced with new ones; students weren’t able to form long-term relationships with their tutors, which researchers say is critical to the success of tutoring. 

When Fulton County schools hired tutors directly, they were more integrated into the school community. However, schools considered them to be “paraprofessionals” and felt there were more urgent duties than tutoring that they needed to do, from substitute teaching and covering lunch duty to assisting teachers. 

Chicago took the burden off schools and hired the tutors from the central office. But schools preferred tutors who were from the neighborhood because they could potentially become future teachers. The MDRC report described a sort of catch-22. Schools don’t have the capacity to hire and train tutors, but the tutors that are sent to them from outside vendors or a central office aren’t ideal either. 

Oakland, Calif., experienced many of the obstacles that schools are facing when trying to deliver tutoring at a large scale to thousands of students. The district attempted to give kindergarten through second grade students a half hour of reading tutoring a day. As described by a December 2023 case study of tutoring by researchers at the Center for Reinventing Public Education (CRPE), Oakland struggled with hiring, scheduling and real estate. It hired an outside tutoring organization to help, but it too had trouble recruiting tutors, who complained of low pay. Finding space was difficult. Some tutors had to work in the hallways with children. 

The good news is that students who worked with trained tutors made the same gains in reading as those who were given extra reading help by teachers. But the reading gains for students were inconsistent. Some students progressed less in reading than students typically do in a year without tutoring. Others gained almost an additional year’s worth of reading instruction – 88 percent more.

3. The effectiveness of video tutoring 

Bringing armies of tutors into school buildings is a logistical and security nightmare. Online tutoring solves that problem. Many vendors have been trying to mimic the model of successful high dosage tutoring by scheduling video conferencing sessions many times a week with the same well-trained tutor, who is using a good curriculum with step-by-step methods. But it remains a question whether students are as motivated to work as hard with video tutoring as they are in person. Everyone knows that 30 hours of Zoom instruction during school closures was a disaster. It’s unclear whether small, regular doses of video tutoring can be effective. 

In 2020 and 2021, there were two studies of online video tutoring. A randomized control trial in Italy produced good results, especially when the students received tutoring four times a week. The tutoring was less than half as potent when the sessions fell to twice a week, according to a paper published in September 2023. Another study in Chicago found zero results from video tutoring. But the tutors were unpaid volunteers and many students missed out on sessions. Both tutors and tutees often failed to show up.

The first randomized controlled trial of a virtual tutoring program for reading was conducted during the 2022-23 school year at a large charter school network in Texas. Kindergarten, first and second graders received 20 minutes of video tutoring four times a week, from September through May, with an early reading tutoring organization called OnYourMark. Despite the logistical challenges of setting up little children on computers with headphones, the tutored children ended the year with higher DIBELS scores, a measure of reading proficiency for young children, than students who didn’t receive the tutoring. One-to-one video tutoring sometimes produced double the reading gains as video tutoring in pairs, demonstrating a difference between online and in-person tutoring, where larger groups of two and three students can be very effective too. That study was published in October 2023. 

Video tutoring hasn’t always been a success. A tutoring program by Intervene K-12, a tutoring company, received high marks from reviewers at Johns Hopkins University, but outside evaluators didn’t find benefits when it was tested on students in Texas. In an unpublished study, the National Student Support Accelerator, a Stanford University organization that is promoting and studying tutoring, found no difference in year-end state test scores between students who received the tutoring and those who received other small group support. Study results can depend greatly on whether the comparison control group is getting nothing or another extra-help alternative.

Matthew Kraft, a Brown University economist who studies tutoring, says there hasn’t been an ideal study that pits online video tutoring directly against in-person tutoring to measure the difference between the two. Existing studies, he said, show some “encouraging signs.” 

The most important thing for researchers to sort out is how many students a tutor can work with online at once. It’s unclear if groups of three or four, which can be effective in person, are as effective online. “The comments we’re getting from tutors are that it’s significantly different to tutor three students online than it is to tutor three students in person,” Kraft said.

In my observations of video tutoring, I have seen several students in groups of three angle their computers away from their faces. I’ve watched tutors call students’ names over and over again, trying to get their attention. To me, students appear far more focused and energetic in one-to-one video tutoring.

4. How humans and machines could take turns

A major downside to every kind of tutoring, both in-person and online, is its cost. The tutoring that worked so well in Chicago can run $4,000 per student. It’s expensive because students are getting over a hundred hours of tutoring and schools need to pay the tutors’ hourly wages. Several researchers are studying how to lower the costs of tutoring by combining human tutoring with online practice work. 

In one pre-pandemic study that was described in a March 2023 research brief by the University of Chicago’s Education Lab, students worked in groups of four with an in-person tutor. The tutors worked closely with two students at a time while the other two students worked on practice problems independently on ALEKS, a widely used computerized tutoring system developed by academic researchers and owned by McGraw-Hill. Each day the students switched:  the ALEKS kids worked with the tutor and the tutored kids turned to ALEKS. The tutor sat with all four students together, monitoring the ALEKS kids to make sure they were doing their math on the computer.

The math gains nearly matched what the researchers had found in a prior study of human tutoring alone, where tutors worked with only two students at a time and required twice as many tutors. The cost was $2,000 per student, much less than the usual $3,000-$4,000 per student price tag of the human tutoring program.

Researchers at the University of Chicago have been testing the same model with online video tutoring, instead of in-person, and said they are seeing “encouraging initial indications.” Currently, the research team is studying how many students one tutor can handle at a time, from four to as many as eight students, alternating between humans and ed tech, in order to find out if the sessions are still effective.

Researchers at Carnegie Mellon University conducted a similar study of swapping between human tutoring and practicing math on computers. Instead of ALEKS, this pilot study used Mathia, another computerized tutoring system developed by academic researchers and owned by Carnegie Learning. This was not a randomized control trial, but it did take place during the pandemic in 2020-21. Middle school students doubled the amount of math they learned compared to similar students who didn’t receive the tutoring, according to Ken Koedinger, a Carnegie Mellon professor who was part of the research team. 

“AI tutors work when students use them,” said Koedinger. “But if students aren’t using them, they obviously don’t work.” The human tutors are better at motivating the students to keep practicing, he said. The computer system gives each student personalized practice work, targeted to their needs, instant feedback and hints.

Technology can also guide the tutors. With one early reading program, called Chapter One, in-person tutors work with young elementary school children in the classroom. Chapter One’s website keeps track of every child’s progress. The tutor’s screen indicates which student to work with next and what skills that student needs to work on.  It also suggests phonics lessons and activities that the tutor can use during the session.  A two-year randomized control trial, published in December 2023, found that the tutored children – many of whom received short five-minute bursts of tutoring at a time – outperformed children who didn’t receive the tutoring. 

The next frontier in tutoring, of course, is generative AI, such as Chat GPT. Researchers are studying how students learn directly from Khan Academy’s Khanmigo, which gives step-by-step, personalized guidance, like a tutor, on how to solve problems. Other researchers are using this technology to help coach human tutors so that they can better respond to students’ misunderstandings and confusion. I’ll be looking out for these studies and will share the results with you.

This story about video tutoring was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Hechinger newsletter.

The post PROOF POINTS: Four lessons from post-pandemic tutoring research appeared first on The Hechinger Report.

Ever since schools reopened and resumed in-person instruction, districts have been trying to help students catch up from pandemic learning losses. The Biden Administration has urged schools to use tutoring. Many schools have purchased an online version that gives students 24/7 access to tutors. Typically, communication is through text chat, similar to communicating with customer service on a website. Students never see their tutors or hear their voices. 

Researchers estimate that billions have been spent on these online tutoring services, but so far, there’s no good evidence that they are helping many students catch up. And many students need extra help. According to the most recent test scores from spring 2023, 50 percent more students are below grade level than before the pandemic; even higher achieving students remain months behind where they should be.

Low uptake

The main problem is that on-demand tutoring relies on students to seek extra help. Very few do. Some school systems have reported usage rates below 2 percent. A 2022 study by researchers at Brown University of an effort to boost usage among 7,000 students at a California charter school network found that students who needed the most help were the least likely to try online tutoring and only a very small percentage of students used it regularly. Opt-in tutoring could “exacerbate inequalities rather than reduce them,” warned a  September 2023 research brief by Brown University’s Annenberg Center, Results for America, a nonprofit that promotes evidence-backed policies, the American Institutes for Research and NWEA, an assessment firm.

In January 2023, an independent research firm Mathematica released a more positive report on students’ math gains with an online tutoring service called UPchieve, which uses volunteers as tutors. It seemed to suggest that high school students could make extraordinary math progress from online homework help. 

UPchieve is a foundation-funded nonprofit with a slightly different model. Instead of schools buying the tutoring service from a commercial vendor, UPchieve makes its tutors freely available to any student in grades eight to 12 living in a low-income zip code or attending a low-income high school. Behind the scenes, foundations cover the cost to deliver the tutoring, about $5 per student served. (Those foundations include the Bill & Melinda Gates and the Overdeck Family foundations, which are also among the many funders of The Hechinger Report.)

UPchieve posted findings from the study in large font on its website: “Using UPchieve 9 times caused student test scores to meaningfully increase” by “9 percentile rank points.” If true, that would be equivalent to doubling the amount of math that a typical high school student learns. That would mean that students learned an extra 14 weeks worth of math from just a few extra hours of instruction. Not even the most highly regarded and expensive tutoring programs using professional tutors who are following clear lesson plans achieve this.

The study garnered a lot of attention on social media and flattering media coverage “for disrupting learning loss in low-income kids.” But how real was this progress? 

Gift card incentives

After I read the study, which was also commissioned by the Gates foundation, I immediately saw that UPchieve’s excerpts were taken out of context. This was not a straightforward randomized controlled trial, comparing what happens to students who were offered this tutoring with students who were not. Instead, it was a trial of the power of cash incentives and email reminders. 

For the experiment, Mathematica researchers had recruited high schoolers who were already logging into the UPchieve tutoring service. These were no ordinary ninth and 10th graders. They were motivated to seek extra help, resourceful enough to find this tutoring website on their own (it was not promoted through their schools) and liked math enough to take extra tests to participate in the study. One group was given extra payments of $5 a week for doing at least 10 minutes of math tutoring on UPchieve, and sent weekly email reminders. The other group wasn’t. Students in both groups received $100 for participating in the study.

The gift cards increased usage by 1.6 hours or five to six more sessions over the course of 14 weeks. These incentivized students “met” with a tutor for a total of nine sessions on average; the other students averaged fewer than four sessions. (As an aside, it’s unusual that cash incentives would double usage. Slicing the results another way, only 22 percent of the students in the gift-card group used UPchieve more than 10 times compared with 14 percent in the other group. That’s more typical.) 

At the end of 14 weeks, students took the Renaissance Star math test, an assessment taken by millions of students across the nation. But the researchers did not report those test scores. That’s because they were unlucky in their random assignment of students. By chance, comparatively weaker math students kept getting assigned to receive cash incentives. It wasn’t an apples-to-apples comparison between the two groups, a problem that can happen in a small randomized controlled trial. To compensate, the researchers statistically adjusted the final math scores to account for differences in baseline math achievement. It’s those statistically adjusted scores that showed such huge math gains for the students who had received the cash incentives and used the tutoring service more.

However, the huge 9 percentile point improvement in math was not statistically significant. There were so few students in the study – 89 in total – that the results could have been a fluke. You’d need a much larger sample size to be confident.

A caution from the researcher 

When I interviewed one of the Mathematica researchers, he was cautious about UPchieve and on-demand tutoring in general.  “This is an approach to tutoring that has promise for improving students’ math knowledge for a specific subset of students:  those who are likely to proactively take up an on-demand tutoring service,” said Greg Chojnacki, a co-author of the UPchieve study. “The study really doesn’t speak to how promising this model is for students who may face additional barriers to taking up tutoring.”

Chojnacki has been studying different versions of tutoring and he says that this on-demand version might prove to be beneficial for the “kid who may be jumping up for extra help the first chance they get,” while other children might first need to “build a trusting relationship” with a tutor they can see and talk to before they engage in learning. With UPchieve and other on-demand models, students are assigned to a different tutor at each session and don’t get a chance to build a relationship. 

Chojnacki also walked back the numerical results in our interview. He told me not to “put too much stock” in the exact amount of math that students learned. He said he’s confident that self-motivated students who use the tutoring service more often learned more math, but it could be “anywhere above zero” and not nearly as high as 9 percentile points – an extra three and a half months worth of math instruction.

UPchieve defends “magical” results

UPchieve’s founder, Aly Murray, told me that the Mathematica study results initially surprised her, too. “I agree they almost seem magical,” she said by email. While acknowledging that a larger study is needed to confirm the results, she said she believes that online tutoring without audio and video can “lead to greater learning” than in-person tutoring “when done right.”

“I personally believe that tutoring is most effective when the student is choosing to be there and has an acute need that they want to address (two things that are both uniquely true of on-demand tutoring),” she wrote. “Students have told us how helpful it is to get timely feedback and support in the exact moment that they get confused (which is often late at night in their homes while working on their homework). So in general, I believe that on-demand tutoring is more impactful than traditional high-dosage tutoring models on a per tutoring session or per hour of tutoring basis. This could be part of why we were able to achieve such outsized results despite the low number of sessions.”

Murray acknowledged that low usage remains a problem. At UPchieve’s partner schools, only 5 percent of students logged in at least once during the 2022-23 year, she told me. At some schools, usage rates fell below 1 percent. Her goal is to increase usage rates at partner schools to 36 percent. (Any low-income student in grades eight to 12 can use the tutoring service at no cost and their schools don’t pay UPchieve for the tutoring either, but some “partner” schools pay UPchieve to promote and monitor usage.) 

The downside to homework help

Helping students who are stuck on a homework assignment is certainly nice for motivated kids who love school, but relying on homework questions is a poor way to catch up students who are the most behind, according to many tutoring experts. 

“I have a hard time believing that students know enough about what they don’t know,” said Susanna Loeb, a Stanford University economist who founded the National Student Support Accelerator, which aims to bring evidence-based tutoring to more students. 

For students who are behind grade level, homework questions often don’t address their gaps in basic math foundations. “Maybe underneath, they’re struggling with percentages, but they’re bringing an algebra question,” said Loeb. “If you just bring the work of the classroom to the tutor, it doesn’t help students very much.” 

Pre-pandemic research of once-a-week after-school homework help also produced disappointing results for struggling students. Effective tutoring starts with an assessment of students’ gaps, Loeb said, followed by consistent, structured lessons.

Schools struggle to offer tutors for all students

With so little evidence, why are schools buying on-demand online tutoring? Pittsburgh superintendent Wayne Walters said he was unable to arrange for in-person tutoring in all of his 54 schools and wanted to give each of his 19,000 students access to something. He signed a contract with Tutor.com for unlimited online text-chat tutoring in 2023-24. 

“I’m going forward with it because it’s available,” Walters said. “If I don’t have something to provide, or even offer, then that limits opportunity and access. If there’s no access, then I can’t even push the needle to address the most marginalized and the most vulnerable.”

Walters hopes to make on-demand tutoring “sexy” and appealing to high schoolers accustomed to texting. But online tutoring is not the same as spontaneous texting between friends. One-minute delays in tutors’ replies to questions can test students’ patience. 

On-demand tutoring can appear to be an economical option. Pittsburgh is able to offer this kind of tutoring, which includes college admissions test prep for high schoolers, to all 19,000 of its students for $600,000. Providing 400 students with a high-dosage tutoring program – the kind that researchers recommend – could cost $1.5 million. There are thousands of Pittsburgh students who are significantly behind grade level. It doesn’t seem fair to deliver high-quality in-person tutoring to only a lucky few.  

However, once you factor in actual usage, the economics of on-demand tutoring looks less impressive. In Fairfax County, Va., for example, only 1.6 percent of students used Tutor.com. If Pittsburgh doesn’t surpass that rate, then no more than 300 of its students will be served.

There are no villains here. School leaders are trying to do the best they can and be fair to everyone. Hopes are raised when research suggests that online on-demand tutoring can work if they can succeed in marketing to students. But they should be skeptical of studies that promise easy solutions before investing precious resources. That money could be better spent on small-group tutoring that dozens of studies show is more effective for students.

This story about drop-in tutoring was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for Proof Points and other Hechinger newsletters.

The post PROOF POINTS: Schools keep buying online drop-in tutoring. The research doesn’t support it appeared first on The Hechinger Report.

One of the few replicated findings in education research is that daily, individualized tutoring during the school day really helps kids catch up academically. The problem is that this kind of frequent tutoring is very expensive and it’s impossible to hire enough tutors for the millions of American public school students who need help.

In theory, educational software could be a cheaper alternative. Studies have shown that computerized tutoring systems, where algorithms guide students through lessons tailored to their individual needs, can be effective when kids use them. But kids are tired of learning over screens and the kids who are the most behind at school are the least likely to have the motivation to learn independently this way.

What if you were to marry humans with technology? Could you substitute some of the tutoring time with time on ed tech without sacrificing how much students learn? That’s exactly what a team of University of Chicago researchers tried with 1,000 students in six high schools in Chicago and New York City. This blend of tutors and technology yielded results in ninth grade algebra equivalent to daily human tutoring alone at a much lower cost: $2,000 per student versus $3,000.

“You really need to get kids to like practicing math and that’s what the tutors do,” said Monica Bhatt, senior research director of Education Lab, a research center at the University of Chicago, who led the study. (The study was funded by both the Overdeck Family Foundation and Arnold Ventures; both foundations are among the many funders of The Hechinger Report.)

The study has not been published or peer reviewed, but I heard Bhatt present her team’s findings at a briefing in New York City on April 26, 2023. I thought it was worth writing about this research because it shows one approach to bringing tutoring to more students. That’s a matter of current urgency given how far behind grade level so many students have fallen during the pandemic. And ninth grade algebra is such an important milestone. Students who fail it are five times more likely to drop out of high school, according to one estimate.

This is just one study with only a year or so of evidence. Bhatt says there’s a lot that researchers still need to figure out about mixing human tutors and technology to reduce costs without losing potency. This particular study had tutors working with students in a one-to-four ratio five days a week while using ed tech half the time. But $2,000 per student remains prohibitively expensive for most public schools, especially after $122 billion in federal pandemic recovery funds run out in 2024. 

Bhatt is now studying how to further increase student-to-tutor ratios and time on ed tech to lower costs even more. She suspects that time needed with a human tutor varies by student and is currently partnering with schools in Illinois, Georgia and New Mexico to identify which students need more human attention and which need less. 

Bhatt uses a metaphor of training for a 5K race. Most people can run this distance if they train in incremental baby steps. “If you showed up at my house every single day, watched me lace up my running shoes and ran with me, then I could definitely do it,” said Bhatt. “And there are some kids, you can just say, ‘Here’s the training schedule, please follow it.’ And that will work for them.” Bhatt is trying to figure out how much personal training each kid needs in math. 

Another tutoring researcher, Philip Oreopoulos at the University of Toronto, is studying whether once-a-week Zoom tutoring sessions at home are sufficient for some students when combined with practice problems from Khan Academy, a nonprofit organization that provides free online learning.

Oreopoulos thinks the amount of tutoring a child needs might depend both on the child and the classroom teacher.  In a separate study, Oreopoulos paired coaches with elementary and middle school teachers to help them differentiate instruction in their classrooms and assign different practice problems to different students on the Khan Academy website. He found that some teachers were far more successful at motivating students to do the practice work and their students’ math achievement gains were as strong as those seen in tutoring studies. Meanwhile, similar students taught by other teachers were less motivated to do the practice work. These students might need tutoring.

In the current University of Chicago study, researchers set up a tutoring lottery for almost all the ninth graders in six low-income schools, two in Chicago and four in New York City. (Roughly 10 percent of the students had severe disabilities or extreme absenteeism – attending school less than 25 percent of the time – and were excluded from the study.) A thousand students “won” the math lottery and were given an extra math class each day operated by the nonprofit tutoring organization Saga Education, whose tutoring program has produced strong results for students in several well-designed research studies. A thousand students “lost” the lottery and had another elective scheduled during this period. Everyone, both winners and losers, had a regular algebra class. 

During the extra math block, about five or so tutors sat at tables in an ordinary classroom, each working with four students. The tutors worked closely with two students at a time using the Saga math curriculum, while the other two students worked on practice problems independently on ALEKS, a widely used computerized tutoring system developed by academic researchers and owned by McGraw-Hill. Each day the students switched:  the ALEKS kids worked with the tutor and the tutored kids turned to ALEKS. The tutor sat with all four students together, monitoring that the ALEKS kids were on task.

This experiment started in the 2018-2019 school year and at the end of the year, the students who had this extra math block learned more than twice the amount of math than lottery losers who didn’t have this tutoring-and-ed-tech experience. More surprising, the math gains nearly matched what the researchers had found in a prior study of human tutoring alone, where tutors worked with only two students at a time and required twice as many tutors. In addition to higher scores on year-end math tests, students who received the extra math block also had higher math grades (by a fifth of a letter grade) and lower rates of failure in their algebra class. “It was remarkable,” Bhatt said.

A principal of one of the schools in the study, the High School of Fashion Industries in Manhattan, spoke at the briefing and said he continues to use Saga tutors, paying part of the tab from his own budget now that the study is over. 

“One thing that they always survey students on is ‘Do you have an adult in the building that you can confide in and trust?’ You can’t underrate having that one ally in the building,” said Daryl Blank, the high school principal. “A lot of times for Saga students, it’s the Saga tutor who’s in that room, because they’re not just teaching them the math, the algebra, they’re just sort of looking out for them, cheering for them as an ally.”

The study was supposed to extend for two years, but the pandemic hit in the middle of the 2019-2020 school year and the experiment was cut short. Before schools closed, Bhatt said that midyear math grades were again higher among a second cohort of ninth grade students who had the extra math block. No standardized math assessments were administered that spring. 

I have a jaundiced view of ed tech, based on the sheer number of studies that have shown null or very tiny results for students. I am concerned about replacing time with teachers and interacting with classmates with time staring at a computer screen with headphones in our own private bubbles. Maybe there is wisdom in incorporating work periods into the school day, when students do their practice work under the guidance of tutors and machines. But I’d hate to lose art and other electives to make room for it. These are tough decisions for school leaders to make.

This story about tutoring and ed tech was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for Proof Points and other Hechinger newsletters. 

The post PROOF POINTS: How can tutors reach more kids? Researchers look to ed tech appeared first on The Hechinger Report.

Education researchers have been urging schools to invest their $120 billion in federal pandemic recovery funds in tutoring. What researchers have in mind is an extremely intensive type of tutoring, often called “high dosage” tutoring, which takes place daily or almost every day. It has produced remarkable results for students in almost 100 studies, but these programs are difficult for schools to launch and operate. 

They involve hiring and training tutors and coming up with tailored lesson plans for each child. Outside organizations can help provide tutors and lessons, but schools still need to overhaul schedules to make time for tutoring, find physical space where tutors can meet with students, and safely allow a stream of adults to flow in and out of school buildings all day long. Tutoring programs with research evidence behind them are also expensive, at least $1,000 per student. Some exceed $4,000. 

One organization has designed a different tutoring model, which gives very short one-to-one tutoring sessions to young children who are just learning to read. The nonprofit organization, Chapter One (formerly Innovations for Learning), calls it “short burst” tutoring. It involves far fewer tutors, less disruption to school schedules and no extra space beyond a desk in the back of a classroom. The price tag, paid by school districts, is less than $500 per student. 

The first-year results of a four-year study of 800 Florida children conducted by a Stanford University research organization are promising. Half the children in 49 kindergarten classrooms were randomly selected to receive Chapter One’s tutoring program during the 2021-22 school year. Almost three-quarters of the students were Black and more than half were low-income – two groups who are more likely to be held back in third grade because of reading difficulties. 

To keep younger children on track, the Broward County school district, where the study took place, wanted all kindergarteners to be able to sound out simple three-letter words by the end of the year and be able to distinguish similar words such as hit, hot and hut. After one year of this short burst tutoring, more than double the number of kindergarteners hit this milestone: 68 percent versus 32 percent of the children who didn’t receive the tutoring in the same classrooms. Tutored students also scored much higher on a test of oral reading fluency. 

“These results are big,” said Susanna Loeb, a Stanford professor of education who was a member of the research team and heads the National Student Support Accelerator, a Stanford research organization that studies tutoring and released this study in February 2023. “What’s so exciting about this study is it shows that you can get a lot of the benefits of high impact tutoring – relationship-based, individualized instruction with really strong instructional materials – at a cost that is doable for most districts in the long run.”

Loeb said the reading gains in this study were at least as large as what has been produced by more expensive tutoring programs. But it remains to be seen whether these short-term benefits will endure, and whether kids without tutoring will eventually catch up. Researchers especially want to learn if these tutored children will become proficient readers at the end of third grade, a crucial marker in academic development. By one measure, a third of U.S. third graders are currently far behind grade level in reading and in need of intensive remediation. 

The 400 children who received the short-burst tutoring in kindergarten in this study are continuing to receive tutoring in first grade during the current 2022-23 academic year. Researchers are tracking all 800 children, with and without tutoring, for an additional two years through third grade.

Loeb cautioned that this short burst model would be unlikely to work with middle or high school students. It might be that short bursts of one-to-one help are particularly suited to the littlest learners.

“We realized at that young age that their attention span runs out somewhere around six or seven minutes if you’re really doing things intensively with them,” said Seth Weinberger, the founder of Chapter One. 

Weinberger stumbled into tutoring after a foray into educational video games. He was originally a lawyer representing video game makers, and collaborated with academics to develop phonics games to teach reading. 

“After about 20 years of honing these computer games, we came to the conclusion that computer games by themselves are just not going to be enough,” said Weinberger. “You really need some combination of computer-assisted instruction and actual real live humans in order to make it work for the kids.”

Weinberger’s tutoring-and-gaming model works like this: a tutor sits at a desk in the back of the classroom during the normal English Language Arts (ELA) period. One child works with a tutor for a short period of time, typically five to seven minutes, rejoins his classmates and another child rotates in. Children work with the same tutor each time, but a single tutor can cycle through eight or more students an hour this way. 

Though it might seem distracting to have an audible tutoring session in the same classroom, kindergarten classes are often a hubbub of noise as children work with classmates at different activity stations. Tutoring can be another noisy station, but I imagine that it can also be a distraction when the teacher is reading a picture book aloud. Weinberger considers it a strength of his program that kids are not pulled out of the classroom for tutoring so that they are not missing much instruction from their main teacher. In disadvantaged schools, children are frequently pulled out of classes for extra services, which is also disruptive.

Technology plays a big role. Behind the scenes, Chapter One’s computers are keeping track of every child’s progress and guiding the tutors on how to personalize instruction. The tutor’s screen indicates which student to work with next and what skills that student needs to work on.  It also suggests phonics lessons and activities that the tutor can use during the session. 

The computer guidance takes the usual guesswork and judgment calls out of reading instruction and that has enabled well-trained laypeople to serve as tutors as well as experienced, certified teachers. (The Stanford team is currently studying whether certified teachers are producing much larger reading improvements for children, but those results are not out yet. In the current study I am writing about here, both laypeople and certified teachers served as tutors.)

Chapter One’s technology also determines how much tutoring each child should get each day and how many times a week. Dosage ranges from a two-minute session every two weeks to as much as 15 minutes a day. More typical is five to seven minutes three to five times a week. Children in the middle who are making good progress get the most. Children at the very top and the very bottom get the least. (Children who are not making progress may have a learning disability and need a different intervention.)

Technology is also used to reinforce the tutoring with independent practice time on tablets. Chapter One recommends that every child spend 15 minutes a day playing phonics games that are synced to the tutoring instruction and change as the student progresses. The researchers did not yet have data on how much time children actually spent playing these educational games, and how important this independent practice time is in driving the results.

A federal survey of principals estimates that half of U.S. students are behind grade level, far higher than before the pandemic, when a third were behind. But it’s really hard to expand high-dosage tutoring programs rapidly to serve the millions of children who need it. Most of the effective programs are rather small, reaching only a tiny fraction of the students who need help. What’s heartening about this Chapter One study is that the organization is already tutoring 25,000 students in U.S. schools (plus 1,000 students in Canada and the United Kingdom). Now we have a well-designed study – as close as you get in education to the kinds of tests that we do on vaccines and pharmaceuticals – showing that it is effective. 

“It’s not that it has the potential to scale,” said Stanford’s Loeb. “Already 10,000 kids are receiving it in this one district, so we know that it’s actually possible.” 

This story about alternatives to high-dosage tutoring was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Hechinger newsletter.

The post PROOF POINTS: Trial finds cheaper, quicker way to tutor young kids in reading appeared first on The Hechinger Report.

Ever since the pandemic shut down schools almost three years ago, I’ve been writing about tutoring as the most promising way to help kids catch up academically. I often get questions about research on tutoring. How effective is tutoring? How many schools are doing it? How is it going so far? In this column, I’m recapping the evidence for tutoring and what we know now about pandemic tutoring. For those who want to learn more, there are links to sources throughout and at the end, a list of Hechinger stories on tutoring. 

Well before the pandemic, researchers were zeroing in on tutoring as a way to help children who were significantly behind grade level. Remedial classes had generally been a failure, and researchers often saw disappointing results from after-school and summer school programs because students didn’t show up or didn’t want to go to school during vacation. 

But evidence for tutoring has been building for more than 30 years, as tutoring organizations designed reading and math programs, partnered with schools and invited in researchers. The results have been striking. In almost 100 randomized controlled trials, where students were randomly assigned to receive tutoring, the average gains were equivalent to moving an average child from the 50th percentile to the 66th percentile. In education, that’s a giant jump. One estimate equated the jump from tutoring to five months of learning beyond a student’s ordinary progress in a school year. There are no magic bullets in education, but tutoring comes as close to one as you get.

What researchers mean when they say “tutoring,” however, is not what many people might imagine. It’s not provided by the kind of tutors that well-to-do families might hire for their children at home. Studies have found that sessions once or twice a week haven’t boosted achievement much, nor has frequent after-school homework help. Instead, tutoring produces outsized gains in reading and math when it takes place daily, using paid, well-trained tutors who are following a proven curriculum or lesson plans that are linked to what the student is learning in class. Effective tutoring sessions are scheduled during the school day, when attendance is mandatory, not after school. Researchers call it “high-dosage” or “high-impact” tutoring. 

Think of it as the difference between outpatient visits and intensive care at a hospital. High-dosage tutoring is more like the latter. It’s expensive to hire and train tutors and this type of tutoring can cost schools $4,000 or more per student annually. (Surprisingly, the tutoring doesn’t have to be one-to-one; researchers have found that well-designed tutoring programs can be very effective when tutors work with two or three students.) 

The Biden administration has urged schools to use their $122 billion in pandemic recovery funds on tutoring. But it’s been hard for schools to launch tutoring operations. For starters, it’s tough to hire tutors amid a strong labor market when there aren’t many people looking for work and “help wanted” signs are everywhere. The logistical issues are complex: tutor training, rescheduling the school day to make time for tutoring periods, finding physical space to hold tutoring sessions and figuring out how to allow a stream of adult tutors to flow in and out of the school buildings all day. There are also tough decisions, such as which students should be tutored, and which curriculums to choose. Educators have to become operations experts and build a whole new organization amid everything else they’re juggling.

So far we have spotty data on how many schools have actually implemented tutoring. Among those who have, it’s unclear how many have launched good high-dosage programs and which students are getting it. 

The U.S. Department of Education estimates that more than four out of five schools were offering a version of tutoring to some of their students during the 2022-23 school year, based on a December 2022 survey of 1,000 schools. The majority said they were delivering “standard” tutoring, such as once a week extra-help sessions after school. Only 37 percent said they were delivering “high-dosage” tutoring. Even among the 37 percent of schools that said they were delivering high-dosage tutoring, only 30 percent of the students were receiving it. This translates into an estimate of 10 percent of public school students nationwide who are receiving high-dosage tutoring – far less than the need. In the same survey, school principals estimated that half of their students were behind grade level. 

Sixteen states are using $470 million of their federal pandemic recovery funds to launch large tutoring programs that will reach millions of children, according to a separate February 2023 report by the Council of Chief State School Officers, a group of public officials who head state education departments that oversee elementary, middle and high schools.  Among them are Arkansas, Colorado, Louisiana and Tennessee. Another four states are sending more $200 million directly to families to hire their own tutors. Indiana, for example, gives families up to $1,000 per qualifying student to spend on high-impact tutoring. (Local school districts are spending much more than a total of $700 million on tutoring. The school officers’ report covers only direct state spending.)

In many cases, tutoring this year is taking place virtually over screens instead of in person. Often, students are texting with tutors and not hearing or seeing one another – akin to a customer service chat session. But there are also tutoring companies that are trying to recreate an in-person tutoring experience through live video and audio. It feels more like a Zoom meeting with a shared whiteboard that both student and teacher can write on.

It remains to be seen if the outsized academic gains from in-person tutoring can be replicated online. A study of low-income middle schoolers in Chicago was disappointing. The program was riddled with problems: poor attendance, technical glitches and a slow recruitment of college student volunteers to serve as tutors. Students who were assigned tutoring didn’t catch up more than those who didn’t get that extra help. But there were some signs of hope, too. Kids who started the tutoring sooner made larger academic gains. 

Another pandemic study of virtual tutoring for low-income immigrant middle schoolers in Italy yielded good results when students received four hours a week, but much worse results when they got only two hours a week. When the hours were halved, the academic gains dropped by more than half.  

Saga Education, an organization which has built an impressive track record with in-person tutoring, is currently testing whether its high-dosage model works as well in the virtual world. I am eager to see their data when it comes out. Earlier this month I observed Saga’s virtual tutoring at a New York City high school, where the students sat in a classroom and connected to their algebra tutors through laptops. I noticed how much more engaged the students were with a tutor who was physically present. Many ninth graders weren’t keen to be seen on camera and angled their laptops away. It was harder to develop an easy, friendly rapport between student and tutor.

School administrators have told me that it is hard to squeeze in three or more tutoring sessions a week, or make sure that students log in when sessions are scheduled. No-shows are common.

Many schools have purchased unlimited online tutoring from for-profit companies, such as Paper, Tutor.com and Varsity Tutors, where students can login anytime for homework help. Companies have marketed this voluntary 24/7 tutoring as high-dosage because, in theory, students could use it frequently. And it is much cheaper for schools; it can cost $40 per student instead of $4,000 for in-person, high-dosage tutoring. But several reports, such as this one in Fairfax County, Virginia, find that students aren’t using it very much, and the students who need tutoring the most are the least likely to use these drop-in tutoring services. 

Efforts by researchers to increase usage through text nudges convinced only 27 percent of the students at one charter school chain in California to try an online tutor even once. More than 70 percent of the students never logged into the tutoring platform. Among students who needed tutoring the most because they had failed a class with a D or an F, only 12 percent ever logged on. Just 26 of the 7,000 students in the charter network used it three times or more a week, which is what researchers are recommending.

Even though the services are marketed as one-to-one tutoring, some tutoring companies, such as Paper, have their tutors handling multiple students at once. Several tutors explained to me how challenging it is to juggle homework questions from different grades and different subjects simultaneously. Students sometimes have to wait patiently for their tutor to reply to a text while the tutor is texting with others. Relying on students’ homework questions, instead of using a structured tutoring curriculum, makes it hard to know if you’re teaching students the topics they need to catch up. Part of the magic of tutoring may be forming a long-term relationship with a caring adult. But tutors at several of these companies rarely see the same student twice. It’s no wonder that most students aren’t eager to log in.

Even though there’s good evidence for the effectiveness of intensive tutoring, districts are struggling to build functional programs. The for-profit tutoring services many schools are buying in the meantime don’t make the grade.

This story about tutoring research was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Hechinger newsletter.

The post PROOF POINTS: Taking stock of tutoring appeared first on The Hechinger Report.

Throughout 2022, the Biden Administration urged schools to spend their $122 billion in federal recovery funds on tutoring to help students catch up from pandemic learning losses. Education Secretary Miguel Cardona said students who had fallen behind should receive at least 90 minutes of tutoring a week. Last summer, the White House put even more muscle behind the rhetoric and launched a “National Partnership for Student Success” with the goal of providing students with 250,000 more tutors over three years.

This federal tutoring campaign is based on some of the best evidence that education researchers have ever found for helping students who are behind grade level. What researchers have in mind, however, is not what many people might imagine. Studies have found that sessions once or twice a week haven’t boosted achievement much, nor has frequent after-school homework help. Instead, tutoring produces outsized gains in reading and math – making up for five months of learning in a year by one estimate – when it takes place daily, using paid, well-trained tutors who are following a good curriculum or lesson plans that are linked to what the student is learning in class. Effective tutoring sessions are scheduled during the school day, when attendance is mandatory, not after school. 

Think of it as the difference between outpatient visits and intensive care at a hospital. So called “high-dosage tutoring” is more like the latter. It’s expensive to hire and train tutors and this type of tutoring can cost schools $4,000 or more per student annually. (Surprisingly, the tutoring doesn’t have to be one-to-one; researchers have found that well-designed tutoring programs can be very effective when tutors work with pairs of students or in very small groups of three.) 

But little is known about how many schools have actually taken up the tutoring cause. And among those who have, it’s unclear exactly what kinds of tutoring programs they have launched and which students are being tutored. The Department of Education provided some answers last week with the release of a national survey conducted in December 2022 of 1,000 schools, from elementary to high school. This School Pulse Panel is far from an ideal survey; the slightly more than 1,000 respondents represent fewer than half of the 2,400 schools that federal authorities surveyed, and some of the responses are inconsistent and confusing. But it’s the best snapshot of the recovery that we have so far.  

More than four out of five schools said they were offering at least one version of tutoring this 2022-23 school year, ranging from traditional after-school homework help to intensive tutoring. But the modes varied: 37 percent said they were giving students high-dosage tutoring; 59 percent said they were administering standard tutoring; 22 percent said they were offering self–paced tutoring, and 5 percent said they were doing other types of tutoring. The numbers exceed 100 percent because some schools are offering several types of tutoring at the same time, administering different kinds to different students in different subjects. (For more details on how each mode of tutoring was defined, here is that question in the survey.)

Tutoring is an expensive catch-up strategy and not every student in each school gets it. Even among the 37 percent of schools that said they were delivering high-dosage tutoring, only 30 percent of their students were receiving it. This translates into an estimate of 10 percent of public school students nationwide who are receiving high-dosage tutoring.  Most schools said they were relying on diagnostic assessments and teacher referrals to determine which students were the most behind and should be assigned high-dosage tutoring, but some were also giving it to children whose parents had requested it.

Greater numbers of students nationwide were estimated to be receiving standard tutoring (14 percent) and self-paced tutoring (19 percent), both of which are much less costly to implement, but do not have as strong an evidence base. 

It remains unclear how much of the tutoring takes place in person and how much is delivered online. Self-paced tutoring is conducted through online software that mixes instruction with practice questions. But both standard and high-dosage tutoring can be done in-person or virtually. And both can be conducted during the school day or after school. 

Many schools have purchased unlimited online tutoring from for-profit companies, such as Paper, Tutor.com and Varsity Tutors, where students can login anytime for homework help. Companies have marketed this voluntary 24/7 tutoring as high-dosage because, in theory, students could use it frequently. Rachel Hansen, a statistician at the National Center for Education Statistics who oversees the survey, said it’s possible that some schools believed their unlimited online tutoring services were a version of high-dosage tutoring and checked that box on the survey, even though it does not meet the Department of Education’s definition of high dosage. I wonder if far fewer than 10 percent of students are actually getting high-quality tutoring three times or more per week.

Another reason to be cautious about this data is that 13 percent of the schools that were offering high-dosage tutoring also said that their students were receiving it only once or twice a week. That is below the survey’s definition of high-dosage tutoring, which is supposed to take place at least three times a week.

The Institute of Education Sciences, the research and data unit inside the Department of Education, launched the School Pulse Panel during the pandemic to track how teaching and learning is changing. Each month, the survey focuses on a different topic, from remote instruction and quarantines to learning lags. This December survey focused on tutoring and it is the final survey for this group of 2,400 schools during the 2022-23 school year. The department plans to begin surveying a new cohort of schools in the fall of 2023. 

One thing from the current survey that’s clear is that principals believe half of the students in their schools – 49 percent – were behind grade level, far higher than before the pandemic, when 36 percent were behind. Even if effective tutoring really is reaching 10 percent of students (which I doubt), it doesn’t come close to reaching all students who need help.

This story about high-dosage tutoring was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Hechinger newsletter.

The post PROOF POINTS: New federal survey estimates one out of 10 public school students gets high-dosage tutoring appeared first on The Hechinger Report.

Leo Salvatore graduated from college in May 2022 and dreams of becoming a philosopher. While he applies to graduate school, the affable 23-year-old holds a part-time job that barely existed before the pandemic: online tutor. From his home in Baltimore, Maryland, Salvatore logs in for one of his four-hour shifts three times a week and earns $20.25 an hour. Often, he has two or three students in different grades simultaneously text chatting with him about different homework assignments. It might be a fourth grader in Los Angeles struggling with English, an eighth grader in Palm Beach, Florida, asking about history, and a 10th grader in Las Vegas needing help with French verb conjugations. 

“It can be overwhelming,” Salvatore said, in an interview, describing his life as an online tutor in the time of coronavirus. 

We want to hear from you. If you have worked in online tutoring or experienced it as a teacher or student, please reach out to Jill Barshay. We won’t share your name or story without your permission. 

A couple of times, Salvatore recalled, he tutored as many as seven students at once. Keeping track of students’ questions and chatting with them in real time can feel more like being a short-order cook during the breakfast rush than an educator. At least his commute to work is great.

Salvatore’s employer is Paper, which is based in Montreal, Canada and says it is the largest online tutoring company in the United States. The company is fueled by more than $120 billion that Uncle Sam has pumped into educational recovery after the pandemic, when students lost months of instruction and fell behind. Schools are required to spend at least 20 percent of these federal funds on academic catch up programs for students, and the U.S. Department of Education is encouraging schools to use “high dosage tutoring,” which has produced impressive learning gains in rigorous studies where students work closely with an in-person tutor every day using prepared lessons.  

Paper has had impressive success marketing its online model of tutoring as a version of high dosage tutoring. Students at schools that pay Paper between $40 and $80 per pupil are entitled to unlimited on-demand tutoring at any hour of the day. More than 300 school districts across the country, serving more than 3 million students, have bought it. Paper has also landed statewide deals with Mississippi, New Mexico and Tennessee, serving millions more students.

The company has hired 3,000 tutors — and says that only 4 percent of those who apply make the cut. Its tutors all have a college degree or are completing college and have at least a year of experience teaching or tutoring.

“Education and experience aside, what we are looking for in our tutors is their ability to approach our students with warmth, positivity and patience, while being able to adapt to each student’s unique needs,” said Philip Cutler, Paper’s CEO, in an emailed statement.

As Paper has grown, so have its services. In addition to unlimited homework help, its tutors give feedback on essays that students upload. Salvatore is spending most of his time on these lately. The pace is grueling. The company expects him to review an essay of 500 words in 20-25 minutes, and gives him only slightly longer, 35 minutes, for 750 words. During that time, tutors are supposed to not only read but also write a paragraph of overall comments, highlighting strengths and weaknesses, plus make five specific notes per page. 

“It’s challenging,” said Salvatore. “We get lots of topics. Sometimes I go from an essay about George Bush’s speech after 9/11, to an essay about Lord of the Flies and Piggy’s demise. So it’s also kind of a brainstorm – in the negative sense.”

Nonetheless, Salvatore says he enjoys the work. Reviewing essays has helped him improve his own writing.  “It’s very fulfilling and, I hope, also helpful,” he said.

Salvatore was born in Italy and immigrated to the United States when he was 15 in 2014. He has a flair for languages and worked as a part-time French tutor in college at Soka University of America in Orange County, California. There, he tutored fellow undergraduates the old-fashioned way, in person, in the library. 

Then, in the spring of 2020 when the pandemic erupted, Soka sent students home. Salvatore moved in with his mom in Brooklyn, New York. An elementary school teacher, she was suddenly teaching five-year-olds online. 

“She was Zooming every day with these little kids in their homes, with some of them with parents behind them, some of them alone,” said Salvatore. “It was a very serious situation. I remember witnessing all that firsthand. It was pretty intense.”

Back on campus during his senior year in January 2022, Salvatore noticed a recruiting ad on LinkedIn for online tutors with Paper. Given his experience as a tutor and the memory of watching his mom struggle with children on Zoom, he was inspired to help and applied. He said he passed a test for job applicants, clicked his way through a short e-learning training module and was directly tutoring students within a month.

The first few weeks were nerve-racking, he said, while he got the hang of keeping students engaged and thinking what to type next in the chat screen. “I remember I was panicking,” he said.  “I think the training had been somewhat effective. But when you’re facing the screen in the first couple of sessions, at least for me, all the concepts can flow away.” 

Salvatore has deep-set eyes, close shaven hair and a trim dark brown beard. When I interviewed him on Zoom in December 2022, he exuded the inner calm of a yoga instructor. But his students never see his face or hear his voice. On the Paper website where students and tutors connect, there is no video or audio. The only communication is through text chatting and a whiteboard where students and tutors can draw and write numbers.

Paper tutors are trained not to give students the answers, but to use the Socratic method to help students find the answers themselves. When a student shares a homework problem, Salvatore begins by asking what the student already knows. “Do you have any resources that you looked at in class? And from there, I would say, well, let’s look at some examples,” he said.

Direct teaching “from scratch,” he said, is discouraged. But Salvatore is finding that so many students lack basic background knowledge that he sometimes teaches a mini lesson from instructional materials that he discovers online. 

By contrast, the kind of high dosage tutoring that’s shown great results in studies involves structured lesson plans. Tutors aren’t making it up on the fly. The same tutor meets with the same student at least three times a week. In the year that Salvatore worked as an online tutor, he says he’s only met with the same student twice a handful of times.  Each was by chance.

I was surprised to learn that tutors are frequently handling multiple students at once, even though the service is marketed as one-to-one tutoring. An algorithm matches students with a tutor within 30 seconds, according to the company’s marketing materials, with no scheduling required. Salvatore doesn’t tutor math, for example, so he wouldn’t be matched with a student who has an algebra question. Even if all the tutors are busy, the algorithm will keep adding students who log in to the system into each tutor’s screen. The tutor toggles among them, but all the student sees is a photograph of his or her one tutor, not the other students.  Students may have no idea that their tutor is helping others, too.

It’s similar to text chatting with a customer service representative online. You get your individual question answered, but behind the scenes, the representative is chatting with several customers at once. Often customers wait several minutes between questions and replies.  That can happen with online tutoring too. I watched one video of a tutoring session, where it seemed to take 30 seconds or more for the tutor to reply to each text that a student typed. I was impatient just watching it. 

Students can log in any time of day, but tutors aren’t expected to be on call at all hours. Tutors submit their availability to Paper and an algorithm determines the schedule, based on expected student demand. Salvatore has never requested a 3 a.m. graveyard shift. “No, I like my sleep schedule,” he said.

The research critique of this kind of 24/7 online tutoring is that very few students are motivated to take advantage of it. Fewer than 30 percent of students even tried it once in one study and students who used it regularly, as recommended, were rare. Researchers say it’s not reaching the students who need tutoring the most; the students at risk of failing were the least likely to try it. 

Salvatore is just one of 3,000 online tutors employed by Paper. Others may have different experiences. But dozens of tutors describe similar stories on a Reddit discussion board, complaining about time constraints and low pay. (Starting wages were recently increased to $18 an hour.) Some disgruntled tutors describe a sweatshop-like atmosphere where tutors quickly burn out and are fired.

Salvatore is not disgruntled, but his experience shows the pressures that online tutors are operating under, and make it seem unlikely that quick homework help like this can effectively help students fill large holes of instruction that they missed.

Not all online tutoring companies are the same. Some like Paper focus on drop-in homework help, but others make an effort to replicate an in-person tutoring experience over video with certified teachers or specially trained tutors in frequent, scheduled sessions. That model is far more expensive to deliver and I plan to continue writing about the experiences of these kinds of tutors too.

Salvatore is also interested in exploring other types of tutoring. He misses the camaraderie that developed when he was an in-person tutor.  “The more I did it,” he said, “the more I realized that it was a very, very meaningful way to help people, with their academics, but also to connect with them and have a conversation and make learning a bit more informal and fun.”

This story about an online tutor was produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Hechinger newsletter.

The post PROOF POINTS: The life of an online tutor can resemble that of an assembly line worker appeared first on The Hechinger Report.

For my year-end post, I’m highlighting 10 of the most important Proof Points stories of 2022. This year, I was proud to write several watchdog stories that use research evidence to highlight poor or ineffective practices in schools. I put a special focus on tutoring – the good and the bad –  as well as test-optional admissions and reading. 

Thank you to everyone who read and commented on my weekly stories about education data and research. I look forward to continuing this conversation with you next year. If you would like to receive my email newsletter and be notified when the column comes out each week, please click here and fill out the form. I’ll be back again on Jan. 2, 2023 with a story about arts education. Happy New Year!

1. PROOF POINTS: Many schools are buying on-demand tutoring but a study finds that few students are using it

Companies market 24/7 online tutoring services as “high-dosage” tutoring to help students catch up from pandemic learning losses. But researchers warn that these products don’t have an evidence base behind them and now a study finds that not many students are using them.

2. PROOF POINTS: Does growth mindset matter? The debate heats up

I took a deep dive inside the scholarly debate over boosting students’ “mindsets” — one of the most popular ideas in education. Dueling meta-analyses conclude it’s either generally ineffective or effective only for low-achievers. 

3. PROOF POINTS: Colleges that ditched test scores for admissions find it’s harder to be fair in choosing students, researcher says

A qualitative study gives us a rare, unvarnished glimpse inside college admissions offices as they struggle to admit students under new “test-optional” policies. Admissions officers often described a “chaotic” and “stressful” process where they lacked clear guidance on how to select students without SAT or ACT scores. This story came out a couple weeks before the U.S. Supreme Court heard two affirmative action cases and it was my most read story of 2022.  

4. PROOF POINTS: Leading dyslexia treatment isn’t a magic bullet, studies find, while other options show promise

New research casts doubt on the most sought-after and expensive way of teaching children with dyslexia to read: the Orton-Gillingham method. 

5. PROOF POINTS: Seven new studies on the impact of a four-day school week

As policymakers debate the schedule switch, some research shows a tiny negative effect on rural students, where the shortened week is most popular. 

6. PROOF POINTS: Researchers say cries of teacher shortages are overblown

I was the first reporter to question the media narrative that teachers were leaving the profession en masse. I discovered that teacher vacancies weren’t that much higher than we have had during previous tight labor markets.

7. PROOF POINTS: The paradox of ‘good’ teaching

Researchers find a tradeoff between raising achievement and engaging students. It’s extremely rare for teachers to do both in the classroom. 

8. PROOF POINTS: Debunking the myth that teachers stop improving after five years

Newer research finds that even experienced educators get better, albeit at a slower pace. 

9. PROOF POINTS: Third graders struggling the most to recover in reading after the pandemic

As the coronavirus pandemic ravaged communities and shuttered schools, many educators and parents worried about kindergarteners who were learning online. That concern now appears well-founded as we’re starting to see evidence that remote school and socially distanced instruction were profoundly detrimental to their reading development. 

10. PROOF POINTS: Six puzzling questions from the disastrous NAEP results

How bad were pandemic learning losses among fourth graders? My best analogy is a cross-country road trip. Imagine that students were traveling at 55 miles an hour, ran out of gas and started walking instead. Now they’re back in their cars and humming along again at 55 miles an hour. Some are traveling at 60 miles an hour, catching up slightly, but they’re still far away from the destination that they would have reached if they hadn’t run out of gas.

This story about the top education research stories of 2022 was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Hechinger newsletter.

The post PROOF POINTS: 2022 in review appeared first on The Hechinger Report.

In the fall of 2020, educators at Aspire Public Schools – a network of 36 charter schools in California that are privately run but taxpayer funded – were worried. As with other schools around the country, pandemic era learning wasn’t going smoothly. Many of its 7,000 middle and high schoolers, mostly Hispanic and low-income, were struggling in their studies and course failure rates had spiked. 

Like hundreds of school districts, Aspire purchased an online tutoring service for the spring of 2021 to help these students. Students could log in to the tutoring service, called Paper, whenever they wanted, 24 hours a day, seven days a week, and connect with a tutor to help with schoolwork in any subject. There was no video or audio, but students could text chat with a human tutor and work together on a virtual whiteboard and share documents. The tutoring was free to students no matter how much they used it.

The nonprofit charter school network also invited a team of university researchers to study whether the online tutoring service was helping students. The results were disheartening for those who hope that on-demand tutoring might be an effective way to help students catch up. (Researchers agreed not to disclose the name of the tutoring company in the study, but Aspire has been public about its 2021 tutoring deal with the Montreal-based tutoring giant Paper, also known as Paper Education Company Inc..)

The researchers, from Brown University and the University of California, Irvine, tried three different ways of engaging students. But no matter what they tried, a majority of students never used the tutoring service. Even their most successful effort, which involved nudging both parents and students with frequent text messages and emails, convinced only 27 percent of the students to try an online tutor at least once. More than 70 percent of the students never tried it. Without the nudges, only 19 percent of the students connected with an online tutor. And, among the students who needed tutoring the most because they had failed a class with a D or an F in the fall of 2020, only 12 percent ever logged on. Students who were doing well at school and not at risk of failure were twice as likely to take advantage of the free tutoring. 

“Take-up remained low,” the researchers wrote, in an October 2022 working paper of the Annenberg Institute for School Reform at Brown University, titled “The inequity of opt-in educational resources and an intervention to increase equitable access.” 

“The real key takeaway from the study,” said lead researcher Carly Robinson, is that just telling students about a tutoring service isn’t enough to make them use it.  “And it happens even less for those students who we think probably need it the most,” said Robinson, a postdoctoral research associate at Brown University’s Annenberg Institute and a researcher at the National Student Support Accelerator, which is promoting the use of evidence-based tutoring at schools.

The students who did log in typically had no more than four tutoring sessions during the entire spring term. Only 26 of the 7,000 students used it three times or more a week, which is what experts are recommending. One student was a power user, logging on 168 times.

It’s unclear how much this optional tutoring helped students academically. Fewer students in the group that used the tutoring the most failed classes. Fifty-nine percent of the students who were nudged (along with their parents) passed all their courses without any Ds or Fs compared to 55 percent of the students who weren’t nudged to use the tutoring. Still, even with the availability of tutoring and the reminders, more than 40 percent of the students failed at least one class. 

Students in the nudged group didn’t get higher grades than students in the control group who were not nudged. In both groups, the students who took advantage of at least one tutoring session did get better grades than those who never had a tutoring session. Math grades, for example, were more than a letter grade higher – an A versus a B minus. But researchers emphasized that’s not proof that the tutoring made the difference. It’s quite likely that students who were motivated to try a tutoring session were generally more motivated students and would have had higher grades regardless. 

Schools are required to spend 20 percent of their $122 billion in federal pandemic recovery funds on helping students catch up academically.  Education researchers and the U.S. Department of Education are calling for schools to set up tutoring programs, especially for the weakest students who fell the most behind during the pandemic. Strong scientific evidence of academic gains has come from a specific type of intensive tutoring that takes place three or more times a week and is often referred to as “high dosage.” Hallmarks of the proven programs are not just frequency, but working in-person with tutors using clear lesson plans, rather than merely helping with homework. And the sessions are scheduled during school hours, when attendance is required. 

“Good tutoring also means working with the same tutor over time and building a relationship, which isn’t usually possible with an on-demand sort of support,” said Amanda Neitzel, assistant professor at the Center for Research and Reform in Education at Johns Hopkins University and research director for ProvenTutoring, a coalition of organizations that provide evidence-based tutoring programs.

Neitzel advises schools not to spend their pandemic recovery money on 24/7 online tutoring. “I think in most cases, no,” she said.  “There is very little evidence to support this, and plenty of better alternatives.” 

Several online tutoring companies have been marketing their 24/7 tutoring services to schools as “high dosage.” Paper has a webpage devoted to “high dosage” tutoring, correctly explaining the model that researchers are advocating, while presenting online products as “newer, more scalable high-dosage tutoring models.” Business has exploded since the pandemic hit. Paper currently has tutoring contracts with 300 school districts around the country, including Las Vegas, Boston and Atlanta, and statewide deals with Mississippi and Tennessee. 

Paper was also the vendor of the online tutoring services in the California study. The company said that it “fully” agrees with the study’s findings and acknowledges that it needed to improve the usage rates of its tutoring services. Paper says that it has since adopted many changes to boost the number of students who log in. Younger students can now record their voices instead of text chatting, for example. And it says that nominating teachers who can “champion” their product in the district and share best practices has been effective in driving more use. However, Paper declined to disclose what its current usage rates are.

In an interview, Paper’s CEO Philip Cutler described his firm’s on-demand tutoring as an “enhanced” version of “high- dosage” tutoring. Better? Yes, he said, because it can serve more students. 

“You need to show that there’s results,” said Cutler. He said that the kind of intensive tutoring that researchers are recommending is “valuable” but it can serve only  “a handful of students.” “It doesn’t move the needle nationally,” he said.

Research-backed tutoring programs, by contrast, are difficult for schools to manage, from hiring and training tutors to finding classroom space for the tutoring sessions and rescheduling the school day to make time for it.  It’s much easier for school leaders to pay for an online tutoring service that takes place outside the school walls.

Cutler admitted that he cannot yet point to proven results for his on-demand tutoring. He’s currently working with independent researchers at Learn Platform and McGill University to evaluate his product.

At first glance, on-demand online tutoring would seem to be more economical. Cutler said his company charges a flat fee of $40 to $80 per student, depending on the size of the school district, regardless of hours used or how many students log in. By contrast, evidence-based high-dosage tutoring can run $4,000 per student for the year. However, given the low usage seen in the California study, per-hour costs can be similar. (Here’s my back-of-the-envelope math: If a 10,000-student district pays $400,000, but only 20 percent of the students log in for four half-hour sessions each, then the district could end up paying $100 an hour for tutoring.).

One influential educator has some advice for administrators who are trying to figure out what to do. Terry Grier, the former schools superintendent of Houston and a mentor to school leaders around the country, said schools that want to offer on-demand tutoring should negotiate tighter deals and pay only for the hours used and only if student test scores increase. He said it’s  “immoral” for schools to sign “blank contracts” without strings attached. In his own experience with “high-dosage” tutoring in Texas, he said that the in-person, intensive version was very effective, especially in math. He said he also tried online tutoring, but it didn’t work well. “Kids wouldn’t use it,” Grier said.

Online tutoring is still relatively new and these on-demand services may prove to be effective.

Tutoring companies describe impressive vetting processes and training programs for their tutors, who might be fantastic. I don’t know. In the California study, lead researcher Robinson noticed that online tutors could relieve teachers from having to answer every small question that students have so that they can spend time with students who need more help. 

“I think there’s a place for this type of virtual on-demand tutoring,” said Robinson. 

Cutler, Paper’s CEO, told me that some teachers are telling their students to submit their first drafts to a Paper tutor to work with students on revising their essays before turning them in. Using online tutors to build good editing habits sounds like a fantastic idea to me, but it might not help students make up for pandemic learning losses. 

Meanwhile, the Aspire schools in California have reconsidered on-demand tutoring and many aren’t using it anymore. The schools that are have shifted to using the online tutors for special projects and as an additional resource when parents aren’t available for on-the-spot help with homework.

This story about on-demand tutoring was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Hechinger newsletter.

The post PROOF POINTS: Many schools are buying on-demand tutoring but a study finds that few students are using it appeared first on The Hechinger Report.

Tutoring is by far the most effective way to help children catch up at school, according to rigorous research studies. The research community urged schools to spend a big chunk of their roughly $190 billion in federal pandemic recovery funds on what is called “high-dosage” tutoring. That means that students have tutoring sessions at least three times a week, working one-to-one with tutors or in very small groups with tutors using clear lesson plans, not just helping with homework. 

Many schools embraced this sort of frequent tutoring. So far, we have a few preliminary data points, which, on the surface, can make it seem like tutoring is working quite well.

• In Tennessee, 50,000 or roughly 5 percent of the state’s elementary and middle school students were assigned high-dosage tutoring during the 2021-22 school year. In June, the state reported huge academic gains for students throughout the state, hitting new five-year records in reading achievement. In math, students are on track to recover all of their pandemic learning losses within three years. 
• Amplify, a curriculum and assessment company that jumped into the tutoring business during the pandemic, said that roughly 60 percent of the students who received online tutoring at least twice a week in 2021-22 were making outsized gains in reading, compared with only about 40 percent of students who didn’t receive tutoring but were also well behind grade level at the start of the year.
• Saga Education, a nonprofit tutoring company that focuses on ninth-grade algebra in low-income schools, reported that 78 percent of the more than 6,000 students it tutored across six cities passed their math classes in the spring of 2022. 

All of this is good news, but none of these data points is proof that tutoring is working.

In Tennessee, the students who were targeted for tutoring – those who are significantly behind grade level – didn’t do as well as students overall. Indeed, the number of low-income children who fell “below basic” – the lowest category on the state’s annual proficiency tests  –  kept growing last year. In 2019, before the pandemic, 31 percent of Tennessee’s low-income students were reading on a “below basic” level. That number grew to 33 percent in 2021 and hit 36 percent in 2022. 

This is evidence that Tennessee’s strong gains last year were driven more by other educational changes that helped top and middle students, who didn’t receive tutoring. Last year, for example, Tennessee revamped the way it teaches reading to all students.

It’s possible that the 50,000 struggling students who received tutoring last year would be doing much worse without the extra instruction. Or, maybe it’s taking a while for schools to set up new tutoring programs, and it’s not yet showing big results. Brown University’s Matthew Kraft is studying tutoring efforts in Nashville to help answer these questions, but methodical research is slow. 

“We need to be prepared for underwhelming results from tutoring operations,” said Kraft, who believes it will take time for schools to figure this out. “Changing educational systems at scale is hard.”

Meanwhile, tutoring companies are reporting impressive but unverified gains from students who are receiving frequent tutoring sessions. It can be unclear whether the students who show up day after day are more motivated and would have done just as well without the tutoring. While we wait for more rigorous results that compare students who did and didn’t receive tutoring – apples to apples – one troubling issue is already emerging: low participation or attendance rates.

In one large city, Amplify contracted to give almost 1,200 students tutoring sessions three times a week with a tutor delivering sessions over a video call, similar to Zoom. More than 100 kids never logged in to connect with a tutor online. Only 200 students – fewer than 20 percent – received at least two sessions a week throughout the school term. More than 80 percent received less, often far less. 

I talked to a school administrator in another school district south of Fort Worth, Texas, who assigned 375 third graders across all 15 of his elementary schools to use Amplify tutors in the spring term. The Crowley school district especially wanted its lowest achieving third graders to receive tutoring because their first and second-grade years were so disrupted by the pandemic when they were just learning to read.

Tutoring sessions were supposed to take place during the school day, during a special half-hour class dedicated to extra catch-up instruction, but teachers had discretion over whether to get the computers out to connect students with their remote tutors. Overall, students attended only 46 percent of the sessions that were supposed to take place.

“Attendance has been a challenge,” said Crowley chief academic officer Nicholas Keith. “Some campuses bought into it. But it was hard for some to find time for the tutoring component.”

Teachers may have been hesitant to put their students in front of screens, Keith explained, and wanted to work with students directly themselves. At the same time, the district was plagued with many teacher absences as the virus variants surged through their community and substitute teachers often didn’t know they were supposed to set up the computers for tutoring. 

Next year, Keith said he plans to continue the online tutoring only at the schools that were making good use of it. In some schools, more than 60 percent of the students attended on a regular basis and the teachers noticed progress in students’ reading abilities, Keith said.  

Meanwhile, Saga, which tutored more than 6,000 ninth graders in math during the 2021-22 year, reported that students attended two thirds of their in-person daily sessions, on average, with attendance rates ranging from a high of 87 percent in Washington, D.C., to a low of 49 percent in Providence, Rhode Island. Among the 62 percent of its students who received at least 80 hours of tutoring, 87 percent passed their math classes this past spring. 

Saga’s tutoring is a scheduled course during the school day called “math lab,” without other competing instructional activities at the same time. “The attendance rate is the same as a student’s school attendance rate,” said AJ Gutierrez, a co-founder of Saga.

An outside research firm, Mathematica, is currently studying Saga’s tutoring results during the pandemic, analyzing the tradeoff between larger tutoring groups and how much students gain from tutoring. Larger groups are more economical and reach more students.

The Tennessee Department of Education said it was seeing much lower attendance rates for tutoring sessions scheduled before and after school. Most schools, however, have opted to provide tutoring during the regular school day, the department said. “Tutors often pull students from their classrooms to ensure that students who are at school receive their tutoring session,” a spokesperson for the department explained by email.

Saga’s Gutierrez says he’s heard stories of after-school and summer programs failing to lure students to tutoring sessions with gift cards, movie passes and food.  “I know of a principal in North Carolina who did everything above and more (i.e. added extra curricular activities) to get 100 students in his school to attend summer tutoring, but only ended up with 21,” Gutierrez said by email.

Tutoring was a big component of the 2001 No Child Left Behind law that aimed to lift the achievement of low-income children. But between poorly trained tutors and outright embezzlement scandals, it was not a success. This time around, many schools are trying to improve tutoring quality. But attendance is uneven.

One suggestion to help tutoring deliver on its promise comes from Bart Epstein, president of the EdTech Evidence Exchange, a nonprofit that aims to help schools make better decisions in buying education technology. He is also a former executive at tutor.com, a tutoring company.  “No school district should be paying for tutoring if kids aren’t showing up,” Epstein said. “That’s ridiculous and wrong for so many reasons.  Anyone who negotiates a contract that results in paying a tutoring organization for service for 1,100 students when only 200 receive service should be ashamed of themselves.”

“If you want tutoring companies to get kids to show up,” Epstein said, “structure their contracts so that they have the incentive to make that happen, even if it requires tutoring companies to hire caseworkers and social media people and customer service people who call parents, and meet with kids to find out what they need.”

This story about tutoring was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Hechinger newsletter.

The post PROOF POINTS: Early data on ‘high-dosage’ tutoring shows schools are sometimes finding it tough to deliver even low doses appeared first on The Hechinger Report.