My Week at Duolingo

In August 2024, I had the truly unique opportunity to participate in a standard-setting workshop for the first time. It was at the Duolingo headquarters in Pittsburgh. We were tasked with reviewing the Duolingo English Test and aligning it with the Canadian Language Benchmarks, so that it could be submitted to IRCC as an option for refugees who may struggle with access to the required language proficiency testing. 

The 15 panelists arrived on Tuesday, some of us previously having met and/or heard of each other. Wednesday was our first day of work. We met at the main building and were given a tour of the two buildings before settling in for a deeper explanation of the DET. We were organized into pods of three to review, discuss, and determine our ratings. I didn’t know any of the people in my pod previously, but what quickly became apparent was how easily we could discuss matters and generally come to a simple consensus. At times, we worked with the pod next to us, and it worked very similarly. I was amazed at how 15 strangers could come together fairly seamlessly to tackle such a project and have a shared understanding and ability to work together so quickly. The level of experience in the room was extremely impressive, and it was interesting and insightful to listen to the reasoning and ideas of others. We started with reading first and worked through each skill area one at a time over the next four days. At times, it was really tough, particularly with the receptive skills. The project leads were great at looking at how things were going and adjusting to make the process as efficient and accurate is it could be.

We worked long hours over those four days, but there were some moments of fun. We had a dinner at the end of the first day with the DET team. It was great to get to chat with them more and hear about their journeys to Duolingo. We also had the opportunity to attend a Pirates game. Although baseball isn’t really my thing, the Pirate’s stadium is incredible with its view of the river and downtown bridges. On our last working day, we had lunch at Duo’s Taqueria which is connected to the main building. I had actually eaten there on my first night when I first heard about it, but the abundance of delicious food was unforgettable.

Our last workday was a tough one though as we struggled to work on listening. It was quite challenging to align the benchmarks, and at one point, we took a much-needed dinner break to clear our heads.  A small group of us went in a direction that we hadn’t previously explored. We stopped at a local Pittsburgh chain restaurant, Primanti Bros.  Allie and I spied the ‘Giant Bavarian Pretzel’ on the menu and ordered it.  I, then, said, “I hope it’s as big as my head!”  Maybe, ten minutes later, the server came out of the kitchen with our two ginormous pretzels.  We all laughed, and it released the stress we had been feeling. Sabine captured photos of us eating them directly off the hooks they were served on, and we all had another restorative laugh over the photos. This was exactly the break that we needed to refocus and return to work.

The experience was truly amazing in every way. The lead researchers, DET team, and flock of Canadian as we were known due to our research aliases were simply amazing. The welcome that we felt from everyone at every level at Duolingo, and in Pittsburgh was unforgettable. For educators who work in funded programs, visiting Duolingo was a different world with the common gourmet lunches, snacks, coffee bar, ramen and other machines. I feel really privileged to have been a part of this project, to meet so many wonderful people, and to learn how to tackle standard-setting. 

Before leaving for the airport, there was a little free time for those of us flying back to Toronto as our flights were in the afternoon, unlike people from other parts of Canada.  I opted to visit the Natural History museum on the recommendation of a friend and visit the dinosaurs. The dinosaurs were impressive, but that was only one part of the huge museum. We did our best to see what we could, but we could have spent days there. Then, it was back to the airport, back to Toronto, and back to reality.