The Orange
and Black Box

Hey, I'm Lawrence Li and this is the Orange and Black Box. It's March, which means there's only a month left before course selection, and honestly, I'm already overwhelmed. Princeton offers more course evaluation tools than I know what to do with — all of them supposedly help me build the perfect schedule. Finding the best courses to take at Princeton is one of our biggest social traditions. But it's also something that's become increasingly data-dependent. TigerJunction, PrincetonCourses and even data articles from the Prince are all part of an increasingly popular data-driven approach to tackling a social-cultural process. In fact, we're even expecting an LLM to soon be added to that list of technical tools to make course selection easier. In this episode, we're exploring how all this data is changing how students make their academic choices, and whether more data is really making course selection any easier. I'm joined by Ibraheem Amin, Gwanpil Son and Olivia Chen — some of the minds and beta-testers behind that upcoming LLM — to see what they have to say about the future of data, courses and the Princeton experience.

I think course selection is a balance between classes that I have to take and then the fun or interesting classes that I want to take. So first, I'll definitely take into account courses for my major, put it into TigerJunction, and then from there, I figure out how much that courseload is gonna be, so I ask upperclassmen, I know the general reputation of the class, and I look at course reviews, and if I have a heavy workload, then I'll go around and ask what easy fifth classes my friends would recommend, or if it's relatively light, then I'll look at classes that are interesting. Within the past three years, there's definitely been more emphasis on using data to decide stuff in course selection. Like we don't see more social mixers to help underclassmen decide what courses to take, we see software that makes it easier to see course ratings more visibly. I think it makes it easier in a way that if everyone is saying the same thing, it's likely that it's accurate. But it doesn't solve everything, because there's definitely a factor of personal interest and strengths. I will say, I have been led astray by a class that was really highly rated, but it ended up being a lot more work than I expected.

TigerApps is a student organization that was founded back in 2017, used to be a part of Princeton's student government, but is now its own student org. And they've made apps like PrincetonCourses and TigerJunction, which are the sites that a huge chunk of undergrads use for course selection. We're talking about 6,000 unique visitors a year. And we get our data from everything available to students through the Office of Information Technology's student API. When we think of doing undergraduate course planning at Princeton, the available data doesn't take into account things like course discontinuations, minor requirements, pathways, or major requirements. Princeton doesn't make available data for future courses ahead of time, releasing course information for upcoming semesters only about a month or two before course selection for that term. What people often want is not just the mean, but the spread around it — the standard deviation, the disagreement, the signs that a class is polarizing. And beyond that, they want to know who liked it. Was it mostly students with strong quantitative backgrounds? Was it people already interested in the topic? Because sometimes the most useful data point is not the overall average, but how someone like you experienced the course.

I can read the full thought process of the LLM and I can see exactly from where in PrincetonCourses the LLM got its information. For example, I can see whether it's looking for the courses with the highest rating, whether it is looking for courses with no final, and so on. When I asked ChatGPT or Grok about COS324, they both said to take those courses since they were "the intro machine learning course" at Princeton and that many people enjoyed it. This is probably because LLMs like ChatGPT and Grok don't have access to PrincetonCourses so they can't see the student reviews. Ibraheem's model, on the other hand, can see student responses so when I asked it about COS324, it still said positive things because it has to, but it also talked about its criticisms, like messy lectures and exams. For "fun courses" it first filtered out for courses with a rating of 4.5/5 or higher and looked for courses that had no final exam. It also looked for more "interesting" courses and looked heavily on music courses, creative courses, and engaging courses.

So we've talked a lot about how incorporating more data is changing course selection at Princeton. And that raises a bigger question: what kind of academic culture do we want at Princeton? The rise of data within the course selection process here at Princeton is a microcosm of a larger cultural shift. But as we've seen today, data can only go so far. While more data and more evaluations of courses can be helpful, there are certainly other aspects of the course selection process — like that piano ensemble sheet or broader four-year scheduling tips — that can't be accounted for with more data points. We see that course selection sort of operates on these two legs — social and data-specific information. Moving forward, it's important to keep in mind what more data is and isn't capable of telling us, and being cognizant of where these limits of human and technical knowledge each lie.