Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, by Cathy O’Neil. Chapter 3 (WOMD-Chapter3.pdf )
Big Data: A Report on Algorithmic Systems, Opportunity, and Civil Rights (https://obamawhitehouse.archives.gov/sites/default/files/microsites/ostp/2016_0504_data_discrimination.pdf)
The excerpt from Weapons of Math Destruction successfully demonstrates that algorithms do not necessarily quantitative like the algorithms reviewed previously in the class, like algorithms that determine risk assessments or where police should dedicate patrolling resources. The U.S. News and World Report’s ranking system measures subjective aspects of the college experience and includes subjective descriptions of colleges from college presidents. While the aspects measured can contribute to a positive college experience, like high faculty-to-student ratios, the application of the same set of standards to every college in the United States results in these universities gaming the system to improve the score.
At the same time, I believe it is important to consider the audience these rankings are presented to and how different audiences may react. In the case of a student living in the United States, they are likely to pay less attention to rankings when compared to a student that does not live in the United States. The student living in the United States has a greater ability to visit the college, shadow current students, or even talk to alumni of the American college because the student lives in the same country as the college. However, an international student would have to pay for transportation, visas that may be required, and housing if they wanted to visit the college and shadow a current student. Of course, an international student may be able to speak to alumni of the college, but differences in location and language can contribute to there being a lack of information available to international students reviewing U.S. colleges. With a lack of information, rankings become more central in deciding whether to attend a college. Thus, rankings can be come the way that colleges market themselves to particular students, like international students.
Regardless, this serves as an important reminder to consider who the results of algorithms are presented to and whether these audiences have an ability to access more information to assess the validity of these algorithms.
Anything about college admissions always reminds me of my college search process, and the conflict I felt between wanting to get into a good school but not knowing what that entailed. I spent plenty of time on US News’ rankings, especially as my search narrowed near the end and I tried to choose between Grinnell and Beloit. On one hand, a lot of things were similar between the two, but ultimately, I made a choice that aligned with prestige. Although ultimately I’m happy with my choice, readings like this WMD one prompt me to consider whether or not I would have been happy regardless, and that the aura of “eliteness” ends up meaning anything in the end. It is hard to accept a notion that the rankings of schools may suffer from the same status-quo feedback loop issues that permeate other socially constructed criteria. There is a dominant group of schools, Ivy’s, standford, etc, and then the criteria of a “good” school is shaped by those institutions, even though the standards are unrealistic and only end up harming the vast majority of colleges that cannot operate in the same ways. And what we see in effect is a vicious cycle of colleges raising tuition and students spending more on college preparation (the ones that can afford it at least), and more students applying to the “best” institutions over schools that are better fits, and more students choosing to go to schools that they can’t afford because they feel like its the only way they can be successful, etc. It makes me feel like I made a choice for reasons that I didn’t have any sway over; that I was forced into choosing prestige because it seemed like the only option.
A final note is the mention of making schools more affordable, which I think is such a widely supported view that it’s comical that colleges face almost no resistance to constantly raising rates. Just last year at this school, tuition was raised 5%, which may not seem like much, but to many students, this represents a drastically higher amount that they must cover. But then again, it’s almost impossible problem for a student to solve, since many are probably a year or two from graduating, don’t have another option lined up, and lack the social power to do anything about it. When I think about it, the college could raise its tuition almost as much and as frequently as it wants, because as brutal as it is to get into college, starting over somewhere else seems infinitely harder, and so we are prisoners to their business.
O’Neil exposes the deep flaws and unintended consequences of the US News & World Report college rankings, arguing it has become a destructive and unfair weapon of math destruction (WMD). The model incentivizes a host of problematic behaviors by focusing solely on proxies for educational quality like test scores, selectivity, and graduation rates while ignoring critical factors like cost and student debt. Schools game the system in ways that improve their ranking but reduce educational quality, like inflating application numbers to appear more selective or spending lavishly on facilities instead of financial aid. Students and parents panic, thinking the rankings dictate success and pour money into test prep and admissions consultants.
A key insight is that the US News model optimizes for privilege, not educational excellence or opportunity. The proxies implicitly embed social advantages, causing a harmful feedback loop. As O’Neil writes, “It’s here that we find the greatest shortcoming of the U.S. News college ranking. The proxies the journalists chose for educational excellence make sense, after all. Their spectacular failure comes, instead, from what they chose not to count: tuition and fees.” This hits on a recurring theme with WMDs – what is measured and optimized often reflects embedded biases.
While O’Neil argues convincingly that the US News methodology is flawed, she also cautions that alternate ranking systems have risks too. Good intentions can still lead to negative unintended consequences if gaming incentives exist. This speaks to the complexity of evaluating something as abstract as educational quality. Transparency and user control may be better than any top-down ranking. By highlighting the damage algorithms can cause, O’Neil compels us to think critically about how we quantify success. blind reliance on models leads to dangerous distortions.
I remember a while back that Columbia University got in trouble for submitting false information to the US News Report. The first thing I thought was, “Why Columbia? They’re a good school, right? Why would they need to pad out rankings?” It is, of course, a good school. Most people would probably agree. That’s exactly why algorithms like the US News Report were built around conforming to what people already think. As soon as there are a set of values that directly correspond to an institution’s value, though, people will immediately begin to game the system. Thus begins an arms race to become the “best” school, though only on paper. As long as you only sacrifice things the algorithm doesn’t consider, such as cost or affordability, you’re golden. And, due to the expectations that people have in general, the schools at the top can’t rest on their laurels. Top schools are forced to game the system, even cheating, if they must, to avoid the feedback loop that plunges them in the rankings and destroys their reputation. Columbia, after the whole debacle, decided not to send anymore data to US News Report. Of course, this was only after they got caught lying and cheating, but the decision still holds a lot of weight. At a certain point, people have to purposefully choose to avoid engaging in systems that corrupt them or their institutions and further damage their reputation.
O’Neil argued on US News’s college ranking. I do not think school ranking based on subjective factors is necessarily bad because rankings can help applicants make decisions during the application process. However, as she argued, it has some negative impacts on schools and applicants such as reporting false information and paying huge amounts of money to education consultants.
One of the interesting points in the reading is the objective of modeler of US News’ college ranking. The objective of the ranking is to list the schools in the correct order top to bottom. They did not consider some factors such as educational values in the school, but included subjective factors. I do not think its approach is totally wrong, but they need to update the formula by adding some factors such as its cost-efficiency as O’Neil argued. The ranking system does not need to be totally new, but updating the system year by year can make it more beneficial for colleges and applicants.
It has always caught my attention how the college admission process in the US works. To start, it’s incredible how many universities and colleges there are. As far as I’m aware, most countries in South America have only a handful of options, and usually, there is only one requirement for admission: the score on the national standardized test. Back when I was applying to study in the US, I remember how much my college advisor would emphasize the importance of having a broad range of extracurricular activities, achieving high test scores, and writing an innovative college essay. She also stressed the importance of applying to multiple colleges and avoiding only applying to prestigious schools that were always at the top of most rankings.
In the third chapter of “Weapons of Math Destruction,” O’Neal argues about how the US News ranking had a negative impact on college culture, mainly due to the algorithm it introduced for ranking colleges. Something I really appreciate about this chapter is all the examples O’Neal gives about what colleges do to boost their rankings, from asking for donations and building modern facilities to recruiting talented athletes and paying students to retake the SAT while manipulating the statistics in the reports they send. It’s terrifying how an algorithm has created the sense that, for someone to be successful, they need to get into the most prestigious colleges. The argument about how an algorithm like this one falls into a feedback loop where the universities on top will always remain on top due to different factors like alumni salaries and donations made. The ranking model proposed under the Obama administration seems like an improvement, but I believe that at the end of the day, each person is looking for different things when deciding which college to attend; therefore, there will never be the perfect ranking.
As this course delves deeper into the world of higher education, it has become apparent that the purpose of schooling is not simply to provide individuals with the opportunity to educate themselves and position themselves for success. Rather, it is becoming increasingly clear that colleges and universities are primarily businesses that prioritize revenue. This emphasis on achieving high rankings and financial success, as discussed in Weapons of Math Destruction and Big Data: A Report on Algorithmic Systems, Opportunity, and Civil Rights, is understandable given the competitive nature of the industry.
It would be natural to assume that colleges have always prioritized the education and well-being of their students, but the truth is that the focus has shifted over time. As with many other aspects of the American system, colleges have become more financially oriented, with a greater emphasis on generating profits and securing funding from various sources (College Sports for example). This has led to a number of changes in the way colleges operate, from the types of programs they offer to the way they approach recruitment and retention of students.
Despite these challenges, it is important to remember that many colleges are still committed to providing their students with a quality education and preparing them for success in their chosen fields. However, it is also important to acknowledge the economic realities of the industry and the pressures that colleges face to remain financially viable. Ultimately, the key to success for both students and colleges is finding a balance between these competing priorities and ensuring that everyone involved is able to thrive in the long run, which starts with colleges prioritizing their students more.
I find it interesting that we are now talking about the algorithm of college rankings and how some colleges manipulate their statistics to appear higher in the college rankings. We see that the same idea of wanting to gain profit rather than the intention of assisting groups is shown, as we see colleges somewhat change for the benefit of gaining more popularity, support, donations, etc. They become more business-related than for educational purposes and I find that insane. The purpose they stood for changes and doesn’t seem like a problem to many higher-ups. This is a far stretch, but I also thought about the possibility of the reputation of a college and how these rankings allow colleges to show if they are better or not. My parents would always talk about how going to an Ivy League college would get me a better job, and I just find it so insane how these rankings of colleges are connected to so many different opportunities, as people have different perspectives of the rankings and how these rankings affect peoples mindsets.
This other chapter of WMD is fascinating, especially in relations to my work at admissions and a book that I just started reading called “Excellent Sheep” which talks a lot about how colleges have become phenomenal at creating students that excel with endless transcripts and resumes but have no real world knowledge, seem to know everything but have little true curiosity, and can do anything but have deeply embedded insecurity and unhealthy habits. This all, to some extent, comes from the US News rankings. And while I know how little the admissions staff actually cares about the rankings, we still have that little checkmark next to “Grinnell College” that says that we contributed additional data (surely in a bid to increase ranking spots) and proudly display our “7th best value” and “11th best Liberal Arts College” rankings to prospective students and parents.
I also found the sports part interesting and wonder how much Jack Taylor’s NCAA (and world) record has helped Grinnell’s rankings and applications. Probably not much but I’d bet the effect was measurable.
I think that the government’s report really emphasized for me the notion that algorithms can be helpful, like the credit report findings that would help low-income people the most, but need to be transparent and constantly checked in on. It’s like the Feynman’s dilemma piece that we read, you can’t just decide that something is ok or moral once and then just continue with that assumption in perpetuity. It needs to be a constant updating and checking of the effects. Plus changing societal values can really change what people are willing to sacrifice and the general calculus around algorithms and their effects.
Often, people look at the US News college rankings as measure of college educational quality and indicator of future success, but rarely question how these rankings are calculated and why colleges may change in ranking. As in the reading, I think because the colleges that top the rankings are the ones people would expect to do so, so credibility isn’t questioned.
In high school I was pretty unaware of how college admissions worked; I think the proximity of the whole USC admissions scandal as well as some acquaintances’ experiences with SAT programs further exposed me to how they operated. I especially agree with the reading when it discusses how this process of applying doesn’t really leave you with important skills, but rather trained to fit into the same machine; people think once you get into college it’s over but it’s not since similar processes occur when applying for further education and jobs. Then, attention was often focused on how applicants game the admissions system, and less about how colleges themselves do the same. But, the need for colleges to check off boxes to rank higher results in feedback loops influenced by the rankings created by US News, which is ultimately why applicants feel they must go to these lengths for college applications in the first place.
Within the church, chapter 3 of Weapons of Math Destruction O’Neal explores, the bias and complexity of the college admission process. I think the college rankings can be helpful in determining information about which college is better for a major or specific focus. But I think that when the college focuses too much on beating other colleges and getting the best ranking, it’s not in the interest of students’ welfare but rather more focused on its reputation and how it is seen. I feel like that mindset can be very toxic. For example, I’ve seen some tiktoks about how stressful it is to be at a high-ranking elite university. If a majority of the high-ranking colleges focused more on the mental health and support of the students I have hope that the students will be happier.
I always found college rankings so bizarre. My high school typically sends 40% of the graduating class to Ivy League schools, and I remember several of the faculty in admissions criticizing my choice to not apply to any Ivy schools, especially because of my “background.” I remember kids in class debating whether or not Brown should be considered an Ivy. I think there is a huge sense of elitism in a lot of the college process, especially in the application process. A lot of the times students with marginalized identities are expected to engage with their trauma in some way to prove their resilience, or grit. And, again as I’ve said before, when these algorithms rank things such as campus life, who are they asking and what are the demographics of these students? The only reason I am still at Grinnell is because I was awarded a fellowship, otherwise I would have transferred. In fact, several of my friends in the year above me have told me that a disproportionate number of students of color (Black students especially) transferred to other schools because of the campus environment. None of these rankings actually reflect the diversity of student experience, and it makes it harder for students who can’t visit these schools to gauge what their life would look like as a student.
What strikes me about what O’Neil mentioned about college rankings is that they were reversed engineered to put Ivy leagues at the very top and that the employment of these Blackbox-like algorithms, as has been a theme in this course, preserves the status quo. I wanted to focus on the claim that the employment of such algorithms decreases the intellectual diversity of a class by briefly considering the example of international students coming to American colleges (although I do think this is probably applicable to domestic students). I think it is a well-known fact that the acceptance rate for international students in every US college is strictly lower than the total acceptance rate. What this then means is that, since most colleges want to be ranked higher and want ‘better’ matriculated students to do so, this creates a situation wherein the people with more privileges ultimately win out in this process. I am not really going to talk about how the majority of colleges being need-aware for international students leads to an overrepresentation of higher income students in a lot of colleges, but structural privileges grant people the ability to, for example, have access to a SAT or ACT testing centre, go to coaching classes for the same, be able to take the test multiple times to better their scores, attend a school that is a well-known feeder schools into Ivy Leagues, etc. If an applicant’s parent is able to donate to the school or, better yet, sit on a board that oversees fundraising if the student was to be admitted, this also helps in the college’s ranking due to the fundraising being a factor considered in determining the same. So, we see that these algorithms allow for individuals with the most structural privileges to max out every aspect of their application that the algorithm truly considers (some of which are weighted more than others). This creates a situation where in the upper-classes are guaranteed admission into these schools, whereas students of any other social class have to be exceptional in order to secure a spot. In this way, these ranking algorithms not only rank the schools, but the students who apply too, and not by merit. Furthermore, reliance on these rankings lists by international students leads to an increased number of these students applying to the top ranked schools and, since the acceptance rates for international students is much lower, this lowers the overall acceptance rate of the school and increases their ranking like a self-fulfilling prophecy.
The timing of the readings which we have for class today could not be more perfect. Many of the concerns highlighted in the report by the Obama administration, Big Data: A Report on Algorithmic Systems, Opportunity, and Civil Rights, have surfaced or continue to persist in the “opaque, black box” today. Specifically, I thought it was a bit ironic that the big data opportunity for criminal justice was that it could increase transparency and build community trust. Additionally, the report continues to outline the challenges of how big data and algorithms could increase racial disparities in smaller crimes and address how crime is largely subjective–thus hard to label with numbers. In the previous class readings, however, we quickly discovered that big data’s use as of 2023 highlights more “challenges” than “opportunities.”
At the time of my writing this, the U.S. News 2023 Liberal Arts College rankings just came out with Grinnell appearing higher (I believe) than it has appeared at any other point in my time here as we claimed the number eleventh spot. To me, I have no knowledge of what has happened at other Liberal Arts schools in comparison to Grinnell, but I still do believe that nothing really (in my view) has changed in regards to the school that much to point to an increase in ranking other than prestige. The rankings thus to me and I am sure to many other students here are just a ranking. Mathematical destruction points this out and how schools like Grinnell have probably figured out ways to target the statistical analysis of U.S. News to improve our spot in the rankings. While I guess I am happy we improved in the rankings, I wonder what this means in terms of actually improving student life. Does this merit less needing to be accomplished in student life because our ranking alone will pull in new students? A question that I think is common amongst students here.
Chapter 3 of Weapons of Math Destruction spoke a lot about the US News College Rankings, and how it has negatively impacted the culture surrounding college admissions. As many have mentioned, it perpetuates the cycles of “top” universities staying on top because of being able to do things like recruit top athlete or even pay students to retake the SAT, artificially increasing their average scores and thus their prestige. This prestige helps their alumnae attain high paying jobs quickly, which leads to alumni donations, which further increases their ability to pay their way to keeping themselves on top.
However, algorithms can and do change. Just this year, US News changed its rankings to include factors like average debt, as well as percentage of Pell Grant and first gen students. This led to a jump in rankings for many large public universities, and a fall for several elite private ones, such as Vanderbilt. I’d seen this article from the Chronicle about Vanderbilt’s response to this which was very telling: https://www.chronicle.com/article/vanderbilt-u-says-u-s-news-emphasis-on-social-mobility-is-deeply-misleading#:~:text=The%20Chronicle%20also%20later%20learned,has%20been%20added%20as%20well.&text=In%20U.S.%20News%20%26%20World%20Report,.%2013%20to%20No.%2018. . For the reasons that the chapter classified making the US News Report algorithm a WMD, this change would be one in the positive direction. However, though Vanderbilt’s response reads to many as a confirmation that Vanderbilt is a score by the social elite for the social elite, Vanderbilt contests it does not factor in the ways financial aid structures at elite private universities may mislead low income students into thinking they could not afford it, when in reality they could because of need-blind policy. They assert that this perpetuates their comparative lack of socioeconomic diversity. While adjusting for social mobillity is very important and is a good step on the US News part towards being a WMD, and even though Vanderbilt’s response was a little tone deaf, the point that this adjustment doesn’t fully resolve a complicated issue such as this and doesn’t make US News no longer a WMD.
The weapons of math destruction reading comes with a lot of details about why the college ranking system brought a lot of issues to the way colleges focused their funds. I thought it was interesting how the department of education decided to forgo the idea of creating their own ranking system, but tried their best at providing information about each school in a more wholistic way. I definitely empathize with the issue of safety school and rejection as many of my peers found themselves rejected from many schools they applied to for this reason. Schools they were certainly capable of attending on paper, but who were ultimately looking to attend a higher rated school.
I think it is kind of crazy that we have a document from the executive branch from back in 2016 talking about issues concerning big data with a lot of the nuances that we are still discussing now. Technology has continued to advance rapidly but the regulations in place have been unable to keep up. There was a very interesting detail I noticed about using location and other data to confirm credit history that I thought was a little concerning. I also have seen that Experian, which has had some security issues, seems to have tried to implement the strategy mentioned about including utilities more regularly into a foco score without it being only focused on punitive missed bills. It does raise privacy and security issues
The WMD chapter really did an effective job of portraying how powerful models like the US News rankings are, and how the ramifications of them can be grand and unforeseen. Like O’Neil said, there could potentially be an argument for how the rankings improve schools, as the proxies that determine the rankings can effect positive change, but that’s only in an extremely limited view of what is defined as “positive”. The TCU example was very effective of exemplifying how to game the system, and it hit close to home. I’m from Ann Arbor, and have seen all my life the embarrassing level of effort universities and towns put into improving their reputation while ignoring their students. While U of M students continue to struggle to get student housing or financial aid, upper class students who went to private school get scholarships and our small city continues to get improvements to the second largest football stadium in the country (which can fit our entire population). While football games are fun, those changes don’t do anything actually positive for the student body. But U of M football has become a cultural force, and that attracts attention. Attention means more applications, which means higher rankings. This backwards method of “improvement” doesn’t actually improve the quality of life or experience for students.
I also really appreciated the second half of the reading, involving the college application process and how that has morphed. The whole process has shifted to only benefit those with privilege and access to resources like private tutoring and extracurriculars. I think I can speak for a lot of people when I say I think it feels like a waste of time to try to fulfill very specific guidelines for what colleges deem to be important; oftentimes it felt like a guessing game to me, there was no definitive resource on what to do. The fact that the college culture has shifted to where 99% of people don’t benefit from the changes made is an indicator to how other institutions will change. Although most people are being hurt by changes made by their colleges, employers, and the services they use, it means more money for these establishments.
In O’Nell’s reading, he talks about the WMD in college ranking and the application process. When things comes to competition, whether big data provides a fair system of ranking becomes an interesting question. On one hand, qualitative studies are hard to involve in the rankings; on the other hand, some of the qualifiable questions are able to be presented in a direct way. However, I think that the “best college” is hard to describe since everyone has different definitions of what is best. The limited variables with the name of bests are not convincing enough in regard to the limited variables and the percentage of the variables since different people have different opinions on what is more important. The big data reduces the biases but also creates biases in another form. The cognitive biases could be reduced through the algorithm and big data. However, since the rule of how to run the algorithm is determined by people, the algorithms are innate with bias.
The “Weapons of Math Destruction” reading about college admissions brought me back to three years ago when I was using the same rankings from US News to inform my own decisions about where I should apply for college. As an international student who had never been to the US, knew nothing about the college culture, and could never afford a plane ticket to the US for in-person college tours, I relied heavily on US News to form my own impressions of prestige and decide which college is worth my family’s investment. For many high school seniors like me who are from outside the States, as well as those who are faced with other constraints that limit their choice of higher education, rankings like this have a great impact on our lives. Despite that, we rarely ever think about how the schools are ranked and which criteria the rankings are based upon.
The reading provides a really in-depth analysis of the history, the arguments as well as the controversy behind the college-ranking algorithm. It really shows how influential input data can be, and how technologies like this can be used to perpetuate and mask historical biases. In this case, the creators of these algorithms looked at the most well-known colleges and identified factors that made them appear to be the most successful in the nation. These surface-level factors were fed into the algorithm and became the measure of success and prestige for a college. With this came all the preconceptions about what a college that can provide a worthwhile education looks like, which are objectively flawed and lacking in many respects. Although the general public, specifically high school students who have to make decisions regarding college choices, is now much more knowledgeable and critical of these rankings, college administrators and admitted students alike feel pressured into the rat race of maintaining the ranking and pursuing the top-ranked schools respectively. This demonstrates how long-lasting and powerful the impacts of technologies can be even when they were created with the best of intention and knowledge a long time ago. Despite efforts to advocate for a new algorithm or do away with these biased ones, college administrators together with past, current and aspiring college students themselves pushed back because it has become too convenient a means of comparison for everyone.
In this chapter, O’Neil describes the widespread effects of the U.S. News college ranking system. Since it’s release, it has cause a reformation in how universities admit students, hire faculty and decide on funding distribution. The rankings create a dangerous feedback loop where universities focus on improving the proxies measured by U.S. News rather than necessarily being the best university it can be.
However, I disagree with the notion that the U.S. News rankings are a WMD like the predictive policing and recidivism algorithms we talked about earlier this week. Instead, I think that the rankings are a secondary problem of the system of higher education in the US more broadly. I think that private universities and colleges are able to spend so many resources focusing on rankings because of the under-funding of public higher education.
Outside of the US, public universities tend to be the largest and most renowned institutions. This is sometimes the case in the US as well, but funding for public higher ed in the US has consistently gone down when adjusted for inflation in most states. As a result, it is instead private schools with massive endowments that are able to spend on the perverse incentives enabled by U.S. News. If public universities instead had more funding, there would be much more room for top students to go to top schools, rather than the current system which funnels privileged students into small elite private schools.
I found the reading about the college/university ranking quite interesting. In the reading, the author talks about how the first ranking systems came out and got a lot of controversy. Besides that, he mainly focuses on how colleges/universities can choose to game the system including manipulating graduating rates, employment rates after graduation, and so on. The author does admit that the model has some credibility on its ranking referring to the academic performance part of the student enrolled, but one of the biggest shortcomings is that the tuition and fees do not count into the model, this may decrease the possibility of poor/low-income kid get into the college and finally enlarge the social dividend. in the end, the author gives an opposite example by pointing out that the students can ask their own questions that they are concerned about, and the software itself behaves like an online travel site that creates individual models for each person. The author argues that in this way, it is transparent, personal, and controlled by the user.
In my own college preparation experience, these types of rankings weren’t really on my radar, so the WMD chapter was honestly a very informative look at how these rankings are constructed (and how reifying they are to societal norms). The analogy used about the caveman diet in the WMD reading was interesting, but I think it fell a little short in reflecting how the college admissions process / these college rankings support existing structures of power and marginalization.
In regard to the data report, it was interesting to see what the officials had identified as possible weaponized uses of data to discriminate. However unsurprisingly, this analysis was very surface level, and seemed to mostly deal with the use of big data as the problem, without addressing the possibility that the underlying systems themselves (say policing) are inefficient and violent