June 13, 2010
Which is the Best AI School?

So in the recent US News rankings that everyone pretends not to care about, the "AI Specialty" rankings came out like this:

  • MIT
  • Carnegie Mellon
  • Stanford
  • Berkeley
  • Texas
  • Washington
  • Georgia Tech
  • UIUC/UMass (tie)
  • Penn

My understanding is that along with the numerical part of the survey, there’s a section in which department heads are supposed to write in names of schools that excel in specific specialties. Then, the number of times each school is mentioned is added and schools are ranked by how many times they appear. Obviously, this ranking methodology is crappy and so here I tried to do something better.

What I came up with was this:

  1. Identify the primary current research affiliation of the final author of every technical paper of the last several years of AAAI/IJCAI.
  2. Rank schools by the sum total of the counts of their affiliates.

This might seem quizzical, but it has some key advantages. It allows me to completely bypass the question of what AI is by answering it this way: AI is whatever gets published at AAAI/IJCAI. Trying to combine other, different conferences gets nasty quickly: comparing acceptance rates and “AI-ness” and numbers of papers and all that mess. Is EC an AI conference? CHI? ICRA? How “good” is UAI? My approach avoids these issues by focusing on the premier big tent conferences. And the ground they cover is really big — a list of AAAI 2010 keywords can be found on the second page of the call for papers.

Because there’s no good automatic way to do author affiliations at the time of publication, I focused on the current affiliation of the last author under the assumption that this would be the faculty member in charge of the lab responsible for the research (or the paper was single-authored by a grad student). In virtually all cases I looked up this turned out to be the case. There are cases where this methodology messes up, like this one, but I figure those are just tiny independent random shocks.

Before the rankings and analysis, a couple notes about my implementation. I looked at conferences from the past five years, so the AAAI programs from 2005, 2006, 2007, and 2008, and the IJCAI programs from 2005, 2007 and 2009. I didn’t count people who were the final author on only one publication. Mainly I just didn’t want to look up several hundred more names in Google, but also if you’ve only had one paper in the last seven major AI conferences you’re probably not making that much of an impact in AI anyway. Finally, just like US News I’m only looking at US schools; sorry, rest of the world. I’ll note in passing that several foreign schools are easily competitive with the schools listed here (Alberta and Southampton come to mind in particular).

So, with all that in mind, here’s the top ten:

  • Carnegie Mellon
  • UCLA/Washington (tie)
  • USC
  • MIT/UIUC/UMass (tie)
  • Texas
  • Harvard
  • Stanford

Rankings 2 through 7 were all within 2 papers and should be regarded as roughly equal, and Texas, Harvard, and Stanford were also within two papers of each other. The gap between CMU and the second-ranked schools was quite substantial, and CMU also had the most faculty members that were tallied as well. To qualitatively summarize the results in the most self-aggrandizing way possible:

  • Superior: Carnegie Mellon
  • Excellent: UCLA, Washington, USC, MIT, UIUC, UMass
  • Very Good: Texas, Harvard, Stanford

If there’s one way to summarize the differences between this ranking and the US News list, it’s that my ranking is more now. This is best seen by the three schools in my list but not the US News: UCLA, USC, and Harvard. All three of these schools have been making strong recent strides to hire good AI people, but certainly USC and Harvard are not “traditionally” considered computer science powerhouses, and that undoubtedly hurts them when it comes time for department heads to brainstorm AI schools.

Interestingly, by my tally no Berkeley researcher has had two publications in the last seven AAAI/IJCAI conferences. Obviously, there are fantastic AI researchers at Berkeley — Stuart Russell and Michael Jordan come to mind immediately — but this ranking shows they have not been particularly active in the recent big-tent conferences. If I had to guess, I would probably say faculty name recognition along with Russell writing the widely-used undergraduate AI textbook are responsible for Berkeley’s performance in the US News ranking.

8:05am  |   URL: http://tmblr.co/ZtlAMyfLyu0
(View comments
Filed under: AI 
Blog comments powered by Disqus