Recruiting and Performance, Part II

This is a continuation of a post from yesterday. I figured we needed some good numbers on the relationship between recruiting and performance, and I was uniquely positioned to do the analysis. If you want details on methodology or more results, check out the other post.

Using recruiting results and performance between 2006 and 2013, I estimated the impact of recruits on performance. Now I want to address an obvious follow up question: which teams have outperformed their recruiting classes and which teams have done worse. The table below has the team's performance in terms of points per game above or below what was predicted for them based on the strength of their recruiting classes. I break the results into two groups - 2006 to 2009 and 2010 to 2013.

We all could have guessed the teams that standout as over-performers. Boise State was first for '06-'09 and second for '10-'13, only because they were passed by Oregon. Boise State, Utah, BYU make the list for both periods, and Utah States makes the latter list; recruiting services might be missing something in the Utah/Idaho area. Oregon and Oregon State make both lists, as does Nevada. Navy and Air Force are both in the top six in '06-'09 and Navy makes the list again in '10-'13. Go America! And you know Saban has a good thing going when Alabama tops the recruiting ranks and is still among the top programs for performance relative to recruited talent.

We should take a moment and put Boise State in perspective. I found that a 4 or 5 star recruit is worth a little more than 1/2 a point per game for the next four years (I'll use .6 for this example). Boise State has been about 20 points per game better than their recruiting would suggest. This means the team was playing as though they had an additional (20/.6=) 33.3 4 or 5 star recruits on the team, or 8-9 more per class.

The under-performing list is dominated by dumpster fires, teams that no one thought would be any good, but turned out to be even worse. A few programs stand out: Michigan, Notre Dame and Florida State for the '06-'09 period and Texas for the '10-'13 period. Texas A&M and Utah State have the proud distinction of moving from under- to over-performing. Eastern Michigan, FIU, Idaho, Memphis, Miami (Ohio), New Mexico State, Tulane and UAB have the less-proud distinction of making both lists. Central Michigan fell from over- to under-performing.

Recruiting and Performance

Executive Summary:

• 4 and 5 star recruits are worth about 1/2 a point per game on average for the next four years. 
• That's about 25 points total
• By recruiting, the top 5 in 2014 should be
• #1 Alabama
• #2 Ohio State
• #3 Florida State
• #4 LSU
• #5 Florida
• Consistent over and underperformers
• (See part II)

****If you're only interested in results, you can skip this part**** 

A note on methods - I measure the independent variable (recruiting) using 247sports composite rankings. I use both a count for 4 and 5 star signings and "points" - 247sports has a clever method for ranking classes that awards points for each recruit, with diminishing returns as class size increases. I consider the past four recruiting classes (e.g., for the 2013 season, I use the 2013, 2012, 2011 and 2010 recruiting classes). Unfortunately, there are not enough 5 star recruits to use this as a unique variable, so in most cases I combine 4 and 5 star recruits.

I measure the dependent variable (performance) using an algorithm that converts game scores into a measure of team performance independent of schedule strength and game site (i.e., home/road). I exclude bowl games and games against FCS competition.

I have performance data from 2006 and recruiting data from 2003. From this, I can match 939 teams over that period on recruiting and performance (117 teams per season). I use regression to estimate performance from recruiting metrics.

Results (for the nerds amongst us):
Stars is the count of four and five stars signed in that year, Stars (t-?) is the number of years between that class and the season in question. For example, Stars (t-1) refers to the class signed earlier that year (the 2013 class when talking about performance in 2013) and Stars (t-4) is the class three years earlier (the 2010 class when talking about performance in 2010). The same is true of the second model, where I use points instead of star counts.

 

****Everyone should start reading here****

First, when we use 247sports points to predict performance, we get an R^2 of .38. This means that this recruiting measure explains about 38% of the total variation in a team's performance. Player evaluation, coaching, and luck (e.g., injuries) should explain most of the remaining 62%.

On average, a team is .68 points better per game for each freshman 4 or 5 star recruit, about .57 points better for each sophomore and junior, and .32 points better for each senior. The big drop from junior to senior is most likely a product of the best juniors leaving early for the draft (or otherwise leaving the program).

Using the coefficients from the second model, I plugged recruiting results from 2011, 2012, 2013, and 2014 to predict performance in 2014. I list the top 50 below.

The top 10 is a who's who of college football. It shouldn't surprise anyone that Alabama comes out on top with 18 5 stars in the last four classes. Ohio State's position at #2 is mostly a product of 48 4 stars in the last three classes; that is more than the bottom 2/3 of programs combined.

Some teams are over-performing their recruiting this season, others failing to meet expectations. Boston College at #69 beat #9 USC. BYU (#71) manhandled Texas (#10). And we were supposed to believe it was an upset when #27 Virginia beat #38 Louisville. Oklahoma is only one spot ahead of Tennessee by recruiting; the gap on the field is much larger. I don't think we will see Florida at #5 or Michigan at #11 at the end of the season. Oregon (#18) and Baylor (#31), on the other hand, are doing more with less.