Quick Thoughts on Recruiting and Results

I noted earlier in the season that, based on recruiting results, the top five teams in 2014 should be Alabama, Ohio State, Florida State, LSU and Florida. The top three are in the playoffs (even if they shouldn't be). Oregon, the fourth, is 18th in recruiting. The Ducks have overperformed their recruiting for most of a decade.

LSU and Florida are on the outside. LSU has an excuse. The NFL has expended 18 draft picks on Tigers the past two seasons. I own a couple on my fantasy team. A number of these gentlemen were elite recruits out of high school and could have played on the 2014 edition of the fighting Les'. (While we are on the topic, let us review NFL.com's perceived weaknesses in Odell Beckham Jr.'s game before the draft: "Adequate height. . . . . Inconsistent making contested grabs -- can be out-muscled in a crowd. Has some concentration drops)."

The emmigation of talent from the Red Stick should have opened up a spot for the Gators. Instead, Florida is playing East Carolina in the Birmingham Bowl. For those new to college football, that is not a glorious conclusion to the season. Muschamp will now be coordinating the defense for Auburn, The other Tigers were looking for someone else to coordinate their defensive efforts after they allowed 130 total points in three of the final four games of the regular season.

The Two Pictures that Prove Ole Miss should be Ranked above Florida State (at least this week)

The web of wins for Ole Miss and Florida State within six degrees of separation:

Ole Miss, connected to 84 teams:

Florida State, connected to 45 teams:

If rankings are about the body of evidence supporting a team's relative quality, the evidence is clearly in Ole Miss's favor at this point in the season.

Recruiting and Performance, Part II

This is a continuation of a post from yesterday. I figured we needed some good numbers on the relationship between recruiting and performance, and I was uniquely positioned to do the analysis. If you want details on methodology or more results, check out the other post.

Using recruiting results and performance between 2006 and 2013, I estimated the impact of recruits on performance. Now I want to address an obvious follow up question: which teams have outperformed their recruiting classes and which teams have done worse. The table below has the team's performance in terms of points per game above or below what was predicted for them based on the strength of their recruiting classes. I break the results into two groups - 2006 to 2009 and 2010 to 2013.

We all could have guessed the teams that standout as over-performers. Boise State was first for '06-'09 and second for '10-'13, only because they were passed by Oregon. Boise State, Utah, BYU make the list for both periods, and Utah States makes the latter list; recruiting services might be missing something in the Utah/Idaho area. Oregon and Oregon State make both lists, as does Nevada. Navy and Air Force are both in the top six in '06-'09 and Navy makes the list again in '10-'13. Go America! And you know Saban has a good thing going when Alabama tops the recruiting ranks and is still among the top programs for performance relative to recruited talent.

We should take a moment and put Boise State in perspective. I found that a 4 or 5 star recruit is worth a little more than 1/2 a point per game for the next four years (I'll use .6 for this example). Boise State has been about 20 points per game better than their recruiting would suggest. This means the team was playing as though they had an additional (20/.6=) 33.3 4 or 5 star recruits on the team, or 8-9 more per class.

The under-performing list is dominated by dumpster fires, teams that no one thought would be any good, but turned out to be even worse. A few programs stand out: Michigan, Notre Dame and Florida State for the '06-'09 period and Texas for the '10-'13 period. Texas A&M and Utah State have the proud distinction of moving from under- to over-performing. Eastern Michigan, FIU, Idaho, Memphis, Miami (Ohio), New Mexico State, Tulane and UAB have the less-proud distinction of making both lists. Central Michigan fell from over- to under-performing.

Recruiting and Performance

Executive Summary:

• 4 and 5 star recruits are worth about 1/2 a point per game on average for the next four years. 
• That's about 25 points total
• By recruiting, the top 5 in 2014 should be
• #1 Alabama
• #2 Ohio State
• #3 Florida State
• #4 LSU
• #5 Florida
• Consistent over and underperformers
• (See part II)

****If you're only interested in results, you can skip this part**** 

A note on methods - I measure the independent variable (recruiting) using 247sports composite rankings. I use both a count for 4 and 5 star signings and "points" - 247sports has a clever method for ranking classes that awards points for each recruit, with diminishing returns as class size increases. I consider the past four recruiting classes (e.g., for the 2013 season, I use the 2013, 2012, 2011 and 2010 recruiting classes). Unfortunately, there are not enough 5 star recruits to use this as a unique variable, so in most cases I combine 4 and 5 star recruits.

I measure the dependent variable (performance) using an algorithm that converts game scores into a measure of team performance independent of schedule strength and game site (i.e., home/road). I exclude bowl games and games against FCS competition.

I have performance data from 2006 and recruiting data from 2003. From this, I can match 939 teams over that period on recruiting and performance (117 teams per season). I use regression to estimate performance from recruiting metrics.

Results (for the nerds amongst us):
Stars is the count of four and five stars signed in that year, Stars (t-?) is the number of years between that class and the season in question. For example, Stars (t-1) refers to the class signed earlier that year (the 2013 class when talking about performance in 2013) and Stars (t-4) is the class three years earlier (the 2010 class when talking about performance in 2010). The same is true of the second model, where I use points instead of star counts.

 

****Everyone should start reading here****

First, when we use 247sports points to predict performance, we get an R^2 of .38. This means that this recruiting measure explains about 38% of the total variation in a team's performance. Player evaluation, coaching, and luck (e.g., injuries) should explain most of the remaining 62%.

On average, a team is .68 points better per game for each freshman 4 or 5 star recruit, about .57 points better for each sophomore and junior, and .32 points better for each senior. The big drop from junior to senior is most likely a product of the best juniors leaving early for the draft (or otherwise leaving the program).

Using the coefficients from the second model, I plugged recruiting results from 2011, 2012, 2013, and 2014 to predict performance in 2014. I list the top 50 below.

The top 10 is a who's who of college football. It shouldn't surprise anyone that Alabama comes out on top with 18 5 stars in the last four classes. Ohio State's position at #2 is mostly a product of 48 4 stars in the last three classes; that is more than the bottom 2/3 of programs combined.

Some teams are over-performing their recruiting this season, others failing to meet expectations. Boston College at #69 beat #9 USC. BYU (#71) manhandled Texas (#10). And we were supposed to believe it was an upset when #27 Virginia beat #38 Louisville. Oklahoma is only one spot ahead of Tennessee by recruiting; the gap on the field is much larger. I don't think we will see Florida at #5 or Michigan at #11 at the end of the season. Oregon (#18) and Baylor (#31), on the other hand, are doing more with less.

Effective Points per Possession (opponent adjusted), 2005-2013

We recently added a new feature here at CFBTN. If you go to the Teams menu, you'll find the option to look at team statistics for 2005 to 2013. To celebrate, I've decided to do a short series on the best performers across a few interesting teams statistics since 2005. Today, we're going to look at opponent-adjusted effective points per possession (EP3+).

EP3+ is a measure of a teams offensive production. It is similar to points per possession, but has three advantages. First, EP3 takes field position into account. Second, it gives little weight to possessions in non-competitive situations. Third, EP3+ is adjusted for the strength of the competition. For the sake of brevity, I've listed those teams that have managed more than 2 effective points per possession.

• Only three teams made the list between 2005 and 2009. Six have made the list in the four season since.
• That makes USC's 2.29 in 2005 even more impressive.  
• Georgia Tech led the country in 2009 with 1.58, 3/4 of a point lower than Texas A&M (2012).
• Only Baylor has made the list twice (though the Bears slip through only if you round up in 2012). 
• Only Hawai'i (2006) tops 2 EP3+ from a non-BCS conference after adjusting for competition. Hawai'i scored 3.88 points per possession that season.

Points per possession, 2005-2013

We recently added a new feature here at CFBTN. If you go to the Teams menu, you'll find the option to look at team statistics for 2005 to 2013. To celebrate, I've decided to do a short series on the best performers across a few interesting teams statistics since 2005. In this first edition we're going to take a look at points per possession.

For the sake of brevity, I've limited the list to teams that score more than 3.5 points per possession, or half a touchdown.

• In nine seasons, 12 teams have managed to cross that threshold. 
• Florida, Wisconsin and Stanford did it twice (each in consecutive years). 
• Only Hawaii (2006) and Tulsa (2008) managed 3.5+ points per possession in non-BCS conferences. 
• The 2008 season was particularly prolific; four teams topped 3.5 and Texas was a few hundredths below. 
• Two of the nine national champions during this period scored more than 3.5 points per possession (Florida 2008 and Florida State 2013), and USC (2005) and Oklahoma (2008) lost title games.
• Louisiana Tech (2012) and Nevada (2009) led the nation in points per possession, but scored fewer than 3.5.
• Oregon has not scored 3.5 points per possession in any one season. Good teams that play fast are at a particular disadvantage in this case, because they can run out to big leads early. 

An Analysis of Recruiting Class Rankings

The most under-appreciated story from the 2014 NFL draft is the Longhorn shutout. The academic juniors and seniors during the 2013 season, the players most likely to be drafted in the 2014 NFL draft, signed in 2010 and 2011. According to 247sports composite rankings, Texas signed seven 5*s and 27 4*s in 2010 and 2011, yet no Longhorns were drafted. Baylor, on the other hand, signed one 4* in 2010 and another in 2011. Five Bears were drafted last month (though they were all drafted on the last day).

Obviously, recruiting class ranking does not translate directly to performance on the field. Good players can be poorly coached, bad players in high school can be confused for good ones, and injuries or off-field issues can stymie otherwise promising careers. But the elite programs, i.e., those that play for national championships, recruit well and develop players once they are on campus, not one or the other.

That said, I used the 247sports composite rankings to determine which teams, in theory, can put the most talent on the field in 2014 based on the strength of recruiting classes between 2011 and 2014.

• Total - total signees
• 5* - total 5*s, rank and percent of all signees that were evaluated as 5*s
• 4*+5* - see 5*
• Points - 247sports has a clever method for ranking classes that awards points for each recruit, with diminishing returns as class size increases. This the average points for each team over the last four years.
• Adj - A weighted average of Points that puts greater emphasis on the senior and junior classes and the least weight on the 2014 class. 

Alabama has signed 17 5*s over the last four years, and almost 3/4 of signees have been 4 or 5*s. The gap between Alabama and everyone else is substantial. Ohio State and Florida State have signed a lot of 4*s and 5*s, respectively, but they fall well short of the Tide in overall points.

The SEC has five teams in the top 9 and ten in the top 24. No other conference has even two teams in the top 10. You have to go down to #23 to find five Pac-12 teams, #26 to find five ACC teams, #32 to find five Big 12 teams, and #35 to find five Big 10 teams. 

One final points deserves attention. Because the adjusted points puts greater emphasis on senior classes, the difference between the average points and adjusted points gives us an indication of how talent is trending on campus. In most cases the two numbers are very similar, but if the adjusted points is higher than the raw average, this would indicate that the most recent classes were not as strong as past classes. By that metric, the team trending most strongly in the wrong direction is none other than our Texas Longhorns (tied with Rutgers). The explanations for this are many, but none is more important than the team that is trending most strongly in the right direction: Texas A&M.

Other teams trending in the right direction are LSU, Ole Miss, Kentucky, Arizona State, Wisconsin and Central Florida. Cal, Utah and Colorado are heading in the wrong direction. In fact, in the Pac-12 only UCLA, Arizona and Arizona State are trending up.

Texas A&M offense (2013) as animated gif

I said before the 2013 season that Texas A&M had the potential to field the best offense in the history of college football. They didn't reach that potential. They didn't have the best offense in 2013. But the potential was there: the Aggies finished 2nd in EP3+ (i.e., they had the 2nd most effective offense in the country), and Sumlin demoted his offensive coordinator before the bowl game.

The charts below show that the Aggies still moved the ball effectively and scored often. They reached the red zone on 45% of possessions, more than any other team in the country. But they fell short in one key area; the Aggies had the most explosive running game in the country in 2012, but they were below average in that statistic in 2013. Christine Michael took a share of the explosive runs with him to the NFL, and defenses were marginally better at bottling Manziel. As a result, the Aggies were 11th in EPA per rush while the top three scoring offenses (Florida State, Ohio State and Oregon) were 1, 2, 3 in EPA per rush.

Evaluating Coach Performance in 2013

With the many high profile coaching moves at the end of this past season, one topic that has been discussed heavily is coach performance.  Often, the conversations play out within the context of two related concepts: expectations and compensation.  Nick Saban is the highest paid coach in college football.  Meeting expectations at Alabama means winning a national championship.  Likewise, Mack Brown is the second highest paid coach in college football.  Texas is expected to win.  Texas did not win and Mack is fired.  Todd Berry at UL-Monroe, however, is not expected to win national championships by the administration, his players, or the fans.

While obviously there is clear relationship between expectations and salary, evaluating how well a coach has done in building a program takes years.  Keeping that in mind, one thing we can do to evaluate coach quality during 2013 is to compare the rank order of coach salary with the rank order of team performance.  Overall, there is a statistically significant correlation between the two, with increasing two spots in the rank order of highest paid coaches increasing your team's end of season rank by about 1 spot.  But who exceeds expectations?  Who fails most miserably?  Below are the top ten best and worst performers, subtracting the ranking of a coach's salary among all BCS teams from the team's end of season network ranking (more on the ranking here).

(*indicates first year; † indicates fired)

Coaches Who Exceeded (Salary) Expectations:

1. Roderick Carey (Northern Illinois; $500,000; Team Rank 38)*
2. George O'Leary (Central Florida; $1,500,000; Team Rank 4)
3. Dave Clawson (Bowling Green; $375,000; Team Rank 60) (hired by Wake Forest)
4. Matt Wells (Utah State; $500,000; Team Rank 52)*
5. Larry Coker (Texas-San Antonio; $350,000; Team Rank 72)
6. Matt Campbell (Toledo; $400,000; Team Rank 65)
7. Pete Lembo (Ball State; $400,000; Team Rank 66)
8. Joey Jones (South Alabama; $400,000; Team Rank 70)
9. Todd Berry (Louisiana-Monroe; $350,000; Team Rank 79)
10. Hugh Freeze (Ole Miss; $1,600,000; Team Rank 17)

Coaches Who Didn't Live Up to Their Pay Grade:

1. Sonny Dykes (California; $2,300,000; Team Rank 117)*
2. Bret Bielema (Arkansas; $3,000,000; Team Rank 93)*
3. Gary Patterson (TCU; $3,500,000; Team Rank 86)
4. Mike London (Virginia; $2,600,000; Team Rank 91)
5. Mark Stoops (Kentucky; $2,000,000; Team Rank 116)*
6. Charlie Weis (Kansas; $2,500,000; Team Rank 90)
7. Jim Grobe (Wake Forest; $2,400,000; Team Rank 88)†
8. Darrell Hazell (Purdue; $1,500,000; Team Rank 119)*
9. June Jones (SMU; $2,000,000; Team Rank 97)
10. Dave Doeren (NC State; $1,800,000; Team Rank 100)*

We can also take a look at who got hired and fired.  The +/- in parentheses is the difference between their ranking in salary and their team's end of season ranking, e.g. Texas finished ranked 44 but Mack Brown is the second highest coach, therefore -42.

Hired Coaches

1. Bobby Petrino to Louisville (Western Kentucky; +25)
2. Bryan Harsin to Boise State (Arkansas State; +15)
3. Steve Sarkisian to USC (Washington; +8)
4. Charlie Strong to Texas (Louisville; -1)
5. Chris Peterson to Washington (Boise State; -11)

Fired Coaches

1. Jim Grobe (Wake Forest; -55)
2. Mack Brown (Texas; -42)
3. Dave Christensen (Wyoming; -34)
4. Rich Ellerson (Army; -23)
5. Charley Molnar (Massachusetts; -11)
6. Don Treadwell (Miami (OH); 0)