Analyzing Social Television: Check-ins vs Nielsen Rating

by kyle on July 6, 2011 · Comments

Introduction

A couple weeks ago we published our first data science blog post where we investigated the connection between movie check-ins on GetGlue and box office draw. In this post we’ll look at how GetGlue check-ins align with a traditional metric of TV performance: the Nielsen ratings.

Check-ins vs Nielsen Rating: Single Show

For this study we looked at check-ins for episodes since the beginning of the year that we also had Nielsen ratings for. Our dataset was comprised of 367,369 check-ins, 1,649 episodes, and 237 shows. We removed episodes for shows with stickers to account for promotional bias and normalized the check-in counts to account for growth in the GetGlue user base. By doing this we were better able to compare episodes with check-ins in May to episodes with check-ins in January, for example.

We’ll start by looking at a single show — Big Bang Theory — a popular comedy on CBS. We chose this show because we had many Nielsen ratings for it and it’s easy to locate in the final graph for this section. Other shows that we plotted using this method correlate equally as well.
getglue_big_bang_theoryThe R2 for the trend line is 0.88. This indicates a very good fit for our limited sample size and tells us that there is indeed a strong relationship between check-ins and Nielsen rating, at least in this case. The mean R2 was 0.69 with a standard deviation of 0.26 for shows with a relatively significant amount of data (more than 10 episodes and more than 500 check-ins on average).

Check-ins vs Nielsen Rating: Broadcast Comedies

Now that we’ve seen that check-ins and Nielsen ratings correlate for a single show, let’s look at how all broadcast comedies correlate.
getglue_comediesThe R2 for the trend line is 0.55. Not as good as a single show, but still a fairly strong correlation. Assuming the Nielsen ratings are accurate, this tells us that the number of check-ins a show receives relative to its audience size is made up of many different factors. Even shows within the same genre may have different check-in patterns.

Check-ins vs Nielsen Rating: All Genres

Next we’ll look at all episodes across different genres and networks. Our hunch was that we would see more variation in the data, but that individual genres would form distinct curves. Indeed, this is what we found. This chart shows the number of check-ins versus the Nielsen rating for that episode. The color of each point on the graph represents the genre of the show for that episode and the shape represents whether the show was on broadcast or cable television.
getglue_genresThere is a lot to gleam from this chart, especially with regard to way the data seems to form clear clusters of TV shows.

We noticed several distinct groups that formed when we made the plot:

  • Supernatural teen dramas include Smallville, Vampire Diaries, and Supernatural — all of which are on cable except for Being Human, which is on SyFy. Tosh.0 and South Park — two cable comedy shows — appear in this cluster as well.
  • Teen dramas include Gossip Girl, Greek, Skins, 90210, and One Tree Hill.
  • Family sitcoms include How I Met Your Mother, Modern Family, Big Bang Theory, and Mr. Sunshine.
  • Crime dramas include NCIS, NCIS: Los Angeles, Criminal Minds, Criminal Minds: Suspect Behavior, The Mentalist, Castle, CSI, Body of Proof, Blue Bloods, and Hawaii Five-0.
  • Music/Dance reality shows include American Idol and Dancing with the Stars.

It is interesting to see the consumption habits of users on GetGlue. For example, teen dramas on GetGlue drive the same amount of engagement as Crime dramas, even though the estimated audience size is much lower for teen dramas compared to crime dramas. One possibility is the fact that GetGlue users are young and tech-savvy and not likely to fit into the crime drama demographic. It may also be due to the fact that young people are more likely to watch on DVR/Internet and the Nielsen ratings fail to capture that portion of the audience. Another reason is there could be something inherent about teen dramas that cause people to check-in more — possibly because there is more to talk about.

The difference between broadcast and cable TV consumption is interesting as well. Most of the low rated/low check-in shows tend to be female oriented cable reality shows such as Millionaire Matchmaker, Kate Plus 8, Real Housewives, etc. The highly engaged shows on cable tend to be Dramas and Comedies.

Check-ins vs Nielsen Rating: Men vs Women

Now that we’ve looked at genre, what happens if we breakdown the data by another factor, say gender? The next chart is the same as the last one, but instead of genres we colored the episodes by whether the show was mainly watched by men, mainly watched by women, or was watched roughly equally by both men and women.
getglue_genderThe first thing to notice is that there is a lot of pink. Although the GetGlue active user base is about 50/50 male/female 62% of the check-ins in our data set came from women. One of the reasons for the disparity may be that we did not include sporting events in the list of shows. Another reason may be that men are embarrassed to check-in to shows that are thought of as feminine and are therefore underrepresented in the sample. Women, on the other hand, do not feel the same way about shows that are considered masculine. Lastly, women may simply watch more TV and/or enjoy checking-in more. In addition to the makeup of users, the chart tells us that Men tend to favor comedies and sci-fi shows while women tend to favor dramas and reality shows.

We also thought it would be fun to list the top shows by male to female ratio. We only looked at shows that averaged more than 500 check-ins an episode to weed out some of the more obscure shows.

Top shows for women
1. The Bachelorette
2. Grey’s Anatomy
3. Dancing with the Stars
4. The Bachelor
5. Vampire Diaries
6. Pretty Little Liars
7. Inside The Royal Wedding
8. Real Housewives of Beverly Hills/Atlanta
9. Secret Life of the American Teenager
10. Gossip Girl

Top shows for men
1. Stargate Universe
2. The Cape
3. Archer
4. V
5. Lights Out
6. Smallville
7. South Park
8. Human Target
9. Outsourced
10. Perfect Couples

Yes, men like watching Perfect Couples apparently — they made up a whopping 60% of the check-ins. Check-ins from women outnumbered those from men almost 5 to 1 for the top women’s show, The Bachelorette and for the top men’s show, Stargate Universe, check-ins from men outnumbered those from women almost 3 to 1.

Conclusion

Solely looking at the Nielsen rating of an episode won’t tell you how many check-ins it received. However, when looking at more variables such as the genre of the show and the number of male and female viewers we can start to build an accurate prediction model.

We are excited about using GetGlue data to provide insights into social entertainment. This is only the first of many blog posts that will involve TV. As we gain more users and more data our insights will only continue to get better. Let us know your thoughts and if there is anything in particular that you would like to see us analyze in the future. Stay tuned for more!

Tags: | | | | |

Previous post: Happy 25th Anniversary to True Blue by Madonna

Next post: GetGlue hits an all-time high: 7,000,000 check-ins in June