Friday, March 30, 2012
Ratings Glossary
We all use ratings information, but it's baffling to me how few users even understand how the metrics are derived. On a day-to-day basis I can honestly say it doesn't matter for most users. Like the speedometer in your car, it is usually enough to know what do do with the data. But there is a downside to ignorance. As a consequence, most of radioland was nonplussed by the legal challenges to PPM. Some of them did have merit, but PPM is undeniably superior to the system it replaces.The problem wasn't that Arbitron faced a possible rebuke. The problem was that the user base was so uninformed as to be unable to take a side.
Take for example the lawsuit Arbitron just settled a in California on March 21st. Arbitron was taken to task for using land lines for recruiting respondents for their sample. Minority households are known to be statistically less likely to have a land line. Many users of Arbitron measurement data didn't know if this was a benefit or a detriment to them. Personally I believe that accuracy is a benefit to all users, but I also understand the argument in favor of advantageous bias. It was the state of California itself that brought the suit, no radio stations were specifically named. More here and here.
In fairness I can hardly blame Arbitron. some of the nuance requires a degree in statistics. While the company has some serious, short-comings, namely their delivery method: CD-R via Fed-Ex one market at a time!... They do make an honest effort to disclose their terminology in a fairly straight forward way. Below I quote and elaborate on their basic terms and acronyms. All users should understand everything below. If you don't, you need to call your rep on Monday. Don't be embarrassed. If you are paying for this data, make it worth your money. More here.
AQH
Average Quarter-Hour: a 15-minute period
Average Quarter-Hour Persons (AQH Persons)
The average number of persons listening to a particular station for at least five minutes during a 15-minute period.
Average Quarter-Hour Rating (AQH Rating)
The AQH Persons estimate expressed as a percentage of the population being measured. This estimate is printed for the MSA and DMA. It can also be computed for the TSA.
[AQH Persons / PUR] x 100 = AQH Rating (%)
Cume Persons
The total number of different persons who tune to a radio station during the course of a daypart for at least five minutes.
Cume Rating
The Cume Persons audience expressed as a percentage of all persons estimated to be in the specified demographic group. [Cume Persons / PUR] x 100 = Cume Rating (%)
Daypart
A time period for which audience metrics are reported. In radio we use divide up the 24-hour day in to segments. These are for scheduling advertisements, but also as a way to easily divide up ratings data. The below only applies to weekdays. More here.
AM Drive: 6:00 AM to 10:00 AM
Midday: 10:00 AM to 3:00PM
PM Drive: 3:00 PM to 7:00 PM
Evening: 7:00 PM to 12:00 PM
Overnight: 12:00 PM to 6:00 AM
PUR
Persons Using Radio. This is described in some documentation as "population." This is somewhat misleading as it's actually just the subset of the population using radio.
Rating (AQH or Cume)
The audience expressed as a percentage of the total population.
[Listeners / Population] x 100 = Rating (%)
Reach
The total number of unique individuals exposed to a song or ad in a given daypart. Reach can be calculated for a single station or multiple stations.
Share
The percentage of those listening to a single radio in a given Metro who are listening to a particular radio station. [AQH Persons to a Station / AQH Persons to All Stations] x 100 = Share (%)
TSL
Time Spend Listening: The amount of time the average listener spent listening to a radio station during a daypart. The estimate may be expressed in number of quarter hours or in hours/minutes.
Thursday, August 04, 2011
If Your Household is Selected...
This is the actual letter that Arbitron sends out to sampled households.You may notice some vague dark smearing in the center of the letter. That is not a bad scan. that is a mark from the fresh, crisp one dollar bill that was enclosed. This letter is from one of my neighbors, I borrowed, scanned and anonymized it. As a member of the media, even in my capacity I would not be permitted to participate in their research.
You can click on the image to see it in high res.
Wednesday, February 17, 2010
PPM Explained
Math is hard, and lots of people are bad at it. They find it intimidating and frustrating. I include myself. It's a shame really since math is one of the few things which can contain actual, unadulterated, indisputable, non-partisan truth. To that point, despite those good qualities, it can be hard to understand. In that category PPM is so complicated that it's nearly impossible to explain to a layperson at all.What's changed is how listening is tracked. So lets start there. In PPM a signal is embedded in the radio stations broadcast. But what kind of signal? How is it embedded? the signal is called a watermark. It is a sub-audible signal in the audio. It's not in the carrier wave, it's not RDS. It's actually in the audio. Arbitron actually has a device in the transmission chain that embeds a station identification code into the audio at each individual radio station.
Watermarking isn't exactly inaudible. I've done a few controlled tests to see if I can hear a supposedly inaudible watermark. Interestingly, I can detect it by ear, but only when comparing to source audio (about 70% of the time.) The watermark is a modification of the source audio that either adds to or changes existing audio. Watermarks are measured in three very specific ways. Robustness, Perceptibility and Capacity. These first two exist in total opposition. RBR reported that this had even come into play with PPM deployment. I quote:
"Anecdotal reports from radio broadcasters say that Arbitron lowered the watermarking energy in response to complaints about the watermarking being audible in certain circumstances."A digital watermark is called robust with if it is detectable in the marked audio even if degraded by a number of transformations. Perceptibility is more binary. Since a watermark you can find by ear is useless we have a mushy adjective called "non-intrusive." Some watermarks are truly imperceptible, but the best ones are perceptible but non-intrusive. Capacity is how much data you can put in it.
There are three primary ways to embed water marks. They can be used in tandem: Spread Spectrum, Amplitude Modulation, and Quantization. You probably figured out by the quote that PPM includes some Amplitude modulation. The PPM system takes 10 audio bands between 1 and 3 kHz. In the image above I've increased the amplitude of that region so you can see all ten peaks. But to reduce perceptibility, all 10 bands don't run full-tilt all the time. Each channel will have 4 frequencies within it. The identifying data will only be injected in one of them at any time, and only then if the energy (amplitude) of the channel is great enough to render the injected watermark imperceptible. This is of course patented.Each of the four "injectable" frequencies represents 2 bits of data. I don't know the refresh rate but 500 ms rate was suggested and seems reasonable. This would produce a total of 40 bits per second or 2,400 bits per minute to encode a watermark. RBR suggested A unique ID might only consist of 50 bits. I think it could be done with less. There are only 14,000 stations in the US. Error rates will eat into this but multiple IDs per minute should be common. Compare that to a diary-based system.
Thursday, February 11, 2010
PPM and College Radio
So when I saw that WERS had a 0.9 Share... that was something that had to be reconciled. can a college station out-draw a commercial station? Hypothetically it's possible, it seems improbably when you think about marketing budgets but programming really is everything. In Boston WODS, an Oldies station is the number 1 station. That' s also unusual. So I took a look at the other PPM markets looks for this result. PPM is currently in the following 16 markets: Philadelphia, Houston, New York, Los Angeles, Chicago, San Francisco, Dallas, Atlanta, Washington DC, Detroit, Boston, Miami, Seattle, Phoenix, Minneapolis, and San Diego.To preface this I'll say now that I excluded NPR talk and Classical as they obviously demo
differently than rock. In New York, WSOU, WFUV, WFMU and WFDU don't even book WBGO came in at the bottom with a 0.3. That pattern continues almost without exception. In San Francisco KCSM gets a 0.5, in Dallas KEOM 0.6, in Houston KTSU 0.5, in Atlanta WCLK 0.5, in Philly WXPN has a 1.5, narrowly bested by WRTI with a 1.6, KPLU scored a 2.2 in Seattle, Minneapolis KCMP 1.6, and in San Diego KSDS got a 1.3.Of that entire set of stations all are jazz except for three: Both KCMP and WXPN are AAA and KEOM runs a Hot AC mix of sorts. KEOM, while it's book was respectable was still dead last in it's market. WERS leans hard toward Triple A, and it's rating was comparable to the other two, this level of scrutiny proves little. The outlier could be instead a consistent and repeatable sampling problem. But, perhaps this is real and can set a new bar for non-commercial stations. Maybe they can compete on their own terms.
Thursday, July 30, 2009
BREAKING NEWS
For months now we've all been reading about civic organizations, state attorney generals and radio stations suing Arbitron over PPM. I've written before about the natural predisposition of radio to reject all new numbers, especially any numbers that imply your listenership is down. But the reaction to PPM was way over the top.. I was even beginning to suspect that there was some validity to this just because of the wide-spread hostility. Then the story came out today. First thing today the Taylor on Radio-Info newsletter was in my inbox.“Hispanic stations lost an average of 42% of their AQH ratings with 18-49s, urban stations an average of 34%…” That’s in the latest analysis across all 20 PPM markets done by consultant Randy Kabrich of Arbitron’s previous-diary-book ratings (not shares, but AQH ratings), compared to the latest June People Meter numbers. Kabrich says by comparison, “the non-ethnic-formatted stations lost an average of 16% of their ratings but held their ranks.” While the more serious loss of AQH ratings by the Hispanic and black stations had notable consequences for their ranks. Hispanic stations dropped an average of five ranks, compared to the diary. Urban stations dropped two ranks. Kabrich’s conclusion – “from diary to PPM, in currency ratings, the urban and Hispanic stations have more than double the loss of the non-ethnic stations.”That just cant be right. First assumption. In the last 3 months 44% of Hispanic radio listeners didn't change the channel. Therefore the number was either too high before or too low. Around 4:30 FMQB sent out their email chiming in on the same topic with similar aplomb. they too were focused on Randy Kabrich, but hits a different more revealing quote:
"Share, which Arbitron uses, is not only not currency, it really can't be used to compare diary to PPM because of the vast differences in the PUR [Persons Using Radio]." Kabrich added, "Arbitron continues to use 12+ share in their comparisons which not only disguises a number of these issues, it is simply not currency and is irrelevant."Randy is calling out Arbitron on the misuse of sample size. Sample size matters. But, here's why I think Randy is full of crap. Arbitron is guilty of using undersized samples. But, even previous to this the data was blatantly, criminally, unreliably, flawed! It was a diary system loaded with omission, falsehood, bias, non-response
, error, and still yet some pretty small samples. If your data sucks it does not matter how big the sample is. It's just wrong. A bigger serving of crap is still crap. 60% more crap? Yes, still crap.The PPM system removes every kind of error except the under-sized samples. If they samples are heinously undersized what we'll see is a wide variance between books. I don't see that. I think the shift is largely a correction. But not solely.
4 weeks ago Randy made a play for the same story but was focused on Urban radio. Story here. Edison makes the same assessment that I did: undersized samples vary, bias samples skew. Edison research makes the appropriate assessment of Randy's report:
"I simply couldn't understand what Randy was seeing. In the case of KMJQ (the blue line), one sees consistent upwards movement, while the PPM sample goes both up and down. KBXX (the red line) sort of stumbles along until their numbers shot up dramatically, again while the PPM sample gyrates around...We also ran an Analysis of Variance test (ANOVA) and found that the sample size is not associated with the AQH for either station."
Thursday, May 14, 2009
Radio Ratings Part 6: Burke Vs. TRAC-7
Burke was founded in 1975 and in 1977 picked up Harry Bolger, the former development director of AC Nielsen to run things. In October of 1978 Burke Broadcast Research entered the radio ratings field with a telephone recall method. Hooper and Crossley had given up on telephone surveys in the 1950s. (Nielsen bought them.) The appeal of telephone surveys is that they are cheap. It's not accuracy, not demographics, and not speed. At the time they were competing with TRAC-7 which also used telephone surveys.In the mid 1970s Arbitron had a virtual monopoly. Pulse went under in 1973, and radio stations lacked options. In 1977 RAB and NAB cooperated in hiring Audits and Surveys Inc. to develop a competing system. Audits and Surveys Inc. did a study that basically stated the
obvious: you cant ask a respondent to recall a whole week of listening in one sitting. TRAC-7 focused on single day recall and large samples. It cost more but was more accurate. In contrast, Burke's sample wasn't even random...Industry endorsement was not enough. When shopping for favorable data radio readily picked the generic over the brand. In the end Arbitron crushed both. TRAC-7 folded up in June of 1979 and Burke at the end of December. Burke still exists but is now dedicated to marketing research. Audits and Surveys appears to as well...
Friday, January 18, 2008
Radio Ratings Part 5: The First Scandal!
I should clarify the title.. the first big scandal. There had already been little ones. Any ratings system that raises any station or programs standing inherently lowers another. It's a zero sum game. So for every step forward there was a flurry of back room deals, bribes, and sabotage to attempt to stop progress. Within the context of the resistance to Arbitron's PPM I am reminded of the previous hullabaloo in 1963. But it was the radio networks that invited statisticians into the fray in the first place. they were hiring them to debunk any system that was a detriment to their ad revenues.I am reminded of a quote from the film A Few Good Men "You can't handle the truth
It was radio station 1100 KJBS-AM that bitch-slapped the entire industry. They used both the
Hooper and Pulse data and were furious about the discrepancies between them. Station Manager Stanley Breyer ran a full-page advertisement in the trade magazine Broadcasting-Telecasting, headed, “Two Umpires Behind the Plate Isn’t Any Good in Broadcasting, Either.” (If anyone has a scan of that ad, or there whereabouts of Stanley Breyer please fill me in.)NAB formed a "Special Test Survey Committee" to investigate radio ratings. Essentially every business attached to marketing, advertising an media chimed on on the issue. The Advertising Research Foundation (ARF) polled its members in 1952, on the issue. Their members asked for an “ending [to] confusion in radio and TV audience ratings." A bigger uglier committee was formed. They took in hundreds of pages of testimony. Then as they were about ready to issue a report the chairman of the committee Dr. E. L. Deckinger actually said "This study will be the Kinsey Report of the TV industry.” The scandal actually really hurt some agency's. For some that had been already losing companies to the already dominant Arbitron and Nielsen it was the beginning of the end.
But somehow in 1954 the Recommended Standards for Radio and Television Audience Size Measurements report lacked the promised Kinsey-level of punch. Both NAB and ARF really failed on their own to produce any action beyond printing thick reports and thinning out the industry. So they took a shot at congressional action. It wasn't' the first time they'd tried to flex that lobbyist money but it was probably the first time they went the brute force method. (It began a long-standing tradition)Senator A. N. "Mike" Monroney, chairman of the Senate Interstate and Commerce Committee.
It was a smaller but far meaner and uglier committee. It was 1956 and they had already been having hearings on TV. Monroney prodded Senator Warren Magnuson (pictured) into investigating the ratings topic. Magnuson sent perfunctory letters to the ratings companies inquiring as to their techniques. Magnuson then held a hearing on June 26th 1958. The concepts were so academic, and the elected officially so ignorant that the hearing degenerated into an educational briefing. Nielsen, Sindlinger, Roslow, and and Seiler escaped unharmed. Later that year the TV quiz show scandal So history repeats itself. The dissenters of PPM are the stations who incur losses under the more granular data. thankfully the Senate Interstate and Commerce Committee is pretty disinterested in ratings. They're much more focused on white space auctions, satellite radio mergers and cross-ownership regulations...
Radio Ratings Part 4: Arbitron
Telepulse, Videodex Trendex, Telerad, TPI ratings, etc please bear with me. With all the buzz going on between Hooper, Crossley and Nielsen you actually could miss that there were other smaller ratings companies selling equally legitimate products. This included; Sindlinger, Trendex, Inc., the American Research Bureau..and many others. Some of them used very similar methods, others basically branded themselves on the fine differences. The three survey methods were:1. The Roster Recall Method
2. The Recall method
3. The Coincidental Method
Take Sydney Roslow and the Pulse for example. They used face-to-face interviewing instead of phoners. The interviewees were selected through the same kind of random sampling as Hooper and Crossley. But instead of asking about the last hour or the last day, The Pulse asked them to name the radio stations the interviewee had listened to in the last day, and week! Interviewees were shown a roster of local radio station callsto help them remember. It was called the "roster-recall" method. It sucked. Under this methodology interviewers went from home to home interviewing. It was fast and cheap, and produces a much larger sample, up to 67,000 households, sixty times the size of some competitors. but the samples were less random and therefore less representative.

Sydney was president of The Pulse for 34 years. He went on to pioneer demographic research breaking our Hispanics early in the game. But in the 1970s after a decade of growth they began to lose clients to their competitors. The diary technique was more trusted, and the radio pool was too small for the number of services. They folded in 1978. More here
Sindlinger and the American Research Bureau were all advocates of the diary method. Sample households are given a diary and asked to record all the radio programs they listen to in a week. On the upside it's cheap and produces a large sample. American Research managed 2,200 radio diaries. As an added bonus, the system also reached low-income households that lacked a phone. The phone surveys of hooper and Crossley eluded that demographic.
Sindlinger was fascinating in it's own right. Eighteen years after even CAB had dropped the recall method Albert Sindlinger brought it back from the dead in 1962. Pulse was using aided recall all throuout, but Sindlinger was using unaided recall alone. The president of ABC networks, Robert Pauley drove Sindlinger to break out the old, dead and debunked method. The reason was the previous to this Sindlinger was running an electronic monitoring system called RADOX but the system was ruled to infringe on one or more of Nielsens patents so proceeding with that platform woudl require paying a royalty to Nielsen.
ABC used the new pathetically innaccurate Sindlinger phone surveys to pick a fight with Nielsen over out-of-home listening. So Robert Pauley convinced Sindlinger to return to radio after being run off once already. The Sindlinger Radio Activity Service (RAS) used about 16,000 samples in his survey so it was at least more sound than the old Hooperatings. he also pioneered the use of computers. He was using an IBM 1620 to help crank out reports faster than Nielsen. ABC just liked it because the ratings were higer with Sindlinger than Nielsen. In the end NBC had enough of the malarkey and hired a NYU statistician to debunk Sindlinger. Af courswe even Albert himself knew it was all crap. In the book "Lies damned lies and Statistics he wrote:
The American Research Bureau's system by comparison was pretty substantial. The diary system worked and is still relied upon today. It was founded by Jim Seiler in 1949, and became Arbitron in the 1960s. Arbitron still uses diaries and still knows they have flaws but it's far superior to all three recall methods and lacks the out-of-home listening problem that plagued Nielsen. Today Arbitron has pioneered the Personal People Meter (PPM) a portable device that records radio listening everywhere the subject may travel. It's only in a few markets so far and it has some demographic issues in its sampling but compared to all previous methods it is the most reliable by far. More PPM here. Friday, March 16, 2007
Puerto Rico, The Market
The Arbitron-defined market of “Puerto Rico” is about the size of
Connecticut and with half a million more people. By comparison, Connecticut is parsed into the seven markets of Hartford, New Haven, New London, Fairfield, Bridgeport, Stamford and Danbury. somthing is already amis and I haven't even addressed topography.The mountain ranges central to the island mean there’s almost no signal overlap among the three main population centers, of San Juan, Mayaguez and Ponce. Almost any operator intending to cover the market needs a stick for each market. But this creates an ownership concentraion problem. The existing rules are based on a 30 percent horizontal ownership cap.
All along of course the Bush-appointed FCC comissioners have been trying to limit ownership caps entirely. This is not being done in defence of Puerto Rico. Mostly this is has been persued by lapdog congressmen the at the behest of their mediaowning campaign contributors. Thankfully in 2005 the U.S. Court of Appeals for the 3rd Circuit blocked their efforts. Of course FCC Chairman Kevin J. Martin continues to be a heathen jackass of perviously unheralded proportions. But let's get back to the island pseudo-state shall we?
As I said the FCC has been known to grant exceptions in Puerto Rico. Most recently Arso Broadcasting was permitted to add another station to its existing cluster of 13 stations (6 AM and 7 FM) signals in the market. On the table was 102.3 WMIO, serving Mayaguez primarily.
But as exciting as all thi sis, why is it that comperably sized connecticut is sliced into 7 markets and Puerto Rico only one? Is it racial? I doubt it. The fact is, that Hawaii has a similar problem, and it too is "one island, one market."
Tuesday, May 16, 2006
What is a listener?
I recently uncovered a fine peice of radio arcana regarding thsi more statistical side of radio. In order to estimate the number of listeners a radio staiton has in any given period of time, one must define "listener." Is that one song long? or the average legnth of one ad, about 30 seconds? Nope, actually the threshhold is currently set to five minutes.
The criteria for inclusion as a "listener" is five continuous minutes in one quarter-hour, per week recorded in an Arbitron diary. That sure isn't much usage, I will admit. Fortunatly for radio stations and their advertisers Arb diaries indicate a lot more individual radio usage than this bare minimum. But does PPM? that I do not know.
Their distant, but only real competitor Bridge ratings takes a harsh view on this "Bridge Ratings has no confidence in average-quarter-hour measurement nor its cousin time-spent-listening. The convoluted calculation required to generate such numbers raises the margin of error for such estimates beyond reliability. At Bridge Ratings more confident estimates rest with CUME and FAVORITENESS. Favoriteness simply seeks listenering behavior related to the station audiences listen to most often. Years of studies tell us that a listener devotes between 66 and 80 percent of their weekly listening to their favorite station. This statistic represents loyalty and when combined with cume reflects a station's ability to convert its cume to favoritness. " Favoriteness? Is that like Truthiness? http://www.bridgeratings.com/
Basically they avoid the traditional ratings schema entirely. they generate useful but unrelated quantizations of a stations listener dominance in a market. RAJAR in the UK has an interesting article on this as well: http://www.rajar.co.uk/documents/admap-paulkennedy.pdf
Monday, July 25, 2005
Markets Merging...
Arbitron releases it's new market ratings and definitions twice a year; each spring and each fall. The biggest shake up I remember in recent years was been the deletion of the Westchester market. Typically market changes involve the addition or deletion of far more tertiary markets. But in the not so distant past Arbitron has changed major markets. Once Dallas and Ft. Worth were separate market areas. Obviously, this was no small change. Even today their downtown areas are 30 miles apart. Can you imagine the chaos if they merged Baltimore and Washington D.C.? That's only 35 miles. It is equally possible. I examine this change focusing on one particular station was went from the top of its game to an unrated station because of this market change.KBOX-AM 1480 was the first all country 24-hour radio station in the Dallas Metro. This was historic. Group One Broadcasting of Texas acquired KBOX from Balaban Broadcasting in 1967 for $2 million dollars. KPCN-730 AM was the first country station in the area, starting in 1962, but broadcasted during the daytime only. KBOX quickly became the ratings leader for country music in Dallas. In its first ratings book, they moved from a 10.0 share Q3 1966 to a 12.1 share in Q1 1967!
KBOX-AM enjoyed six years of solid ratings, posting a high of 14.4 in the Q3 1967 book. In 1972 WBAP-AM began giving them stiff competition as a country-formatted 50k watt powerhouse. KBOX's ratings started to slowly sliding as a result. The Clear Channel owned WBAP was trying to chip away at their audience and it was working.
In the year 1973, the Arbitron markets for Dallas and Fort Worth were combined into one book, to reflect the merging of the two cities into one metropolitan area. KBOX, like other smaller stations, was hurt by this redefinition. KBOX did not even appear in the new ratings book. WBAP-AM became the clear country winner at that point, as KBOX barely reached the Ft. Worth half of the newly defined market.
In 1976 Group One applied for a nighttime power increase from 500 watts to 1,000 watts. The FCC approved it. This gave them a somewhat less marginal signal. It's probably what allowed them to struggle into the 1980's providing a decidedly more local service to the Dallas area. But by 1980 FM radio was growing in dominance and their ability to compete with a music format was waining.
on November 14, 1982 the KBOX call letters were dropped by Group One. KBOX-AM became KMEZ-AM and switched from its country format to a simulcast of the easy listening format of sister station KMEZ-FM. These days 1480 AM is occupied by KNIT-AM A Salem Media property running a decidedly more mundane gospel format.
You can hear a 2:00 .WMA of classic KBOX news from 1958 here:
http://www.michaelspears.com/kbox56k.wma
Great thanks to Michael Spears Entertainment just for existing.
great KBOX-AM history here:
http://www.knus99.com/kbox1480pt2.html
Tuesday, June 21, 2005
What market am I in?
Some portions of America are more inportant to a radio man than others. and some do not even exist at all. (in the tree falls in the woods sense) These are called "un-rated market areas." Wyoming contains a whole lot of un-rated market area. Thsi is beacuse its population density hovers around 2 per square mile. or about 400,000 people. i.e. less than the City of Providence Rhode island.
This handy-dandy map color codes everything a radio-man should care about:
http://www.arbitron.com/downloads/Arb_US_Metro_Map_02.pdf#search=
The kind People at Arbitron study this so that I dont have to.
The list is a tad out of date. but you can get a hard biannually updated list here:
http://www.arbitron.com/downloads/bluebook_sp05.pdf

