This paper identifies the different ways in which media developments lead to ever smaller audiences being examined within people meter systems. The consequent unreliability of much people meter data is assessed and strategies proposed for dealing with this. A greater use of aggregation in the use of audiences and trading practices is proposed. Alternative techniques are also considered. A key principle is that the measurement of a number of small audiences in aggregate can be as reliable as the measurement of a large audience. Not only does research have to change but also the use of research.
This paper reports on an investigation of the relative performance of one, two and four week diaries for radio audience measurement. A special four week diary study carried out in 1992 by Research Services Limited for RAJAR has been re-analysed to evaluate the trade-offs between respondent fatigue, the performance of the extended reach model, sample efficiency and relative cost.
The paper reviews the concept of sampling error applied to various methods of sampling and television panels in particular.Approaches are described for calculating sampling errors in relation to: Programme/Commercial Break Ratings Commercial Impacts/Average Hours of Viewing Channel Reach Channel Share Schedule Reach and Frequency Changes Over Time An approach to the use of sampling error is suggested given that audience measurement figures have to be used in making decisions even when the results lie within statistical confidence limits. Panel analyses which can throw light on the contribution of sampling error are also discussed. An awareness of broad levels of sampling error are essential to users and should play a part in designing research and fixing sample sizes.
RAJAR, the new joint industry measurement system for UK radio, was launched in 1993. As in the past, the RAJAR survey uses a one-week self-completion diary and a model is required to estimate station or schedule reach beyond seven days. Changes in the radio market have led to changes in listening habits and a consequent need to update the extended reach model. The new model is probability based and has been validated using a one-off four week diary study. The published station reach build curves are used by several bureaux as a basis for providing the advertising industry with a practical system for schedule reach and frequency analysis. Two bureaux - IMS and Telmar - collaborated to ensure consistency in their approach to the estimation of schedule reach. However, each system offers several ways in which the base data can be used, essentially depending upon whether a schedule is being planned in broad time segments or using exact spots. These permutations are seen to generate greater inconsistencies in estimates of frequency rather than reach. Without an understanding of how each bureaux system will interpret an analysis specification, buyers and sellers of airtime could well be negotiating with inconsistent and misleading reach and frequency results.
There are currently 21 peoplemeter panels operating in 18 countries in Europe. At a European industry level, a great deal of work has been done on the subject of harmonization of television research. The new comprehensive survey from EAAA was conducted over a period of 18 months by Dr. Toby Syfret, media research advisor to the EAAA. The report is invaluable in identifying opportunities to reduce variability and increase the unity between systems. From an advertisers point of view, some of the variations that were shown in the report appear significant. At a very basic level, a multi-national advertiser needs to know if a GRP in one country is the same as a GRP in another. Concentrating on the methodological variations in the calculation and reporting of GRPs, Carat identified three areas for investigation: operational definitions, the components of a commercial rating (eg. guest viewing) and the definitions of reporting categories. In order to evaluate their effects, Carat commissioned RSMB to re-calculate the ratings for a selection of UK schedules using the respondent level data from the BARB panel. This simulation demonstrates that the conventions adopted in some countries can lead to large variations in reported commercial impacts.
The basis of a reach and frequency analysis is a count of the number of commercial spots seen by each individual in an advertising schedule. When this analysis is constructed from a people meter panel that reports on a daily basis, the obvious start point is to extract the sample of continuous reporters. This is defined to be the sample of individuals from whom a valid record of viewing was received for at least every day on which a spot in the schedule was transmitted. It is only for those individuals that we can construct a complete account of which spots they did or did not view. This was the standard approach adopted within the BARB Television Audience Measurement System in the UK prior to the launch of the new service in August 1991. However, this meant that users were forced to live with potential shortfalls in data quality and certain practical difficulties: -The continuous sample base decreases as the length of the schedule increases. A loss of 1% or 2% of the panel each day can easily compound to a loss of 10% to 20% over a four week campaign. Sampling errors would increase and there is potential for bias in the continuous sample. The demographic weighting of every continuous panel to target population profiles is not a practical option given a large number of reach and frequency analyses required on a very fast turn-around. Guest viewing could not be incorporated into the reach and frequency analysis because there was no such thing as a continuous panel of guest viewers. In fact, this would probably not be meaningful because, in the context of reach and frequency, guest viewing is really a surrogate measurement of panel members viewing in other (un-metered) households. Given the problems above, reach and frequency analyses could never be consistent with the published currency which estimated individual spot audiences from the full daily reporting samples, using a more sophisticated calculation procedure and incorporating guest viewing. The introduction of the new BARB Television Audience Measurement Panel last year created a requirement for change and an opportunity to re-visit the issues listed above.
A new BARB Audience Measurement Service for the UK was introduced in August 1991 using larger more dispersed samples. Among its most important innovations has been disproportionate demographic sampling in favour of certain key target audience sub-groups in all but two of its 14 regional panels. This paper summarises the nature and degree of the disproportionate sampling employed. Due primarily to the regionality of terrestrial commercial television, the BARB Audience Measurement Service is based on geographically disproportionately sampled regional panels. Now that demographic disproportionate sampling is also employed within them, rim weighting procedures are used at both the regional and national level. This paper summarises the actual and effective sample sizes achieved. The Replication Study published by Arbitron in 1977 demonstrated that for a television audience measurement panel the averaging of separate estimates increases the efficiency of the result In 1980 a corresponding study of the UK measurement service was commissioned by JICTAR from AGB, and its findings endorsed those of the Arbitron study. The authors have conducted a repetition of the 1980 JICTAR study on the new BARB panel using the data for February 1992 in order to evaluate the gains and losses in efficiency due to disproportionate sampling and the possible increases in efficiency due to larger more dispersed samples, people-meter measurement, and a more fragmented market. The results show that there has been an increase in the efficiency gains as a result of averaging, which more than compensate for the small loss in efficiency from weighting. Such gains in efficiency tend to be greater for small audience categories and lower ratings.