The BMJ has published an analysis which highlights a number of flaws in the interim evaluation of the Responsibility Deal pledge to remove 1 billion units by 2015, which earlier this year suggested the pledge had been met ahead of schedule.
Carried out by the Sheffield Alcohol Research Group, the researchers say the findings do not mean the pledge is 'bad for public health', but that a number of factors undermine the validity of the Department of Health's evaluation report, stating:
"...we believe the data used in the analysis may not be fit for purpose, that the report makes simplistic assumptions about consumer responses to the pledge, and takes insufficient notice of confounding factors."
Data issues?
The Sheffield analysis says a fundamental challenge exists to evaluating the pledge given data available. Whilst beer and spirits are taxed relative to their alcohol content, wine and ciders are taxed by volume of product, irrespective of strength. As such tax data does not reveal the number of alcohol units in the market for wine and cider.
The authors highlight the market research data instead used to estimate wine and cider consumption was between 11% and 14% lower than equivalent HMRC estimates. The analysis appendix also presents "a case that 870 million of the 1.3 billion units removed from the market are accounted for by a change in 2011 to the way HMRC recorded beer data, rather than the pledge."
As such the Sheffield analysis states the claimed interim findings may be invalid irrespective of other concerns, but also explains the other shortfalls it believes exist. It argues that only one consumer response to lower strength drinks is considered - that people will consume the same volume of a drink but at a lower strength. As such this "raises the question whether new low strength products are primarily designed to reduce alcohol consumption, simply reflect what consumers want, or seek to increase consumption in new drinkers."
Other issues such as changes in tax and product strength lead the authors to state that "robustly separating the effect of the pledge from these factors requires a careful and detailed statistical analysis which the report does not provide".
An alternative approach?
The authors acknowledge that a robust evaluation of the pledge may not be possible without more detailed individual level longitudinal data being available. However they suggest that independent market research could "help understand how consumers are responding to new and revised products, in particular who is buying them and whether they are substituting for or adding to existing drinking."
Responding the the report, Miles Beale of the Wine and Spirit Trade Association said "It is disappointing that the public health lobbyists have again spurned an opportunity to achieve public health goals through joint working with industry and government”, and that the deal had meant "there are more low alcohol products on the shelves than ever before".
With a final official evaluation of the report still due, it will be interesting to see whether any future Government analysis takes account of the recommendations made - indeed a new Government may well opt to abandon the pledge for more 'measurable alternatives' as suggested.
Comments