Towards a greater understanding of the illicit tobacco trade in Europe: a review of the PMI funded 'Project Star' report.

BACKGROUND: Following a legal agreement with the European Union (EU), Philip Morris International (PMI) commissions a yearly report ('Project Star', PS) on the European illicit cigarette trade from KPMG, the global accountancy firm.
METHODS: Review of PS 2010 report. Comparison with data from independent sources including a 2010 pan-European survey (N=18,056).
FINDINGS: Within PS, data covering all 27 EU countries are entered into a model. While the model itself seems appropriate, concerns are identified with the methodologies underlying the data inputs and thus their quality: there is little transparency over methodologies; interview data underestimate legal non-domestic product partly by failing to account for legal cross-border sales; illicit cigarette estimates rely on tobacco industry empty pack surveys which may overestimate illicit; and there is an over-reliance on data supplied by PMI with inadequate external validation. Thus, PMI sales data are validated using PMI smoking prevalence estimates, yet PMI is unable to provide sales (shipment) data for the Greek islands and its prevalence estimates differ grossly from independent data. Consequently, comparisons with independent data suggest PS will tend to overestimate illicit cigarette levels particularly where cross-border shopping is frequent (Austria, Finland, France) and in Western compared with Eastern European countries. The model also provides data on the nature of the illicit cigarette market independent of seizure data suggesting that almost a quarter of the illicit cigarette market in 2010 comprised PMI's own brands compared with just 5% counterfeited PMI brands; a finding hidden in PMI's public representation of the data.
CONCLUSIONS: PS overestimates illicit cigarette levels in some European countries and suggests PMI's supply chain control is inadequate. Its publication serves the interests of PMI over those of the EU and its member states. PS requires greater transparency, external scrutiny and use of independent data. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Introduction

In the 1990s, overwhelming evidence emerged, including from the industry's own documents, that smuggling formed part of transnational tobacco companies’ (TTCs) business model.1–6 This prompted investigations7 8 and lawsuits, including a 2000 civil action filed by the European Community in New York against Philip Morris International (PMI) and RJ Reynolds (now part of Japan Tobacco International; JTI), which led to legally binding agreements between these companies and the European Union (EU).9–11 As part of the agreement with PMI, the tobacco company pays KPMG, the global accountancy firm, to produce an annual report, known as Project Star (PS), on the illicit tobacco trade in the EU.9 While PS reports have been prepared since 2006 and used by PMI in lobbying against regulation,12 until 2011 the reports remained unpublished and therefore void of scrutiny.

It is in the TTCs’ interest to exaggerate levels of illicit tobacco and particularly the problem of counterfeit for two main reasons. First, TTCs use the illicit trade argument to appeal against tobacco control measures, including tobacco excise tax increases and packaging legislation;13–15 the counterfeit issue being particularly emphasised in plain packaging debates.16–20 Second, the deal on cigarette smuggling reached between PMI and the European Community,11 similar to deals reached with the other TTCs,21–23 requires PMI to make payments (known as ‘supplemental payments’) in the event of large seizures of its own contraband products, but does not hold PMI responsible for counterfeit product (see box 1).9 These issues and the apparent emphasis on PS as the main data source on illicit tobacco in Europe,24–26 including its use by the European Commission and national governments,27 underline the importance of assessing the accuracy of the PS data.

Illicit definitions (based on definitions in11 except where otherwise stated)*:

Contraband product: product that has been imported into, distributed in or sold in the territory of a member state, or was en route to the territory of a member state for sale in that member state, in violation of the applicable tax, duty or other fiscal laws of that member state or the European Commission (EC).

Counterfeit products*: products bearing a trademark of a cigarette manufacturer that are manufactured by a third party without the consent of that cigarette manufacturer.

Contraband and counterfeit (C&C)*: contraband and counterfeit product (as defined above) combined.

Philip Morris International (PMI) contraband: PMI product that has been imported into, distributed in or sold in the territory of a member state, or was en route to the territory of a member state for sale in that member state, in violation of the applicable tax, duty or other fiscal laws of that member state or the EC. Excludes counterfeit.

Transnational tobacco company (TTC) contraband: product of one of the TTCs that has been imported into, distributed in or sold in the territory of a member state, or was en route to the territory of a member state for sale in that member state, in violation of the applicable tax, duty or other fiscal laws of that member state or the EC. Excludes counterfeit.

‘Illicit whites’: non-TTC branded cigarettes that are legally produced but have no legitimate market. These are defined by the EC as: ‘brands manufactured legitimately in one market, either taxed for local consumption or untaxed for export, and sold knowingly to traders who transport them to another country where the products are sold illegally without domestic duty paid.’62

NOTE: In Project Star, PMI is the only TTC for which C&C is broken down into these two components, that is, PMI contraband and PMI counterfeit. The idea behind this is that TTCs bear responsibility for their own-brand contraband product but not for counterfeit.

Data definitions (based on details in

Legal domestic sales (LDS)=sales of genuine domestic product through legitimate channels. Derived from industry shipment data.

‘Outflow’=legal domestic sales taken out of the country. Estimated using empty pack surveys.

Legal domestic consumption=LDS—outflow.

Non-domestic legal (ND(L))=product brought into the market legally by consumers, for example, during legal cross-border trips, legal tourist shopping and legal duty-free. Estimated using consumer interview survey.

Non-domestic product (NDP)=product not originally intended for the market in which it is consumed. This is subdivided into legal and illegal product, that is, NDP=ND(L)+C&C.

*Our use of definitions such as ‘contraband and counterfeit’ does not signal an endorsement of their use (rather we believe the separation of these terms serves tobacco industry efforts to highlight counterfeit).

We obtained a copy of the PS 2010 results from the European Commission via public access to documents legislation. The date of this report is 24 May 2011.28 Substantial sections of this 362 page report obtained have been redacted. Around the same time, PMI placed a version of the same report on its website for the first time. This online report is dated 22 August 2011.29 This version is only 294 pages long, indicating that both copies provide only partial results.

This paper aims to examine the reliability of the PS methodology and results and to highlight key findings from this report which, due to its length, has hitherto largely been ignored by the tobacco control community as a source of data on illicit trade. To assess the results, this paper compares PS data with those from publicly available sources, including data from a recent, independent pan-European survey.30 Page numbers referred to are the page numbers from the version obtained under access to document legislation (24 May 2011) and correlate quite closely with page numbers in the online version.

Terminology used

For the purposes of this paper, we use terminology used in the PS report. Thus, contrary to usual practice where the term ‘contraband’ is used to cover all illicitly traded products,9 like PS28 and the relevant EU deal with PMI,11 we separate out two components—contraband and counterfeit (box 1). This does not signal an endorsement of this approach which is confusing and serves PMI's interest in emphasising counterfeit.

A review of the PS methodology

Model and primary data inputs

PS pulls together a large amount of data in a complex, iterative model that aims to provide data on the extent and nature of the illicit cigarette trade across the EU. There are three primary data inputs into the model: legal domestic sales data, empty pack survey (EPS) and consumer interview data (pp. 325, 328 and box 1). The starting point is legal domestic sales data for each country. EPSs in each country are then used to identify non-domestic packs and thus to estimate the outflow of cigarettes from each EU country to others. Non-domestic packs include cigarettes legally purchased abroad as well as contraband and counterfeit cigarettes. Consumer interview surveys, which ask about cigarette purchases while travelling abroad, are therefore used to estimate non-domestic legal product (ie, cigarettes legally purchased abroad, eg, via duty-free or legal cross-border sales). It is noteworthy that within the EU there are no limits on purchases for travellers returning from other EU countries as long as the tobacco is for personal use.31 Consequently, non-domestic legal cigarette use may be greater than in other regions, underlining the importance of accurately taking this into account when attempting to estimate illicit.

Data for all 27 EU member states have been collated since 2006 although not all data are updated annually. Below we review the primary and corroboratory data sources used in the model and the methodologies used to obtain these data inputs.

Legal domestic sales (sales of genuine domestic product through legitimate channels)

The methodology section indicates that legal domestic sales figures are based on In Market Sales data derived in most instances from shipment data (p. 327). Elsewhere (p. 11), it clarifies that these data are provided by PMI management although it remains unclear whether PMI provides data on other companies’ sales/shipments. Where shipment data are unavailable, AC Nielsen Retail Audit data or tax stamp data are used with PMI management inputting on the most suitable source or calculating an appropriate uplift (p. 342). The report specifically notes that shipment data are ‘insufficient’ (p. 340) for the Greek Islands. As the World Customs Organisation has previously concluded that Greece plays a key role in the illicit tobacco trade,32 33 PMI's apparent inability to supply accurate shipment data for these islands raises concerns.

Interview data used to estimate non-domestic legal sales (including cross-border shopping, legal tourist shopping and legal duty-free)

Insufficient methodological details are given to determine whether the results of the consumer interview surveys are likely to be nationally representative and thus valid. Although PS states that from 160 818 individuals who were contacted and agreed to be interviewed, 14 629 interviews were conducted—a response rate of 9.1% (data on p. 335)—this figure excludes non-responders (those contacted who refused to participate in the survey) and no data are given on respondents versus non-respondents. Such information is essential in order to accurately project results to national level particularly as the final sample sizes in some countries were small (200–300 in four countries and under 500 in another four) and the surveys are used to provide brand specific data.

A further issue is that these surveys aimed only to assess cigarettes bought while travelling abroad (p. 334). This would appear to overlook the legal purchase of cigarettes abroad by those living in border areas and by foreign students, immigrants and tourists. As such, these surveys may underestimate non-domestic legal sales which, in turn, will lead to an overestimate of the illicit trade. This is likely to be a particular issue in countries where cross-border sales are common place,34 35 as explored below.

EPS data (used to estimate non-domestic and counterfeit packs in each market)

A number of potential methodological issues are apparent with the EPSs (box 2), some of which are flagged in the report (p. 341). Importantly, EPSs measure non-domestic product and cannot distinguish which of this is legal (present as a result of legitimate cross-border trade, duty-free shopping or brought in by tourists, foreign students or immigrants) and which is illicit. PS attempts to estimate the legal element using consumer survey data but, as outlined above, these fail to account for some aspects of legal non-domestic consumption. Furthermore this, we suggest that the EPS data will tend to overestimate illicit trade in large part because of their urban bias and the role of industry in undertaking or commissioning the surveys (box 2).

Legitimate cross-border trade: A fundamental limitation of EPSs is that, while they can identify non-domestic product, they cannot distinguish which of the non-domestic product is legal (legal cross-border and duty-free shopping and products brought in by tourists) and which is illegal.

Timing of pack surveys: Other than for Germany (pp. 332 and 341), where packs are collected monthly, the timing of pack collections is not stated. Yet, pack collections undertaken during the tourist season will likely see more non-domestic packs collected. While this is acknowledged as a limitation, the report claims it only adjusts for this when the timing of EPSs varies between years and the method of adjustment is not clearly detailed (p. 341).

Location and methodology of pack surveys: EPSs may provide an accurate means of assessing the extent of tax avoidance when conducted in a well-documented, systematic and representative manner (eg, with random selection of areas using probability proportional to size).63 However, the methodological details given in Project Star are inadequate to determine whether such standards are met. For example, no description is given of how neighbourhoods and streets are selected and although the EPSs are used to provide national data, they are undertaken only in the largest cities in each country with no sampling of non-urban areas. The choice of area surveyed will impact on the findings: choice of brand and tobacco type varies with geography;64 those of lower socio-economic or educational status are both more likely to use illicit cigarettes65–67 and to live in cities;68 69 and both tourists and foreign university students will be over-represented in large cities. Consequently surveys undertaken in major cities are likely to over-represent non-domestic/illicit packs. Furthermore, growing evidence suggests that EPS sampling may be designed to increase the likelihood of finding non-domestic/illicit packs: industry surveys in the UK were specifically undertaken at sports events (p. 318)63 which are largely attended by men who have higher rates of illicit use;66 industry EPSs in Poland have provided estimates of tax avoidance greater65 (in once instance double)27 than that of independent estimates; while those in Germany have been found to systematically over-represent regions along the border and around US military bases where use of non-domestic product would be greater.70 This is supported by the reluctance of industry to publish or provide methodological details15 27 on EPSs. Finally, as littering is illegal in many jurisdictions,71 72 it could be hypothesised that those that litter may be more likely to engage in other illegal activity, including use of illicit product, and thus that the EPSs may overestimate illicit.

Investigator: In some countries (Austria, Belgium, Luxembourg), the national manufacturers’ associations (NMAs) carried out the surveys. As NMAs, like the companies they represent, have an interest in overestimating the extent of both illicit trade and counterfeit versus transnational tobacco company (TTC) illicit product, this represents a conflict of interest. A comparison of UK Tobacco Manufacturer's Association (TMA) data on illicit trade with that of Her Majesty's Revenue and Customs (p. 318) illustrates this point, showing that the TMA data give a much higher estimate.

Determination of counterfeit: Once packs are collected, tax stamps (or, where tax stamps are absent, health warnings and packaging characteristics) are used to identify domestic and non-domestic packs. Then packs are sent to the manufacturers to determine which are TTC contraband and which counterfeit (p. 330) despite the obvious conflicts of interest in the TTCs making this determination given that they only make supplemental payments for TTC contraband.

Corroboratory sources: reliance on industry data

Attempts are made to corroborate data from each of the three primary sources outlined above. The corroboratory sources of evidence (p 329) include an ‘expert interview programme’, ‘external public research’ and various data, but close examination shows that all of these draw heavily on industry sources.

Expert interview programme

As part of the 'expert interview programme' (pp 329 and 337), KPMG claims to have undertaken interviews with ‘specialists across a broad spectrum of areas and backgrounds’ (p. 337). Yet, none of the 15 experts in the field of illicit tobacco we contacted (including all those who have published extensively in the academic literature on this issue) were approached for interview.

External public research

KPMG claims ‘extensive research into external data sources’ in the 27 EU member states, yet the first three sources identified are ‘third party information available within PMI, press articles, retail trade and tobacco industry associations’ (p. 337). This, and a more detailed list of external data sources (pp. 352–7; three pages of which are redacted), suggests that these sources were rarely ‘external’ but that, instead, great emphasis was placed on industry generated data. While press articles might be seen as independent, evidence suggests they may be biased towards industry sources which tend to exaggerate illicit compared with independent estimates36 (see box 3). This in turn increases the likelihood of bias towards higher estimates and helps ensure the corroboratory sources concur with the primary sources as they are likely to be based on the same data.

The ‘external data source’ list (pp. 352–7) identifies eight press articles on Bulgaria (all English language) dated between 2009 and 2011 (although other data sources date back to 2006). The only other data sources cited on Bulgaria are National Statistical Institute data on GDP and the Labour Force Survey. Online investigative journalist reports that should have been recovered via routine internet searches, one critical of industry involvement in the illicit tobacco trade in Bulgaria73 and another on illicit in the region,45 are both omitted.

An examination of the seven of these eight articles we were able to obtain shows that in relation to illicit and tax they quote only from industry sources or ‘opponents’ to tax hikes (which one assumes to be industry or industry affiliated sources). Of the seven press articles listed, three claim tax increases will boost smuggling, two report that a third of cigarettes are smuggled and two are on smoke-free legislation and do not cover illicit trade. The articles cited in Project Star are as follows (the sources for the data given within each article are italicised):

On smoke-free legislation: no data on illicit.

http://tobaccoreporter.com/home.php?id=498&art=2424: A third of the tobacco products sold in Bulgaria are illicit, according to a Novinite story quoting sources within Bulgartabac.

http://www.novinite.com/view_news.php?id=111533: Opponents to the tax hike claim that it will boost cigarette smuggling and may lead to the bankruptcy of the dominant state cigarette maker Bulgartabak.

Trud Daily: unable to obtain.

http://www.stumbleupon.com/su/3Urbja/www.tobacco-news.net/bulgarias-new-cigarette-prices-come-into-force: No data but ‘Tobacco business representatives have criticised the idea of raising the excise duty, saying that it would only stimulate contraband and will not bring the expected revenue to the Budget’.

http://www.novinite.com/view_news.php?id=114806: Almost identical to (3) above. States ‘Opponents to the tax hike claim that it will boost cigarette smuggling and may lead to the bankruptcy of the dominant state cigarette maker Bulgartabak.’

http://sofiaecho.com/2011/01/19/1027294_bulgaria-to-introduce-complete-ban-on-smoking-in-small-cafes-in-mid-2011: ‘The decision was immediately opposed by small-size businesses, which said that the new restriction would produce a wave of bankruptcies.’

http://www.novinite.com/view_news.php?id=106476: ‘One third of the tobacco products in Bulgaria are sold on the black market according to state owned tobacco monopoly Bulgartabac.’

Recent research on tobacco industry conduct in Bulgaria43 highlights that the figures the industry uses in its discourse and lobbying on illicit tobacco in the country exaggerate the extent of the problem when compared with independent sources and yet industry figures feature prominently in press coverage. For example, an industry representative claimed in 2007 that planned excise increases would raise smuggling by around 20%,74 although an independent survey indicated that a hypothetical 20% increase in price would prompt only 11.2% of smokers to switch to smuggled cigarettes75 and the excise increase that was implemented (ultimately in 2008) increased weighted average prices by only 8%.76

In January 2010, Bulgartabac spokesmen warned in the press that excise increases would push the illicit trade up to 40% of Bulgarian cigarette sales that year,77 and in 2011, Philip Morris International (PMI) released data estimating that illicit trade had reached 34% of total sales in 201078 similar to the 30.7% figure for 2010 recorded in the KPMG Project Star report.28 Yet these figures are higher than three other data sources, two of which (Euromonitor and Eurobarometer data) would have been available to KPMG when they wrote their 2010 report. Euromonitor data estimate illegal sales as comprising 27% of the market in 200979 while Eurobarometer survey data suggest that 21% of Bulgarians thought they had bought smuggled cigarettes over a 6-month period in 2008.80 The 2010 Pricing Policies and Control of Tobacco in Europe survey suggests the extent of the problem is lower still with just 14.5% of Bulgarians reporting that their latest cigarette pack was smuggled.75 Those interviewed in the course of the Bulgarian research also highlighted how PMI attempted to widely publicise its 2010 data on smuggling. One interviewee, for example, stated: ‘...they [PMI] waved it around in the Parliament and in the media to the left and to the right...It caused a scandal in the Parliament: some people accept it, others do not.’43

Moreover, it is clear that exaggerated claims by industry sources regarding levels of illicit trade have continued in Bulgaria. A November 2011 press report for example gives even higher industry figures for the illicit trade in Bulgaria: ‘cigarette makers in Bulgaria have alarmed that the illegal cigarette market is continuing to expand and its current share is about 40%….This share has been 35% in 2010.’ Yet, the article simultaneously notes that tobacco excise revenues are increasing.81

Corroboratory data sources

Many of the corroboratory data sources are provided by PMI. The data obtained on ‘outflows’ (see box 1) are corroborated via ‘PMI management estimates’; estimates of legal domestic consumption are corroborated using prevalence and consumption data from PMI's Global Consumer Tracking Survey; non-domestic legal estimates are corroborated from PMI's duty-free market estimates and so on (p. 329). No further details (including sample sizes, sampling, definitions, etc) of these data sources are given and for some (consumption and prevalence estimates) it is unclear why more independent, routinely available sources are not used. As we show below, independent data would, in some instances, have provided very different results.

Review of results

Comparing prevalence data with routinely available sources

Much of the smoking prevalence data (provided by PMI) is inconsistent with independent data. Although PS prevalence figures would be expected to slightly outstrip WHO and OECD Organisation for Economic Cooperation and Development (OECD) estimates due to their narrower age range (table 1), some of the most notable discrepancies are lower PS prevalence estimates. For example, PS suggests smoking prevalence in Germany and Netherlands is 16% and 8%, respectively (p. 20). Anyone with knowledge of European tobacco control would suspect this is wrong; it would give Netherlands the lowest smoking prevalence in the world. Independent data suggest smoking rates are at least double these PS estimates: WHO data suggest prevalence in Germany is 34%, OECD data that it is 23%, while a peer reviewed study finds a prevalence of 30%.37 Similarly in the Netherlands, WHO and OECD estimate prevalence rates of 21% and 28% respectively while Nagelhout et al cite an estimate of 32%.38 Marked discrepancies are also noted in other countries (table 1).

Table 1

Comparison of Project Star (PS) smoking prevalence data with independent estimates (all figures are smoking prevalence, %)

Country	PS (legal age of cigarette use-64 years) (p. 20)	WHO HFA (daily smokers aged 15+)	Year of WHO HFA data	Difference between PS and WHO HFA*	OECD (daily smokers aged 15+)†	Difference between PS and OECD	PPACTE (smokers aged 15+)	Difference between PS and PPACTE	EB (smokers aged 15+)	Difference between PS and EB
Austria	28.0	23.2	2006	4.8	23.2	4.8	30.4	−2.4	34.0	−6.0
Belgium	12.0	20.5	2008	−8.5	20.0	−8.0			30.0	−18.0
Bulgaria	49.0	32.7	2001	16.3	29.1	19.9	40.9	8.1	39.0	10.0
Cyprus	29.0	26.5	2008	2.5	24.7	4.3			32.0	−3.0
Czech Republic	25.0	22.8	2010	2.2	24.3	0.7	29.1	−4.1	26.0	−1.0
Denmark	17.0	20.0	2010	−3.0	23.0	−6.0			29.0	−12.0
Estonia	33.0	26.2	2010	6.8	26.2	6.8			32.0	1.0
Finland	19.0	19.0	2010	0.0	20.4	−1.4	26.3	−7.3	21.0	−2.0
France	25.0	29.1	2010	−4.1	26.2	−1.2	27.5	−2.5	33.0	−8.0
Germany	16.0	33.9	2003	−17.9	23.2	−7.2			25.0	−9.0
Greece	30.0	31.9	2009	−1.9	39.7	−9.7	38.9	−8.9	42.0	−12.0
Hungary	31.0	31.4	2009	−0.4	25.9	5.1	35.5	−4.5	38.0	−7.0
Ireland	28.0	27.0	2008	1.0	29.0	−1.0	36.0	−8.0	31.0	−3.0
Italy	23.0	23.1	2010	−0.1	22.4	0.6	22.0	1.0	26.0	−3.0
Latvia	33.0	33.7	2010	−0.7	27.9	5.1	28.8	4.2	36.0	−3.0
Lithuania	27.0	21.8	2010	5.2	26.5	0.5			30.0	−3.0
Netherlands	8.0	20.9	2010	−12.9	28.0	−20.0			24.0	−16.0
Poland	28.0	27.0	2009	1.0	26.3	1.7	28.0	0.0	33.0	−5.0
Portugal	30.0	20.9	2006	9.1	19.6	10.4	32.4	−2.4	23.0	7.0
Romania	41.0	26.7	2011	14.3	20.2	20.8	26.1	14.9	30.0	11.0
Slovakia	21.0	19.4	2009	1.6	25.0	−4.0			26.0	−5.0
Slovenia	26.0	18.9	2007	7.1	18.9	7.1			26.0	0.0
Spain	25.0	26.2	2009	−1.2	26.4	−1.4	28.6	−3.6	35.0	−10.0
Sweden	10.0	13.6	2010	−3.6	14.5	−4.5	16.3	−6.3	16.0	−6.0
UK	15.0	20.0	2010	−5.0	22.0	−7.0	24.9	−9.9	28.0	−13.0

*Differences of 5% or over between PS and other prevalence estimates are indicated in bold text. Where there are consistent differences between PS and other datasets (ie, differences are seen in at least three datasets) these differences and the country name are indicated by shading.

†Uses data from 2008 or nearest year available.

Sources: WHO data: http://data.euro.who.int/hfadb/ downloaded1/3/13and17/04/2013; OECD: ‘Tobacco Consumption among adults’, in Health at a Glance: Europe 2010, OECD Publishing (Eurostat Statistics Database): http://www.oecdilibrary.org/docserver/download/8110161ec026.pdf?expires=1366276217&id=id&accname=guest&checksum=B1837069B1EEF937830FFDD8405DC2B5; EB: Eurobarometer 2009 data, published 2010.79

EB, Eurobarometer; OECD, Organisation for Economic Cooperation and Development; PPACTE, Pricing Policies and Control of Tobacco in Europe; PS, ‘Project Star’.

Not only do these data appear inaccurate but there appears to be a bias. PS prevalence estimates in Western European countries are lower (by 5% or more in Belgium, Germany, Greece, the Netherlands and the UK), but higher in Eastern European countries (by 5% or more in Bulgaria, Estonia, Hungary, Lithuania, Romania and Slovenia) compared with independent sources (table 1). The only country not fitting this pattern is Portugal. These prevalence data, which come from PMI's Global Consumer Tracking Survey on which no methodological details are given, are used to validate the sales/consumption estimates, also provided by PMI, which (once outflows have been deducted) form the basis for the denominator when calculating the proportion of consumption that is illicit—the key finding from the report (p. 42). Consequently, this bias will tend to underestimate legal consumption in, and thus potentially the proportion of illicit cigarettes in, the West.

Comparing the KPMG results with other data on illicit cigarettes

The most recent pan-European data on illicit cigarettes that can be compared with PS come from a 2010 survey conducted as part of the Pricing Policies and Control of Tobacco in Europe (PPACTE) project.30 This face-to-face representative survey conducted on 18 056 participants (8653 men and 9403 women) from 18 European (16 EU) countries (approximately 1000 participants per country) is described in detail elsewhere.30

The strengths of this survey include the representativeness of samples, the survey tool being developed by tobacco control experts, the standardised questionnaire and face-to-face interviews which enabled direct questions on illicit cigarette trade to be validated through the use of a ‘show your pack’ section. A relatively high consistency was found between the survey findings and those of the 2009 Eurobarometer survey (rsp ranging between 0.73 and 0.75 for various smoking patterns).30 The limitations were the sampling method not being identical in all countries, some very low response rates (range 11%–79%) and relatively limited individual country sample sizes.30

Data collected on the latest pack of cigarettes, based both on self-reported purchase behaviour and interviewer pack inspection, enabled estimates of the extent of illicit and non-domestic legal tobacco use (box 4). To compare PPACTE survey estimates with PS, we limited PPACTE data to subjects showing a 20- or a 10-cigarette pack (thus excluding smokers showing hand-rolled tobacco packs or other packs of cigarettes) from 16 EU countries. We then used cigarette consumption data (the number of cigarettes consumed per day or per week for occasional smokers) from PPACTE, combined with the data on provenance, price and pack verification (box 4), to estimate the proportion of cigarettes consumed that were illicit and non-domestic legal.

Illicit data*

Current smokers were first asked to identify the provenance of their latest pack from one of seven sources: (1) national legal tobacco shops; (2) vending machines; (3) the internet; (4) shops in other countries; (5) duty-free shops; (6) ‘individuals selling cigarettes independently at local markets, delivery service, door-to-door, just in the street’, or, for UK and Spain, cheap cigarettes sold from legitimate retailers; (7) offered by peers. Cigarettes purchased from source (6) were considered illicit.

To validate these direct questions on provenance, smokers were also asked the price of their latest purchased pack and to show the interviewer this pack from which the interviewer collected information on the health warning and tax stamp. The data on provenance, price and pack observations were then combined to provide a validated measure of illicit67 in which a pack was defined as illicit if it had at least one of the following four tax evasion indicators: (1) it was bought from illicit sources as reported by smokers (ie, source (6) above); (2) it had an inappropriate tax stamp (ie, a pack with a foreign stamp or absent tax stamp unless the pack had been bought over the internet, in another country or duty-free shop. According to the European Commission,82 no banderole was present on manufactured cigarette packs in Austria, Finland, France, Sweden and England and so packs from these countries with no tax stamps were not therefore regarded as illicit); (3) it had an inappropriate health warning (ie, a pack with health warnings in a foreign language or without health warnings, unless the pack had been bought over the internet, in other countries or in duty-free shops); and (4) its price was substantially below the known price in that market (for packs not bought over the internet, in other countries or in duty-free shops or offered and with a price lower than 70% of the lowest price of cigarettes in that country in 2010).

Non-domestic legal (ND(L)) data†

The proportion of cigarettes consumed that, based on both self-report (ie self-reported purchase from sources (4) or (5) above) and inspection of the latest pack, were identified as avoiding taxes via either purchase from shops in other countries or duty-free shops.

*These PPACTE data are comparable with Project Star (PS) counterfeit and contraband cigarettes as a share of total consumption (p. 42).

†These PPACTE data are comparable with PS non-domestic legal (ND(L)) cigarettes as a share of total consumption (pp. 31–32).

Comparisons of PPACTE and PS data on illicit cigarettes (Figure 1A) show that in some countries the estimates are very similar (Spain, Hungary), but the tendency is for PS to give larger estimates: PS estimates exceed PPACTE estimates in 12 countries, with absolute differences greater than 5% or more observed in eight (Ireland 15% gap, Bulgaria 15%, Finland 12%, France 11%, Austria 8%, UK 7%, Romania 6%, Greece 5%), while PPACTE estimates exceed PS in four countries, absolute differences of over 5% being observed in two of these (Poland −9% gap, Sweden −8% gap). The overall estimate for the 16 countries combined differs by more than three percentage points (9.7% in PS; 6.5% in PPACTE).

Figure 1

(A) Comparison of Project Star and Pricing Policies and Control of Tobacco in Europe (PPACTE) latest pack data on the extent of illicit cigarette trade in Europe (the proportion of cigarettes identified as illicit). (B) Comparison of Project Star and PPACTE latest pack data on the extent of the non-domestic legal cigarette market in Europe (the proportion of cigarettes identified as non-domestic legal).

The gap between PS and PPACTE estimates of illicit is greater in Western (the gap averages 5.8% across 10 Western European countries) than Eastern Europe (gap averages 0.8% across six Eastern European countries) (see online supplementary appendix table S1). The same geographic pattern is observed when comparing Euromonitor with PS data on illicit (see online supplementary appendix table S2 and text).

The comparison of PS and PPACTE non-domestic legal data (figure 1B) shows that the overall estimates are similar (2.7% in PS and 2.5% in PPACTE across the 16 EU countries combined). However, as with illicit, large discrepancies are seen in some countries. On non-domestic legal, PPACTE finds larger estimates in Austria (12.3% PPACTE vs 7.4% PS), the UK (4.7% vs 2.6%), France (8.5% vs 6.3%) and Finland (7.9% vs 6.0%), but smaller estimates in Latvia (1.4% vs 4.5%, Ireland (7.9% vs 10.2%) and Italy (0% vs 1.2%).

It is noteworthy that in three of the four countries where PS data provide lower estimates of the extent of non-domestic legal, Austria, France and Finland, PS data also provide higher estimates of the extent of illicit trade compared with PPACTE. This suggests that the higher illicit estimate is likely explained by the underestimate in non-domestic legal. These are also the three countries that a recent Eurobarometer survey finds have the highest level of cross-border shopping in the EU—over 10% of respondents had used cross-border shopping (Austria 11%, Finland 10%, France 14%) compared with 5% of EU respondents overall.35 These Eurobarometer data and official estimates from France and Finland on levels of cross-border shopping suggest that in these countries the PPACTE data are slightly more accurate than PS. In France, it is estimated that in 2010, 15% cigarettes in France were cross-border purchases39 (8.5% in PPACTE and 6.3% PS) and in Finland 12% were (7.9% PPACTE and 6.0% PS) (personal correspondence, Meri Paavola Ministerial Adviser, Ministry of Social Affairs and Health, Finland, 14/3/2012).

Perhaps, unsurprisingly in light of the above, official estimates of the illicit trade in France (5%40 and 6%39) are closer to the PPACTE estimates (2.4%) than the PS estimate (13.7%) as are official Finnish estimates (3%–5%, personal correspondence Meri Paavola, 14/3/2012) compared with PPACTE (3.5%) and PS (15.9%). We have been unable to obtain official estimates for Austria. However, where we have official data from other countries—the UK and Ireland only—PS illicit cigarette estimates are closer to national estimates in Ireland (19.3% PS, 4.6% in PPACTE and 14% in official figures),41 and in the UK (10.5% in PS, 3.4% in PPACTE and 11% in national data).42

Contraband, counterfeit and cheap whites

PS only splits the illicit data into separate contraband and counterfeit components for PMI brands. It finds (p. 40) that, in 2010, PMI contraband accounted for 24% of the total EU illicit market, five times the amount accounted for by PMI counterfeit (5%) and 2.5 times the amount accounted for by the three leading illicit white brands, Classic, Jin Ling and American Legend (10%) (p. 52). Moreover, the most frequently smuggled brands were genuine PMI brands—Marlboro (10% of illicit inflows), L&M (8%), followed by other PMI brands (6%) (p. 53). These figures outstrip the market shares for the leading illicit whites—Classic (5%), Jin Ling (3%) and American Legend (just 1%) (p. 53). Despite these ratios, the PS results section focuses far more heavily on illicit whites: of 34 pages on the illicit market (other than seizures which are dealt with separately), 17 focus exclusively on illicit whites while virtually no details are given on PMI contraband.

Seizures

PS's seizure data (pp. 75–8) focus only on PMI products. They suggest that seizure data overestimate the extent of counterfeit product when compared with the model estimates and that, despite the model suggesting that counterfeit declined between 2009 and 2010, the share of counterfeit among seized product increased. This suggests seizure data are unrepresentative of the illicit market and increasingly so. Thus, PMI counterfeited products reportedly accounted for 89% and PMI contraband just 11% of seizure volumes in 2010 despite model estimates that PMI contraband products outnumbered counterfeit by approximately five to one.

Discussion

Monitoring the illicit tobacco trade is difficult and publicly available data are scarce. PS therefore provides a potentially useful contribution to the debate. Its strength lies in the production of a useful model and providing estimates of the nature of the illicit cigarette market across Europe that are independent of seizure data on which, despite their bias, such estimates are usually based. Nevertheless, concerns are identified with lack of transparency, inadequacy of methodological details and subsequent quality of the data inputted to the model, the over-reliance on PMI data, and the lack of external validation. Consequently, it is difficult to be certain about the accuracy of the data inputs and emergent results. Furthermore, although the 2010 results provide estimates of the nature of the illicit cigarette market, identifying the proportion of illicit that is genuine PMI product, we note that this is not provided in all reports, limiting their utility.

Methodological and data concerns include the following. First, methodologies used to obtain the data inputs for the model, such as the EPSs, are inadequately described while the survey methodology underlying PMI's smoking prevalence data is not mentioned at all. Second, considerable reliance is placed on EPS data yet such surveys are limited in what they can provide and both theory and growing empirical evidence27 36 suggest that industry EPSs may overestimate levels of illicit (box 2).36 Third, interview data appear to fail to account for a number of sources of legal non-domestic product including cross-border sales in border areas and it is unclear how they are accurately projected to the national level given the lack of data provided on non-respondents. Fourth, over-reliance on industry data (as both primary and secondary data sources serving to corroborate each other) with what appears to be little effort to validate the accuracy of these data could further bias the results. Fifth, much of the prevalence data appears inaccurate and biased in a way that would tend to overestimate illicit in Western compared with Eastern Europe. Any bias will be exaggerated by the reliance on industry experts and press reports as corroboratory data sources which are likely to confirm rather than challenge the primary data. Indeed, press coverage on illicit has been shown to rely heavily on industry data which tends to exaggerate levels of illicit compared with other sources (box 3).27 36

The paucity of public data on the illicit trade makes it difficult to ascertain the accuracy of the PS model results. We have, however, identified two particular problems where concerns with the methodology and data inputs are supported by inconsistencies in the findings when compared with independent data. First, in countries where cross-border sales are a significant issue (Austria, France and Finland), levels of illicit are overestimated by PS. In other countries, the discrepancies were generally smaller and in two countries where official estimates were also available, PS would appear, in 2010, to provide accurate estimates. Second, the data will tend to overestimate illicit in the west compared with the east of Europe. This concern is identified through both apparent bias in the inaccuracies of the prevalence data and through comparisons with two external datasets on illicit trade which find greater overestimates of illicit in PS data in Western than Eastern European countries.

Despite these limitations, a number of PS findings are of interest. First, PS indicates that seizure data are unreliable, being heavily weighted towards counterfeit product, and should not therefore be relied upon to estimate the nature of the illicit cigarette trade. Second, it indicates that a significant proportion of the illicit cigarette trade in Europe is PMI's own brands, suggesting that PMI has yet to adequately control its supply chain. According to PS estimates, a quarter of illicit cigarettes in the EU in 2010 were PMI cigarettes; no figures are given for other TTCs so this is a minimum figure for the extent of illicit TTC cigarettes in the EU. The inability of PMI to provide adequate shipment data for the Greek market further suggests it has inadequate controls on its supply chain. That TTCs more generally are failing to adequately control their supply chains is suggested by work showing significant TTC overproduction of cigarettes in Ukraine and recent involvement in the illicit tobacco trade.43–45 The British law enforcement agency responsible for tobacco smuggling, Her Majesty's Revenue and Customs, likewise remains concerned that supplies of certain brands by TTCs to specific countries ‘are considerably higher than legitimate local demand.’46 Third, PS model data indicate that illicit PMI cigarettes are a significantly greater problem than counterfeit and cheap whites. At 24% of the illicit cigarette market in 2010, PMI illicit outstripped PMI counterfeit and leading cheap white brands by ratios of 5:1 and 2.5:1, respectively. The failure to provide data on PMI's illicit product in the most recent report therefore limits the utility of the report.

Despite these findings, approximately half of the 34 results pages on the illicit market focus on illicit whites while virtually no details are given on PMI contraband. This reporting imbalance, coupled with the use of seizure data rather than PS model data to describe the nature of the illicit cigarette market, and the focus on cheap whites and counterfeit, rather than TTC illicit, continues in other TTC communications. For example, at PMI's annual general meeting, Louis Camilleri, Chairman of the Board and Chief Executive Officer, argued that the ‘illicit trade is a growing threat to legitimate business. The threat comes from different sources, the most important of which are counterfeit, and … ‘illicit whites’’.47 Nowhere did Camilleri tell his shareholders that, according to PS, PMI genuine product was being smuggled in far greater quantities in Europe than cheap whites.47 Similarly a JTI presentation at a European Parliament hearing used PS data but emphasised only contraband and cheap whites.25 A further misrepresentation occurred in relation to the latest PS report which presented the 2012 data48 and was launched unusually early in an apparent attempt to influence debates over standardised packaging legislation in the UK. PMI's press release,49 which accompanied the report claimed ‘The UK, Greece, Italy, and Estonia are home to the sharpest increases in illegal cigarette consumption since 2011’.49 Yet, it failed to mention that the method of estimating illicit changed between 2011 and 2012 (from a pack to a cigarette based measure) in a way that will tend to overestimate illicit compared with previous years in two of these countries—the UK and Italy—where 10-packs are common. Even in the report, details of this change were obscured in a tiny footnote on page 199 of the report which had to be read in conjunction with a methodological appendix on page 215.

Implications for policy and practice

The key concerns underlying the above issues are those of transparency and external scrutiny. While recognising that law enforcement agencies addressing the illicit tobacco trade need to keep some data and investigations confidential, we believe much can be gained from greater transparency in line with broader EU commitments50 and obligations under Article 5.3 of the Framework Convention on Tobacco Control.51 Our results suggest that independent data were rarely used in triangulation, the methodologies and data presentation were insufficiently clear and PMI misuses the report's findings. Instead PMI control the data inputs and can thereby control the results, while recent press reports suggest that PMI also devised the PS methodology.52 These findings highlight that, at a minimum, there is a need for full transparency over, and independent scrutiny of, the methodologies underlying the data inputs (including details of the EPS and prevalence data methodologies) and the quality thereof and, wherever possible, for independent data to be used in the model (at the very least in data triangulation). Full, independent scrutiny of the results prior to publication should also occur although our experience shows this is both difficult and time-consuming.

We also recommend that the contract through which PS is commissioned, the way in which the methodology is devised and agreed, and details of any external scrutiny and sign off should all be made public. Furthermore, the full PS results should be made publicly available with no redactions. Specific changes could also be made to the PS methodology and report. First, our findings suggest a different approach is needed to deal with non-domestic purchases in border areas. Second, as all TTCs have now signed cooperation agreements with the Commission, and as other evidence suggests very recent JTI involvement in illicit,44 53 details of other TTCs’ contraband should also be made available.

The feasibility of the changes suggested above is difficult to determine without knowing more about the contractual arrangements underpinning PS. Although the production of PS reports was part of the legal settlement reached between PMI and the European Commission9 and PMI publicly claim that PS is ‘conducted by KPMG for PMI, the European Commission, and all 27 EU Member States’,54 the European Anti-Fraud Office dispute this stating that the report is in fact commissioned and financed by PMI alone.55 This underlines the need for transparency while indicating that the feasibility of change is limited and PS data should be treated with extreme caution.

More broadly, our findings, combined with what is already known about historical TTC involvement in the illicit tobacco trade,1–6 concerns that this involvement may be ongoing,43–45 53 overwhelming evidence of the TTCs’ efforts to undermine the science supporting tobacco control56 and growing evidence that industry data on illicit (including its EPSs) are misleading,27 36 raise serious questions about the appropriateness of PMI's role in PS. Concerns about the EU deals with industry on illicit have previously been raised,57 but have yet to be adequately addressed despite growing evidence that the tobacco industry sees the illicit trade as a means of securing access to and influence over regulators and undermining Article 5.3.26 58 59 The European Commission should, therefore, consider whether PS could be commissioned and undertaken independently of industry in line with recent UK National Audit Office recommendation that work on tax evasion should involve other sources of expertise, including academic input.46

Finally, our findings also suggest that the ‘supplementary payments’ (fines) that PMI are required to make as part of the deal reached between PMI and the European Commission60 are inadequate to deter ongoing involvement in the illicit trade. PMI pay five times the tax value of genuine products seized but only for seizures of over 90 000 cigarettes. On smaller seizures they pay only the tax that is due. As it is estimated that only one in 10 smuggled cigarettes will be seized,61 fines would have to be at least 10 times the value due to have the desired deterrent effect. Furthermore, PS shows that seizures significantly underestimate PMI contraband, suggesting that PMI will not suffer financially from involvement in illicit trade. In line with this we note evidence suggesting recent TTC involvement in illicit trade.43 44 53

This is the first rigorous evaluation of Philip Morris International's (PMI) Project Star report widely used as a source of data on the illicit cigarette trade in Europe.

It identifies major concerns with the quality of data inputs, indicates that Project Star tends to overestimate illicit cigarettes in countries where cross-border shopping is frequent and in Western Europe, and suggests that PMI has inadequate controls on its supply chain: almost a quarter of the illicit cigarette market in 2010 was found to comprise PMI's own brands compared with just 5% counterfeited PMI brands.

Project Star requires and would benefit from far greater transparency, external scrutiny and use of independent data. Its findings should be treated with caution.