MJA: Jamrozik et al., How the NHMRC got its fingers burnt

How the NHMRC got its fingers burnt

Konrad Jamrozik, Simon Chapman and Alistair Woodward

MJA 1997; 167: 372-374

On 20 December 1996, Justice J Finn of the Federal Court of Australia handed down his judgment in the case of the Tobacco Institute of Australia (TIA) Ltd and others v the National Health and Medical Research Council (NHMRC) and others. Justice Finn concluded that, in developing its recommendations for control of environmental tobacco smoke, the NHMRC's Working Party on Passive Smoking erred significantly in regard to the consultative procedures that it employed. As the following discussion shows, the legal decision has profound implications for the NHMRC and the provision of expert advice to Australian governments on matters of health and health policy. The discussion has been prepared by three members of the NHMRC Working Party, but reflects their personal views and not necessarily those of the Working Party as a whole or those of the NHMRC.

Introduction - The judgment - Implications of the judgment for the NHMRC - References - Authors' details

Make a comment - Register to be notified of new articles by e-mail - Current contents list - ©MJA1997

Introduction
The action brought by the Tobacco Institute of Australia (TIA) Ltd and others against the National Health and Medical Research Council (NHMRC) and others was concerned with matters of process, not of science. In his Reasons for Judgment statement,¹ Justice Finn said, "My concern is not with the science of the Draft Report nor with the public policy recommendations made" (p2). He ruled that the Working Party "failed to have regard to the submissions received" (p38) from the TIA, in that it was obliged to but failed "to give positive consideration to their contents as a fundamental element in its decision making" (p31).
Early in its deliberations the Working Party had decided that, in responding to its term of reference requiring it "to review the relevant scientific evidence linking passive smoking to disease in adults and children",² it would only consider evidence that had been published in the peer-reviewed scientific literature. It then applied transparent processes in making an extremely conservative calculation of the impact of passive smoking on the health of Australians. Next, it considered the present situation with regard to public policy and public opinion in relation to smoke-free policies. Finally, it produced a series of recommendations which proposed that, given the increasing amount of peer-reviewed scientific evidence of harm from environmental tobacco smoke and significant public concern and support for change, all workplaces and confined public places should become smoke-free.
The judgment
There were two major issues in the judgment.
1. What emphasis should be placed on different types of evidence?
Section 12 of the National Health and Medical Research Council Act 1992 (Cwlth) provides that, whenever the NHMRC proposes to develop guidelines or recommendations, it must publish a notice to that effect, seek submissions from the public and interested bodies on the matter, publish the draft recommendations and seek further submissions on these before issuing a definitive report. In the first phase of consultation initiated by the Working Party on Passive Smoking, in April 1994, the TIA submitted a large volume of material, much of which fell into three broad categories:
Reports on aspects of passive smoking. These were openly acknowledged as having been prepared at the request of the tobacco industry and as not being peer-reviewed.
Articles from the scientific literature that were readily identifiable as having been written by people known to have been associated with tobacco companies at some time (as judged by criteria developed by Bero and Glantz³). A considerable proportion of these consisted of critiques of previous original, peer-reviewed research on passive smoking rather than independently produced new research. The tobacco industry has been quite successful in pursuing advice received from consultants in 1978 that it should have as much material published as possible that casts doubt on the scientific evidence that passive smoking is harmful to health.^4,5 This dissident opinion then gains a life of its own by being repeatedly cited in peer-reviewed literature, often, but not always, by authors with known links to the tobacco industry.
Published proceedings of apparently reputable scientific meetings. However, these were, in fact, meetings convened by the tobacco industry and allegedly attended by invitation only.⁶
The Working Party was faced with the question of how to define "relevant scientific evidence". Errors may result from excluding material from non-peer-reviewed literature. However, in this instance, inclusion of reports from outside the peer-reviewed domain could produce a bias of another kind, as the volume of material generated by the tobacco industry far outweighs that sponsored by other interest groups.⁷ The Working Party judged that the literature on a long-defined and closely investigated topic such as the effects of passive smoking on health was likely to be relatively unaffected by orthodox publication bias, which favours early, false positive reports.^8,9 Accordingly, the Working Party chose to restrict its description of the scientific evidence to that available from peer-reviewed literature. This is the same position as that adopted by the International Agency for Research on Cancer in its regular reviews of the causes of human cancer.¹⁰ However, Justice Finn found that the NHMRC Act did not provide for the Working Party to apply a selective standard of whether the material it received was peer reviewed, and rejected the argument that application of such a standard was justified by its wide acceptance by the professional community in reviewing scientific evidence.
2. What constitutes proper consideration of submissions made to the NHMRC or one of its committees or working parties?
Justice Finn found fault with a mechanism adopted by the Working Party on Passive Smoking of having research officers summarise the individual submissions received and, in the first instance, circulate these summaries to members of the committee (with full submissions available to all members on request). The summaries were an aid to identifying efficiently to which of the evolving chapters of the Working Party's report submissions might be most pertinent (notwithstanding that no weight was given to non-peer-reviewed material).
Justice Finn did not overturn the Working Party's conclusion that passive smoking poses a significant risk to the health of Australians, but he stated that:¹
". . .'positive consideration' of a submission (i) would preclude the adoption of an a priori criterion which itself excluded a part or parts of that submission from actual consideration; and (ii) would involve 'an active intellectual process directed at that . . . submission'." (p31).

Justice Finn did not say explicitly what evidence of "proper consideration" or "an active intellectual process" would have satisfied him, nor did he limit or define a submission. Not defining these terms opens the door for any organisation with sufficient resources to forward excessive amounts of material for "active intellectual processing".
The lack of a definition of "proper consideration" is significant. A close reading of the judgment suggests that its strongest defence would have been for each member of the Working Party to have prepared a written critique on all of the four books, 122 scientific papers and other supporting documents sent to the NHMRC by the TIA, for those critiques to have been tabled and discussed at full meetings of the Working Party, for the minutes of those meetings to have recorded those discussions and their outcomes in detail, and for all members of the Working Party to have been prepared to have taken the witness stand to answer questions about the whole process and every document. This extremely onerous standard would stretch the present NHMRC and its system of committees to their very limits. While it is possible that in future a court might accept a less demanding standard, it is predictable that those appealing against the NHMRC would seek evidence of "consideration" in its widest sense.
Implications of the judgment for the NHMRC
Given that the 1992 NHMRC Act obliges the Council to embark upon a process of public consultation whenever it is proposing to issue regulatory recommendations (recommendations that require responses from relevant Australian governments) or guidelines on any matter falling within its charter, we now have a situation in which any party with a vested interest and sufficient resources can effectively slow down the NHMRC's provision of independent expert advice on important matters of public health policy, simply by submitting an overwhelming number of opinions and commissioned, non-peer-reviewed reports on the matter at issue. There would appear to be nothing stopping any manufacturer or other group involved in the production, sale or distribution of any possibly hazardous product from doing exactly the same thing.
The NHMRC Secretariat has a limited budget (approximately $5.5 million, half of which is for staff salaries)¹¹ and only a very small permanent workforce of 96 people that it supplements by temporary secondments from the Department of Health and Family Services. A very large proportion of its work is undertaken by academic and other experts who, while being mindful of the honour of being invited to serve the Australian public in this way, also feel duty-bound to accept the responsibility and to give their time and effort to providing the best possible advice to the community and its various governments. The recent case on passive smoking shows that, as the law stands, this rather unsophisticated arrangement is not adequate for dealing with potentially far-reaching questions of health and health policy where large vested interests are involved.
One response would be for the Federal Government to expand the Secretariat and budget of the NHMRC to the point where dedicated, in-house experts would be permanently available to consider any matter that was referred to the Council for consideration. Alternatively, the NHMRC Act might be changed to define more closely the requirements and mechanisms for consultation to prevent the process of formulating policy becoming unworkably bogged down. The NHMRC should not retreat to a professionally dominated model that excludes participation by the wider community. The challenge is to create processes that are open, inclusive and democratic, in the sense that all interests are treated fairly. The NHMRC must juggle efficiency, equity and scarce resources -- not an easy task. One practical suggestion is that, in future, it should restrict the length of submissions to ensure that the Council, with limited resources, can give fair consideration to all.
A third alternative is suggested by the history of tobacco control in the United Kingdom, where expert reports from the Royal College of Physicians have driven the debate on public policy.^12-15 In the present circumstances, the Federal Government might do well to commission outside expert groups such as the universities or the medical colleges to produce reports. The government would get its advice, but the process of providing it would not be held up by legal challenges.

References

Federal Court of Australia. Tobacco Institute of Australia Ltd & Ors v National Health & Medical Research Council & Ors [1996] 1150 FCA 1 (20 December 1996). Full text decisions of the Federal Court of Australia from 1977 on the Internet at: < URL: http://www.austlii.edu.au/au/cases/cth/federal - ct/ > >.
National Health and Medical Research Council Working Party. The health effects of passive smoking. Canberra: NHMRC, 1995: 3.
Bero LA, Glantz SA. Tobacco industry response to a risk assessment of environmental smoke. Tobacco Control 1993; 2: 103-113.
Roper Organization Inc. A study of public attitudes towards cigarette smoking and the tobacco industry. Vol 1. New York: Roper Organization Inc., 1978.
Chapman S. Tobacco industry memo reveals passive smoking strategy. BMJ 1997; 314: 1569.
Bero LA, Galbraith A, Rennie D. Sponsored symposia on environmental tobacco smoke. JAMA 1994; 271: 612-617.
Bero LA, Glantz SA, Rennie D. Publication bias and public health policy on environmental tobacco smoke. JAMA 1994; 272: 133-136.
Begg CB, Berlin JA. Publication bias: a problem in interpreting medical data. J Roy Stat Soc A 1988; 151: 419-463.
Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical resarch. Lancet 1991; 337: 867-871.
O'Neill IK, Chen J, Bartsch H, editors. Relevance to human cancer of N -nitroso compounds, tobacco smoke and mycotoxins. Lyon: International Agency for Research on Cancer, 1991. (IARC Scientific Publications No. 105.)
National Health and Medical Research Council. Review of the Implementation of the Strategic Plan 1994-1996. Document tabled in Federal Parliament, 26 August 1997.
Royal College of Physicians. Smoking and health. London: Pitman, 1962.
Royal College of Physicians. Smoking and health now. London: Pitman, 1971.
Royal College of Physicians. Smoking or health. London: Pitman Medical, 1977.
Royal College of Physicians. Health or smoking. London: Pitman Medical, 1983.

Authors' details

University of Western Australia, Perth, WA.
Konrad Jamrozik, DPhil, FAFPHM, Associate Professor in Public Health.

University of Sydney, Sydney, NSW.
Simon Chapman, PhD, Associate Professor in Public Health and Community Medicine.

Wellington School of Medicine, Wellington, NZ.
Alistair Woodward, PhD, FAFPHM, Professor of Public Health.

Reprints will not be available from the authors.
Correspondence: Associate Professor K Jamrozik, Department of Public Health, University of Western Australia, Nedlands, WA 6907.
E-mail: konrad @ talbot.epidem.uwa.edu.au

Make a comment - Register to be notified of new articles by e-mail - Current contents list - To top of article - ©MJA 1997

Introduction	The action brought by the Tobacco Institute of Australia (TIA) Ltd and others against the National Health and Medical Research Council (NHMRC) and others was concerned with matters of process, not of science. In his Reasons for Judgment statement,¹ Justice Finn said, "My concern is not with the science of the Draft Report nor with the public policy recommendations made" (p2). He ruled that the Working Party "failed to have regard to the submissions received" (p38) from the TIA, in that it was obliged to but failed "to give positive consideration to their contents as a fundamental element in its decision making" (p31). Early in its deliberations the Working Party had decided that, in responding to its term of reference requiring it "to review the relevant scientific evidence linking passive smoking to disease in adults and children",² it would only consider evidence that had been published in the peer-reviewed scientific literature. It then applied transparent processes in making an extremely conservative calculation of the impact of passive smoking on the health of Australians. Next, it considered the present situation with regard to public policy and public opinion in relation to smoke-free policies. Finally, it produced a series of recommendations which proposed that, given the increasing amount of peer-reviewed scientific evidence of harm from environmental tobacco smoke and significant public concern and support for change, all workplaces and confined public places should become smoke-free.
The judgment	There were two major issues in the judgment. *1. What emphasis should be placed on different types of evidence?* Section 12 of the National Health and Medical Research Council Act 1992 (Cwlth) provides that, whenever the NHMRC proposes to develop guidelines or recommendations, it must publish a notice to that effect, seek submissions from the public and interested bodies on the matter, publish the draft recommendations and seek further submissions on these before issuing a definitive report. In the first phase of consultation initiated by the Working Party on Passive Smoking, in April 1994, the TIA submitted a large volume of material, much of which fell into three broad categories: Reports on aspects of passive smoking. These were openly acknowledged as having been prepared at the request of the tobacco industry and as not being peer-reviewed. Articles from the scientific literature that were readily identifiable as having been written by people known to have been associated with tobacco companies at some time (as judged by criteria developed by Bero and Glantz³). A considerable proportion of these consisted of critiques of previous original, peer-reviewed research on passive smoking rather than independently produced new research. The tobacco industry has been quite successful in pursuing advice received from consultants in 1978 that it should have as much material published as possible that casts doubt on the scientific evidence that passive smoking is harmful to health.^4,5 This dissident opinion then gains a life of its own by being repeatedly cited in peer-reviewed literature, often, but not always, by authors with known links to the tobacco industry. Published proceedings of apparently reputable scientific meetings. However, these were, in fact, meetings convened by the tobacco industry and allegedly attended by invitation only.⁶ The Working Party was faced with the question of how to define "relevant scientific evidence". Errors may result from excluding material from non-peer-reviewed literature. However, in this instance, inclusion of reports from outside the peer-reviewed domain could produce a bias of another kind, as the volume of material generated by the tobacco industry far outweighs that sponsored by other interest groups.⁷ The Working Party judged that the literature on a long-defined and closely investigated topic such as the effects of passive smoking on health was likely to be relatively unaffected by orthodox publication bias, which favours early, false positive reports.^8,9 Accordingly, the Working Party chose to restrict its description of the scientific evidence to that available from peer-reviewed literature. This is the same position as that adopted by the International Agency for Research on Cancer in its regular reviews of the causes of human cancer.¹⁰ However, Justice Finn found that the NHMRC Act did not provide for the Working Party to apply a selective standard of whether the material it received was peer reviewed, and rejected the argument that application of such a standard was justified by its wide acceptance by the professional community in reviewing scientific evidence. *2. What constitutes proper consideration of submissions made to the NHMRC or one of its committees or working parties?* Justice Finn found fault with a mechanism adopted by the Working Party on Passive Smoking of having research officers summarise the individual submissions received and, in the first instance, circulate these summaries to members of the committee (with full submissions available to all members on request). The summaries were an aid to identifying efficiently to which of the evolving chapters of the Working Party's report submissions might be most pertinent (notwithstanding that no weight was given to non-peer-reviewed material). Justice Finn did not overturn the Working Party's conclusion that passive smoking poses a significant risk to the health of Australians, but he stated that:¹ ". . .'positive consideration' of a submission (i) would preclude the adoption of an a priori criterion which itself excluded a part or parts of that submission from actual consideration; and (ii) would involve 'an active intellectual process directed at that . . . submission'." (p31). Justice Finn did not say explicitly what evidence of "proper consideration" or "an active intellectual process" would have satisfied him, nor did he limit or define a submission. Not defining these terms opens the door for any organisation with sufficient resources to forward excessive amounts of material for "active intellectual processing". The lack of a definition of "proper consideration" is significant. A close reading of the judgment suggests that its strongest defence would have been for each member of the Working Party to have prepared a written critique on all of the four books, 122 scientific papers and other supporting documents sent to the NHMRC by the TIA, for those critiques to have been tabled and discussed at full meetings of the Working Party, for the minutes of those meetings to have recorded those discussions and their outcomes in detail, and for all members of the Working Party to have been prepared to have taken the witness stand to answer questions about the whole process and every document. This extremely onerous standard would stretch the present NHMRC and its system of committees to their very limits. While it is possible that in future a court might accept a less demanding standard, it is predictable that those appealing against the NHMRC would seek evidence of "consideration" in its widest sense.
Implications of the judgment for the NHMRC	Given that the 1992 NHMRC Act obliges the Council to embark upon a process of public consultation whenever it is proposing to issue regulatory recommendations (recommendations that require responses from relevant Australian governments) or guidelines on any matter falling within its charter, we now have a situation in which any party with a vested interest and sufficient resources can effectively slow down the NHMRC's provision of independent expert advice on important matters of public health policy, simply by submitting an overwhelming number of opinions and commissioned, non-peer-reviewed reports on the matter at issue. There would appear to be nothing stopping any manufacturer or other group involved in the production, sale or distribution of any possibly hazardous product from doing exactly the same thing. The NHMRC Secretariat has a limited budget (approximately $5.5 million, half of which is for staff salaries)¹¹ and only a very small permanent workforce of 96 people that it supplements by temporary secondments from the Department of Health and Family Services. A very large proportion of its work is undertaken by academic and other experts who, while being mindful of the honour of being invited to serve the Australian public in this way, also feel duty-bound to accept the responsibility and to give their time and effort to providing the best possible advice to the community and its various governments. The recent case on passive smoking shows that, as the law stands, this rather unsophisticated arrangement is not adequate for dealing with potentially far-reaching questions of health and health policy where large vested interests are involved. One response would be for the Federal Government to expand the Secretariat and budget of the NHMRC to the point where dedicated, in-house experts would be permanently available to consider any matter that was referred to the Council for consideration. Alternatively, the NHMRC Act might be changed to define more closely the requirements and mechanisms for consultation to prevent the process of formulating policy becoming unworkably bogged down. The NHMRC should not retreat to a professionally dominated model that excludes participation by the wider community. The challenge is to create processes that are open, inclusive and democratic, in the sense that all interests are treated fairly. The NHMRC must juggle efficiency, equity and scarce resources -- not an easy task. One practical suggestion is that, in future, it should restrict the length of submissions to ensure that the Council, with limited resources, can give fair consideration to all. A third alternative is suggested by the history of tobacco control in the United Kingdom, where expert reports from the Royal College of Physicians have driven the debate on public policy.^12-15 In the present circumstances, the Federal Government might do well to commission outside expert groups such as the universities or the medical colleges to produce reports. The government would get its advice, but the process of providing it would not be held up by legal challenges.
References	Federal Court of Australia. Tobacco Institute of Australia Ltd & Ors v National Health & Medical Research Council & Ors [1996] 1150 FCA 1 (20 December 1996). Full text decisions of the Federal Court of Australia from 1977 on the Internet at: < URL: http://www.austlii.edu.au/au/cases/cth/federal - ct/ > >. National Health and Medical Research Council Working Party. The health effects of passive smoking. Canberra: NHMRC, 1995: 3. Bero LA, Glantz SA. Tobacco industry response to a risk assessment of environmental smoke. Tobacco Control 1993; 2: 103-113. Roper Organization Inc. A study of public attitudes towards cigarette smoking and the tobacco industry. Vol 1. New York: Roper Organization Inc., 1978. Chapman S. Tobacco industry memo reveals passive smoking strategy. BMJ 1997; 314: 1569. Bero LA, Galbraith A, Rennie D. Sponsored symposia on environmental tobacco smoke. JAMA 1994; 271: 612-617. Bero LA, Glantz SA, Rennie D. Publication bias and public health policy on environmental tobacco smoke. JAMA 1994; 272: 133-136. Begg CB, Berlin JA. Publication bias: a problem in interpreting medical data. J Roy Stat Soc A 1988; 151: 419-463. Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical resarch. Lancet 1991; 337: 867-871. O'Neill IK, Chen J, Bartsch H, editors. Relevance to human cancer of N -nitroso compounds, tobacco smoke and mycotoxins. Lyon: International Agency for Research on Cancer, 1991. (IARC Scientific Publications No. 105.) National Health and Medical Research Council. Review of the Implementation of the Strategic Plan 1994-1996. Document tabled in Federal Parliament, 26 August 1997. Royal College of Physicians. Smoking and health. London: Pitman, 1962. Royal College of Physicians. Smoking and health now. London: Pitman, 1971. Royal College of Physicians. Smoking or health. London: Pitman Medical, 1977. Royal College of Physicians. Health or smoking. London: Pitman Medical, 1983.