|Assistant Secretary for Planning and Evaluation||
Guidelines for Ensuring the Quality of Information Disseminated to the Public
J. Substance Abuse and Mental Health Services Administration
The Substance Abuse and Mental Health Services Administration (SAMHSA) was established by Congress under Public Law 102-321 on October 1, 1992, to strengthen the Nation's health care capacity to provide prevention, diagnosis, and treatment services for substance abuse and mental illnesses. SAMHSA works in partnership with States, communities and private organizations to address the needs of people with substance abuse and mental illnesses as well as the community risk factors that contribute to these illnesses.
SAMHSA serves as the umbrella under which substance abuse and mental health service Centers are housed, including: the Center for Mental Health Services (CMHS), the Center for Substance Abuse Prevention (CSAP), and the Center for Substance Abuse Treatment (CSAT). SAMHSA also houses the Office of the Administrator, the Office of Applied Studies, and the Office of Program Services.
SAMSHA will ensure that disseminated information meets the standards of quality set forth in the OMB, HHS and SAMSHA guidelines. It is SAMSHA’s policy to ensure and maximize the quality, objectivity, utility, and integrity of information that it disseminates to the public. We strive to provide information that is accurate, reliable, clear, complete, unbiased, and useful. We are committed to integrating the principle of information quality into every phase of information development, including creation, collection, maintenance, and dissemination.
The pre-dissemination review described in the guidelines only applies to information disseminated on or after October 1, 2002. The administrative mechanism for correction applies to information that the agency disseminates on or after October 1, 2002, regardless of when the agency first disseminated the information.
They apply to information in all media - printed, electronic, audiovisual, verbal, and other. The Guidelines focus primarily on the dissemination of substantive information (i.e., reports, studies, summaries) rather than on the information pertaining to basic agency operations. Information that is disseminated at the request of SAMHSA or with specific SAMHSA approval through a contract or grant is subject to these Guidelines.
Scientific research papers, books, journal articles, and similar materials unless they have disclaimers alerting the audience that they do not represent official views of SAMHSA;
Other official reports, brochures, documents, newsletters, electronic documents, and audiovisual productions (i.e., a unified presentation, developed according to a plan or script, containing visual imagery, sound, or both, and used to convey information);
Oral or written information, including speeches, interviews, expert opinions, if representing SAMHSA's official positions, views, or policies;
Statistical information, including statistical analyses, aggregated information by program, Center or Office, or for SAMHSA as a whole, including funding information and histories (by funding mechanism, dollars, and/or other criteria);
All publications that carry the SAMHSA imprimatur, i.e., that are considered official SAMHSA publications or releases, must follow SAMHSA policy and procedures for preparation, review, approval, and distribution. The types of information disseminated by SAMHSA to the public include the following, however, the OMB guidelines are not directly applicable to all the information in these categories (See Section II):
Scientific Research Papers, Books, Journal Articles, etc. These may be in the form of a book, chapter of a book or textbook, monograph, journal article, proceedings, or the like. These are generally authored or co-authored by SAMHSA staff as part of their official duties and may be authored by working groups convened by SAMHSA. This category also includes grant project description (e.g., abstracts of funded grant proposal available through the NIH Computer Retrieval of Information on Scientific Projects database), bibliographies, collections of abstracts, reviews, and recurring reports. Summaries of grant findings may be released in the form provided by the investigator, or the investigator-supplied information may be used as the basis of a narrative describing a particular program area. These summaries may be used for many purposes, including in meeting annual reporting requirements, such as the Government Performance and Results Act (GPRA). Highlights of findings may be posted on the SAMHSA web site and may be found in testimonies and speeches by SAMHSA staff in many venues, including annual Appropriations Hearings, presentations on SAMHSA funding opportunities, and literature reviews.
Official Reports, Brochures, Documents, and Audiovisual Productions. This category includes:
Summaries and Analyses of Data Collection Activities. Periodically, SAMHSA publishes summaries of its data collection activities, such as the annual Summary of Main findings from the National Household Survey on Drug Abuse and preliminary estimates from the Drug Abuse Warning Network.
Evaluation and Program Findings. SAMHSA sponsors a variety of evaluation and other activities which lead to major publications such as the recently completed National Treatment Improvement Evaluation Study. SAMHSA also publishes information derived from its grant and contract sponsored programs such as the National Estimates of Expenditures for Mental Health and Substance Abuse Treatment.
Grant and Contract Funding Opportunities. The SAMHSA web site is the official vehicle for announcing the availability of SAMHSA funds for discretionary grants and contracts. Current and past SAMHSA funding opportunities are available on the SAMHSA web site, along with other information on grants policy, peer review, award data, contacts, application forms, and CRISP database.
Guidelines. This type of information is issued after careful review and deliberation of available scientific evidence, usually with the assistance of a panel of outside experts and is generally associated with a formal meeting or consensus panel specifically convened for the purpose. Prime examples are the Treatment Improvement Protocols published by the Center for Substance Abuse Treatment, SAMHSA.
Oral or Written Information. This category includes official speeches, editorials, commentaries, Letters-to-the-Editor only if they are provided by SAMHSA staff representing official SAMHSA viewpoints.
Statistical Information. Examples of statistical information published by SAMHSA include annual appropriations by SAMHSA's Centers or Offices, employment data (e.g., numbers of staff and staffing by professional degree), and data books (e.g., the annual Summary of Findings for the National Household Survey on Drug Abuse).
SAMHSA information is disseminated in many media, with the following four being the most common:
Print -- publications, books newsletters brochures, booklets, pamphlets, and reports.
Oral -- formal speeches, oral presentations, interviews, or commentaries for publication or broadcast; letters to the editor or correspondence likely to result in similar publications.
Audio-Visual -- broadcast scripts, audio or videotapes, and videocasting.
Electronic -- The SAMHSA web site at www.samhsa.gov.
At SAMSHA, the quality assurance process begins at the inception of the information development process. Further, SAMSHA reviews the quality (including the objectivity, utility, and integrity) of information before its is disseminated and treats information quality as integral to every step of the development of information, including its creation, collection, maintenance and dissemination.
All SAMHSA documents and audiovisuals must be prepared in accordance with professional and ethical standards, as well as generally accepted standards of good taste. They must be appropriate for dissemination by SAMHSA and must undergo appropriate review and approval prior to release. SAMHSA adheres to the laws and regulations applying to publications and audiovisual materials, including OMB Information Quality Guidelines, the HHS Printing Handbook, and relevant SAMHSA policy issuances. SAMHSA efforts to ensure and maximize information quality begin at the preparation stage and continue through the review and approval stages. When published electronically, existing SAMHSA policies developed in concert with Federal computer security laws provide appropriate security safeguards to ensure integrity of SAMHSA documents, i.e., that the information is protected from unauthorized access, revisions, corruption, or falsification. Further, SAMHSA is committed to demonstrating in its Paper Reduction Act (PRA) will be collected, maintained, and used in a way that is consistent with OMB, HHS and SAMSHA information quality guidelines.
Each publication must be accurate, both in specific details and in general impressions, and meet accepted standards of high quality. The OMB Information Quality Guidelines define quality as including objectivity, utility, and integrity. SAMHSA documents and presentations containing text and summary data must be objective and scientifically sound. Sources should be referenced for the convenience and further information of the reader. Where appropriate, supporting data should have full, accurate, and transparent documentation. Potential error sources affecting data quality should be identified and disclosed to users. Disclaimers should be used to distinguish the status of information (e.g., based on preliminary data or partial data set).
OMB Information Quality Guidelines hold "influential" scientific, financial, or statistical information to a higher standard of quality, requiring that the results be "substantially reproducible" by qualified third parties were the original or supporting data to be independently re-analyzed using the same methods. Making the data and methods publicly available will assist in determining whether analytical results are capable of being substantially reproduced.
A Government-wide directive requires Federal agencies to use plain language in all communications with the public. Plain language is writing that is geared to the target audience (i.e., a plain language document for a scientific audience may be different from a plain language document for the general public). Plain language is grammatically correct, with accurate word usage. It is clear and expresses exactly what readers need to know without unnecessary words.
Each publication should be prepared in a pleasing, dignified, and finished format appropriate for its intended use and audience. Use of color and typography must be in accord with Government Printing and Binding Regulations (www.house.gov/jcp/jcpres.pdf).
The purpose of the review process is to improve the quality of SAMHSA documents and to ensure the accuracy and validity of information intended to benefit the targeted audience, such as professional associations and the general public. All materials distributed by SAMHSA must be reviewed for accuracy, propriety, completeness and quality (including objectivity, utility, and integrity). The structure of the review and the types of reviewers will depend on the nature of the information as well as the targeted audience. For scientific and technical documents, peer review provides a level of quality control that is well recognized in the scientific community. According to OMB Information Quality Guidelines, material subject to formal, independent, external peer review can be considered to be of acceptable objectivity. However, this presumption of objectivity is refutable based on a persuasive showing by a complainant in a particular instance. The single most important determinant of a review groups' excellence and credibility are its members. Reviewers must: have appropriate scientific knowledge (as demonstrated by grant and publication records, academic degrees and honors); be respected in the scientific community; have breadth of expertise; be fair and objective; and, not be influenced by inappropriate personal interests (competition, scientific bias, personal antagonisms, or similar irrelevant factors). Because utility is one measure of quality, the review group should include representatives of the targeted audience to the degree practicable.
Statistical compendia and documents providing "influential scientific, financial, or statistical information" must be reviewed carefully. The Director of the Office of Communications must determine that these documents and the data on which they are based conform to the standards set forth here, and, if applicable, that the reported information and/or statistics reported therein be substantially reproducible.
A document that has obtained publication clearance for paper printing is often posted on the sponsoring Center's or Office's web page for greater accessibility. Such documents should not need additional approvals. Web documents with no print counterpart require content clearance by the Office of Communications to ensure that the information observes all applicable requirements governing information for release to the public.
Scientific papers, books, journal articles, brochures, documents, statistical compendia, newsletters, electronic documents, audio-visual productions, and similar materials.
SAMHSA encourages dissemination of scientific research and other information by its employees. Professional and scholarly writing, lecturing, editing, and publishing are an essential part of research, are in the public interest, and bring credit and distinction to SAMHSA and to the employees themselves. In assisting employees to share information about their official and professional activities, SAMHSA seeks to advance scientific knowledge and contribute to professional education. The first report of any scientific research results or other professional findings may be made by publication in a scientific or professional journal or presentation at a meeting of a professional organization. The choice of the journal or meeting to which reports are offered is the prerogative of the author(s).
To ensure and maximize the quality of information disseminated by SAMHSA employees, any non-extemporaneous presentation (written or electronic) by a SAMHSA employee on a subject related to his/her SAMHSA duties must be reviewed and approved through an internal SAMHSA process prior to submitting for publication consideration. With few exceptions, non-extemporaneous oral presentation on health policy or practice, or presentations with policy implications, must also be cleared in advance. Manuscripts intended for publication are customarily subjected to an external peer review process directed by the interested publisher, volume editor, or journal editor.
Publication or oral presentation of scientific and professional information by individual employees must conform to applicable laws and regulations, including OMB Information Quality Guidelines and the HHS Standards of Conduct Regulations.
Customary professional practices impose certain constraints on the degree to which SAMHSA employees maybe identified with the results of research and development work, including that obtained in collaboration with extra-mural grantees.(2)
All SAMHSA components have a formal internal operating procedure for identifying printing requirements and tracking publications. In addition, the concept clearance process for new publications is often the vehicle to track the development of a new publication and to identify its attendant printing requirements. Each prospective publication must be cleared through the originating SAMHSA component and then be approved by the Office of Communications, SAMHSA and the Office of the Assistant Secretary for Public Affairs (OASPA), HHS.
In brief, any official publication (including book, bibliography, chapter of a book or textbook, booklet, brochure, collection of abstracts, fact sheet, house organ, index, leaflet, manual, monograph, newsletter, pamphlet, review, periodical, proceeding, recurring report, statistical compendium, Internet document, audio-visual or the like) prepared by any SAMHSA component directly or through a contract must be sent by SAMHSA for HHS clearance. This clearance requirement does not apply to publication of articles in journals.
All audio-visual projects and exhibits must be cleared through OASPA, whether produced in-house or under contract. To obtain clearance for audiovisual products, including exhibits, Form HHS 524A must be completed, approved by the Office of Communications, and filed and approved with the Office of the Assistant Secretary for Public Affairs before production may begin.
Statistical compendia, including statistical analyses, aggregated information by program, Center, Office or for SAMHSA as a whole, including funding information and histories (by funding mechanism, dollars, or other criteria), require approval by the Director, Office of Communications, SAMHSA. The Director of the originating Center or Office is required to determine that the data conform to accepted scientific and quality standards and that the reported statistics are substantially reproducible.
In general, any writing by a SAMHSA employee on a work-related subject, whether or not intended for electronic or print publication, or for oral delivery, must be prepared according to accepted SAMHSA standards of quality, reviewed for substantiative content, and administratively approved. The purpose of the clearance process is to improve the quality of information and to ensure the accuracy, objectivity, utility, and validity of information.
Consistent with HHS Standards of Conduct (73.735-705 Writing and Editing), employees are encouraged to engage in outside writing and editing when such activity is not otherwise prohibited. If the writing or editing activity is related to the employee's official duties or other responsibilities and programs of the Federal Government, the employee must: (1) make no mention of his or her official title or affiliation with the Department, or (2) use his or her official title or affiliation with the Department, or (3) submit the material for clearance within the operating component, under procedures established by the component. When clearance is denied at any lower level, the employee shall have recourse for review up to the head of the principal operating component. This clearance will show there are no official objections to the activity, and the employee may then use his or her official title or affiliation usually without a disclaimer. Except where the requirement for disclaimer is waived as a result of official clearance, disclaimers shall be used in all writing and editing related to the employee's official duties or other responsibilities and programs of the Federal Government: (1) in which the employee identifies himself or herself by official title or affiliation with the Department, or (2) when the prominence of the employee or the employee's position might lead the public to associate him or her with the Department, even without identification other than by name. Disclaimers shall read as follows unless a different wording is approved by the Assistant General Counsel, business and Administrative Law Division, Office of the General Counsel: "This (article, book, etc.) was (written, edited) by (employee's name) in (his or her) private capacity. No official support or endorsement by (name of operating component or of Department) is intended or should be inferred."
Oral Information including speeches, interviews, expert opinions, only if representing SAMHSA views, official position, or policies
Any public statements, comments, or discussion of Federal policies or practices that are relevant to the employee's position or duties, draw conclusions, advocate or oppose professional practices or positions on subjects related to SAMHSA duties or that might otherwise be construed as reflecting an official position by SAMHSA, HHS, of the Federal Government, are covered by the OMB Guidelines and must be approved in the Office of Communications, SAMHSA.
No review or approval is required for non-official and private writing, speaking, and publishing by an employee unless his/her SAMHSA employment is likely to be regarded as influencing the content.
SAMHSA employees are responsible for the statements they make, regardless of whether or not they have been cleared. If one presents material that requires clearance but that has not been cleared prior to presentation, then the employee must inform the audience of the personal or unofficial nature of his or her views. An example of an appropriate disclaimer follows:
"This material is presented from my own perspective and should not be taken as representing the viewpoint of the Department, SAMHSA, or the Center for...."
SAMHSA employees shall not identify themselves as SAMHSA employees in unofficial materials prepared for dissemination to non-professional audiences, such as a letter to the editor. These materials must be reviewed prior to presentation in the Office of Communications, if the employee's identification with SAMHSA is to be shown, can be inferred, or is well known.
SAMHSA Clearinghouse Information
Clearinghouses often serve as a public point of contact and provide access to information about SAMHSA programs, conferences, and grant activities. These are often in electronic as well as hard copy format. Clearinghouses have been contracted to provide varying levels of service, including distribution of fact sheets, information packages, and publications; storage of materials; conducting outreach and promotion; and performing training and quality control for the clearinghouse staff. Clearinghouses may respond to inquiries about particular issues, ranging from information about available educational materials to statistical data. Clearinghouses are challenged to ensure accuracy and reliability of information, while continually striving to improve performance and response times. Clearinghouses also must determine which organizations are worthy of referral when customers need information that is not available at the clearinghouse and how to avoid implying endorsement.
SAMHSA has developed World Wide Web (www) Guidance which is available on the SAMHSA Intranet to SAMHSA staff. Web page creators must periodically review material on the web page to determine whether or not it is accurate and up-to-date. Information, particularly time-sensitive information, should be posted as soon as possible. Web page creators are expected to promptly update or remove out-of-date information or to notify Division of Information Resources Management that such actions must be taken.
Unless otherwise noted, it is safe to assume that information posted on public web sites within the "SAMHSA.gov" domain is considered to be "in the public domain." As such, others are free to establish links to SAMHSA online resources. In establishing such links, SAMHSA requests that others avoid creating the impression that SAMHSA is endorsing or promoting any particular product or service. In the same vein, any outside link to an external resource from a SAMHSA web site needs to be examined and approved on a case-by-case basis.
Web pages containing links to external pages not located on the SAMHSA server should include a link to a statement that releases SAMHSA from responsibility for the material included in the external web page. Again, it is important to avoid giving a user the impression that SAMHSA is endorsing information or a commercial product described in an external site. Disclaimers on copyright, endorsement (general and external links), liability, and medical information are also used, as appropriate, for individual web sites.
Each Center or Office designates a principal contact for information and approvals related to the development of web pages. SAMHSA personnel, contractors, and other authorized users must notify this contact person prior to setting up a web page on the SAMHSA server.
The SAMHSA Division of Information Resources (DIRM) is charged with providing, coordinating, and managing information technology for SAMHSA. In terms of computer security, there are three distinct objectives: (i) confidentially -- ensuring that there is no deliberate or accidental improper disclosure of sensitive automated information; (ii) integrity -- protecting against deliberate or accidental corruption of automated information; and, (iii) availability -- protecting against deliberate or accidental actions that cause automated information resources to be unavailable to users when needed. Information is accorded protection against disclosure, alteration, loss, or destruction, based on the degree of sensitivity.
DIRM staff use appropriate safeguards to protect data from improper disclosure by backing up critical data periodically, and, if a security incident occurs, by following proper incident response procedures. Supervisors are responsible for ensuring that employees, both Government and contractors observe all security requirements and that employees receive appropriate security training.
SAMSHA has developed administrative mechanisms to allow affected persons to seek and obtain correction of disseminated information that does not comply with OMB, HHS and SAMSHA guidelines.
At SAMHSA, the Director, Office of Communications (OC) will have overall responsibility for implementing SAMHSA's Information Quality Guidelines and coordinating this effort with other SAMHSA Centers and Offices. All complaints may be sent to:
Office of Communications
Substance Abuse and Mental Health Services Administration
1 Choke Cherry Road
Rockville, MD 20857
OC is the central office for communications at SAMHSA. As such, OC takes the lead across SAMHSA for setting communications policy and for communicating information about SAMHSA programs, issues, and accomplishments to the public, public interest groups, and to the scientific community. OC is the communications link between SAMHSA's components and the Office of the Assistant Secretary for Public Affairs (OASPA) in HHS and serves as the coordinating office or central source for matters related to SAMHSA publications, including printing, HHS/SAMHSA clearance and review procedures, Joint Committee on Printing, U.S. Congress, and Government Printing Office printing and binding regulations, and copyright rules. Among its many activities, the office produces and distributes a number of publications that highlight SAMHSA achievements. OC also support and coordinates the principal contents of the SAMHSA web site (www.samhsa.gov).
As the lead office for SAMHSA Information Quality, OC responsibilities include:
Developing policies and procedures to effectively meet the requirements of the OMB Information Quality Guidelines;
Providing information and/or training to SAMHSA staff on their responsibility in meeting Federal requirements and SAMHSA policies on ensuring the quality of information disseminated to the public;
Assisting in the review of information quality complaints;
Reviewing the proposed SAMHSA Center or Office response for appropriateness and assisting in finalizing a response;
Establishing a tracking database for complaints, with information on the type of complaint and its disposition, whether the complaint was deemed inapplicable or frivolous, and any resolution or corrective action taken;
Submitting an annual report on behalf of SAMHSA to HHS with the number and types of complaints, and the actions taken, in time for the HHS to report to OMB by January 1 (beginning in 2004);
Posting on the SAMHSA web site any further clarifications, guidelines, and Frequently Asked Questions (FAQs) about handling SAMHSA information complaints;
Making available examples of typical complaints and appropriate responses collected from SAMHSA Center
To seek a correction of information disseminated by the agency, individuals should follow the procedures described below.
The complaint shall contain
Complainants should be aware that they bear the ‘burden of proof’ with respect to the necessity for correction as well as with respect to the type of correction they seek.
Based on a review of the information provided, the agency will determine whether a correction is warranted and, if so, what action to take. The agency will respond to the requestor by letter or e-mail. The agency's response will explain the findings of the review and the actions that the agency will take, if any. The response will consider the nature and timeliness of the information involved and such factors as the significance of the correction on the use of the information and the magnitude of the correction. The response will describe how the complainant may request reconsideration. The agency will respond to all requests for correction within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date.
If the individual submitting the complaint does not agree with the agency's decision (including the corrective action, if any), the complainant may send a written hard copy or electronic request for reconsideration within 30 days of receipt of the agency's decision. The appeal shall state the reasons why the agency response is insufficient or inadequate. Complainants shall attach a copy of their original request and the agency response to it, clearly mark the appeal with the words, Information Quality Appeal and send the appeal to the specific agency appeals address.
The agency official who resolved the original complaint will not have responsibility for the appeal.
The agency will respond to all requests for appeals within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date.
SAMHSA may respond in a number of possible ways, including:
The Information Quality Guidelines require that "influential" scientific, financial, or statistical information in official Government documents must be based on studies that can be substantially reproduced if the original or supporting data were to be independently re-analyzed using the same methods. "Influential" when used in the phrase "influential scientific, financial, or statistical information" means that SAMHSA can reasonably determine that dissemination of the information will have or does have a clear and substantial impact on important public policies or important private sector decisions or will have important consequences for specific health practices, technologies, substances, products, firms, etc.
SAMHSA is committed to applying rigorous scientific standards to ensure the accuracy and reliability of research results. For scientific and technical documents, the scientific community recognizes peer review as the primary mean of quality control.
To facilitate the replication of scientific and other influential information by qualified third parties, SAMHSA continues to encourage the sharing of data and methods where practicable. Since the influence and implications of SAMHSA-disseminated information cannot always be truly anticipated, all SAMHSA scientific reports are expected to state clearly how analytic results were generated -- the specific data and sources used, data collection methodologies, various hypotheses and assumptions, specific analytic methods, statistical procedures, sources of bias or measurement error -- making the analysis sufficiently transparent so as to be capable of being reproduced. SAMHSA advocates the archiving of data through the Substance Abuse and Mental Health Data Archive or similar archive of publicly available data to facilitate the reproducibility of influential information.
HHS Standards of Conduct Regulation (45 CFR 73.735-705). Updated October 1, 2000.
HHS Printing Handbook, September 1998.
Office of Management and Budget Guidelines for Ensuring and Maximizing the Quality, Objectivity, utility, and Integrity of Information disseminated by Federal Agencies. Final Guidelines. January 3, 2002. www.whitehouse.gov/omb/fedreg/reproducible.html
Office of Management and Budget Circular No. A-130 Revised (Transmittal Memorandum No. 4) Management of Federal Information Resources. November 30, 2000.
(1) In general, special terms and conditions of a grant award will specify ownership rights to data. For this purpose, "data" means recorded information, regardless of the form or media on which it may be recorded, and includes writings, films, sound recordings, pictorial reproductions, drawings, designs, or other graphic representations, procedural manuals, forms, diagrams, work flow charts, equipment descriptions, data files, data processing or computer programs (software), statistical records, and other research data. Grantees are required to place an acknowledgment of SAMHSA grant support and a disclaimer, as appropriate, on any publication written or published with such support and, if feasible, on any publication reporting the results of, or describing, a grant-supported activity. An acknowledgment shall be to the effect that: "This publication was made possible by Grant Number ____ from _________" or "The project described was supported by Grant Number ___ from ______" and "Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the (name of awarding component or SAMHSA)."
(2) The conditions allowing SAMHSA staff to be (co-)authors of publications under SAMHSA extramural awards ordinarily arise only from contracts and cooperative agreements, where, by definition, there is substantial programmatic (i.e., scientific-technical) staff involvement. To merit approval for (co-)authorship on publications from extra-mural awards (including grants, contracts, and other award mechanisms), SAMHSA staff must have played a substantial role beyond normal program officer duties, including the following: originating the specific ideas that led to the research activity and manuscript; performing significant portions of the activity, and participating actively in preparing manuscripts.
Last revised: December 13, 2006