You are here
Home > Medical Calculator > Grace Principles Validated Checklist

GRACE Principles


GRACE VALIDATED CHECKLIST


Background



A Validated Checklist for Evaluating the Quality of Observational Cohort Studies for Decision-Making Support

GRACE: Good ReseArch for Comparative Effectiveness

The GRACE Checklist is designed for the assessment of observational studies of comparative effectiveness in terms of their quality and usefulness for decision-making. The checklist was developed from a review of the literature with guidance from recognized experts in this field. The content includes questions about data and methods. One hundred and thirteen (113) volunteer testers have rated 280 articles. Validation activities have documented the usefulness of all 11 questions in this checklist. Approaches to scoring are under consideration. The GRACE Initiative has been spearheaded by Quintiles Outcome in collaboration with the National Pharmaceutical Council. GRACE contributors represent perspectives from academic, government, and private sectors in the US, Europe, Asia and Africa. A listing of contributors and collaborators can be found at www.graceprinciples.org. More information is available in the American Journal of Managed Care 2010; 16(6): 467-471 (Dreyer NA, Schneeweiss S, McNeil B et al.) The methods and results for the validation have been submitted for publication.

To join the GRACE Initiative or for more information, please contact them at
coordinator@graceprinciples.org
Nancy A. Dreyer
Leader, GRACE Initiative

Data

D1. Were treatment and/or important details of treatment exposure adequately recorded for the study purpose in the data source(s)?
Note: not all details of treatment are required for all research questions.





Yes: reasonably necessary information to determine treatment or intervention was adequately recorded for study purposes (e.g., for drugs, sufficient detail on dose, days supplied, route or other data important. For vaccines, consider the importance of batch, dose, route and site of administration, etc. For devices, consider type of device, placement, surgical procedure used, serial number, etc.).

No: data source clearly deficient or not enough information in article.


D2. Were the primary outcomes adequately recorded for the study purpose (e.g., available in sufficient detail through data source(s))?



Yes: information to ascertain outcomes were adequately recorded in the data source (e.g., if clinical outcomes were ascertained using ICD-9 diagnosis code(s) in an administrative database, the level of sensitivity and specificity captured by the code(s) was sufficient for assessing the outcome of interest.)

No: data source clearly deficient (e.g., the code(s) captured a range of conditions that was too broad or narrow, and supplementary information such as that from medical charts was not available), or not enough information in article


D3. Was the primary clinical outcome(s) measured objectively rather than subject to clinical judgment (e.g., opinion about whether the patient’s condition has improved)?



Yes: clinical outcomes were measured objectively (e.g., hospitalization, mortality) N/A--primary outcome not clinical (e.g., PROs)

No
:
e.g., clinical opinion about whether patient’s condition improved, or not enough information in article


D4.  Were primary outcomes validated, adjudicated, or otherwise known to be valid in a similar population?



Yes:  outcomes were validated, adjudicated, or based on medical chart abstractions with clear definitions, e.g. a validated instrument was used to assess patient-reported outcomes (e.g., SF-12 Health Survey); a clinical diagnosis via ICD-9 code was used, with formal medical record adjudication by committee to confirm diagnosis or other procedures to achieve reasonable sensitivity and specificity; billing data were used to assess health resource utilization, etc.

No
: No, or not enough information in article


D5.  Was the primary outcome(s) measured or identified in an equivalent manner between the treatment/ intervention group and the comparison group(s)?



Yes.

No
  or not enough information in article


D6. Were important covariates that may be known confounders or effect modifiers available and recorded? Important covariates depend on the treatment and/or outcome of interest, (e.g., body mass index should be available and recorded for studies of diabetes; race should be available and recorded for studies of hypertension and glaucoma).




Yes. most if not all important known confounders and effect modifiers available and recorded, e.g., measures of medication dose and duration.

No
  at least one important known confounder or effect modifier not available and recorded (as noted by authors or as determined by user’s clinical knowledge), or not enough information in article




Methods

M1. Was the study (or analysis) population restricted to new initiators of treatment or those starting a new course of treatment? Efforts to include only new initiators may include restricting the cohort to those who had a washout period (specified period of medication nonuse) prior to the beginning of study follow-up. 




Yes
. Only new initiators of the treatment of interest were included in the cohort, or for surgical procedures and devices, only patients who never had the treatment before the start of study follow-up were included.

No
or not enough information in article



M2.  If one or more comparison groups were used, were they concurrent comparators? If not, did the authors justify the use of historical comparisons group(s)?




Yes
. Yes--data were collected during the same time period as the treatment group (“concurrent”) or historical comparators were used with reasonable justification, e.g., when it is impossible for researchers to identify current users of older treatments or when a concurrent comparison group is not valid--(i.e., uptake of new product is so rapid that concurrent comparators differ greatly on factors related to the outcome)

No
. No--historical comparators used without being scientifically justifiable, or not enough information in article


M3.  Were important covariates, confounding and effect modifying variables taken into account in the design and/or analysis? Appropriate methods to take these variables into account may include: restriction, stratification, interaction terms, multivariate analysis, propensity score matching, instrumental variables or other approaches.




Yes
Yes--most if not all important covariates that would be likely to change the effect estimate substantially were accounted for, e.g., measures of medication dose and duration.

No
. No--some important covariates were available for analysis but not analyzed appropriately, or at least one important covariate was not measured, or not enough information in article



M4.   Is the classification of exposed and unexposed person-time free of “immortal time bias”? Immortal time in epidemiology refers to a period of cohort follow-up time during which death (or an outcome that determines end of follow-up) cannot occur.



Yes
.

No
.
  No, or not enough information in article



M5.  Were any meaningful analyses conducted to test key assumptions on which primary results are based? E.g., were some analyses reported to evaluate the potential for a biased assessment of exposure or outcome, such as analyses where the impact of varying exposure and/or outcome definitions was tested to examine the impact on results.




Yes (1)
. Yes--and primary results did not substantially change 
Yes (2) Yes--and primary results changed substantially
No.
None reported, or not enough information in article







 
 



 


References top of page



Dreyer NA, Bryant A, Velentgas P. The GRACE checklist: a validated assessment tool for high quality observational studies of comparative effectiveness. J Managed Care & Specialty Pharmacy 2016; 22(10):1107-13.


Grace Principles Validated Checklist