Original Research

Stakeholders’ feedback on the proposed recommendations for updating the patented medicine prices review board (pmprb) budget impact analysis guidelines

Naghmeh Foroutan PhD^1,2, Jean-Eric Tarride PhD^1,2,3, Feng Xie PhD^1,3,4, Bismah Jameel BS¹, Fergal Mills⁵, and Mitchell Levine MD, MSc^1,2,3

¹Department of Health Research Methods, Evidence and Impact (HEI), McMaster University, Hamilton, ON, Canada;

²Programs for Assessment of Technology in Health (PATH), The Research Institute of St. Joe’s, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada;

³Centre for Health Economics and Policy Analysis (CHEPA), McMaster University, Hamilton, ON, Canada;

⁴Program for Health Economics and Outcome Measures (PHENOM), Hamilton, ON, Canada;

⁵Innomar Consulting, Oakville, ON, Canada

ABSTRACT

Introduction: The present study aimed to obtain Canadian stakeholders’ feedback on a list of proposed recommendations for updating the Patented Medicine Prices Review Board (PMPRB)’s 2007 budget impact analysis (BIA) guidelines.

Methods: A mixed-methods study was designed to obtain feedback from two stakeholder perspectives—(public and private) payers and manufacturers—on the proposed BIA recommendations. We obtained policymakers’ opinion through one-on-one interviews and collected feedback from manufacturers and their consultants using a survey. The interview guide and the survey were developed based on the list of recommendations related to BIA key elements, which were either not discussed or addressed differently in the PMPRB 2007 BIA guidelines. The list was derived from 16 Canadian or other national and transnational BIA guidelines. A thematic analysis was applied for analysis of the qualitative (interview) data.

Results: Thirty-five policymakers and manufacturers participated in the study. Stakeholders supported the inclusion of 56% of the proposed recommendations into the guidelines pertaining to the use of expert opinions, data extrapolated from the payers’ database, scenario analysis, and dynamic population. Inclusion of indirect costs, and cost transfers from other jurisdictions, were not approved. There was no consensus regarding the inclusion of patients’ adherence/compliance and cost offsets.

Conclusions: The present study has provided sufficient insights to enable the creation of a penultimate version for updating the PMPRB BIA guidelines. This penultimate version will be subject to a broader consultation among stakeholders prior to a final revision and approval. Further Canadian stakeholder feedback is required for reaching consensus on inconclusive recommendations.

Keywords: stakeholder analysis; public drug plans; private payers; pharmaceutical industry; PMPRB budget impact analysis guidelines

INTRODUCTION

In Canada and other developed countries, spending on pharmaceuticals is expected to increase significantly with the population aging and the introduction of highly specialized, expensive medicines.¹ Health technology assessment and budget impact analysis (BIA) are important components to determine whether a drug will be approved by the drug benefit programs. In Canada, the fourth version of the guidelines for the economic evaluation (EE) of healthcare technologies was updated in 2017, and manufacturers are required to comply with these guidelines when submitting cost-effectiveness analyses to support the public or private reimbursements of their products. Similarly, manufacturers have to provide a 3-year BIA to reimbursement authorities. The first and the only Canadian BIA guidelines were published in 2007 by the Patented Medicine Prices Review Board (PMPRB).² These BIA guidelines provide recommendations on BIA analytical model structure, data input, and sources and reporting format, and have been approved and adopted by most of the Canadian provincial drug plans.³

Since 2007, many BIA guidelines have been published or updated in several jurisdictions, but contrary to the Canadian guidelines for EE, which have been updated thrice to incorporate new developments in the conduct of EEs,⁴ the Canadian BIA guidelines have never been updated. To this end, we conducted a systematic review of BIAs^5–11 to identify recommendations from the literature that were either not included or discussed differently in the 2007 PMPRB BIA guidelines. In this paper, we present the results of a qualitative and quantitative (mixed) analysis of Canadian stakeholders’ views and feedback on the proposals for updating the Canadian PMPRB BIA guidelines.^11,12

METHODS

Study Design

This study was premised upon the central research question, “What is the Canadian stakeholders’ perception about the practicality and relevance of the newly proposed BIA recommendations for updating the Canadian BIA guidelines?” This mixed-methods study obtained research ethics approval from Hamilton Integrated Research Ethics Board (project number: 2923). The study includes both qualitative and quantitative data collection.

For the qualitative analyses, a semi-structured interview guide was developed collaboratively by the research team to ensure that the questions would address the research objectives of the study.^13,14 The interview guide was developed based on the discordance that was observed between the PMPRB 2007 and the other BIA guidelines that were reviewed^5,7,9 (Table 1). These include, for example, proposing “dynamic” (vs. closed) population,¹ including catch-up effect in the case of chronic conditions, scenario analysis for managing uncertainty, in the PMPRB 2007 BIA guidelines. Based on this information, we developed a semi-structured interview guide, which included 10 interview questions around major themes (i.e., BIA usage in drug reimbursement decisions and price adjustments in Canada, BIA usage in disinvestment decisions, and linking of incremental cost-effectiveness ratio [ICER] with BIA [affordability] and BIA key elements). Appendix Table 1 presents the semi-structured questionnaire. In addition, it was agreed that as interviews progressed, interview probes and follow-up questions could be amended given the completion and content of previous interviews, and in order to elicit the most comprehensive information possible from the research participants. The interview guide also included 14 closed questions (Appendix Table 2) for which a Likert-type ordinal scale was used to rate the responses, ranging from 1 to 5 (1 = “strongly disagree,” 3 = “neither agree nor disagree,” 5 = “strongly agree”).

All interviews were conducted by the lead author (NF), who had extensive knowledge of the BIA context. The methodological principles of interpretive description were applied to sampling, data collection, and analysis procedures for the interviews.¹⁵

Interviews were mostly conducted in a single meeting using the Google Hangout application (n = 6) except for three participants in whose case a telephonic interview was conducted. The 30-minute (average) interviews were recorded, and they were completed between March 2018 and September 2018.

In parallel with the interviews, an online written survey consisting of 30 questions was developed using SurveyMonkey (Appendix 2). We converted each BIA recommendation that is not considered or discussed differently in the Canadian BIA guidelines (Table 1) to a question in order to get participants’ opinion to assess whether they agreed, disagreed, or neither to include a recommendation in the Canadian BIA guidelines. It was mandatory to answer all the questions, and there was no option to move backward through the survey. The survey was open from May 10, 2018 to December 31, 2018.

Participants

Candidates for the interviews were purposively selected from public drug plans, the Federal Ministry of Health (Health Canada), PMPRB, CADTH² (CDR³ and pCODR⁴), pCPA⁵, NIHBP⁶, and private payers. For each interview, a maximum of three email invitations were sent out to the representatives of these key stakeholder organizations inviting them to participate in the study. The initial email included brief information about the project, the identity of the interviewer (NF), the purpose of the study, the number of questions, the expected interview duration, and that the study was voluntary. Recipients were also told that the interview would take place through an online meeting at a time convenient to them. It was stated that their agreement to be interviewed was inferred as their consent to participate in the study. A second reminder was sent 2 weeks after the initial request, followed by a third reminder, which was sent a month later. Those who did not respond to the third reminder were contacted by phone.

For the online survey, we sought participation from pharmaceutical manufacturers and industry reimbursement consultants in order to obtain a pharmaceutical industry perspective. Recruitment was conducted with an invitation and a URL link that was sent by email to these stakeholders. Similar to the interview, a second reminder was sent 2 weeks after the initial request, followed by a third reminder, which was sent a month later.

With consent obtained prior to the interviews, all interviews were audiotaped and stored in an MP3 format. Two authors (NF and BJ) transcribed the data. The interviews, transcription, and analyses were conducted concurrently, allowing an opportunity for new themes to emerge across participants and for further exploration of these themes throughout the remaining interviews.¹⁵ A deductive content analysis that is based on previous knowledge and framework was deemed appropriate for this study.¹⁶

Given that the present study was one of four studies for the author’s (NF) doctoral dissertation, NF independently coded all of the transcripts. Specifically, NF completed iterative readings of each transcript. This allowed her to gather, label, and compare keywords from the text that captured key thoughts and concepts described by the participants, referred to as “codes.” The author generated overarching themes among the codes through a process of identifying patterns of coding within and across participants. She generated an initial codebook with definitions of each code and the linking themes. The online survey data were analyzed using the “results analysis” feature of the SurveyMonkey application.

RESULTS

We conducted nine interviews with policymakers, including public and private payers (62% response rate), and collected 27 online surveys from reimbursement experts in the pharmaceutical industry (51% response rate). The audio data for one participant was lost due to an unexpected error in the recording process (although interview notes were still available). Thus, transcripts from eight interviews entered the thematic analysis. The results are reported below in three sections: (1) feedback from policymakers (BIA reviewers), (2) feedback from manufacturers or their consultants (BIA producers), and (3) comparative analysis between policymakers and manufacturers/consultants.

Feedback from Policymakers

Through a thematic content analysis, we identified the following major themes in the interview results: (1) BIA usage in drug reimbursement decisions and price adjustments in Canada, (2) BIA usage in disinvestment decisions, (3) linking ICER and BIA (affordability), and (4) BIA key elements (e.g., time horizon) including additional recommendations for improving the guidelines.

BIA Usage in Drug Reimbursement Decisions and Price Adjustments in Canada

Most interviewees believed that BIA could be useful in drug reimbursement decisions and price adjustments. This was captured by the following comments: “I would say they are very useful” or “it is the crucial part of assessing the affordability to pay for new technology, …, very important, the most important [part is] when it comes to the reimbursement because it shows the … how the new technology may impact the budget…” The remaining interviewees did not have a strong idea about it due to the fact that they were representing organizations that do not actually review BIAs or do not usually use BIA for price adjustments (e.g., private payers). In the private drug benefit programs, BIA could be helpful in setting insurance premiums; however, according to an interviewee, premiums tend to be set about 12–18 months in advance to the arrival of a BIA, so they usually don’t have BIAs in time for setting the premiums. “There [are] sort of two parts … the traditional BIA that we get as part of a submission [which] tends to be a little more directional in nature. There’s starting to be a little more attention paid to sort of creating what I call miniature BIAs on pipeline drugs to try and get a better feel for what those BIAs are [going to] be with closer about a 2- or 3 year time horizon, based on what’s in the pipeline and what the major drugs are looking like, in terms of what their indications may end up being” according to a private payer representative.

BIA Usage in Disinvestment Decisions

Most participants believed that, at least theoretically, BIA could be helpful in disinvestment decisions (delisting drugs); however, in practice, there are many other factors that would need to be taken into account in delisting a drug (e.g., clinical efficacy and safety, cost-effectiveness, ethical considerations, patient access), which makes it a rare occurrence in the formulary management process. “The way we use it right now is when we look at the BIAs we just generally look at it when we do listing decisions for our formularies at the very front end, when we decide whether or not we list that drug, so we don’t ever look at them later to make disinvestment decisions pretty much, but I assume they [BIA] could come in handy” was mentioned by one of the provincial drug plan’s representative. According to the latter interviewee, BIA could be useful in disinvestment decisions in the circumstances that they do consider disinvestment such as when a new more cost-effective drug becomes available and is associated with a considerable cost saving.

Linking Incremental Cost-Effectiveness Ratio and Budget Impact

In a constrained budget, the higher the budget impact of a new drug, the lower the ICER threshold for appraising the drug for reimbursement.¹⁷ We asked the Canadian stakeholders for their opinion about the use of BIA (affordability) in price adjustments through defining the ICER thresholds. Some found the idea of linking ICER and BIA helpful in the sense that they are complementary to each other, and having both pieces together could provide policymakers with a better understanding of value for money and affordability in assessing a new medication for reimbursement and price adjustments. A few stakeholders preferred keeping them separate (as they are), for example, “I think keeping it [BIA] separately would make more sense … so it gets down to what the purpose is, and that is to determine the BIA for the drug plan.”

BIA Key Elements and Additional Comments for Improving the PMPRB BIA Guidelines

With regard to the perspective of adopting in a BIA, in addition to the public or private payer perspective, some interviewees believed that asking manufacturers to include the patients’ perspective as a complementary component to the BIA base-case analysis would not be practical or feasible. The main concern was that in some jurisdictions, “there are so many different scenarios/plans that they could have for co-payments and co-insurance, which makes it really hard for the manufacturers to capture any of that in their submission, or in their BIA.”

Participants were asked for their opinion about the advantages and limitations of the current 3-year time horizon in the Canadian BIA guidelines. Some participants believed that a longer time horizon (≥3 years) could be more helpful in Canada (e.g., 5 years). The advantages and disadvantages of a 3-year or a longer time horizon are summarized in Table 2. Increasing the uncertainty, especially for market size estimation and lack of real-world information, was the main concern of most participants on using a longer time horizon.

Cost analysis is directly related to the perspective of the adopted budget holder. If the new drug represents one of a class of drugs, the least cost alternative (LCA) within the class as defined by the drug plan could be used to set the price of the new comparator. Among public plans, “Alberta and British Columbia [use LCA]… Ontario doesn’t” and “it varies by the provinces based on what they deem to be interchangeable.” Private payers have programmed their system “to recognize what the lowest cost is, and [the] price for that [which] generally is the generic [version of the drug], but sometimes it might not be.”

From the policy-making standpoint, there may be an opportunity cost associated with introducing new technology, as it may use additional resources that must be taken from the existing services.¹⁸ Having this definition in mind, participants believed that manufacturers could not estimate the opportunity cost, “obviously in terms of specific decisions that provinces make, it’s literally impossible for the drug company to know what we would do with the money instead of funding this so… that’s not really something they can answer.”

Modeling may be needed to calculate the budget impact for bringing together the best available data from different sources.^1,3,18–23 If there is an EE in advance to the BIA, assumptions should be consistent with EE.^{3,18–20,22,23} All participants believed that modeling could be helpful as long as it is as simple as possible. Usage of complicated models (e.g., Markov) is not required. One of the participants mentioned that, “certainly we try to model out the condition or population as much as we can, to get to a level that gives us confidence while providing say enough certainty, so that’s an extra level of complexity that usually is not needed.” One participant brought up the discussion related to BIA models in the United States in comparison with the Canadian approach. “I think in the US there’s less of a reliance on cost-effectiveness analysis (CEA). So there’s a lot of emphasis on BIA, and so they might not be getting some of this additional information regarding some of the complexity of the condition with BIA alone. So I guess it really depends on the purpose of the BIA and from what perspective it is being conducted. … if you already have a CEA that accounts for some of this already, and the BIA is completely from a pharmacy perspective or a drug plan perspective, then maybe there’s not the same level of need to have components of the CEA [as] part of the BIA. But … if you don’t have a CEA or the perspective of the BIA needs to be broader, [then it is another story]… it really depends on the question that the BIA is intended for.” From a private payer’s perspective, sensitivity analysis is a better way to take into account the complexity of the disease (e.g., acute plus chronic conditions). In general, participants believed that the extent of required modeling complexity depends on the disease condition and payers’ perspective (e.g., in one province, they are sometimes interested in a more dynamic model if the treatment reduces the disease mortality, disease rate of complications, and when it changes the duration of treatment). Moreover, the idea of reflecting both chronic and acute conditions is more about using incidence and prevalence-based approaches in BIA. A “re-assessment approach” was also recommended by a payer to determine the long-term consequences of medications in chronic conditions.

All but one participant had read the 2007 PMPRB BIA guidelines (that individual was familiar with the US guidelines for BIA), and all believed that there is a need to update the PMPRB BIA guidelines mainly because they are considered out of date. All participants provided comments for improving the new version of the PMPRB BIA guidelines, which are summarized in Appendix Table 3.

Based on the results of the survey component of the interview guide (Appendix Table 4), a majority of interviewees believed that (1) treatment switches and (2) changes in the rate of mortality and disease progression are important to be captured over the time horizon. Most of them welcomed the idea of providing a list of acceptable databases as reliable references for input data in BIA calculations in the updated version of the Canadian BIA guidelines. Most of them also agreed with a cap or threshold for the budget impact of new drugs to signal the need for negotiation with manufacturers for lowering the price. They thought it is important to build in a reassessment process of BIAs in a future real-world, post-market environment. The opinions regarding the inclusion of patient adherence and compliance in the target population assessment, including the cost of adverse events, clinical outcomes, and disease complications in the BIA cost analysis, were inconclusive.

Feedback from Manufacturers or Their Consultants

The results from the online written survey of 27 participants with an industry perspective are summarized in Appendix Table 5. On the major issue of time horizon, 20 agreed with the 3-year time horizon in BIAs as reasonable in Canada. Only a quarter of respondents (n = 7/27) believed that there should be a change in the time horizon in the new PMPRB BIA guidelines. However, there was the sense that flexibility (variability) depending on, for example, the disease area and patent duration should be considered and justified after 3 years in different cases.

With respect to the target population issue, 11 of the 27 (41%) survey participants agreed on conducting subpopulation analysis in addition to an aggregated analysis for the whole population. Thirteen participants believed that the target population should be dynamic (open) in BIAs, meaning that patients could be added to or removed from the analysis based on whether they meet the inclusion criteria or not over time. Including off-label indications in the target population assessment was not supported by the majority of participants (74%; n = 20/27 disagree).

Pertaining to the costs, either of the comparators or those included in the base-case, using the LCA in the cost analysis, was acknowledged as appropriate by the majority of the survey participants. Approximately 50% of survey participants disagreed with the inclusion of either indirect or non-healthcare-related costs (e.g., training or introduction cost, transportation, productivity, and caregiver-related costs) or the taxes, for example, harmonized Sales Tax (HST) in BIA. Including the cost transfer from other jurisdictions (where there is a lack of real-world data for the proposed medicine) was not acceptable to 70% of the survey participants. A majority (62%; n = 16/27) of the participants supported including the total and incremental impact on the budget (cost analysis for all new and currently covered indications) in the BIA, but opined that the effects of inflation and discounting should not be included in the BIA.

Most participants disagreed with using probabilistic sensitivity analysis (PSA) in BIA. Scenario or deterministic sensitivity analyses were highly recommended (96 and 85%, respectively). There was support for describing the direction and magnitude of the impact of uncertainty on the overall estimates, but risk-sharing agreements and longer introduction phase were not favored for decreasing model and data uncertainty. The majority of respondents felt that data from manufacturers, clinical data from other jurisdictions, expert opinions, and extrapolating data from similar (or proxy) drug experiences on the payers’ database could all be considered reliable sources of data in BIA.

Over 80% of the respondents supported reporting the gross and the net impact on the budget based upon the anticipated sales of the drug of interest for each of the first 3 years after the coverage is granted. There was less support for a schematic representation of the uncertainty analysis (e.g., Tornado diagram). Two-thirds felt that aggregated and disaggregated budget impact results should be reported for each year of the time horizon.

The support was 52% for both cost outcomes being presented separately for different payers and for cost outcomes being presented in monetary units. There was no consensus on whether some cost outcomes should be in natural units (e.g., number of unpaid working days) or not.

Comparative Analysis of Stakeholders’ Feedback (Policymaker vs. Pharmaceutical Manufacture Perspectives)

We performed a comparative analysis between the two groups of stakeholders for seven survey questions. Table 3 summarizes the questions that are common in the data obtained from both groups. Figure 1 illustrates all survey results, including the comparative results, between the two groups.

FIG 1. Summary of written survey results of all participated stakeholders (n = 36).
Note: Gold refers to “policymakers” and “blue bars” represent manufacturers/consultants.

Both groups did not support the inclusion of staff training or introduction costs, non-healthcare-related costs or taxes in a BIA. Both groups believed that expert opinions and data extrapolation from similar drug in the payers’ database are reliable sources of data to be used in BIAs where there is a lack of real-world data for the proposed drug. While 62% of industry participants supported that reporting both the total and incremental impact on the budget for a new medication is important in BIA, the results from policymakers were indecisive (38% neither agreed nor disagreed).

DISCUSSION

In the present study, the Canadian stakeholders’ feedback on the BIA recommendations, obtained through qualitative and quantitative methods, provides additional insight to help define BIA guidelines from a Canadian perspective. This information may also be of value for updating or creating BIA guidelines worldwide. The present study arose from the results of the literature reviews describing Canadian, international (e.g., France, Australia, Belgium, Ireland, Brazil, and the United Kingdom) and transnational (e.g., ISPOR) BIA guidelines, and was designed to capture feedback and expert inputs from the stakeholders (policymakers and manufacturers) in the field of pharmaceutical pricing and reimbursement in Canada.^5,24 The authors identified discrepancies between the PMPRB 2007 BIA guidelines and the more recently published or updated BIA guidelines either in Canada or outside. This generated a list of BIA recommendations, which were not included or discussed differently in the PMPRB 2007 guidelines. While the PMPRB BIA guidelines are clearly intended to serve the interests of the people who will be using them, that is, the policy makers, we also obtained inputs from the stakeholders who are tasked with creating the BIA documents for submission to the policymakers. On a positive note, for the items where we were able to obtain common data, there was a general agreement between these two groups. Nevertheless, had there been disagreement, the BIA guidelines would have to reflect the needs of the policymakers or the BIA submissions; otherwise the guidelines would be unhelpful in the drug reimbursement regime. While there was consensus on many recommendations, some recommendations will need further inputs to determine whether they should be included in an updated version of the PMPRB BIA guidelines, especially involving policymakers from both public and private perspectives (e.g., 3-year or longer time horizon, patient adherence and compliance, cost offsets, reporting total and incremental impact on the budget, and providing a list of reliable databases as data sources).

Obtaining stakeholders’ feedback was part of the process to create many of the previously published national BIA guidelines.^4,5,9,11 The PMPRB 2007 BIA guidelines were initially developed based on a needs assessment and a literature review and then improved upon with inputs from the NPDUIS Advisory Committee, including drug plan managers from multiple provinces in Canada and a representative from the CADTH.⁴ In Poland, the BIA guidelines were initially conducted internally within the Agency for Health Technology Assessment and Tariff System, and then within the Guidelines Update Team. The Guidelines were submitted for public comment and for review by the Minister of Health.¹¹ In Belgium, the preliminary BIA guidelines were developed based on a literature review and then stakeholders’ feedback was obtained involving the Belgian Health Care Knowledge Centre (KCE) and different Belgian stakeholders from both government and industry.⁹ In France, as a part of the French BIA guidelines development, a public consultation process was conducted including international expert reviews and approval from the HAS Board and the Economic and Public Health Evaluation Committee of HAS.⁵ Unfortunately the stakeholder analyses that were conducted as part of the above-mentioned guidelines were not published with any methodological detail. Our study is unique in terms of: (1) the rigorous study design (mix methods), (2) the scope (including policymakers from both public and private drug plans), (3) inclusion criteria (clear definitions for selecting stakeholders), (4) the one-on-one semi-structured interviews providing a rich description of the stakeholders’ opinion on improving the PMPRB BIA guidelines, and (5) publishing the stakeholder analysis in the public domain.

There are a number of conclusions that arise from the results of the feedback. Similar to Pearson and Ghabri et al., we found that using the BIA as an affordability factor for ICER⁷threshold adjustments and price cap estimation, especially in chronic conditions such as hepatitis C virus drugs, is a practical benefit of using BIA in a real-world scenario.^1,25,26 Moreover, considering a BIA cap or threshold received positive feedback in our study. There are some examples for using BIA threshold internationally such as a budget impact test of £20million for NHS England since 2017²⁷ and a budget impact threshold linked to the growth in the national economy GDP⁸ in the United States.²⁵ Similarly, a real-world reassessment process for BIA results (in the post-market surveillance phase) would be recommended, as already used in the United Kingdom.

One should note that in the Irish¹⁸ and ISPOR²³ BIA guidelines, opportunity costs are defined as the costs that arise when implementing the technology or clinical guidelines that might not be reflected in the “actual costs” at the time of doing a BIA, and that they are different from the opportunity cost definition in policymaking of “whether the improvement in health outcomes that the proposed new drugs offer exceeds the improvement in health that would have been possible if the resources had, instead, been made available for other health care activities.”²⁸ Based on the results, we concluded that, it is not feasible to ask manufacturers to calculate in BIA the opportunity cost of investing in a new drug. Therefore, there is a need for a method to calculate the opportunity cost without relying on the BIA, and an alternative method has been proposed by Ochalek et al. where the opportunity costs in the healthcare expenditures are represented by the threshold value for the CEA.²⁸

Regarding the time horizon, a minimum of 3 years is favorable in Canada and a time horizon beyond 3 years should be justified by the manufacturer (e.g., for a specific drug, a disease area, or patent duration). In some BIA guidelines, such as in France,¹ ISPOR,²³ and Brazil,²⁹ there is a range for time horizon, for example, 3–5 years, whereas in the British and Irish guidelines, they introduce a punctual time horizon of 5 years. Further stakeholder feedback is required to reach a consensus on the most favored approach for a BIA time horizon in Canada.

The results show that usage of complicated modeling techniques (e.g., Markov models) is not recommended in BIA, but it is advised that the disease condition should be modeled as much as possible to capture the long-term consequences (at least within the adopted time horizon) associated with using the proposed medication in a chronic condition. Providing either incidence- or prevalence-based (or both) models would help to better understand drug costs related to the acute and chronic conditions. This is especially important for responding to a methodological gap, which was highlighted by Mauskopf et al.³⁰

Different terminologies are used in different guidelines/countries to define comparators. In the Canadian context, the multidrug treatment strategy for defining comparators is called “strategy-based treatment,” which is different from France¹ (treatment set), ISPOR²³ (treatment mix or set), and Australia¹⁹ (treatment mix). In the new version of the PMPRB BIA guidelines, usage of a “treatment mix” is suggested to be the most consistent with international terminology.

One should note that comparator mix does not necessarily always match the comparator mix in the utilization (real world); for example, the treatment mix that private payers and public plans see in their database maybe different because of inherent differences in their formularies. This type of difference between public and private payers should be addressed in BIA. It is also recommended that the choice of comparators in BIA should be consistent with the health EE (e.g., cost effectiveness study) unless there are clear justifications for not taking the same treatment strategy as the health technology assessment (e.g., in the case of non-drug/surgical alternatives). Similar to the PMPRB 2007 BIA guidelines, our study participants confirmed that off-label indications should be considered only in sensitivity analysis (not in base-case analysis). As it is recommended in the Irish¹⁸ and ISPOR²³ BIA guidelines, the catch-up effect (treatment switch) is also recommended in BIA in Canada. Mauskopf et al.³⁰ raised a methodological gap relating to the “treatment switch” or “drug discontinuation,” in the chronic conditions, which might not be appropriately addressed in many published BIAs in the United States.

Similar to the PMPRB 2007 BIA guidelines, cost offsets, that is, costs associated with changes in clinical outcomes, costs associated with clinical consequences/complications (e.g., adverse drug reactions), and resource utilization (e.g., hospitalization, emergency room admission) are still excluded from the analysis. The impacts on indirect costs (e.g., productivity, transport, capacity, and workforce) are not included in a BIA base-case analysis, and cost data from other jurisdictions are not acceptable.

Uncertainty in input data is a general concern in BIA. PSA is used in the Irish and Belgian guidelines,^9,10 whereas it is not recommended in Canada. In contrast, scenario (for structural uncertainty) or deterministic (for input data uncertainty) sensitivity analyses are highly recommended in Canada. A methodological review of US budget impact models³⁰ showed that sensitivity and scenario analyses presented in published BIAs are “typically too limited to allow a budget holder to assess the likely budget impact for their health plan.” We would expect that the gap would be covered by recommending scenario and sensitivity analyses being included in the PMPRB BIA guidelines for dealing with uncertainty. Risk-sharing agreements and longer introduction phase for decreasing the uncertainty in new drug submissions, as they are recommended in Australia¹⁹ and the United Kingdom,²⁰ are not favored in Canada.

There are a number of limitations to the present study. There was a limited sample size for interviews providing the qualitative data. Nine policymakers agreed to participate in an interview, and the audio data for one participant was lost. Also, two participants from the same province were interviewed at the same time. While the sample size was limited, we were able to obtain data from representatives of different jurisdictions across Canada, and a similarity in their responses was noted suggesting that new themes or ideas might not have been forthcoming even if the sample size was larger. Nevertheless, in order to make any meaningful comparison between the opinions of public and private payers’ (subgroup analysis), we would have required substantially larger numbers.

CONCLUSION

We obtained Canadian stakeholders’ opinion on a list of recommendations prepared based on a comparative literature review of national and ISPOR BIA guidelines using a mixed-methods approach. A mandate for submitting a companion CEA/CUA along with a BIA in a new drug application in Canada could be a reason for some observed differences between the Canadian stakeholders’ perspective and recommendations from other jurisdictions (e.g., ISPOR), especially with respect to the inclusion of cost offsets (e.g., clinical outcomes) and complicated modelling techniques in BIA.

The present study is an integral step towards creating a proposal for updating the PMPRB BIA guidelines. The present study aimed to gain initial Canadian stakeholders’ feedback and opinion on potentially new recommendations. A penultimate revised PMPRB BIA guidelines will be developed based on the results of the present study. In Canada, when guidelines are proposed by a government agency or board, there is a mandatory comprehensive consultation process with stakeholders. The results generated by the current study will form the template for the new draft guidelines document that the PMPRB will produce. PMPRB will then conduct a broader consultation with stakeholders than could be achieved in this study. After that step, there will be a final revision and subsequent adoption of updated BIA guidelines.

COMPLIANCE WITH ETHICAL STANDARDS

FUNDING

This work was supported by the Mitacs Accelerate research award (IT09958).

CONFLICTS OF INTEREST

Naghmeh Foroutan, Jean-Eric Tarride, Feng Xie, and Bismah Jameel have no conflicts of interest that are directly relevant to the contents of this article. Mitchell Levine is the chair of the PMPRB, and Fergal Mills is a director at Innomar Consulting.

DATA AVAILABILITY STATEMENT

We have provided the required data as much as possible (in a summarized format), and there are no additional data to be shared.

REFERENCES

1. Ghabri S, Autin E, Poullie AI, et al. The French National Authority for Health (HAS) Guidelines for Conducting Budget Impact Analyses (BIA). PharmacoEconomics 2018;36(4):407–417. https://doi.org/10.1007/s40273-017-0602-5
2. Guidelines for conducting pharmaceutical budget impact analyses for submission to public drug plans in Canada. Patented Medicine Prices Review Board; 2007. Available at: http://www.pmprb-cepmb.gc.ca/cmfiles/bia-may0738lvv-5282007-5906.pdf.
3. Marshall DA, Douglas PR, Drummond MF, et al. Guidelines for conducting pharmaceutical budget impact analyses for submission to public drug plans in Canada. PharmacoEconomics 2008;26(6):477–95. https://doi.org/10.2165/00019053-200826060-00003
4. Guidelines for the economic evaluation of health technologies: Canada. Canadian Agency for Drug and Technology in Health (CADTH); 2017. Available at: https://www.cadth.ca/sites/default/files/pdf/guidelines_for_the_economic_evaluation_of_health_technologies_canada_4th_ed.pdf.
5. Foroutan N, Tarride J-E, Xie F, et al. A methodological review of national and transnational pharmaceutical budget impact analysis guidelines for new drug submissions. ClinicoEcon Outcomes Res 2018;10:821–54. https://doi.org/10.2147/CEOR.S178825
6. Foroutan N. Proposals for updating the Canadian Patented Medicine Prices Review Board pharmaceutical budget impact analysis guidelines for new drug submissions to public and private payers. McMaster University, Hamilton, ON, Canada; 2019.
7. Foroutan N, Tarride J-E, Xie F, et al. A comparison of Pharmaceutical Budget Impact Analysis (BIA) recommendations amongst the Canadian Patented Medicine Prices Review Board (PMPRB), public and private payers. PharmacoEconomics 2019;3:437–451. https://doi.org/10.1007/s41669-019-0139-y
8. Foroutan N, Levine M, Tarride J-E, et al. The methodological review of Budget Impact Analysis (BIA) models to update Canada’s pharmaceutical BIAs guidelines for new drug submissions to public and private payers. J Popul Ther Clin Pharmacol 2017;24(3):e51–78.
9. Foroutan N, Levine M, Tarride J-E, et al. Updating Canadian pharmaceutical budget impact analysis guidelines. Health Technology Assessment International (HTAi) annual meeting 2018; Vancouver International Journal of Technology Assessment in Health Care 2018. Available at: https://htai.org/wp-content/uploads/2018/08/AM18_Abstract-Book.pdf
10. Foroutan N, Mills F, editors. Proposed changes to Canada’s PMPRB budget impact analysis guidelines for new drug submissions to public and private payers. Canadian Agency for Drug and Technologies in Health symposium 2018; Ottawa. Available at: https://www.cadth.ca/symposium2018/proposed-changes-canadas-pmprb-budget-impact-analysis-guidelines-new-drug-submissions
11. Foroutan N, Tarride J-E, Xie F, et al. Penultimate revised version of the Patented Medicine Prices Review Board (PMPRB) pharmaceutical budget impact analysis (BIA) guidelines. The Canadian Association for Population Therapeutics / Association Canadienne pour la Thérapeutique des Populations presents; Toronto: The Journal of Population Therapeutics and Clinical Pharmacology; 2019. p. e23. Available at: https://jptcp.com/index.php/jptcp/article/view/654/568
12. Foroutan N, Levine M, Tarride J-E, et al. Comparative analysis of stakeholder views on proposals for updating the Patented Medicine Prices Review Board (PMPRB) Budget Impact Analysis guidelines. The Canadian Association for Population Therapeutics/Association Canadienne pour la Thérapeutique des Populations presents; Toronto: The Journal of Population Therapeutics and Clinical Pharmacology; 2019. p. e14–34.
13. Rubin HJ, Rubin IS. Qualitative interviewing: the art of hearing data. Thousand Oaks, CA, USA: Sage; 2011.
14. Creswell JW. Qualitative inquiry and research design: choosing among five approaches (ed.). Thousand Oaks, CA, USA: Sage; 2007.
15. Thorne S. Interpretive description. Abingdon-on-Thames, UK:Routledge; 2016.
16. Elo S, Kyngas H. The qualitative content analysis process. J Adv Nurs 2008;62(1):107–15. https://doi.org/10.1111/j.1365-2648.2007.04569.x
17. Paulden M, O’Mahony J, McCabe C. Determinants of change in the cost-effectiveness threshold. Med Decis Making 2017;37(2):264–76. https://doi.org/10.1177/0272989X16662242
18. Guidance on budget impact analysis of health technologies in Ireland. Dublin, Ireland; 2015. Available at: https://www.hiqa.ie/system/files/Guidance_on_Budget_Impact_Analysis_of_Health_Technologies_in_Ireland.pdf.
19. Guidelines for preparing a submission to the Pharmaceutical Benefits Advisory Comittee. 2016. Available at: https://pbac.pbs.gov.au/.
20. Assessing resource impact process. National Institute for Health and Care Excellence (NICE); 2017 [updated May]. Available at: https://www.nice.org.uk/Media/Default/About/what-we-do/Into-practice/assessing-resource-impact-process-manual-ta-hst.pdf.
21. Ferreira-Da-Silva AL, Ribeiro RA, Santos VC, et al. [Guidelines for budget impact analysis of health technologies in Brazil]. Cad Saude Publica 2012;28(7):1223–38. https://doi.org/10.1590/S0102-311X2012000700002
22. Neyt M, Cleemput I, Van De Sande S, et al. Belgian guidelines for budget impact analyses. Acta Clin Belg 2015;70(3):175–80. https://doi.org/10.1179/2295333714Y.0000000118
23. Sullivan SD, Mauskopf JA, Augustovski F, et al. Budget impact analysis—Principles of good practice: report of the ISPOR 2012 budget impact analysis good practice II task force. Value Health 2014;17(1):5–14. https://doi.org/10.1016/j.jval.2013.08.2291
24. Foroutan N, Levine M, Tarride J-E, et al. PP51 updating Canadian pharmaceutical budget impact analysis guidelines. Int J Technol Assess Health Care 2018;34(S1):86. https://doi.org/10.1017/S0266462318002131
25. Pearson SD. The ICER value framework: integrating cost effectiveness and affordability in the assessment of health care value. Value Health 2018;21(3):258–65. https://doi.org/10.1016/j.jval.2017.12.017
26. Ghabri S, Mauskopf J. The use of budget impact analysis in the economic evaluation of new medicines in Australia, England, France and the United States: relationship to cost-effectiveness analysis and methodological challenges. Eur J Health Econ 2018;19:173–175. https://doi.org/10.1007/s10198-017-0933-3
27. Budget impact test England. NICE. Available at: https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/nice-technology-appraisal-guidance/budget-impact-test.
28. Ochalek J, Lomas J, Claxton K. Assessing health opportunity costs for the Canadian health care systems. 2018. Available at: http://www.pmprb-cepmb.gc.ca/CMFiles/Consultations/new_guidelines/Canada_report_2018-03-14_Final.pdf.
29. Faleiros DR, Alvares J, Almeida AM, et al. Budget impact analysis of medicines: updated systematic review and implications. Expert Rev Pharmacoecon Outcomes Res 2016;16(2):257–66. https://doi.org/10.1586/14737167.2016.1159958
30. Mauskopf J, Earnshaw S. A Methodological review of US budget-impact models for new drugs. PharmacoEconomics 2016;34(11):1111–31. https://doi.org/10.1007/s40273-016-0426-8

APPENDIX 1

TABLE 1. Interview Guide, Open-Ended Questions

Semi-structured interview: We asked for participants’ personal opinions on some of the important issues related to BIA methodology and guidelines. The questionnaire was developed for stakeholders from different perspectives (e.g., federal and provincial levels), and some questions may not apply to everyone’s perspective and position.

A) Open-ended questionnaire for interviews (version#1)

BIA: budget impact analysis; CEA: cost-effectiveness analysis; pCPA: pan-Canadian Pharmaceutical Alliance; LCA: least cost alternative; PMPRB: Patented Medicine Prices Review Board

Note: Question#6 was replaced by the following question after six interviews mainly because we received the same answer from participants repeatedly: “In the case of co-payments, the new recommendations indicate that the patient perspective should be considered complementary to base-case analysis. What are your thoughts about this recommendation? What do you consider are the benefits of including the patient perspective? What do you consider are the limitations to including the patient perspective?”

A) Open-ended questionnaire for interviews (version#2)

TABLE 2 Interview Guide, Closed Survey

We used Likert-type ordinal scales to rate the responses, ranging from 1 to 5 (where: 1 = “strongly disagree,” and 5 = “strongly agree”), with a middle neutral category (3 = “neither agree nor disagree”). Please answer the questions by choosing a number between 1 and 5.

1. In your opinion, how important is it to include the treatment switch in the target population assessment in BIAs?

2. In your opinion, how important is it to include patient adherence in the target population assessment in BIAs?

3. In your opinion, how important is it to capture changes in the rate of mortality and disease progression over time horizon in BIAs?

4. Do you agree with including staff training or introduction costs for a new medication (if applicable) in a BIA?

5. Do you agree with including direct non-healthcare-related costs (e.g., transportation) or indirect non-healthcare-related costs (e.g., productivity) in Canadian BIA guidelines?

6. Do you agree with including the cost of adverse events, clinical outcomes, and disease complications in BIA cost analysis?

7. Do you agree with including the appropriate rate of value-added tax (e.g., HST)?

8. Do you agree with using costs and tariffs data from other jurisdictions (other countries) in the absence of local information?

9. Do you agree with considering expert opinions as a reliable source of data in BIAs?

10. Do you agree with using market-share (or claim-based) data extrapolated from similar drug experience on the payers’ database in case of lack of real-world data for the proposed medicine?

11. Do you think BIA guidelines should provide a list of acceptable databases as reliable references for input data in BIA calculations?

12. In addition to an incremental impact on the budget for a new medication reported in BIAs, how important is it to also report “total impact,” which includes previous expenditure plus the costs related to the newly proposed indications for that drug in BIAs?

13. Do you agree with considering a cap or threshold for the budget impact of the new drugs to signal the need for negotiation with manufacturers, for instance, to decrease the prices or set risk-sharing agreements?

14. In your view, how important is it to build in a reassessment process of BIAs in a future real-world post-market environment?

APPENDIX 2: SURVEY MONKEY

Link to the survey: https://www.surveymonkey.com/r/F6KRRTZ

Footnotes

¹ Patients could be added to or removed from the analysis based on whether they meet the inclusion criteria or not over time. In some cases, when a drug applies to a well-defined group of patients, the BIA may require a defined closed population.

² Canadian Agency For Drugs And Technologies In Health.

³ Common Drug Review.

⁴ Pan-Canadian Oncology Drug Review.

⁵ Pan-Canadian Pharmaceutical Alliance.

⁶ Non-Insured Health Benefits Program.

⁷ Incremental cost-effectiveness ratio.

⁸ Gross domestic product.

⁹ Régie de l’assurance maladie du Québec.