The Asthma Return on Investment (ROI) Calculator Web Conference: Transcript

---

On May 22, 2008, Rosanna Coffey presented the AHRQ Asthma Return-on-Investment (ROI) Calculator on a Web conference. This is the transcript of the event's presentation.

---

May 22, 2008
1:00-2:15 p.m. ET

Moderator-Margie Shofer: I'm Margie Shofer in the Office of Communications and Knowledge Transfer at the Agency for Healthcare Research and Quality. Thank you for joining us for this Webinar on the Asthma Return on Investment (ROI) Calculator. This Webinar is the second of three followup events featuring AHRQ tools that were shared at two workshops held this past December and January. We focused on the State Snapshots at the end of April and will hold another event June 16 on the preventable hospitalizations county-level mapping tool. We see all these events as the first step in what we hope will be a series of follow-on technical assistance opportunities. If, after learning more about the calculator today, you're interested in further assistance from AHRQ in using this tool, please let us know.

We're hosting this audioconference in response to interest in the ROI Calculator expressed by participants at both workshops. We know that some of you attended one of the workshops, whereas others may have had less time to interact with the tools. As such, we will spend some time reviewing the basics of the tool. We will then move on to a demonstration of the tools, will address questions about underlying data raised at the workshop, and will discuss interpreting the results for a variety of policy applications. We will conclude with a few caveats and further discussion.

We would appreciate your active participation, because one purpose of today's Webinar is to probe further into underlying data sources and other technical issues you may encounter when using the tool. We also want to explore practical applications for the tool, collect your suggestions for future tool enhancements or modifications, and learn how you envision using the tool and for what purposes. And again, as this last point, we really hope you tell us about the technical systems that you might need in order to be able to make full or better use of the ROI Calculator. It's really the purpose of this call today.

We're going to start the Webinar with a brief introduction from Jeff Brady, Director of the U.S. National Healthcare Quality Report here at AHRQ. He is an M.D. with a background in primary care and preventive medicine and public health. And he oversees the National Health Care Quality and Disparities Reports, the State Snapshots, and all derivative products, which include the Asthma ROI Calculator. Following his introduction, Jeff will introduce Dr. Rosanna Coffey from Thomson Reuters Healthcare, who will give today's presentation. Now I'm going to turn this over to Jeff.

Jeff Brady: Good afternoon everybody. It's my pleasure to be here today and involved with this WebEx teleconference. As you'll hear today, the Asthma Return on Investment Calculator can help you estimate the financial benefits of asthma quality improvement programs in your State. A real success of the calculator is that it combines information from the literature with real world data about patients to estimate the net impact of an asthma care quality improvement program. As you'll further see, it has lots of information about medical utilization and asthma prevalence that you can really input to apply this to real world examples.

And so what I'd like to do is introduce not only Rosanna Coffey, but also Ginger Carls, who unfortunately can't be involved in today's conference but has been very involved in both the production and the dissemination of the tool. As you participate further and perhaps engage in some technical assistance with the tool, if you have any questions that go beyond today's conference, it's very likely that Ginger will be involved in those.

Ginger's an economist in the Health and Productivity Research Department at Thomson Reuters Healthcare. She's worked on various projects to evaluate the performance of disease management programs, as well as health burden studies. She's currently a doctoral candidate pursuing a degree in the Department of Economics and Health Management and Policy. For today's presentation, Rosanna Coffey will lead that. Rosanna is a Vice President at Thomson Reuters Healthcare, formerly Thomson Healthcare, which before that was Thomson Medstat. Rosanna and her team support the development of the Asthma ROI Calculator. Rosanna is a Ph.D. economist with more than 30 years of health services research experience. So, I'd like to turn it over to Rosanna to start.

Rosanna Coffey: Thank you, Jeff. Good afternoon, everyone. I want to acknowledge Ginger's hard work on this. She is the energy and the brains behind this calculator. And we also had other colleagues at Thomson that we worked with: Ron Ozminkowski, and Greg Lenhart, who was our Excel wizard, and you'll get to see some of his work later. I'm stepping in for Ginger since she had a death in the family. If I get stuck on anything, we will get back to you in e-mail; that is if you ask me questions that I can't answer, but I hope to be able to answer them all.

So, before we start, I would like to ask you all a question. And that is, we're going to open the polling option, and we want to know, have you used the calculator, or are you planning to use the calculator? You should see that question, and if you just click on "yes" or "no," we'll get the results as they come in.

So I'm going to continue with the presentation at this point.

The plan for today is to give you a very brief review of the calculator, because many of you have already seen it and perhaps have used it, and I'm going to focus on how to use the calculator. The data that you can enter, the data that underlies the calculator, some of the default information that's already there, and we'll spend some time later on interpretation of results and looking at limitations and solutions for those limitations.

You may ask questions at any time through this, by going to the Q&A option on your right-hand menu. Jeff Brady is going to watch those questions for me, because I'm going to be busy concentrating on what I'm trying to say to you. He may interrupt me, if there's a clarifying question he thinks we ought to address as I'm going through.

Let's start with a review of the calculator.

The purpose of the calculator is really to help policymakers-primarily State policymakers-although private health care can use this tool too. The purpose is to help with program design for some kind of asthma care quality improvement. The calculator helps to estimate the financial gains or losses that you might observe if you institute a quality improvement program. And keep in mind, financial gains may not be the only reason that you may want to institute a quality improvement program, but this tool, at least, will help you get a handle on the financial aspect.

The calculator has summarized a huge literature pool; 52 studies are used behind the calculator, to come up with impact estimates. When you look at that literature, it is no mean feat to summarize it. People do analyses differently; they look at different aspects of health care costs. In most of these studies, the outcome that they were looking at was the change in utilization, and there were very few studies that attempted to convert that into what that means in terms of cost. So we did that translation for you.

I see we have the results of our poll. How cool. We're doing this mainly because I wanted to do it, to see how it would work. And we see that we have 52 percent of you who have used the calculator, which is great. So we hope that you will ask your questions. If you have them, we have two places in the presentation where we ask for your questions, so we're going to address them at that point.

Back to what I was saying: The calculator's purpose, also, is to help you focus on particular populations that you're interested in.

The message behind the calculator would boil down to something very simple. It's not quite this simple, but just to get it understood, we take the evidence that comes out of the literature, which is primarily utilization information. We add to that external information on cost of health care. And the combination of the two gives us the financial impact, tied into a lot more information, which I'm going to show you as we go through this.

Let's define what we're talking about. We're talking about asthma care programs that typically follow the national asthma education and prevention program guidelines. And those guidelines focus on patient education, as well as provider education. On the patient education aspect, we're talking about self-management of the disease, helping people with asthma understand how to avoid the triggers that cause asthma attacks, how to anticipate problems, and how to use their medications appropriately. On the provider side, we're talking about providers, and getting them to make accurate diagnoses, to prescribe medications appropriately, and also to monitor their patients' success and their attempts to self-manage their disease, as well as to provide patient education.

So in what we're talking about, we have combined the literature. Some programs may focus on different aspects, one on patient education, another on provider activities and education. But we have lumped them all together. So in some sense, we have an amalgam here of all of these different types of activities that have gone on.

So, we're going to turn now, to how to use the calculator, and we're going to focus on the data.

The data that you need to use this calculator are four main types of information. You need to figure out who the eligible asthma patients are. You need to know something about their baseline use of health care and the costs associated with that. You need to have some evidence around what happens if you implement an asthma improvement program-that comes directly out of the literature. And then you want to know what it's going to cost to implement a program, and we'll talk about that.

So, start with the eligible asthma patients. You have two options. One, you can calculate the number of eligible patients using your own data. If you are a Medicaid agency, or you're responsible for a State employee benefit program, or maybe a high-risk pool, you may have some information on people in that population. Maybe information from medical claims. You can, and you should, use your own data to populate the calculator.

If you don't have that information, you can use the default information that we provided. So let's assume for a minute that you have the information and you're going to calculate the eligible people with asthma from your own data. One of the very important things you need to do is think about which asthma patients you're targeting, and you need to analyze your data, or tabulate your data for the right population that you're interested in.

You may be interested in any patient with an asthma diagnosis. And that's the first bullet here, under "defined criteria," or you may want to target patients who have persistent asthma. We did include both of these, as well as the third option. But the patients with persistent asthma, we defined based on the NCQA HEDIS definition, and here's the Web site for finding that. The other place that you'll find it is in our detailed report.

If you're using the calculator, you should have a copy of our detailed report. It tells you everything that we had to do to come up with these estimates and lists all of the literature with abstracts on all of the studies that went into the calculator. And in tables 8 and 10 of that detailed report, we have the definition for persistent asthma. The third category that we give you is an option to choose if you want to target those very high-cost patients. You're likely to find patients with persistent asthma and an acute health care visit to an emergency department or a hospital within the year.

To use the calculator to estimate the eligible asthma patients, you would need the percentage of enrollees in each age and gender cell, or you'll see that in the calculator. And the percentage of enrollees by each race and ethnicity is available if you're looking at Medicaid. And you'll have the total number of enrollees.

I am going to jump us to a demo.

Thank you, Jose, for reminding me that I did have a question. But I wanted to ask our participants again, have you tried to enter data into the calculator? If you would go to your polling on the right-hand side and answer that question for us, we'll have a sense of how many of you have actually tried to do this.

Now, I'm going to share my desktop with you.

In a few seconds, you should be able to see the Asthma Care Return on Investment Calculator. You can see it is in Microsoft Excel, and at the top are a number of tabs and these really are the main aspects of the calculator: the population sheet, a participant sheet, the baseline cost and utilization data, program impact information that comes out of the literature, the program costs where you may want to enter actual values for the cost of the program you're considering, and the results sheet.

We have a lot of information in the appendix. Like I said before, we also have more detail in a paper.

Let's look at that population sheet for a second. You see you have an option here. You can look at Medicaid as a population; you can go to the employer-sponsored insurance population, or even State employees and their dependents, if you choose. And you also have the geographic option-you can select your particular State. If you're from Alabama, I'm selecting you now. And you can see the size of the population went down, and distribution of age, gender, and race changed, and this is for Medicaid in Alabama. And it was about 2005 data. Things may have changed; you may not think that this reflects you and Alabama very well. So you would want to enter the size of your population, and you may want to even enter your age, gender, and race distributions if they have changed.

I am going to go back and select the Nation here to show you the participant page. This is where you make some choices. Are you targeting children? Are you targeting adults? If it's both you keep them clicked. If you really just want to look at children, take off the adult. What asthma severity are you looking at? Is it all asthma patients, persistent asthma only, or persistent asthma with acute visits? When I choose the last one, you see the eligible patients go down. You also have the choice of making a judgment about how many people that you target are going to participate in this program. We have put in 25 percent, being very conservative. You may think you'll have a higher number of people who will be willing to go through the program, and you want to change that.

Let's jump back to the slide presentation.

As you see here, we have the option of selecting asthma severity, the type of coverage that you're dealing with, or the population. The prevalence rates that are in the calculator come out of a Thomson Reuters data product, which is the MarketScan® Medicaid or MarketScan® Commercial Database. And those prevalence rates are given as I showed you by the age and gender, and also race/ethnicity if it comes out of Medicaid. There is no race/ethnicity in most commercial claims, and we don't have it in ours. Also, we have made all those calculations for different asthma severity levels.

Now the next piece of information you will see, looking at the calculator, is the baseline utilization and cost information. This is the big cost that your population is incurring before you institute an improvement program. And the data fields relate to all the components of cost that you ought to consider when evaluating an asthma quality improvement program or you're thinking about designing one. What costs might be affected by your program?

So we have the per-patient per-year estimate of emergency department visits or hospital stays or outpatient visits. On the cost side, we're looking at the cost per visit, or the cost per stay for each of these components, as well as the asthma medication costs per patient per year, and the asthma ancillary costs, such as lab, imaging, and pulmonary function tests. Immunizations are even in here.

Notice that we have paid amounts. We're not talking about charges. We are talking about what was actually paid for medical claims. You have an option as to whether you include the patient's copayment. The other data field you have as part of the baseline is number of missed work or school days. This is an option for you if you want to estimate the productivity gains from instituting quality improvement.

Our advice in calculating estimates for these data fields if you're putting in your own data: You need to decide which use and cost components you want to include. We recommend you be comprehensive. If you've got information on one department, like emergency department, use what's in the calculator for the other components. Just make sure that for the emergency department you define and calculate your use and cost piece in the same way that you would set up the population. That is, if you're coming up with utilization rates on emergency department visits, and it's for the people with persistent asthma, make sure you've made that selection at the front-that you're looking at all the costs in our calculator that relate to people with persistent asthma. That's one of the most important things about consistency in using this tool.

So we're going to jump now to share my desktop with you again, and we're going to look at the baseline data. Here, you do have an option to select asthma treatment costs, or any treatment costs for anyone with asthma. All of these light boxes are data that we have calculated from the MarketScan claims data. I believe we're still selected on the Medicaid population. So these all come out of our MarketScan Medicaid claims data: the emergency department visits per patient, the hospital stays per patient, and on the right-hand side, the cost of an emergency department visit and hospital stay. If you have information on the baseline for your population, you really should enter that in here to get some idea about how a quality improvement care program will work for your population.

Also note here on the top right side. You can take two different perspectives. You can take just the perspective of your program or plan-for example, how much Medicaid will save. Or you can take a social perspective and include the patient's cost in the calculation.

Let's jump back to our slides.

Let me just run through quickly what is behind that baseline information. I told you about MarketScan claims. There are eight Medicaid States; they are geographically dispersed. They represent about 7 million covered lives. On the employer-sponsored health insurance, the private health insurance component, if you would select that, back at the beginning of the calculator, the baseline information would relate to over 100 large, self-insured employers, about 15 million covered lives. These employers are located all over the United States. But one thing to keep in mind, when you're talking about insurance coverage by large employers, they are generally more generous plans than smaller employers, or certainly than individual coverage.

For the missed schooldays, or workdays, we borrow an estimate. It's a total U.S. estimate that comes out of the National Health Interview Survey for 2003. And we evaluate those missed work and school days, using the average wage rate, that comes out of the Bureau of Labor Statistics. That particular wage rate, we have available by State. So we apply the State's average wage rate when we're coming up with these cost estimates. For people who are on Medicaid, we use the Federal Poverty Level for the value of a missed workday.

Now we're finished with the baseline.

Let's talk a bit about those impact estimates that come out of the evidence. The data fields that we have in the calculator relate to the number of asthma-related hospital stays, ER visits, outpatient visits, payments for prescription drugs in the retail setting, for ancillary services, as well as those numbers of missed work and school days. Where do these data come from? This is the big effort that we undertook. The results come from our meta-analysis of the literature and that is what is in the calculator.

You may have estimates from a pilot test that you've already done on some asthma improvement program, and you may want to enter your own pilot test information in here. Or you may know about a study that's more recent than the ones we looked at. Our review ended in March 2007. If you know about a new study and you want to see how that plays into the calculator, you can enter those data in as well.

Jeff Brady: This is Jeff. Can I interrupt at this point for a moment?

Rosanna Coffey: Yes.

Jeff Brady: One question that I think fits into this part of your presentation.

Rosanna Coffey: Absolutely.

Jeff Brady: One question was about the paid amount for facilities for inpatient care and the fact that that varies, and what the default amount is. Is it just an average cost of that? You've already kind of touched on how folks can modify that. But I think, "What's the default amount based on?"

Rosanna Coffey: The default amount in the baseline is the average for the population that we're talking about. So if we go back to the calculator, this baseline cost of $5,705 is for Medicaid. It's for children, because I had checked children. And it's for those who had persistent asthma with an acute visit. Those are the things I have selected, and it is an average of paid amount to the hospital for people in that population. Does that help?

Jeff Brady: Yes. I think so.

Rosanna Coffey: O.k. Let me see where I was. I think we're ready to move on.

I was going to take you to the program impact page. This is where we have all of the information summarized from the literature. We went through 52 studies, we broke them down, and we captured information on all of the characteristics of those studies. And we noted whether they were randomized controlled studies, statistically controlled studies, or studies with no control group. There are all types of controlled studies, including the randomly controlled studies, as well as the statistically controlled studies. We put both in here because there are so many more studies that are statistically controlled, and that increases the precision of our estimates.

But we included the randomized controlled studies as a separate option, because if you're trying to convince physicians that this is going to be a good thing to do, they are going to want to hear about randomized controlled studies. The interesting thing here is that the numbers change very little. The results were very similar for the randomized and statistically controlled studies in our work. When I go to studies with no control group, the estimates change a lot, and this is the problem of regression to the mean, because the people that were high cost will come down in cost anyway over time.

We only include this no-control group in here, because if you're doing a pilot study, and you've got some initial results, and you don't have a control group fully analyzed at some point, and you want to compare your pilot study results, you can compare them to these studies with no control groups. But I think it is so powerful to see that there is a big difference here between the controlled and uncontrolled results, that you'll realize that you really do want a control group when you're doing your evaluation.

Back to our slides. This slide tells you that if you're going to enter your own data for your pilot test, or you're going to use some study that's more recent than we have, you're going to have to do the calculation of the percent change in visits.

You'll do that differently, depending on the type of study you have. If it's no control group, you'll just do this simple before-and-after percent change. If it's randomized, you can simply look at the treatment and the control group difference.

I have a little bit of information here about the studies behind the evidence base. There were 52 studies. They don't all look at all the components of cost. That's why it's important to pull them all together and synthesize it. These are in order by the components that have the most evidence behind them. The ED visits were the most frequent type of study, followed by hospitalizations, outpatient visits, and so on.

Look at the medication costs. We've only had 10 studies that actually looked at medication costs. You have to moderate your excitement about those numbers and what they mean.

Ancillary costs had only three studies. So in those cases, we didn't actually do a meta-analysis, but we included the average cost effects.

Now we're turning to the program costs, and the design features of the program. The data fields are the annual costs per participant of the program. This, you might get from a disease management vendor, or you might tally it up for your own organization if you're going to incur the cost yourself.

For the number of years until the program has full impact, you decide that. Also, you'll choose the discount rate and duration of the program. If you're looking at a program for 3 years, you would specify that, and we can show you quickly the program costs. It's a simple spreadsheet.

Here is where you put in the annual program costs for participant: $395 is the average that we found in the literature. It may be high, but at a minimum, you have a place to enter an amount you may be negotiating with a vendor and see what the implications are of that particular cost.

Now I will exit the desktop and go back to the slides.

How do you choose a discount rate? What we used basically was the rate of inflation. It really is just a method to equalize that stream of costs that you incur over the time of the program and the stream of benefits that you expect. The cost can be lumped up front, the benefits can be lumped at the end, and so you're trying to equalize that time differential, and the value of time and money spent or received is different across time periods. Money is much more valuable when you've got it in your pocket than when you're waiting for it out into the future.

You may want to think about how you would estimate the annual cost per participant. I think here, you may want to call some disease management vendors and try to get some costs. We tried to do that. We weren't very successful. We didn't have a real program that we were trying to set up with them, so they weren't too interested in helping us out on this.

This wraps up the types of data needed. We've talked about all four of these types of data. When putting it together, I just want to say that it's ideal if you get it from a single source. It's not essential that you have it from a single source; you may get one thing from your Medicaid claims, you may get something else from another study in the literature or your pilot study.

The important thing is that you make sure you're talking about the same population of asthma severity, Medicaid, privately insured, or children/adults, and that all of those definitions are the same in the underlying data that you're bringing into the calculator.

I want to jump to the desktop and show you the results of the calculator.

This is the results page. It has the results related to all of the things that you've selected in the prior sheets. On the bottom right is the overall impact. The net present value is just the difference between the total program costs that you'd see here on the left and the total health care savings per participant that you see at the top. In this case we did not include productivity gains, so we have a positive net present value for this population. The return on investment means that for every dollar that you invest, you're getting $1.05 back. So you are definitely covering the costs of your program. In this case, if it costs $395.

One thing you do have to do is hit the Update button if you need to refresh those. Even though the estimate of costs we had in here was $395, you can see from here, you can go up to a program cost of $412 and still cover the costs of that program in what we expect to return.

What if I had selected a population that was persistent asthma, not just those with an acute visit? Say, I was trying to reach a higher number of people. So when I go to the results page, I don't have a positive net present value here. I get, for every dollar I spend on all people with persistent asthma, only 32 cents back. Thus, I'm going to be subsidizing the improvement and care for these people.

By playing around you will see where you will get a return and where you will not get a return but will be eating some of the costs of the program. If we could get our costs down to $126 for the cost of this program, we would be just covering our costs. If I included productivity gains, I could go up to $213 for the cost of the program and cover the cost.

So that shows you how the calculator works. And how you can put your own data into it. Shall we stop for questions? Jeff, do we have questions?

Jeff Brady: I've actually answered a few questions offline that maybe we should open it up for questions verbally now.

Rosanna Coffey: O.k.

Jeff Brady: Does anybody have questions so far?

Rosanna Coffey: Do we have a question on the line? Identify yourself and let us hear your question.

Jeff Brady: Sounds like there are not any questions. I think it was somebody just some offline discussions.

Rosanna Coffey: So, for those of you who are having discussions and we are listening to them, you may want to mute your phone so we don't hear them. Shall we move on?

Jeff Brady: Yeah, I think so.

Rosanna Coffey: O.k.

Now we're going to turn to interpretation of the results. I'm going to give you a little sense of how the results change with different populations. To start with, I have some default decisions that I made, and I'm going to do this in the slide, and show you some tables of different scenarios.

I selected the population nationwide. I selected a program that lasts for 5 years. I want to evaluate it within and at the end of 5 years. I assume that it will take me a couple of years to get my participation up to where I assume it should be. And the discount rate is 3 percent. You may want to up that, with all the talk about inflation that's going on. But we did this earlier, and we used the 3 percent rate. I'm going to use the evidence from randomized controlled studies, because I have to convince physicians about this.

Here are four scenarios. I have four more on the next slide, which I'll show you in a minute. In the first three, we're looking at the Medicaid population.

The first scenario is for adults and children, or children only. All of these are populations with persistent asthma. Annual cost of the program in the first scenario is $395 and in the second as well; then I lowered it to $100 in the third. For my assumption about which costs I'm worried about, I'm looking at this from society's point of view. So I'm counting Medicaid's cost as well as the patient's costs if there are any copays under Medicaid. I'm not going to include productivity costs. You can see, nationwide, I expect with my assumptions about participation of 25 percent that I'd have about 2 million people nationwide in an asthma program, or 850,000 children.

The net present value is negative in the first two cases, and when I lowered the program cost from $395 to $100, I got a positive net present value of $123 per person. The return on investment is 27 cents that I receive back in addition to covering my costs in the third scenario. If I had lowered the cost of the program in the first case to $56, I would have covered my costs, or if I had lowered it to $126 for the children-only program, I would have covered my costs.

For the employer-sponsored scenario, I have the same assumptions that I had in the first three scenarios. The difference is that the employer-sponsored children don't have as high a baseline cost and utilization costs as Medicaid children do, and so my return on this is not as high as it was for Medicaid-actually it's a negative net present value for the employer-sponsored group. From each dollar that I spend, I only get 87 cents back.

Now the next four scenarios are employer sponsored. The first two are for children (scenarios five and six) for acute visits for persistent asthma. These are the people that went to the hospital, were admitted, or were treated in the emergency room and released. Again, we kept their program costs pretty low and assumed all medical costs for the patient as well as the plan in the first case and only the plan costs in the second case.

We did not include productivity, but we have here a net present value, because we're looking at very high cost cases, those with acute visits. We get a big return on this particular population when we're looking at those with a lot of utilization in the past year.

Let's look at this last one, because we have such a huge return on this one. These are adults, they have persistent asthma, and they have $100 program cost per year. We include not only the plan cost, but also productivity gains. When you include the benefits to society of having people not miss work, and you evaluate that, you have a big return on investment.

How do you use the calculator? You use the calculator to forecast the financial impact of the program that you might design. Keep in mind that financial return is not the only thing you may be trying to achieve.

And, you want to assess the key assumptions about your proposed program. Make sure that you know whether you're doing this with children or adults. The calculator may help you make that decision. What asthma severity are you going to target? What are the costs of the program? Are you going to consider social costs, and is productivity something you think you should be counting as a benefit or not?

One of the things you can do with this is figure out if your own assumptions are reasonable compared to the evidence in the literature.

You can enter your own data. You might have a pilot test that you've done, and you want to enter that.

You may be talking to vendors about how much a program will cost. You may want to put that in here and see how it plays out. You may want to use it to negotiate with a vendor.

Another way you can think about using the calculator is, suppose you already have a program going on. What kinds of things should you be evaluating and including in your evaluation of your asthma care program?

We went through that scenario with the New York Department of Health. I don't know whether anyone is on from New York, who worked with us. They asked for some technical assistance. We spent 45 minutes on the phone listening to what it was they were doing and tried to apply the calculator to their situation. They were looking at a program that was much bigger than asthma. It was a housing improvement program but they thought that that improvement program should help reduce asthma symptoms and they were interested in pulling that aspect into the evaluation. At the end of our call, we decided the calculator itself wasn't something they needed to put data into and work through. But they found it really helpful in terms of thinking about the actual components that they should be looking at. They hadn't thought about looking at emergency department visits and hospitalizations, counting the medication costs, and things like that. So they were interested in it for the conceptual framework that it provided them.

I also want to mention that we got into this calculator because AHRQ had a program to work on pediatric asthma. The Michigan Pediatric Asthma Coalition was part of that. I think Betsy is on the call. We worked with Betsy and her colleagues in the Michigan Department of Health, and they influenced us a lot in terms of designing this. I understand that they actually used it in making a decision about a county-level asthma program. They had to enter county-level data into a calculator, which you can do, and figure out whether it made sense for them to do an asthma improvement program in a particular county.

We've also had a request for the calculator from Iowa, the Iowa Medicaid Medical Officer. I don't have details on that, but they were looking at it in terms of whether they should start an asthma care improvement program.

The last thing that we wanted to cover if we have time, or if there are burning questions, we can jump to those. But the last-

Jeff Brady: Actually, Rosanna, let me interrupt for a few questions that we've received since the last question period.

Rosanna Coffey: Sure. Great.

Jeff Brady: One is generally relating to our plans for updating the tool as more literature becomes available. What are our plans for that? I can answer that, actually. We don't have specific plans yet to update the tool that was performed a year ago, and that involved reviewing the literature for new findings that could be incorporated into the literature summary that Rosanna provided at the beginning of the session today. I think a lot of our future plans for that will depend on interest in the tool and, in fact, use of the tool.

The other big variable is what actually comes out. Of course, if there are big studies that would have a more significant impact on the input, then that's something that we'll take into consideration too. That answers that question.

There have been a few questions that have come through about obtaining the calculator and how to get that. Presently it's available by request to Margie Shofer. Her e-mail's at the end, which Rosanna will get to.

There was a specific technical question about the 12-month Medical Consumer Price Index (MCPI) change as a discount rate. I don't know if Rosanna you want to take that. The question was: Did you decide to use a 12 month MCPI change as a discount rate?

Rosanna Coffey: The discount rate that we used was an annual inflation rate from the Consumer Price Index. It wasn't the medical care component. We're just trying to equalize the cost of money. Not the medical care cost inflation, so that's why we used the 3 percent rate.

Jeff Brady: O.k. I think that catches us up. By all means, if anybody has a question that we haven't addressed that you sent by text, please let us know, and then we can unmute your phone if you'd like to ask a question that requires maybe a little more explanation. Otherwise, I guess we're good to go ahead, Rosanna.

Rosanna Coffey: O.k.

Let's go to limitations/caveats. We just want to tell you when the results are less than robust.

We only had seven studies that reported program costs. They had a very wide range from $81 to almost $1,000. We put in the average amount. But I think that's a place where you'd want to be snooping around and finding out where you can get cost data, what a program will cost, what it looks like and what it includes. And maybe enter that information directly, rather than using what we have in there from the literature.

The other studies that we were disappointed in were: Few studies looked at medication costs. There were 10 of them. They are the more recent studies. There may be more recent ones now. The baseline asthma medication cost in those studies really varied by a lot. So that's another place, maybe if you're going to play around with it. Maybe look at the literature, think about better-cost estimates.

Jeff just discussed the next bullet, which is the plan for updates. They want to hear from you about that. Literature continues to grow and you can monitor the literature. We did it through March of 2007, so anything more recent won't be in the calculator.

Baseline data also do become obsolete. I think I showed you that we used 2005 data for most of this. We did inflate it to 2006 dollars. You may want to inflate it now if you're looking at 2008 dollars.

And those are basically the main limitations of the calculator.

I think the real advantage of this calculator is that it gives you a real handle on what the literature tells you. I did not go into how we went about getting these average impact estimates. We have a very detailed paper that walks through all of that and the analysis and all of the things we controlled.

We were just impressed with the amount of information that was out there and that we were able to synthesize, and I can't emphasize enough that it's not an easy thing to do. Even if you have the time to read all that literature, you can't wrap your head around it and figure out what the average effects are, unless you do something quantitative like we did in our meta-analysis.

Let's open the line for other questions at this point.

Jeff Brady: Rosanna, can you hear me?

Rosanna Coffey: Yes, I can.

Jeff Brady: There's a question about Medicaid programs with a majority of members that are enrolled in managed care or full-risk capitation programs. The question is: Other than the apparent benefit of improved health outcomes or productivity gains, how can you apply the ROI tool in that case? How could it be used?

Rosanna Coffey: If you have a population, if I understand the question correctly, with the majority of your Medicaid members in a managed care program, the data that we have that underlies the calculator is based on about 60 percent of Medicaid enrollees in managed care in the data that we use to populate the calculator.

If your managed enrolled population is about 60 percent, this is probably a pretty good estimate for you, although if it were convenient to put your own numbers in here, I would do that. If your managed care enrollment is 100 percent, I would definitely put your own numbers in to see, because your baseline utilization numbers are going to be different. And you want to take that into account when you're looking at a return on investment.

If your baseline costs are low, you may not get the same kind of return on investment that we get out of this with the average effect. It is in here-the managed care component. It's reflected in these data, but it just depends on where you are compared to 60 percent coverage, as to whether this is realistic for you.

Jeff Brady: O.k. Some other questions that have come in. It's fairly short. Where are the payments for physician care during an inpatient stay or ER visit captured?

Rosanna Coffey: They're going to be captured in the inpatient stay or the ER visits because in the claims that we get, we bundled those into that component. If the care were provided in the ER, if the care were provided in the hospital, it would be bundled in there. If the care were in the doctor's office, of course it's going to be under the outpatient part.

Jeff Brady: O.k. Then there's another question about access to the articles and the literature. That's part of the tool, right, Rosanna?

Rosanna Coffey: In the paper we wrote, we have the abstracts for all the articles that we included in the study as well as some that we did not include, because their metrics were not set up so that we could use them. You could go to our appendix in that paper and look at all the study abstracts. Then go and obtain the articles online or from the journal itself. That's how you would get access to it. But you can get a lot of information out of our paper.

Jeff Brady: O.k. Just to clarify another question that's come through a couple of times, is this just for an Asthma ROI Calculator, just for this disease only at this point?

Rosanna Coffey: That's definitely correct. It's just for asthma. It's the asthma literature that drives the impact estimates. Asthma was the population we looked at for the baseline estimates, and you could not use this for, say, diabetes or some other disease. We wouldn't put our Good Housekeeping Seal on it.

Jeff Brady: Right. O.k. I think that catches up all the questions that I've seen by text, but it's possible that I've missed a few. If that's the case and somebody wants to ask their question over the phone line, that's something we can certainly do now or, additionally, if there's followup on one of the questions that has already been asked, and some additional clarification is needed, we can do that at this time, too.

Rosanna Coffey: I think the other thing we ought to do before we sign off is do another poll and ask the question: How many organizations out there think they would like to have some technical assistance on the calculator? So, would you be interested in receiving technical assistance on the calculator? Just if you wouldn't mind answering that question at this point, and we'll get some idea.

It could be providing some presentations to a group in your organization, or it could be working through the calculator with you and your particular situation, like we did with New York. It could be holding your hand while you calculate numbers. All kinds of things.

We are willing to listen to your voice if you'd like to ask a question verbally and don't want to type. That works too for us.

O.k. Would you be interested in technical assistance? Seventy-six percent said yes. I have to find out how many were under that. Jose, can you tell us how many respondents there are? I see the percent, but I don't see the number. We'll figure that out.

We have a little more than 5 minutes left. Who am I trying to get back to, Margie? Are you on mute, Margie?

Margie Shofer: Hello.

Rosanna Coffey: There you are.

Margie Shofer: Thank you, Rosanna. Thank you, Jeff, for the presentation today and thank you everyone on the line for your thoughtful questions and your participation in this Webinar. We hope you found this discussion helpful. If you have questions about follow-on technical assistance opportunities, please do not hesitate to contact me, Margie Shofer.

Rosanna Coffey: I have that up on the screen.

Margie Shofer: That's my contact information in the first bullet on the slide. If you have any questions or comments about the tool, or would like to request a copy of the calculator, please send an e-mail to the AHRQ Quality Tools mailbox. That's the second bullet there. For more information about the suite of AHRQ tools developed for the State Quality Improvement Project, please visit the site on that final bullet. Under that final Web address, a bunch of you asked if there was going to be a recording of this.

And there is. We're going to have a recording of events, as well as a transcript available. I'm not sure how quickly it will be up, but it will be up there hopefully sometime soon. So I want to thank you again for being on this Webinar. And this concludes the Webinar and we look forward to hearing from you. Bye.

Rosanna Coffey: Goodbye.

Current as of May 2009

Return to Contents