Quality measurement of children’s health care: A new stage of growth

Quality measurement for pediatric health care is accelerating – and maturing, some say – with an increasing focus among policy makers, providers, and other experts on developing common quality measures that can be used more consistently and widely and that better capture outcomes and the patient experience.

The ideas aren’t new, but current efforts may be more successful than past ones in addressing what have been unrelenting challenges. Finding quality measures that work in pediatrics is even tougher than in adult health care: Children are developing, dependent, and less often fit into neat categorical condition groups. The scientific evidence base is much weaker, and the overall goals for children in health care involve maximizing development and growth – an upward trajectory that is challenging to measure.

"The measures we use today for child and adolescent health care are mostly process measures, and it’s not always clear that these correlate with the outcomes we want to see," said Dr. David Keller, professor of pediatrics at the University of Colorado and vice chair of clinical affairs and health care transformation at Children’s Hospital Colorado, Aurora.

Increasingly, however, experts involved in measuring child and adolescent health are pursuing common and consistent quality measures that, when used more widely, will facilitate more comparison and the collection of more data to be used in looking at long-term outcomes and population impact.

"It’s mind-boggling to see all the available [pediatric] measures ... all the indices and projects out there. Yet there’s little pooled data, and no common language – especially for routine, general pediatric primary care," said Judith S. Shaw, Ed.D., M.P.H., R.N., executive director of the Vermont Child Health Improvement Program (VCHIP).

Dr. Shaw’s organization is housed at the University of Vermont, in a small state with an advanced health information infrastructure and a history of multi-insurer payment reforms that support medical homes and community health teams. VCHIP has supported and led a string of quality improvement projects over the years, but "we realized we have no longitudinal database to look at what’s transpiring across the state," she explained.

Dr. Shaw recently led a team of pediatricians and other experts from across the state in culling through almost 200 pediatric quality measures developed by the National Committee for Quality Assurance (NCQA), Bright Futures, and an array of other state and national organizations and efforts. They narrowed the list to a core set of about 40 measures that VCHIP will monitor over the long term in an attempt to bring measures in line with each other, identify gaps and areas for refinement, and more easily inform improvements in child health and health care. Chart auditors will collect data from a random sample of charts from participating practices.

Over time, Dr. Shaw hopes to pool her state’s data with data from other states. VCHIP leads and advises a network of more than 15 states with similar Improvement Partnerships (IPs), or collaborative efforts to measure and improve children’s health care. The network provides an opportunity to create the large data pools that are important for pediatrics and measurement of long-term outcomes. First, though, measures will need to be streamlined.

Dr. Shaw and her colleagues recently surveyed the IPs in 12 states and found that, of 100 primary care measures described in the survey, only 8 measures were utilized by more than one state IP. Similarly, among three states measuring asthma care, only 2 of the 14 available measures were used by all three states. "There’s not a lot of overlap in the measures that state QI projects are using," and in the big picture, this is limiting, said Dr. Shaw, also associate professor of pediatrics at the University of Vermont, Burlington.

On a national level, development of the 2013 first "core set" of children’s health care quality measures for voluntary use by Medicaid and State Children’s Health Insurance Programs was driven by the desire to "get everyone [across states] aligned and to be able to make fair comparisons of children’s health across the country," said Sarah H. Scholle, Dr.P.H., vice president of research and analysis at NCQA.

A core set of children’s health care quality measures was mandated by the Children’s Health Insurance Program Reauthorization Act of 2009 (CHIPRA) and developed under the auspices of the Agency for Healthcare Research and Quality (AHRQ) and the Centers for Medicare & Medicaid Services (CMS). It is meant to be annually updated and enhanced – a 2014 version is now in use – and to be usable across different settings and by multiple stakeholders including payers, providers, and consumers. This isn’t an easy task: Many experts maintain that data can rarely be used for both accountability and quality improvement.

A CHIPRA-mandated $55-million Pediatric Quality Measures Program (PQMP) has been supporting seven Centers of Excellence in an effort to refine existing measures in both primary and specialty care and develop new ones that are more evidence based. "[Finally], there has been substantial federal investment for the design of pediatric measures to be used nationally," said Dr. Mark Schuster, who is William Berenberg professor of pediatrics at Harvard Medical School and the principal investigator for CHIPRA’s Center of Excellence for Pediatric Quality Measurement at Boston Children’s Hospital.

Thus far, it "has been hard to learn from each other ... To get a sense of what and how well your peer groups are doing," he said. "We want to be able to go to a hospital that’s consistently doing better on a certain measure and ask, ‘How are you doing this? Can you help us?’ "

Experts in quality measurement say that interest and efforts are increasingly focused on measures that address more than the structure and process of care, as well as on measures that address behavioral and mental health. The national Children’s Core Set includes three measures relating to mental and behavioral health.

NCQA’s Healthcare Effectiveness Data and Information Set (HEDIS) measures – which comprise a large portion of the Children’s Core Set – have "provided a basic floor of measures" for various stakeholders in the health care system by looking at factors such as immunization status and whether or not children receive well-child visits, for instance, "but they don’t say anything about what happens when the child is there or" how the child fares afterwards, said Dr. Charles Homer, chief executive officer of the National Initiative for Children’s Healthcare Quality in Boston, which assists practices, hospitals, and others in using data and measurements to improve.

"You can do very well on the HEDIS measures and still not deliver the kind of care you or I would want our children to get," he said. "As part of our efforts to improve quality, we need data-driven, higher-bar measures. ... I don\'t think we’re very far from adding some of these kinds of metrics."

NCQA’s Dr. Scholle doesn’t disagree. Like Dr. Schuster, she also is serving as a principal investigator for the PQMP. One of her main areas of focus has been the development of measures on adolescent depression care. "We need to see whether adolescents [who receive depression care] are actually improving, with a reduction in symptoms over time," she said.

Also of growing importance in pediatric quality measurement is what experts call the patient or family "experiences of care." Dr. Scholle, for one, can attest to the value of such input when it comes to measuring the quality of depression care. NCQA has been incorporating measurement of the patient/ family experience in some of its HEDIS measures, and Dr. Schuster and his colleagues in Boston have been testing a family experience of care measure in pediatric hospitals across the country.

"We’re not talking about patient satisfaction – we’re talking about the experience of care and the outcomes," said Christina Bethell, Ph.D., M.B.A., M.P.H., professor of pediatrics at the Oregon Health & Science University, Portland, and founding director of the Child and Adolescent Health Measurement Initiative, which focuses on developing patient- and family-centered measures of quality.

"We can’t ask patients about what kinds of lab tests they had, but we can ask them about their behaviors and their functioning and their outcome," she said. "Is the child going to school, for instance? Sleeping well?"

Growing use of electronic medical records and other technologies – as well as financial incentives – may help support the collection of such information. And, Dr. Bethell maintains, there is a wealth of information to tap into from national surveys like the National Survey of Children’s Health that can be used to support state and regional-level measurement and quality improvement.

Overall, "we have a lot of what we need right now to" better measure the quality of children’s health and health care. "We just have to use it better," she said, adding that the main supportive pillars of good quality measurement – policy, culture, technology, and financial incentives – "are coming together." That, she said, "is what’s really changing."

Quality measurement for pediatric health care is accelerating – and maturing, some say – with an increasing focus among policy makers, providers, and other experts on developing common quality measures that can be used more consistently and widely and that better capture outcomes and the patient experience.

The ideas aren’t new, but current efforts may be more successful than past ones in addressing what have been unrelenting challenges. Finding quality measures that work in pediatrics is even tougher than in adult health care: Children are developing, dependent, and less often fit into neat categorical condition groups. The scientific evidence base is much weaker, and the overall goals for children in health care involve maximizing development and growth – an upward trajectory that is challenging to measure.

"The measures we use today for child and adolescent health care are mostly process measures, and it’s not always clear that these correlate with the outcomes we want to see," said Dr. David Keller, professor of pediatrics at the University of Colorado and vice chair of clinical affairs and health care transformation at Children’s Hospital Colorado, Aurora.

Increasingly, however, experts involved in measuring child and adolescent health are pursuing common and consistent quality measures that, when used more widely, will facilitate more comparison and the collection of more data to be used in looking at long-term outcomes and population impact.

"It’s mind-boggling to see all the available [pediatric] measures ... all the indices and projects out there. Yet there’s little pooled data, and no common language – especially for routine, general pediatric primary care," said Judith S. Shaw, Ed.D., M.P.H., R.N., executive director of the Vermont Child Health Improvement Program (VCHIP).

Dr. Shaw’s organization is housed at the University of Vermont, in a small state with an advanced health information infrastructure and a history of multi-insurer payment reforms that support medical homes and community health teams. VCHIP has supported and led a string of quality improvement projects over the years, but "we realized we have no longitudinal database to look at what’s transpiring across the state," she explained.

Dr. Shaw recently led a team of pediatricians and other experts from across the state in culling through almost 200 pediatric quality measures developed by the National Committee for Quality Assurance (NCQA), Bright Futures, and an array of other state and national organizations and efforts. They narrowed the list to a core set of about 40 measures that VCHIP will monitor over the long term in an attempt to bring measures in line with each other, identify gaps and areas for refinement, and more easily inform improvements in child health and health care. Chart auditors will collect data from a random sample of charts from participating practices.

Over time, Dr. Shaw hopes to pool her state’s data with data from other states. VCHIP leads and advises a network of more than 15 states with similar Improvement Partnerships (IPs), or collaborative efforts to measure and improve children’s health care. The network provides an opportunity to create the large data pools that are important for pediatrics and measurement of long-term outcomes. First, though, measures will need to be streamlined.

Dr. Shaw and her colleagues recently surveyed the IPs in 12 states and found that, of 100 primary care measures described in the survey, only 8 measures were utilized by more than one state IP. Similarly, among three states measuring asthma care, only 2 of the 14 available measures were used by all three states. "There’s not a lot of overlap in the measures that state QI projects are using," and in the big picture, this is limiting, said Dr. Shaw, also associate professor of pediatrics at the University of Vermont, Burlington.

On a national level, development of the 2013 first "core set" of children’s health care quality measures for voluntary use by Medicaid and State Children’s Health Insurance Programs was driven by the desire to "get everyone [across states] aligned and to be able to make fair comparisons of children’s health across the country," said Sarah H. Scholle, Dr.P.H., vice president of research and analysis at NCQA.

A core set of children’s health care quality measures was mandated by the Children’s Health Insurance Program Reauthorization Act of 2009 (CHIPRA) and developed under the auspices of the Agency for Healthcare Research and Quality (AHRQ) and the Centers for Medicare & Medicaid Services (CMS). It is meant to be annually updated and enhanced – a 2014 version is now in use – and to be usable across different settings and by multiple stakeholders including payers, providers, and consumers. This isn’t an easy task: Many experts maintain that data can rarely be used for both accountability and quality improvement.

A CHIPRA-mandated $55-million Pediatric Quality Measures Program (PQMP) has been supporting seven Centers of Excellence in an effort to refine existing measures in both primary and specialty care and develop new ones that are more evidence based. "[Finally], there has been substantial federal investment for the design of pediatric measures to be used nationally," said Dr. Mark Schuster, who is William Berenberg professor of pediatrics at Harvard Medical School and the principal investigator for CHIPRA’s Center of Excellence for Pediatric Quality Measurement at Boston Children’s Hospital.

Thus far, it "has been hard to learn from each other ... To get a sense of what and how well your peer groups are doing," he said. "We want to be able to go to a hospital that’s consistently doing better on a certain measure and ask, ‘How are you doing this? Can you help us?’ "

Experts in quality measurement say that interest and efforts are increasingly focused on measures that address more than the structure and process of care, as well as on measures that address behavioral and mental health. The national Children’s Core Set includes three measures relating to mental and behavioral health.

NCQA’s Healthcare Effectiveness Data and Information Set (HEDIS) measures – which comprise a large portion of the Children’s Core Set – have "provided a basic floor of measures" for various stakeholders in the health care system by looking at factors such as immunization status and whether or not children receive well-child visits, for instance, "but they don’t say anything about what happens when the child is there or" how the child fares afterwards, said Dr. Charles Homer, chief executive officer of the National Initiative for Children’s Healthcare Quality in Boston, which assists practices, hospitals, and others in using data and measurements to improve.

"You can do very well on the HEDIS measures and still not deliver the kind of care you or I would want our children to get," he said. "As part of our efforts to improve quality, we need data-driven, higher-bar measures. ... I don\'t think we’re very far from adding some of these kinds of metrics."

NCQA’s Dr. Scholle doesn’t disagree. Like Dr. Schuster, she also is serving as a principal investigator for the PQMP. One of her main areas of focus has been the development of measures on adolescent depression care. "We need to see whether adolescents [who receive depression care] are actually improving, with a reduction in symptoms over time," she said.

Also of growing importance in pediatric quality measurement is what experts call the patient or family "experiences of care." Dr. Scholle, for one, can attest to the value of such input when it comes to measuring the quality of depression care. NCQA has been incorporating measurement of the patient/ family experience in some of its HEDIS measures, and Dr. Schuster and his colleagues in Boston have been testing a family experience of care measure in pediatric hospitals across the country.

"We’re not talking about patient satisfaction – we’re talking about the experience of care and the outcomes," said Christina Bethell, Ph.D., M.B.A., M.P.H., professor of pediatrics at the Oregon Health & Science University, Portland, and founding director of the Child and Adolescent Health Measurement Initiative, which focuses on developing patient- and family-centered measures of quality.

"We can’t ask patients about what kinds of lab tests they had, but we can ask them about their behaviors and their functioning and their outcome," she said. "Is the child going to school, for instance? Sleeping well?"

Growing use of electronic medical records and other technologies – as well as financial incentives – may help support the collection of such information. And, Dr. Bethell maintains, there is a wealth of information to tap into from national surveys like the National Survey of Children’s Health that can be used to support state and regional-level measurement and quality improvement.

Overall, "we have a lot of what we need right now to" better measure the quality of children’s health and health care. "We just have to use it better," she said, adding that the main supportive pillars of good quality measurement – policy, culture, technology, and financial incentives – "are coming together." That, she said, "is what’s really changing."

Quality measurement for pediatric health care is accelerating – and maturing, some say – with an increasing focus among policy makers, providers, and other experts on developing common quality measures that can be used more consistently and widely and that better capture outcomes and the patient experience.

The ideas aren’t new, but current efforts may be more successful than past ones in addressing what have been unrelenting challenges. Finding quality measures that work in pediatrics is even tougher than in adult health care: Children are developing, dependent, and less often fit into neat categorical condition groups. The scientific evidence base is much weaker, and the overall goals for children in health care involve maximizing development and growth – an upward trajectory that is challenging to measure.

"The measures we use today for child and adolescent health care are mostly process measures, and it’s not always clear that these correlate with the outcomes we want to see," said Dr. David Keller, professor of pediatrics at the University of Colorado and vice chair of clinical affairs and health care transformation at Children’s Hospital Colorado, Aurora.

Increasingly, however, experts involved in measuring child and adolescent health are pursuing common and consistent quality measures that, when used more widely, will facilitate more comparison and the collection of more data to be used in looking at long-term outcomes and population impact.

"It’s mind-boggling to see all the available [pediatric] measures ... all the indices and projects out there. Yet there’s little pooled data, and no common language – especially for routine, general pediatric primary care," said Judith S. Shaw, Ed.D., M.P.H., R.N., executive director of the Vermont Child Health Improvement Program (VCHIP).

Dr. Shaw’s organization is housed at the University of Vermont, in a small state with an advanced health information infrastructure and a history of multi-insurer payment reforms that support medical homes and community health teams. VCHIP has supported and led a string of quality improvement projects over the years, but "we realized we have no longitudinal database to look at what’s transpiring across the state," she explained.

Dr. Shaw recently led a team of pediatricians and other experts from across the state in culling through almost 200 pediatric quality measures developed by the National Committee for Quality Assurance (NCQA), Bright Futures, and an array of other state and national organizations and efforts. They narrowed the list to a core set of about 40 measures that VCHIP will monitor over the long term in an attempt to bring measures in line with each other, identify gaps and areas for refinement, and more easily inform improvements in child health and health care. Chart auditors will collect data from a random sample of charts from participating practices.

Over time, Dr. Shaw hopes to pool her state’s data with data from other states. VCHIP leads and advises a network of more than 15 states with similar Improvement Partnerships (IPs), or collaborative efforts to measure and improve children’s health care. The network provides an opportunity to create the large data pools that are important for pediatrics and measurement of long-term outcomes. First, though, measures will need to be streamlined.

Dr. Shaw and her colleagues recently surveyed the IPs in 12 states and found that, of 100 primary care measures described in the survey, only 8 measures were utilized by more than one state IP. Similarly, among three states measuring asthma care, only 2 of the 14 available measures were used by all three states. "There’s not a lot of overlap in the measures that state QI projects are using," and in the big picture, this is limiting, said Dr. Shaw, also associate professor of pediatrics at the University of Vermont, Burlington.

On a national level, development of the 2013 first "core set" of children’s health care quality measures for voluntary use by Medicaid and State Children’s Health Insurance Programs was driven by the desire to "get everyone [across states] aligned and to be able to make fair comparisons of children’s health across the country," said Sarah H. Scholle, Dr.P.H., vice president of research and analysis at NCQA.

A core set of children’s health care quality measures was mandated by the Children’s Health Insurance Program Reauthorization Act of 2009 (CHIPRA) and developed under the auspices of the Agency for Healthcare Research and Quality (AHRQ) and the Centers for Medicare & Medicaid Services (CMS). It is meant to be annually updated and enhanced – a 2014 version is now in use – and to be usable across different settings and by multiple stakeholders including payers, providers, and consumers. This isn’t an easy task: Many experts maintain that data can rarely be used for both accountability and quality improvement.

A CHIPRA-mandated $55-million Pediatric Quality Measures Program (PQMP) has been supporting seven Centers of Excellence in an effort to refine existing measures in both primary and specialty care and develop new ones that are more evidence based. "[Finally], there has been substantial federal investment for the design of pediatric measures to be used nationally," said Dr. Mark Schuster, who is William Berenberg professor of pediatrics at Harvard Medical School and the principal investigator for CHIPRA’s Center of Excellence for Pediatric Quality Measurement at Boston Children’s Hospital.

Thus far, it "has been hard to learn from each other ... To get a sense of what and how well your peer groups are doing," he said. "We want to be able to go to a hospital that’s consistently doing better on a certain measure and ask, ‘How are you doing this? Can you help us?’ "

Experts in quality measurement say that interest and efforts are increasingly focused on measures that address more than the structure and process of care, as well as on measures that address behavioral and mental health. The national Children’s Core Set includes three measures relating to mental and behavioral health.

NCQA’s Healthcare Effectiveness Data and Information Set (HEDIS) measures – which comprise a large portion of the Children’s Core Set – have "provided a basic floor of measures" for various stakeholders in the health care system by looking at factors such as immunization status and whether or not children receive well-child visits, for instance, "but they don’t say anything about what happens when the child is there or" how the child fares afterwards, said Dr. Charles Homer, chief executive officer of the National Initiative for Children’s Healthcare Quality in Boston, which assists practices, hospitals, and others in using data and measurements to improve.

"You can do very well on the HEDIS measures and still not deliver the kind of care you or I would want our children to get," he said. "As part of our efforts to improve quality, we need data-driven, higher-bar measures. ... I don\'t think we’re very far from adding some of these kinds of metrics."

NCQA’s Dr. Scholle doesn’t disagree. Like Dr. Schuster, she also is serving as a principal investigator for the PQMP. One of her main areas of focus has been the development of measures on adolescent depression care. "We need to see whether adolescents [who receive depression care] are actually improving, with a reduction in symptoms over time," she said.

Also of growing importance in pediatric quality measurement is what experts call the patient or family "experiences of care." Dr. Scholle, for one, can attest to the value of such input when it comes to measuring the quality of depression care. NCQA has been incorporating measurement of the patient/ family experience in some of its HEDIS measures, and Dr. Schuster and his colleagues in Boston have been testing a family experience of care measure in pediatric hospitals across the country.

"We’re not talking about patient satisfaction – we’re talking about the experience of care and the outcomes," said Christina Bethell, Ph.D., M.B.A., M.P.H., professor of pediatrics at the Oregon Health & Science University, Portland, and founding director of the Child and Adolescent Health Measurement Initiative, which focuses on developing patient- and family-centered measures of quality.

"We can’t ask patients about what kinds of lab tests they had, but we can ask them about their behaviors and their functioning and their outcome," she said. "Is the child going to school, for instance? Sleeping well?"

Growing use of electronic medical records and other technologies – as well as financial incentives – may help support the collection of such information. And, Dr. Bethell maintains, there is a wealth of information to tap into from national surveys like the National Survey of Children’s Health that can be used to support state and regional-level measurement and quality improvement.

Overall, "we have a lot of what we need right now to" better measure the quality of children’s health and health care. "We just have to use it better," she said, adding that the main supportive pillars of good quality measurement – policy, culture, technology, and financial incentives – "are coming together." That, she said, "is what’s really changing."