The American Academy of Pediatrics launched its program to review and critique practice management software for pediatric practice environments during the Spring Session in San Diego. The Practice Management Software Review Program (PMSRP) was first proposed by the AAP Task Force on Medical Informatics and is being implemented by the Council on Pediatric Practice, drawing experts in computers and practice administration from SCOT and SOAPM.
Twenty-one pediatricians attended a six-hour training session during the Spring Session of the American Academy of Pediatrics to become PMSRP reviewers. The training session was led by James V. Lustig, MD, former chair of the AAP Task Force on Medical Informatics (1995-1996), former member of the Committee on Practice and Ambulatory Medicine and charter member of the Section on Computers and Other Technology.
Timothy V. Tincknell, MA, MBA, principal staff to the PMSRP, presented an overview of the program, its objectives, and three levels of review. The PMSRP is to be a standardized review of practice management software; it is not to be an endorsement. Only those programs dealing with practice management would be eligible for the PMSRP (software related to professional or patient education may be eligible for other review programs hosted in either the AAP Department of Education or Division of Public Education).
According to preliminary data from a recent AAP Periodic Survey of the 90% of the respondents that have computers in their office, less than 50% use them to submit claims for federal programs (ie, CHAMPUS), state programs (ie, Medicaid), or private third-party payers. With HIPAA and the HHS pushing new deadlines for the implementation of various informatics standards, many pediatricians will need to re-tool their practice information systems. PMSRP will provide them with the confidence they need in determining how the strengths and weaknesses of various practice management software products may affect their medical office.
Dr. Lustig presented the evaluation tool that had been drafted and reviewed by a workgroup prior to the meeting. The workgroup included members of SOAPM and SCOT (Douglas Stetson, MD, Donald Lighter, MD, Thomas Selva, MD, Ed Gotlieb, MD and James Lustig, MD). Roger Suchyta, MD also reviewed and critiqued the evaluation document. It was suggested that the evaluation tool be used in conjunction with a "Minimal Data Set" of those essential elements of a practice management software program. The SCOT policy statement still under development, "The Minimal Record Set for Electronic Transfer of Computerized Medical Records," would prove useful in this regard.
The participants were divided into small groups and each assigned a portion of the evaluation tool to critique. Each group presented its recommended changes for each section. It was decided that these comments would be collated into the evaluation document and disseminated to the 21 participants via e-mail for additional comments. Each participant was asked to indicate a particular area of expertise in which to focus their efforts but that all could comment on other areas as well. Dr. Lustig volunteered to synthesize the comments into a second generation of the document. Comments from PMSRP participants would be returned to Tim Tincknell in the Division of Pediatric Practice and incorporated into the third generation of the document. The final product would be a spreadsheet version of the evaluation tool to facilitate the scoring process.
Two sample software programs were presented and participants were asked to evaluate each one using the appropriate elements of the evaluation tool. The purpose of this exercise was to assess the evaluation tool and build participant familiarity with the review process. Half of the participants were asked to conduct their evaluations individually. The other half were asked to evaluate each product as a group. Unfortunately, time did not permit a thorough evaluation of either product, but participants were able to comment on the relative utility of the evaluation tool.
It was suggested that the Academy subsidize reviewer attendance at PMSRP training sessions in light of the potential profitability of this project. Under the description of the PMSRP, reviewers will receive an honorarium commensurate with the level of review for each review they complete. In addition, all PMSRP reviewers will receive complimentary copies of PMSRP evaluations (the narrative and the annual summary card) as a courtesy for serving as reviewers. As one of the few volunteer activities with honoraria in the AAP, both the COPP and ACBOPP, at their May 1997 meetings voted against subsidizing the travel/hotel expenses for PMSRP reviewers to attend training sessions.
The group debated the appropriateness of pediatricians reviewing various "financial" and "administrative" software and discussed the possibility of having office managers or non-pediatricians perform those reviews for the PMSRP. Staff reminded the participants that the PMSRP is designed to be software reviews "by pediatricians for pediatricians" and that other sources of reviews were already available through a variety of publications. Several participants indicated that they would feel very comfortable reviewing "billing" and "accounting" software by virtue of their MBA and/or experience. At the May 1997 meetings of the Council on Pediatric Practice and Advisory Committee to the Board on Pediatric Practice, each group confirmed that reviewer status be limited to members of the Academy by virtue of the uniqueness of the program's original intent: "by pediatricians for pediatricians."
Pediatricians involved in the development of practice management software or with financial ties to practice management software vendors will not be eligible to serve as PMSRP reviewers. They are welcome to attend PMSRP sessions and contribute their expertise to discussions regarding the development of our review process; however, they will not be allowed to review either their own or other software products. Their exclusion as reviewers stems from a need to remove any bias or appearance of prejudice that could be alleged by having a potential competitor review another software product. It was suggested that this narrow restriction may deprive PMSRP of the benefit of some of the most expert pediatricians. The Council on Pediatric Practice, as well as the Advisory Committee to the Board on Pediatric Practice considered this "conflict of interest" issue at their May 1997 meetings and concurred with the above ruling. (Please note: Regarding "financial ties" to software vendors, holding a very small percentage of stock in a large publicly traded corporation would NOT disqualify an individual from being a reviewer.)
Participants were asked to complete an evaluation of the training session. The overall session was rated "excellent" by 21% of respondents, "good" by 50%, and "satisfactory" by 29%. The highest percentages of "excellent" ratings went to Meeting Location at 64%, Room Set-up at 79%, and Assistance of Academy Staff at 64%. Although the evaluation tool was in first draft format, it managed to rate "good to excellent" with 40% of respondents. Due to the quantity of work left to complete the evaluation tool, as well as fully master the review process, 93% of respondents stated that they were willing to attend a PMSRP update training session at the 1997 Annual Meeting in New Orleans.
Dr. Lewis Wasserman agreed to disseminate the revised evaluation tool via his electronic mailing list and Dr. Julius Edlavitch offered to designate free time on his international pediatric chat for discussion of PMSRP issues.
Because of the need to continue this program on its "fast track" and resolve several pending issues (conflict of interest, finalization of the evaluation tool, scoring method, user survey, and subsidized participation), it was suggested that a follow-up PMSRP training session be held at the Annual Meeting in New Orleans. The COPP and ACBOPP concurred that a second training session at the 1997 Annual Meeting is necessary. The PMSRP update training session will be held on Sunday, November 2, 1997 from 11:30 am - 5:30 pm. The location will be announced in a few weeks.
To ensure the most effective and efficient use of Academy resources, it was agreed that the PMSRP update training session would be primarily limited to those individuals (21) that attended the initial training session in San Diego. The New Orleans update training session will be limited to 25 reviewers. Thus, a minimum of four pediatricians off a waiting list compiled from the first training session will be contacted as to their availability for the November 2nd meeting. Those knowingly not on the current waiting list that wish to have their name added should contact Tim Tincknell, within the Division of Pediatric Practice, at 800/433-9016 x4089 or via email at
If you have a practice management software product you would like to see included in the Academy's Practice Management Software Review Program, please contact Tim Tincknell, within the Division of Pediatric Practice, at 800/433-9016 x4089 or via email at
Thanks to All Spring Session PMSRP Attendees