How to test-drive medical software

Whether you're buying an electronic medical record or a practice management system, you must prepare for this moment of truth. We've collected advice from computer experts as well as medical practices that have kicked some software tires.

Identify deal breakers. If you're in the market for a car, a model with heated front seats or eucalyptus wood trim on the dash might impress you. But what about the gas mileage? Likewise, in evaluating software, you must distinguish between features that would be nice to have and those that you must have—what consultant Rosemarie Nelson in Syracuse, NY, calls deal breakers.

Deal breakers vary from practice to practice. In an EMR, they might be the ability to write quickie electronic prescriptions, pull in data from diagnostic equipment like an ECG machine, or track whether orders for lab tests and diagnostic imaging actually get performed. Deal breakers for practice management systems might include tools to double-check diagnostic and CPT coding or flag insurance payments that fall short of contractual amounts.

Once you know your deal breakers, says Nelson, list them in a "demo report card," with a separate column for each system you evaluate. Grade each system on how well it handles each deal breaker, using a scale of 1 to 3—3 being tops—and jot down your observations.

Develop a patient-visit scenario. If you're buying an EMR, you'll want to see how a system helps you chart a patient visit from start to finish. Accordingly, create a hypothetical, representative visit and use it in each demonstration so you can make apples-to-apples comparisons, says Nelson. "A good scenario for a primary care doctor would be a patient with two diagnoses—diabetes and hypertension—who needs two prescriptions. Scenarios suggested by the vendor might be simpler and therefore less helpful."

Get a live demo. Before the Internet, live software demonstrations had to be given in person. Now we're in the Webinar era—you can go online and watch a salesperson mouse through a software program while you talk with him on the phone. Webinars have their pluses, including the ease of hooking up doctors in multiple offices any time of the day. Some doctors rely on Webinars exclusively, but experts say live demonstrations are still indispensable.

"In most Webinars, you're watching someone else use the software," says Nelson. "In a live demonstration, you can take over the controls and get a better feel for the program."

Mark Johnson, president of MediNetwork, an IT firm in Dallas, cites a subtler reason for being in the same room with the salesperson. "With a Webinar, you can't look someone in the eye," says Johnson.

Schedule wisely. You'll need at least two hours to put an EMR or practice management program through its paces. And those should be distraction-free hours. If the salesperson is conducting the demonstration at your office, carve out the time by opening several hours later than usual or closing several hours earlier.

Another option is arranging private demonstrations at a healthcare IT conference such as Toward an Electronic Patient Record (TEPR). All the major vendors will be there, and you'll escape your office routine. Anchor Health Centers, a group practice in Naples, FL, did just that at TEPR last May. A contingent of physicians, nurses, and administrators along with consultant Rosemarie Nelson tested six EMR systems in two days. "It helped us focus," says internist Rick Francis. "We felt like we were on a retreat."

Make the vendor show you. At this stage of the buying game, you've probably already heard the vendor's standard sales pitch. You don't need to hear it again during the demonstration. Take a cue from the Anchor Health EMR team—it asked one embarrassed company exec to turn off a lengthy PowerPoint presentation and get to the main event.

Likewise, you must be assertive in your questioning. When you ask whether the system can perform a particular function—say, view three doctors' appointment schedules simultaneously to find the earliest open slot—you must insist that the salesperson make the system do it in front of you. Otherwise, he or she will invariably respond with the notorious "Yes, it can"—or the mushier "Yes, it could"—and move on.

"When the salesperson says, 'Yes, it can,' he may mean, 'Yes, it can, if you pay extra for custom programming,' or 'Yes, it can, when we bring out a new version this fall,' " says Susan Jones, a healthcare IT expert in Lebanon, IN. Hence the expression "vaporware."

Count the clicks. Besides finding out what a system can actually do, you want to evaluate its ease of use. Does the screen overwhelm you with text? Does it take eight mouse clicks to write a prescription when another system requires only four? Do you have to flip through three screens to register a new patient?

Visit a practice that uses the software. A sales demonstration of medical software provides a narrow picture of its capability. With an EMR, you're limited to hypothetical patients. And you normally can't try out electronic claims and remittance on a demo version of practice management software since that requires sending data to an insurer, notes Susan Jones.

It's imperative, then, to visit a practice that's running the software you might buy. "Some people think you should conduct a practice visit for each of your top three vendors," says Mark Johnson. "That's easy if you live in a big city and references are 15 minutes away. Another school of thought says select your vendor and then do the site visit to confirm your decision. The latter route makes sense for doctors in isolated areas who can't afford too many fact-finding trips."

Find a practice that roughly matches yours in specialty and size. Your fact-finding team should include staffers who would use the system—billers, receptionists, nurses. Each employee should talk to counterparts at the other practice to find out how the system performs for them in real life. And go prepared with questions. (You can find helpful questions in How to Select and Implement the Right Computer Solution for your Practice, co-authored by Susan Jones and available for $67.50 at www.medicalbusinesspublishing.com.)

Shake off the escorts. You probably won't receive candid appraisals from the folks you're interviewing if salespeople are hovering nearby, so insist on private face time.

Walk away if you witness significant problems. A few complaints or software glitches encountered during a site visit shouldn't necessarily rule out a vendor, notes Mark Johnson. "Whether you buy a car, a house, or software, nothing will be perfect," he says. "That's all the more reason to know what features and capabilities are your top priorities."

You may learn enough on a site visit to kill a deal, though. "You have to assume that these are their happiest clients," says Johnson. "It won't get any better than this. If there are serious problems, that means something."

You're better off experiencing a dismal moment of truth before you sign a contract. The salesperson may say, "Yes, it can," but your reply should be "No, I won't."

 

Robert Lowes. Technology Consult: How to test-drive medical software. Medical Economics Sep. 3, 2004;81:17.