Healthcare’s Investment in Machine Learning and Artificial Intelligence

The “Rise of the Healthy Machines” sounded more like a screening of a sci-fi movie than a Medstartr Health 2.0 NYC event. But to this group of over 180 attendees, this title conjures images of “machines” that help us stay healthy, help doctors give better care and help researchers give us better answers.  Moderator Fard Johnmar, President of Enspektos  fielded questions from the audience to the expert panelists on the newest analytics and algorithms now embedded in health technology. Dialogues highlighted, on a personal level, the struggle healthcare is having with the hype of machine learning and artificial intelligence and the whether it is a “one-size fits all” solution.

A 30-second definition of machine learning and what may be its biggest benefit for healthcare was the first request of the panel.  Spencer Greenberg, Ph.D., CTO of UpLift volunteered and introduced machine learning as the study of how to make accurate predictions from data, at its simplest form.  He explained that there used to be a field peopled called AI, Artificial Intelligence, which included a lot of things.  Machine learning, being part of it, has done so well that usually when people say AI they mean Machine Learning.   Greenberg feels that where ever you can imagine being able to make a prediction in healthcare, perhaps about a patient outcome or on which doctor will prescribe which drug, machine learning can be incredibly valuable.

Skepticism about ML and AI Solutions: A Hype or Help?
A member of the audience posed her concern and shared her red flag moment when she asked a question to her “Alexa” and it gave a response completely out of context.   As a physician, the point she was making was that many of her patients aren’t predictable and don’t behave like an algorithm.  She feels that 95% of medicine probably fits normal curve but fears the outliers are probably going to do really badly.  Panelists commented on why the benefits of machine learning depend on the continued learning from more and more data, and how outliers should not be discounted but serve an important role in research.

“I use AI to find rare disease. Outliers. One in a million patient,” Fernando Schwartz, Ph.D., Chief Data Scientist at Prognos.AI  told the audience.  His company uses reams of lab data to uncover biomarkers that can predict disease onset at the earliest point in time.  And even though the biggest impact of AI may be on trying to scale up and model the masses, there is also huge value in identifying those outliers. He pointed out that pharma companies, working on new drugs for rare diseases, can be doing really well until they need to find the next new patients. “Alexa may be wrong, but if you have a rare disease it is very likely we will find you,” replied Schwartz.

“Machine Learning is definitely over hyped but it is an incredibly valuable set of technologies,” proclaimed Greenberg.  He has encountered many companies that may want to use machine learning, but not only don’t know what it is, or are applying it fruitlessly where it doesn’t apply.  He also explained that machine learning is a predictive science, not a causal science.  To predict whether a patient with a given set of conditions is more likely to get cancer, and with the right data, machine learning can do that.  A study of whether a patient will respond better to treatment X versus treatment Y is not in the realm of machine learning.   He noted that unfortunately, at times the application gets misapplied and it does get overhyped.   Greenberg clarified that a wonderful benefit of machine learning is the continuous “training” from data that occurs, and in a sense, Alexa gets smarter from making a misclassification error. “All the data that you are feeding Alexa, you are making ‘her’ smarter. Every year Amazon is working with Alexa, it gets better and better and better,” he imparted to the audience.

Plausible Solutions for Reducing Tech Interference
An observation from the audience put an interesting spin for a use case for machine learning.  She relayed that having attended market research studies on EHR/EMR usage, the negative feedback from physicians was extremely potent. A major sentiment was technology caused friction during the patient and physician visit.  Was there anything in the tech world that could address this problem? Several panelists shared their opinions on AI and tech solutions for reducing the burden clinicians have right now with the EMRs.

“AI for a purpose is a good way to put it.  What is AI for the EMR? What is AI going to do? ” questioned Schwartz.  He pointed out from a business perspective, you may have to start developing some ideas like Alexa where you can have a voice command.  From the investors’ point of view, he sees this type of AI lacking a strong business case.  Maybe AI will help us interact better with the system in the future, but he contends the problem may rest in demonstrating how to monetize the technology.

“It is not merely a technology problem, but doctors have to be willing to adopt technology to make EMRs more seamless,” interjected Gary Cheung, Sr. Software Architect from Cigna. From his perspective, the pain point is on transferring paper documents to EHRs and taking steps to move away from writing notes and records to an iPad when seeing patients.

Jeff Nadler, CTO of  Teledoc, had a pragmatic approach “There is friction in the patient-doctor relationship. I think we are going to see technology not eliminate that friction but make it less, like the voice recognition and natural language processing,” he stated unequivocally.  Nadler sees more of a complimentary adoption over time by both younger physicians, having grown up with smartphones and tablets, and the expansion of sophisticated technology and tools that will guide the physician through the visit.   Teledoc is leading with branching logic decision support permitting intuitive prefilled menus for speed and ease of exposition.

Companies like the new San Francisco-based Forward are taking on this communication challenge with its super hi-tech primary care system. “And one of the interesting things that they do,” Johnmar described, “is take natural language processing, basically taking speech, and making it understandable. Their patient and doctor consult technology essentially allows them to have a conversation without having to take notes into the EHR/EMR.”

Learn more about Medstartr Heath 2.0 NYC and please join us at our next event.

This article appeared in RCM Answers and has been reprinted with the permission of Answers Media Network LLC.

A Virtual Clip Board Pilot Study

A Mobile Healthcare Registration Process

By Sarianne Gruber
Twitter: @subtleimpact

Patient self-service kiosks in hospitals and physician offices have helped speed up the patient intake process, collect co-pays as well as improve data quality such errors and duplication of information. The versatility of the technology permits foreign language options, the signing of patient consent forms, the gathering of demographic, clinical and insurance information, even the administration of satisfaction questionnaires. The success of these mobile solutions, has prompted the Sullivan Institute for Healthcare Innovation, in collaboration with theWorkgroup for Electronic Data Interchange (WEDI), HIMSS and MGMA, to streamline the patient matching experience by improving interoperability and health information exchange through the new Virtual Clipboard.. The Virtual Clipboard would also eliminate the need for patients to fill out new forms by hand every time they visit their physician or the hospital, simplifying the patient intake process for patients and providers. Along with other advanced IT initiatives, improvements are place for the way basic patient data is created, stored, transmitted, and used in healthcare. The goal of the initiative is to automate the engagement model between patients, providers and payers with the incentive to improve business value to all.

This summer the solution definition and design document was released for Health Benefits and Health Record Mobile Solution (Virtual Clipboard Initiative) Pilot Study The document outlines the specifications for the pilot study includes functionality, scope as well as requirements and procedures. The initiative is being launched to define a set of industry standards for exchanging and securing healthcare information within mobile applications used by patients and / or their advocates. The expected outcome of this mobile and portal application is to improve healthcare efficiency, improve quality of care and empower consumers with healthcare decision making. The scope of Virtual Clipboard Pilot is in two phases to include both administrative and clinical information portions of the clipboard:

  • Pilot Phase 1: Demographics & Benefits Coverage
  • Pilot Phase 2: Medications and Allergies

“We expect the Virtual Clipboard Initiative to significantly improve the burdensome patient intake process – a critical and overlooked component of the healthcare delivery system,” said Devin Jopp, Ed.D, president and CEO of WEDI. “In an unprecedented collaborative effort, key stakeholders from across the healthcare continuum have come together to define initial standards for mobile healthcare applications. Leveraging the technology that many patients already use, the pilot seeks to demonstrate dramatic improvements to the healthcare registration process.”

The first phase of the Virtual Clipboard Initiative pilot will be to facilitate the automated collection of patient health insurance and demographic information. The ultimate goal of this project is to set standards, integration points and security that will add value to the entire healthcare delivery system. Through this process, it anticipated to see better quality of care, improved patient safety and reduced administrative costs. “We are very excited to see the Virtual Clipboard project move into the pilot phase,” stated Robert Tennant, director of health information technology policy for the Medical Group Management Association. “Once implemented, this automated approach to patient intake and data transmission will significantly improve administrative efficiency- while at same time increasing patient satisfaction. By bringing together a powerful alliance of patient advocates, providers, health plans and vendors, the Sullivan Institute, along with WEDI, are forging a clear pathway forward to better patient care.”

In 1993, the original WEDI report brought together public and private industry to develop a roadmap for healthcare information exchange. The impetus for the project began with the release of the 2013 WEDI Report. Recommendations were sited on leveraging smart technologies to dramatically enhance the patient experience while improving patient safety. The Sullivan Institute and its partners having been working since to create a solution blueprint for a “smart” mobile solution that will ease the patient check-in process by automating demographic, insurance and critical clinical information (e.g. allergies, medications and lab results) elements. Learn more on the Virtual Clipboard at on the WEDI website.

This article was originally posted on, August 18, 2015.

The Science of Population Health: Epidemiology

By Sarianne Gruber
Twitter: @subtleimpact

Having once been a graduate student in Epidemiology and Public Health, certain books still remain my bookshelf as iconic references for studying disease and our healthcare system. There is the bright orange, soft covered, Foundations of Epidemiology by the father and son team Abraham M. Lilienfeld and David E. Lilienfeld, a relic from the required reading list. Reviewing chapters on retrospective, cross-sectional and prospective studies with classification tables and relative-risk calculations, all has a vague familiarity to reading a Girl Scout handbook before going on an overnight camping trip. The study design you select, the sample size you determine, the conceptual hypothesis for the study plus the rationale for conducting the research, the selection of subjects and the measure of disease status, all demands a well-thought out plan and creates a distinct anatomy of the epidemiological study. Each study type is consistent with specific conditions otherwise the associations between variables could be spurious. There is a lot of value to understanding the basic concepts, principles and methods of population-based epidemiologic research within the framework of managing patient panels, communities of patients and preventing disease.

An Iconic Population Health Study: The Framingham Heart Study
In 1948, the Framingham Heart Study, a cornerstone study of Population Health, was initiated by the National Heart Institute and Boston University. Participants included 5,209 men and women between the ages of 30 and 62 from the town of Framingham, Massachusetts. All the volunteers had baseline physical examinations and lifestyle interviews. A follow-up visit would take place every two years, where detailed medical histories, physical examinations and lab tests were taken. In 1971, a second generation of 5,124 of the original participates’ adult children and their spouses joined the study. Then in 1994, a new study commenced with a more diversified population of participated. It has become a prototype for other epidemiological studies around the world; it is among the most cited references in medical literature. Over the years, the findings have led to the identification of risk factors including high blood pressure, high cholesterol, smoking, obesity, diabetes and physical inactivity.

Starting with the Physician Practice: Taking on Population Health
Dr. Thomas Royle (Roy) Dawber, the legendary founder of the Framingham Heart Study, was head of his time ,a pioneer in the practice of population health management. He was not a traditional epidemiologist; his primary interested was in providing information that could be directly useful to –prevention-minded doctors in practice. He viewed epidemiology clinical investigation on a community level. The citizens of Framingham volunteered in the study for over more than five decades. Hard to imagine, but this study was conducted without the use electronic health records and electronic data warehouses. The success of the Framingham Study was in gathering information on the prevalence, incidence, clinical manifestations and predisposing risk factors that could lead to heart disease and cardiovascular disease. Expanding and beginning programs designed for population health surveillance for CHD, diabetes and other chronic disease risk factors. The Heart of New Ulm Project at the Minneapolis Heart Institute is a large 10-year intervention designed to reduce the incidence of myocardial infarction and improve heart disease. Dr. Dawber passed ten years ago in 2005. Yet, today’s EHRs, longitudinal records, are a documentation tool for individual patient-provider encounters. The potential exists to serve as a tool for managing the health of communities is just starting to be realized with more epidemiological studies in the future.


Did You Take Your Pills Today? Keeping Healthy Through Communication

Reducing ReadmissionsBy Sarianne Gruber
Twitter: @subtleimpact

What process is used to communicate the medication a patient should be taking? How do you verify that they understand the instructions? Can a healthcare professional provide the most effective communication when educating the Congestive Heart Failure (CHF) patient on medication? The medical and administrative teams at St. Mary’s Hospital in Amsterdam, New York wanted to know the answers to these questions with the objective to reduce CHF readmissions. In many cases, the reason for the readmission is medication non-compliance. One patient will find it easy to “take a pill” daily while another patient is unable to comply the treatment. What can you exactly say to the CHF patient to motivate adherence to the specific medication plan? The research team from iNovum developed a segmentation solution and better yet, helped reduce CHF readmissions. Here is the how the science behind the study works.

A preference-based conjoint analysis was used to define patient attitudes and preferences. The goal was to determine which messages would convince a patient to adhere to a medication plan. First, consider the major silos related to taking medication. Step two, select six classification silos: (1) Education, (2) Cost, (3) Reminders, (4) Quality of Life, (5) Belief and (6) Trust. Step three, create messages as the stand alone pieces of information that communicate attributes and descriptors for taking your medication. Listed in Table 1 are messages related to a Patient’s Belief about Taking Medications.

Table 1. Silo E: Belief about Taking Medications

Silo E: Belief
E1 You believe that taking your medication helps your condition
E2 You believe that your medication is safe
E3 You believe that the side effects of your medication are manageable
E4 You believe that the side effects of your medication are tolerable
E5 You believe that your medications do not interfere with each other
E6 You believe that your condition warrants taking your medication

A total of 226 patients participated via the internet. The respondents were tested on a variety of Concept Screens. The study had 6 categories (noted above) and 6 messages per category allowing for 36 unique combinations. A study patient is shown 48 concept screens. For example, the screen below shows 4 messages from 4 categories. These concepts were developed by a systematic combination of the messages via an experimental design and are comprised of 2-4 messages. No two messages from the same silo can appear on the screen at the same time. Messages must appear independently so that the messages are statistically independent of each other so that a regression analysis can be done. For this study, IdeaMap®NET, a patented proprietary optimizer, pulled the viewed messages. Table 2 is the Positioning Page that sets the framework for the study. Table 3 is a Study Sample Screen with a vignette to get the patient’s reaction.

Table 2. Positioning Page

Sarianne #1


Table 3. Sample Survey Screen


Sarianne #2

Concept interest scores show the reaction of the patients to the different messages. Concept scores for sample of the 36 messages are shown in Table 4 for the total panel. Here we see a few messages drive compliance on the total sample. Most messages have no effect. And the purple messages should be avoided. The next step is to take a closer look at the sample by using segmentation. Table 5 shows that certain messages are have better appeal within different segments, and that patients’ personal preference perspective is not homogeneous.

Table 4. A Portion of All Tested Messages

Sarianne #3


Table 5. Segment 1 – It’s My Decision – Key message: “You have the information and trust what you were told”

Sarianne #4

The research at St. Mary’s Hospital uncovered three segments that differentiate patients’ compliance. Segment 1 patients (It’s My Decision) want to retain control and when following instructions. Segment 2 patients (“It Takes a Support Structure”) want to feel that they are not going through this alone and need to have support to motivate them to take their medication. Segment 3 patients (“Faith in the Medical Field”) trust in their medical team given their prior relationships. New patient segment classification is done based on a five question Typing Tool. The CHF medication compliance tool will identify to which Viewpoint Group a patient belongs and subsequently, what programs and messages work best with that patient to help them achieve medical compliance. And most importantly, St. Mary’s Hospital has started to see a decline in readmissions. In 2012, out of a total of 262 non-Inovum patients admitted, 45 patients readmitted within 30 days (17.04%). Compared to 116 inovum segmented patients from February 2013 through June 2013, only 5 patients readmitted within 30 days (3.38%). To cite a staff testimony,” I started the survey with patients, followed the instructions, gave out the handouts, but then something changed”.

This article was originally published on and is republished here with permission.

Extend, Flatten, Jump. A Point in Time Changes the Data Story

Extend, Flatten, Jump.  Sounds like diving instructions or some other sports coaching.  But that is not what I am writing about.   Instead it is a way of thinking about graphs, and how ‘Extend, Flatten, Jump’ can affect how we make decisions about quality.    Much of quality work uses time series.  A time series is a list of numbers where each number is labeled with a unique time value, whether in seconds, minutes, days or years.

In medical quality, the time series used are often counts of undesirable events in a given time period. Examples are falls, outbreaks, device malfunctions, or relapses.  Time series can also show averages of clinical measurements such as cell counts, chemical analytes or tumor size across several people.   Although the time series might represent several events, it is still one number with one time point (1).    Time series are typically shown as line graphs, with the time on the x axis and the result on the y axis.   Adding more time points, ‘extend’, can make the recent events look less severe.   Increasing the range of the y-axis to ‘flatten’ the graph can make things look less noisy.   And a sudden jump in values near the end can affect perception of the whole series, even when not much has changed.  And perception can affect actions. Let’s say we have 20 weeks of event counts, (pick your adverse event) starting with the first Monday of this year.  The order of the dates gives a position to each event where the count of events during the week of 2014-01-06 has position 1, the 2014-01-13 has position 2, through position 20.

count of events 2


If we are interested in the last 10 weeks, we can graph as follows.  I will use numbers on the x-axis in order to avoid clutter.  I use the range for count as 0 to 16, with a tick mark every 4 units.

main graph 2


Extending back to position 1, and compressing the data in a graph of the same width looks as follows: extend graph 2 What does adding prior data give to your perception of how well this clinic is doing in managing the adverse event?


The following graph shows the last 10 points but note that the y-axis is extended by 50%.  This flattens out the results. flatten graph 2 The maximum of the time series, 16,  at week 14 looks less severe but it is the same data.


jump graph 2

 This is a situation that can make managers jump.   We were doing so well, and then what happened last week?  The truth is that these are 20 simulated numbers from a Poisson distribution (2) with a mean of 10 and a sample mean, the average of the 20 numbers, of 10.1.    They are all independent of each other.  While often in time series this isn’t the case, it is a good place to start.

I have taken inspiration for this post from an article in the Spring 2014 issue of CHANCE, a statistics magazine, called “Your Textbook Can’t Help You Here.” (Michael Lopez, Adrian Esparza, Michael Lavine, Jenna Marquard.  Professor Jenna Marquard of the University of Massachusetts School of Engineering researches decision-making in health information technology.

Georgette Asherman July 16, 2014

(1) Longitudinal studies track a group of subjects at several time points.     In longitudinal studies the major interest is how different subjects change at each measurement.  In time series the major interest is the pattern of the series over at least 8 time points.

(2) It is common to use the Poisson distribution to fit count data.   It provides integer values to describe a count of events in a give unit of time or space.   The mean and variance are equal so a higher average count will have more spread.

Draw the Line: The straight and narrow of using regression lines.

I have four books about linear least square regression models. I have no interest in writing another one. But I find that at whatever level, none step back and explain the underlying philosophy for this horribly misnamed[1], powerful but deceptive technique. It is worthwhile to think of some basic concepts before getting overwhelmed in complicated equations and model assessments.
Read more