Real World Evidence: Using In-Office Dialogues to Understand Shared Decision-Making

Editor’s Note: In the fall of 2023, GreenBook’s IIEX Health event took place in Philadelphia, bringing both useful and inspiration content to insights and analytics professionals spanning the healthcare, pharmaceutical, medical, and wellness industries. Attendees found the content so valuable that we wanted to make much of it available to all who could not attend this in-person event. Before even reading this post, know this: You can view all the sessions on-demand now!

If you aren’t in those industries … how might you apply the learning within your own? At GreenBook, we believe that IIEX is more than a conference series. It’s a mindset. These are the forums in which the most important insights innovations are revealed, demonstrated, debated, and championed. What starts at the events drive change in our world. It is in that spirit that we bring you, directly, some of the poignant content we heard at IIEX Health. We continue this series with a session from the Manager of Strategy & Insights at Verilogue, Zach Hebert.

---

Enjoy our On-Demand Video

The Covid-19 pandemic and vaccines have been, and are still, a point of concern for many people all over the country. When it comes to childhood vaccines, vaccine hesitancy among parents is on the rise, doubling over the last several years to reach approximately 18% for pediatric vaccines. Delivering the right message has never been more important. Click to view the video (courtesy of Civicom).

 

View more 2023 IIEX Health content on-demand!

Whether you were able to attend, or you were not, join us online to see what was shared by some of the biggest brands, the newest startups, and expert-level researchers across healthcare, pharma, and consumer experience. Here’s just two of the amazing sessions you’ll find on-demand:

• Dr. Katelyn Jetelina speaks to the process she created for translating the complex science of epidemiology into plain English and communicating it in such a way that informed and empowered individuals across the globe.
• Greg Hewitt and James Bauler speak to the innovation process behind Fuse Oncology, a spin-out of Cone Health, after a critical examination of the lag between a patient’s diagnosis and start of treatment.

Online you’ll find other fantastic sessions by speakers from Pierre Fabre Group, Novartis, Hinge Health, and more! If you want to stay on top of the trends in the healthcare industry — one of the largest spends in market research — you won’t want to miss IIEX Health On-Demand!

Not familiar with the Insight Innovation Exchange (IIEX)?

Ten years ago, GreenBook embarked on a simple idea: Could we create opportunities for market research leaders to share ideas and collaborate to define the future of insights?

If there was something new to our industry — a company, methodology, or platform — that didn’t exist 10 years ago and is now considered a “best practice” … well, you probably saw it first at an IIEX event.

What starts here will change our world!

---

Transcript

(Transcript courtesy of TranscriptWing)

Female: Okay. So, our next talk is called Real World Evidence: Using in-office dialogues to  understand shared decision-making. Zach Hebert who is the manager of linguistic insights and analytics at Verilogue is going to be illustrating how unique in-office conversation data can be used as a real-world evidence to support a wide range of  research goals. Welcome Zach.  

Zach Hebert: Okay. My turn to look for the clicker. Okay, great. I assume green means go.  Perfect. Yes. So, thanks everyone. So, I’ll start off before I start going through  some of these slides by talking a little bit about myself. My name is Zach Hebert. I  am a manager over at Verilogue. Some background on me, I never really expected  to find myself in market research. I imagine that might be the case for many  people, but my specific path came from academic linguistics and anthropology. So, if you all want to talk about sign language phonology, about grammar, about  running some fun perceptual studies that hook up to people’s brains after this, we  can do that too. If you want to talk about culture and identity, come to me or  really anyone here. [Laughter] 

So, anyway, let’s start moving through these. It’s a nice slogan. We’ll keep going.  Okay. So, I imagine some folks in here are familiar with Verilogue. For those of you  that are not, one of our I guess biggest treasures is our collection of data.  Verilogue has the largest collection of in-office conversations audio recordings with  transcription of those. Right now, I assume it’s somewhere over 200,000. It goes  back a little over a decade now across lots of different specialties with lots of  different conditions, lots of different sort of moments in our lives and patient’s  lives. We have a team of linguists and this is maybe the other bigger, more  valuable, I think treasure at Verilogue that explore these conversations. This  dialogue data both from qualitative and sort of quant light sometimes modes to  look at how we use language in the exam room to relate accomplished tasks and  also kind of a little bit of like how social categories emerge for our client teams. We tend not to say those kinds of things because those are really loosey goosey scary semiotic terms. So, we talk about things that work and what they do and yadda-yadda, but that’s how I like to think about what I do.  

So, yes, we jump into these real-world conversations on the lived experience of healthcare stakeholders. So, for those of you who saw the title, we’re going to be talking more about like outcomes and real-world evidence sorts of stuff rather than maybe the more traditional market research and sites kind of lens. So, part of this presentation is sort of me if I don’t sound excited is because I’m kind of nervous. I see the previous speakers also saying things about first time talking in front of a bunch of people in person since the pandemic, but excited to share that like we’re starting to break into this kind of area of research as well and it’s really great. I’ll say, personally, it’s kind of nurturing the scientist in me that sort of had to take a backseat sometimes to somebody who works with marketers and consults and has a really diverse audience that he communicates with.  

Anyway, our work in this kind of area, Verilogue’s built on four key pillars. Of course, there’s the conversations. We also do other kinds of qualitative research sort of the traditional interviews, journeys, that kind of thing. This is kind of a recapitulation of some stuff as I said earlier by identifying needs that patients have things across their disease journey. What I’m going to be telling you the story of today, specifically, is kind of a hybrid. It’s a little bit of a case study. It’s a little bit of ongoing journey and it’s a little bit of best practices and what we’ve learned so far. Both coming in as primarily a market research team but also as a team that has sort of unique data and I’ll also take a moment there to say that like we’ve said the word conversation a lot today. I think that’s really exciting. It can mean lots of different things, right? So, just to be sure that I place our data correctly and looking at how people talk to one another or express themselves out in the world. What we’re looking at really is recordings of a person goes to their PCP for example. They do a typical consenting process and all that but microphone you know starts recording and they go about their visit.

So, how do we get contacted with the client that I’m going to be talking about? This is the GSK value evidence and outcomes team. It was kind of a typical thing, so not terribly interesting there. We initially met with them in October 2019 to talk about Verilogue in this kind of sales BD meeting, right? The team said that they do their own sort of patient-centric outcomes research then it feeds into a lot of the clinical trial design. They act as a sort of bridge between the real-world data, analytics, the R&D, market access and everyone.

At the first meeting, they saw our data they thought maybe this could be used to cover some questions that we really haven’t been able to get into because it’s hard to get into what’s going on, what’s actually happening there in an exam room. So, initially they wanted to look at treatment decision-making and what role patients in particular had in the process. This is with a team that was focused on the Nucala brand. For those of you who don’t know about Nucala, it’s a biologic for severe asthma and this research project kind of expanded scope to just decision-making around biologics more generally. From the beginning, they were clear that they were interested in this research becoming kind of publicly visible which is great because it means I get to talk about it with you all, which is another little bonus of working with some of these teams.  

So, about getting to the point of doing the research. So, this is where things might start to get a little less typical, at least, for my experience with sort of insights teams and this might be the case for many of you. We had to develop a protocol, right? It wasn’t just as easy as saying we’re going to do some interviews. This is what we call the interviews. The protocol process is weeks. It can be months long.  It involves a lot of back and forth and thorough documentation of your data collection methods, the reasons for them how you’re defining variables, how you’re going to go about analyzing them, why the heck are you even doing this to begin  with, right? What is the scientific value of doing this kind of a project? So, we did develop this protocol in collaboration with the video team, submitted it to the protocol review committee and that led to a sort of formal meeting anybody who’s done an IRB. Kind of you’ll know what this is like, basically was that – unsurprisingly, they encountered our data and this happens all the time. I’m like, well, now wait a minute, this is kind of weird. We had to answer a few more questions about how we protect people’s identity, that kind of thing but they wanted to know as well about like our small sample size and how we could still make use out of that which for anybody who does primarily coal work. Well, no that’s kind of your eternal battle of the fact that stories actually do matter and you can get a lot out of a little. Anyway, we got approval. So, that was great.  

Let’s keep moving. So, about the actual research. So, the overall research objective was to look at decision-making points in the severe asthma journey. To do this, we selected 50 conversations between severe asthma patients and their pulmonologist, allergist or primary care physicians. This took place between September of 2018 and August of 2020. So, we did the analysis of those after that. I’m not going to go too, too far into like the selection process and the analysis process mostly because I want this more to be about what we learned about what’s different doing this kind of research. You will have an opportunity to kind of ask me more about it and there’s a QR code in there where you can scan to learn more.  

We broke these 50 conversations up into three different points and the journey. First, there was a consideration of a biologic therapy, so pre-initiation and these were crucially with patients who were naive to biologics. Ideally, completely naive but certainly naive to using biologics for treating severe asthma. Then we had some three months after their initiation of biologic, see how it was going, how they talked about their symptoms experience of their asthma and of the treatment. Also 20 conversations where the patient and doctor entertained the possibility of switching treatment or stopping the current treatment.  

Some things that we coded, just an example of like what we were looking at here, were share of the conversation. I’m happy to talk about what that exactly means in the Q&A, but there are a lot of ways to measure the idea of share. The big ones are time. I’m more of a fan of grammatical units like words, but – okay. Topic discussion length assessment style like the doctors asked kind of an open-ended questions. Is there an apparent questionnaire that’s being used? Is there an intake form being referenced? Then, of course, what is in the EEO outcomes.  

So, some key findings, we saw that when starting a biologic patient had a kind of passive role in establishing treatment goals. Some cute quote examples here. Instead, the initial goal setting was driven mostly by doctors and it relied on patient some responses to doctors, closed assessment. So, if the doctor asked the patient has a symptom patient confirms or affirms that they have that the doctor might mention that when talking about what a treatment does. Kind of a nice example of that is this little bottom half here where the doctors have to deal with this problem. There are several injectable medicines, most work regularly relatively okay. Most have reduced their asthma symptoms like coughing and wheezing, which kind of counts as a goal. A few other interesting findings. Doctors drove that initial goal setting but the decision on whether to stay on therapy was found to be a lot more patient driven about patient’s kind of pushing the issue of their symptoms continuing to be unacceptable and the doctor putting the option of switching in the ball in the patient’s court there. In particular, it was patients’ characterizations of their symptoms. So, for example this patient here saying  “Monday night, I couldn’t sleep, I was coughing and wheezing so bad on the doctor later suggest switching.” 

So, like I said, I don’t want to talk too much about the findings there because it’s more about doing this kind of work. Another key difference between this and some of the market research stuff is how the findings are getting disseminated specifically like that they’re getting disseminated. It can go live outside of the pharma company instead of going to some other internal business units and as a researcher can’t even kind of see the life of my work after I sort of complete it. So, our DSK partners were the ones who were determining kind of where they want to submit this thankfully because I am not a doctor of medicine certainly. Yes, so in this case, he sort of – thing that was up for consideration was still we went back to logistics the dates versus the conference fit. We wanted to submit the American thoracic society, but it was a little too late to make it so although all of our data was us based, we ended up submitting to ERS. Processes will vary I think from partner to partner on how this works, but in this case both Verilogue and GSK worked with Fishawack as a medical writing service. So, this was really exciting for me because my conference experience was kind of DIY. I made my own poster and my own everything. So, this meant that while Fishawack works from the original report that I was part of writing which was a manuscript itself. Our role in prepping the conference and subsequent submissions for journal publication is much more editorial which is so nice that I can say, “I think this needs to happen” and it goes out and does that. Yes, thank you to the official people. They were quite – and are quite lovely to work with.  

So, yes, like I said we submitted to ERS, 2021. It was going to be in Barcelona and we had a pandemic and so, it was much closer to home and online only unfortunately. This is the poster of our findings presented actually by one of our GSK partners Ember Lu. For anybody who actually wants to look at the poster, that’s – that is a QR code, it’s a cool looking one, but it’s a QR code, it’ll bring you to our website. It has a larger version of it along with an audio recording of Ember presenting the work. I see some people trying to take the picture, so sit and if it’s not working come find me. Great. 

So, I’m going to start to wrap up and this is more just thoughts here. This story is not finished, right? So, what we’re doing right now is we’re in manuscript prep mode for people who especially are from the softer social sciences. Know the publication process is arduous and each person that you add to the authorship team sort of compounds that. So, we’re on draft. I’m not sure which but we are targeting the Journal of Asthma right now for this paper. So, some considerations that I think are more relevant to everyone and some of these are going to be truisms I think, but partnership is key here. We come in as experts on the kind of research that we’re doing on the data that we’re working with but I think that even Nikki’s presentation just before me touched on this that it’s really important to rely on the expertise of other people, it can bring to the table, right? In this case, for the VEO team, we were fortunate to have pharmacologists and physicians who could contextualize the work within asthma and the treating asthma space and who were able to bring in other kinds of researchers to lend expertise and opinions to help develop things on the topic of research questions. This is I think the biggest departure. I think the research questions can and they really should be fewer for one of these engagements. They need to be really narrowly defined much more so than typical insights or MR work. For those with sort of traditional scientific research training who are thinking about this really frame what you’re doing in terms of a null hypothesis as opposed to what’s it like because that can kind of send you in all sorts of directions.  

Protocol development as I mentioned, it can take time. It can be hard but really, it’s your friend. It’s the first draft of a peer reviewed article basically because it’s where you’re putting in what you’re going to be doing while you’re doing it. If you were traps that you leave for yourself and your protocol the easier time you’re going to have down the road. So, yes, don’t expect to approach it like it’s necessarily an insight certain MR project. The research feels a lot more sober. I was trying to think of what kind of words to use here to describe the difference without putting too much of a sort of valence on these two modes. I think the focus in this case is a lot less on sort of an insight story on implications. We got really interested in the data which is great because if we didn’t like it that would kind of be terrible for us and I would need to find other job but we had all these ideas about like what works, what could you do to improve you know things. Unfortunately, this is not necessarily the space for putting those in a formal scientific report. That set the goals findings rather than sort of strategy recommendations. Lastly, just this is kind of a no brainer but it’s important to know your expected audiences and in particular that they’re going to be different. In this case, we’re thinking about conferences, we’re thinking about journals, we’re thinking about people out there in the world rather than a sales team or a marketing team that might pick up a report and try to make sense out of it and like something or not like something because it matches with whatever strategic plan they have already. 

Also, think about outputs. It surprised some people on our team. The extent to which our GSK partners were not interested in PowerPoint whatsoever. It was exciting to me personally because I love writing longer form, but it comes with the territory and writing for an article is much different than writing for a report that is sort of to be given orally, right? So, yes, your outputs can include PowerPoint, but primarily, I think they’re going to lean more on things like manuscripts, conference abstracts and presentations that are not to sort of a company internal audience.

So, all in all, sort of a recap. I don’t really have a graceful way to land this deck, so I’m going to just go to the next slide. All in all, it’s a great growth experience. I would encourage anybody who hasn’t dipped a toe in that water to do so because it has a great feedback on sort of sharpening your skills as a market researcher as well in terms of affording some creativity on your part and designing research, building your confidence and ways to approach problems. I see time’s almost running out, so last but not least. I’m Zach Hebert, but I also want to say thank you to my research collaborators. Not all of them are Verilogue folks, so that’s Katya Solovyeva, Lisa Kietzer, Raphael, Ember and the rest of the GSK VEO team. We’re all on it kind of together and that’s what’s made sort of the process more engaging and collaborative. Thank you.  

Female: I don’t have any questions in the app. Are there any questions for Zach? All right.  Thank you, Zach. 

– End of Recording –

A special thank you to: