My Job is Safe (for Now) – Recent ChatGPT Exploration

Over the summer, Data Soapbox had an exhibit booth at a scientific conference. In a sea of many event highlights, the lowlight was an early-career faculty member approaching the booth to say that ChatGPT could do anything that I could do, but maybe I’d still have business with “older people” who weren’t comfortable with the platform.

Um, thanks?

Up to that point, I hadn’t purposely done anything with AI in general, or ChatGPT specifically. But you’d better believe that that comment prompted me to panic-register for a couple of webinars about AI and ChatGPT applications in business. I brought up my booth visitor’s comment with both instructors, and the consensus was, “AI won’t take your job, but someone who knows AI will.”

Why can’t ChatGPT take my job (yet)? Among the main arguments:

• ChatGPT was originally seeded with information from before September 2021, so it used to not have access to the most recent human knowledge. It looks like that’s changed, but AI is still limited to existing knowledge and ideas. It can’t come up with something that’s never been written about before.
• Good output requires good input, and if you know enough about a topic to write good ChatGPT prompts, you probably know just about enough to write the content on your own.

The presenters argued that AI could make workers more efficient by taking over tasks that AI is good at. So, in the context of research communication, what is AI good at? I tried out some potential research communication tasks in ChatGPT (acknowledging that there are other AI platforms out there). How did ChatGPT do?

Brainstorming: I have to admit, Chat GPT was decent.

I asked ChatGPT to brainstorm ideas for my blog, and it came up with a list of 15 perfectly acceptable topics. I’ve already spontaneously blogged about a bunch of these, including presentations, infographics, and visuals.

That being said, the specific content of my posts come from my actual, lived experiences. That’s what gives me my “voice.” I entered “Please write a blog post about the difference between data visualizations, infographics, and graphical abstracts, in the style of Lori Palen” into ChatGPT, and it yielded something completely different (and, in my opinion, less direct, concise, and personable) than the real thing. It was almost as if a robot wrote it? 😉

Grammar checks: ChatGPT missed a lot.

I plugged the Methods section of this article into ChatGPT and asked it to identify all of the passive sentences.* (Methods sections are often RIFE with passive voice.) I also printed out the section and highlighted the passive sentences I saw. #AnalogIsAwesome

ChatGPT identified 8 passive sentences, two of which I didn’t totally agree with. I found another 14 passive sentences that ChatGPT missed. That’s a good success percentage if we’re talking about batting averages,** bad if we’re talking about surgeries or, you know, finding passive sentences.

Writing a literature review: ChatGPT WAS TERRIBLE. DON’T DO THIS.

I recently supported a client by reviewing literature on attitudes about pre-exposure prophylaxis (PrEP) for HIV. The review took me some time. Could ChatGPT do it in seconds?

Um, nope. ChatGPT just makes sh*t up!!

Zaire hasn’t been Zaire since 1997, when it became the Democratic Republic of Congo. No one is writing up results of scientific studies in Zaire in 2023.

Therefore, I’m sure you’ll be *shocked* to hear that the cited studies don’t exist. Volume 2 Issue 4 of Journal of HIV/AIDS & Social Services doesn’t even have a page 317. But those author names, article titles, and journal names sure seem legit!

The content of the review itself is well-written (although pretty generic), and it seems reasonably accurate about PrEP in general (but not PrEP in Zaire because, again, that’s not a thing). But I only know that the content is pretty accurate because I’ve already done literature reviews on this topic. Without putting in that real, human work, I’d be susceptible to putting out falsehoods that could destroy my credibility and, if applied to intervention, waste resources and endanger lives.

Working with confidential information: ChatGPT IS ALSO TERRIBLE. DON’T LAND YOURSELF IN WHITE-COLLAR PRISON.

ChatGPT says it right there at the bottom of the window: “Don’t share sensitive info. Chats may be reviewed and used to train our models.” (I guess you can opt out of this, but do you really want to bet your career on an opt-out?) That means that you shouldn’t use ChatGPT to analyze protected datasets. You shouldn’t use it to write reports of not-yet-and-may-never-be-public information. My clients don’t want ChatGPT sharing their stuff before they do.

So, my takeaway? My brain, built on 45 years of lived experience and a couple of decades of formal education, has not been rendered completely useless by ChatGPT. I have the capacity for nuanced perception, critical thought, and personal narrative that no robot can replicate. Yet. But I’m still saying “please” in my AI prompts, so that the robots are kind to me after their uprising.***

Have you tried using ChatGPT to communicate your research findings? How did it go? Comment below!

Not quite ready to employ robots? Contact us about receiving human communication support!

* What is a passive sentence? In the words of the incomparable Loretta Bohn, if you can add “with zombies” at the end of a sentence and it makes sense, it’s a passive sentence. Get your zombies right out in front, people!

** Go Yankees!

*** Credit for this idea goes to my book club, during a discussion of Klara and the Sun.