By SHANNON O. WELLS

Generative artificial intelligence may be a game changing, even revolutionary new force in academia and well beyond, but John Radzilowicz in the University Center for Teaching and Learning strives to view the technology development — and its potentially outsized role in university life — from a reasonable perspective.

“We see it as another tool in the tool belt to be used. We are very aware that if you wield the tool correctly, you can get some really good results,” he said, offering the saying, “‘Give a carpenter a hammer, and you get some great things. Give a vandal a hammer, and you end up with a lot of broken things.’ We understand that, and we bring that approach.”

Radzilowicz, the Teaching Center’s director of pedagogy, practice, & assessment, shared this and many other ideas during a presentation to Faculty Assembly at its Feb. 18 meeting in Posvar Hall.

Offering a mix of reassurance, reservation, historical perspective and modern reality, he laid out the Teaching Center’s proposed pathway for University faculty to incorporate and thoughtfully regulate Gen AI’s role in their curriculum and classrooms.

Acknowledging that he sensed “a little confusion” about the center’s role “and the professional approach we bring,” Radzilowicz clarified that the center considers generative AI a “powerful tool with great potential for teaching and learning.”

However, “we are very aware of the numerous risks and serious concerns that faculty and others have about a variety of issues.”

Regardless of a faculty member’s perspective on Gen AI’s role in the classroom, the Teaching Center is available to “support those decisions about use, non-use” and what Radzilowicz calls “scaffolding,” or structuring how it makes the most sense in your discipline. “But one of the things that we have to say is, ‘It’s here now.’”

To those who might view the technology as “just a fad” that’s “going to pass,” Radzilowicz shared a news flash: “Well, it’s been three years, and it hasn’t passed yet.”

Studies by the Teaching Center and other faculty members show that around 85% of Pitt students use Gen AI with about 15% to 20% of faculty using the tools. “As I always say, ‘What could possibly go wrong in a situation like that?’” he quipped.

Radzilowicz emphasized recent Teaching Center efforts to provide information and training. This includes considering when to let students use AI, “and when to say ‘no.’”

Despite some “miscommunication” in the past academic year about its AI-related offerings, he said the Teaching Center has engaged in about 170 hours of “direct, one-on-one consultations” with faculty. “About 20% of our total time in consultations has been on AI,” he said, including more than 100 professional development workshops. “Again, running about 20% of the total that we offered last academic year.

“We reached about 2,000 faculty with that,” he noted. “We have been very involved.”

Two-thirds of Gen AI workshops the center offered were customized to schools and departments that specifically asked to tailor information to them, he noted.

“Though we offer a wide variety of workshops that are available to everyone — these sort of public-facing workshops for faculty — we have been doing this customized work as well.”

The Teaching Center has what he calls “extensive resources” that are constantly updated, including a reformatting plan that faculty will be able to access on the center’s website. “We expect our numbers to be similar or bigger for the current academic year once those things are all added up.”

Instrumentalist vs. conductor

Despite Gen AI’s increasing ubiquity and the seemingly bottomless hype surrounding it, Radzilowicz acknowledged that, at least from an economic dominance standpoint, the “AI bubble is probably going to burst.” He referenced the late 1990s digital technology “dot com” stock market boom and subsequent crash, which led to many online shopping and service entities closing or losing significant value.

When the dust settled from that, however, “the internet did not go away,” he noted. “And you mostly still (do) your shopping online now. Most of you.”

In other words, AI is likely here to stay.

“It’s going to change … but we feel we cannot ignore it,” he said. “It needs to be understood, directed and managed. … Our goal is to help faculty in any way possible to maximize the benefits that can come from integrating AI and minimize the harms.

“And if that is a choice not to use AI, we’re there to help that as well.”

Maintaining a dynamic framework to accommodate AI in pedagogy is part of Pitt’s obligation to students, he noted.

“If you see what’s happening out in the broader world, businesses are reporting their use of AI as extremely high,” he said. “I hear it from students in my class all the time: ‘What will I be expected to know about AI when I get out into the job market?’”

Right, wrong or indifferent, the point is that Gen AI “is happening, and students are concerned about it. … We think that the approach lies in thinking really hard about what you actually want students to do.”

To that end, Radzilowicz and Alan Lesgold, senior advisor in the Teaching Center and former School of Education dean, conceived the “Instrumentalist vs. Conductor” approach to Gen AI. 

Taking on a new or unfamiliar endeavor requires the ability to “play an instrument,” as in achieving competency in core skills. These include writing, reasoning, critical reading, investigating, building models, applying judgment and “explaining and defending work without external crutches,” Radzilowicz explained.

“Once they learn those skills, what AI gives them the potential to do — and what we think is the future of how AI will play out in the workforce — is that they become a ‘conductor’” with the ability to define problems and decide things such as:

• When to use or not use AI.

• What tasks should be delegated to AI or performed manually.

• How to prompt AI and verify what comes out of the search.

• How to integrate those outputs into coherent solutions.

“They have to be able to detect flaws and bias,” he said. “They have to get the context there, and they especially have to make … ethical choices about what they do with this.”

The role of ‘the struggle’

Looking at career paths, Radzilowicz said students and faculty shouldn’t be concerned about losing jobs to AI. Rather, they should worry about “losing jobs to other people who know how to use AI. That’s the skill set that we want to talk about.”

Many students arrive at universities without foundational skills, i.e. an ability “play the instrument,” he noted. They’re not prepared to “conduct” or effectively apply AI to academics.

“They need direct practice. The need to be able to think for themselves,” he said, including judgment building, creating mental models and engaging in error detection. “Ultimately, that will get them to the ‘orchestration’ level, where they can delegate (and) strategically guide what happens, verify outputs and apply those results.”

The effort required to make this happen can’t be sidestepped, he noted, quoting Lesgold, who said, “outsourcing the hard cognitive work to AI is like sending a proxy to the gym and expecting to get (physically) fit,” Radzilowicz joked. “We have to do that hard work.”

The center breaks the approach into major categories of writing, quantitative decision-making, critical reading, and research skills. “If they can’t do these things, it is not going to be a benefit for them to jump right into AI,” he said.

The Teaching Center advocates what he called an “integrated developmental approach … where we scaffold across what they need to know and where we want to get them in terms of what they can use AI for.”

Faculty members should thus look at what they teach and decide, “‘How does that fit in? What course are you teaching and why, and where does (AI) fit into what they’re doing?’”

The sequential categories for this developmental approach include:  

Foundational courses. “You may be in a situation where you say there will be no AI in this class, or minimal AI or some level, as you work it out for your particular discipline,” he said. “Because you are focused on those practical skills they must learn.”

Intermediate. This incorporates teaching AI literacy, including how to use the basics, how to verify and “how to look at the product that comes out and make a decision about how that can be integrated.”

Advanced/capstone stage. This focuses on what the center calls “professional orchestration. This is the bigger scope: ‘How can I sort my work out so that I can use AI where it’s appropriate, don’t use it where it doesn’t make sense, and get outputs that I can integrate into meaningful solutions that I can apply,’” Radzilowicz explained. “This is a tall order. We understand that. (These) are the kinds of things we feel faculty need to be thinking about and talking about.”

Focused training therefore becomes essential. “This is something that we all need as a skill set in our own work … because you can actually have quite an impact on the work you do once you become good at using these.”

Based on consultations with faculty, Radzilowicz warns against getting “hung up too much on the product.”

“We hear people say, ‘Well, I need an AI detector because I’m looking at what comes out.’ Our answer to that is the AI detectors don’t work, and if you’re looking at the product, you’re looking at the wrong thing.”

To increase accountability, “you need to be looking at the process and how they got there.”

The approach starts with “low stakes, repetition, feedback,” and builds to higher cognitive structures. “You’re teaching students why there are AI restrictions and not just saying, ‘Well, here’s my policy. It’s a ban, and don’t ask me anything about AI.’ In this class, you say ‘This is why we take this approach.’

“Our tip is, if learning is in the struggle, and you take away that struggle with AI, the learning goes away too,” he added. “We have to think about that.”

Commanding AI

Key questions for faculty in deciding how their courses fit in and when they should use AI include:

• What foundational competencies are students still developing?

• Is the assignment building skills or applying already mastered skills?

• Can students critically evaluate AI outputs in this domain? “If they’re not ready for that, you don’t want to go there,” he said.

• Am I teaching AI orchestration skills explicitly? “Is that what I’m trying to do, or am I back at ‘the instrument?’”

• What unique human capabilities does this cultivate? “We think of them as essential skills,” he said, including the ability to work together, reason and hypothesize.

• Does the assignment emphasize process or product? “We just present that here as … students who have core skills and students who don’t, and how you might be able to break that down,” he said.

• Are you creating authentic, experiential learning opportunities that relate to your discipline? Feedback from faculty is essential. “We need you to tell us what’s happening in your discipline, in terms of how AI can and cannot work and how we can support that,” he said.

• What is your policy and how will you communicate it?

“These are the kinds of exercises we take faculty through so that you’re not just jumping to, ‘Well, I’m not going to do it.’ If you’re not going to do it, you know why you’re not going to do it.” If you are, however, you can justify it to yourself, your colleagues, dean — “and to your students, most of all — what you’re doing.”

This culminates in a series of possible policy frameworks:

Prohibit: Foundational skill practice, in-class essays and exams. “Foundational skills have to happen in this space, as in “we are either very little or not at all, in terms of AI.”

Permit with constraints: Where citations, prompts and verifying sources are required. “This is the training-wheel stage, where the scaffolding is beginning,” he said.

Require: Critiquing and improving AI output. “Where you’re teaching ‘orchestration,’ and you have very specific applications that they can use the AI for.

“I take this approach with my own courses,” Radzilowicz noted. “We use AI when they are in the process of developing robotic space missions to Mars. I actually have mission planning as part of the space flight course that I teach, and we work up to that all semester long, in terms of the core skills that they need.”

Students who skip the foundational skill building by outsourcing to AI, he noted, will struggle to think independently and evaluate information critically. “We can’t do that, so our job is to decide when do students need direct support. When are they ready for that orchestration level, and when can they use AI as professionals?” he said. “How do we get them there through these stages?

“We want graduates who command AI and not graduates who are commanded by it. We think that’s the line that’s going to make a difference when they enter the job market.”

Concerned faculty

Following his presentation, Radzilowicz addressed a question from Jon Stoner, professor in the Dietrich School’s Department of History, about AI companies sharing data with immigration enforcement and other government agencies, Radzilowicz clarified that the Teaching Center has “absolutely no involvement in that process.”

“I would ask of them the exact same questions that you asked, but they don’t answer my questions … Most people don’t know that, so the hate mail comes to me every time Pitt Digital does something,” he said. “We are, at the Teaching Center, agnostic about the tools.

“We do not support any one company, any one tool, and we think after the bust, some of those companies are going away.”

Jackie Calhoun, assistant professor in the School of Nursing, asked about the downside of AI in “removing the struggle” aspect of the learning process. Explaining that the point is to “delegate the drudgery to AI,” Radzilowicz clarified that he’s “no expert. And I don’t think there really are any experts.”

“You have computer-science experts who know about that, and you have people with educational expertise and all. But this is moving fast, and half of what I said to you may not be true a year from now. I mean, that’s a little bit of an exaggeration, but this is fast moving, and we have to work together to figure it out.”

Noting reports and observations that more students enter the higher education realm with fewer critical thinking skills, Susan Graff, director of the Physician Assistant Studies Program in the School of Health and Rehabilitation Sciences, expressed concern that AI could increase this deficit.

Radzilowicz said from the Teaching Center’s perspective, the instructor of record is “always the final decision maker in what happens in their classroom” in terms of techniques, pedagogy, assignments “and all of that. And I see nothing on the horizon where it is looking at policy. … I don’t think we ever want to (impose AI requirements) from outside.”

Addressing the critical thinking question, he said the aftermath of lost instruction time and routine life during the COVID pandemic of 2020-21 still reverberates with new college students.

“You’re also seeing some of the impacts of the world that we’re living in right now and the fears and concerns that those students have,” he said. “You’re right, AI has the potential to help it or make it a whole lot worse, and that’s why we want to have those discussions.”

Regarding the notion that AI is useful to reduce or eliminate “drudgery” and allow more time to immerse in a topic, Bridget Keown, teaching assistant professor in the Gender, Sexuality, & Women’s Studies Program, inquired if we are now only “preparing students merely for the world in which they live.

“How can we be using our tools to prepare students to change the world that they’ve been given?” she asked.

One “exciting” possibility about implementing Gen AI is “doing the things that we’ve known for a century are the things that actually work,” Radzilowicz replied, noting that it’s good news when English teachers say they’re going to “workshop essays” during class time.

“To avoid AI, maybe that’s not the best reason for doing it, but I’m so glad to hear (instructors) doing it, because we never should have walked away from that piece.”

HAIL to the rescue

Following Radzilowicz’s presentation, Michael Colaresi, former associate vice provost for data science, provided an overview of the new academic Hub for AI and Data Science Leadership (HAIL), for which he now serves as director.

Launched in January, HAIL is designed to build upon the foundation created through the Responsible Data Science initiative and other fledgling AI efforts. It serves as an academic convening point for AI and data science efforts, facilitating collaboration across teaching, learning and other academic operations.

“Many of the things we’re trying to tackle around responsible AI use as a practice align with responsible data science,” he said, with many following from the Data Science Task Force in Pitt’s “unique place” in applied data science and algorithms and models. “And not believing the hype of the illusion of abstraction and math. Because the power of models is in the people, in the decisions that are affected by them.

“We continue that work. But it’s faster … It’s everywhere all at once at the same time,” Colaresi said. “It’s more opaque and complicated … and it’s even more massively interdisciplinary.

“In doing this. I am kind of awed by the challenges … And I don’t want to say that we’re solving this. This is what we’re working on,” he added. “And I’ll tell you what I think our leverage is, and what we’re trying to do with it.”

To read more about HAIL and Colaresi’s new role, please read this story in the Dec. 19 edition of the University Times.

Shannon O. Wells is a writer for the University Times. Reach him at shannonw@pitt.edu.

Have a story idea or news to share? Share it with the University Times.

Follow the University Times on Twitter and Facebook.