Artificial intelligence: boon or doom?

Jenny Williams. Photo by Roy Murphy

The Wellington book launch at Unity Books of a fictional AI agent competing for a literary prize, rapidly became a free-for-all on the virtues and vices of AI itself. Daniel Foote, founder of a new start-up investigating the risks of AI, spoke with Jenny Williams, author of House of Liars, but their audience soon took over.

For several years, Jenny Williams wrote the text for AI responses at Google, and worked with the AI ethics team at Microsoft. At that time Microsoft made public commitments to AI and to transparency and accountability. The ethics team was eventually dismantled.

“We had a moral philosopher on the team,” said Williams. “We had designers. We had project managers. We had different disciplines all coming together to think very deeply and hard about ethics questions. During that time we started working with a Large Language Model, which no one at that time had ever heard of.”

Artificial Intelligence (AI) mimics human intelligence, while a Large Language Model (LLM) is a specific AI tool that generates human text.

“I put text in and this miracle occurred – it responded in coherent text as if a person is right there on the other end writing back,” Williams said. “It felt like science fiction.”

Her novel is about six writers brought to a remote island to compete for a million-dollar publishing prize. One of the writers is an AI agent competing against the five human writers. Williams explores the big questions of writing, creativity, motherhood, and AI and the risks that it poses.

Williams said: “I don’t use AI in my creative writing. I don’t find it useful and I have ethical problems with it.” She thinks some of that is because AI is replacing artists and writers, and it’s stolen the content of a massive load of books, without compensation.

“One of the things that feels very powerful about AI, is that it’s designed to find and exploit our weakest points,” she said. “It’s targeted to say, Do you have writer’s block? AI can help you with that. Or if you’re not quite sure how this chapter should end, AI can give you feedback on that. Or, you don’t have time to read that email, AI will summarize it for you.”

The audience jumped in with disagreements.

One member said AI is biased. He said a research project asked AI to write versions of a short crime fiction novel. It nominated the place, the victim and the perpetrator, one black and the other white. He said: “If it was a black perpetrator, the whole situation would be set in a back-alley neighbourhood. If it’s a white perpetrator, the crime would happen in an affluent suburb.”

A woman chimed in: “You are rightly critiquing and challenging the ethical basis. We are continually referring to these impressive algorithms as if their sentience is human.”

Williams responded: “It is not an entity operating with its own independent agenda. It’s something built by humans, deployed by humans, and used by humans. I got into so many arguments with people saying, AI can’t say I’m thinking. AI can’t say ‘I’m looking at it.’ Because the model can’t think and it can’t look. We shouldn’t use the word hallucinate because it gives a sense that AI is sentient, that it has a mind of its own. We’re reinforcing this idea, when in fact it’s just an algorithm.”

Daniel Foote founded Good Ancestors New Zealand, an independent, non-profit policy organization focussed on how to manage the harms and risks of AI.

He said: “AI is only a tool that can be used for a whole range of purposes, some of which are quite disturbing,” including people having romantic relationships with AI. Worse, people’s thinking process and critical thinking are being outsourced to AI.

The woman cyber specialist said: “The entry level jobs are where you learn by doing simple tasks. That’s how you learn the job. But now the simple tasks are given to AI. The new generation won’t have the opportunity to learn.”

One woman posed the interesting question: “Do we have to be concerned that AI is inherently about the past? It builds on what is already recorded, so it’s always going to be reflecting the past, not the present.”

A man in the audience had a different perspective.

“I’m a heavy user of AI,” he said. “While it might be a statistical distributed model, in many respects maybe we are as well, as human beings. We respond to the world based on our previous experiences.”

Foote pointed out that there is a genuine debate in philosophical circles about AI consciousness. “There are very strong and different views about that. In many philosophy departments this is a fair and reasonable question, and is one that’s being asked today.

“AI tools are more reliable and lead to better results than human experts. AI tools may also enable humans to create new novel diseases. The benefits and capabilities can lead to useful benefits to society. The flip side is that these capabilities come with risk and possible harm.”

Williams said the big danger is that the industry is moving so fast.

“Sadly, my findings are that people cannot speak up when something is going in the wrong direction. People speak about it as an arms race. The sense of competition is so urgent among the front-line AI labs that even when the leaders of these companies say, we need to slow down, there’s no mechanism to do that. Instead, there’s a lot of pressures to keep building the next model even when we haven’t understood the risks right now.”

A nonfiction writer and historian has no intention of using AI because he doesn’t consider it reliable. He said: “AI will invent answers to questions that it can’t answer.

“I work a lot with archivists, historians, finding source materials, primary sources for the subjects I’m writing about. They tell me that people are increasingly coming to them saying, ‘I can’t find this file that’s referenced here’. The archivists will say, ‘it doesn’t exist. It is an invented reference to a made-up fact.’ This AI slop is being fed back into AI, producing more slop.”

Williams said: “Someone asked me, ‘what gives you hope?’ I looked around the room and said, ‘People showing up to talk about these questions and have conversations, that gives me hope.’ There are people like Dan who are starting organizations that say we think this is so important we are going to work full time to try to change minds and to help people see things with more clarity.”

Foote said: “A lot of change at the national level happens through political will and policy change, and that doesn’t come out of nowhere. It arises when talking about an issue becomes mainstream. Raising those issues with your local MP or the relevant minister is a thing they hear more and more often. Everybody can play a small role. It can lead to change at the national level, at a policy level.”

The woman cyber specialist offered support to those around her. “I know individuals in security and AI safety. I’m happy to run workshops about AI safety for senior AI users. If anyone wants to get a group together, I’m happy to teach and provide advice free. Come and see me afterwards.”

Jenny Williams

Daniel Foote