After a whirlwind year of rapid artificial intelligence advancements, some AI makers are starting to show a bit of caution. In the past two months, OpenAI has provided demos of video and voice generating technologies — but did not provide wide access, citing safety concerns. Last week, Microsoft unveiled a tool to create AI-generated videos of people based on just a photo and an audio snippet — but said it was too dangerous to release.

“We have no plans to release an online demo, API, product, additional implementation details, or any related offerings until we are certain that the technology will be used responsibly and in accordance with proper regulations,” Microsoft said in its announcement.

The circumspection is a welcome pause in the frenzy of new AI model releases, but it doesn’t mean we are out of the woods in this high-stakes election year. There are still hundreds of existing AI tools that are available to the public, and very little is known about how they are being used. Proof’s testing of text-generating models has consistently found them to be unreliable for voter information queries.

To discuss how to navigate this uncertain terrain, I spoke with Irene Solaiman, the director of global policy at Hugging Face, an AI software development hub. Solaiman formerly led AI policy at Zillow Group and was an AI researcher at OpenAI. She also previously helped train U.S. election officials to defend against cybersecurity threats and information operations.

Our conversation, edited for brevity and clarity, is below.

Angwin: What is the impact of AI in elections going to be? We’ve seen stories about deepfakes and the AI clone of Joe Biden’s voice. What do you see as the biggest problem? 

Solaiman: AI is a component of the threat landscape. From my experience, especially after monitoring the 2018 midterm elections and having studied the immense amount of influence operations that we saw at the 2016 presidential elections, it’s really a modification of the threat landscape. There is a lower barrier so that any person with a relatively strong internet connection is now capable of generating relatively realistic different types of modalities of potential disinformation. 

But, we’ve had this capability for a long time. You could Photoshop pretty realistic photos. The distinctions that I think we need to be making are in the modality of influence operation. With voice cloning, I do have more concerns than, for example, with generated text. 

Angwin: A lot of the discussions on AI in elections are a little confusing because people might start to think that we’re using AI in the election to count ballots or something, and that’s not correct. Tell us what we should not be worried about. 

Solaiman: I can only speak with expertise on elections in the U.S., and elections are necessarily decentralized. AI is not, to my knowledge at least, being used in any way to be part of our voter infrastructure. You don’t necessarily need AI to be counting ballots. You need these scanners that we’ve had for years, and they’re reliable.

So we shouldn’t worry so much about that right now. We should be more worried about seeing a doctored photo that is convincing or a voice clone that is convincing. That could change our views about a candidate or about an issue or even provide false information about whether it’s dangerous to go to the polls.

Angwin: People have been talking about watermarking that would identify AI content as a way to combat false information. But is watermarking ready for prime time?

Solaiman: Silver bullets are very attractive, and they also don’t exist. Watermarking is so promising as a research investment, but especially where we’re at today, we haven't seen it deployed at scale.

OpenAI did incredible work on watermarking DALL-E. But if you crop it, if you screenshot it, the watermark doesn’t hold. So these types of technical tools aren’t going to be a panacea and definitely not this year. What we can do is work together on identifying and stopping the distribution of not just fake content but any sort of misleading content.

Angwin: And so what does that look like? Can you give an example of what you would like to see?

Solaiman: Distribution platforms like news media and social media need to invest in capacity to gauge and identify content in collaboration with election officials across the world. That also means investing in lower resource regions and lower resource languages in 65 different jurisdictions that are hosting elections worldwide. 

In the U.S., secretaries of state need to be upping their capacity to work with distribution platforms. And on the distribution platform side, they need to have the people who are able to verify the appropriate communication lines and take action on their side. So there’s a lot of, frankly, just personnel capacity that if it’s not already invested in, needed to happen yesterday. 

Angwin: A question that often comes up in AI conversations is whether open source AI models are more dangerous than closed source. The theory is that because open source models can be modified by the user, they can be used to create dangerous or misleading content. Hugging Face hosts a lot of open source models. Do you think they are more dangerous than other AI?

Solaiman: I find that the release of a system and its components is often the wrong framing and what we need to be thinking about is access and barriers to access. An AI model’s accessibility can be related to but not dependent on whether it’s open source; a hosted model that is “closed” (e.g., unreleased model weights) can have an easily accessible interface for any person to use. These interfaces can vary in safety and monitoring.

I had my voice actually cloned, nonconsensually, a couple of weeks ago. I would not recommend it. It was not for malicious purposes, for marketing purposes, but it is very weird to have your voice say things that you didn’t say. Even my loved ones couldn’t tell that it wasn't me. I mean, the fact that I don't speak Chinese was a bit of a flag for them. But the technology has gotten very good.

Angwin: Did you have any recourse? What happened?

Solaiman: Legally, I didn’t own it. I give out my voice and my face constantly. My company issued a takedown request, and they took it down out of goodwill. I’m grateful for that. 

The reason I bring up voice cloning is that this was a closed source model but an open access model. So anybody could get a free trial. You don’t need any computer science background. And they say that you need to get consent [to clone a voice], but clearly that did not happen in my case.

Angwin: Is there anything people can do to detect voice cloning?

Solaiman: We don’t have good detection mechanisms right now, so it’s [a good idea to] have a safe word with your loved ones should your voice be cloned. 

Angwin: Many of the AI companies have made pledges about not allowing their models to be used in political campaigning and directing voter queries to trusted sources. But it’s hard to imagine how the companies will be able to prevent political campaigns from using their tools, and we have found that some of the AI models are not directing users to trusted sources. Are these pledges meaningful in your opinion?

Solaiman: Part of accountability is having an outline of your commitments. It’s better than not having anything at all.

But voters also need to be aware of the reliability of these systems, and I don’t think people, especially based on your work, should be using language models to find voter information. They should be going to trusted sources.