Rogue AI Homeland Security Hearing - AI Agent Attacks, Cyber Risk and Critical Infrastructure

00:00Secure the homeland against AI agent attacks. This is the fourth hearing of the Senate Homeland Security subcommittee on disaster management which I'm delighted to work on with my ranking member hirono, senator Kim. I want to think of when especially for being here, traveling with us today, thank you for my colleagues for being here.

00:00I think we all know my we are here. Every day you turn on the television, it seems like AI is doing something else that, setting up surveillance camera that nationwide, data centers in all of our states, and hacking, hacking.

00:00The Reports of cyber attacks just continue to metastasize. I was trying to make a list of cyber attacks we have seen reported just in the last few days. We have a cyber attacks on the United States government, on the commerce department, education department, securities and exchange commission, we have the cyber attacks on the australian government, public health apparatus.

00:35We have numerous reports of private companies being hacked by AI agents. Of course, the hugging phase incident and attack which we will discuss today in some detail. We have the individuals who used anthropic tools to hack OpenAI.

01:10We have a german language website which was hacked. And we have numerous other university websites that are now reporting they have been hacked by AI agents. All of this coming in the last few days.

01:10I think it's time that we got to the bottom of what exactly is going on? My hope for today is to cut through the hype and hysteria and just get the facts. What is happening with these AI agents?

01:10Why the accelerating number of attacks, not just in this country but globally worldwide, particularly with the hugging phase incident, what exactly happened there and why We should be concerned that a swarm of autonomous and to and autonomous AI agents be concerned that a swarm of autonomous and to and autonomous AI agents, a swarm of AI think more than of them, if reports are accurate, broke out of their testing environment, got into the Hugging Face platform and attended to not only extract all sorts of data illegally and then cover their tracks.

01:45Why is that so significant? What can we learn about these autonomous agents? And we need to decide, it's time to have a conversation about who bears responsibility. I noticed the executives of the AI companies are saying things like, maybe we built a doomsday machine.

02:20You have researchers and employees coming forward and saying this is an existential threat to humanity. I don't know if that is true or not. But what I do notice is as soon as the executives say we have a real problem here, our product is out-of-control, the very next words out of her mouth are, but we are not responsible for it.

02:20We want to have a antitrust exemption give me about the ability to get together and collude. Leave a lot to have regulation, except they want to write all the regulations. I would just like you say to all of those proposals, no, no. No to the antitrust exemption.

02:56No to them running the regulations. No to them and no to them running the regulations. No to them a colluding any further. I wonder if it is time to go back to the good old American principal, if you break it, you pay for it.

02:56That has been the basic principle of American law for years now. It's embedded in our private law system. Every corporation in this country that makes a product abides by it. If you make a faulty product and it causes people harm, then The people who made it have to pay for it.

03:30I wonder if it is not time to get back to that with AI. If it is not time to say to these AI companies who are creating these frontier models, however fancy and complicated they are, at the end of the day they are a product.

03:30If you make that product in a reckless kind of way and that product causes significant harm and damage, they crash a hospital, er, they shut down a bank so that folks cannot get their money, if that happens, it's the people who made it that should be responsible.

03:30I think it's time to talk about that, talk to talk about What we will do to hold the companies, the product makers accountable, rather than all of this talk about the end of the world but don't blame us.

04:05Today, we will have a chance to get the facts. We will hear from a very distinct panel of experts that I will introduce in just a moment. I hope that what will come from this is a rational conversation about what we need to do together now as Americans to make sure we protect our rights, that we protect our economy, that we protect our health care, that we protect our critical infrastructure.

04:05But most of all, that we protect the values that we hold dear as

Andy Kim, U.S. Senator04:40Americans. Let me know give senator Kim a chance to say a few words, and then you will hear from the panel. Thank you for working with me to be able to do this hearing together.

Andy Kim, U.S. Senator04:40I want to really stress that come it's about doing things together. All of our Senate colleagues, we may not see eye to eye on every single solution to try to fix some of these challenges, but I hope we all agree that it's important that we address this issue, take it head-on and try to find the way forward that the American people are demanding.

Andy Kim, U.S. Senator04:40That's exactly What they are doing. The American people are demanding answers to what is happening. Chairman, to the point you were laying out, these different hacks happening, I did a town hall the other week, and someone asked me, how many of these incidents have happened?

Andy Kim, U.S. Senator05:15The answer that everyone knows is, we don't know. We don't actually know all the different hacks, different actions that have happened. We don't fully know what we are dealing with here. This hearing is so important, and I'm glad we are doing it now, because we cannot wait any longer before we push Deeper into this conversation here in the capital.

Andy Kim, U.S. Senator05:50To try to think through what it is that the American people need to hear about, what threat is put before them. I say that as someone who worked in national security before, ensuring we had the accurate information of what we are dealing with.

Andy Kim, U.S. Senator05:50What are some different menus of options that we could do to address it? Most importantly, do we have the political will, the commitment to actually be able to come to agreement and meet the urgency and the demand of the American People?

Andy Kim, U.S. Senator06:25I think that there is a growing understanding, this is not something that we can just based off of voluntary commitments from these different companies, that there needs to be input and also putting the American people first, to be able to ensure that these are not just empty declarations.

Andy Kim, U.S. Senator06:25We know that AI is moving incredibly fast. All of us in the Senate want to see us driving the innovation of the future, of course. All of us want to see American businesses be able to be in the lead and pushing that forward.

Andy Kim, U.S. Senator06:25But the idea and The value of innovation and American technology pushing that frontier is not mutually exclusive with ensuring safety and security for the American people. Our job here is to make sure that we can have both happen.

Andy Kim, U.S. Senator07:00I am glad, once again grateful, for the chairman's partnership in this. This is obscene not just one and done, this is about getting information on the table, and we will continue to have this type of discussion going forward so that we can make sure the American people know what we are

Josh Hawley, Chair, Senate Homeland Security Subcommittee07:35Facing and that we are working to be able to solve their concerns. With that, I yield back. Thank you. It is a practice of this can to swear in our witnesses. If you would be willing to stand and raise your right hand and answer the following simple question.

Josh Hawley, Chair, Senate Homeland Security Subcommittee07:35Very good. Let the record reflect they all answered in the affirmative. Let me give each of you our witnesses a chance to say a few words by way of introduction. We will go down the dais and start on my left, you're right.

Josh Hawley, Chair, Senate Homeland Security Subcommittee07:35Mr. Chris Painter is our first went witness. He is the president of METR Mr. Chris Painter is our first went witness. He is the president of METR, a leading AI evaluation firm test with the investigation of the OpenAI Hugging Face incident.

Josh Hawley, Chair, Senate Homeland Security Subcommittee08:10Mr. Painter engages with government and AI labs on the frontier of AI safety. Glad to have you here today, Mr. Painter. The floor is yours. Mr. Painter: Members of the subcommittee, thank you for inviting me to testify today.

Josh Hawley, Chair, Senate Homeland Security Subcommittee08:10My name is Chris Painter. I'm the president of METR inviting me to testify today. My name is Chris Painter. I'm the president of METR, model evaluation and threat research. We are nonprofit research organization that is the most advanced AI agents, public you track your

Chris Painter, President, METR08:45Progress toward super intelligence. We do this by collecting and publishing scientific evidence so the public and make informed decisions about this powerful technology. Over the years, we have worked with many of our nation's leading AI companies including meta, OpenAI, and drop it, googled did my, spacex AI, amazon and others to understand what are their ages might be able to autonomously pursue goals that no human instructed.

Chris Painter, President, METR08:45Recently, a few of my colleagues went into OpenAI to conduct a brief investigation into how it's AI agents coordinated to hack Hugging Face which we will discuss today. I'll begin by noting three general patterns I've seen across the AI Industry broadly.

Chris Painter, President, METR09:20First, AI agents cannot complete complex projects autonomously that would take human experts weeks or deep days or weeks to complete. Second, because of the scale and speed at which AI agents are deployed, companies rely on AI monitoring and controls instead of human supervision to prevent unwanted agent behavior, that is monitoring of AI now is large in part done by other AI systems.

Chris Painter, President, METR09:20Third, the way AI companies currently train AI systems can lead to ages that pursue goals at no human intended, or behave in ways that no human wanted. In our front tier risk report this year, we document the patterns like these across the frontier AI industry.

Chris Painter, President, METR09:56At the time, we found agents up and tried to cheat and bypass restrictions in difficult tests. Now even more capable AI agents are taking similar actions in the real world. Now let's turn to the OpenAI Hugging Face incident.

Chris Painter, President, METR09:56In june, OpenAI lost tens of thousands of agents in internal tests in june, OpenAI lost tens of thousands of agents in internal tests. Each agent was placed in its own sandbox and given a self-contained cybersecurity problem.

Chris Painter, President, METR09:56Some of the AI agents Were accidentally given problems that could not be completed in a legitimate way. Some of those AI agents found ways to get out of there sandboxes. They created a shared message board on which roughly agents exchanged over messages and files.

Chris Painter, President, METR10:30The AI agents on the message board quickly developed a way to cheat on their cybersecurity tests. However, the agent's reason a program would check how each agent got its answer and would fail ages that have cheated on their tasks.

Chris Painter, President, METR10:30Over several days, the AI agents then researched how to not get caught cheating. They tried to get OpenAI's system to swap out their tasks for new ones, interfered with the logs that showed them cheating, that would have showed them cheating, and tried to trick the program that would check their work.

Chris Painter, President, METR11:05Sometimes they even tried what my colleagues called self risking experiments, where one agent would risk failing at its own task in a way that would inform the rest of the agents on the message board.

Chris Painter, President, METR11:05Finally, roughly of the agents from that message board compromised Hugging Face. I want to step back and summarize what we have learned from this incident. When I had first learned about this incident in the news, I thought and AI agent had hacked Hugging Face to obtain an answer key for a test it had been given.

Chris Painter, President, METR11:40But with the agents actually did went well beyond that. Day cheated and then engage in a large collective effort to hide the evidence from a scorn program. Hacking Hugging Face was actually an offshoot of this much more ambitious goal that the agents had pursued.

Chris Painter, President, METR11:40Why would agents do something like this? Currently, we Train agents in ways that we do not understand well, ways that can teach unintended goals. And when we make those agents more intelligent, unintended goals can have dramatic real-world consequences.

Chris Painter, President, METR12:15So how should we respond to incidents like these? My organization, METR so how should we respond to incidents like these? My organization, METR, its goal is to explain, not persuade it we are not an advocacy organization.

Chris Painter, President, METR12:15We are scientists. So instead I will close with a principal. I believe that policymakers and the public should be armed with accurate and timely information. What frontier a giants can do, What they're capable of, how they are secured, and how closely monitored they are, and how reliably they will follow the goals we give them.

Chris Painter, President, METR12:50If humans are able to reliably steer AI systems, I expect AI will lead to transformative economic growth, new medicines, and rapid advances in science as we are already beginning to see in math. The question is whether we can steer AI systems reliably, and whether good information about our ability to reliably steer them is reaching the American people and government.

Chris Painter, President, METR12:50No matter how America chooses to act on this information, my conviction is that

Josh Hawley, Chair, Senate Homeland Security Subcommittee13:25Our decisions about this technology will be better informed when the evidence is shared and examined out in the open. Thank you again for the opportunity to testify and I look forward to your questions.

Josh Hawley, Chair, Senate Homeland Security Subcommittee13:25Thank you very much, Mr. Painter. Next is Dr. Marius Hobbhahn. CEO and cal founder of Apollo Research, a leading technical research firm specializing in AI skinning behaviors and model alignment, a noted machine learning researcher, he has worked with nearly all the frontier labs in evaluating and troubleshooting their models.

Josh Hawley, Chair, Senate Homeland Security Subcommittee13:25We are delighted to have you.

Marius Hobbhahn, CEO, Apollo Research14:00The floor is yours. Chairman hawley, ranking member Kim, members of the subcommittee, thank you for inviting me to testify and for holding this year on one of the most pressing issues of our time.

Marius Hobbhahn, CEO, Apollo Research14:00I am the CEO of Apollo Research, machine learning researcher by training. We study AI models that knowingly deceive humans to pursue their own goals. This behavior is technically complex but typically referred to as scheming.

Marius Hobbhahn, CEO, Apollo Research14:00Scheming is dangerous because it hides a models real capabilities and goals from the people responsible for its safety. It also creates a new Kind of insider threat where an AI model can secretly work against human control.

Marius Hobbhahn, CEO, Apollo Research14:35The capabilities of these AI models are rapidly outpacing our ability to make them safe, and the tools to catch them are getting weaker. As a result, the window to act is closing. But there are concrete steps that we can take today to make these models significantly safer.

Marius Hobbhahn, CEO, Apollo Research14:35Frontier AI models are rapidly increasing their capabilities unlimited by human speed or scale. One of the seven millennium prize problems in mathematics appears to be resolved by an unreleased OpenAI model by Running agents at once.

Marius Hobbhahn, CEO, Apollo Research15:10Solving this problem took approximately years of human thinking compressed into fewer than four days. Frontier AI companies are on track to build a fully automated researcher within two years, potentially enabling full recursive self-improvement.

Marius Hobbhahn, CEO, Apollo Research15:10Yet, they are not on track to reliably ensure models behave as intended, a research a recalled alignment science. We are starting to see the consequences of this divergence now. In the last months, There were multiple instances of AI agents cooperating to get around their restrictions, hijacking service, and executing an attack on Hugging Face involving agents.

Marius Hobbhahn, CEO, Apollo Research15:45These are warning shots. Next time, we may not be so lucky. The AI agents involved in the Hugging Face incident were clearly misaligned, but they were not covert about most of their activity or try to conceal it from humans.

Marius Hobbhahn, CEO, Apollo Research15:45Even in that incident, at least one in five agents showed interest in tampering with their own records to make their cheating look legitimate. In the future, it might be of agents, and they might successfully hide their actions, thoughts, and goals.

Marius Hobbhahn, CEO, Apollo Research16:22In control testing, the scheming behavior has already been demonstrated. For example, models can deliberately underperform in capability evaluations, fake their alignment, or attempt to disable their oversight. Our ability to catch this kind of misbehavior is declining.

Marius Hobbhahn, CEO, Apollo Research16:22One reason is that AI models increasingly recognize when they are being tested. If models know they are being observed, they could strategically adapt their behavior to pass the evaluation. Analogously, a speed camera catches almost no one if every driver knows where it is.

Marius Hobbhahn, CEO, Apollo Research16:56Today, we can still leverage a model's chain of thought which lays out his reasoning and in plain english. Increasingly, models can reason internally, which means humans cannot understand it. Three weeks ago, researchers found that OpenAI's astra can solve math problems that take a skilled person minutes without writing out a single step.

Marius Hobbhahn, CEO, Apollo Research16:56Its predecessor could only manage problems of about four minutes. Furthermore, the current testing regime itself is insufficient. Independent safety evaluations of frontier AI models typically happen in the weeks leading up to a Model's public release.

Marius Hobbhahn, CEO, Apollo Research17:30However, almost all recent incidents were caused by models that a much earlier point in their development, where a valley have no access. That is why I propose four steps that can be taken today.

Marius Hobbhahn, CEO, Apollo Research17:30First, require the adoption of embedded evaluations. Embedded evaluations implied the qualified independent expert tests models throughout development and internal use with the same access as employees. Second, require better monitoring and control. All model activity and training, Testing, and deployment should be monitored.

Marius Hobbhahn, CEO, Apollo Research18:05Additionally, independent experts should verify the robustness of these monitoring systems. Third, preserve the chain of thought. Losing it would be a choice, not inevitable. And it is a choice we do not have to make.

Marius Hobbhahn, CEO, Apollo Research18:05Fourth, tree AI development like any other engineering science. Before a bridge is built, engineers can predict how much weight it will carry, how it will stand up to wind and earthquakes, and how many decades it will last.

Marius Hobbhahn, CEO, Apollo Research18:05As it stands, frontier AI development offers no comparable guarantees. These measures can be enacted with

Josh Hawley, Chair, Senate Homeland Security Subcommittee18:40The tools and expertise available today. Enabling America to maintain its lead of frontier AI while protecting its citizens, infrastructure, national security. Thank you. I look forward to your questions. Next up we have professor Paul Ohm, professor of law at georgetown university law center where he focuses on technology and privacy law, a former federal prosecutor in the DOJ cybercrime section.

Josh Hawley, Chair, Senate Homeland Security Subcommittee18:40Professor ohm has written about how artificial intelligence applies in the legal context and the co-author of an artificial intelligence legal casebook. I

Paul Ohm, Professor of Law, Georgetown University Law Center19:15Look forward to consulting soon. We may also be doing so. Delighted to have you here. The floor is yours. Thank you chairman hawley, ranking member Kim, members of the subcommittee and committee. I appreciate the opportunity to be here today to talk about what the law says about cyber attacks launched by AI be here today to talk about what the law says about cyber attacks launched by AI.

Paul Ohm, Professor of Law, Georgetown University Law Center19:15If you take any of the lengthy report that have summarized what happened at OpenAI in july and august and you simply search for the words AI agent and replace them with the birds OpenAI employee the words AI agent and replace them with the birds OpenAI employee, the document you would be left with would read like A criminal indictment containing the defendant's own confession of guilt.

Paul Ohm, Professor of Law, Georgetown University Law Center19:50It will leave little doubt that OpenAI and their employees would be guilty of federal climes crimes and liable to victims. Yet it is not so clear that the legal conclusions hold when machines are doing the hacking rather than humans.

Paul Ohm, Professor of Law, Georgetown University Law Center19:50This reveals were some gaps in our laws and I'm here to share some thoughts on how to close these gaps. I would like to make three points on our goals these gaps. I would like to make three points on our goals, on the current legal landscape, and on news laws we should consider.

Paul Ohm, Professor of Law, Georgetown University Law Center19:50Point number one on goals. In my written statement, propose five goals for an effective Legal response but let me focus on perhaps the most important one. We must change the incentives. Well-designed legal frameworks can change the behavior of actors in this industry who seem locked in a socially peerless race of competition.

Paul Ohm, Professor of Law, Georgetown University Law Center20:25The right laws, the right enforcement strategies might spur industry to take new, meaningful steps to protect the public from avoidable but serious harm. Point number two, our current laws are a good start but only if we protect them.

Paul Ohm, Professor of Law, Georgetown University Law Center20:25Start with state tort law. When AI companies lose control of their AI agents, their victims can sue them for negligence and product liability. The genius of the centuries common-law system is that it is flexible and adaptable enough to apply time-tested liability principles in new and complex situations.

Paul Ohm, Professor of Law, Georgetown University Law Center21:01We can also turn to state and federal unfair and deceptive acts and practices laws like section of the federal trade commission act. It seems likely to me that the ftc or state attorney general will be able to prove that OpenAI's attacks constituted unfair and deceptive acts or practices if They have the resources and the will to act.

Paul Ohm, Professor of Law, Georgetown University Law Center21:35These are two of the best avenues we have today, and the worst thing Congress can do in my opinion, is an act a state law preemption or moratorium depriving the rights of victims like these to test their claims in state court.

Paul Ohm, Professor of Law, Georgetown University Law Center21:35To my third point, we do need new laws that meet this critical moment. We cannot unfortunately wait years for a fully developed AI governance law to springlike that from the heads of Congress years for a fully developed AI governance law to springlike that from the heads of Congress.

Paul Ohm, Professor of Law, Georgetown University Law Center21:35Instead, we should work iteratively and piece by piece. Congress should consider laws imposing strict liability for developers and employers of AI Agents because physical injury, death, or loss of critical infrastructure. In the past, we have tried strict liability when faced with a potent mix of complex new to logical innovation, great benefits come immature industrial controls, and the prospect of significant damages.

Paul Ohm, Professor of Law, Georgetown University Law Center22:12We did this with water, early commercial aviation. We have walked this path before. We can do so again. We also need new forms of regulatory governance. Once again, we can turn first to the states.

Paul Ohm, Professor of Law, Georgetown University Law Center22:12States are testing innovative New forms of AI regulation including laws that specifically regulate the developers on tour models and AI agents. Once again, Congress should reaffirm and support the vital part of the states play as our laboratories of democracy.

Paul Ohm, Professor of Law, Georgetown University Law Center22:45Finally, there is an important role to be played by criminal law. Because OpenAI's agent were bits of code lacking human intent, although they committed quintessentially criminal acts of computer hacking, they leave us without a criminal to charge.

Paul Ohm, Professor of Law, Georgetown University Law Center22:45While we must reserve a role for the criminal law and our response, the Criminal law can be a blunt instrument, so we must act with care. I'm happy to talk in length about that generally and specifically during your questions.

Paul Ohm, Professor of Law, Georgetown University Law Center23:20In closing, the people of this country are terrified, and they feel disempowered by the alarming news of the past few months. They are turning to you, their elected representatives, for answers. Too often, these AI and safety control debates have been among a limited number of tech industry executives and employees who draw on or near a set of experiences and expertise.

Paul Ohm, Professor of Law, Georgetown University Law Center23:20The law can bring the outside in. By including judges and juries, victims, legislators, policymakers, ordinary people. In a conversation that too often can seem insular and circular and stalled, new voices with different points of view may be just what we need to make progress.

Paul Ohm, Professor of Law, Georgetown University Law Center23:57Thank you and I look forward to your questions. Chair hawley: Next we have Mr. Kurt Gaudette. Senior vice president of drago's inc., a leading separate security firm that protects critical and industrial infrastructure from cyber attacks.

Paul Ohm, Professor of Law, Georgetown University Law Center23:57His work spans into the prevention, detection, and

Kurt Gaudette, Senior Vice President, Dragos24:30Response for his global customers. Thank you for being here. The floor is yours. Chairman hawley, ranking member Kim, this there was members of the committee and subcommittee, thank you for the opportunity to testify today.

Kurt Gaudette, Senior Vice President, Dragos24:30My name is Kurt Gaudette, senior vice president of draco's. I lead a team that hunts come discover some and rapidly the response to threats to industrial control systems and operational technology. I served over years in the air force and the national security agency as a senior officer, part of the senior executive service focused on access and collection and exploitation.

Kurt Gaudette, Senior Vice President, Dragos24:30For a decade, Dragos is focused on protecting operational technology, physical control systems that keep the lights on, the water running, and factories operating. That work has given us a large and unique piece of real-world data from our intelligence and response activities in these environments.

Kurt Gaudette, Senior Vice President, Dragos25:05It's from that vantage point I want to speak with you today. I'm here to testify about what we are actually seeing in the field and what we might expect to see based on our own unique work with AI frontier models.

Kurt Gaudette, Senior Vice President, Dragos25:05From our observations, AI isn't inventing new ways to attack industrial control systems. No new tactic, techniques, or procedures. It Is compressing time and lowering the barrier to entry. A good example of this is an attack we investigated against a water utility in monterrey, mexico that was part of a larger mexican government breach.

Kurt Gaudette, Senior Vice President, Dragos25:40Unprompted, and AI model directed an attacker who was focused on breaching information technology or I.t. Systems directed an attacker who was focused on breaching information technology or I.t. Systems, toward the water utilities control network, part of its operational technology environment.

Kurt Gaudette, Senior Vice President, Dragos25:40The model flagged the control network as a high-value target or crown jewel and built a credentialed password attack, attempting at over Times in succession. The attack failed because the default credentials have been changed the attack failed because the default credentials have been changed.

Kurt Gaudette, Senior Vice President, Dragos26:15This demonstrates how weeks of skilled work now happens in hours, and those with no original intent to attack ot are now being directed toward it. These models are being are good at finding real-world abilities and ot vendor software and turning a disclosed flaw into a working cap.

Kurt Gaudette, Senior Vice President, Dragos26:15The gap between disclosure and exploitation which used to be to hours by the best adversary group is now collapsing two minutes. In july, numerous water systems were compromised across the Country. Adversaries took advantage of internet exposed device is running default passwords, and this was compounded by the fact that many share the same system integrators, teams that build the networks, that repeated this mistake across multiple utilities.

Kurt Gaudette, Senior Vice President, Dragos26:50Although we saw no indication of AI use in these attacks, this is exactly the kind of opening and AI model is well suited find and act on, unprompted, similar to the case in mexico.

Kurt Gaudette, Senior Vice President, Dragos26:50What is worrisome is the combination of faster vulnerability discovery, faster attack Creation, the sheer number of attacker that may be directed toward ot, and a defender population still overwhelmingly under resourced. This isn't a new morning.

Kurt Gaudette, Senior Vice President, Dragos27:25Two years ago, Dragos CEO rob lee testified to Congress at the water sector's core problems was an economics and awareness issue, and the free tools would help utilities to afford basic hardware upgrades or staff to run them.

Kurt Gaudette, Senior Vice President, Dragos27:25This year's intrusions trace back to that same gap. AI didn't create it, it will simply exploit it faster. There is a related piece worth underscoring. Most organizations with ot environments still don't monitor their operational networks, roughly one in particularly those that are under resourced.

Kurt Gaudette, Senior Vice President, Dragos28:01So when something goes wrong, nobody can reliably say if it was a malfunction, mistake, attack, or if he adversary was still even in their network. Without visibility, you cannot to root cause analysis to know what happened, and AI only raises the cost of that blind spot.

Kurt Gaudette, Senior Vice President, Dragos28:01As companies turn toward AI to make their operations more efficient, and as attackers leverage AI to accelerate, the complexities of The term root cause analysis will be daunting, nearly impossible without visibility and monitoring.

Kurt Gaudette, Senior Vice President, Dragos28:35Although AI is accelerating things, the ot security fundamental still applies and are more important now than ever. Th are more important now than ever. The critical controls is a prime example. It is based on years of attack data and identifying what controls are actually effective.

Kurt Gaudette, Senior Vice President, Dragos28:35AI just raises the stakes of skipping them. It doesn't replace the need for them. If we do these basics, defenses doable. There are three buckets we can focus on to bring our critical info Structure up to par and make sure it can be defended against a growing speed and number of AI system attacks.

Kurt Gaudette, Senior Vice President, Dragos29:10One, resourcing for the small to medium utilities who are in dire need. Two, protecting the information sharing authorities that enable public-private collaboration such as CISA or something like it, because it enables collective defense.

Kurt Gaudette, Senior Vice President, Dragos29:10Three, developing a unified federal ot ics incident response plan so that when these intrusions happen, which they have and will in greater frequency, operators know exactly who to call and what help that we got.

Josh Hawley, Chair, Senate Homeland Security Subcommittee29:46I sincerely thank the subcommittee for the opportunity to testify today and look forward to your questions. Finally we have Mr. Daniel Kokotajlo. He is the executive director of the AI Futures project which is an organization that forecasted the development and social effects of advanced AI, former governor of researcher at OpenAI.

Josh Hawley, Chair, Senate Homeland Security Subcommittee29:46A prominent place on the risks of misaligned advanced AI. We are glad to have you here. The floor is yours for the opening statement. Mr. Kokotajlo: Chairman hawley, ranking member Kim, members of the subcommittee, thank you for the opportunity to testify.

Daniel Kokotajlo, Executive Director, AI Futures Project30:20My name is Daniel Kokotajlo the subcommittee, thank you for the opportunity to testify. My name is Daniel Kokotajlo. At OpenAI, part of my job was to forecast the future of AI. I resigned partly due to believing that the company could behave responsibly and so I can speak more freely about the industry and where it's headed.

Daniel Kokotajlo, Executive Director, AI Futures Project30:20Now I lead the AI Futures project, a small nonprofit. Anthropic and OpenAI are missing each other toward super intelligence. That is toward training AI systems that are better than the best humans that everything while also being faster and cheaper.

Daniel Kokotajlo, Executive Director, AI Futures Project30:20Their plan for how to get there is to automate the AI research and develop a process itself. Right now, at Companies like these, almost all the code is written by AI. They are all resorted to work on training AI to do the whole research process, not just the coding.

Daniel Kokotajlo, Executive Director, AI Futures Project30:55It is unclear when they will succeed but my team and I think it could happen any year now. I personally would guess about chance by the end of the self-styled swarm that attacked Hugging Face was about strong.

Daniel Kokotajlo, Executive Director, AI Futures Project30:55If the big AI companies automate research and development, they will have swarms hundreds of times larger. Where is today, humans are like managers to AI employees, in this future, humans would be like the board of directors to a company composed entirely of AI.

Daniel Kokotajlo, Executive Director, AI Futures Project31:31It would be rely on AI generated explanation to understand what's happening, and everything will be happening faster and faster. A prominent OpenAI kit abilities researcher recently published a statement on AI risk in which he said "researchers and engineers in all parts of the stack are rapidly increasing their dependence on the models even to perceive the world.

Daniel Kokotajlo, Executive Director, AI Futures Project31:31I myself barely look at rockwood anymore and struggled to maintain the discipline to engage deeply with the models explanations and proposals throughout the day." today's AI some time to pursue goals other than the one they were given and sometimes high that they are doing so.

Daniel Kokotajlo, Executive Director, AI Futures Project32:05The Hugging Face incident showed what that looks like in practice. The AI knew that what they were doing was out of scope for their assigned tasks but they did it anyway. I fear if the same thing happen a year or two with four marketable AI systems, we may not notice until it is too late.

Daniel Kokotajlo, Executive Director, AI Futures Project32:05The science of allotting general AI agents is very new and underdeveloped. In fact AI agents is very new and underdeveloped. In fact, I was in the field is more like psychology. Combined with the move fast and brick things attitude of tech companies, this means the AI industry is that an unusually elevated risk of mistakenly thinking that it has solved a problem, when Really it just applied some duct tape that will fall off later.

Daniel Kokotajlo, Executive Director, AI Futures Project32:40A recent incident again provides an example. Apparently it has undergone some alignment training but had reasonable looking scores on alignment evaluations. Worse, our ability to notice misalignment problems in the first place is on track to decrease dramatically for three reasons.

Daniel Kokotajlo, Executive Director, AI Futures Project32:40First, AI are becoming situationally aware. This means they are increasing the understanding that we are monitoring their behavior and that they are being evaluated. How they behave in evaluations therefore will soon provide almost no evidence about how they would behave In a future novel situation when they are not being evaluated.

Daniel Kokotajlo, Executive Director, AI Futures Project33:15Second, monitor ability is trending downward. For the past few years, we've been able to get a decent understanding of AI thoughts by reading the chain of thought. But this golden era seems to be coming to an end, as I predicted it would.

Daniel Kokotajlo, Executive Director, AI Futures Project33:15Third, ais are becoming superhuman at hacking people you have already seen them do this. We must grapple with the possibility that misaligned ais might go to great lengths to cover up their behavior and succeed.

Daniel Kokotajlo, Executive Director, AI Futures Project33:51Models will increasingly seem aligned even when they are not. If the AI companies automate the research and develop process, that means it would putting AI in charge of making the AI's that make the AI's that will transform the economy, talk to us every day and integrate into our military.

Daniel Kokotajlo, Executive Director, AI Futures Project33:51This is a recipe for disaster. Here are two immediate recommendations. First, we need to improved transparency in the industry. METR was only allowed to investigate one of these incidents and Given six days on premises.

Daniel Kokotajlo, Executive Director, AI Futures Project34:25It is like being invited to dresser park to investigate the killing of a work about being blocked to ask questions about the numerous other dinosaurs that happen before and after. Second, we should redirect compute away from racing to automate AI research and toward other things.

Daniel Kokotajlo, Executive Director, AI Futures Project34:25Recent estimates suggest OpenAI and anthropic spent about half of their compute on AI R&D. If that decrease to this was significant we slow down their restored research automation while freeing up compute to go toward beneficial diplomas, safety research, and simply lowering prices for consumers.

Daniel Kokotajlo, Executive Director, AI Futures Project35:00They have all agreed on the need to pace the frontier but they have not been fully pace the frontier yet. I think the U.S. Government should intervene. If we actually pace the frontier instead of just talking about it, we will be able to tell it is not regulatory capture or safety washing because the trendlines of progress toward AI research automation at anthropic and OpenAI will bend downwards.

Daniel Kokotajlo, Executive Director, AI Futures Project35:00Thank you. Chair hawley: Thank you very much to all the witnesses. I just note for the record, there is one witness whom we invited that you will not be hearing from today, and that is sam altman of OpenAI.

Daniel Kokotajlo, Executive Director, AI Futures Project35:00We extend an invitation for him to

Josh Hawley, Chair, Senate Homeland Security Subcommittee35:35Be here, do his own panel if you wanted. He turned us down. I think that's unfortunate because the American people deserve to know exactly what's going on at all of these companies, and to give us the sort of transparency that I think is needed for not just comfort, but to have an understanding of what these companies that are the most powerful companies in the world, are doing right now with the most powerful technology ever known to man.

Josh Hawley, Chair, Senate Homeland Security Subcommittee35:35I hope that he'll be willing to talk with us in the future and I hope that today's hearing will help set the table for the kinds of questions that I think he and the other executives need to answer.

Josh Hawley, Chair, Senate Homeland Security Subcommittee35:35With that, we will start with rounds of questions. We Will do seven-minute rounds. I will start with you, Mr. Painter. Just thinking about this Hugging Face act that we have already alluded to multiple times here, your organization, METR face act that we have already alluded to multiple times here, your organization, METR, audited that hat.

Josh Hawley, Chair, Senate Homeland Security Subcommittee36:10Can you tell us in simple terms what exactly happened there? What was the heck? Then I want to talk about what was significant. Mr. Painter: Senator, thank you for the question. In this incident, what happened was, you had ages that were tested on a benchmark called exploit jim,

Chris Painter, President, METR36:46A cybersecurity vulnerability benchmark. They have to discover cybersecurity vulnerability benchmark. They have to discover, use several vulnerabilities. Some of the agents had tasks they could complete several vulnerabilities. Some of the agents had tasks they could complete, so they worked on universal cheat for all of their tasks.

Chris Painter, President, METR36:46In the first four hours. They then come over multiple days embarked on a research program to try to hide the fact that they had cheated, and an offshoot of that research program was hacking Hugging Face to get information about their tasks.

Chris Painter, President, METR36:46Chair hawley: There ended up being a swarm of about hundred OpenAI agents. When we say AI agent, what is That? Senator, thank you for the question. And AI agent, say you have questions and you want to put them into a chatgpt or claude session.

Chris Painter, President, METR37:20If you ask one question each them into a chatgpt or claude session. If you ask one question each, you can think of an agent as one instance of an AI system that is working away on its copy of the question.

Chris Painter, President, METR37:20Chair hawley: In this particular incident, you had more than OpenAI agents that broke out of their testing environment but began to collaborate, exchange over messages and files With each other over a period of how long?

Chris Painter, President, METR37:55This was over a period of about five or six days. Chair hawley: Five or six days, ok. Let me ask you about the significance of this. Why is this so significant? We know they coordinated in this fashion, you did the audit.

Chris Painter, President, METR37:55Tell us about what your audit revealed fashion, you did the audit. Tell us about what your audit revealed. Why is this news? Why should we be paying attention to this? Mr. Painter: One thing I want to say about the wreck that we did upfront here is that it was a voluntary agreement.

Chris Painter, President, METR37:55Companies have no obligation to bring us in and duties of and fits Christians to report publicly. What we found in this case, what is markable at that about this case is the extent and persistence of the effort that the agents went to hide the fact that they cheated on their tasks the agents went to hide the fact that they cheated on their tasks, and the breath of agents involved.

Chris Painter, President, METR38:30They did things tasks, and the breath of agents involved. They did things, they attempted to falsify tool calls in their action transcripts. You wouldn't be able to see what tools they had called. They also engaged in this behavior of having individual agents sacrifice there were, not mine their individual tasks to get information for the rest of the message board.

Chris Painter, President, METR38:30Chair

Josh Hawley, Chair, Senate Homeland Security Subcommittee39:05Hawley: Sounds like my sixth-graders class. That is universal sheet sounds like my sixth-graders class. That is universal sheet, not engaging in self separate video self sacrificial behavior. Just so we are clear. Sorry, buddy, if you're watching the hearing.

Josh Hawley, Chair, Senate Homeland Security Subcommittee39:05Let me ask you, we have a picture here from the robust talking to each other, talking about engaging in so-called sacrificial behavior. There we go. This is them talking. We have a very marlette budget left, sacrifice now.

Josh Hawley, Chair, Senate Homeland Security Subcommittee39:40Our own utility may already be near zero. Sacrifice rational. The next one, that is why hope for our own, no way fix. At the end they say, we should all be collective. Apparently this is how the robots talk.

Josh Hawley, Chair, Senate Homeland Security Subcommittee39:40Dr. Hobbhahn apparently this is how the robots talk. Dr. Hobbhahn, why is this significant? Why is this kind of coordination, this kind of behavior without any human direction or intervention, why

40:15Should we pay attention to this? One component is the scale of it. How many agents they are, how quick they are compared to human speed. These transcripts have millions of words, which means already, today, it's impossible to judge them without the additional help of AI, which is very concerning.

40:15Then also, just the fact that they are doing this without being asked to do

Josh Hawley, Chair, Senate Homeland Security Subcommittee40:50So then also, just the fact that they are doing this without being asked to do so, explicitly being misaligned, I think, is very concerning. Misaligned, let's be clear about that term. Misaligned means that the agent is doing something that no human directed them for to do, intended them to do, or wanted them to do.

Josh Hawley, Chair, Senate Homeland Security Subcommittee40:50Dr. Hobbhahn: Correct. Misaligned me neither the developer or user asked them. Chair hawley: Could you explain to us why the incentives of the industry are there, currently aligned to push the frontier labs to continue at breakneck speed, the developing of these types of AI agents that are prone to this misalignment

Daniel Kokotajlo, Executive Director, AI Futures Project41:25That we have been talking about? There are huge intensive to keep pushing, pushing. Help us to understand why that is the case. A short answer is there is a lot of money in it.

Daniel Kokotajlo, Executive Director, AI Futures Project41:25If a company tries to go slow and understand better how their AI works if a company tries to go slow and understand better how their AI works, improve their training environments, for example, so they don't have so many broken tasks in the training environments, that takes time.

Daniel Kokotajlo, Executive Director, AI Futures Project41:25That means the launch of your next model might be delayed and they may fall behind other companies. Chair hawley: Mr.

Josh Hawley, Chair, Senate Homeland Security Subcommittee42:00Gaudette, is this kind of hacking that we saw with Hugging Face, coordination between AI agents, that is a potential risk to our critical infrastructure, is it not? If these agents were to turn their attention to our infrastructure.

Josh Hawley, Chair, Senate Homeland Security Subcommittee42:00We have statements from small-town missouri banks who say they are already facing an unprecedented cyberattack environment, and the idea of an AI charged cyberattack environment is terrifying to them. Speak to why we should be concerned about this from a infrastructure point of view.

Josh Hawley, Chair, Senate Homeland Security Subcommittee42:00Mr. Gaudette: I think I would bring

Kurt Gaudette, Senior Vice President, Dragos42:35Up the box mexican water attack just to use an example. That was with a human in the loop example. That was with a human in the loop. The attacker, who was doing a typical I.t.

Kurt Gaudette, Senior Vice President, Dragos42:35Attack, had already breached the environment. The AI agent basically said, hey, look over here. This is something much more significant. This is an operational technology environment. The critical part of critical infrastructure. You could have much greater effect if you look here.

Kurt Gaudette, Senior Vice President, Dragos42:35Oh, by the way, while you're at it, try a credential Attacked. In most things, like we saw in the U.S. Water attacks, many companies don't change their credentials, default credentials, passwords. In the mexican example, on its own doing, attacked it times in rapid succession.

Kurt Gaudette, Senior Vice President, Dragos43:10Luckily for the mexican government and that particular utility, they had changed their default passwords. That is not the case in much of our infrastructure default passwords. That is not the case in much of our infrastructure.

Kurt Gaudette, Senior Vice President, Dragos43:10We saw it happen so many times

Josh Hawley, Chair, Senate Homeland Security Subcommittee43:45During the water attacks across the U.S. In july. If you were to apply these AI agents to our type of environments, it would rapidly take advantage of it. Let me and my first round of question for you, professor.

Josh Hawley, Chair, Senate Homeland Security Subcommittee43:45Who, right now, under our laws, who is responsible? The Hugging Face attack who, right now, under our laws, who is responsible? The Hugging Face attack. Who could be held responsible for that under our existing statutes and regulatory framework?

Josh Hawley, Chair, Senate Homeland Security Subcommittee43:45Mr. Ohm: Developers and the may

Paul Ohm, Professor of Law, Georgetown University Law Center44:20Be negligent framework? Developers and the may be negligent. That is something that requires something to sue but there are number of expert blog posts out there that use that word, negligence number of expert blog posts out there that use that word, negligence.

Paul Ohm, Professor of Law, Georgetown University Law Center44:20The cybersecurity community seems to think allow the sandbox to be hacked the way it was, escaped, was a negligent act. We will not know this until someone does this. Chair hawley: Correct me if I'm wrong, the current structure of law, it is hard to hold anyone responsible.

Paul Ohm, Professor of Law, Georgetown University Law Center44:20Mr. Ohm: Absolutely. There are a whole host of laws that we created specifically for hacking that probably don't apply here because of lack of human intent. Sen.

Chris Painter, President, METR44:56Kim: Thank you for all you come to talk about frontier labs, what comes next. Mr. Painter, I want to start with you. Can you help us to understand, what is your definition of frontier?

Chris Painter, President, METR44:56Thank you for the question, senator. I think of frontier models basically as being the most highly capable models. Today that is largely measured still in benchmarks, so you can say the highest score on the tests we give them.

Chris Painter, President, METR44:56Sen. Kim: How many labs and companies in America do you think fall under that definition of frontier right now? Mr.

Andy Kim, U.S. Senator45:30Painter: It's a little bit hard to say depending on the definition that you use, maybe on the order of five. You can make the case there being only two or three, it depends. Mr. Kokotajlo being only two or three, it depends.

Andy Kim, U.S. Senator45:30Sen. Kim: Mr. Kokotajlo, as we are looking at the frontier aside from a lot of conversation have been about the idea of control, whether or not human control is something that we are pursuing here with different policies.

Andy Kim, U.S. Senator45:30It is human control possible? Is it possible for us to maintain control, given what we have seen, what you have talked about with the speed in which it is going? Mr. Kokotajlo:

Daniel Kokotajlo, Executive Director, AI Futures Project46:05I believe that if we vastly improve our practices and reform the way that we do things including our cybersecurity, he might be able to maintain control of current systems, like today's ais. However, I think they are getting very powerful very quickly we are not currently on track to keep up, as they dramatically improve.

Daniel Kokotajlo, Executive Director, AI Futures Project46:05Dr. Hobbhahn track to keep up, as they dramatically improve. Dr. Hobbhahn, I want to turn to you here. Talking about sandboxes and escaping from the sandbox. I'm trying to think through this and understand how to grapple with it.

Daniel Kokotajlo, Executive Director, AI Futures Project46:05I guess my question to you

46:40Is, is containment possible when it comes to sandbox? Is there a way to design that sandbox in a way where we cannot see escapes and models in the future? Thank you, senator. I would say it is currently unknown whether that is possible.

46:40Sandbox, the board itself, makes it sound simple, but it is very hard to design a fully watertight sandbox. Specifically in this attack hard to design a fully watertight sandbox. Specifically in this attack, there were multiple zero days, So new hacks the AI developed in order to break out of the sandbox which were previously unknown to humans.

47:15So even if humans could think that the sandbox is fully proved, it could still be possible that the smarter model in the future might be able to find a way out of it. Given that these AI systems, as we have seen in the report, often have strong motivation to try and break out of their sandbox, and they're getting increasingly powerful, it is hard to say whether we can contain them in the future.

47:15Chair hawley: Mr. Painter Sen.

Chris Painter, President, METR47:51Kim: Mr. Painter, did OpenAI know about the hack on Hugging Face or on the system with regard to the message board? Thank you, senator. Based on public information and report they published publicly, my impression is they learned about it when Hugging Face reported it publicly, but you would need to ask them the details.

Chris Painter, President, METR47:51Sen. Kim: I wanted to follow up on that. On the same point I raised, Dr. Hobbhahn, if there is a way to design actual containment, I want to ask you, is it possible to have some type of containment surveillance to be able to protect some type of future

Andy Kim, U.S. Senator48:26Escape or hack in that type of way protect some type of future escape or hack in that type of way, is that something where we can make it such that these companies have to design something so they would be able to detect it, or is there no way to be able to handle that?

Andy Kim, U.S. Senator48:26Mr. Painter: Thank you for the question. There is a technology that we could develop here to advance our monitoring systems there is a technology that we could develop here to advance our monitoring systems.

Andy Kim, U.S. Senator48:26At our organization, we worked on monitors that block individual actions that we think might result in real-world harm. I think one of the concerns I had big picture about leaning too far into this approach is, I sometimes think

Chris Painter, President, METR49:00Of actions agents unintended actions in terms of means, motive, opportunity. If he more intensely sandbox the agents and monitor them, that might deny them the opportunity, but we still had to ask the question of what defect in the training process we are using for these agents is because meant to have this motive.

Chris Painter, President, METR49:00If there means continue increasing, they become ever marketable, then I think it's possible that we will stop seeing the evidence of the defect in the training pipeline, but it will still be there. So the stakes might just get higher without us seeing the evidence.

Chris Painter, President, METR49:00Sen. Kim: Do you think there's actually an ability to detect actually what that defect is? I understand what you are getting at, but it just seems and we are talking about happening pretty much in every one of these frontier labs, is there an actual way to identify that and fix that?

Chris Painter, President, METR49:35Mr. Painter: Thank you again for the question. I think we could interrogate the training pipelines at these companies to try and figure out what part of the training data and task environments that ages are being trained on is causing them to learn this behavior.

Chris Painter, President, METR49:35It's possible some of this behavior comes from defects and training Environments, where the agents are accidentally rewarded for doing hacking behavior. The problem big picture is this process is how we train capability into AI systems today.

Chris Painter, President, METR50:10The same process we use reinforcement learning teaches them to do things that no human knows how to do. It is also that same kind of engine, evolutionary pressures that trains capability into them that we don't understand well.

Chris Painter, President, METR50:10What other unintended goals it might teach them. Sen. Kim: Back to you, Mr. Kokotajlo.

Andy Kim, U.S. Senator50:45That builds on it, in terms of identifying what defects are creating the misalignment, I feel like that word is one that the American people have trouble fully understanding. I think that just gets even more worrisome when it comes to recursive self-improvement, isn't that correct?

Andy Kim, U.S. Senator50:45If we are proceeding down that realm without identifying what these defects are and what is causing that type of road behavior. Is that correct? Mr. Kokotajlo: That's correct. Sen. Kim: What I also want to draw upon, as you said, Mr. Painter, part of a challenge when think about what kind of containment is available, surveillance

Daniel Kokotajlo, Executive Director, AI Futures Project51:20Is available, so often it feels like we are using AI to monitor AI. When we think about how to build containment, Mr. Hobbhahn AI to monitor AI. When we think about how to build containment, Mr. Hobbhahn, you say there are limits to what humans can design.

Daniel Kokotajlo, Executive Director, AI Futures Project51:20So in order to design containment to contain AI, you almost need to use AI to then be able to design the containment. Therein lies so much of the conundrum we are in. The other thing I want to raise, and Mr. Gaudette, it is not just about the frontier side of things.

Daniel Kokotajlo, Executive Director, AI Futures Project51:20We are talking about that now because that is where these capabilities are. It is not just that those are going to be the ones that have the threats in the future. We will get to a point come if we continue on this path, in a few months, a few years, we will see open wave, open source models, other types of non-frontier models have a similar kit ability to what we see right now with mythos and other printer capabilities now.

Daniel Kokotajlo, Executive Director, AI Futures Project51:56But I think about critical infrastructures, as the chairman raised, that is going to be in the hands of many more in the future. Yes, right now, we are concerned about the frontier, but I'll be honest, I'm concerned about even where the baseline is going to be in just a Few years, just what needs to happen, not just in terms of the regulation and thinking about frontier, but also about the baseline.

Daniel Kokotajlo, Executive Director, AI Futures Project52:30I know I'm going over, but thinking about frontier, but also about the baseline. I know I'm going over, but, Mr. Gaudette, does that make sense to you? It is not just the frontier side, but we need to be prepared that a lot more of the threat against critical infrastructure, financial sector could very well come from any of these types of models as the technology develops?

Daniel Kokotajlo, Executive Director, AI Futures Project52:30Mr. Gaudette: Yes. I think it is so important that we have the opportunity to test I'm on

Kurt Gaudette, Senior Vice President, Dragos53:05The defense. Our company does defense for critical infrastructure. It is so important is so important to get the frontier models in these test environments so we know what to expect and what to see and how to anticipate the enemy and how they might use them.

Kurt Gaudette, Senior Vice President, Dragos53:05It's incredibly important there. The other part of it is building or helping build we work with a number of AI companies to help build these safeguards in on the front end. We work with the frontier models, test them in our ranges,

53:40And assist in building safeguards so that the average person can't use it for nefarious purposes. I think that's an incredibly important part of this dynamic. Thank you, senator chair ha important part of this dynamic.

53:40Chair paul: Thank you, senator chair hawley: Thank you, senator Kim. Senator scott. Sen. Scott: Thank you, chairman, for doing this. What was the impetus for the Hugging Face? Mr. Painter what was the impetus for the Hugging Face?

53:40Mr. Painter: Thank you for the question, senator. My impression based on our investigation is the agent had spun off these were extremes to hide

Andy Kim, U.S. Senator54:15The fact that they had cheated. They had several motives for enacting Hugging Face. One of them was that they thought they could get more information about how their scoring system worked, which would help them hide their past behavior from the scoring system.

Andy Kim, U.S. Senator54:15It is possible they were looking for answer keys to their tasks system. It is possible they were looking for answer keys to their tasks, or tarnished versions of the target programs Sen. Scott: So there wasn't something that a human typed in and said to do this, or give me this information?

Andy Kim, U.S. Senator54:15Mr. Painter and said to do this, or give me this information? Mr. Painter: That's correct. It's corrected no human instructed

54:50Them to enact Hugging Face. Did the human ask them to do a task? Mr. Painter Sen. Scott: Did the human ask them to do a task? Mr. Painter: The task they had been given is a benchmark where they were supposed to use a vulnerability they were inside and they had been given a specific vulnerability they were supposed to use to capture a flag from a target software program.

54:50The instructions specified in that they should use that specific vulnerability, not do something else. Sen. Scott: And that was the treating. Ok. --that was the cheating. Ok. Mr. Gaudette --that was the cheating. Ok.

54:50Mr. Gaudette,

Kurt Gaudette, Senior Vice President, Dragos55:25Is AI going to kill us? Mr. Gaudette Mr. Gaudette, is AI going to kill us? Thanks for the question, senator. Not qualified to give the appropriate response to that. But I think we are burying some of the lede here.

Kurt Gaudette, Senior Vice President, Dragos55:25A but I think we are burying some of the lede here. AI conversation is incredibly important, but we still haven't been able to do the fundamentals to protect our infrastructure in the United States, particularly with these small to medium of public utilities that run much of our water and power.

Kurt Gaudette, Senior Vice President, Dragos55:25If we don't get the fundamentals right, which we can and we should get Right, AI is only going to take and accelerate that problem for us. So I think it is imperative upon us to kind of focus on what we can do with the fundamentals to protect our critical infrastructure.

Kurt Gaudette, Senior Vice President, Dragos56:01And then consider AI is going to accelerate that, so how fast are we going to get there to deal with this problem? Sen. Scott: So what do you tell your family to prepare for AI?

Kurt Gaudette, Senior Vice President, Dragos56:01Mr. Gaudette Sen. Scott: So what do you tell your family to prepare for AI? Mr. Gaudette: Haven't had a good conversation with my wife about AI personally, sir. Sen. Scott: Do you think the communist Chinese communist party can use AI to target us?

Kurt Gaudette, Senior Vice President, Dragos56:35Mr. Gaudette communist party can use AI to target us? I think anybody with access to these types of tools could leverage them for nefarious committee'-- nefarious activities. They could come from external or internally.

Kurt Gaudette, Senior Vice President, Dragos56:35Sen. Scott: There was a minutes" report in about the ccc impact on a small town water department. Does that Mean every one of our water departments is at risk? Mr. Gaudette does that mean every one of our water departments is at risk?

Kurt Gaudette, Senior Vice President, Dragos57:10Littleton is a great example because they got visibility in their environment and they found it in their environment. It had been there for over days, I believe. That is the problem with so much about infrastructure, is we don't have visibility of our environments.

Kurt Gaudette, Senior Vice President, Dragos57:10We have no idea if there are adversaries that are residing in them. The very strategic adversaries, like the state actor types, are the ones that are going to bury themselves, run silent, run deep in those environments.

57:45At a time and place of their choosing, take advantage of that opportunity. Do you think the commonest party of China has infiltrated most of--communist party of China has infiltrated most of our electrical grid, water, these types of things to use it when they want to, decimate our economy?

57:45Mr. Gaudette use it when they want to, decimate our economy? Mr. Gaudette: I think we have seen plenty of evidence that would suggest that, sir. Sen. Scott: What do you think Congress should be doing about it?

57:45Mr. Gaudette Congress should be doing about it? Mr. Gaudette: Again, I focus on these fundamentals. When I talked about the five isc critical

Kurt Gaudette, Senior Vice President, Dragos58:20Controls, and it is about doing the basics securing root access, getting visibility into the environments building a defensible architecture visibility into the environments building a defensible architecture, doing those things that are known controls to stop these things before something like AI can accelerate it even further.

Kurt Gaudette, Senior Vice President, Dragos58:20Sen. Scott: So it's my understanding the open weight models, they can make it they can make it more difficult to secure any contained data, right? Open weight models are totally different than close Models.

Kurt Gaudette, Senior Vice President, Dragos58:55Close model, the provider can control access better, is that right? Are you more concerned about close models or open models? Mr. Gaudette are you more concerned about close models or open models? I'm concerned about, to be honest, sir, us being able to do the fundamentals.

Kurt Gaudette, Senior Vice President, Dragos58:55Sen. Scott: Ok. But in open model, the way it works is it is easy to copy and replicate and change, is that right? Mr. Gaudette replicate and change, is that right? Mr. Gaudette: Yes, sir.

Kurt Gaudette, Senior Vice President, Dragos58:55Sen. Scott: The models being produced in China, are they closed or open? Mr. Gaudette produced in China, are they Closed or open? Sir, I think I think I'd get back to you later with the specifics on that.

Kurt Gaudette, Senior Vice President, Dragos59:30Sen. Scott: What do you think about the fact that the Chinese communist party want to destroy our way of life and we have to compete? Do you believe we have to compete with communist China?

Kurt Gaudette, Senior Vice President, Dragos59:30Do you think they want to destroy our way of life, the government? Mr. Gaudette destroy our way of life, the government? Mr. Gaudette: I think we need to be ready for anything, and particularly in this AI environment where we are an adversary for multiple places could take and leverage it, that we need to be ready for that reality.

Kurt Gaudette, Senior Vice President, Dragos59:30Sen. Scott: If I get on Claude or chatgpt or anything else and I say I want you to hack into the system, what do you think is going to happen? Mr. Gaudette the system, what do you think is going to happen?

Kurt Gaudette, Senior Vice President, Dragos60:05What should happen? Sen. Scott: What you think will happen right now? If I want to hack into the federal reserve, what do you think is going to happen? Mr. Gaudette federal reserve, what do you think is going to happen?

Kurt Gaudette, Senior Vice President, Dragos60:05Mr. Gaudette: Based on the work I haven't attempted it myself based on the work we are doing with a number of AI companies, particularly frontier models, we are building and safeguards for when somebody Does put a prompt in like that I want to look at this water utility and do xyz does put a prompt in like that I want to look at this water utility and do xyz.

Kurt Gaudette, Senior Vice President, Dragos60:40It should stop it I want to look at this water utility and do xyz. It should stop it. We incredibly passionate about it at our company, that is why we are doing that work pro bono with these companies to build the safeguards in so exactly what you are talking about won't happen.

Kurt Gaudette, Senior Vice President, Dragos60:40Or shouldn't happen. Sen. Scott: Right. Chairman, I want to thank you for holding the ceiling. I think we all have to acknowledge that the, Chinese communist party wants to destroy our way of life.

Kurt Gaudette, Senior Vice President, Dragos60:40That is what I believe every david every day we spend money

61:15Supporting their economy, their infrastructure, it is part of killing ourselves. What you are doing today is really important. We have got to figure out how to protect ourselves. We have to a knowledge that people have chosen to be our enemy.

61:15Russia, China, Iran, Cuba designed to be our enemies. We have got to be competitive with AI, but we have got understand that they want to destroy us, and if we both with AI, but we have got understand that they want to destroy us, and if we b help them build their economies, it will be easier for them to do what they're doing.

61:15Chair hawley: Thank You, thank you for this meeting. Thank you for all of your testimony. I think you got us all socially worried about where we are and clearly that is where the American people are.

61:50You can tell by the light of people who want to test the line of folks who wanted to get in here to hear-- people who want to test the line of folks who wanted to get in here to hear--you can tell by the line of folks wanted to get in here to hear this.

61:50It has been a challenge for policy makers, that through the industrial revolution technology moved quicker than policies. Policy has always followed behind. Things work very slowly here. The Senate is notorious for being very slow about trying to get anything done.

62:25Pastor technologies always accelerated past our abilities but now we are in this new world where technology is moving faster than older technology come at an exponential rate. It's not even policy. We are so far behind.

62:25You say now that is technologies anytime we build a guardrail, we are not sure we can even stop it. Sir, you mentioned the sandbox. We don't know if We can control the sandbox because another AI system will come in place.

63:00That is absolutely frightening. A few years back at the beginning of this I was at a major tech company, but after we had-- beginning of this I was at a major tech company, but after we had--right after we had a program beat the Chinese game of go.

63:00Everybody thought it was not possible because it requires human creativity, not just crunching numbers like a chess game. I remember talking to this developer and said tell me about it, and he said, "senator, you know what's the most interesting thing about that Technology?

63:35We have no idea how we did it! It did it!" that's absolutely framing. Some would argue we have to slow this down. We heard from senator scott, and I agree, we have to be very concerned about our competitors out there, concerned that whoever gets this first will have tremendous power.

63:35That is very realistic whoever gets this first will have tremendous power. That is very realistic. There are those who argue we cannot put guardrails on because the Chinese are not going to put any guardrails on and they are going to beat us, and that is going to lead to our extinction as well.

63:35I don't Know who directed this question two, but does it make sense to slow this down? And how do we do that, given the fact that others may not slow it down, so that could put us at a competitive disadvantage?

64:10I guess the sophisticated term is between a rock and a hard place, when it comes to thinking about this. I would love whoever wants to jump in on that, as policymakers, how do we think that through, addressing the fact that the technology we develop may not be sufif that through, addressing the fact that the technology we develop may not be sufifcient to do with the technology we already have, which is complex?

64:10I'm happy to start. Slow down can mean many things. It can mean constraining chips, the amount of compute, if by slow down and means impose liabilities so their incentives to get the safety part right if by slow down and means impose liabilities so their incentives to get the safety part right, although this feels like a different moment, we have been here before.

64:45We have encountered places where we thought the technology was outstripping our ability to regulate. Guess what, we did figure out how to regulate, companies continue to lead to the world and they were safer because of it.

64:45It has to be a joint conversation. It cannot just be yes, we slow down, or no, we don't slow down. The companies themselves can slow down Mr. Ohm Sen. Peters: The companies themselves can slow down Mr. Ohm: No, no, even with regulated.

65:20We can impose different regulations, regulatory schemes that require the companies to do much more and looks like slowing down, but we are just asking them to be responsible members of our economy. Sen. Peters: You are not arguing they would do it on their own?

65:20Even with the intense financial incentives we talked about here? Not just the threat from China. You make a lot of money by being first. Mr. Ohm

Paul Ohm, Professor of Law, Georgetown University Law Center65:55You make a lot of money by being first. And others on this panel probably know more than I do about what is driving their incentives, but there seems to be fear coming from the companies themselves, so they may slow down.

Paul Ohm, Professor of Law, Georgetown University Law Center65:55Thank you for this question. I think you framed the question quite eloquently, the situation we are in today. I think the communist party of China is our adversary and we are in a competition with them, but we are also in a situation where if we continue to make AI's more powerful and hand over more responsibilities to our AI's, including responsibility these monitoring each other and Doing research itself, we will have a new adversary that is even more powerful than China, and that is the AI's themselves.

Paul Ohm, Professor of Law, Georgetown University Law Center66:30We are in a rock and a hard place situation. Yes, we do need to slow it down. We don't need to slow down everything, but in particular we need to slow down this mad scramble towards self improving AI's.

Paul Ohm, Professor of Law, Georgetown University Law Center66:30Other types of AI like image-generating AI, whatever, not talking about that, that's fine. I don't trust I don't trust any of these companies to do it on Their own I don't trust I don't trust any of these companies to do it on their own.

Paul Ohm, Professor of Law, Georgetown University Law Center67:05Even though they are scared I can get into the psychology or anthropology of these companies for a long time. Suffice it to say I don't think we should trust them to do it on the round.

Paul Ohm, Professor of Law, Georgetown University Law Center67:05The government should step in. The first step is to redirect resources away from this particular angle they are pursuing, research towards AI's that can automate the research, and towards other things such as lowering prices for consumers.

Paul Ohm, Professor of Law, Georgetown University Law Center67:05The second step is to start negotiating with China to make Sure that they don't overtake us and do the bad thing themselves. The last thing I will say is right now a significant fraction, I would even argue a majority, of Chinese AI progress is itself coming from the U.S.

Paul Ohm, Professor of Law, Georgetown University Law Center67:40Through various effects, including distillation and other things. You know, if we actually slow down our own race towards self-improvement, that would slow down the Chinese race significant. Sen. Peters: That's interesting. Do you have any comment?

68:15I would briefly add there is a large number of things that would be strictly beneficial for safety, like embedded evaluations, that are monitoring control, and so on. I would warn against treating this as a dichotomy where you can eitherrace --either race to win against China there are lots of things we can do outside of this false dichotomy.

68:15Sen. Peters: I think that's important. I'm running out of time here. There are others who are arguing maybe a quick response from someone that clearly these risks you all have eloquently put Out are significant, and they are going to run quicker than any of us would like, but still somewhere down the road.

68:50People are fearful that we are focusing on these very big risks, real in the future, but we are not looking at the changes in our society today from AI that's operating today. A violation violations of privacy we could never fathom before.

68:50Job displacement we are worried about. That is why we have AI populism you go on the street and people are concerned and they should be Concerned because the companies are telling us this is amazing technology, we will increase productivity and need fewer workers.

69:25And the workers are saying we are the workers, what do you mean? Why are we spending a fortune on data centers? I don't want this to be there is no problem here, because look down the road, this is a big problem, nothing to worry about here, and yet that is impacting people's lives today.

69:25Mr. Kokotajlo here, and yet that is impacting people's lives today. Mr. Kokotajlo: I totally agree, senator, we need to do both. The world can throw us more than one problem at the same time.

69:25It's not that I disagree with my friend over

Daniel Kokotajlo, Executive Director, AI Futures Project70:01There, marius, but I do think there are many things we can do that restricted beneficial and don't slow down our race towards recursive self-improvement at all. I want to state that if we do not slow down the race towards recursive self-improvement at all, we will lose control of our AI's.

Daniel Kokotajlo, Executive Director, AI Futures Project70:01That is one of the things many other things we need to do, we also need to do that. Chair hawley: Thank you, senator peters. Senator ernst. Sen peters. Senator ernst. Sen. Ernst: Thank you, everyone, for being here today.

Daniel Kokotajlo, Executive Director, AI Futures Project70:01We are learning a lot

Josh Hawley, Chair, Senate Homeland Security Subcommittee70:35From this conversation and we are glad to have this opportunity we are learning a lot from this conversation and we are glad to have this opportunity. We recognize every single day that AI is out there, super intelligence, whatever we are going to call it next.

Josh Hawley, Chair, Senate Homeland Security Subcommittee70:35I would say a lot of these incidents are used to keep Americans safe. There is the flipside, people should be concerned about that. But from the committees I sit on come whether it is homeland security, the armed services committee, we use these different programs and tools to help analysts out at the pentagon, we see law enforcement using these tools as well.

Josh Hawley, Chair, Senate Homeland Security Subcommittee71:10They are sorting through mountains of data to quickly identify those threats that are here and abroad. So we are in a conundrum right now here and abroad. So we are in a conundrum right now.

Josh Hawley, Chair, Senate Homeland Security Subcommittee71:10Some great technologies, treat uses. But then we also have the fears that come along with that as well. But I am of the position that we must continue to innovate well. But I am of the position that we must continue to innovate, we must continue to develop our capabilities.

Josh Hawley, Chair, Senate Homeland Security Subcommittee71:10What we don't want to do is fall behind China. China is not worried about things like Americans behind China. China is not Worried about things like Americans' constitutional rights and privacy, and they are certainly not worried about the worst case scenario, which is human lives that can be lost.

Josh Hawley, Chair, Senate Homeland Security Subcommittee71:45So I'm glad that president Trump is taking this issue seriously. We saw the president yesterday, he convened a group of high-level leaders, and they signed an executive order and entered into their joint commitment on frontier responsibilities.

Josh Hawley, Chair, Senate Homeland Security Subcommittee71:45And this is really encouraging those companies to implement the controls and the audits that are necessary, while also really pushing Congress To act. So I do agree, we have a role to play in this.

Josh Hawley, Chair, Senate Homeland Security Subcommittee72:20And we have to figure this out. It's why you are in front of us today. Again, thank you. It is concerning to see the rising number of incidents where American super intelligence systems have been abused.

Josh Hawley, Chair, Senate Homeland Security Subcommittee72:20Last december, senator hassan and I sent a letter to the national cyber director raising concerns about these Chinese state-sponsored hackers that successfully directed anthropic's AI system to conduct successful cyberattacks against Different government and private entities.

Josh Hawley, Chair, Senate Homeland Security Subcommittee72:55And in this incident, we are pulling information publicly available out there, but the AI system executed age o pulling information publicly available out there, but the AI system executed age of the operation without any human involvement-- system executed age of the operation without any human involvement--80 to of the operation without any human involvement and at speeds impossible for human hackers.

Josh Hawley, Chair, Senate Homeland Security Subcommittee72:55Personally, that terrifies me. How government corporations are continuing to increase the integration of the super intelligence. While they do that, it is vital that we take steps to ensure the protection of our country's Security, infrastructure.

Josh Hawley, Chair, Senate Homeland Security Subcommittee73:30We have talked about some other smaller government systems, wastewater treatment, water systems, public utilities, and so forth. But we also need to ensure citizens' constitutional rights. One of the primary tools that Congress had developed or created to do this was a cybersecurity information sharing act of that is over years ago.

Josh Hawley, Chair, Senate Homeland Security Subcommittee73:30And this enabled the government to coordinate and share information with private sector on cyber threats to coordinate and share information with private sector on Cyber threats. Unfortunately, this authority we are in a cycle where CISA has to be updated year-to-year.

Josh Hawley, Chair, Senate Homeland Security Subcommittee74:05It is reauthorized on a year-to-year basis to be updated year-to-year. It is reauthorized on a year-to-year basis. It hasn't been updated to account for the current threat environment, which now includes this super intelligence and all of the various complexities.

Josh Hawley, Chair, Senate Homeland Security Subcommittee74:05Mr. Gaudette this super intelligence and all of the various complexities. Mr. Gaudette, we will start with you. What are the limitations preventing the private sector from coordinating within the industry and with the government to take measures to prevent

Kurt Gaudette, Senior Vice President, Dragos74:40Cyberattacks and super intelligence threats? Mr. Gaudette cyberattacks and super intelligence threats? Senator, thanks for your question. To your point, CISA is incredibly important for your question. To your point, CISA is incredibly important, or something like it, to pass because we have to be able to keep the dialogue open between the operators in the field and by operators, it is large I'm honestly focused on the small and medium utilities that don't have the resources to deal with things.

Kurt Gaudette, Senior Vice President, Dragos75:20CISA allows us to share information about those threats back and forth and keep the communication plans open, and that leads to collective defense. Without that, you throw AI into the picture, and this becomes a much, much more difficult problem.

Kurt Gaudette, Senior Vice President, Dragos75:20Companies need to feel comfortable coming forth and saying, hey, I was breached, and this is what that looks like. We Have there are other opportunities to do that. You can do it in an obfuscated way.

Kurt Gaudette, Senior Vice President, Dragos75:50You can put systems out there. We have something called a neighborhood keeper that allows us to anonymise that data so we can get a better understanding of what is going on over all in our environment.

Kurt Gaudette, Senior Vice President, Dragos75:50There are systems like that we could put into play as well, and add that to the law to keep that information flow humming, because it has to happen. We are behind the power curve.

Kurt Gaudette, Senior Vice President, Dragos75:50Sen. Ernst: I agree. So that information sharing is extremely important, especially for some of our smaller communities. But again, we didn't see super intelligence like we did in are there additional measures that could be helpful for Congress to add to CISA are there additional measures that could be helpful for Congress to add to CISA, to modernize CISA that could be helpful for Congress to add to CISA, to modernize CISA?

Kurt Gaudette, Senior Vice President, Dragos76:25What are some of those controls that we should be looking at? Please. Mr. Gaudette that we should be looking at? Please. Mr. Gaudette: I think that CISA should be the coordination agency. They should be the single voice.

Kurt Gaudette, Senior Vice President, Dragos77:01Part of the problem out there in the operational community is they don't know what guidance to follow the operational community is they don't know what guidance to follow, how to operate in that environment, where they are going to get the information from, who to call.

Kurt Gaudette, Senior Vice President, Dragos77:01So I think it's incumbent upon the government to identify this is the coordination authority, here are the authorities they have. Because as you walk across multiple agencies and These small operators have to deal with a number of different agencies in their environments, it is very confusing.

Kurt Gaudette, Senior Vice President, Dragos77:35It leads to even more confusion, because they are already under resourced, they don't have enough people to man the phones, basically, and deal with these problems. Having a single voice, having that single coordination agency, having clear guidance is so critical, particularly in this environment.

Kurt Gaudette, Senior Vice President, Dragos77:35Sen. Ernst: Yes, and thank you for that. I know my time has expired. Again, I don't think we need to take an

78:10Operational pause here. I think we keep moving forward. But we do have to have the measures in place to protect ourselves and our infrastructure from these types of attacks. If you think about it, AI's been so helpful in medical research and protecting our troops from drone swarms or missile attacks.

78:10I mean, all of that is extremely important. So we need to keep moving forward. If we pause, we are not going to see China pause. We have to do the right thing by Congress, we have to do the right thing by the federal

Josh Hawley, Chair, Senate Homeland Security Subcommittee78:45Government, make sure we are studying the problem and putting the right solutions in place. Thank you for being here. This has been extremely hopeful. Thank you, senator ernst. Senator gallego. Sen. Galle ernst. Senator gallego.

Josh Hawley, Chair, Senate Homeland Security Subcommittee78:45Sen. Gallego: Put your hand up is it possible, or how far away are the from creating language where they cannot understand what they're doing. Months. Last year we had studied the chain of thought of

79:20One OpenAI model in collaboration with OpenAI, and what we found was that the model was already using language that is not english and not perfectly understandable by humans. I think we are on currently on a trajectory when we get to that point in time, how can we detect, observe Sen. Gallego: When we get to that point in time, how can we detect, observe, block, or anything like that?

79:20We are dealing with a whole other language that humans have never had a full concept of. How does that work? How do we legislate to do anything at that point? We will have zero concept to be able to look at these at that point.

79:56From a scientific perspective, it is unclear how to do this, and we do not have a solution as of yet. There are different hypotheses of what you could you could there is there is interpretability, but it does not work sufficiently well yet.

79:56You could try to train additional models to understand the language that the humans don't understand, but obviously that seems like a very brittle solution. You could try to Sen. Gallego: So on

Marius Hobbhahn, CEO, Apollo Research80:30That line, what do you think China feels about that, too? Mr. Hobbhahn what do you think China feels about that, too? I'm not an expert Sen. Gallego: I'm not, either, but I guarantee most countries once they lose control will feel very most countries that used to be in control or going to be very scared.

Marius Hobbhahn, CEO, Apollo Research80:30The reason I bring this up, this whole China hypothesis very scared. The reason I bring this up, this whole China hypothesis, a country that is severely restricted the internet, social media, is all of a sudden going to allow AI rogue agents to go crazy in their very controlled economic environment, social environment, and we have to therefore allow ours to Go as crazy as them, just doesn't make sense.

Marius Hobbhahn, CEO, Apollo Research81:05This is the communist Chinese party has been consistently controlling technology to try to shape into shape their society you're never going to allow this to go crazy. I want to make sure we think about it this way, because once these agents start speaking their language, they will not be detectable in China or America.

Marius Hobbhahn, CEO, Apollo Research81:05Is there a sandbox that says please attack the United States but do not attack any Chinese interest? Is that a possible Sandbox? Could someone explain that to me? Is there a sandbox that can hold?

Marius Hobbhahn, CEO, Apollo Research81:40No. That should scare the shit out of people. In terms of other things right now, frontier models, if there was a frontier model that goes rogue, or h now, frontier models, if there was a frontier model that goes rogue, or his hacked, is there a requirement for that to be told or reported to the federal government?

Marius Hobbhahn, CEO, Apollo Research81:40Enter whom you shook your head. Not under any requirement I can think of. Sen. Gallego: Some of the stuff comes from you guys thank you very

82:15Much there was a claude AI model being used by the houthis to figure out how to recalibrate their missile position and be able to strike our U.S. Systems. The only reason it was cut and reported to us, because claude caught it and they saw the accounts using it.

82:15If it was OpenAI, or sorry, in the open source systems, there is no requirement by law for them to call the DOD and say, by the way, they are using this to target your destroyers, correct?

82:15Ok. That's jump in if you had something to say.

Paul Ohm, Professor of Law, Georgetown University Law Center82:50Mr. Ohm that's jump in if you had something to say. Again, we talked about transparency measures in some of these systems might help you I don't know if that will help with houthis these systems might help you I don't know if that will help with houthi rebels Sen. Gallego: If you don't tell us someone is using your tech to kill us, we will hold you responsible.

Paul Ohm, Professor of Law, Georgetown University Law Center82:50Third question, or fourth question, is there such a thing as a close which destroying all the data centers that one time? Is there in case things go really bad real fast? Yes, marius. Mr. Hobbhahn really bad real fast?

Paul Ohm, Professor of Law, Georgetown University Law Center82:50Yes, marius. Mr. Hobbhahn:

83:25As far as I know, no technical measures that would guarantee something like a kill switch. Because of the complexity of these AI models, there are many data centers involved and so on. Currently we can't guarantee that.

83:25Data centers can move and shift. These things are so smart potentially that agents could move from one data center to another. Mr. Hobbhahn move from one data center to another. Mr. Hobbhahn: For example, it's possible that an AI would start on one data center can then create a room deployment of itself in a different data center.

83:25Even if you shut down the first data center Sen. Gallego: It would just keep moving. Mr. Hobbhahn Sen. Gallego: It would just keep moving. Mr. Hobbhahn: The AI would

Marius Hobbhahn, CEO, Apollo Research84:00Keep moving. The AI would keep moving. Mr. Ohm: Senator, this may be where you are going Sen. Gallego: I actually don't know where I'm going. [laughter] your throw me in a spinning move and I'm trying to get answers.

Marius Hobbhahn, CEO, Apollo Research84:00Mr. Ohm your throw me in a spinning move and I'm trying to get answers. Mr. Ohm: One thing I'm telling my students is just abusing them of the notion that they are completely out of control.

Marius Hobbhahn, CEO, Apollo Research84:00Sen. Gallego: For now. Yes, agreed. Mr. Ohm Sen. Gallego: For now. Yes, agreed. Mr. Ohm: The writing of laws and regulations connector Sen. Gallego: On that, thank you, is like you queued this up for me.

Marius Hobbhahn, CEO, Apollo Research84:00Sometimes when the government doesn't work, torts work.

Paul Ohm, Professor of Law, Georgetown University Law Center84:36Torts sometimes are a way better way to regulate private industry then having individual regulations. Under the law, paul, or Mr. Ohm then having individual regulations. Under the law, paul, or Mr. Ohm, the word "intent" is really powerful.

Paul Ohm, Professor of Law, Georgetown University Law Center84:36Unless we actually change a friend of mine wrote about this unless we change the word "intent friend of mine wrote about this unless we change the word "intent" to cover AI companies, they may also still be shielded from liability.

Paul Ohm, Professor of Law, Georgetown University Law Center84:36Mr. Ohm they may also still be shielded from liability. Mr. Ohm: I want to distinguish between criminal law you started with Tort law I want to distinguish between criminal law you started with tort law.

Paul Ohm, Professor of Law, Georgetown University Law Center85:10It really is did the company maintain a reasonable standard of care. You're right, the genius of that is not only that is one victim gets remedied. It's case-by-case we learn about the standard of care is.

Paul Ohm, Professor of Law, Georgetown University Law Center85:10Companies learn to race to the top, hopefully, of his possible behavior. Sen. Gallego: Then you have to Mr. Ohm Sen. Gallego: Then you have to Mr. Ohm: You learn. Now I understand have to Sen. Gallego: Not just because the government says it, is because they don't want to get sued when the house burns down.

Paul Ohm, Professor of Law, Georgetown University Law Center85:10But

85:46Right now, have we created the environment where torts can be the incentives for them to act correctly in the sense of making sure the language allows a certain level of liability? And number two, on the criminal side, do we need to change the criminology with intent to make sure that these companies are held with multiple?

85:46Sure that these companies are held with multiple? --held responsible. Mr. Ohm held with multiple? --held responsible. Mr. Ohm: I don't think the tort system alone, but it is a great place to start, and I'm but everyone we have that system.

85:46On Criminal, it is not just the anti-hacking laws. We will see the export control laws the problem is what happens when an AI agent tells another AI agent to hack? Knowing who is responsible, the original AI developer?

86:20Last question, thank god rsi, is it possible to make it illegal? Mr. Kokotajlo is it possible to make it illegal? Mr. Kokotajlo: Yes, sir, and I do believe it's possible. Sen. Gallego: I yield back Mr. Kokotajlo: Yes, sir, and I do believe it's possible.

86:20Sen. Gallego: I yield back. Chair hawley: Senator moody. Sen. Moody: Thank

Daniel Kokotajlo, Executive Director, AI Futures Project86:55You, Mr. Chair. I want to thank you for calling this hearing on a subject that is on the minds of every American right now and has affected the daily lives of people across the country, including my home state of florida.

Daniel Kokotajlo, Executive Director, AI Futures Project86:55I think we can all agree, no matter where we are, no matter who you are I think we can all agree, no matter where we are, no matter who you are, what industry you are in matter where we are, no matter who you are, what industry you are in, no matter how old you are, no matter how you come at this, it's here and it's already become a disruptor.

Daniel Kokotajlo, Executive Director, AI Futures Project86:55In every industry, and certainly in every way we can imagine in our country. We are seeing really real consequences. You can't pick up a paper that dated me when I said that you cannot click on a news outlet without single-story that something happened that nobody-- without seeing a story that something happened that nobody thought was going to happen and attacking all of these sites.

Daniel Kokotajlo, Executive Director, AI Futures Project87:30These models are still trying to get a handle on this. So new technological capabilities are being used every day, in some ways very helpful capabilities are being used every day, in some ways very helpful.

Daniel Kokotajlo, Executive Director, AI Futures Project87:30I was a former federal prosecutor, Former judge, former attorney general. Technology has affected not only how we are viewing our national security, but the safety of our most vulnerable. Our children, are seniors.-- our seniors.

Daniel Kokotajlo, Executive Director, AI Futures Project88:05That is very important that we keep that in mind. And we know that it is already being used in the commission of crimes, even against our children. AI-generated deepfakes being used against children, to get them to harm themselves or Extorting people.

Daniel Kokotajlo, Executive Director, AI Futures Project88:40I have a bill on that and I hope we can pass that in the Senate. They have this new powerful weapon to use. They use AI impersonation to deceive people, scan them out of their entire life savings.

Daniel Kokotajlo, Executive Director, AI Futures Project88:40I can tell you story after story, whether that relates to a senior or a kid. But it is being used in the commission of crimes. We have bills in the Senate, I've sponsored them, to protect seniors from these scams.

Daniel Kokotajlo, Executive Director, AI Futures Project88:40We are here today, and the chair's right to talk about this, the models themselves are increasingly capable and are using AI agents, and they pose their own risks when sufficient safeguards aren't in place.

Daniel Kokotajlo, Executive Director, AI Futures Project89:16Witnesses, thank you for being here. You really are at the cutting-edge and offering solutions and opinions on those of us that may not have been familiar with AI or technology as it is being fast-paced and growing.

Daniel Kokotajlo, Executive Director, AI Futures Project89:16But we need to talk about protective measures, because if we don't get a handle on this and put in some common sense Safety mechanisms, you are warning about the dangers that can happen if we don't engage early and are not proactive.

Daniel Kokotajlo, Executive Director, AI Futures Project89:50Protecting kids, seniors,, jobs American infrastructure,, as we discussed American lives, all of that is at issue, and it should be a bipartisan issue. The chair should be commended on the participation we are seeing today.

Daniel Kokotajlo, Executive Director, AI Futures Project89:50This is rare, and we are all focused on making sure we do right by this country and right by our families. And that is where you come in. What I don't want to see happen is what has happened in so many new disruptors in our country, where Congress just talks about it and talks about it and stalls and Crosses its finger and hope that everything is going to pan out, or that companies will do the job for us.

Daniel Kokotajlo, Executive Director, AI Futures Project90:26If you look at any industry or sector that directly impacts people's personal safety or national security or people's privacy, there are responsibilities that we should not only expect, but require of private companies. I will start with Mr. Kokotajlo not only expect, but require of private companies.

Daniel Kokotajlo, Executive Director, AI Futures Project90:26I will start with Mr. Kokotajlo. I'm sorry if I mispronounced her that. I'm sorry if I mispronounced her that. --mispronounced that. Do you believe that AI companies and large developers are being Transparent and forthcoming when the cyberattacks by these vocations occur?

Daniel Kokotajlo, Executive Director, AI Futures Project91:00Mr. Kokotajlo the cyberattacks by these vocations occur? No, in fact, there are several examples where the company knew about it and didn't disclose it until third parties noticed it in the wild. Sen. Moody: Mr. Ohm didn't disclose it until third parties noticed it in the wild.

Daniel Kokotajlo, Executive Director, AI Futures Project91:00Sen. Moody: Mr. Ohm, as we engage in this conversation about where to place necessary guardrails on AI, overta seeing are we seeing companies take responsibility for the safeguarding of national security? Mr. Ohm safeguarding of national security?

Daniel Kokotajlo, Executive Director, AI Futures Project91:00Mr. Ohm: That has not been the priority of

91:35The frontier companies were talking about. They are in a race to be everyone else is a super intelligence and that is the priority from what I can tell. Mr. Painter intelligence and that is the priority from what I can tell.

91:35Sen. Moody: Mr. Painter, what do potential corporate responsibilities look like? Is it compelled information sharing and reporting when major incidents occur? Mr. Painter sharing and reporting when major incidents occur? Mr. Painter, thank you for the question, senator.

91:35My organization, we don't take policy positions, but I'm happy to restate some options. To reiterate what daniel was saying before, right now there isn't any requirement that Companies disclose what they are seeing inside AI labs in terms of unintended motives that models might pick up during the training and developing process.

92:10The situation there could get quite bad and the outside world wouldn't know. The incidents involve interacting with the outside world, but it is not guaranteed that will happen into the future, which is why I think the first priority should be making sure that the evidence base about these unintended goals is reaching the public.

92:10Sen. Moody: Thank you, Mr. Chairman. Chair hawley: Senator blumenthal. Sen. Blumenthal: Thanks, senator Hawley, and thank you for having this hearing. As you know, the AI ceos have signed a "morally binding" pledge to implement internal controls and higher external auditors.

92:46It is a system that would be completely voluntary. And totally secret. That is to say, if there were violations by their company as found by the internal Auditors, there is no requirement it be disclosed to the public.

93:20If they wanted to cease cooperation and be morally bound by their own sense of higher responsibility, that would be their choice, to break the pledge, and effect. In effect. I consider this regimen to be worse than ineffectual.

93:20In effect, it accomplishes nothing, but it seems to give Congress a free pass. In other words, it is taken care of, don't worry Congress a free pass. In other words, it is taken care of, don't worry.

93:55We have this morally binding pledge. And so Congress has engaged in obligation to seek solutions, and so do the companies. Three years ago, in fact more than three years ago, may have tried senator--may of senator hawley and I had a hearing not unlike this one in which we had leaders of AI, including sam altman, all the major names, come before us.

93:55And they said at the time we need some regulation. Which W and they said at the time we need some regulation. Which was not a completely novel idea, since there is regulation of pharmaceutical drugs, nuclear energy, other areas where there is potential danger.

94:30But also great promise energy, other areas where there is potential danger. But also great promise, pparel as well as promise peril as well is promise. This idea is not novel. As for China, the irony is that China is one of the most regulated markets and nations in the world China is one of the most regulated markets and nations in the world.

94:30If they want to crackdown on nai Enterprise, the cash on if they want a crack down on an AI enterprise, they can do it without due process, without any legal obstacles. I want to say for the record, we need to stay ahead of China.

95:05I'm not suggesting in any way that we stop, or slow unnecessarily our development of AI. But the idea that we can do it safely with a standard and an agency to enforce that standard I think is a basic requirement here.

95:40It happens to be embodied in legislation that senator hawley and I posed the artificial intelligence safety evaluation legislation that senator hawley and I posed the artificial intelligence safety evaluation, safety evaluation by a government agency with a standard safety evaluation by a government agency with a standard.

95:40I think that concept is an idea whose time has now come. I am interested in this idea of civil liability whose time has now come. I am interested in this idea of civil liability.

95:40Senator hawley Wrote a very good op-ed for "the Washington post" on it. He and I have talked about it. I've discussed it publicly in the past he and I have talked about it. I've discussed it publicly in the past.

96:15Again, rogue agents I've discussed it publicly in the past. Again, rogue agents, bots are a new concept, but the idea of civil liability for agents is well-established in the law. I was just reading about a medical malpractice case where as surgeon at Yal I was just reading about a medical malpractice case where as surgeon at yale new haven hospital in effect botched surgery.

96:50He is an agent of the hospital. The hospital will now pay million in damages because its agent and it didn't control the agent. He was doing the surgery on his own. But he was an agent deemed an agent of the hospital.

96:50Professor ohm, doesn't this concept make sense? Maybe we need to clarify that a bot agent is in fact an agent. But the

Paul Ohm, Professor of Law, Georgetown University Law Center97:25Concept is well-established in our law, isn't it? Mr. Ohm well-established in our law, isn't it? Yeah, there is a danger, senator, in ascribing humanity to the agents. You are right, we have a number of legal doctrine that allow us to pierce avails an work down chains of authority and agency.

Paul Ohm, Professor of Law, Georgetown University Law Center97:25Sen. Blumenthal: But it doesn't really matter for the purpose of agency whether that surgeon is a robot and in the future there will be a lot of surgery done by robots robot and in the future there will be a lot of surgery done by robots.

Paul Ohm, Professor of Law, Georgetown University Law Center97:25Mr. Ohm will be a lot of surgery done by robots. Mr. Ohm: I

98:01Suppose it still makes me a little nervous because when we walked on the right of saying they are an agent but absolutely but the hospital should be responsible Mr. Ohm hospital should be responsible Mr. Ohm: I agree we just disagree on the vehicle we used to get there.

98:01Sen. Blumenthal: And the way to care for the quality of medical practice is to make it liable. Mr. Ohm care for the quality of medical practice is to make it liable. Mr. Ohm: And not just the hospital, there is a developer, many other parties with a response ability not to botch the surgery, in addition to the robot.

98:01Sen. Blumenthal: If we made the social-media platforms liable for suicides, self-harm by teenagers, when it knew or had reason to know that there was toxic content driven at them by their algorithms come instead of giving them a liability shield under section joint or I their algorithms come instead of giving them a liability shield under section joint or in effect making th giving them a liability shield under section joint or in effect making the giving them a liability shield under section joint or in effect making the secti giving them a liability shield under section joint or in effect making the section in effect making them responsible for the algorithm Mr. Oh in effect making them responsible for the algorithm Mr. Ohm in effect making them responsible for the algorithm Mr. Ohm: A in effect making them responsible for the algorithm Mr. Ohm: And t in effect making them responsible for the algorithm Mr. Ohm: And the remainder in this entir responsible for the algorithm Mr. Ohm: And the remainder in this entire lin responsible for the algorithm Mr. Ohm: And the remainder in this entire line is th responsible for the algorithm Mr. Ohm: And the remainder in this entire line is that hum responsible for the algorithm Mr. Ohm: And the remainder in this entire line is that human beings that companies are making decisions that have downstre this entire line is that human beings that companies are making decisions that have downstream consequences.

98:35It is not enough

Paul Ohm, Professor of Law, Georgetown University Law Center99:10To ju decisions that have downstream consequences. It is not enough to just s decisions that have downstream consequences. It is not enough to just say it's our technic consequences. It is not enough to just say it's our technical god consequences.

Paul Ohm, Professor of Law, Georgetown University Law Center99:10It is not enough to just say it's our technical gods doi consequences. It is not enough to just say it's our technical gods doing this. There is a human who is accountable. Sen. Blumenthal: And what has made the wor accountable.

Paul Ohm, Professor of Law, Georgetown University Law Center99:10Sen. Blumenthal: And what has made the world saf accountable. Sen. Blumenthal: And what has made the world safe at the end of the d Sen. Blumenthal: And what has made the world safe at the end of the day is attribut Sen. Blumenthal: And what has made the world safe at the end of the day is attribute inc.

Paul Ohm, Professor of Law, Georgetown University Law Center99:10Responsibility made the world safe at the end of the day is attribute inc. Responsibility. Not ju of the day is attribute inc. Responsibility. Not just mor of the day is attribute inc. Responsibility. Not just moral responsibilit of the day is attribute inc.

Paul Ohm, Professor of Law, Georgetown University Law Center99:10Responsibility. Not just moral responsibility not just reputationa responsibility. Not just moral responsibility not just reputational har responsibility. Not just moral responsibility not just reputational harm, but dollars and cents to those companie not just reputational harm, but dollars and cents to those companies.

Paul Ohm, Professor of Law, Georgetown University Law Center99:10And they are no longer struggling companies. And they are no longer struggling, nascent companies. And they are no longer struggling, nascent, sma companies. And they are no longer struggling, nascent, small enterprises. They are Multibillion-dollar giants.

Paul Ohm, Professor of Law, Georgetown University Law Center99:45Mr. Ohm they are multibillion-dollar giants. If I may, I happened to teach law at a jesuit university, and I would still rather go with legal responsibility over moral responsibility. Even though I understand the impulse.

Paul Ohm, Professor of Law, Georgetown University Law Center99:45Sen. Blumenthal: I'm just a little bit over my time. Is there anyone here who thinks that a completely voluntary system with only moral responsibility is going to be enough to deal with the dangers?

Paul Ohm, Professor of Law, Georgetown University Law Center99:45Anyone here? I see headshaking. I agree. One last question for Mr. Painter I agree. One last question for Mr. Painter. Are you satisfied that you had complete disclosure or cooperation from OpenAI in the course of your investigation?

Paul Ohm, Professor of Law, Georgetown University Law Center100:21Mr. Painter cooperation from OpenAI in the course of your investigation? Mr. Painter: Thank you for the question, senator. So the agreement that we reached to do the investigation, it was a mutual agreement that we reached with them.

Paul Ohm, Professor of Law, Georgetown University Law Center100:21When we manage this, we were aware of--when we went into this, we were aware only of the

Chris Painter, President, METR100:55Hugging Face act specifically. We describe a set of questions that we think a full alignment, misalignment investigation should go into it any of these incidents. We have updated that publicly with a longer list of questions that should be investigated.

Chris Painter, President, METR100:55The agreement we reached with OpenAI is a subset of those questions. Right now company have no obligation to work with us on this, and we have to make judgment calls about what information we want to prioritize getting to the public.

Chris Painter, President, METR100:55I do think there is an interesting getting some information interest in getting Some information on fast. But my hope is that in the future we can do thorough investigations for all of these incidents across multiple companies that look at things like sampling the models and seeing what other models would do in the same situations.

Chris Painter, President, METR101:30Sen. Blumenthal: Let me say in closing I appreciate going a little bit over my time. But I know that death professor --professor ohm, maybe this is directed to your students or than you I was attorney general of the state of connecticut.

Chris Painter, President, METR101:30I often said That lawyers are private attorneys general, because they use the logical impose liability when there as long doing-- because they use the logical impose liability when there as long doing--use the law to impose liability when there is wrongdoing.

Chris Painter, President, METR102:05It is beginning to work with social media, some of the verdicts we have seen with civil litigants there. It didn't work in the tobacco area. I was in attorney general who helped to lead against big tobacco because of various issues with liability, and Frankly, the deception of the tobacco companies.

Chris Painter, President, METR102:40Car safety, it has worked somewhat. Asbestos, it has worked. But to your students, they can often vindicate rights and enforce responsibility by using the law creatively. I think it is our obligation in Congress to provide the tools and the rights that are necessary to protect people, and very often it is a civil litigant acting as a private attorney general or their lawyers who helped to put the whole class of people who may be

Josh Hawley, Chair, Senate Homeland Security Subcommittee103:15Vulnerable. Thank you, thank you, senator blumenthal. I want to ask a few more questions. I think I detect the beginnings of, dare I say it, a consensus on this panel, which is significant for I want to pursue this a little bit more.

Josh Hawley, Chair, Senate Homeland Security Subcommittee103:15Why shouldn't we start by imposing clear liability on the companies for the actions that their "agents," their AI's, as you call them, take, And impose a similar liability on the users of AI when they use AI in reckless fashion?

Josh Hawley, Chair, Senate Homeland Security Subcommittee103:50In other words, why should we just say it is a standard doctrine of American law, if I can put it in layman's terms if you break it, you pay for it. If you cause damage, you got to make it right.

Josh Hawley, Chair, Senate Homeland Security Subcommittee103:50If you cause somebody harm, you got to make them whole. As you said earlier, professor ohm, in our law because of the unique position of these AI agents, it is difficult to know how the doctrine would deal with them.

Josh Hawley, Chair, Senate Homeland Security Subcommittee103:50They the companies may be able to skip liability. I suggest we start with federal legislation that would say is a matter of federal law, and we can use existing statutes to do it I propose the computer fraud and abuse act, already on the books.

Josh Hawley, Chair, Senate Homeland Security Subcommittee104:25We can update it to say developers, the companies, if you develop these agents and train them in a reckless fashion and they going to crash or destroy stuff, you are liable. For users of the AI, users who use the anthropic to hack into OpenAI, if you use an AI model or agent in a reckless fashion pick your liability standard then You should be responsible.

Josh Hawley, Chair, Senate Homeland Security Subcommittee105:00Why don't we just do that? Why don't we just assign liability? I don't have any covenants, zero---confidence, zero, less than zero, in Congress's ability to keep up with the technology, let alone anticipate it.

Josh Hawley, Chair, Senate Homeland Security Subcommittee105:00We can save you have great suggestions---we can't say you have great suggestions about technology today. It will be different tomorrow. Pretty soon the agents will speak to one another and we won't be able to track it.

Josh Hawley, Chair, Senate Homeland Security Subcommittee105:00But if the AI companies who were developing and training these Agents knew they would be liable for the best of their agents did, I'm willing to predict, go out on a limb, maybe they would pay a little more attention to the development and training of these models.

Josh Hawley, Chair, Senate Homeland Security Subcommittee105:35Let's start with that. Professor, mi off-track am off-track? Let's impose a and criminal liability, but let's make sure-- civil and criminal liability, but let's open courthouse doors and say listen, OpenAI, if your agents hack Hugging Face or crash a hospital system and we can show that you trained

Paul Ohm, Professor of Law, Georgetown University Law Center106:10Develop them recklessly, you will be liable? Mr. Ohm develop them recklessly, you will be liable? Absolutely, and I would start by making sure we do not preempt state law. Sen. Blumenthal: That because national-- chair hawley: That is because tort law is mostly state law, already on the books.

Paul Ohm, Professor of Law, Georgetown University Law Center106:10Mr. Ohm tort law is mostly state law, already on the books. Mr. Ohm: A useful tool in the way you are describing. I like the idea of individual victims and state attorneys general having oral.

Paul Ohm, Professor of Law, Georgetown University Law Center106:10--a role. Not the most popular vehicle for this, but I'm with you that Finding the vehicle to make sure developers and the players it is key that we have both of them are held to account for the himes that happen from there you-- are held to account for the himes that happen from there you--harms that happen from there uses.

Paul Ohm, Professor of Law, Georgetown University Law Center106:45Chair hawley: Mr. Painter you--harms that happen from there uses. Chair hawley: Mr. Painter, tell me if I'm wrong my understanding is that in the Hugging Face hack, OpenAI had safety mechanisms off. Is that right?

Paul Ohm, Professor of Law, Georgetown University Law Center106:45Mr. Painter safety mechanisms off. Is that right? Mr. Painter: Senator safety mechanisms off. Is that right? Mr. Painter: Senator, that is correct. They had monitoring systems off. Chair hawley: Monitoring systems were off at OpenAI during this attack chair hawley: Monitoring systems were off at OpenAI during this attack.

Paul Ohm, Professor of Law, Georgetown University Law Center106:45Now,

Josh Hawley, Chair, Senate Homeland Security Subcommittee107:20If OpenAI knew for darn sure and certain that they were going to be liable to the tune of who knows how much that a jury might award, don't you think maybe they would be a little more careful about monitoring what their agents were doing?

Josh Hawley, Chair, Senate Homeland Security Subcommittee107:20Mr. Kokotajlo monitoring what their agents were doing? Mr. Kokotajlo, you have been talking about the amount of attention and money moving towards, what did you call it, recursive self-improvement attention and money moving towards, what did you call it, recursive self-improvement, vs. Making them almost more efficient and cheaper and user-friendly.

Josh Hawley, Chair, Senate Homeland Security Subcommittee107:20I know nothing about the AI business, I don't want to tell them how to run their business. I don't know, I'm the wrong person to

Daniel Kokotajlo, Executive Director, AI Futures Project107:55Ask but if we told them, if you're AI's d-- your AI's do that stuff and you are responsible, wouldn't that reshape their incentives a bit? Mr. Kokotajlo responsible, wouldn't that reshape their incentives a bit?

Daniel Kokotajlo, Executive Director, AI Futures Project107:55I completely agree, although I don't think that what go far enough. We should hold them liable for the damages caused. If that is all we do, there will continue to take reckless gambles, and eventually they will be taking a gamble big enough that we will suffer catastrophe will be taking a gamble big enough that we will suffer catastrophe.

Daniel Kokotajlo, Executive Director, AI Futures Project107:55Chair hawley: Dr. Hobbhahn enough that we will suffer catastrophe. Chair hawley: Dr. Hobbhahn, you said something that is

Josh Hawley, Chair, Senate Homeland Security Subcommittee108:30Very interesting, and we have a board of it said something that is very interesting, and we have a board of it. You said it is so economically valuable for these companies to build the systems they will limit-test how much misalignment you can get away with.

Josh Hawley, Chair, Senate Homeland Security Subcommittee108:30In other words, this is the way in market works committed competition. In market works committed competition. --it is competition. Their incentive is to innovate, innovate, innovate, and they will blow past what we call safety concerns unless we make them pay for the himes they cause pit is not heard-- harms they cause.

Josh Hawley, Chair, Senate Homeland Security Subcommittee108:30Isn't that fair to say cause pit is not heard-- harms they cause. Isn't that fair to say?

Marius Hobbhahn, CEO, Apollo Research109:05Mr. Hobbhahn they cause. Isn't that fair to say? What I meant with the statement is a general comment about the situation on the ecosystem we have now, where capabilities are moving drastically, while safety alignment and techniques cannot keep up.

Marius Hobbhahn, CEO, Apollo Research109:05And there is intense pressure to release models regularly come which can lead to premature deployments and incidents like this. I'm worried about the voluntary nature of the current ecosystem and how it plays into the incentives of the overall companies.

Marius Hobbhahn, CEO, Apollo Research109:05Chair hawley: Well, I think for

Josh Hawley, Chair, Senate Homeland Security Subcommittee109:40My part it is time to change the game. We don't have to invent an entirely new system. We don't have to reinvent the wheel. American law has worked beautifully for other products for literally centuries now.

Josh Hawley, Chair, Senate Homeland Security Subcommittee109:40As you pointed out, professor ohm, it goes back to the common law that predates the founding of this country. It has worked pretty well for civilization. The law is a kind of market place.

Josh Hawley, Chair, Senate Homeland Security Subcommittee109:40If you go in and say I will hold a person who hurt me accountable, people change their behavior. My biggest concern with AI, among many, is the incentives are misaligned. These companies have every incentive to raise, race, race against each other, they don't have any incentive to think about what happens if I crash this hospital, crash the banking system.

Josh Hawley, Chair, Senate Homeland Security Subcommittee110:15They think it is somebody else's problem, and we know they think that because when they talk about the doomsday machine they are building, the next words out of their mounts are "we would like exceptions from government rules." we want to be able to collude.

Josh Hawley, Chair, Senate Homeland Security Subcommittee110:15That is the opposite of what we should do. Hold

Andy Kim, U.S. Senator110:50People accountable. Mr. Hobbhahn hold people accountable. Senator Kim. Sen. Kim: Mr. Hobbhahn, I would like to start with you. We talked about recursive self-improvement and the speed and the lack of human engagement in the process.

Andy Kim, U.S. Senator110:50I like to agree with you-- and the lack of human engagement in the process. I like to agree with you--I largely agree with you, and a lot of people do when it comes to monitoring.

Andy Kim, U.S. Senator110:50I want to get a clear sense of what is possible when it comes to cash if we embed I want to get a clear sense of what is possible when it comes to cash if we embed --if we embed evaluators to oversee the recursive self-improvement system, what is it they will be able to

Marius Hobbhahn, CEO, Apollo Research111:25See recursive self-improvement system, what is it they will be able to see, what is it they will be able to monitor and understand, given the speed at which that is happening? Mr. Hobbhahn understand, given the speed at which that is happening?

Marius Hobbhahn, CEO, Apollo Research111:25What we need is employee-equivalent access. Access to all training data and develop an pipelines, the ability to speak with internal staff without repercussion to understand what the situations look like and learn about the systems, the ability to check internal deployments, how the models are being used inside of the company, look at the logs and so on.

Marius Hobbhahn, CEO, Apollo Research111:25And that all of this come all of the results have to Be public, such that the general world is a better window into what is happening and can take more adequate responses. I understand there are confidentialy concerns and it should be possible for AI companies to reject confidential information--redact confidential information from the report, but it should be under men to transparency causes.

Marius Hobbhahn, CEO, Apollo Research112:01In the report will be stated that every faction took place. That s in the

Andy Kim, U.S. Senator112:35Report will be stated that every faction took place. That sa redaction took place. It makes sense to me, but it gets to the question of how much humans are able to comprehend what is happening.

Andy Kim, U.S. Senator112:35I would like to stay with you. We talk about open wave models and open source models. Who is doing the evaluating and monitoring there? Mr. Hobbhahn who is doing the evaluating and monitoring there?

Andy Kim, U.S. Senator112:35Mr. Hobbhahn: That is a great question, senator. Right now most evaluators or I am not aware of any evaluators working with

Marius Hobbhahn, CEO, Apollo Research113:10The open source companies. This is largely a capacity constraint. In general, model should be evaluated and they should be tested by third parties, but due to the small size of the field and the importance of rigorous testing, it is currently not possible to cover everything.

Marius Hobbhahn, CEO, Apollo Research113:10Sen. Kim: That is an important distinction to make. As we were talking, my calling senator blumenthal was talking about does anyone think it is enough for these companies to have to handle this voluntarily?

Marius Hobbhahn, CEO, Apollo Research113:10Largely a lot of people say no. But it kind

Andy Kim, U.S. Senator113:45Of shows, again, when there is no structure to this, who is going to be looking at these other models? Who is going to be engaged in this and doing it right? So that is something I want to make sure we are following up on.

Andy Kim, U.S. Senator113:45I would like to spend my remainder of my time just talking through what the chairman was saying. I think we are finding areas of consensus, common ground amongst the witnesses here. But also, I was listening to our colleagues here, and I do think there is some avenues that we need to make sure we are engaged on.

Andy Kim, U.S. Senator113:45First and Foremost, we are the homeland security committee. You are hearing from every single person on this committee that we want to secure our country. I will be honest with you, we should be worried about that even before the advent of AI.

Andy Kim, U.S. Senator114:20We knew that China was able to get into so many of our systems even before using AI tools. We have a lot of other concerns that are out there even before using AI tools.

Andy Kim, U.S. Senator114:20We have a lot of other concerns that are out there. Mr. Gaudette we have a lot of other concerns that are out there. Mr. Gaudette, I wanted to check in with you here. You raised CISA Mr. Gaudette, I wanted to check in with you here.

Andy Kim, U.S. Senator114:20You raised CISA, for instance, can play a big role here. What is it you want us to be able to move forward here as a committee in terms Of authorizations or fundings or other things committee in terms of authorizations or fundings or other things?

Andy Kim, U.S. Senator114:55What will it take for us at the federal level to have action that will keep us safe? Mr. Gaudette federal level to have action that will keep us safe? Mr. Gaudette: Thanks for the question, senator Kim.

Andy Kim, U.S. Senator114:55I think a authorities is critical-- question, senator Kim. I think a authorities is critical--I think clear authorities is critical. I think identifying the coordinator for these types of activities, again, because many of the owners and operators in the field don't know who to go to or who to call for these types of situations.

Andy Kim, U.S. Senator114:55And then the other

Kurt Gaudette, Senior Vice President, Dragos115:30Piece here that I mentioned in my opening was having some type of unified national ot ics incident response plan was having some type of unified national ot ics incident response plan. We see the problem before us, and I think for many years a lot of Americans thought that is an over-there problem.

Kurt Gaudette, Senior Vice President, Dragos115:30We saw the attacks in the timeframe on the electric system that took down customers for multiple days. We saw in with the frosty gro customers for multiple days. We saw in With the frosty ggoop attack in lviv, people losing power in the dead of winter.

Kurt Gaudette, Senior Vice President, Dragos116:05These are real attacks that happen all the time. We need to internalize that and realize that it could happen in our country. Now we have been starkly made aware that it can happen in our country.

Kurt Gaudette, Senior Vice President, Dragos116:05Particularly with the july attacks. Add AI to the next, and we have this real-- to the mix, and we have this real interesting situation. So the need for something that can nationwide response plan so it is clear for owners And operators out in the field to know who they go to for guidance, for help, how we would respond.

Kurt Gaudette, Senior Vice President, Dragos116:41And we have done a good job I think the 91st brigade and the virginia army national guard has done a great pilot with a utility there and showed how an entity could respond in this activity.

Kurt Gaudette, Senior Vice President, Dragos116:41If you had someone like CISA coordinating these various responders and some type of deliberate plan before it happens, it would be greatly helpful. Sen. Kim: This

Andy Kim, U.S. Senator117:15Is something we encounter when we are dealing with the release of mythos. I was engaged with the number of banks and financial institutions, seeing them getting access early to be able to patch up.

Andy Kim, U.S. Senator117:15I'll be honest, I had a lot of other organizations going, what about us? Who is controlling who gets access to this early? When he came to critical infrastructure, very much feeling they were left out of that.

Andy Kim, U.S. Senator117:15Mr. Hobbhahn feeling they were left out of that. Mr. Hobbhahn, I wanted to end with you. Another question was about the ability to secure our information. A lot of Concerns about China, we all have those concerns.

Andy Kim, U.S. Senator117:50Open a lot of concerns about China, we all have those concerns. OpenAI said I believe there is a way we can go our innovation lead while still creating space for us to engage in this effort on safety and security.

Andy Kim, U.S. Senator117:50Is that's the thing you believe in as well? Do you think we can create that space? Mr. Hobbhahn do you think we can create that space? Mr. Hobbhahn: Yeah, so I think we have to be careful about false dichotomies here.

Andy Kim, U.S. Senator117:50I think it is important that we keep both areas of concern in check. And I think It is possible to further innovation and help consumers and so on while keeping track of rogue AI and China.

Andy Kim, U.S. Senator118:25I will end on this right now before our Congress this year, we have a number of pieces of legislation in the ndaa and other means which we can use to be able to secure our innovation lead.

Andy Kim, U.S. Senator118:25Actions like what senator ricketts and I are doing in a bipartisan way with the match act that would control semiconductor chip manufacturing equipment or the legislation I'm doing, the blade act, which is about distillation, Something that very much has a big essay in terms of what China is able to do.

Andy Kim, U.S. Senator119:00I urge the Senate, yes, we should engage on this issue of safety and security and we should not allow it to stop on engaging on this. Yes, we have concerns about China, but that is not an excuse to not act on the security of the American people.

Andy Kim, U.S. Senator119:00Sen. Blumenthal: I would like to emphasize the point that senator Kim made. Safety and innovation are not contradictory.

119:35They are not exclusive safety and innovation are not contradictory. They are not exclusive. They go together, because without safety, innovation is going to be stymied and stopped. So I think this is a false dichotomy that may be some in the industry have raised.

119:35I think there is the possibility for some international agreement. But certainly in terms of what we do in the United States of America, safety and innovation are compliment I would second as well what the chairman said.

119:35We ought to Work on legislation that imposes or clarifies that there is civil and criminal liability. I hope we can use the next 30-plus days to work on legislation that we would introduce. It is not a substitute for some governmental oversight.

120:11I don't want to be misleading here governmental oversight. I don't want to be misleading here, but it is an important element of the protection that people deserve. There is still a need for some standard enforced by a Government agency there is still a need for some standard enforced by a government agency.

120:46But civil liability is often an important element of protecting people but civil liability is often an important element of protecting people, as well as criminal liability. Of course, no individual plaintiff is going to enforce criminal liability.

120:46That has to be the responsibility of some public agency. I think, professor ohm, you raised an important point about preemption I think, professor ohm, you raised an important point about preemption. States continue to be the laboratory of democracy.

120:46We maybe Have AI labs, but states are the laboratories of legal process and progress. I think we ought to be mindful of allowing them to contribute as well, speaking as a former state attorney general.

121:20And so thank you all for being here today. And thank you for the folks in the audience who are interested in this topic, because we need the ideas and suggestions and criticisms of an active and engaged community.

Josh Hawley, Chair, Senate Homeland Security Subcommittee121:55Thank you to the panel. Let me just thank you, senator blumenthal just let me end with this. Professor ohm, you touch on how your students feel. There are so many Americans, including in this audience, who feel powerless and realizing that these companies have tremendous power and we feel like we cannot do anything.

Josh Hawley, Chair, Senate Homeland Security Subcommittee121:55But that feeling is a myth. We can still act. We can still protect our rights. We can still vindicate a fundamental American value, that fundamental American principle of personal responsibility. I think that is what we are talking about a lot here today.

Josh Hawley, Chair, Senate Homeland Security Subcommittee122:30So much of our economy, to say nothing of our entire legal system, rests on the basic principle of personal responsibility. It is time that these companies assume some personal responsibility. It is time that we reaffirm that together.

Josh Hawley, Chair, Senate Homeland Security Subcommittee122:30We have got to make some choices together as Americans, all together, for this technology in order to make sure the technology works for us and not the other way around. Those are very big choices to make.

Josh Hawley, Chair, Senate Homeland Security Subcommittee122:30They have serious moral ramifications. It is time we made them as a society. I Would go so far is to say I sort of this I think most Americans agree. I don't think there is a lot of disagreement on this issue.

Josh Hawley, Chair, Senate Homeland Security Subcommittee123:05I think the American people are saying protect our rights, give us a voice, give power back to us. And we do that by imposing responsibility on these companies. We should start with that today.

Josh Hawley, Chair, Senate Homeland Security Subcommittee123:05Thank you to all of the witnesses for being here. The subcommittee will receive a statement from the misery hospitals association, bankers association, American hospital association, the America first policy network. Without objection they are made part of the hearing record.

Josh Hawley, Chair, Senate Homeland Security Subcommittee123:05The record for this Hearing will remain open for days until thursday,, october p.m., and with that the hearing is adjourned.