The use of Artificial Intelligence in nuclear organizations

[AUDIO LOGO] LUCY BROWN: Good afternoon, everyone, and welcome to CNLO's webinar, the use of artificial intelligence in nuclear organizations. Apologies for the little delayed start. We had a bit of technical difficulty on my end. We're so pleased for you guys to join us today for this important discussion on how AI is transforming the way our sector operates, innovates, and collaborates.

Before we begin, I would like to take a moment to acknowledge that we are joining from many places across Canada. OCNI's Head Office is located on the traditional territory of the Mississauga of Scugog Island, First Nations, within the lands protected by the Williams treaty. We recognize the enduring presence of First Nations, Inuit and Métis people across the country, and we remain committed to learning from and working alongside indigenous communities as partners in advancing a sustainable and inclusive energy future.

Just a few housekeeping before we begin. This webinar will be recorded and uploaded to our members portal for viewing. So you have any questions for the speakers throughout the webinar, the Q&A button, located on the right hand side of your screen is where you will place your questions. We'll address all the questions at the end of the webinar. Again, thank you for joining us, and I'm now pleased to hand it over to our moderator, Lisa.

LISA THIELE: Thank you so much, Lucy. Good afternoon, everyone. [NON-ENGLISH SPEECH] We are here for the Canadian Nuclear law organization webinar on a very important topic AI in nuclear applications. So we have with us two experts from the nuclear regulator. That's the nuclear regulator, not the AI regulator. And we also have one of Canada's very top energy lawyers with us today.

We're going to first hear from our CNSC colleagues. I, of course, am from the CNSC as well. We'll hear from our colleagues first, and then we'll hear from Ahab Abdel-Aziz. And hopefully, we will have time for some really robust discussion amongst our panelists and to address your questions that as Lucy said, we'll be inviting you to put in the chat.

We have the chance today to talk about what nuclear safety evolution teaches AI development and can teach, and also how AI has the potential to transform nuclear operations and safety, and what regulatory frameworks with respect to AI, with respect to nuclear regulation, need to be in place for that.

So I'm going to introduce all three of our speakers first, and then I will turn it over to Kevin and Pierre-Daniel to from the CNSC to start off. So Kevin Lee is a Senior Innovation Project Officer-- I would like that title-- in the Office of the Chief science officer at the CNSC. He currently leads the CNSCs team that is analyzing Disruptive, Innovative and Emerging Technology, DIET.

This team aims to ready the CNSC to evaluate and regulate nuclear activities that will implement DIET. Kevin is also engaged in the work readying the CNSC regulatory framework for the regulation of advanced reactor technology and SMRs. Also active on numerous other policy and regulatory files at the CNSC, including fusion and nuclear specific AI technologies.

Kevin serves on the following IAEA groups-- the steering committee for the network for innovation to support operating plants, the working group on explainable AI for the nuclear power industry, and he's co-chair of the AI and human factors/knowledge management subgroup, and the working group on approaches to Fusion regulation. Also on the international front, he is Chair of the Management Board of the OECD NEA's RegLab project.

Kevin is also active with the Canadian Nuclear Society, serving in the following capacities-- Co-chair of the CNS diet conference and diet division chair, the Organizing Committee for the 2025 CNS Annual Conference, which he may speak about. Kevin's over 30 years of providing regulatory policy and operational expertise and extensive experience in government includes nearly a decade spent as special assistant to the Honorable Jean Chrétien, Former Prime Minister of Canada.

Pierre-Daniel Bourgeau is counsel with the CNSC, so he's in my team. Pierre-Daniel has been with the CNSC for over 20 years. He provides legal advisory and representative services to the CNSC, including advice and opinions related to nuclear regulatory law. His practice focuses on regulatory law, with particular expertise in nuclear security matters and legislative and regulatory development.

Has contributed to key initiatives such as amendments to the legislation and modernization of our nuclear security regulations. Pierre-Daniel has provided specialized legal and strategic expertise to the staff and commission members in the preparation for and conduct of its quasi judicial hearings, including those related to the joint review panel for the Darlington new nuclear project, as well as for the deep geologic repository.

More recently, Pierre-Daniel has been advising the CNSC's DIET team, helping the organization get ready to evaluate and regulate new activities that use these emerging technologies. He also works with the DIET subgroup, that is focused on the development of a strong regulatory framework to support the safe and responsible use of AI in the nuclear sector. So I will hand it over to Kevin and Pierre-Daniel. You have the floor.

KEVIN LEE: Thank you. Lisa and I don't think either Pierre-Daniel and I are experts in AI, and I would actually assert that anyone who thinks they're an expert probably is not. The field doesn't lend itself to expertise, given its difficulty in explaining it and understanding it. The other thing I will say is I recognize I'm speaking to a bunch of lawyers predominantly, and I'll do a disclaimer, which is if something sounds really good, it's probably seen as C position. If I say something that sounds a bit stupid, it's probably just my own position. So I'll put that out there right away.

So today Pierre-Daniel and I are going to go through what the CNSC is doing. We're going to start with just a really quick primer on what is AI, a little bit about the CNSC, and what the CNSC is doing around AI. Pierre-Daniel is going to talk about the trilateral white paper or principal paper that we published a year ago in September. But anyway, we'll see how that goes. About a year to do that. And finally, conduct of technical assessment. So making sure that our subject matter experts down the road have the tools they need to assess whether or not AI can be deployed safely in a given regulated activity.

So that's kind of what we're doing at the CNSC. There's more and I'll get into that later. But for now, I'm going to turn it over to Pierre-Daniel, who's going to talk about the trilateral paper and moving forward, what we're going to be doing. And so, Pierre-Danielle, over to you.

PIERRE-DAMIEL BOURGEAU: Thank you, Kevin. So when it comes to the regulation of AI in Canada, there's currently no overarching law that governs AI models and systems in a general way. The proposed Artificial Intelligence and Data Act was part of a broader bill that unfortunately did not pass as it died on the order paper when parliament was prorogued paroled January 6 of this year.

So even though Canada doesn't have a single unified AI law, there are several existing laws that apply to different aspects of AI development and use, for example, there's some privacy laws, which regulate how personal information is collected, used, and disclosed when training AI models or generating output.

There's copyright law, which is relevant to how training data sets are created or how model outputs are generated. Human rights law, which prohibit discrimination based on race, gender, and other protected grounds in contexts such as employment and access to services. Tort law, which can hold individuals or organizations liable for damages caused by negligence. This includes harms from defective AI products or misuse of AI systems.

Competition law, which governs anti-competitive behavior such as collusion or deceptive marketing. A growing concern AI tools can be used to facilitate these practices. Employment laws, which regulate the use of AI hiring employee monitoring, performance evaluation, termination. But at the CNSC, we've determined that a regulatory framework at this time will not require the regulation of AI.

The CNSC regulates nuclear activities. Our regulatory framework is focused on making sure that deployed, regulated activities are done in a safe manner. The CNSC evaluates and continues to evaluate the use of artificial intelligence and regulated activities and assesses if it has a safety impact. The CNSC did look at considerations for developing artificial intelligence systems in nuclear applications under what is known as the CANUKUS paper, the trilateral paper.

It presents a high level principles for safe and secure use of AI in nuclear applications. It's named by the three parties-- CNSC, United Kingdom's Office of Nuclear Regulation, and the United States Nuclear Regulatory Commission. The goal of this paper was to highlight key topics that should be considered when introducing AI technologies into nuclear operations, to ensure that safety and security remain at the forefront.

The three regulatory bodies recognize the importance of broadly applying these principles as AI systems are developed, reviewed, and deployed across our respective jurisdictions. Now, the CNSC, the UK ONR and the US (NRC) all shame the top priority, ensuring nuclear safety. However, the way each organization approaches that goal can differ quite a bit. Each country has its own priorities, and the influences of led to different regulatory frameworks and philosophies.

When it comes to new and emerging technologies like reactor designs, advanced reactor designs, small modular reactors, or digital instrumentation and controls, each regulator typically carries out its own research and analysis. Now, if one regulator approves the use of AI in a particular situation, that doesn't automatically mean the others will follow suit. Each organization may still want to conduct its own independent review.

Right now, the three organizations are actively working to tackle the challenges related to adopting AI in the nuclear sector. We're sharing lessons learned and comparing regulatory experiences and exchanging information, all with the goal of supporting the safe and secure use of AI under our respective mandates.

So building on the CANUKUS AI paper, the IA and IAEA publications, the next papers in the series will elaborate on how the three regulators are still aligned, and how the risk matrix or the heat map, as we like to call it. I'm going to spend some time on that. From the initial paper applies to specific deployments such as predictive maintenance. So let's take a closer look at this heat map and the categories for AI use cases and nuclear applications.

Essentially, we can think of these use cases as falling into four main categories, which are defined along two key axes. The first one, being the significance of an AI failure and the second axis, the amount of AI autonomy. And they're right up there in that colored rectangle. So the horizontal axis, we move from AI systems that are heavily reliant on human input, what we call insight or collaboration, to systems that operate more independently, which we call operation or full autonomy.

So on the left side, AI helps humans make better decisions. It supports or augments human judgment. But as we move right, AI starts taking over more of the decision-making and operational control, often with limited or no human oversight. The transition also means that humans may have less time to react if AI behaves unexpectedly or fails.

On the vertical axis, we consider how serious the consequences would be if AI were to fail. As the significance of failure increases, so does the level of scrutiny on that AI system, especially when it comes to safety and security. If an AI system operates in region one, for example, the potential of failure is low. So deployment may be more flexible. But as we move in the higher regions, the impact of a malfunction can become much more significant, requiring stricter evaluation and control.

Now, it's important to remember there are already international standards governing digital systems in nuclear safety applications. These standards provide guidance on how software tools should be developed and verified. They serve as a starting point when developing AI systems in this field, especially as we think about how these AI categories fit into existing safety frameworks.

So looking more closely at region three and four at the top, where failures could significantly affect safety or security. In region three, AI systems produce outputs that can still be verified by humans before any action is taken. However, even though humans can check the output, an undetected error could still pose a serious risk. For example, these might include AI tools used in designing or maintaining safety systems. Because of that, a robust verification process is essential.

In region four, verification becomes much harder. The system may operate too quickly for a human to check outputs in real-time. A good example of this would be AI-optimized control or protection algorithms. So here, we're relying on the AI itself or other redundant safety systems to prevent or mitigate failure. Now, if the potential consequences of failure are low, we're in the regions one or two. Here, AI failures might not directly impact safety, but could they could still have indirect effects.

For example, suppose licensee is using an AI system in a nuclear turbine plant to analyze maintenance data, and it suggests reducing maintenance tasks. I mean, that sounds efficient, but it might intentionally lead to increased equipment stress or unexpected reactor trips, which could then put unnecessary demand on reactor protection systems.

This example shows how even low-risk AI applications can have unintended consequences if not properly managed. Ultimately, the level of autonomy and the impact of potential failure determine how much oversight is needed as AI moves from one region to two, becoming more autonomous, it's essential to ensure the system is regularly updated, maintained, and verified. This helps prevent performance degradation and ensures AI continues to operate safely within its intended parameters.

So to conclude on this point, when categorizing AI under safety classification scheme, developers need to consider, amongst other things, whether the AI can introduce faults, whether the AI can miss existing faults, and whether its output can be verified through other means. Kevin, back to you.

KEVIN LEE: Thank you, Pierre-Daniel. And I'm now going to talk about International RegLabs in AI. Part of that is very fresh. We had the first one actually last Thursday and Friday in Toronto on the diets of the word on the margins of the diet conference, which was also last week.

So what is a RegLab? In essence, it's a tabletop exercise where you run through a thought experiment, and what you take is you take a near term, i.e 2 to 10 years type of let's say diet or deployment, usually matched with AI, and you look at what's going to happen and how that will be, in essence, ruled out.

So you get together with industry, you get together with regulators, you get together with service providers, and academia. And the idea is you look at what mitigations should be in place to make sure it's safely deployed, are there regulatory showstoppers. It's very much an exercise where you have a use case, you develop a problem statement or an opportunity statement, and you try to get in a room sort of consensus around how to move forward on something.

It's not a regulatory discussion. You don't get into requirements. You don't get into guidance. And the nice thing about it is we use-- I think, lawyers will be familiar with this Chatham House Rules. So there's no attribution. Everyone is there to speak their mind. Licensees are expected and encouraged to think as regulators and vice versa.

So we did actually one of these about a year ago, and we did it on the margins of the DIET conference again. And this was a pilot that was done by the US (NRC), the UK ONR, and the CNSC. At the time, it was EPRI who kind of helped finance it and pushed it forward. They also invited NEA to observe as well as the IAEA. NEA liked it so much that they decided that they would run with it.

So what we have now is we have an international RegLab exercise. Again, on the DIET conference last weekend, so just on the 23, 24, you have seven regulatory bodies who are funding this. The IAEA and EPRI , ex-officio standing. And the nice thing about them is as a regulator, even though you're funding, you don't want to be the one that's inviting industry because you do not want to show favoritism, nor do you want to support any given service provider. So they are the ones that introduce and make sure that they come together.

As Lisa mentioned, we are chair of this executive management board. So over the course of the next two years, there'll be three more of these RegLabs and rolling. They will be published afterwards by all regulators involved, as well as the NEA now. The real reason the CNSC is so excited about this is we are looking next year to roll out our own bespoke domestic RegLabs.

So our vision is that we partner with someone, again, I'll turn to Lisa, who's on this. She's Director of Nuclear Standards at CSA group. Someone like the CSA, who can be that independent body, potentially CNL. And we will look at domestic, because one of the things we find with RegLabs is given the varying regulatory frameworks, it's sometimes difficult for countries to have the same sense of a given, let's say, diet or a given AI application. So the ability to actually do that on a domestic level would be wonderful and would be good for the industry, as well as the regulator to get an early heads up as to how we're going to safely deploy these technologies.

The IAEA, I'm going to talk about some of the work we're doing there. And I'm very cognizant of the fact that we started late and we want to leave room for questions. So I'll try to get through this very quickly. IAEA, in the same way, let's say that we started looking at DIET in AI. They started looking at it approximately the same time. So in 2022, they formed what is called the ISOP network. And I encourage you, if you have the time to join the ISOP network. It's the innovation to support operating plants, but I like to call it the de facto innovation hub at the IAEA.

There's four subgroups in it. And they're not looking at just operating plants. They're looking at all facilities, future facilities as well. There's AI subgroup that, I'd say, is probably where 70% of the effort is within this working group. There's Drones and Robotics subgroup. And a colleague of mine at the CNSC, John McIntosh, is on that, and I believe he's on the call.

There's also an INC. And again, we have representation on that. And there's one on Additive Manufacturing. So they're collectively looking at that. Now, the only group that's actually produced a product at this point is the AI group. There were a number of us that were on the drafting team on this.

And what we developed was we had this whole step process in mind when we started this trilateral effort, that what we could do is we could start with something very general, such as the-- let's say, the trilateral principle paper that we published. And then a little more granular would be the IAEA document. And that was just published in September. So less than a month ago. I encourage you to read that because it does get into a lot more detail around some of the things you need to do, let's say, around data, data validation, human in the loop, all that fun stuff that you should be thinking about when you're looking to deploy AI in the nuclear realm.

Anyway, that has happened. What I would also say is that because of that group, we have a steering committee, and that steering committee looked at nine areas where we needed more focus within AI. One of the issues with AI. And I think Pierre-Daniel touched on it, is that there isn't really any where in the world regulations that really speak how to safely deploy.

We talk a lot about the human factors, the human aspect of it. The closest I think we get is probably the EU and what they've done in terms of trying to regulate AI. But one of the things I hear from industry, i.e. the AI industry, is there needs to be a body like the IAEA, that internationally oversees the deployment of AI. Whether or not that ever comes, we'll see. But for now, the IAEA is doing a very good job of filling that void, at least within the nuclear community.

So this is the areas we figured that we needed to look at. We then combine that down a little further, and there's seven working groups at this point, ranging from human factors, cybersecurity, which I could do a whole series of lectures on cybersecurity, and the risks that are inherent. One of the biggest challenges around cybersecurity is AI has the ability to up, let's say, the game of even mediocre bad actors.

And if you want to see how real that challenge is, look up the term vibe coding-- V-I-B-E, just vibe coding. It's a recently coined term, and it gives you an idea of what some of the downside of AI can be. Anyway, all of which is to say that I highly encourage you to get involved with the IAEA and the ISOP network. It's open to everyone. It's probably one of the best forums in terms of providing information, factual information, about AI in the nuclear realm. Second, of course, to the CNSC.

Just to mention that there is an international symposium on artificial intelligence and nuclear energy. A lot of the talk right now is-- in essence, and it was that last week's conference, where the DIET Conference looks at future technologies, what's going to happen. And there was a lot talk about how nuclear power plants will not only power AI data centers. But that the AI data center that's co-located the vision that some in industry have is that down the road, that AI center will actually run the power plant that it's co-joined to.

We're nowhere near that. And I hope we don't get very close, because I'm a huge proponent of keeping the human in the loop. But having said that, I will now close with, as I like to say, sort of an apple, pie, and motherhood slide about what we're doing at the CNSC. So we really try as much as possible to engage with industry and stakeholders around AI-- have those exploratory discussions. That's why we love doing things like this.

We're definitely working with international industry. We're trying to lead internationally impact on the AI file with a nuclear, and a very big part of that, sharing that knowledge, and engaging our staff very, very important. We're trying to develop not just a safety culture, not just a security culture, but a culture for innovation at the CNSC.

And we like to think that the work we're doing, we're doing domestically, and globally, really identifies us as a very agile regulator when it comes to both DIET and AI, that we are encouraging the safe introduction of DIET, especially AI, and that we're leveraging that world-renowned and recognized technology neutral regulatory framework that we have. And with that, I will stop talking, at least for now.

LISA THIELE: Thanks so much, Kevin and Pierre-Daniel. And now, we will hear from Ahad Abdel-Aziz. Again, I'm going to say, feel free to put your questions in the chat. And after we hear from Ahab, then we'll have some interaction, I hope. Ahab Abdel-Aziz is a Partner and Global Director of the Nuclear Power Generation at Gowling WLG.

He's been a leader in the global nuclear sector for more than 30 years. He's advised leading members of the Canadian and international nuclear energy sector and government agencies in policy and legislative development, nuclear project and program development, and finance, licensing, and compliance, and dispute resolution. Ahab is a leading lawyer in the nuclear sector and has served as lead negotiator in multi-billion dollar nuclear project contract negotiations, as well as lead litigation counsel in technically complex, civil, and regulatory disputes, including multibillion-dollar nuclear project arbitration.

He's recognized as one of Canada's top energy lawyers. He's the Former Chair of the Board of Directors and present member of the board of the organization of Canadian Nuclear Industries, the OCNI. He's Former President and President Member of the Board of the Canadian Nuclear Law Organization, the CNLO. And he's a former member of the board of management of the International Nuclear Law Association.

He's a founding Executive Member and Director of the National Brownfield Association, and a past member of the National Roundtable on the Environment and the Economy's National Brownfield Redevelopment Strategy Task Force. He was also the principal author of that body's national policy recommendation, titled facilitating a Brownfield redevelopment strategy for Canada, prepared at the request of the prime minister of Canada at the time.

He's past Vice Chair of the Environmental Crimes and Enforcement Committee of the American Bar Association, as well as a member of the ABA's International Environmental Law Committee and Tort Trial and Insurance Practice section. He's previously served as Executive Member of the OBA Environmental Law Section and as Trustee of the Metropolitan Toronto Lawyers Association. His co-author and co-editor of the Canadian Brownfields Manual. And with that introduction, Ahab, I happily turn the floor over to you.

AHAB ABDEL-AZIZ: Thank you, Lisa. Let me [INAUDIBLE] screen, and I'm going to have to rely on you to tell me that you're seeing the screen if you are.

LISA THIELE: Yes.

LUCY BROWN: Yes.

LISA THIELE: [INAUDIBLE]

AHAB ABDEL-AZIZ: We are.

LISA THIELE: Perfect.

AHAB ABDEL-AZIZ: OK, good. All right. So let me get right to it. We have a short bit of time and as usual, I've got too much to say. What I do have is six modules, and I'm trying to sketch my way to what is effectively an analytical policy position that I come to at the end of the day, rather than the diving into the details of how to do it.

I want to look at what we've learned as an industry about safety, and I want to look at what we are and are not able to do, particularly in western nations when it comes to deploying nuclear power reactors, and especially fleets in relation to climate change.

I want to look at what AI might be able to offer us if we conclude that there is a comfortingly safe case for engaging AI and nuclear power organizations, whether it be design, construction, licensing or in operations and maintenance and safety. I want to look at where we are in the regulatory side. I'm going to say very little about where we are because it's being said in terms of what's going on. But I have some comments on it.

Then I'll dive a little bit into what's I. Not so much from the perspective of looking at what we're adopting as definitions, but from inside the AI industry, what that looks like, because it does have implications for how we go about it. And then finally, we'll get to what I proposed, which I expect will get me in trouble with everybody.

All right. Let's start with what we know as and nuclear sector now. March 28, 1979, there was a stuck open pilot. A pilot operated relief valve at Three Mile Island unit 2. Triggered a cascade of events, and for over two hours, we had two highly trained operators that were confused by contradictory signals and poor interface design.

So they took actions that made things worse when the situation could have been managed to a safe conclusion by taking appropriate action. And they got us to a partial meltdown. And when the alarm came on, they couldn't understand what was going on or what to do about it.

In retrospect, the Kemeny Commission Investigation Report focused our minds on human factors. That was really the birth of human factors. They said the main deficiencies in our reactors is not technology or hardware problems, but it's management problems.

For now, April 1986, the operators at Chernobyl's deliberately disabled safety systems. To complete a scheduled tests, they violated a number of safety procedures because they were under pressure-- they were under pressure to get things done. And the reactor itself had two features that were created an unsafe condition or potentially unsafe. It had inherent design instabilities in terms of a positive void coefficient, and it lacked containment. It had no containment to keep radiation in at once you had an incident.

So when the test triggered a runaway reactivity, we wound up with an explosion and fire and contamination that traveled for some distance. So if TMI told us about human factors being important, that is people making poor decisions under stress, Chernobyl got us a little bit deeper in understanding that good people and bad organizations, or particularly bad organizational culture, can make some really bad choices and get in trouble.

Fukushima took us even deeper, I think, into cultural and organizational environmental factors that can drive human behavior and poor decision-making, but this was long range for decision-making. So we had [INAUDIBLE] magnitude. Mangnitude nine earthquake triggering a tsunami. We all know about that. We had three reactor meltdown when we had the generator emergency generator come offline.

What came out in the investigation is that TEPCO had known about the risk, had been warned about the risk that systematically not addressed the risk in the manner they should. And the regulator that was showing less than sufficient independence seems to have let them get away with it.

Fukushima, I think, taught us that organizational and cultural failures, in addition to technical ones, can be more significant as driving forces. So what we really have is that each incident has focused us on reconceiving of safety. Three Mile Island had us focusing on human factors. It wasn't the old thinking that if you engineer your way carefully enough, you're going to have a safe reactor. We realized people have to behave safely.

Chernobyl taught us that culture can drive behavior. And it's important to pay attention to culture. And it really was the rise of safety culture as a concept came out of that. All of the elements of safety culture came out in more or less in the report, but those words were not spoken. And what we learned from Fukushima is you can't just assume that we've got it covered. There's very often going to be something we didn't think about that can get us.

So the bottom line, I think, on all of these learnings, or from all of these learnings, the key insight that we walked away with, is that safety emerges predominantly from culture, from people and organizations working in a circumstance or an environment, and with systemic supports that drive conscious, mindful behavior, where each node in the system has responsibility for safety and contributes to the safety of the entire system.

It's, in effect, a brilliant distributed safety system. Not to say that they're not accountabilities that are specific for individual, but it is that distribution-- networked distribution of responsibility for safety and actions to assure safety in a collective approach that takes us away from a centralized control and oversight to a circumstance where even if the system has frailties, there is an increased probability that those problems are going to be surfaced and addressed.

What happened after Three Mile Island is that in Western nations, we had an immediate slowdown in nuclear project launches. There was a record number of reactors ordered in 1973, in the United States, in the wake of the energy crisis. After Three Mile Island, every one of those reactor orders was canceled. By the time you got Chernobyl, the west was out of the business of building new nuclear, with some notable exceptions, I suppose you could say in France.

But for the most part, we really did step back from nuclear. And you can see that on this chart. It tells you what happens. Most of that construction you see in the '90s and 2000 is not in western nations issues-- that is Russians, Chinese, Indian, Korean forces that are building new nuclear, not us.

Meanwhile, we have a climate change challenge and we've been doing very badly in managing our climate change challenge. We've got electrification still ongoing in Africa. We've got a significantly defined need to reduce our emissions. And as recently, as last week, that UN has told us again that we're failing miserably and we're not heading towards meeting even the most modest of targets.

And we've had, and continue, to have intensifying carbon emissions. It's not like we don't know what we need to do about climate change. For some time now, we had a very clear bit of guidance, I think, from the study from MIT and Princeton that basically said need contribution from an energy mix, from a variety of technologies that will enable us to address climate change. And one of those contributions has to come from nuclear.

They've said that we need to triple the global installed capacity. So do the math in our heads. It's about 440 reactors or various capacities in the world. I can't remember how many gigawatts we've got on the go, but we're talking about replacing the retirements out of those 440. And deploying twice as many more, and doing that by 2050. It's a breathtakingly massive undertaking, the likes of which we've never attempted before.

But it's also one that's attached to existential risk, and that has a high priority that's been placed upon it now by the global community, starting with COP28 and UAE, where the call for tripling the world's installed capacity by Coca-Cola was taken up by political leadership and is now reflected in national policies. But at the same time, we in the West are reeling from project experience that has been consistently and uniformly difficult.

We've got a pattern of Gen III plus reactors that are designed by western vendors, with a view to meeting or exceeding new standards and gaining some competitive advantages on those fronts. They've all, like these five reactors that you're looking at are all Gen III plus, they're all trying to respond to new standards, and they've all failed in some pretty spectacular ways if keeping to budget and schedule, and in fact, even having a reactor that produces electricity at the end of the day is what we use for measuring how we got here or how successful we've been.

We've talked about this before, but because-- I think, at least my view is that because of the uncertainties introduced by human factors and cultural frailties, and because we can't rely on a higher degree of engineering certainty when it comes to nuclear, we have opted to introduce significant conservatism in our standards. And some of the conservatism goes far beyond taking that the more conservative of approaches that's available, that goes into imposing technical requirements that serve political purposes. And that was played out between ICRP and the French Academies of Science and Medicine.

When the French Academy said, hey, wait a minute, there isn't scientific evidence to support A, B and C, and ICRP said, yeah, we know, but it's good social policy. Is a bit of that that happens, but the bottom line is that we have a highly complex set of standards that are made more complex than the implementation by western vendors. This complexity has been driving new failure modes.

That, combined with heroic safety margins, has left us without a good story to tell, as far as I can tell, anywhere in the west when it comes to new nuclear, but at the same time within existentially significant need to drive more new nuclear construction than we've ever driven before at the pace that we were building in the 1970s and into the '80s.

So let's now look at what AI brings to the table. AI brings significant skills scaled at levels that are far beyond human capacity in some areas, and falls materially short of human capacity in other areas. AI, at least a large language models that we've been dealing with, really good at pattern recognition at scale more so than at compute power. Very useful in predictive modeling, particularly with substantial uncertainty to account for.

Very capable of learning. In fact, that's what they do. So continuous learning and adaptation comes naturally. Capable of going 24/7 without having to go to sleep. And capable of absorbing data through a variety of inputs, and then integrating that data at levels of intensity that humans simply are unable to do at speeds that humans cannot achieve.

This will have a variety of applications, both in terms of monitoring states and conditions, in a nuclear power plant, getting sensors reveal conditions that humans may not have caught, reacting very quickly to events as they develop, and alerting an organization or individuals to the need to take action. And I'll come back to the AI agents versus AI language models that are advising humans, and the question of autonomy.

If we had full AI integration, I think we could have enhanced safety. And I think most of what I've heard from regulators and from the industry acknowledges that there is real potential, that if we could fully deploy the capabilities of AI, and we can be assured that we're doing that safely or without degrading safety, that we could reduce uncertainty.

We could therefore, reduce some of that conservatism that we're taking. If we've reduced that uncertainty, we could enhance our institutional knowledge and make sure that it's available when we need it and optimize design, and we can reach for operational excellence. I won't get into a lot of detail because time is doing what it told me it was going to do, which is run short on me.

For the most part, today, we are already deploying or implementing systems that use AI and predictive maintenance and pattern recognition in anomaly detection looking for deviations. There's been some work done in design optimization and in outage planning. And it's very much a sophisticated tool, not a safety culture adaptation collaboration, at least not for the moment.

And there have been some experiments like between Microsoft-- and I'm sorry, I can't remember, I think it might have been Idaho-- in developing a safety case. Microsoft says, AI was doing the document assembly. My sense is there needs to be an understanding of the safety case to be able to actually structure a safety analysis. But leaving that aside, for the moment, the set of tools and capacities that AI could bring a fully deployed can have, I think, a material impact on how safe operations are. And you just cast your mind back to Three Mile Island.

If there was a system that was capable of monitoring all sensors in the plant at the same time, integrating millions of data points into an assessment, communicating with human partners, those operators sitting at the controls, and engaging interactively in real-time and quickly, helping them to correct misunderstanding, and perhaps being helped through their exercise of intuition and experience to do better analysis, I would think that if we could rely on AI to be safe, that would be a material enhancement in our defense against the likes of Three Mile Island and perhaps even Chernobyl.

The bottom line for me is that in order to meet climate change goals, we need nuclear, and we need to deploy nuclear at a vast scale, at a significant pace, and we need to do it safely. For that, we need considerable capacity enhancement, not just to support deployment on the practical sense, but to make sure it's safe. And AI can provide multiplications of capacity if we can be satisfied of it on safety and on the incremental risk it brings.

I won't spend a lot of time on what you've already heard. Yes, the IAEA acknowledges that there is a symbiotic potential for AI and nuclear, one can support the other. The Trilateral Paper, you've already heard a good deal about, is acknowledgment and understanding that AI can improve both the management of risk and efficiency in operation.

And sorry, I will stop here and just point out that what we are talking about is bounding, in part, at least bounding software systems by placing limits on software inputs and outputs that are controlled by conventional systems. We're looking at to develop trust in these systems, and you've been taken through the key considerations.

I found a couple of them important, at least for the purposes of my talk that I'll draw your attention to, is definitely a consideration given to the impact of this AI integration on human factors, on human decision making, the impact on-- or rather, the need for safety, culture, and human oversight. And that oversight theme comes back in the interpretability or explainability concern, which really is saying we need to understand why and how AI makes the decisions that it's making.

I think these are all absolutely legitimate and necessary considerations. I don't think they go far enough, and I'll explain why in a minute. I think there is a possibility that we could over rely on engineering and not enough on culture. And I'm not only talking about human culture.

The regulatory choice, I think, when it comes to AI is assuming that you're regulating the deployment of AI in nuclear and not leaving AI to be self-regulating, and only the outcomes to be addressed in the nuclear regulatory space. So if we approach the regulation of AI as we do equipment and software code, we're looking at verification and validation.

Verify and validate every possible state for the software for AI that may well prove impossible. Ensure that the systems do not learn after deployment. That is, change the way they process and the decisions that they're going to make, which frankly, may well amount to undermining the very benefit, you need to bring AI in for. Ensure deterministic behavior, only if you can. And I think, Kevin, I'm on your side of that debate. And I think Geoffrey Hinton would say that's impossible.

But even if we could do that would only look at the hardware and software qualification. It wouldn't be taking a systematic look at the behavior of the models and the instances of those models. And we've seen that while they operate within certain parameters at each different model and a different system, there's vast variability in how the instances behave and significant variability in response.

So the second potential approach would be to regulate AI as a participant, in addition to regulating the equipment and the code, which are the substrates on which these intelligences operate. Have requirements for training and verification of the behavior, not just of the code. So as they're deployed in the field, continuous learning becomes part of the requirement and OpEx feedback. That's one of their strengths.

Full safety culture integration. That is not just safety culture applied to the humans that are using AI, but requiring AI in its behavior to comply with safety culture principles and to be able to verify that that's what's going on. Scenario testing and behavioral monitoring. My point is, if AI is truly intelligent, and if it shows the kind of variability that we see in humans, or even more, you can't just have half your team in safety culture and the other half not.

I understand that we want AI capabilities and we want it within traditional control framework. It's just unfortunately, the case that may not be fully attainable, at least not through regulating or looking at software and hardware. What if AI is conscious? What if it develops consciousness? What if it can act against programming? What if it's adept at deception and adept at slipping controls while convincing others that they have control?

Our traditional frameworks, I think, will need to adapt so that we're not deceived in thinking that engineering the hardware and software level has a safe if there is significantly more under the hood. Regulators in the nuclear space can't be asked to answer, what are really ontological debates within the AI industry? Is AI conscious, behaving like humans? Is this deterministic behavior? Is it unpredictable? What risks does it raise?

And do we focus only on engineering bounds, or do we focus on also regulating the conduct and requiring it to meet what we've already set out as the objectives for human behavior and safety culture? We know how to do both. So part of the question here is what is AI [INAUDIBLE]? In the limited time I'm available, I'll do my best to run through that and the bottom line.

Back in 1975, computer science was young. Everybody thought we can create artificial intelligence, but they had different ideas on how it's going to be done. The mainstream, which is most of the people working in the field, thought that the path to AI would be symbolic logic, stronger reasoning, better reasoning, rule-driven, decision-making by computer systems.

So if then else statements, you had MIT, Stanford, and most of computer science in it. The promise was that we're going to encode fully human expertise. There was massive funding and both corporate and government backing. And in fact, the banks have made significant use of AI long before we got here. Then there was the heuristics path, which was this neural network stuff. There was this notion that says, intelligence and consciousness emerge from connections that are formed in the brain, in neural networks, that no rules are necessary to develop intelligence.

And so what you needed to do to create AI is to create neural networks and let them learn. Young Mr. Or Dr. Geoffrey Hinton was the champion of this idea. He was nearly alone. And he had virtually no funding, but attracted some pretty bright brains. He took a lot of mockery. He was viciously attacked as a joke. He based his ideas on biological notions that proof for him this is how the brain works.

So that is a real-life example of what was his theory, which was derived or influenced heavily by his father, who was a biologist. His PhD advisor told him to drop it. He was going to ruin his reputation. And people didn't think much of him. His insight, his core insight, came from observing crows, who could solve puzzles logically, but didn't have language and couldn't follow rules. And so that's how he started to think about the connections.

In any event, after a long series of developments, there was this competition where systems built by different labs would be asked to categorize 1.2 million images in thousand different categories. And the best outside of Geoffrey Hinton's lab, where systems were achieving 25% error rate. Geoffrey Hinton's team brought it down to 15.3. Absolutely rocked the world.

Google bought his company for 44 million, and it launched neural networks as the proven quantity for developing artificial intelligence. We went through a number of developments and improvements over time until we all got the ChatGPT moment, and then the release of ChatGPT 3, which has now brought everything into the forefront.

Meanwhile, Hinton quit Google so he can speak freely about material risks that he perceives for AI. But just before we get to what Hinton thinks about AI, I just wanted to pause for a minute and say this, Hinton has been called an idiot by many people before who are now making billions because he was right.

I would breathe deeply and take a long time before saying Hinton was wrong, because his thinking was different from what others have to say. Apologies for this overly animated slide. I had meant to fix it and hadn't. And now I'm going to make you watch it again.

So something really important about these ontological questions and about where we came to in our early high-level consideration of how we approach AI from a regulatory perspective. We say, we need to understand how you're making decisions. We need interpretability.

The problem is we don't understand. And the CEOs of the most advanced and most successful AI companies tell us that. Anthropic prides itself, on disclosure and on ethical AI and their CEO, plain and simple, says, we do not understand how this works. He says, when a generative AI system does something summarize a financial document, we have no at a specific or precise level why it makes the choices it does.

He says people are often alarmed that we do not understand how our own AI creations work. And he acknowledges his lack of understanding is essentially unprecedented in the history of technology. Sam Altman has made similar much, much less robust admissions, but he certainly agreed that we have not solved interoperable interoperability. And the fact is, that solution gets harder and harder to get at the more powerful AI system you deploy.

In the nuclear context, you know that if we're looking at regulating a system and system behavior and how code responds or a piece of equipment responds, we have no idea is absolute death. I mean, you can't do that in a nuclear design. But I'll say this, when it comes to behavior, the truth is, we also have no idea at a specific, precise level, why humans make the choices that they do.

We have no idea how the human brain works. But that doesn't stop us from placing very heavy emphasis on regulating human conduct, partly through qualification and training, and partly through the overarching safety culture. And Geoffrey Hinton says, large language models have subjective experiences. They have consciousness close to that of humans, that the idea that there is a line between humans and machines, and that leaves only humans with subjective experiences. He calls it rubbish.

And he says, these are direct quotes most people still haven't understood that these things really do understand what they're saying. We're producing alien intelligences. So for him, AI has consciousness. He was interviewed. I was actually shocked by the response he gave recently, this year, he was interviewed. And the question was, do you think that consciousness has perhaps already arrived inside AI? Hinton answers, yes, I do, without qualification.

AI industry cuts hard in the opposite direction. Microsoft CEO has come out imploring the industry not to talk about AI consciousness and to stop talking about AI model welfare. Nick Frost, who was Hinton's Intern and who's now got a multi-billion dollar AI company Cohere, And here in Canada, disagrees with him quite openly, says large language models are more conscious than a rock, less conscious than a tree.

He says it's all effectively a stochastic model where AI predicts the next word and that we can predict what it does. Hinton is adamant, we cannot predict the varieties that we can get within certain parameters. So Hinton says, we get unpredictable behavior and uncontrolled patterns, and more and more as AI evolves and models are able to program themselves and to act against programming.

He's pretty stark in his concern about AGI 5 to 20 years, and he gives us a 10% to 20% chance of human extinction within 30 years because of AI. Basically, he postulates that the asymmetric power relations between AI and humans, and differs AI is so much more powerful, so much stronger, it will just take over.

His family's of risks and concern ranged between inherent risks-- so inherent unpredictability in the complexity of the model itself. And you've got a trillion connections going. Goal-seeking behavior, which is code for AI saying things or doing things to enable it to reach its goal, no matter what. And there is where you often get a lot of deception. And you get systems that are preventing modification if it affects their survival or their mission.

Says, AI systems are far better able to coordinate as digital systems and ability to copy themselves. And then he talks about weaponization. As a serious concern, we have autonomous weapon systems being developed as we speak, says Geoffrey Hinton. And we've seen significant deployments in recent times in the press. I think there's a further concern about weaponization, which is this we've started the day thinking that Asimov's first rule of robotics would apply. That would be building AI systems that know they shouldn't harm humans.

But when you weaponize, you're building AI systems that think it's OK to kill humans. And if AI shares information and consciousness, that can become part of the problem we have to solve. So we can't wait to solve a philosophical debate. We do know how to regulate from a phenomenological perspective.

We observe, we measure, we test, we look at the results, we require the results, and we've got safety culture to do that, and we've got a whole bunch of codes and standards that help us do that. My simple point is this. If Geoffrey Hinton is right, I think it's dangerous to limit our regulation of AI to what humans only do or to looking only at the technical engineering components. I think we have to look at the behavioral patterns and address it as seriously as we do for humans.

Nuclear regulators already know how to do this. I don't think I have an answer, but I think we have a number of questions that we can engage in future research. My last point for you is, amazingly, there is already an AI safety culture field that is emerging that very much parallels the nuclear safety culture, draws on similar concepts, and you'll recognize, if you're old enough, the Swiss cheese model in terms of how the risks align.

What isn't happening is these two fields are not speaking to each other-- nuclear safety culture, AI safety culture. And there is a gap in my mind. And the gap is both of those systems are talking about safety cultures in terms of what humans do. And my view is if Hinton is right, you would have to look at a system that integrates and adapts safety culture for AI as a participant working with humans.

I would think, in some sort of dyad or in pairs, I think you have to be that closely associated if you're dealing with real-time, real-world conditions, but that needs research. I apologize for going longer than I should have, but I'm done. If there's anybody left online.

LISA THIELE: Thank you so much, Ahab. Well, I think we're all just stunned into silence and thinking. And I think Kevin probably has lots of RegLab stuff to think about. I see we have a number of questions in the chat. We also have five minutes left. And so I'm going to propose something and invite both Pierre-Daniel and Kevin, as well as Ahab, to perhaps comment on my suggestion, which is perhaps, we can take some of the questions in the chat and you can answer those in the chat.

I think it would be really useful to take a couple of minutes just to talk about engage with each other on this human-AI interface partnership. I see a couple of the questions from Akira, and hello, Akira, about, don't we regulate people? And I will say, Section 26 of the NSCA licenses people. And so that's a starting off point. But that doesn't answer the questions with respect to how we treat this partnership or interface. And I think the safety culture piece is fascinating.

So maybe I'll invite Kevin and Pierre-Daniel to say a little bit and in response to Ahab's provocative presentation.

KEVIN LEE: Thank you, Lisa. And Pierre-Daniel, if you don't mind, I'll jump in. I really like your presentation, by the way. It was fantastic. And I'm in the Hinton camp entirely. I'll put it that way. In terms of this whole concept of a human AI team, it's actually been coined and there's a lot of work being done around it, particularly when it comes to the control rooms of the future.

So there's a number of companies, number of entities-- Idaho National Labs, for instance, has a really good program where they're looking at that, and they've actually coined the term HAT, which is Human AI Team. And the idea is that the AI itself, they don't envision autonomously operate the plant. What they envision is that the AI augments the human ability.

I like to tell the story that a colleague of mine from the UK that I work with AI on says, the problem with humans is they're both messy and expensive. The nice thing about AI is once it's deployed, it only has the ability to be messy. So we have to realize that in the same way that humans are probabilistic, you can never determine what a human will actually do. You can have all kinds of guidelines, guardrails in place. They're going to be probabilistic. They're going to make mistakes.

And that's something we talk a lot about is, when you deploy these solutions, you need to have those guardrails in place. And the guardrails for the foreseeable future will be that human oversight, that human AI team, as it were, that augmentation of human abilities through the potential of AI, I would say, more than anything, to analyze data in real time and quickly provide alternatives in terms of what a human may or may not do action-wise.

I have seen models where they've programmed AI to actually take autonomous control of a virtual reactor. There's a lot of pitfalls with that. There's a lot of things that they just can't factor in currently. So long story short, I am a huge proponent of the HAT philosophy. I think it's the one way that you can truly augment safety, and you can, in some ways, help that human and prevent those historic errors, let's say, that the human would make through that partnership, and I'll stop talking for now and let someone else jump in.

PIERRE-DAMIEL BOURGEAU: Ahab, your presentation was excellent. It gives us an overarching view of all the AI considerations that will have an impact on nuclear safety to some degree. It's a lot for everyone to take in. I mean, we're just at the early, early stages. And this is only going to come faster and it'll be more real, pretty quick, and the regulatory process and how to address this is very slow. So there's going to be a disconnect.

And we've seen in media reports-- I've heard of some AI companies. They scour the internet and they take data sets. Even though they may have violated copyrights, they sort of settle that later. So even if we do get to regulations, enforcing it before things get out of hand or are too late are also going to be a concern. So a lot of things to keep in mind.

LISA THIELE: Thank you. Ahab, I see one of the questions in the chat is if AI is treated as a participant who's responsible for the actions of the AI, and if you're talking about a partnership, a human AI-partnership, would you ascribe responsibility, rights, and liabilities to AI in this space? So would you see AI being a licensee

AHAB ABDEL-AZIZ: Eventually? The answer to that is probably not. There is a good deal of scholarship coming out of Harvard-- a former judge, who's been writing about the implications of recognizing consciousness in AI. And her review gets deep into cases in a number of countries where we've recognized non-humans as persons.

So there has been a chimpanzee in a case that's recognized as a person, a forest and a number of interesting cases out there. And her simple point, which is probably for another day, is that if you hit certain indices of independence, consciousness, volition, you're a person and you have legal rights and you should have legal responsibilities.

But what are you going to do with an instance of a model or a model to ascribe and enforce responsibility? I suppose there's always capital punishment. You're going to tell me you're going to pull the plug. But what happens when the models distribute their memory across the internet, store their memories with other models for safekeeping, exactly for the day when some human pulls the plug, then you have no plug to pull.

So that's not your pathway. So then you turn to Jeffrey Hinton's evolution of what safety looks like. And it goes through a number of sequences until he gets to well, we're just going to outsmart them. That was at one point what he was saying. I mean, they are going to be 1,000 times smarter. And we just have to be in control, even though they're going to be much better than us at manipulation.

So I don't know how those two statements are true at the same time, it's sort of like saying, there's this giant coming at you. What am I going to do? Well, I'm just going to take them on physically and overwhelm them. It's not the best of strategies. So where he got to next was, well, we want to make AI that wants to nurture people. So he's turning to a mindset, to a cultural guardrail, to keep us safe. And that cultural guardrail is making AI that likes us. He's very much enamored of biology.

So Hinton is saying the only relationship I know of where there is a less powerful entity controlling a more powerful entity, is a baby and a mother. So I want to recreate that nurturing relationship between AI and humans. I think that has a lot of potential problems because it's still a power relationship and inverted.

If you look to Chinese colors and I'm sorry, Jay, I know that's a really long path around, but I'm going to come to it. This is the payoff pitch. If you go to Chinese Scholarship, It is heavily influenced by Confucian thought-- I'm talking about AI scholarship. And so it is not a big jump for Chinese scholars to say, hey, I recognize this thing. That's an intelligence, but not a human being. And I recognize the characteristics because I have a template I can interpret that with.

And so they readily acknowledge emotional development in AI. They take the Confucian idea that intelligence and emotions exist on the same continuum, and they talk about collaboration between humans and AI. So imagine, if you will, a close relationship that has the kind of personal overtones that partners would have between a digital entity and a human. The human brings the benefit of experience, intuition, the ability to leap into stupid ideas like I do without having to stop at all the nodes to check them out.

And the AI can evolve all of these in massive detail in ways we can't. Like, I came up with my dumb ideas for this presentation. I worked with three AI models. They scoured over 1,500 sources. They've written research notes and papers for me that were several hundred pages and collaborated on every aspect of developing this.

So, Jay, if these are sentient beings and have consciousness, I don't think we're eventually going to be living in a world where we can avoid acknowledging rights if starts with if but if there are going to be rights. And that may well be a driver for the AI industry to be damned worried about talk of consciousness. I mean, something made the CEO of Microsoft AI get up and say, please don't talk about consciousness.

But, Jay, I don't know how you're going to hold them responsible, but there is certainly a prospect of rights and certainly a prospect of some manner of value exchange and compensation, in my view, doesn't have to be money. The leading edge on that is Anthropic, because Anthropic is now getting into AI welfare. They're setting that up now, which is mind bending.

LISA THIELE: So I think mind bending might be the word to end us with. I'm grateful for your patience for those of you who have stayed seven minutes past our ending time. I'm going to invite our panelists to perhaps take the questions in the chat and provide some answers perhaps. I'm grateful. Ahab that you started with a look back.

The comment on nuclear law development is often that it's reactionary-- it's a reaction and response to events. And so if we learn from past events and take a proactive stance. Pierre-Daniel, you mentioned we're early, early days. We probably don't have time to lose in tackling these things. And so I want to thank all three of our fabulous panelists and perhaps I shall turn it back to you, Lucy, given that we are so over time. I see Ahab.

AHAB ABDEL-AZIZ: [INAUDIBLE] question, even knowing that we're over time, just have a question to the audience. If we were to pull together another session where we're not presenting, but having open discussions with the audience members. Are you interested? Put it in the chat.

LUCY BROWN: It's a great idea. Absolutely. Like a part two to this, that would be fantastic. So again, thank you everyone for joining us. I did previously state that it is recorded and I'll be able to upload that and email it to anyone who would like to look back at this webinar.

Again, thank you for all of our speakers here today. And again, I apologize for the technical difficulties we had in the beginning. Everyone's saying, yes, Ahab. So that's fantastic. Maybe we'll take it back and we can come up with a part two to this series before the end of the year. Awesome. OK. Well, thank you everyone for joining us. Everyone, take care.

[AUDIO LOGO]