The latest success of Google’s AI group: fusion reactor control

As the world waits to build the largest fusion reactor to date, called ITER, smaller reactors With similar designs still running. These reactors, called tokamaks, help us test both hardware and software. Hardware testing helps us improve things like the materials used for the container walls or the shape and location of the control magnets.

But it can be said that the program is the most important. To enable fusion, the tokamak’s control software must monitor the state of its plasma and respond to any changes by making real-time adjustments to the system’s magnets. Failing to do so can result in anything from low energy (resulting in any fusion failure) to seeing plasma leaking from the containment (and burning the container walls).

Obtaining the right control program requires a detailed understanding of both the control magnet and the plasma that the magnet handles. Or it would be more accurate to say, get this control software right you may ask. Because today, Google’s DeepMind AI team announces that its software has been successfully trained to control tokamak.

out of control

The development of control software for tokamak is a complex process. Based on previous experience with similar designs, engineers can glean some of the basic principles needed to run the software, such as which sensor inputs to read and how to respond to changes in them. But there are always quirks that depend on the design of the devices and the energies of the plasma used. Therefore, there tends to be an iterative process of scaling and modeling, followed by adjustments to the control process, while still maintaining enough performance for near-real-time adjustments.

The resulting control program tends to be somewhat specialized. If researchers want to experiment with a completely different plasma geometry in the tokamak, a major revision of the program may be required.

Researchers in this field had already identified artificial intelligence as a possible solution. Give enough examples that are suitable for artificial intelligence, and they can tell which control configurations produce the desired properties in the plasma. This will free people to focus on the desired end state they want and then let the software produce it for them so they can study it. AI must also be more flexible; Once trained in how to control the system, it should be able to produce very different plasma configurations for study without the need for reprogramming.

To move forward with this idea, all we needed were experts in AI and tokamak. For the new research paper, the AI ​​team came from Google’s DeepMind division, which is known for developing software that can handle everything. protein folding to StarCraft. Tokamak comes courtesy Swiss Plasma Center At the EPFL in Lausanne.

practice fusion

Since putting AI on actual machines during the training process could be a disaster, the team started with a tokamak simulator intended for Swiss Plasma Center machines. This was pretty accurate, and they programmed a boundary into the AI ​​which prevented it from directing plasmas into the configuration where the simulator would produce inaccurate results. DeepMind then trained its deep learning software to access a variety of plasma configurations by letting them control the simulator.

During training, an overlapping layer of the software provided a reward function indicating how close the properties of the plasma were to the desired state. Another algorithm, called the Critic, learned the expected rewards for various changes of the tokamak’s control magnet. These were used by the actual control neural network to know what actions to take.

The critic was elaborate and computationally expensive, but was used only during the training part. When trained, the control algorithm learned what actions to take to reach a variety of situations, and the critic could be ignored.

In order to allow real-time performance, the trained console was compiled as an executable. A standard control program will be used to activate the tokamak and bring the plasma to high energies. Once the plasma was stable, I handed control over to the AI.

it’s working!

The resulting software performed pretty much what you’d like it to be when you put it on actual hardware. The software could control experimental runs that targeted different conditions over time — in one test case, it boosted the power, kept the plasma steady, then changed the plasma geometry, then moved the plasma inside the tokamak before increasing the power again. In another case, it contained two separate structures of plasma in the same tokamak simultaneously.

The paper describing this work contains a large list of things the authors need. This list includes a tokamak simulator that is detailed enough to be accurate but compact enough to provide feedback quickly enough to make reinforcement learning possible. The training set had to include both common conditions similar to where the control was handed over and unusual conditions that would allow it to learn how to transfer those formations to the experimental ones. In addition, the researchers needed to develop software detailed enough to evaluate a wide range of potential control options, but also be able to train a fast-performing controller that could be compiled into an executable file.

The people behind this work are also excited about what might portend future work. Rather than limiting things to the modeling of existing devices, they suggest that it should be possible to give an iteration of this program the desired configuration of the plasma, and allow it to define the hardware geometry that allows it to create that. Alternately, it can improve the performance of existing devices.

Now we just need to wait for an AI-worthy fusion reactor.

Nature, 2022. DOI: 10.1038 / s41586-021-04301-9 (About DOIs).