It was about quarter past seven on the evening of October 7, 2008, when the winners of the Nobel Prize for Physics were announced by the Royal Swedish Academy of Sciences. Researchers and engineers at Japan’s High Energy Accelerator Research Organization (KEK), a vast research facility 60 km northeast of Tokyo, celebrated the news, which included an award to one of their own.
The monitoring and control center at KEK B tracks the performance and collision frequencies of beams, and beam orbital deviations, and monitors disturbance factors and beam-position locators. Source: Design News Japan
It was about quarter past seven on the evening of October 7, 2008, when the winners of the Nobel Prize for Physics were announced by the Royal Swedish Academy of Sciences. Researchers and engineers at Japan’s High Energy Accelerator Research Organization (KEK), a vast research facility 60 km northeast of Tokyo, celebrated the news, which included an award to one of their own. Makoto Kobayashi, professor emeritus of KEK and an associate of the facility for three decades, and Toshihide Maskawa, professor emeritus of Kyoto University won Nobel Prizes in physics for their discovery of the origin of broken symmetry, known as CP violation, which predicts the existence of at least three families of quarks in nature. KEK has played a pivotal role in verifying the matter-antimatter asymmetry, successfully predicted by the Kobayashi-Maskawa theory almost 30 years after it was originally published in 1973.
The KEKB-factory is a charged particle accelerator and storage ring system for experimental validation of the CP violation model. It consists of an electron/positron injector (linear accelerator, or Linac), a dual-ring accelerator, and a Belle detector. The accelerator is an asymmetric double-ring collider, with rings storing an 8-GeV electron beam and a 3.5-GeV positron beam. Beams run at nearly the speed of light in opposite directions through 3 km tunnels buried 11 m beneath the ground, and collide in the center of the Belle detector. The Belle detector record the process of B-meson decay. It was the Belle experiment that firmly established the validity of the Kobayashi-Maskawa theory of CP violation in 2002 and led to Kobayasi’s Nobel prize.
While running through the accelerator rings, beams lose energy.
Accelerator control and EPICS
In the KEKB central control room, researchers and engineers monitor locations and sizes of beams, checking activities of feedback systems and vacuum levels, analyzing orbital data, and operating servers that install new programs. The control system runs Experimental Physics and Industrial Control System (EPICS) application software. Originally written jointly by the Los Alamos National Laboratory (LANL) and the Argonne National Laboratory (ANL), EPICS is now used by many large scientific facilities throughout the world.
“EPICS first became favored for use in particle accelerators in advanced large-scale research institutes and organizations,” says Dr. Noboru Yamamoto, who is in charge of operational control at the KEK Accelerator Laboratory, “and then in astronomical observatories throughout the world. Because it adopts an open-source scheme, users can share development results and skills among their communities.”
The KEKB control system includes approximately 100 VME-based I/O controllers (IOCs). For the field interfaces, 200 VXI mainframes through MXI-2 interfaces, 50 CAMAC crates through serial highways, and 200 ARCnet segments are installed in addition to the VME frames. Many IEEE-488, RS232C devices and PLCs are also employed.
EPICS was introduced initially with the VxWorks real-time operating system, but growing numbers of IOCs now run on Linux, RTEMS,
The electron/positron Linac has been in operation since 1982. Its control system was revised between 1991 and 1993, just before the approval of the KEKB project. It runs on a separate control system comprising 30 VMEs, 170 PLCs, 15 CAMACs, 30 VXIs, 24 intelligent oscilloscopes, many Unix computers, and redundant gigabit Ethernet networks. This system’s design was based on the use of de-facto standards such as Unix, VME, and TCP/IP with the use of optical Ethernet networks for all device controllers without any special field networks.
Furukawa points out that script languages, such as SADscript, Python and Tcl are an important part of the software landscape at KEK. “While hardware controllers are programmed in real-time environment on IOCs, many of the operational algorithms are implemented using scripting languages,” he says. “Researchers and engineers rely heavily on scripting languages for rapid-prototyping activities.”
The SAD (Strategic Accelerator Design) script was developed by KEK to calculate beam operation of the accelerator. It includes Mathematica-like list-processing functions that enable rapid development of online operational software. “SAD allows researchers to test beam-adjustment ideas in model-based design,” Furukawa says.
KEKB involves slightly over 100 engineers and researchers, while the Belle project involves 200 to 300. “Utilizing the common tools of SADscript and EPICS allows sharing a common project overview,” says Furukawa.
Beams deviating from their orbits are at risk of jumping into a damaging the Belle detector. Beam-mask equipment installed in two places per ring prevent this. A Linux PLC controls position of this beam-mask. Source: Design News Japan
EPICS and PLCs
“In order to use resources efficiently to save cost and construction time,” says Yamamoto, “existing facilities at KEKB have reused many kinds of I/O equipment from a previous electron-positron collider called Tristan. We also installed ARCnet I/O boards in the power supply controllers when we began to build KEKB control systems. With such a big project, management has to weigh cost-performance and ease of maintenance when choosing field communication systems.”
“But from the viewpoint of application level,” he continues, “all differences of field bus specifications will be absorbed at the IOC level. EPICS will ‘hide’ all differences, so maintaining EPICS integration is the highest priority for us.”
For some bus systems, such as GPIB and Modbus, EPICS standard interfaces are available from other user groups in the EPICS community. But KEKB has developed other interfaces on its own, such as for PLCs.
PLCs were introduced into KEKB rings to control the power supply system, and they seemed to belong not to the control group but to the equipment group. Additional installations were made when KEKB introduced a system to protect engineers from radiation exposure. “PLC use expanded even to accelerator control in areas where high speed is not required,” Yamamoto reports.
There were several reasons that KEKB accelerated the adoption of PLCs in their control systems:
PLCs cost less per channel than VME boards. “Compared with VME boards, we saved one third to one fourth in costs,” Furukawa says.
Engineers and researchers felt dividing job functions would be easier because of the PLC hardware-software structure versus that of VME boards. “PLC installations and programs are handled by the equipment group, while networks and upper EPICS connection are taken care of by the control group,” he explains.
Testing of programs and systems was also easier with PLCs. “For several reasons, it was not as easy to test software for VME boards in collaboration with board manufacturers compared to PLC manufacturers,” he adds.
Some control groups cite PLCs’ ability to provide HMI functions through touch-panel displays.
It takes 10 microseconds for particles to orbit the 3 km KEKB ring. 2000 electro-magnets regulate that circulation. KEK B is testing two sets of Linux PLCs to control one of the pulsed quadrupole magnets. Source: Design News Japan
Recently, advanced particle collide accelerator facilities in Japan have installed EPICS PLCs and adopted Ethernet for control. This helps reduce costs and software development time, but there have been some problems, such as compartmentalization of the logic between ladder language software and upper level software. Furukawa says: “Engineers would develop PLC logic programs in ladder, but occasionally control-group engineers have designed logic functions in EPICS, complicating control-system maintenance.”
“Increases in programming learning costs is another issue, because you now need to learn ladder programming as well as EPICS.”
Future accelerator control
In terms of network protocols, Furukawa says, “when we hook up ladder CPUs through Ethernet, we have a two-layered communication system; EPICS server software communicates with PLC protocols in the lower layer, and with EPICS’ protocol on the upper layer. If we could make a single-layered network, we could simplify development tools and debugging procedures.”
Yamamoto added, “Because there is network communication between the CPUs of PLCs and VME computers (IOCs) that handle protocol exchanges, programming of the device/driver support tends to become quite complicated. But it is clear that the entire control system should be integrated in EPICS. That’s why we started to investigate running PLCs on embedded EPICS.”
Recently, automation manufacturer Yokogawa introduced a PLC having a CPU running on Linux. Using the beta version of this model, KEKB and two other EPICS users launched a joint project to develop a PLC that can be operated as an EPICS IOC.
There are several advantages to this approach. Linux follows open source license procedures under the General Public License (GPL), while EPICS is also open source. Thus, the collaborators envision creating a completely open source system. The Linux PLC can be connected with conventional ladder-programmed PLCs, allowing users to reuse both hardware and software resources.
If users try to design a new control system from the ground up, such functionally shared processing could also be possible, as complicated logic would be processed by the Linux PLC, while simple processing can be handled in Ladder. EPICS-embedded PLCs have already been installed, and have been used in both test and live operations. “Since the end of July,” Furukawa says, “we have been evaluating the first EPICS-embedded PLC. It controls a beam-mask in a KEKB ring. Then, we added two of them for control of pulsed quadrupole magnets. We were cautious at the beginning, but there has been no trouble so far.”
Furukawa says, “Ethernet connectivity will be the mandate selected for future accelerator control in devices and components. It’s our hope that equipment engineers can develop system components without any influence on the entire control system.”
“Once they complete the development of system components,” he continues, “it’s preferable that the components be able to receive services from the entire control system. At the same time, it will be important for components to funnel their new functionality to the entire control system.”
“To achieve this,” he says, “EPICS IOC software should run on front-end controllers. Thus, in remote I/O control, Linux PLC applications will expand in the accelerator field in the area of the relatively slower controls at the millisecond-level. But also, there could be other solutions developed for faster control.”
Author Information
Shin Kai is editor-in-chief of Design News Japan. Contact him at [email protected]
