Doing a Double-Take: SCADA replication software refines Carlsberg plant operations

At the Leeds, U.K. brewing & bottling plant of Danish brewing group Carlsberg, an asynchronous fail-over and replication solution from Double-Take Software is keeping production going when server failures take critical systems off-line.


At the Leeds, U.K. brewing & bottling plant of Danish brewing group Carlsberg , an asynchronous fail-over and replication solution from Double-Take Software is keeping production going when server failures take critical systems off-line.
Brewing three million hectoliters of beer each year—including the company’s popular British brand, Tetley's—and operating 24 hours a day, server failure used to hit production hard.
Over the last five years, explains Senior Systems Engineer Graeme Walker, server failure has knocked out the brewery’s SCADA system twice. On each occasion, he relates, it has taken IT engineers between two and four hours to replace the hardware and reinstate the system. And with its SCADA systems inactive, brewing operations would grind to a halt—with a four-hour interruption to brewing the equivalent of a loss in production of approximately 190,000 liters of beer.
In mid-2007, explains Walker, the decision was made to upgrade the plant’s Intellution SCADA to a more current version—now sold and supported under the GE Proficy banner. But while this contained an improved fail-over capability, the extent of the data replication wasn’t total. Recognizing that the upgrade would require new servers, Carlsberg began to explore other options.
"Given that the fail-over mechanism didn’t meet our requirements, we decided to look at an external solution,” says Walker. “We initially considered a Microsoft cluster, but it seemed very complex and the cost was prohibitive. Instead, Polestar , our IT partner, suggested we take a look at the data replication and fail-over capabilities of Double-Take Software."
A test system was set up to see if the Double-Take solution would deliver the required fail-over capability. A model of the proposed infrastructure was built by establishing an ethernet connection between a production unit on the plant floor and a "target" server located elsewhere in the facility, to which data would be replicated. Using a variety of methods to recreate realistic failure scenarios, Carlsberg engineers then caused the production server to fail—monitoring whether the fail-over process worked, and if all the required data was present.
“In every test, the Double-Take solution failed over and failed back seamlessly. After the trials were completed, we had no hesitation in giving Polestar the green light to implement the full solution,” recalls Walker.
It was an evaluation process that was fairly typical, adds Double-Take’s Worcester, U.K.-based Sales & Marketing Director, Ian Masters. Some of the company’s largest customers are in the manufacturing industry, he notes, with a common requirement being a low server load.
“With asynchronous replication, there’s no real performance impact: The CPU load is just 3 or 4 percent,” says Masters. “Asynchronous replication doesn’t guarantee total replication. Over IP networks, packet loss can happen, but the data on the target system will be usable,‘crash-consistent’ data. We’re hardware- and application-agnostic, and all the customer needs is a Windows operating system.”

Carlsberg Senior Systems Engineer Graeme Walker says asynchronous fail-over and replication software from Double-Take is keeping production going when server failures take critical systems off-line.

Carlsberg’s server upgrade took place in January, relates Walker, with the SCADA upgrade following immediately. From a previous installation that he characterizes as “15 servers and no redundancy”, the plant now has five primary SCADA servers; two secondary servers used for server databases, pictures, and recipes; and two servers dedicated to the role of replication target servers.
And in addition to their back-up role in the event of server failure, the replication servers also offer operational benefits, he adds. Server maintenance, for example, has been greatly simplified.
“If we need to investigate problems, install patches, or implement server or software upgrades, we manually fail over to the target units, leaving us free to work on the production servers without compromising performance,” says Walker. “Another benefit of having two identical datasets is that we are able to run our tape backup from the target servers. This means that the back-up process doesn't interfere with production performance, and we run the backups during standard working hours.”
Carlsberg recently had genuine need to call on its new fail-over capability—just weeks after going live. “Some plant engineers accidentally cut the power supply to a production server,” says Walker. “On the shop floor, the operators didn’t notice a thing.”

No comments
The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by...
The System Integrator Giants program lists the top 100 system integrators among companies listed in CFE Media's Global System Integrator Database.
The Engineering Leaders Under 40 program identifies and gives recognition to young engineers who...
This eGuide illustrates solutions, applications and benefits of machine vision systems.
Learn how to increase device reliability in harsh environments and decrease unplanned system downtime.
This eGuide contains a series of articles and videos that considers theoretical and practical; immediate needs and a look into the future.
Choosing controllers: PLCs, PACs, IPCs, DCS? What's best for your application?; Wireless trends; Design, integration; Manufacturing Day; Product Exclusive
Variable speed drives: Smooth, efficient, electrically quite motion control; Process control upgrades; Mobile intelligence; Product finalists: Vote now; Product Exclusives
Machine design tips: Pneumatic or electric; Software upgrades; Ethernet advantages; Additive manufacturing; Engineering Leaders; Product exclusives: PLC, HMI, IO
This article collection contains the 5 most referenced articles on improving the use of PID.
Learn how Industry 4.0 adds supply chain efficiency, optimizes pricing, improves quality, and more.

Find and connect with the most suitable service provider for your unique application. Start searching the Global System Integrator Database Now!

Cyber security cost-efficient for industrial control systems; Extracting full value from operational data; Managing cyber security risks
Drilling for Big Data: Managing the flow of information; Big data drilldown series: Challenge and opportunity; OT to IT: Creating a circle of improvement; Industry loses best workers, again
Pipeline vulnerabilities? Securing hydrocarbon transit; Predictive analytics hit the mainstream; Dirty pipelines decrease flow, production—pig your line; Ensuring pipeline physical and cyber security