DeviceLine Radio: Joe Weiss On Industrial Control System Security

My guest today is Joe Weiss, Managing partner at Applied Control Solutions. He is perhaps best known for his work with securing electrical utilities and his testimony on Capital Hill on the danger in not addressing various vulnerabilities facing that industry today. But Joe points out that Industrial Control Systems are everywhere—in our cars, even in our bodies. So the safeguards introduced in one area should apply to all areas. I started the conversation by asking why, in the two years since Dillon Beresford and Luigi Auriemma first poked holes in SCADA systems, we haven't seen more progress on securing those systems.

You can hear the full conversation, along with a recap of this week’s news here.

Or subscribe to DeviceLine Radio on iTunes.

Or read the transcript below.

Robert Vamosi:
OK, so let's start off. It's been a while since Dillon Beresford and Luigi Auriemma and others hacked into various electrical grid systems. What has been the response from utilities and government officials that you've spoken with about those theoretical attacks?

Joe Weiss: From certain people in the government there was concern, but in general from the industry, there's been very little response. In fact, I think there are an awful lot of people in the industry that don't even know that happened.

Robert: Hmm? Is that a fault of communication within the industry or the fact that it appeared in the media and therefore it can be discounted as not being accurate?

Joe: Everything I'm giving you is my thoughts. These are very subjective things, so it's not like being able to point and say, "This is precisely why." But I think there's a couple of things going on. One if you're talking about electric, electric in general is dominated by NERC CIP. What Dillon did, what Luigi did were systems that generally were not addressed by NERC CIP. It sounds crazy, but they weren't. You have an awful lot of people who basically brushed it off because it ostensibly doesn't affect them.

Number two is there's a tendency to brush it off as just another researcher throwing out possibilities. I think the other is...and it's this huge dichotomy we're living with to this day, which is you've got the security world that understands security, but doesn't understand our systems, and so takes this pretty seriously, and then in general, you have the people that operate these systems, whether it's a grid or a pipeline or whatever, whose job is not security, they don't necessarily understand this, and so just basically brush it off, "It doesn't bother me, doesn't affect me. Why should I care?"

We've got that gulf, and that gulf is getting, I believe, wider, not narrower. I just put a blog out literally maybe a half hour ago at, about a meeting I had with a utility and a very, very major ICS vendor, literally last week, where we're trying to do a test bed. The person who was there from the ICS vendor was the security person who came, if you will, from outside the industry, so looking at life from the traditional security perspective, and just this gap between what he's looking at versus what operational end users are looking at is humongous.

Robert: It sounds like the utilities that you're speaking of are compliance driven, that if it's not something that they might be penalized for, they're not going to necessarily worry about it?

Joe: I'm going to turn it around and say it a different way.

Robert: All right.

Joe: I am working with the only electric utility in the United States that I am aware of that number one, is doing an Aurora hardware mitigation project, and is willing to be a test bed for evaluating control systems cybersecurity solutions, because this particular utility, this is the ultimate irony, it's small enough that it doesn't have any NERC Critical Cyber Assets, which means it can be an engineer and do the right thing. Isn't that an incredible statement, that arguably about the only utility in the United States that is going to be secure or trying to be secure is doing so because they don't have to meet the NERC CIPs?

Robert: Hmm? Now...

Joe: Is that nothing but an irony?

Robert: Yes. Now, before the call you pointed out to me that the phrase "industrial control systems" goes beyond electrical. Can you expand on that a little bit?

Joe: Yes. An industrial control system, in fact, even the word "industrial" gets to be a bit interesting, but a control system is something that measures and then controls a physical process. It could be you're measuring temperature, you're measuring the amount of flow, you're measuring the amount of oxygen. I deliberately used those three, because that could be a power plant. Believe it or not, I just described the human body. I'm also describing what goes on inside the hood of a car, and what goes on inside a water treatment facility. Any time you're measuring, this is even an amusement park, you measure speed, and then you automatically adjust based on speed, a control system.

The reason these are so important is these are the systems that are trying to control physics, and you can't fool Mother Nature. If you do something wrong here, physics will react in a most likely nice way. That's why this is so important is you're dealing with the control of physical processes. Does that makes sense?

Robert: Yes, that makes sense. Again, to pick on the media, we think of the lights going out in our city, but can you give us another example that's equally important, or maybe even more important?

Joe: We had a water utility inadvertently, this is cyber, pump water from a Superfund Site into the drinking water system, very not good. We had a train crash into the rear of another because the signals weren't working properly, the alarms didn't go off, et cetera. Nine people died. We had a couple of cases where pipelines ruptured because of control system issues. People died in these. Not in the water case, that I know of, but in the others, people died. What you're talking about, we've had any number of cases with, for example, major discharges of sewage. We've had cases where assembly lines stopped.

If you look at Stuxnet, it was the destruction of equipment. Aurora, the aurora that was demonstrated at the Idaho national lab, destroys alternating current, three phase motors, and generators. This is more than just lights. This is any industrial process.

Robert: Now, we set the stage for all the bad things that are going on. What are some of the mitigations that can be done to keep these from happening?

Joe: A few things. Number one, which I think is first and foremost, senior management has to understand it's real. Our biggest problem to date is senior management believes cyber security is a problem, particularly on the IT side, because of things like Sarbanes Oxley, so they take that very seriously. But senior management doesn't take cyber security of the control systems near as seriously. If they did, there's no way in the world they would be satisfied with something like the NERC Critical Infrastructure Protection standards, because they won't protect systems from cyber attacks. Certainly, they won't protect them from many cyber attacks. I'll put it that way.

Number one, senior management must get involved and take it seriously. Number two, you need to find out what you actually have installed. I use the word "almost," never say never. Almost no entities know what they actually have installed, and what of it is actually cyber vulnerable. If you don't know what you have, how do you know how to protect it?

The third thing is to have control system cyber security policies and procedures. Not IT, these are not IT systems. If you had those things done, I believe you've got 75 percent of the problem solved. Now, the next part is, I'm working with, like I say, arguably the only utility in the country willing to be a test bed for securing control systems.

One of the things we're doing here is we're really trying, in a real environment, to find out what really does work. Most of the solutions [clears throat] out there for securing control systems, and I use "most" because there are a number that are not this, but certainly most, essentially are repackaged IT solutions with the word "SCADA" in front.

Part of what we're doing is trying to find out, "Do these solutions really work?" Moreover...really, more importantly, "Do these solutions actually hurt the control systems?" In other words, "Is the cure worse than the disease?"

This utility is giving us an unparalleled chance to really figure out how to secure a real industrial facility and maintain the reliability of the systems in that utility, which will be transferable to every other industry.

Robert: That's a good segue into my next question. You and I were both recently at the San Francisco Electronic Crimes Task Force meeting, and it focused on medical device security. You noted, in the discussion that followed, that a lot of the things that they were talking about could apply to ICS. How do these different groups talk among themselves? I think we've gotten good at creating the ISACs and the InfoGuard's for respective critical infrastructures, but how does that knowledge base jump out of that box and over into another box and another box, where applicable?

Joe: First of all, I wanted to say one thing. I believe that InfoGuard, and a lot of these other organizations, really don't work for the control system community. If you'll notice how few people were at that meeting we were at two weeks ago Thursday, on medical device security, who were actually from the control system community. There's only maybe 10 or 12, maybe 15, major control system suppliers throughout the world, and they all supply internationally and multiple industries. It is really, really, really important that the information be shared. [clears throat] There are not very many good vehicles for doing that. The International Society of Automation, ISA, has a standards body called ISA99, which is automation and process control cyber security. It was deliberately decided early on not to do it for electric, and not to do a separate one for oil/gas, and a separate one for chemicals, because we all use the same systems.

[clears throat] The meeting we were at that you're referring to, I went up and asked a couple of the people there, one from FDA and the other from the Medical Device Innovation, Safety and Security Consortium, about working with us in 99, because what they were talking about are truly industrial control systems. That includes the big pharmaceutical companies themselves, the pharmaceutical manufacturing. We need to get people to understand, at the cyber level, these things really aren't different, and what happens in one industry directly affects another.

Robert, you're aware, I've been holding a conference since 2002, strictly and only on cyber control systems, and it doesn't use the word "electric" in it because we want to have representatives from all of these different industries, because they all have the same issues, and it's really important if...I don't believe this is really that well understood yet by many in Washington. The point too is, as soon as we can secure one industry, any industry, we can secure most of the other industries, which also means none of the industries yet have been secured.

Robert: You mentioned your work with the standards body. How far along is 99, and how is the participation? Is it just starting, or are you guys wrapping things up and we'll soon see that standard?

Joe: Well, [clears throat] first of all, the standard started probably about six years ago. It's a consensus standard, so that means it takes a very long time to get people to agree. There are something like, I forget the numbers, I should know better. But probably two or three hundred organizations worldwide participating. Cyber security of control systems is a very big animal. There are a whole bunch of different pieces to it. [clears throat] There are a number of ISA99 documents that are already on the street. There are a number more that are in process. This is not like putting out one document, because you're talking about risk assessment. You're talking about technologies. You're talking about certification, certification both for equipment, for people. You're talking about, like I say, assessment methodologies.

There is a whole slew of standards, or recommended practices, that are needed, and so it's an ongoing process, and we are still learning. We're still seeing incidents that surprise us, that we didn't think could happen, that have. We've got to go back and start asking ourselves again, "Is this comprehensive enough?"

Robert: Is there a projected end date?
Joe: No
Robert: [laughs] OK
Joe: I say that with somewhat tongue in cheek. There are end dates for the different products within the standards organization, but two things happen why I say no. They're...every four or five years, or six years, I forget the number, you have to essentially reissue the standard. But number two, this is a technological area where things change. It's not fixed where you can have a standard that's based on the fracture mechanics of a pipe. That physics isn't going to change. You know what's going on there.

The industrial controls world has found itself...which have fairly stayed old line area, because of networking and everything, has found itself in the middle of this technologically evolving area. We have to be able to stay abreast because we use it. The same question would be to the IT world or to the Internet world, "When are you going to be through with your standards process?"

Robert: Fair enough. All right Joe, thank you very much for your conversation, I appreciate your insight into this area.

Joe: Thank you very much for the offer, and allowing me to do this.