I wonder why they needed to actually destroy a relatively expensive piece of equipment, plus spend time & money on all the orchestration around this test.
They clearly knew they were capable of reprogramming the control logic remotely (otherwise this idea for a test would've gone nowhere) and any engineer could've predicted this obvious outcome without the need for a real-world test.
Telling a business person they have a software problem is meaningless. Showing them losing a LOT of money because they ignored it gets attention. Ask any hospital about ransomware. Our local hospital is still running Windows 7 and older versions.
Everybody loves network connected new cars. Nobody thinks about the fact that they are "fly by wire", using software to control everything. Who thinks about changing the battery controller to cause a battery fire? Or controlling the acceleration to maximum? Or setting off the airbags? Or...
Use your imagination. What could go wrong in a 2 ton computer on wheels? Especially one that has a camera and software that can recognize "soft targets" (i.e. people).
It really should be illegal to have network connected cars. Any software security engineer knows that. But you can't convince a business person.
> It really should be illegal to have network connected cars. Any software security engineer knows that. Everybody loves network connected new cars. Nobody thinks about the fact that they are "fly by wire"
I fully agree that automotive and medical equipment/IT could greatly benefit from not having security as an afterthought.
However, I don't know any vehicle with connectivity (other than that Jeep Cherokee controversy), which does not have safety critical CAN/FlexRay buses segregated from user facing 'infotainment' systems.
What that means is that the network bus in which your 'compromised' infotainment system is able to operate is completely separate from Engine, ABS, AEB, ESP, Airbags etc.
The solutions vary but there is usually a physical gateway that prevents a passthrough MITM attacks, so for example you cannot simple send a message frame from your infotainment pretending to be an Emergency braking request to your ABS system.
What justification do you have for asserting that the safety critical systems are actually separated/isolated? The people at Jeep probably thought their systems were secure, yet the researchers on the Jeep Cherokee attack demonstrated they were not and claim that they could have simultaneously affected 471,000(!) vehicles [1]. Do you think anybody at a car company would dare to claim that their systems are safe enough to bet the lives of 471,000 people on them and take responsibility if they were wrong? Do you think any engineer would, in good conscience, support such a statement and share that responsibility if asked directly? I doubt you could find a single engineer that would feel even remotely comfortable that their procedures are good enough for 1/100th that number and you would never find an executive who would dare claim such a thing in a legally binding way.
Absolutely zero benefit of the doubt should be given to systems where a single error could cause grievous harm to tens of thousands of lives. Such systems should require an absolutely ironclad public legally-binding positive assertion of security with strict criminal liability for failure (blame can not be shifted) that is independently validated before we should even think of accepting their use. We already accept nothing less for systems that are less dangerous such as nuclear reactor meltdowns and systems that are equally dangerous such as nuclear weapons systems. If systems with failure modes many times worse than the atomic bombings of Japan can not be made safe, then they must not be made. Anything less would be so criminally irresponsible that we lack a word for the magnitude of irresponsibility.
> What justification do you have for asserting that the safety critical systems are actually separated/isolated?
I don't need to justify anything, I am stating they are isolated in all the vehicles I am familiar with. [0] The example from the Jeep Cherokee is a pretty crude architecture defect, the only way that could happen is in vehicles with a single CAN bus, which really is something out of the 1990's.
As constructive criticism you really ought to read the "In Comments" section of the guidelines at the bottom of the frontpage.
This is wrong. The Cherokee hack existed because both CAN buses in question travel to the infotainment system. The vulnerability came into being by finding a path between them. It became remotely exploitable when they found a way to do the same thing over the onboard cellular modem. So no, your definitive crude architecture defect and reference to the 1990s is a misunderstanding of the actual problem.
This is a common FCA design. CAN HI and CAN LO often go to the same device on different pins. My Jeep has an instrumentation bus as well and all three arrive in one box in multiple places within the vehicle. Friend with a Rubicon Wrangler discovered both engine buses connect to the electric motor to disconnect the sway bar. He discovered this when water got in it, shorted it out, caught the electric motor on fire while he was watching, and subsequently totaled the ECU and TIPM by transmitting electrical noise on both CAN ECU buses.
You’ve assumed one bus goes to one thing to reach your conclusion. That is faulty logic, which is no shortage of ironic given the parts of your comment I’m mostly ignoring in this reply.
Something doesn't parse for me in your second paragraph - CAN HI and CAN LO are 2 parts of the same bus (each bus has 2 wires per the differential spec) and so of course they go to the same device on different pins. Am I missing something about the Jeep architecture?
They’re two separate buses, high rate and low rate, for different purposes throughout the engine and components. Not the literal high and low pins of each bus. High rate, low rate. Think cylinder timing communication speed versus, say, fuel usage. CAN operates at a fixed speed, and I understand high and low rate buses to be a common design (they’re two CAN standards, high is something like a megabit).
My code reader has fetched “CAN HI” before and the code was contextually referring to the high-rate bus, not the physical wire. I’m not a deep Jeep tech, just dangerous electrically and with a spanner, so I may be wrong in how I’m spelling those and I’m following the lead of a trouble code. If you’re nice to a service dealer with a laptop you can get a quite lovely wiring diagram that explains it better.
You state in a different comment chain that bus segregation does not mean no communication, so I do not understand how that is consistent with your claim that they are isolated.
The Jeep Cherokee also did not have only a single CAN bus as evidenced by the diagram on page 8 [1]. The problem was that the radio unit was on both CAN buses.
I ask for justification of your statement because you counter-argued the previous comment that stated: “It really should be illegal to have network connected cars. Any software security engineer knows that.” by stating that the safety critical systems are segregated and thus “the network bus in which your 'compromised' infotainment system is able to operate is completely separate from Engine, ABS, AEB, ESP, Airbags etc.” which implies that solution achieves the desired security properties. I claim the desired security properties are: “safe enough to bet the lives of 471,000 people on them and take responsibility if they were wrong”. I think, and I think most people would agree, that is accurate assessment of the potential consequences of catastrophic failure. That is a consequence comparable to the detonation of a nuclear warhead in a city, so we should apply similar standards to the required level of security. Hardly any software engineer would make a claim that their software can be trusted with an amount of lives within 3 orders of magnitude of that number (note 3 orders of magnitude is 471 which is comparable to an airplane crash, so 3 orders of magnitude less is critical avionics level). Therefore, making a claim that implies that level of security has already been achieved is an extraordinary claim and should thus require equally extraordinary evidence to be believed.
Even with such justification I would still caution anybody else reading it since it would not be a legally binding claim by a liable entity, so even assuming you are commenting honestly and to the best of your knowledge (which I assume you are), and even correctly describing the truth as it is today, there are no consequences for any liable entity if they betray your trust. It should always be up to the liable entity to justify themselves to the people to our satisfaction before deploying systems with such societal-level consequences.
This is exactly the sort of justification that I am cautioning against. Would it be great if they did take the problem seriously and solve it? Of course. But it is not our duty to justify their decisions when the consequences of their decisions may cause grievous harm to thousands. It is their job, the company's, to justify to the satisfaction of engineers and the people that we should trust the lives of thousands to millions on their systems. If anything, it is our job to be skeptical of their claims and grill them ruthlessly before they should even be allowed to consider deploying such systems. I mean, think of any other software system, would you trust anybody's lives to any of them? Would the engineers on those projects agree? Most software engineers I know would be horrified if their systems were used in something responsible for even a single life let alone thousands. The claim that we should trust these companies with hundreds of thousands of lives is an extraordinary claim that they should be required to provide extraordinary evidence for. And, even if we are convinced, they should still be ultimately liable for their choice of decision to make sure they are actually putting their money where their mouth is. It is what we demand from bridge builders, it is what we demand from nuclear reactor designs, it should be what we demand from mass-produced internet-connected safety-critical devices.
That's not what "by wire" means though. My understanding is the steering wheel and brakes are mechanically attached to the steering and braking mechanisms.
The article we're discussing would suggest that the attack surface for any kind of complex machinery is very wide. A car doesn't need to be drive-by-wire to be potentially endangered by malicious code affecting the engine or battery.
The problem is that there is an increasing overlap between physical vehicle control systems, cabin controls and displays, and remote communications. For example, how do you design a system that can automatically call for help in the event of an accident without giving that system access to the sensors that also trigger airbag deployment and similar safety features? How do you build "self-driving" automation (or even less ambitious driver aids that are already in widespread use) without relying on sensors observing the environment around the vehicle, which may be subject to interference or deliberate deception?
All the use cases you described are already implemented in today's cars, bus segregation doesn't mean zero communication.
Gateways can allow for certain messages to pass-through, the point is that they preserve the garantee that certain safety-critical messages can only originate from within a specific network. It's basically a crude MITM mitigation.
> All the use cases you described are already implemented in today's cars, bus segregation doesn't mean zero communication.
That's the problem, though. Once bidirectional communication exists, you can pretty much guarantee there's an exploitable hole in it that can be used to break the firewall.
Data diodes exist and have been used in high security or other risky cases when one circuit or bus absolutely has to be isolated except for one way communication.
In practice it's not typically necessary to go quite that far if a trusted communication processor is able to pass messages or set registers but only ever takes logic control flow input from the "safe" side. Plenty of equipment that wants to broadcast state over radio but never accepts any sort of input back beyond acks or whatever has done this for a long time.
I was talking about two-way communications. Maybe you can securely isolate a bus that has the insecure->secure side reduced to super trivial communication. But I won't trust anything that runs multi-layer software stack and a complex communications protocol; from what I've learned about software security, there's always a bug or a backdoor that can be exploited to elevate access or just break the secure-side components.
(This is, of course, entirely my opinion. I haven't seen the code of these systems, but looking from the outside, nothing I've learned makes me feel like I can trust their security.)
When I saw arbitrary code execution on the NES in Super Mario 3 done with only controller inputs I decided any notion of software security in a system more complex than, say, a microwave oven, wasn't achievable.
I recently heard an "On Star" ad that proclaimed that they can "slow down stolen cars". Consumers are wowed by features that, by definition, are enabled by unwise security decisions. Manufacturers will cash in. IT security in cars will get worse.
Data diodes exist and have been used in high security or other risky cases when one circuit or bus absolutely has to be isolated except for one way communication.
But how can you include adequate firewalls in a car that, for example, receives OTA updates for autonomous driving software? It is inherent in any such system that incoming signals are received and that the software that may be affected by those signals has access to pretty much everything.
These systems are crude but they implement a physical airgap.
You would have to hack a gateway (on-site) or physically access a segregated bus, at which point you could argue that you could also physically tamper with brakes or engine without any software being involved.
Unfortunately, a physical air gap won't prevent a malicious actor from, for example, projecting a misleading image designed to confuse your automated driving systems when they process that image and so prompt an adverse and potentially dangerous reaction. This is not only about the communications channels in the internal architecture, it's a much broader problem than that.
Please read the whole thread, this is going completely off-topic from the original discussion in the start of these comments.
The original claim was that vehicles should never be connected to any network because of unspecified online attacks that could actuate brakes or steering. I explained why this was unfounded.
Adversarial patterns against computer vision based ADAS are only a real issue for system which are not sufficiently redundant . Autonomous systems in particular should also apply a degree of sensor fusion between multiple sources of data such as optical computer vision, radar, long range ultrasound and LIDAR (once it becomes cost effective). If any of those systems provides erroneous data the remaining ones can negate that and allow for a fail-safe behaviour.
If you want my opinion, as someone who has moved away from automotive R&D a few years ago, Tesla's decision of depending too heavily in computer vision systems without addtional sensor redundancy seems like an architectural defect that has already cost lives. Either they are not integrating other sensor data sources or their voting weight appears underestimated.
I really wish people on HN would stop with the "I disagree, so you must have not read everything" comments. I keep seeing these recently, but they are unconstructive, and they are insulting to other participants in the discussion. You might like to consider the alternative possibilities that you weren't clear in your earlier comments and people are responding to what you actually wrote and not what you thought you wrote, or that you might not be fully informed and someone who disagrees with you might simply know something you don't.
The original claim was that vehicles should never be connected to any network because of unspecified online attacks that could actuate brakes or steering. I explained why this was unfounded.
Actually, what you have repeatedly said is that you aren't aware of any vehicles (except the infamous Jeep case) that don't fully isolate infotainment from safety critical systems, which is not the same thing at all.
You also said that the Jeep case was due to a poor architecture that came from the 90s. Even if this is true, the exploit using it to trigger multiple dangerous behaviours remotely was demonstrated in 2015, so apparently the manufacturer was a bit slow on the uptake of what you think they should have been doing for the last 20 years.
Moreover, at least one well-known auto manufacturer is (in)famous for performing OTA updates that can change fundamental car behaviour, so evidently there are still networked vehicles where safety-critical functions and remote communications can not be fully isolated. It doesn't take a genius to extrapolate from this to the not-so-distant future when auto manufacturers are pushing ever more autonomous functionality combined with OTA updates, either.
Then we have all the cars that now have remote controls that do more than just unlock the vehicle, affecting things like environmental controls, or even summoning a driverless vehicle over a short distance (in theory, at least) with some of the newer developments.
Next we have the security systems providing remote access, such as OnStar's Stolen Vehicle Assistance system that someone else already mentioned, which can in some cases interfere with speed or ignition systems remotely.
I think by this point it's safe to say that whatever explanation you think you gave, the evidence doesn't support a conclusion that modern network-connected vehicles are safe because their critical safety-related systems are fully isolated from external influence, which is what really matters here.
Adversarial patterns against computer vision based ADAS are only a real issue for system which are not sufficiently redundant.
And yet not so long ago I read this, which if you're interested in the field you've surely seen as well:
So again, evidently there are systems out there in production that are "not sufficiently redundant". You might have some personal opinions on what should be happening, but that doesn't mean what actually is happening respects your view.
Tesla’s has over the air updates for it’s autopilot software, you even edit destinations from the infotainment system. So, it’s clearly both on the same network and could be hacked remotely.
The car can even be turned on and summoned remotely. Which means if nothing else it could be remotely driven onto a freeway to cause accidents even if it’s ‘autopilot’ had some kind of independent backup safely system to avoid collisions.
> It really should be illegal to have network connected cars.
The important systems should not be permitted to be connected to the Internet. If an Internet-enabled entertainment system is fully isolated from the systems that really matter, there's no problem.
Most older infotainment systems have a physical power switch which is a hard interrupt to power-off. It is in my view a design mistake to have full touch-screen interfaces on infotainment.
It could just be a soft commanding voice that says, "this is not an emergency. Please pull over to one side of the road and get out of the car and wait". Just that will cause enough chaos and accidents.
True, but realistically the line has to be drawn somewhere. A software malfunction in a microwave oven, or some other device with a heating element, could cause a house fire, but that software isn't regulated the way avionics software systems are.
In terms of overall fatalities, distracted drivers on their phones can be expected to continue to kill far more than vehicle hackers. It's troubling though that hacking vehicles' brakes could allow for targeted attacks with little chance of being caught. The same applies to insecure medical solutions.
You apply different standards of rigor based on the failure modes. If the avionics software catastrophically fails in a commercial airliner hundreds of people could die. If there is a critical software malfunction in a single gas stove that could result in it filling a house in the middle of a night with gas and there are no other safety precautions then maybe four people could die. This is already a 1-2 order of magnitude difference in risk and easy to mitigate by having a hardware safety interlock which reduces dependence on software reliability.
The real problem in these cases is internet-connectivity + software-control. Without internet-connectivity, your primary failure mode is random chance. The probability of failure of each system is largely independent from the others as there are few ways to induce simultaneous failure in all such systems. This means that the worst-case failure mode of any individual error is the worst-case of single-system failure which, for most systems, is relatively limited. However, once you add in internet-connectivity, you now introduce an easy way to induce simultaneous failure. Since these systems will all be running the same software, an error in one is an error in all, and you can now exploit the error in all simultaneously. This instantly adds a gigantic single point of failure whose failure mode is critical failure in all systems at the same time.
Suddenly, the software failure mode of an internet-connected gas stove with remote turn-on [1] is no longer one house blowing up in the middle of the night killing possibly 4 people, it is every one of the houses buying the 3.5 million gas stoves sold this year [1] blowing up killing possibly 14 million people. The introduction of internet-connectivity raises the worst case failure mode from a tragedy to approximately a nuclear first strike and surpassing Hitler as a mass murderer. They should not be releasing internet-connected gas stoves unless every engineer on their team is comfortable that their security would be good enough to connect the US nuclear arsenal to the internet since the worst-case consequences are equally bad.
> If there is a critical software malfunction in a single gas stove that could result in it filling a house in the middle of a night with gas and there are no other safety precautions then maybe four people could die.
You've made an assumption about the failure pattern. Perhaps thousands of stoves could fail at once, all encountering the same defect. Something like this could happen if the software fails after 10,000 hours of continuous operation, say. (A bug of this sort once resulted in the deaths of 28 US soldiers, when a missile defence system failed. [0])
> The real problem in these cases is internet-connectivity + software-control
I agree this is a very obvious and severe problem, and it's rather shocking that it's even legal to do this. Avionics software is famously strictly regulated, but the software in road vehicles isn't.
> Without internet-connectivity, your primary failure mode is random chance
It depends. There could be a failure after some number of hours of continuous operation. Perhaps a particular stretch of road could be dangerously mishandled by all self-driving cars of a certain model. There's also a risk that deliberately crafted fake road signs could cause the system to drive dangerously, opening the door to targeted attacks.
> once you add in internet-connectivity, you now introduce an easy way to induce simultaneous failure
That's a good point. It's similar to how we place enormous trust in Windows Update, except we're talking about a life-and-death system.
> They should not be releasing internet-connected gas stoves unless every engineer on their team is comfortable that their security would be good enough to connect the US nuclear arsenal to the internet since the worst-case consequences are equally bad.
Well, no, they plainly aren't, but I do agree with your general point.
I wonder if we'll see any kind of general software regulation (that is, not specific to a domain such as aviation) to cover this sort of recklessness. Currently, the company behind such a product would only suffer after the fact, were something to go badly wrong.
Yes. I endorse your corrections. I was eliding systemic errors in the interest of brevity.
The main point I was trying to make is that mass failures that occur with a degree of simultaneity where the failure can not be corrected between failures are relatively rare outside of internet-connected software due to an access problem. Even the cases you mentioned are generally less dangerous because you can stop usage when an error is detected. If you discover that after 10,000 hours of operation the brakes on all of your cars stop working while driving, it is unlikely that everybody will encounter this problem at the same time. A few will encounter the problem first which will let you detect the error and then work to rectify it or recall your cars before it becomes a truly catastrophic error. It is a truly rare problem where a mass produced product contains a catastrophic defect that will affect a large percentage of them simultaneously. The only ones I can really think of are drugs with long-term unexpected side effects or a response to an equally massive cause like the Carrington Event coronal mass ejection which blew out the electrical grid in 1859.
> The only ones I can really think of are drugs with long-term unexpected side effects or a response to an equally massive cause like the Carrington Event coronal mass ejection which blew out the electrical grid in 1859.
I'd count the mass ejection as a natural disaster rather than the failure of a system, but the harm is just the same of course.
Related to your medicine example: bad dietary advice could be similar to this. We're straying some ways from technological systems here, of course.
>A software malfunction in a microwave oven, or some other device with a heating element, could cause a house fire...
Now I'm curious. Does anyone know the manufacturer and model number for a microwave for which this is true? Have we really replaced hardware thermal switches, fuses, etc. with firmware?
Perhaps microwaves have hardware protections to prevent heating when the door is open, but a software error could presumably still have it spring to life unexpectedly, which could potentially be dangerous.
It's a (perversely) interesting creative exercise to think of catastrophic consequences that might follow from the failure of a system we don't normally think of as critical. Let's restrict it to physical systems, excluding concerns like being banned from Gmail, and issues like misinformation and extremism spreading through social media.
• A bug in a sound driver could cause hearing damage, especially if someone is using earphones
• Perhaps a bug in a graphics driver could be dangerous to people with epilepsy
• A bug in a laptop's battery-management system could cause a fire on an airliner
• A bug in an overclocking tool, or in motherboard firmware, wouldn't typically cause a fire, but perhaps
• We know that bugs in satnav systems can cause inattentive drivers to cause serious accidents
• A bug in an app about food intolerances could have severe consequences. A graphics/rendering bug, perhaps in the OS, could cause the app to misrender.
Perhaps microwaves have hardware protections to prevent heating when the door is open, but a software error could presumably still have it spring to life unexpectedly, which could potentially be dangerous
AFAIK, the magnetron power supply is physicaly connected through multiple door switches, so this is an actual unoverridable and redundant disconnect. One switch shorts out the supply when the door is opened, so that if the others fail, it will still blow the fuse and cut power. I'm pretty sure that design is due to regulation, so I don't see it ever being replaced by software.
Nobody thinks about the fact that they are "fly by wire", using software to control everything. Who thinks about changing the battery controller to cause a battery fire? Or controlling the acceleration to maximum? Or setting off the airbags? Or...
Automotive engineers think about those things. When we drive our cars we're trusting them to have thought about those things, just as we're trusting that the engineer who designed the wheel nuts did his work well enough to stop the wheels coming off, or the engineer who designed the steering linkage considered how we might need to go round corners reliably.
Not always. Hacking a vehicle's brakes is no longer the stuff of sci-fi, it's already happened in a poorly-designed Internet-enabled jeep, see [0]. (As an aside, I have no idea why they thought it was acceptable to do their proof-of-concept demonstration while driving at speed on public roads.)
That story made me angry. There’s no way they could have known precisely what would happen when they executed that hack; software has bugs (the whole point of these exercises) and complex interactions between systems cause all sorts of weird behavior.
setting aside unforeseen interactions between complex systems, the article seems to say that they deliberately disabled the brakes on a public road, while the jeep was moving! that goes well past a fun prank for a story.
irresponsible on the journalist's part too to consent to this. the point could have been made just as well on a private track.
Ask literally any ICS security professional their main concern, it's "[organisation(s)] think a system is airgapped when it isn't".
Whether a system is software-defined/hardware-defined, networked/standalone - these are not technically hard questions to answer, correctly. (No more so than, say, whether polymer O-rings will completely fail to do their job when cold.)
That they're not answered correctly indicates a big communication issue, and whilst I'm sure management has much to answer for, I'm not sure engineers escape the blame for that either.
If that were a reliable safeguard, we wouldn't already have several examples about the behaviour of connected vehicles being compromised remotely in one way or another. Since unfortunately we do have such examples, evidently whatever is going on within the engineering groups at the auto manufacturers is not sufficient to ensure our safety and security in relation to these new generations of connected vehicles.
Looking at CANBUS, I guarantee that they do not. Any device trusts any other device, access to the bus is the only security control. Connect your entertainment system - probably an unpatched Android tablet - so it can display any info from the rest of the car, instant compromise of everything.
That's the reason why there are standards such as the ISO2626, so when a vehicle malfunctions and causes mayhem the vendor isn't at fault but the standard (assuming that the vendor did meet the standard). And then the blame can be pointed at the standard (i.e. "we did everything by the book not our fault") then nobody is liable.
Yes exactly, people instinctively deny to themselves inconvenient things if they can find even a flimsy out. So sometimes you need a spectacular demonstration to pierce the psychological defenses.
> What could go wrong in a 2 ton computer on wheels
I don't see it this way. Right now, there are so many people in cars that should not drive. Alcohol, drugs, lacking skills - to me, computers add to safety.
There will always be a risk. I trust computers more in this case.
Easily hacked computers don't add safety anywhere. They only add random failure and the possibility of large scale disasters.
If you want to push the safety related tasks into the computer, you have the obligation to make them hard to hack. And on the case of cars, that includes not plugging them into the internet.
They only add random failure and the possibility of large scale disasters.
And the latter is the really big danger with autonomous vehicles. The same consistency that means they aren't vulnerable to as much random variation as a human driver could also become a weakness. Yes, a drunk or some idiot on their phone or an unwitting patsy whose drink was spiked at the bar can cause a fatal crash. But they can't simultaneously cause 500,000 fatal crashes by doing the exact same stupid thing 500,000 times at once or by being the victim of the exact same crime 500,000 times at once.
The point is that automation causes us to cede control to the machines which can be hacked, or worse abused. It is a transfer of power to whoever controls the machines. This centralising power can be abused in ways people can only imagine. Of course, people will give up what’s left of their control and power in the name of “safety and security”, they’ve been doing that since 9/11. So much so that we have the term post-9/11 to remind them to toe the line (and to kneel and take off their shoes).
Imagine for instance there is a bush fire and your centrally controlled car decides that the fire is fake news and prevents you from leaving since it’ll cause congestion on the roads. No different from Twitter banning the NY post story. Maybe they decide NY Post headquarters is not a real place and refuse to navigate the car there.
Yeah we should implement some kind of social credit system so that people who do drugs can’t drive or take public transport. They’ve got that in the modern utopia of China right now.
> I wonder why they needed to actually destroy a relatively expensive piece of equipment, plus spend time & money on all the orchestration around this test.
Because uneducated people (like Congressmen and executives) tend to believe stuff more when they can see it with their own eyes.
It's like, in terms of information security, putting up sensitive information on walls that can be seen from the outside. Almost no one sees a problem with that because they believe it takes expensive equipment... until you go, take a 600mm+ tele lens and prove to them that their sensitive information can be stolen for under 200$ worth of camera rental fees.
Or, remember the immigration debate in 2015? The picture of three year old Alan Kurdi on the beach and its consequences in politics? Never underestimate the power of images and videos.
If you're threat modeling, you need to know what you're dealing with. The equipment obviously wasn't designed with the goal of being able to destroy itself (even though it's possible). If equipment starts failing, identifying the source could be critical in limiting damage, and the best way to be able to recognize this kind of attack is do it and record the result.
We can theoretically understand concepts in the abstract, but sometimes more is needed for us to take them fully on board.
The article mentions the people responsible not grasping the full scale of the problem. Giving them the visceral image of the potential destruction might help them take on board the risk emotionally as well as intellectually.
1) This is clearly a stunt meant to impress people. Information security is black magic and deals with unknown risk. People hate to think about unknown risks.
2) It's very difficult to justify increasing security budget every time some infosec people says so - especially if their salary depends on such budget.
Because there is nothing like a real-world test to verify if your ideas and predictions are correct and to reveal other things you may have not considered.
This was obviously a grant money grab. Anyone with a basic working knowledge of electrical machinery and mains power will understand this and would apply to any engineer working in power generation. It's a huge no duh.
it’s a technicality, but if you do this to all the machines at precise same time it would not work. the attack relies on the grid being there and leveraging the power it generates to destroy one piece of equipment.
that said, you could take out a significant number by performing the attack in a serial fashion
You can definitely do this to all of them at the same time. And only one would survive. Why? Because out of sync does not mean the out of sync frequency is the same to all of them. SO you have machine1 at 70 HZ, machine 2 at 30 HZ and so on. Connect all them together out of sync in the grid and you get them all fight each other. Do this maliciously 10 times and in under a minute all but one are totaled, like the one in the video from article.
Inductance in the grid would probably be sufficient to destroy them all, even if they were the only equipment on the grid and they were all destroyed at once. It's a lot of power flowing.
Given that the failure mode seems to rely on force applied to a specific generator from other generators this particular attack could only take down some percentage of all running generators at any given time correct?
Sure, but the OC was clearly alluding to the fact that the real risk is a true mass attack. If you could do anything to every generator, what is the worst thing you could do? If you use this technique you could probably blow out some percentage of generators. Would that take down the grid? For how long? How hard would it be to recover? We already know power grids can be taken down by cyberattacks, just look at the cyberattack Russia did against Ukraine's power grid [1]. The main difference is that they were probably not trying to permanently take down their grid. If you could do something that would destroy enough capacity to keep the grid down for months the casualties would be immense. There would be immediate consequences like the loss of heating and cooling resulting in exposure deaths, but these pale in comparison to losses of fresh water, fuel production and thus food transport, fertilizer production and thus food production, manufacturing and thus goods, cargo port throughput and thus shipping. Modern society in the developed world utterly depends on our power grid.
I think it's not unlikely that a very high percentage of these protective relay is from the same manufacturer running the same "operating system" with the same vulnerability.
With that it doesn't matter if it's the same hardware model and also not which generator sits behind it. From the article it sounds like that force would have killed any generator.
So they messed with the synchroniser and the synch check relays, on a reciprocating deisel.
I design such systems and it sort of reads ok, but a few key details not right or not explained - those size of generators grid connected will have reverse power, pole slip, instantaneous over current electrical protection as well, AVR excitation limiting, plus overspeed relays which would not have let this happen. (gas turbines traditionally have a hydraulic control oil relief overspeed "bolt" that comes out with centrifugal force and no software could mess with, these days electronic replacements are coming in vogue, but...)
As for synch check, rule of thumb is two independant systems, one hard wired, three overall if you can swing it because often one of the multifunction protection relays will have that function available and you wire it and/or code it in line.
But I have seen big reciprocating diesels go into synch at 30-60 degrees out of phase and if they hold without tripping the generator protection then you might get a brief flicker of the lights, some stress on the crankshaft that who knows what it does long term, but some of them were old and had it done to them many times.
Yeah, so they did it, waste of a good machine for known outcome if ask me, very wasteful with no need.
And it is a very particular kind of attack, if you knew it was a problem it could be protected against for hundreds or thousands of generators literally in 24 hours or less - hardwired synch check which can be achieved with a voltage relay at it's simplest.
So I am not seeing anything in that excerpt other than highighting bad design practices.
> Yeah, so they did it, waste of a good machine for known outcome if ask me, very wasteful with no need.
And yet there are plenty of people who don't understand that software problems can cause permanent damage to hardware or people. Or they think of it in the abstract. I don't think it's wasteful to demonstrate.
It was definitely costly in this case. I think the author could have done with a smaller demonstration. But there is something to be said for grandeur.
> So I am not seeing anything in that excerpt other than highighting bad design practices.
Indeed. I think that was exactly the point of the demonstration.
There's 2 reasons why security researchers go beyond a theoretical demonstration into a destructive one:
1) Security researchers have an intense need to show their exploits actually work in case they're accused of not providing proof.
If you go to Defcon, most of the presenters spend half their talk on showing use of the exploit, no matter how tedious it is to the audience.
(When I supervise IT security audits, I have to tell pentesters just to document what they find instead of making changes, which introduces more problems and wastes time and effort. I've had auditors find a configuration file with passwords on a production server that uses them, then try to present a report on how they logged into each one. Duh, of course they work.)
2) This is related to #1, but business people, especially across silos in an organization, may not fully comprehend an exploit unless there is obvious proof. I've seen laughable levels of denial by otherwise intelligent people once they get into management. As in, "Yeah you showed it to me on the screen, but I'll need to confirm it with other sources" who then look at the same screen.
Pro tip 1: by default AWS AMI's are not encrypted. (The reason is that the main list of AMIs doesn't use your KMS keys.) So if you have a compliance program that requires encryption at rest, get cracking!
Pro tip 2: All theoretical exploits have shown to be practical ones later. Stop deceiving yourself.
Leaked extracts in yesterday's Washington Post describe how the operation caused "the most monumental non-nuclear explosion and fire ever seen from space" in the summer of 1982.
"In order to disrupt the Soviet gas supply, its hard currency earnings from the West, and the internal Russian economy, the pipeline software that was to run the pumps, turbines and valves was programmed to go haywire after a decent interval, to reset pump speeds and valve settings to produce pressures far beyond those acceptable to pipeline joints and welds," Mr Reed writes
I dunno. It's good to know that it's contested, but all the intelligence organs (from both sides) denying a hacking took place is how things always go.
They denied that the Stuxnet was a cyberweapon too, for years after all the details were public known.
That said, the argument that there wouldn't be a computer controlled pipeline in 1982 is kinda strong. If I were interested enough on the history, I'd start looking from there.
I picked up his book from the library and it's been a good read. Makes me frustrated at just how little the higher ups in the U.S. Government seem to care about cyber-security. They'll learn the hard way I guess
This is the difference between "Bob the engineer says the bridge will fall down if you cut this cable", Vs "Bob the engineer cuts cable to demonstrate how easy it is to knock bridge down".
We should trust our engineers, and not require them to demonstrate something at great cost to get the attention of people who make decisions...
Which engineers to trust? The ones saying Y2K needed billions of investment or cars all crash with planes falling from the sky as we usher in the post-technology age Jan 1, 2000. Remember that? Some people still believe them that we headed it off successfully, everywhere on earth in every industry, regardless of how little was spent, including nothing at all...
You can't trust engineers anymore than you can trust lawyers (which lawyers?) or Doctors (Second opinions ever much?) Or indeed scientists, managers and politicians. The question to ask is "Do you have good reason to find this convincing."
When what is at stake has sufficient value proving the obvious is indeed, also correct, beyond doubt has very real benefits.
This is the same kind of thinking that has people questioning quarantine protocols (in general, not just this one). A precautionary measure that prevents a worst case scenario doesn't suddenly become stupid because the scenario it was designed to prevent does not happen. We hedge our bets all the time, and strategy people who don't hedge will end up driving busloads of people off a cliff sooner or later. Reckless gambling has no place in policy decisions.
We also had some pretty fair warning from industries that use long-term thinking. Banking software was already breaking in 1969 (30 year mortgage logic). For many industries this was a long emergency.
Even by a cynical view that these were pork projects that didn't 'need to get done', so what? We do these all the time. The cash infusions prop up industries or divisions, and often serve as cover for attacking deferred maintenance. Over beers you can complain about how stupid it is to have to get things done this way, but at work you take the money and run as far as you possibly can.
What bothers me most, I think, is that if you've worked in software for very long at all, you should know all of this. Politics and proxies come up if you work at a mature company, or one that matures while you're there.
(On December 31st 1999, I watched the New Year's celebrations for Tokyo and Australia. When they didn't blow up, I agreed to make plans involving alcohol for the evening. By the time Europe didn't blow up, I figured we were gonna be okay, so Happy New Year to me.)
>This is the same kind of thinking that has people questioning quarantine protocols (in general, not just this one).
There are competing protocols from different expert sources. Which to trust?
The ones that make a convincing case. See how that works?
Who to believe about Y2K? The engineers who make a convincing case.
(Y2K wasn't nothing, but a huge part of the investment in mitigation was a con. Cables certified where I worked. Every windows95 desktop. Do you believe those engineers who said that was necessary? What about the ones who said, "Yeah probably should check your billing system and interest rate calculations. Airplanes, not so much..." But they usually weren't selling using the fear-frenzy for leverage. Do you remember all that?
As someone who worked through Y2K, over 90% of the money spent accross society was straight-up con. It wasn't nothing, but what a massive sales beat-up it became.
Disagree, go nuts. Who do you trust? The engineer making the convincing case. "Trust engineers" not so much.
It's like the story of the factory hiring the experienced engineer and getting a massive bill for tapping a machine with a hammer. Paraphrasing from memory:
The factory had a machine break down. They hired a few cheap engineers who were unable to diagnose and fix the problem. Eventually they found an experienced engineer who fixed the machine by tapping it with a hammer and charged them $1,000. When they asked for an itemised invoice they got:
Tapping with a hammer: $5
Knowing where to tap: $9,995
It's also super expensive to develop current generation offensive and defensive military hardware. It's super expensive to play wargames.
The end result is know about an issue someone else could exploit if they have the time and will to do so. The US is going against state sponsored actors who will spend the time and energy to figure out ways they could cripple our country. And we do the exact same thing to them.
Security is done in layers. If you're a nation state, you need to be prepared to defend against other nation state actors. And that costs a lot of money.
There is nothing interesting here. So called "Hackers" simply told the generators relay to close out of phase causing the generator and grid currents to violently crash into each other. This means the magnetic fields of the rotor and armature were not aligned. Obviously the grid has much larger generators backing it so the little diesel gen set rotor was no match for the now grid energized armature. The armature field tries to violently pull the rotor into phase resulting in catastrophic physical forces on the generator mechanics. After a few of these devastating blows the diesel engine's crank shaft, connecting rod or perhaps the flywheel coupler violently fails.
The only code needed would be to completely ignore line and gen set phasing or flip them 180 to really let the bolts fly.
Any generator will succumb to this no matter the prime mover. They must be in sync with the grid or other operating generators before connected across the line/bus. This is power house practice 101 level stuff.
Indirectly through serial concentrators and RTUs and Scada access, almost 100%.
The FERC / NERC security standards that fell out of this event are largely a bad joke, but had the benefit of at least requiring North American utilities to do something at all.
Any transmission operator of size and most generation operations effectively require real time Scada for anything of meaningful capacity and they're sure not paying someone to sit next to it reading numbers off over a phone. They may not be IP addressed individually but to put it in perspective, I had a communication engineer tell me with a straight face that the fiber mux I telneted into over their local wifi wasn't internet connected, and thus the protection relays on the 500 kv substation the next building over were serial only -- over said fiber cables.
As an architectural building block we have been stuck with Firewalls, IDS etc. We need a new kind of isolation device –
an air-gapped one-way data transmission gate – that can sit between critical safety control systems and their monitoring systems. These read-only monitoring systems can be connected to networks (presumably reachable to hackers through some means). As long as the safety control systems cannot be modified by any means, these types of failures should not happen.
Also, irrespective of software safety, for a system like this grid power generator why can't they design physical safety relays/cutoffs? If the generator sped up beyond it's tolerances, it could cut-off. Why should it generate shocky torque that can damage the rubber grommets? That could be prevented by design?
"We need a new kind of isolation device – an air-gapped one-way data transmission gate – that can sit between critical safety control systems and their monitoring systems."
Those are called "data diodes". They involve optical isolation. True one-way is very limiting, though.
One-directional data transfer isn't really a problem: just have a wire that only sends data in one direction. If you want to be extra paranoid, make it an optical link. Or wireless transmit-only if that fits your application. Downside is you need more bandwidth since you can't have the receiver ask for a specific subset of data.
The trouble always starts when someone wants remote inputs for some reason. Then you really need to start to think about a) how to protect the gateway systems and b) how to make sure that even if the gateway is cracked the damage is limited.
> If the generator sped up beyond it's tolerances, it could cut-off. Why should it generate shocky torque that can damage the rubber grommets?
This was not the generator speeding up beyond its tolerances. From what I understood from this article, it sped up slightly (still safely within its tolerances), but the difference in frequency made it out-of-phase with the power grid. Once it was connected, the misaligned electromagnetic field instantly tried to align the shaft (and match its rotation speed); that's where the sudden torque came from.
The generator wasn't sped up beyond tolerances, the safety system that kept it synchronized with the wider grid deliberately brought it in out sync, like ramming a car engine into a much lower gear at high speed. The force has to go somewhere, and from the perspective of the generator the mass of the grid might as well be infinite.
As for better architectures, we're working on that, but industry is lethargic, lazy, and often organizationally dumb.
But there are limits to isolation. AC grid frequency and voltage and load flows are changing constantly, if normally by small degrees, and propagate at effectively the speed of light. It's impossible to have purely static immutable protection schemes for all but the most trivial of equipment.
> why can't they design physical safety relays/cutoffs?
They can and they do. Or at least "they" used to. Nowadays such safety devices are very often software enabled. Because it's cheap and also because it checks a box on the marketing brochure that every customer wants: "Software upgradability for lower maintenance cost and reduced down time." Oh the irony.
I would imagine the chance is higher than you think.
Being able to remotely monitor and manage such systems may lead to big cost-savings, if it enables the same number of employees to monitor multiple locations.
Seeing as how many tech companies routinely cut corners on security, one could easily imagine a non-tech company doing the same in an attempt to maximize profits.
It's not just remote monitoring. Protective relaying involves some disconnects sending signals to distant disconnects to tell them not to disconnect. The idea is that when you have a sudden overload like a line down, you want to trip the breaker closest to the trouble, and not others elsewhere upstream. Without this, small disruptions take out too much of the grid. See NYC blackout of 1965.[1]
This is why large transmission networks require some capability for fast transmission of data between circuit breakers. Not much data. But it's so much cheaper to send it over the Internet than to have separate microwave repeaters or separate cables or gear to send data over high-tension power lines.
The incident itself is over a decade old and available under FOI, so it's possibly fair to file it under "things that make the front page from time to time". It's certainly not news.
Reader mode is what you want, and it is a built-in Firefox feature. You should see an icon on the right of your address bar. On GNU/Linux Ctrl+Alt+R activates it.
I got tired of struggling against unreadable fonts.
So now Firefox is set to present me with Lucida Grande for all fonts, viz. serif, sans-serif, and monospace. I don't allow pages to choose their own fonts.
I don't think I'm missing much in general, but I do switch to Safari in special situations.
Causes some little useless icons to display as missing-character boxes. I've never found it to make pages unusable, or even particularly worse: those icons are often pretty inscrutable.
(presumably pages that became unusable would have ADA problems...)
one engineer can do it in about four hours. three and a half hours to research the ideal replacement bulb, twenty five minutes to set up the ladder safely, and five minutes to replace the bulb itself. source: I'm an engineer and I replaced a lightbulb this week.
the real problem arises when you have multiple engineers replacing a lightbulb. the time needed to decide on a replacement bulb and agree that the ladder setup is safe is quadratic in n, where n is the number of engineers. it still only takes five minutes to replace it though :)
Didn’t read article but had some fun imagining what happened
Code:
While true {
Get_current_demand()
If current_demand > current_supply then increase_supply()
}
Post Mortem
Eng mgr: where is the code to check the capacity of the supply?
Eng: we considered it but the PM said it was a p1. Increasing supply to meet demand was the MVP solution
PM: our extensive market research indicated that customers would come nowhere close to capacity constraints. In fact our contracts limit the amount of energy they get. But client X went way beyond that, then the generator blew up
The TL;DR as I understood it is that they paused the synchronization with the (60 hz) grid for just a little bit, then reconnected it; running out of sync with the grid caused a sudden backlash, making the generator tear itself apart from the sudden shock.
Yeah. It's meant not to connect until sufficiently in sync. Running out of sync apparently results in the grid nudging the generator into sync, but here the discrepancy was so large the nudge became a percussive encouragement of significant enough magnitude for the generator to permanently take a seat and reconsider. (that sentence was fun to write)
The classic protective mechanism for this is big fuses.
Classic manual synchronization of a generator involved three lamps, showing the voltage difference between the phases of the generator and the grid. The operator has to adjust the speed of the generator until all lights stop flickering and go out. Here's that being done with a desktop training setup.[2]
Because this is a touchy operation to do manually, it's usually automated.
However, there's no reason you couldn't have a hard wired interlock so that the breaker won't close until all three difference voltages are low enough.
But it's so tempting to cut costs and put all that functionality into one convenient package. Like this one.[3]:
The SEL-700G is the right solution for utility and industrial generator protection, with an autosynchronizer, flexible I/O, and advanced communications. It provides a complete protection and synchronization solution for synchronous generators, so you can eliminate the complexity and cost of standalone synchronizer packages. Integrating the synchronization capability into the generator protection relay provides the most cost-effective and reliable solution.
The 5-inch, 800 x 480 color touchscreen display option allows you to directly set, monitor, and control your system, including up to five two- or three-position disconnect switches. A small form factor and multiple mounting adapters make it easy to upgrade protection without cutting or drilling existing cutouts. You can quickly integrate the SEL-700G into serial or Ethernet-based communications with IEC 61850 Edition 2, EtherNet/IP, the IEEE 1588 Precision Time Protocol (PTP), IEC 60870-5-103, the IEC 62439 Parallel Redundancy Protocol (PRP), Mirrored Bits communications, Modbus, DNP3, and other protocols.
It's useful to imagine this way: When a motor is forced to spin by some external influence, drawing more current out of it on the other end means it has to spin with more torque in order to maintain the same voltage. Here, the generators are connected together to supply power in unison. They must all be in phase with each-other otherwise the out of phase generator will find itself fighting with the other generators, and as beautifully demonstrated here will very violently lose the fight. You could describe this as a very cruel demonstration of destructive interference!
Put another way: every electric motor is also a generator and vice-versa. If you induce the motor to absorb energy from the power supply rather than emit it, that energy has to go somewhere. As usual, when not specified, "somewhere" tends to mean heat, sound, and inelastic deformation of anything in relatively close proximity to the excess energy.
They clearly knew they were capable of reprogramming the control logic remotely (otherwise this idea for a test would've gone nowhere) and any engineer could've predicted this obvious outcome without the need for a real-world test.