What Is Threat Modeling

Enter a topic name to show or a new topic name to create; then press Enter

.

Security is a lot about tradeoffs. Rarely can you apply a security countermeasure to a system and not trade off convenience, privacy, or something else that users of that system hold dear to their hearts. Bruce Schneier talks a lot about these tradeoffs in real-world systems such as airports (Schneier 2000). In computer systems, the same tradeoffs apply. Forcing users to run with least privilege (as opposed to administrators) is a huge hurdle that many organizations cannot seem to get past, for example, simply because it's painful for users. Most software breaks when run without administrative privileges (which is stupid and should be fixed, as I discuss in WhatIsANonPrivilegedUser).

It stands to reason that when designing secure systems, you should not simply throw random countermeasures at the design, hoping to achieve security nirvana, but you'd be surprised how often this happens. For example, there's something magical about the acronym RSA. Just because your product uses good cryptographic algorithms (like RSA) doesn't mean it's secure! You need to ask yourself some questions.

• Who are my potential adversaries?
• What is their motivation, and what are their goals?
• How much inside information do they have?
• How much funding do they have?
• How averse are they to risk?

This is the start of a threat model. By sitting down with a small group of bright people who span a product's entire life-cycle (product managers, marketing, sales, developers, testers, writers, executives), you can brainstorm about the security of that product. Once you figure out the bad guys you’re up against (Schneier 2000 has some great guidance here), you can start to think about the specific threats to your system. Now you'll be asking questions like these:

• Is my system secure from a malicious user who sends me malformed input?
• Is my database secure from unauthorized access?
• Will my system tolerate the destruction of a data center in a tactical nuclear strike?

I'm not being facetious here. Someone who asserts an unqualified "My system is secure" either is a fool or is trying to fool you! No one can say a system is "secure" without knowing what the threats are. Is your system secure against a hand grenade? Probably not. You can have security theater or you can have real security, and if you want the latter, you'll need to think about the specific threats that you want to mitigate. As you'll see, you’ll never be able to eliminate all threats. Even if you could, you'd be eliminating all risk, and businesses rarely prosper without a certain margin of risk. Heck, if you disconnect a computer and bury it in 20 feet of freshly poured concrete, there's very little risk that anyone will steal its data, but accessing that data yourself will be a bit challenging. Real security has a lot to do with risk management, and one of the first steps to achieving a good balance between threat mitigation and ease of use is to know the threats!

But how can you possibly analyze all the threats in a nontrivial system? It's not easy, and you'll likely never find them all. Don't give up hope, though. Due diligence here will really pay off. Most threat models start with data flow diagrams that chart the system. Spending the time to build such a model helps you understand your system better, and this is a laudable goal on its own, wouldn't you say? Besides, it's impossible to secure a system that you don't understand. Once you see the data flows, you can start looking for vulnerabilities.

Microsoft has an acronym that they use internally to help them find vulnerabilities in their software, STRIDE (Howard and LeBlanc 2000):

• Spoofing
• Tampering
• Repudiation
• Information disclosure
• Denial of service
• Elevation of privilege

Spoofing is pretending to be someone or something you're not. A client might spoof another user in order to access his personal data. Server-spoofing attacks happen all the time: Have you ever gotten an e-mail that claims to come from eBay, and when you click the link, you end up at a site that looks a lot like eBay but is asking you for personal information that eBay would never request (like your Social Security number or PIN codes)? This attack is now so common that it’s earned a specific name: phishing.

Tampering attacks can be directed against static data files or network packets. Most developers don't think about tampering attacks. When reading an XML configuration file, for example, do you carefully check for valid input? Would your program behave badly if that configuration file contained malformed data? Also, on the network most people seem to think that encryption protects them against tampering attacks. Unless you know that your connection is integrity protected (WhatIsCIA), you’re better off not making this assumption because many encryption techniques allow an attacker to flip bits in the ciphertext, which results in the corresponding bits in the plaintext being flipped, and this goes undetected without integrity protection.

Repudiation is where the attacker denies having performed some act. This is particularly important to consider if you plan on prosecuting an attacker. A common protection against repudiation is a secure log file, with timestamped events. One interesting consideration with these types of logs is the kind of data you store in them. If the log file were to be included in a court subpeona, would it be more damaging to your company to reveal it? Be careful what you put in there!

Information disclosure can occur with static data files as well as network packets. This is the unauthorized viewing of sensitive data. For example, someone running a promiscuous network sniffer such as NETMON.EXE can sniff all the Ethernet frames on a subnet. And don't try to convince yourself that a switch can prevent this!

Denial of service (DOS) is when the attacker can prevent valid users receiving reasonable service from your system. If the attacker can crash your server, that's DOS. If the attacker can flood your server with fake requests so that you can’t service legitimate users, that's DOS.

Elevation of privilege allows an attacker to achieve a higher level of privilege than she should normally have. For example, a buffer overflow in an application running as SYSTEM might allow an attacker to run code of her choosing at a very high level of privilege. Running with least privilege is one way to help avert such attacks (WhatIsThePrincipleOfLeastPrivilege).

Another technique that is useful when rooting out vulnerabilities is something called an attack tree. It's a very simple concept: Pick a goal that an attacker might have—say, "Decrypt a message from machine A to machine B." Then brainstorm to figure out some avenues the attacker might pursue in order to achieve this goal. These avenues become nodes under the original goal and become goals themselves that can be evaluated the same way. I show a simple example in Figure 3.1.

Figure 3.1 Building an attack tree

You can continue the analysis by drilling down into each new goal (Figure 3.2).

Figure 3.2 Further developing an attack tree

The beauty of attack trees is that they help you document your thought process. You can always revisit the tree to ensure that you didn't miss something. Entire branches of an attack tree can sometimes be reused in different contexts.

Once you have a list of vulnerabilities, you need to prioritize them. Remember that, just like in business, good security really comes down to good risk management. The simplest way to prioritize threats is with two factors: damage and likelihood. Rate each vulnerability on a scale of one to ten based on the amount of damage a successful exploit might cause (financial damage, reputation damage, or even physical damage to persons or property). Calculate a second rating on the likelihood of someone being able to pull off the attack. To prioritize, calculate the overall risk factor for each vulnerability: Risk = Damage * Likelihood. Sort your vulnerabilities into a list of decreasing risk, and address the highest risk items first. This is a highly subjective analysis, so you'll be glad you built a well-rounded threat modeling team when it comes time to rank the threats.

Now you need to figure out how you're going to manage the risk for each vulnerability. You've got four choices.

• Accept the risk.
• Transfer the risk.
• Remove the risk.
• Mitigate the risk.

Accepting risk is part of everyday life in business. Some risks are so low and so costly to mitigate that they may be worth accepting. For example, accepting the threat of a nuclear strike that destroys two data centers in two different locations simultaneously might just be the cost of doing business. But each threat model is different: For a military system, this might be worth mitigating.

Transfer of risk can be accomplished many ways. Insurance is one example; warnings are another. Software programs often transfer risk to the users via warnings. For example, try enabling Basic Authentication in IIS and you'll be warned that passwords will be sent in the clear unless you also enable SSL.

Remove the risk. Sometimes after analyzing the risk associated with a feature, you'll find that it's simply not worth it and the feature should be removed from the product. Remember that complexity is the number-one enemy of security. In many cases this simple approach is the best.

Mitigating a risk involves keeping the feature but reducing the risk with countermeasures (WhatIsACountermeasure). This is where designers and developers really need to be creative. Don't be surprised if this means reshaping the requirements, and perhaps the user's expectations, to allow the feature to be secured.

Threat Modeling (Swiderski and Snyder 2004) is an entire book dedicated to threat modeling. The second edition of Writing Secure Code (Howard and LeBlanc 2002) also has a chapter dedicated to this topic, and Secrets and Lies (Schneier 2003) is an invaluable resource as well. To learn more about social engineering, check out The Art of Deception (Mitnick 2002). And the next time someone claims that her feature is secure, ask to see her threat model!

PortedBy KeithBrown