The Four Question Framework for Threat Modeling

Introduction: Doing More with Less

[00:00] Adam Shostack: none of us have extra budget these days none of us are getting extra headcount we're all trying to do more with less we're all thinking about how can we leverage tools to be more efficient more effective and and I'm just going to be frank threat modeling has something of a bad rap.

The Misconceptions of Threat Modeling

[00:21] Adam Shostack: It has a wrap as being heavy, it has a wrap of being slow and people sometimes shy away from it and I think that's a shame because one of the key principles of lean is do it right the first time, don't keep making the same mistakes and one of the things that happens in appsac is we develop our software, we run it through a pen test, oh my gosh there's big problems, who' have thought, and then we get into these ugly escalations and I think threat modeling—no no I don't think I know—I see every day threat modeling changes that dynamic by helping you think about security upfront.

Understanding Threat Modeling

[01:14] Adam Shostack: and so let's let's start out with what is threat modeling. It's it's actually super simple, it's using models to help us think about security. It's getting rid of some details so we can see the whole of what we're working on at a given time and think about what can go wrong with it. And when I say threats in the threats in threat modeling, excuse me, means possible future problems. You know nice story you got here, it would be a shame if something were to happen to it. And this you can do this if you're producing software from scratch, you can do this if you're assembling enterprise systems or operational systems out of a mix of open-source commercial projects and some business logic that you're writing. You can threat model each of these and the way I like to think about it, the way I like to sell it to be honest is threat modeling is the measure twice cut once of security.

If you're in a machine shop or a wood shop and you cut a piece of wood once and you cut it too short it goes in the scrap pile in the corner and that's visible. We all see, we learn very quickly, oh I destroyed that. In software, in building technology that that pile of wood in the corner is wasted development time, it's missed schedules and it's still there but it's less visible. The other thing I like to say as I'm convincing people that they ought to give threat modeling a shot is this quote from Frank Lloyd Wright. He said, "You can fix it on the drawing board with an eraser or on the job site with a sledgehammer." The same thing applies to the technology we're building.

The Four Question Framework

[03:10] Adam Shostack: so how do we threat model? We like to threat model using the four question framework. These four incredibly simple questions are: what are we working on, what can go wrong, what are we going to do about it, did we do a good job? And talk about ways to answer these in just a second, but the thing I want to start from is that nobody has ever said to me, "Adam, that's an irrelevant question. i do X in technology and I don't think I need to answer one of those or one or more of those." And that gives it some power.

These questions which I created when I was working on my first threat modeling book or in the threat modeling manifesto which is a consensus sort of thing about how to threat model and my company showstack associates has a white paper titled, "Understanding the four question framework" because the questions are actually sort of surprisingly nuanced. Um and so these, as I said, simple questions and they lead us to these very scenario appropriate ways of answering the questions and I'll talk about those and it allows us to align threat modeling with the engineering process.

Each of you works at a unique place that has found a way to deliver value, the way we do agile here is—and if I were to come in as a consultant as a trainer and say, "Jen I want you to rip up your playbook for shipping value and replace it with mine." What would you say?

[04:59] Jenn Gile: She'd say, "I got to turn my mic on." Say, "No."

[05:02] Adam Shostack: Exactly. You'd say, "No." And so if we ask these questions we can think about how do we build this into the way we deliver value. The other thing they do for us is they give us a common language. When I'm talking to the CEO and I ask, "What can go wrong?" I get an answer. It's different than the answer engineer might give, but if we start asking the question of everyone we get this continuum of answers, we get this set of answers and we should be doing things about those potential problems and so it's really powerful.

Practical Threat Modeling Techniques

[05:42] Adam Shostack: The one thing I want to mention is as I've taught more and more people about threat modeling I've learned that this concept of boundaries is one—you know and and a boundary just to define it before I talk about it—the user kernel boundary, the firewall imposes a boundary, it's that simple. We're isolating things from each other and finding your boundaries, enhancing your boundaries, possibly adding boundaries is something you can do as you're threat modeling and it's high value.

But let me talk about how to answer the questions. We can answer the questions sort of a classic way of threat modeling that many people learned in an apprenticeship model is they go to a whiteboard and start talking about, "Hey what are you working on here?" Um we can use Lego models, there's some people who have done work on that, it's a lot of fun. Um we can use data flow diagrams which are a classic because data flow diagrams are easy to draw what we draw on the whiteboard, they're simple ble to learn and importantly threats tend to follow data. This message contains an attack or the response to an attack and so data flow diagrams are really useful for helping us understand what are we working on in a way that helps us find the threats to ask what can go wrong.

And we can just brainstorm, we can talk about that in a room. Um and the other day I was chatting with Dave Leblanc and if you don't know Dave and his work, he's one of the authors of the Writing Secure Code books, he's been in security. I first met Dave not even going to admit exactly how long ago it was but the year started with a one not a two. Uh and so Dave was saying, "You know I remember this session where we had no process, no formality, we just got in a room and talked and as we talked about it it became clear that different people had very different perspectives on how the system was put together and where security was happening and surprise, everyone thought security was someone else's job." And that came out of some brainstorming, it came out of Dave's skill as a moderator, his experience as a smart security guy.

And as we scale up threat modeling—so I've I've been threat modeling for a long time and when I joined Microsoft in 2006 and realized I had to scale, I realized brainstorming wasn't the only answer we needed. And Microsoft had a system called STRIDE which is a pneumonic for spoofing, tampering, remediation, information disclosure, denial of service and expansion of authority—I say it once in a while. Um and STRIDE helps you find threats and some people like to say that STRIDE is old and tired, I like to say it's a proven classic and I still use it as one of my go-tos.

We can also use killchains proactively to think about the question of what can go wrong with a system and these structured ways of answering help us take threat modeling from that conversational approach to an engineering approach which we can replicate. And the challenge that we run into is as we add that structure to give people predictability, definitions of done, um that can lead to that heavy weight and so I focus in on the questions as a way to keep us on track in thinking about the things that matter as we're trying to threat model.

Let me give you an example of what the output might look like. So here we've got a simple data flow diagram  um you can see my mouse, okay. So over here we've got a browser, it connects to a user interface and in this case I've got an internet enabled camera right, and so there's a database system over here, there's an updater to get more information about or to get the camera new software because it's buggy and maybe I've got an app on my phone that talks to the camera. Um and I've divided up, I've got a little trust boundary here around the camera, I've got another one inside because there's the camera app that's running in the kernel. And I might do a more uh and I can start by the way with just this and say, "How does the how does the user interface know who's operating the browser? How does the cloud system know which camera that user is authorized to go and talk to?" Those are problems of spoofing or expansion of authority if I can get to other people's cameras.

And so we can start asking questions with these super simple diagrams. We can get a little bit more detailed about what's happening there right, so maybe we'll put a user interface in one process and business logic in a second, an updater in a third and we divy that up with some more boundaries. We can choose to do that, we can choose to convey that in a diagram especially as our companies are growing using these diagrams to say this is your lane, this is the piece that you're working on and you need to stay coordinated can be helpful.

So we can use diagrams at different levels of abstraction and we can think about the question of what can go wrong and I I uh jumped ahead a little bit but also access to the database might allow information to go between customers. The version one diagram didn't have compartmentalization, the blast radius if you like that terminology was bigger and that leads to lower reliability because more errors can cause more um can cause problems in more areas. One of the things that we get from better isolation, better trust boundaries, things running as different users is that the code is simpler, the interfaces are more defined and the bugs we get are simpler allowing us in a lean way to go faster by going by taking a minute at the start to be thoughtful about what we're doing so we have a clean organized workspace and we can ship quickly and confidently and that's so valuable in today's world.

The Importance of Consistency

[12:51] Adam Shostack: And structure, I talked about structure a little bit, structures lead to consistency and so if I am brainstorming, if that's the way I'm threat modeling, I go into one room I get one set of answers, I go into another room I get a different set of answers. And that happens because people in those rooms have different experiences, different skills, maybe they saw different things in the newspaper that morning or as they were doom scrolling and so you get different answers. And so we build these mechanisms, these structures, these ways of doing our work like STRIDE, like killchains, like DFDs that allow me to go in and look at a threat model and say, "Oh this is what they did." And I don't have to learn the diagramming type, I don't have to learn um a new programming language. I can say, "Yep I understand what's happening here, let's talk about the important questions," and that lets me collaborate broadly across lots of different teams with different personalities, different immediate goals.

And one of the places where threat modeling falls down is we move away from those structures because a lot of security people—and I know there's a lot of security people on this call—say, "I want to do it this way." Awesome, you you can do it your way, I'm not saying don't, but recognize that every time you go in and you do something different there is a chance that something will go wrong, you're going to confuse people and so one of the things we do about it is we develop more consistency in how we're threat modeling.

So I like to think about threat modeling as a big tent. It's very much like developing software, there's lots of tasks that we do. You know when we're developing software we um write code to implement functionality but that comes after the task of writing a user story. If we're doing testdriven development I've written some tests before I start writing code, excuse me, there's a lot of tools that we use. You know you might be a Visual Studio user, you might like Jet Brains, um you might like GitHub, you might like GitLab, you might even be old school and using SVN or some other source control system. The way we think about our deliverables, the way we think about our skills so variable and threat modeling can be the same. It can be broad, flexible, more than one way to do each task, more than one tool, people form different deliverables, etc.

And if you think about threat modeling as a monolith, um you must do it Adam's way or you must do it I don't want to pick on anyone else so you must do it someone else's way, um you're going to be slower but when we you're going to be less able to adapt it to the way your organization does stuff and so one of the things the four question frame does for us is it allows us to think about building blocks rather than saying we threat model using the STRIDE methodology, we say we threat model using the four questions and in this instance and in most instances here at the Acme company we're going to start with STRIDE and use something else if STRIDE isn't working. That allows people who are strap modeling, excuse me, for the first time to get a sense of what it is. Okay, I'm answering this question what can go wrong? Okay, weird. I don't quite understand why but someone wants us to do it differently today, that's fine. We're still answering those four questions.

Engaging Early in the Process

[17:02] Adam Shostack: The other thing I want you to think about as a security person is you want to engage early while there's time to fix it on the drawing board. When you set up a review process, when you set up an approval process, the natural engagement with that is I'm going to do my work, i'm going to dot my eyes, i'm going to cross my tees and I'm going to bring you a completed document looking for your approval, your review and that review I don't want to make changes right, i want to go through this approval process and get my sign off. And if you engage early rather than in a review you end up in a much more productive, much more lean sort of world.

And so to get started I really want to encourage you to simply ask, "What can go wrong?" If you want to go a little bit deeper there's a gamified version of this called Elevation of Privilege. It's a card deck, you can buy the card decks, there's the showstack.org/eop link there, one of my partners makes them. Um you can also just download it, there's online, there's free versions at GitHub. Microsoft was nice enough to creative common license it after I created it and so Elevation of Privilege will draw people in a little bit more.

And in a couple of weeks at global at OWASP Global AppSec in DC I'm going to be doing a training um threat modeling intensive using AI with AI. Um I'm going to be delivering the keynote at uh global apps come say hi and also there are six other threat modeling talks at that conference and it's followed by threat modcon also in Washington DC that Saturday and so there's an amazing opportunity in just a couple of weeks to come to Washington DC and learn an enormous amount about threat modeling and with that I am going to um Yeah let's get into some Q&A right.

Q&A with Audience

[19:12] Jenn Gile: yeah exactly all right I'm gonna hide your slides and Katie will be there looking forward to meeting you. I know we're going to be there, uh andor labs will be there. Um so I don't know if anyone's going to be at uh global abstack drop it in the chat let us know where you'll be and in the meantime I have a first question from the Q&A Adam for you. I have a feeling I know what you're going to say to this but I'm going to ask it and I'm gonna add a little bit to it. Um so the question is, "Adam what is your preferred or favorite threat modeling tool?" Now I have a feeling you're going to say it's going to depend a lot on your organization and things like that so perhaps a little of like how would you choose a threat modeling tool to use? So what's what's your take there?

[20:05] Adam Shostack: so I I I'll give you the honest answer to to this. My favorite threat modeling tool is a whiteboard marker. Um and the reason people pick up tools is they're looking to solve a problem. That problem might be we're in a regulated industry and we can't give our regulator uh photographs of a whiteboard, they'll uh it's the word I want, they'll be disappointed in us. Um it doesn't look official enough.

[20:34] Jenn Gile: It doesn't look official enough.

[20:36] Adam Shostack: Um what again while I was at Microsoft the SDL threat modeling tool which I created while there was designed to help people who didn't know how to get started answering the question what can go wrong. There are more modern tools, things like OAS threat dragon does a similar sort of thing to the SDL tool. There's a lot of tooling that is attempting to build AI into the process to help people get going. I was talking to a buddy recently works for a company you all know them um and he said, "We're getting between 75% and 90% of the way there with AI but that last bit is a real challenge, it's a stumper." And my friends at Areas Risk are doing great work on automating it. There's I mean there's a new startup every week that's trying to automate it but but Armando the thing I want you to focus on is what problem do I want my tool to solve for me because that will help you pick one that solves that problem.

[21:49] Jenn Gile: That's great advice and we have two more questions. Um the first one is, "Should threat modeling be done per feature and should it be enforced uh to the SDLC?"

[22:04] Adam Shostack: yes and yes and let me add a whole bunch of nuance to my yes. Um my friend is our tarand do has a set of talks he's given titled things like continuous threat modeling or threat model every feature. And if you think about threat modeling as something that's going to produce a document you can't threat model every feature, you have to make it lightweight and I like to say think of threat modeling as a dial not a switch. You want a little bit of threat modeling for every feature, "What can go wrong with this thing that we're working on?" right this story this epic. And my my experience is something between 60 and 90% of features have no real security impact that requires deeper threat model. You don't have to turn the dial and get big. Some features do and if you just get in the habit of saying in in the user story or in the backlog grooming that you're doing, saying, "We considered security and we believe this story has no security impact," to me that is enforcing it in the SDLC in a lean way that enables you to threat model every story.

If what you want—and I was talking to someone recently—um they spent six figures getting a consultant to do a threat model of a thing they were working on. You can't do that in every story. Um and yeah that can get really expensive really fast.

[23:48] Jenn Gile: It's expensive, it's not lean.

[23:54] Adam Shostack: It's expensive, it's not lean, it's review and it doesn't fit into the way that we develop product. And I think that's that is so important that you think about threat modeling as a dial so that you can apply it to everything you're doing.

[24:08] Jenn Gile: Well said. Um okay next question. Um let me make sure I don't lose track here. Uh "How do you figure out when you're done? How do you know if you've done enough?"

[24:25] Adam Shostack: um don't don't worry about that. What you want to do is you want to time box. You want to say, "We're going to spend up to an hour threat modeling this and then we're going to stop." And if somebody says, "You know I think more would be useful here," then do the more. But if we start from the checklist-y I need a diagram with at least four boxes and four arrows and I need one STRIDE threat for each of those, people very very quickly start gaming your system. And and when they're gaming the system because your definition of done is too onerous, you're not going to get to the value that you want. And if you spend a little time with each story and you recognize that it's going to take us time—you know at a larger company it can take a year or more for threat modeling to really become a part of how you do things. And if you make big demands at the start because you have big aspirations and big hope—and I love those big aspirations and big hopes—but if you push too hard people will push back. And if you you can turn around and ask them, "Do you think you're done? Do you think you did a good job?" And the reason we have this "Did we do a good job?" question is to help people reflect on the work they've just done. You can use a standard agile retro style of what went well, what went poorly, what do we keep doing, what do we change and that will get you towards the what is the right thing for us here. The only other thing I want to say about this real quick is if you work at a medical device maker the Food and Drug Administration has a very very specific set of uh pre-market guidance which explains to you exactly what a definition of done is which is more heavyweight, but then again you're making medical devices so you're sort of used to it.

[26:44] Jenn Gile: Okay we have time for one more live answer and then I promise to all of you whose questions we didn't get to live, I will connect with Adam afterwards and we will uh record some quick answers. You can follow our LinkedIn um channel to see those. Um so the last question that we have time for before we pivot topics is from Muhammad. I love this question. Um "Are there any threat model based projects for students or security reachers researchers uh can do to improve their knowledge of threat modeling?"

[27:18] Adam Shostack: so there's a whole bunch of things you can do and those include coming to OWASP Global AppSec. It includes the threat modeling community that the threat mod team is putting together. Um it includes my books honestly and the other thing where is my where is the chat? Let me go into the chat. I'm just going to um drop a link in here. Something I do when I see a threat model that I like is I talk about it on my blog in a category that I call Threat Model Thursday. And what I do is I pick this thing up that say that I like and I say, "What do I like about it? Where do I wish they had done something different? Why do I wish that thing?" And and my entire goal is to share with you my belief and I would encourage you to do the same thing. And when you see a threat model you like, pick it up and ask, "What makes me think they did a good job?" And write down your answers, talk about your answers and say, "What can we learn from this?" We're all learning, we're And this is this is to tie back to the title of the event this is what lean is about is learning, evolving our practices, making them better and making the next thing that comes through the system better than the last one.

[28:51] Jenn Gile: I love it. Well Adam, thank you for taking the time today. I know you've got a lot of stuff going on. You won't be able to stay for the rest of the event so uh I'm grateful you were able to make this uh half hour or so.

[29:02] Adam Shostack: You're welcome and if there are more questions uh Jen you're going to send those to me and we'll get emailed answers to folks.

[29:10] Jenn Gile: Exactly, we'll we'll address that after you and I are both back at home.

[29:13] Adam Shostack: Sounds good, thank you all for your time, I appreciate us.