I'm this security guy. I have a sweet resume with lists of security stuff I did. I got security skills certifications to show I can actually do security and not just be a moderately adequate opponent in Trivial Pursuit Security Edition. So people come to me and ask me to solve their security problems like, “Our client accesses our mojingle over the doobywassy blah blah hackers.

And it sounds exactly like that.

I'm in a meeting room with the CISO and the subjectively important IT staff is sitting at a table all muttering, “Rhubarb rhubarb rhubarb.

I look at the whiteboard on the wall and it has a big picture of a spastic spider made of a kind of a rectangle in red marker with blue and green lines coming out of it. Next to it are a few smaller cubist spiders, some squat, some long, all floating around it next to clouds.

And it looks exactly like that.

The CISO goes to the whiteboard and draws what's probably not a seagull but looks just like one and I think okay, maybe there's a sunset coming into the picture but there isn't. The CISO just looks at me and says, “We can place the menangel here.” Which causes the volume of the staff saying “Rhubarb rhubarb rhubarb” to rise to levels so high I can't hear the fluorescent lights anymore.

I look good and hard at the picture. I consider the color scheme. It's nice. Not many offices have that kind of pastel-ish colored markers. They always buy those nasty, harsh ones that aren't just green but GREEEEEENNNNNN and that's really the screechy noise they make as you draw a line with them.

I stare at the board so as to avoid eye contact. I take a step closer to it because I feel like I'm supposed to. I feel like I'm supposed to unleash all this bad-ass security low-down and make everyone in the room go “oooooooo” and then I'd wink at the hottest employee in the room with this kind of knowingness.

But that doesn't happen. So I keep staring and wondering what I'm supposed to see.  I can taste the toothpaste in my mouth. The room is so quiet I can hear the fluorescent lights sneering at me now, “Nnnnnnnnnnn… You donnnnnnnnnnnnnn't knnnnnnnnnnnnow….

And I'm quiet too long because the CISO feels uncomfortable and so erases the seagull, which is a shame because it added balance to the picture, and moves it over a cloud, which I have to admit makes more sense. Then in a move that would rock the art world, he draws two more clouds OVER the seagull and stabs them with arrows! He looks at me in expectation. I nod approvingly yet in a noncommittal way. Still, that makes the CISO smile despite causing more “rhubarb” volume at the table.

there is a pressure on security professionals to just know every new technology and standard that falls, literally, out the cloud

Then it's silent a long time again. It occurs to me they won't be dropping more hints because now it's my turn to say something about the picture. I'm supposed to know how the mojingle interfaces with the menangel over the doobywassy despite the fact that before just now, just this very second, I have never heard of those things. 

Well, that's not really true. I had heard of them but never got around to looking into them. Of the million things I have gotten to there's a million more I haven't. Not yet.

See, there is a pressure on security professionals to just know every new technology and standard that falls, literally, out the cloud. It's a pressure that every family doctor has felt when a patient walks in with a tentacle hanging out of their nose and the Internet is down.

But application security is so big and has so many technologies, development languages, and operating systems that it's truly impossible to know how to pronounce all of it, let alone be an expert in all of it.

So what I learned over time is that being a professional doesn't mean knowing all the answers rather it means knowing where to begin. Properly. Because that's what you learn eventually in every profession, even the oldest profession, is that how you begin is what effects the client's satisfaction regardless if they ever finish.

So I point to the big rectangle because it seems to be the most important and ask, "How do you have that configured to work?”

I really want to ask what the hell it does. I mean really, a Picasso spider? Why? But that's the wrong way to begin. The right way is a loaded question that tells me what it's interacting with and how's it working.

It takes connections from the client browser using bumblescript and passes them to the cloud application with the fookinbachs as the back-end with the sensitive data.

And that's exactly what I hear.

The room remains quiet. No rhubarbs. So it must be what it is. But I still have no idea. I try again, this time with more precision.

Tell me more about how the client connects to this fookinbachs,” I ask trying to sound like I know about every damn fookinbachs that ever was. “Show me step by step.

And this is how it goes.

I point at things and somebody has to tell me what it does, how it works, what it interacts with, and how they currently think they secured it, or not.

That's a technique to secure anything. Those are the things that NEED to be asked if you expect to NOT run an application security program into the ground. And you won't just run it into ground but you'll do it faster than if you put a jet engine on it and actually raised the ground.

I look around. I'm making progress. The CISO seems interested. And that's better than happy because people are happy when you give them something but people only get interested if they are learning something.

The room is starting to feel really hot but it's probably just me moving around in front of the whiteboard. I'm on fire. I'm getting it and they're getting it. Well, I still don't get the point of the seagull yet.

People are happy when you give them something but people only get interested if they are learning something.

This is going well. While it's impossible for one person to know everything about application security, it is possible to leverage the knowledge and expertise of all the people working with the systems to figure out the security that's needed. Some people call that consulting. But I'm not some people.

For me, consulting is being a hired gun. You show up, shoot everything you're paid to shoot, and then go home not caring where they bury the bodies. But I care. I'm committed. I have a reputation in security so I don't want to just shoot out the suspected problems but actually figure out the real problems and make security happen. And it will only happen if the people involved with the systems are involved in the security process. So because they don't do this, these hired guns often shoot themselves and the whole team in the process. Which is why, kids, you literally never bring a security consultant to a gunfight.

At some point during this show and tell, one of the people who drew the short straw during career moves and ended up at this IT table points out that the application doesn't actually do that in that way.

That will always happen. Whenever you get a group of people together to discuss how something works there will always be at least one who has to speak up on the contrary and probably it's that same kind of person who messes with the settings on the car radio while you're driving, the bastard.

So I hear him out. Because I have to. And because sometimes guys like this make sense. But I don't have to acknowledge it out loud. And of course he's right, the bastard. The system doesn't actually work in the way the rest of his team thinks it does.

Applications often interact with many more systems and in many more ways then the managers think it does whether it's because of logging, storing data, verifying credit card data, verifying something's in stock, and so on. It also does it all with the customer's data. And so it's just that this kind of thing puts me in a tough position with a client between making their security and making their liability.

You see, application security costs money. Say what you want but it's the cost of doing business and whether you're selling kittens from a sack, underwear made from butterfly wings from the back of a car, or bags of electrons from a web server, there is a cost to make sure you can not only do it but do it without losing more money.

We live in a world where people steal things and so to placate them we built this amazing, interconnected, resilient communications system that helps them steal from really far away. So if you also want to do anything over this amazing Internet then you need to spend money to keep stuff from getting stolen or worse. And by worse I mean terrible things like fill your browser history with hours of adult tap dancing recitals, set your servers to download and update automatically, and move all the missed appointment emails out of your junk folder and back into your inbox so it looks like you were just making excuses.

So when I talk to them about application security I'm really talking about protecting the money now and the money that could possibly be made. But see, I'm not a monster. I might dress like one sometimes as a consenting adult but never mind that. As not a public monster, I know application security is also about the people, particularly the ones who get caught up in the schemes of thieves and so they need to be protected. It's regular people who don't use fancy shmancy security browser settings and traverse the shopping aisles of the web applications without so much as yellow caution wet floor sign to warn them of any danger. Or to shift the liability to them when they do slip and fall.

This is where I need to side with the way too smart guy at the IT table who seems to know how the system actually works and probably also knows what the “tone” function on a modern television actually does. And so using this IT guy's observations is the perfect opportunity for me to talk to them about what they might really want, really really want, from application security.

I tell them application security has two parts, the security part and the liability part. The security part, I say, is to make sure we control all the interactions and separate out what we can't. There's a rumble of nodding and agreement.

Security means investing in people, processes, and software though. Especially for complex applications and systems. Because technology is really good at consistency and humans are not so much good at that. A person could miss a dozen vulnerabilities just because a cool helicopter flew by.

So security technology is needed to help secure the gazillion interactions in any standard inter-operating application. Because it's not realistic that a person can check it all without it. Consider that when I was a little kid I wanted to marry my 2nd grade teacher and win a gold medal standing on a saucer sled downhill in the winter Olympics and THAT was more realistic.

I look at the CISO and then at the whiteboard, pointing at the flow from cloud to spider-cube to seagull. The liability part, I say, is about how much you want to spend to secure the transactions across this flow to protect the data, your company, and your customer.

The CISO looks at me blankly.

This is the thing many security people and even more business people don't get. Whether you spend the money on products, training, people, effort, or insurance, security's going to cost something. Wait, no, they do get that. What they don't get is that you don't have to pay it all at once. And that's the thing about liability. You can defer investment in security by shifting the liability to the customer.

it's too easy to do a crappy job in application security, shove much of the liability on the users, act surprised when you get hacked

That's not something I'm making up. It's true. There's people who look much more intelligent than me with glasses and diplomas and stuff that wrote papers on this. So I know I need to get serious with them.

Listen,” I say seriously. “Enough time has passed for us as a nation to have this talk. I'm not going to sugar coat this. You're not going to like some of this and someone's feeling will get hurt. Just the one. You know there's a lot of talk about give and take in security. The common one is performance and security. But that's not the only one. There's also those risk presentations where they talk about security spending versus protection, getting a sweet deal on a horse for a $100 for which you're supposed to put some fence around it that costs a reasonable $1000. That's not bad if you've seen what even a crappy fence costs. So I never got that example because I'm like, hey, just be happy you got such a good deal on the horse.

They are staring blankly at me but I'm on a roll now. I go over to the grade-school art on the white board to finalize my point by pointing at it.

But the real trade off is how much security you will make and how much liability you shift on your customers or the public.

I shove my hands in my pockets and try to pretend I'm bulletproof.

We don't want to shift any liability onto the public,” says the CISO.

Really? Because your whiteboard shows a different story.

How so?” the clever IT guy asks.

I tell him. I tell them all.

Say you don't spend money on a good authentication system with stuff like tokens, single-use pads, or out of band something or other. That automatically puts the liability on the customer to make and protect their own passwords. That puts the liability on them because they are not authentication experts and you are setting them up to fail.

Silence. The fluorescent lights flicker helpfully with my next example, “Nnnnnnnnnnnnnnncrypt….

Or you don't want to encrypt the records in the database because you don't want to pay for the performance hit to decrypt each one as it's called. If that table gets cracked you are pushing the liability on the customer to deal with the mess of identity theft or future loss. Sure maybe you'll cover a year of credit monitoring but that's like losing your neighbor's solid gold lawnmower and offering to pay for the kid down the street to mow their lawn for a summer.

The table erupts, “Rhubarb rhubarb rhubarb.

From what you're saying it's normal to push liability on the public,” the CISO says. “So why don't they get in trouble or fined or something for doing that?

Because it's a best practice,” I say truthfully. “If you break the law you will be fined. But do the bare minimum required of you by best practices and any compliance objectives. That will protect you legally.

Unfortunately, that's true.

What about our customers? They're used to making their own passwords.

Right, it is socially acceptable for the public to do that. It is also socially acceptable to lose other people's identity records at the moment. Just make sure you only shift the liability for the things that the public will accept. And be sincere when you apologize for losing it in your email to all of them.

I can see the IT team thinking this through. The CISO seems to be running numbers in his head.

And this works?” the CISO asks.

It works. But it's like getting a sharp stick in the eye. It's really really bad. And it's also something you needed to know about,” I say.

Why do we need to know?

What do I say? The truth?

I tell you this because if you don't go through each interaction and define the controls for it then that's what you will end up doing by default. You needed to see what it looked like from the other side so you're aware it exists. Otherwise you'll think you figured it out for yourself and will actually love the idea like a fourth grader loves the clay candy dish they made themselves in art class despite the fact that it's a horrible candy dish.

The CISO stands still, interested.

The thing is it's too easy to do a crappy job in application security, shove much of the liability on the users, act surprised when you get hacked, and then publicly fire somebody from the IT team to make it all good.

As I've said before, nothing good has ever come out of an art class. Which brings me to the conclusion here.

If you want real security because you care then you need the right plan, you need to recognize that it's a daily thing that takes time, you need to bring on security professionals not gunslingers, you need good security software, you need to watch where the liability falls, and you need a good beginning.

And truthfully, doing it right means you'll never reach the end.

Which looks exactly like this.

About Pete Herzog

Pete knows how to solve very complex security problems. He's co-founder of the Institute for Security and Open Methodologies (ISECOM). He created the international standard on security testing and analysis and Hacker Highschool.

Comments (9)

Deepak Suri | March 29, 2016 6:17 am

Hey hi Pete nice to read this article...even i am also learning security certification and agreed with your points. Thanks for sharing article and keep posting

Zac1 | March 29, 2016 1:11 pm

A great write up! I love all the art references and trying to figure out what they mean and in return their information being wrong. That is completely par for the course!

Chana | March 29, 2016 3:19 pm

Hysterical and oh, so true!!!! This is making the rounds of our security folks right now...thanks so much!

Being Khan | March 30, 2016 2:22 am

That's absolute truth. Great job man.

Jay Libove | March 30, 2016 5:36 am

Fantastic article, Pete. I've had this experience many times, both when I worked internally as an organizational information security consultant in multinationals, and now when I do external information risk management consulting for other large companies.
Although, to be fair to some of the CISOs I've met, they often do understand that this is our experience, and they do want us to do exactly what you describe. This is no way changes the truth of your article - we're not to know everything about everything (although we do have an annoying habit of knowing a lot about a lot!); we're to know how to help others think about security effectively and practically.

Andy Boura | March 31, 2016 12:54 pm

Great article. Too many people don't know what they don't know. Or if they do they hide it well.

Martin O'Neal | March 31, 2016 1:06 pm

As ever, Pete does a fine job of hitting the seagull on the head. Sorry, nail. ;)

-pete. | April 1, 2016 4:57 am

Thanks all for your stories and praises! I'm really glad you liked it. I think it's one of those things we never talk about because we think it makes us look bad but it's really a hell of a skill to be able to get to the security essence of anything, even if you need to be told how it works. Actually, I'd love to hear more stories from people on cases where they had no clue walking in and yet were able to turn it around. I know it happened to me more than once so let's hear about yours!

Pradeep Sanyal | April 4, 2016 2:58 pm

Funny and so-true. Loved your writing style.

Please Post Your Comments & Reviews

Your email address will not be published. Required fields are marked *

Love to learn about Application Security?

Get all the latest news, tips and articles delivered right to your inbox.