Episode 34: Program vs Hacker Debate

Justin Gardner (@rhynorater):
Sup

Joel Margolis (teknogeek):
I just

Justin Gardner (@rhynorater):
dude.

Joel Margolis (teknogeek):
saw you get like prepped. You, you like

Justin Gardner (@rhynorater):
I...

Joel Margolis (teknogeek):
at the countdown started, you backed up. You like hyped yourself up,

Justin Gardner (@rhynorater):
You just gotta

Joel Margolis (teknogeek):
did a little stretch.

Justin Gardner (@rhynorater):
get a little, get a little litty, you know? Dude, I'm getting myself ready for this one because this is gonna be, this is gonna be a feisty one today. I'm gonna have to bring the feist to the show

Joel Margolis (teknogeek):
Oh boy, alright.

Justin Gardner (@rhynorater):
because we are representing, we got, in the left corner, we have Justin Gardner representing the hackers. On the right, on the right corner, we have Joe Margolis representing the hackers and the program managers.

Joel Margolis (teknogeek):
Other programs. Yeah. Yeah, I'm playing devil's advocate here

Justin Gardner (@rhynorater):
Yeah, yeah, no, it should be a fun one, man. I'm excited and yeah, I'm gonna, you know, we're pretty tight, you know, and so it's gonna feel good to like, just kind of rip you a new one in this debate,

Joel Margolis (teknogeek):
Yeah?

Justin Gardner (@rhynorater):
I think, you know, and

Joel Margolis (teknogeek):
Okay.

Justin Gardner (@rhynorater):
just,

Joel Margolis (teknogeek):
I mean,

Justin Gardner (@rhynorater):
you know,

Joel Margolis (teknogeek):
listen,

Justin Gardner (@rhynorater):
so we'll see.

Joel Margolis (teknogeek):
I was looking through this doc and there was a lot of very feistily written points and I had some very feistily written answers in my head, so... Uh...

Justin Gardner (@rhynorater):
That's great. In true Joel fashion, you've got the feistily written stuff in your head. That's

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
good.

Joel Margolis (teknogeek):
yeah.

Justin Gardner (@rhynorater):
All right. So let's take a little breath, take a little breather, calm down. We got a news segment to hit before we get

Joel Margolis (teknogeek):
Hahaha

Justin Gardner (@rhynorater):
into the meat.

Joel Margolis (teknogeek):
Yeah, yeah

Justin Gardner (@rhynorater):
All right. So first up on the news, well, first of all, we're kind of a little bit light on news this week. I don't know if that's just because everyone dumped all their stuff at... Defcon, yeah, I was gonna, I actually haven't even asked you about Defcon. How was Defcon?

Joel Margolis (teknogeek):
Yeah, I mean, Defqon was pretty good. I got COVID, so

Justin Gardner (@rhynorater):
Oh no.

Joel Margolis (teknogeek):
that was

Justin Gardner (@rhynorater):
RIP.

Joel Margolis (teknogeek):
the less good side of it. But yeah,

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
other than that, it was good. It was fun. It was great to see everybody. Good to hang out with people who sometimes I don't even get to see people like,

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
you know, once a year. So yeah, it's always a nice time.

Justin Gardner (@rhynorater):
Did you even, did you go to Defcon itself or just H1702?

Joel Margolis (teknogeek):
I didn't even step foot in the conventions that I didn't buy a badge

Justin Gardner (@rhynorater):
Ha ha

Joel Margolis (teknogeek):
but like

Justin Gardner (@rhynorater):
ha!

Joel Margolis (teknogeek):
honestly it doesn't seem like I missed out on much given what the badges were this

Justin Gardner (@rhynorater):
Yeah

Joel Margolis (teknogeek):
year, but yeah, no, I'm my personal opinion is that most of the time the talks are all uploaded

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
except for I think a couple of the villages sometimes

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
don't do it. There's like a whole thing with that, but

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
for the most part, pretty much every single talk is uploaded on YouTube. So if there's stuff that I didn't get to see. I can just go see it. Oftentimes they're really good talks. I'll just hear what they are and I'll go look

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
them up. And like I said, all the people that I... tend to see, I end up seeing outside of Defcon. I really like going to Defcon to see the talks, but

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
one of the disadvantages is that you can't see everything, so

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
I figured I'd save my money, enjoy Vegas a little bit, a little more chill, a little bit less

Justin Gardner (@rhynorater):
save

Joel Margolis (teknogeek):
walking

Justin Gardner (@rhynorater):
your money

Joel Margolis (teknogeek):
around

Justin Gardner (@rhynorater):
and then

Joel Margolis (teknogeek):
conventions.

Justin Gardner (@rhynorater):
enjoy Vegas. Sure.

Joel Margolis (teknogeek):
Yeah, save

Justin Gardner (@rhynorater):
Like,

Joel Margolis (teknogeek):
my money by losing

Justin Gardner (@rhynorater):
go

Joel Margolis (teknogeek):
it

Justin Gardner (@rhynorater):
to

Joel Margolis (teknogeek):
all.

Justin Gardner (@rhynorater):
the

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
freaking

Joel Margolis (teknogeek):
yeah.

Justin Gardner (@rhynorater):
tables

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
or whatever.

Joel Margolis (teknogeek):
yeah,

Justin Gardner (@rhynorater):
Geez. Did you gamble at all?

Joel Margolis (teknogeek):
yeah. Yeah.

Justin Gardner (@rhynorater):
All right. We're not from the tone of that. Yeah. I say we're not going to

Joel Margolis (teknogeek):
Not.

Justin Gardner (@rhynorater):
go down that path. All right.

Joel Margolis (teknogeek):
I didn't walk away a mega millionaire, I'll tell

Justin Gardner (@rhynorater):
didn't

Joel Margolis (teknogeek):
you

Justin Gardner (@rhynorater):
walk

Joel Margolis (teknogeek):
that.

Justin Gardner (@rhynorater):
away with the dub. All right, well that's part of the experience, I suppose. Yeah, and about the talks, yeah, I kind of agree. I think there's something cool about being able to go in person, especially if it's your friends and support them and ask questions afterwards if you have questions,

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
but yeah, most of the time on those talks, you're gonna wanna rewind and take notes and dive

Joel Margolis (teknogeek):
for

Justin Gardner (@rhynorater):
a

Joel Margolis (teknogeek):
sure.

Justin Gardner (@rhynorater):
little bit deeper. So it's nice that they put them up on YouTube a little bit later.

Joel Margolis (teknogeek):
Yeah, yeah.

Justin Gardner (@rhynorater):
Sweet,

Joel Margolis (teknogeek):
Yeah, actually

Justin Gardner (@rhynorater):
man.

Joel Margolis (teknogeek):
they're already up on the Defqon YouTube. So

Justin Gardner (@rhynorater):
Yeah, yeah they are.

Joel Margolis (teknogeek):
yeah,

Justin Gardner (@rhynorater):
Yeah,

Joel Margolis (teknogeek):
it's

Justin Gardner (@rhynorater):
so

Joel Margolis (teknogeek):
only been

Justin Gardner (@rhynorater):
that's

Joel Margolis (teknogeek):
a couple

Justin Gardner (@rhynorater):
good.

Joel Margolis (teknogeek):
of weeks. So if you wanna go check out those talks, they're all available.

Justin Gardner (@rhynorater):
Yeah, yeah, it's been, it's been a long couple of weeks, man. You know, I realized we haven't actually had a Justin and Joel episode since then. So yeah, that's,

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
that's

Joel Margolis (teknogeek):
first

Justin Gardner (@rhynorater):
good.

Joel Margolis (teknogeek):
you had COVID, then I had COVID,

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
now

Justin Gardner (@rhynorater):
I

Joel Margolis (teknogeek):
we're

Justin Gardner (@rhynorater):
got a COVID on

Joel Margolis (teknogeek):
finally,

Justin Gardner (@rhynorater):
a cruise. That was,

Joel Margolis (teknogeek):
oof.

Justin Gardner (@rhynorater):
that was rough. Um, let's see here. All right. Well, let's, let's hit the news then. Um, first item up on the list I'll talk about was, uh, was Rezo, uh, a loyal, a loyal follower of the podcast. Bless his heart.

Joel Margolis (teknogeek):
Drew?

Justin Gardner (@rhynorater):
released something really cool called a prompt injection primer for engineers. And essentially it's a document outlining what kind of things you need to be worried about as an engineer with regards to prompt injection. And yeah, I know we have a lot of engineers listening to this pod, so I wanted to shout it out. And if any of you are working with AI-related tools and are kind of... a little bit worried about what prompt injection may, as it is an unresolved bug, we still don't have a solution to prompt injection that's perfect. It's not like just escaping our double quotes and single quotes in SQL and stuff like that. It's unsolved problem. I think Joseph does a good job here of breaking down what exactly the attack vector is. He's got a little flow chart on whether you need to be worried about it or not. So I think this can make it a lot easier for people to understand what kind of impact this bug could have.

Joel Margolis (teknogeek):
Yeah, it's super interesting. Um, I saw that he had posted on Twitter and it

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
seems like a great resource to just sort of like put some stuff out in, in

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
this space of security. Uh, it's such a new technology and there's so many

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
things that we haven't really figured out. Like you said, it's, it's like an unresolved problem. So

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
I think it's great to have some awesome minds like Rezo and

Justin Gardner (@rhynorater):
Hmm.

Joel Margolis (teknogeek):
I think yourself as well, right? Um.

Justin Gardner (@rhynorater):
You know, I may have seen it before it was released.

Joel Margolis (teknogeek):
Yeah, yeah, so it's good to have these types of minds already working on the problem, already putting out resources out there, helping companies get secure before they even know it.

Justin Gardner (@rhynorater):
Yeah, yeah, for sure. So definitely check that out if you're working with AI stuff at all. Pass it around to anybody who is working with AI because we all kind of saw what happened with Web3 when everyone went crazy boom with Web3 stuff. Everyone was developing, it was happening so fast. Volums were flying all over the place. Money was flying out of wallets left and right. And so luckily AI is not directly linked to currency like Web3 was, but...

Joel Margolis (teknogeek):
Not yet.

Justin Gardner (@rhynorater):
Yeah, right, exactly. Not yet. So hopefully we won't have quite as severe of a fallout, but definitely something to check out if you're working with it. All

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
right,

Joel Margolis (teknogeek):
yeah, totally.

Justin Gardner (@rhynorater):
let's

Joel Margolis (teknogeek):
Did

Justin Gardner (@rhynorater):
see what else we got.

Joel Margolis (teknogeek):
you see this thing from Portswinger?

Justin Gardner (@rhynorater):
Ah,

Joel Margolis (teknogeek):
There's

Justin Gardner (@rhynorater):
yeah, dude,

Joel Margolis (teknogeek):
another

Justin Gardner (@rhynorater):
I

Joel Margolis (teknogeek):
XSS vector.

Justin Gardner (@rhynorater):
freaking love when ports of your research drop something. So they

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
dropped a, I guess Chrome now supports the on scroll end sort of event handler. So this is a great way to get past those. pesky laughs for, I don't know, hopefully a month or two until they update their word list or whatever, but

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
it works on any element. So in the little tweet that we'll link in the description, they have it working on the XSS element, right? The little fake XSS element. And if you combine this with the auto scroll to a specific element feature that they have in Chrome, if you put like the ID and the hashtag and the hash fragment up at the URL, then you can just pop it without any interaction.

Joel Margolis (teknogeek):
Nice, nice, that's awesome. Yeah, this is one of those accounts. You mentioned this way early in the pod and I

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
followed suit right after you did, but I have tweet notifications turned off for Portswigger Research because it's just one of those accounts that is constantly pumping out really, like immediately useful information

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
just like this, which is a screenshot and a link to their post about this. specific vector and how you can use it. You know, it doesn't get much better than that. So, yeah, this is super, super interesting, especially if XSS's are your bread and butter. On scroll

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
and there you go.

Justin Gardner (@rhynorater):
Yeah. So XSS that's, that's that for XSS. The next thing that I wanted to talk about was, uh, this talk that Gunnar Andrews, AKA Golden InfoSec did at recon village. I saw this pop up on YouTube and I know that he's been working. What the heck? My Google Home just started freaking out in front of me.

Joel Margolis (teknogeek):
hahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahah

Justin Gardner (@rhynorater):
Stop! Unplug! Maybe Gunnar Andrews, does it activate my... Gunnar, what did you do?

Joel Margolis (teknogeek):
Oh

Justin Gardner (@rhynorater):
Does

Joel Margolis (teknogeek):
no,

Justin Gardner (@rhynorater):
it activate my Google Home? No.

Joel Margolis (teknogeek):
he got you remotely.

Justin Gardner (@rhynorater):
Yeah, no, I know he does some crazy recon stuff with regards to specifically with regards to serverless architecture. So I was really thrilled to see that recon village put up a talk by him. Um, just like Joe was talking about in the beginning of the episode, you know, you don't necessarily have to be at Defcon to get all this wonderful information that they released. And, uh, I watched through this talk and it's super awesome. And not, not only does he come in, he's not like a serverless fanboy either. Like he comes in and he says like, Hey, this is the These are the pros and cons of serverless. This is when you should use serverless. This is when you should just use a server, you know? And so he does a really good job of outlining what kind of tasks work best in serverless architecture and how to use that to power your bug bounty hunting and your recon. So definitely a hard recommend on that one.

Joel Margolis (teknogeek):
Yeah, I'm gonna I've got that on my list and I'm gonna have to watch it after we finish recording

Justin Gardner (@rhynorater):
Yeah, it's got, I want to say, how long is it? 40 minutes? Yeah.

Joel Margolis (teknogeek):
Yeah, not bad.

Justin Gardner (@rhynorater):
I put it on 2x and I was still getting the full value out of it. So definitely worth a listen.

Joel Margolis (teknogeek):
Awesome. All right, before we hop into the main content,

Justin Gardner (@rhynorater):
Mmm.

Joel Margolis (teknogeek):
real quick,

Justin Gardner (@rhynorater):
True.

Joel Margolis (teknogeek):
Jason Haddix, former guest of the show, Jay Haddix, Jason Haddix, whatever you know him by, he is doing a course, it's the Bug Hunters Methodology live training course, and he has very generously given us a invite code to give away. So one of our listeners will be randomly selected to get a free access to his live Bughunters methodology

Justin Gardner (@rhynorater):
$550

Joel Margolis (teknogeek):
course. Yeah,

Justin Gardner (@rhynorater):
value.

Joel Margolis (teknogeek):
I mean, huge value. So. First of all, major shout out to Jason.

Justin Gardner (@rhynorater):
Mmm.

Joel Margolis (teknogeek):
Thank you. We can't appreciate it enough.

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
We're hoping that whichever of you ends up winning this will get the most value out of it as

Justin Gardner (@rhynorater):
Give

Joel Margolis (teknogeek):
possible.

Justin Gardner (@rhynorater):
us your notes.

Joel Margolis (teknogeek):
Yes, give us your notes.

Justin Gardner (@rhynorater):
It's funny

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
actually, funny story. So I had talked to Jason about this and Jason's like, yeah, I can probably give you a code to give away on the pod, something like that. And so he messages me the link and then he's like, oh, I should probably just message Joel the link as well. So he just drops the discount code in Joel's DMs, right? And Joel's like, oh, heck yeah. I can't wait to go to this course.

Joel Margolis (teknogeek):
I was

Justin Gardner (@rhynorater):
This

Joel Margolis (teknogeek):
like,

Justin Gardner (@rhynorater):
is gonna

Joel Margolis (teknogeek):
oh wow.

Justin Gardner (@rhynorater):
be awesome. And I'm like, no Joel, that's for. That's for the listeners, okay?

Joel Margolis (teknogeek):
Yeah, I was like I saw it I was like oh, I'm gonna have to claim that later And then

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
like a couple hours later. I was talking about it with Justin and he was like oh no We're gonna give that away. I was like okay It looks like it's the same code and then like within a few minutes I get a message from Jason He's like by the way, that's the giveaway on the podcast Yeah,

Justin Gardner (@rhynorater):
Yeah, you can see the disappointment in Joel's eyes.

Joel Margolis (teknogeek):
so definitely you're gonna want to enter to win this we have

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
a very easy way to entry just go to ctbb.show slash giveaway

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
and that's, you'll be taken somewhere, probably Google form, haven't really figured it out yet, but

Justin Gardner (@rhynorater):
We'll

Joel Margolis (teknogeek):
somewhere,

Justin Gardner (@rhynorater):
put it on the new website potentially.

Joel Margolis (teknogeek):
yeah,

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
on our new website. Yeah, so ctbb.show slash giveaway and that's how you can enter to win a free entry to Jason Haddix's Bug Hunter methodology course.

Justin Gardner (@rhynorater):
Yeah, and I'm looking through this course, man, and I was reading through the description and it says explicitly, so I think this matches well with the critical thinking audience. Unlike other courses, TBHM Live is not an A to Z or beginners-oriented course. And I'm looking through the syllabus and this is a lot of content that he's gonna try to cover in two days. So I imagine he'll be moving pretty quickly. Looks like a very... a very interesting course, so definitely check that out. And like I sort of vaguely mentioned just a second ago and totally forgot to announce at the beginning, we have launched a new website for Critical Thinking. It is live. It will have, it's going to be a little bit better organized than our past website, so we've got each episode popping up. We've got the video, we've got the audio, whichever your preferred medium is. We've also got transcripts, which is something a lot of you guys have been asking for a long time. Those will be attached to each podcast release. I'm not sure that it'll be on the instant that the episode drops, but hopefully within the first couple hours of an episode dropping, we should have the transcript up there and ready for you guys to consume. So there's also a contact form on the website that you can get in touch with us via. So shoot any questions or concerns about that over there.

Joel Margolis (teknogeek):
Awesome, all right, should we kick off the main

Justin Gardner (@rhynorater):
Dun-dun-dun,

Joel Margolis (teknogeek):
show? I don't

Justin Gardner (@rhynorater):
here we go.

Joel Margolis (teknogeek):
even, the main attraction.

Justin Gardner (@rhynorater):
Let's see, all right, Richie, we're going to need you to, Richie's our editor, Richie, put in like some sort of, you know, dun-dun-dun, or something like that here, some epic start. Okay,

Joel Margolis (teknogeek):
Okay, wait, real quick, real quick.

Justin Gardner (@rhynorater):
all right.

Joel Margolis (teknogeek):
I do need to get some formalities out of it. Okay, I do currently work at a tech company. I'm not even gonna name them. We're just

Justin Gardner (@rhynorater):
Okay.

Joel Margolis (teknogeek):
gonna keep it completely out of this.

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
I do manage bug-managing programs there. However, everything that we're gonna say today has nothing to do with my employer, has nothing to do with my job, it has nothing to do with my day-to-day. If anything... remotely like sensitive or anything. We're gonna cut it out. So yeah, I just wanna clarify that like, this is based on my experience

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
as a program manager and a bug bounty hunter. This is not reflective of my employer or my current work or anything like that. Okay,

Justin Gardner (@rhynorater):
Nice,

Joel Margolis (teknogeek):
red tape,

Justin Gardner (@rhynorater):
fully disclaimed.

Joel Margolis (teknogeek):
red tape out of the way.

Justin Gardner (@rhynorater):
There

Joel Margolis (teknogeek):
Yes.

Justin Gardner (@rhynorater):
you go, very nice. And you've had, you know, you've managed other programs in the past before too,

Joel Margolis (teknogeek):
Yeah, yeah, and I'm

Justin Gardner (@rhynorater):
so,

Joel Margolis (teknogeek):
happy to say that like I, you know,

Justin Gardner (@rhynorater):
yeah.

Joel Margolis (teknogeek):
before my current role, I was at Uber. I managed,

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
I helped, well, I didn't manage it, but I helped manage it

Justin Gardner (@rhynorater):
Right.

Joel Margolis (teknogeek):
alongside the main manager.

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
So yeah, I definitely have had my fingers in a lot of different programs from the program side for many, many years now, so.

Justin Gardner (@rhynorater):
All right, all right. Disclaimer's all over. Boxing gloves are on, Joel.

Joel Margolis (teknogeek):
boxing gloves off

Justin Gardner (@rhynorater):
Let's

Joel Margolis (teknogeek):
on...

Justin Gardner (@rhynorater):
go. Wait, off? What the heck? This

Joel Margolis (teknogeek):
Ah, wait,

Justin Gardner (@rhynorater):
man's gonna

Joel Margolis (teknogeek):
yeah.

Justin Gardner (@rhynorater):
knuckle to the face. All

Joel Margolis (teknogeek):
Suit off, boxing

Justin Gardner (@rhynorater):
right,

Joel Margolis (teknogeek):
gloves on.

Justin Gardner (@rhynorater):
so I'm going into my state. I'm going into my hacker, hacker representation state. Dude. What is up with the zero day policy, man?

Joel Margolis (teknogeek):
Okay.

Justin Gardner (@rhynorater):
If your scope is vulnerable and it's in scope, why am I not getting paid for this awesome zero day that I found?

Joel Margolis (teknogeek):
Okay, so for the record, I think a lot of things we will probably semi-agree on,

Justin Gardner (@rhynorater):
Probably.

Joel Margolis (teknogeek):
and I think some of this should really be sort of to the other program managers if your program

Justin Gardner (@rhynorater):
I'm out.

Joel Margolis (teknogeek):
isn't adhering to this. You know, this is sort of my take that I think

Justin Gardner (@rhynorater):
I need you to devil's

Joel Margolis (teknogeek):
programs

Justin Gardner (@rhynorater):
advocate

Joel Margolis (teknogeek):
should be

Justin Gardner (@rhynorater):
a little

Joel Margolis (teknogeek):
adhering.

Justin Gardner (@rhynorater):
bit though, okay

Joel Margolis (teknogeek):
Okay,

Justin Gardner (@rhynorater):
Joel?

Joel Margolis (teknogeek):
so

Justin Gardner (@rhynorater):
Like I know you're a hacker

Joel Margolis (teknogeek):
sure.

Justin Gardner (@rhynorater):
and I know you're a good program manager, right? And so I realize, you know, we may

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
agree on some things, but also I want to hear the other side as

Joel Margolis (teknogeek):
Okay.

Justin Gardner (@rhynorater):
well, you know?

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
So

Joel Margolis (teknogeek):
So, so, so I'll tell you sort of both aspects from it. So

Justin Gardner (@rhynorater):
yeah.

Joel Margolis (teknogeek):
like why, why shouldn't you get paid? I mean, there's, there's a couple of reasons. I think the main reason is that there's nothing that can be done from the company side to fix this. And I think we discussed this a little bit when we talked with Shubs about. sort of how to navigate that kind of communication when you're reporting a zero day vulnerability, especially

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
if it's something that you've found, uh, to a company and whether

Justin Gardner (@rhynorater):
Mmm.

Joel Margolis (teknogeek):
or not they should be paying it. And a lot of it boils down to what is the company supposed to do? Because for one, it's nothing that their engineers caused. It's nothing that they accidentally added

Justin Gardner (@rhynorater):
Whoa, whoa,

Joel Margolis (teknogeek):
or, or

Justin Gardner (@rhynorater):
whoa.

Joel Margolis (teknogeek):
knew was there. Like, this is just an artifact of the fact that they use a piece of software and that other company made a mistake.

Justin Gardner (@rhynorater):
Well,

Joel Margolis (teknogeek):
Right.

Justin Gardner (@rhynorater):
okay, their engineers did do something here because they put it, they put a domain, you know, presumably they put a domain pointing to this asset. You know, I don't know if it's a cloud asset, you know, if it's a cloud asset, then they just pointed their domain at it. But you guys made the decision just like you would use, you know, to use open source software to run work with this provider. And that has inherently put your users at risk.

Joel Margolis (teknogeek):
Sure, yeah, and so that's where I'll start to agree with you is that there are certainly systemic steps that you can take from

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
a company side to help stop these things. And that generally involves like... doing audits ahead of time with the company, discussing what their current security team and current security practices look like. How do they store data? What are their password policies? What is their internal security policy? Like there's a lot of things that you can do as sort of checks and balances to try and make sure that the company you're about to

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
work with and whose software you're about to use will not compromise you. But at the end of the day, you didn't write the code, you probably don't even have access to the source code. And so depending on what it is, there might not be much you can do and yes they the engineers probably did create a DNS

Justin Gardner (@rhynorater):
Don't give

Joel Margolis (teknogeek):
record

Justin Gardner (@rhynorater):
me that shit, Joel.

Joel Margolis (teknogeek):
okay

Justin Gardner (@rhynorater):
Okay?

Joel Margolis (teknogeek):
but they didn't do anything wrong from a security

Justin Gardner (@rhynorater):
I don't

Joel Margolis (teknogeek):
like

Justin Gardner (@rhynorater):
know what

Joel Margolis (teknogeek):
from

Justin Gardner (@rhynorater):
to...

Joel Margolis (teknogeek):
a from a secure development standpoint okay there's nothing that the engineer could have done differently that would have stopped them from being vulnerable it was completely out outside of their control

Justin Gardner (@rhynorater):
No,

Joel Margolis (teknogeek):
okay

Justin Gardner (@rhynorater):
okay, that's sort of true. That is true, and I see that point. But also, heck that. And I need the bounty right now. No,

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
no.

Joel Margolis (teknogeek):
right. Yeah, sure. So like, again, I agree with you. And I think this is sort of where like from a program management perspective, where I draw the line is basically, is there impact, yes or no, pay

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
for impact,

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
right? Like it doesn't necessarily matter if it's your fault. It matters whether or not something can happen because of that vulnerability,

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
right? And

Justin Gardner (@rhynorater):
All

Joel Margolis (teknogeek):
so at the

Justin Gardner (@rhynorater):
right.

Joel Margolis (teknogeek):
end of the day, like you should be taking responsibility for the impact to your business, to your users, to your customers, whatever it is that, like, if you're using a piece of software that makes you vulnerable, you should pay for that to some extent. Now,

Justin Gardner (@rhynorater):
Mm-mm.

Joel Margolis (teknogeek):
I think it's fair to not pay a full bounty because

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
there's a limited amount of things that you can do. I also think that there needs to be some sort of like... You know, how do I

Justin Gardner (@rhynorater):
conversation?

Joel Margolis (teknogeek):
fix this?

Justin Gardner (@rhynorater):
Oh,

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
okay,

Joel Margolis (teknogeek):
yeah.

Justin Gardner (@rhynorater):
sure,

Joel Margolis (teknogeek):
Well, yeah,

Justin Gardner (@rhynorater):
sure.

Joel Margolis (teknogeek):
sure. But like, how do I fix this, right? Like, okay, you're reporting a zero day to me. Can I set up firewall rule up? Can

Justin Gardner (@rhynorater):
Yeah,

Joel Margolis (teknogeek):
I add something

Justin Gardner (@rhynorater):
I mean...

Joel Margolis (teknogeek):
to my WAF? Like, what do I do to fix it? What do you want me, the company who uses this piece of software to do? Because

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
if the other company doesn't even know about it, the only thing that we can do is say, hey, by the way, there's a vulnerability in your software. You need to fix this.

Justin Gardner (@rhynorater):
Okay, so I got the little hyperbolized hacker out of my system a little bit there and the raging, but I'm gonna try to approach this from an intellectual perspective as a hacker here. So my answer to that would be that, sure, you guys don't have access to the source, the vulnerable source code that's running on the target, assuming it's a closed source project or whatever. And so how do you remediate that situation? Totally fair question, I think. And I, whenever I report these, I always try to include some sort of remediation step, maybe, um, you know, changing some configuration in the actual third party app that, you know, you can find in the settings, but at the end of the day, you can just stick a WAF in front of it or you can stay, I mean, how hard is it to like spin up an Nginx reverse proxy, right? And, and like, just block off a specific endpoint.

Joel Margolis (teknogeek):
significantly

Justin Gardner (@rhynorater):
Like.

Joel Margolis (teknogeek):
easier said than done.

Justin Gardner (@rhynorater):
What,

Joel Margolis (teknogeek):
I promise.

Justin Gardner (@rhynorater):
are you serious?

Joel Margolis (teknogeek):
Yes, because, okay, here's the thing. In a perfect universe where every company is just like five engineers with an AWS box,

Justin Gardner (@rhynorater):
Hahaha!

Joel Margolis (teknogeek):
yeah, that works. You could just like spin up a service and be done with it. But in most modern tech companies, there's so much more process and procedure and like trying to fit into the existing technologies and. They might not even use nginx. Like there's so many other layers that don't just make it that easy where, yes, that would be a super easy solution if it worked, but it's completely dependent on implementation and use case.

Justin Gardner (@rhynorater):
Okay, but like almost every organization is using some sort of reverse proxy, right? Like, you know, whether it be, uh, you know, Apache and genetics, traffic or however you pronounce that thing

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
or, or have a laugh, right? Like, you know, and so just, I mean, I don't know, maybe this is my lack of enterprise experience coming into play, but, but if I were on a security team and I got a report of this and I, you know, there's an RCE on this box, you know, that you can only access through a certain point, I would want to be you know, spinning up a, you know, AWS, you know, reverse proxy, what are they, I forget what they're called, but, you

Joel Margolis (teknogeek):
I mean,

Justin Gardner (@rhynorater):
know,

Joel Margolis (teknogeek):
like the thing is that

Justin Gardner (@rhynorater):
I'm

Joel Margolis (teknogeek):
like, yeah,

Justin Gardner (@rhynorater):
pointing

Joel Margolis (teknogeek):
if

Justin Gardner (@rhynorater):
at

Joel Margolis (teknogeek):
it's

Justin Gardner (@rhynorater):
it right

Joel Margolis (teknogeek):
that

Justin Gardner (@rhynorater):
away.

Joel Margolis (teknogeek):
case, there's a few things that you can do, right? If it's not something business critical, IP restricted, take it offline,

Justin Gardner (@rhynorater):
Yeah, that's

Joel Margolis (teknogeek):
put it

Justin Gardner (@rhynorater):
true.

Joel Margolis (teknogeek):
behind a firewall,

Justin Gardner (@rhynorater):
Mm hmm.

Joel Margolis (teknogeek):
put it behind VPN,

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
like it's changing configuration. There's usually like a handful of things that you can sort of generically do that will probably help. If it's an actual zero day, there's like very few things. that the company can do outside of like preventative measures

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
around like

Justin Gardner (@rhynorater):
Yeah,

Joel Margolis (teknogeek):
preventing

Justin Gardner (@rhynorater):
you can't

Joel Margolis (teknogeek):
requests

Justin Gardner (@rhynorater):
you can't

Joel Margolis (teknogeek):
from even

Justin Gardner (@rhynorater):
patch

Joel Margolis (teknogeek):
hitting the

Justin Gardner (@rhynorater):
it.

Joel Margolis (teknogeek):
box.

Justin Gardner (@rhynorater):
Yeah

Joel Margolis (teknogeek):
Right? So there's not much you can fix. I see that you also mentioned one days in here. Honestly,

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
my opinion is that if it's a one day, like that's fair game. If there's

Justin Gardner (@rhynorater):
Mmm.

Joel Margolis (teknogeek):
an exploit out there publicly, especially,

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
and you have the software and it's publicly exposed, that is fair game.

Justin Gardner (@rhynorater):
So

Joel Margolis (teknogeek):
That

Justin Gardner (@rhynorater):
what you're

Joel Margolis (teknogeek):
is

Justin Gardner (@rhynorater):
saying

Joel Margolis (teknogeek):
like...

Justin Gardner (@rhynorater):
is before I report, I should disclose and then report the same day. Or no, the next

Joel Margolis (teknogeek):
I mean

Justin Gardner (@rhynorater):
day. Sorry, so it's one day.

Joel Margolis (teknogeek):
Sure if you want to time it so they you get your own CVE the CVE gets public then you publish your own POC

Justin Gardner (@rhynorater):
Yeah...

Joel Margolis (teknogeek):
and you exploit it before

Justin Gardner (@rhynorater):
Wait...

Joel Margolis (teknogeek):
everybody else. Yeah, I mean, yeah, I guess that's a public exploit, but

Justin Gardner (@rhynorater):
gonna be a busy couple days in that scenario.

Joel Margolis (teknogeek):
Yeah, yeah

Justin Gardner (@rhynorater):
No, okay, yeah, that makes sense. And I do agree, if it's got a public CV, then even if you're aware of it, I always add that having to do less asset enumeration or checking of your assets to say like, hey, yeah, this bug is vulnerable to this, I think that adds value. And so there's definitely some other areas where the bug bounty hunters add value with their reports And also it's an easier sell to the people that need to fix it say, okay Yeah, sure that somebody posted this paper that affects our blah-dee-blah But it's a different story when you can say actually a hacker shelled this box this morning and was in here and could have deleted The whole system, you know, I feel like that's

Joel Margolis (teknogeek):
Right.

Justin Gardner (@rhynorater):
an easier sell to your bureaucratic organization or whatever

Joel Margolis (teknogeek):
Yeah, and I think like generally each of these topics probably has some sort of useful takeaway from a program perspective. So like for this

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
case, like having better management and knowledge of your assets and what software is running on each asset.

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
that's probably a decent takeaway. If you're getting affected by zero day reports and one day reports a lot, that should tell you that from a security program perspective, you need to be identifying what software is running on your boxes better, excuse

Justin Gardner (@rhynorater):
Mm-hmm. Yeah.

Joel Margolis (teknogeek):
me, in a more efficient way and being able to update that stuff in a more efficient way. And hand in hand with that, also knowing what vulnerabilities are in the pipeline that are affecting your software so that you can fix it ASAP.

Justin Gardner (@rhynorater):
Yeah, no, that totally makes sense. Mariah's texting me something. Let me just send her this. Okay, all right, yeah. And I think that piece about patch management is very important, you know? And oftentimes people... I guess maybe I shouldn't say oftentimes, but I've seen several times when people have started bug bounty programs and they don't have a solid patch management platform out. And I just can think of this one time with my automation where like every week I was getting this ping, this is vulnerable, this is vulnerable, this is vulnerable, this is vulnerable. They didn't patch the CVE, this critical CVE that leaked AWS creds, right, for over a year. And I'm like, what are you doing? So anyway, so all right, you know, now I gotta get.

Joel Margolis (teknogeek):
What are you doing? We'll get to that shortly because

Justin Gardner (@rhynorater):
Okay, we'll get

Joel Margolis (teknogeek):
I know

Justin Gardner (@rhynorater):
to

Joel Margolis (teknogeek):
that's I know that's in there

Justin Gardner (@rhynorater):
that. Don't, that's not the next question. The next question is, okay, so let's talk a little bit about disclosure, okay?

Joel Margolis (teknogeek):
Sure.

Justin Gardner (@rhynorater):
I have done some research. I've found a vulnerability. I reported to you because I want to give you a heads up, but I also wanna be able to disclose this. But I would also like to be paid for my research. Why, why not? Why, like, you know, a lot of programs add do not disclose policies. What's up with that, homeboy? Ha

Joel Margolis (teknogeek):
Yeah. Okay.

Justin Gardner (@rhynorater):
ha.

Joel Margolis (teknogeek):
So, I mean, the reality is that a lot of this boils down to the fact that a lot of programs have a legal and comms team

Justin Gardner (@rhynorater):
Yeah, yeah.

Joel Margolis (teknogeek):
or both, a legal teams and a comms team.

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
And oftentimes security vulnerabilities are viewed positively by the security community, negatively by everybody else. Right.

Justin Gardner (@rhynorater):
Mm, mm, mm.

Joel Margolis (teknogeek):
And so if you think about it, it's

Justin Gardner (@rhynorater):
Yeah,

Joel Margolis (teknogeek):
like

Justin Gardner (@rhynorater):
that's interesting.

Joel Margolis (teknogeek):
the company doesn't really want to brag that they had a security vulnerability. Right.

Justin Gardner (@rhynorater):
Sure.

Joel Margolis (teknogeek):
The incentive there is good for the security team to be like, look, our bug bounty program's working great.

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
Terrible from a comms and publicity perspective around like, Oh, customer data could have been at risk or whatever. Right. Like I think that is always the challenge. And some companies are very good about owning security and being like, we don't give a shit, we're going to be public about everything and

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
like. If we have a security vulnerability, that's great. We don't care. We'd rather be public about it. And it doesn't really matter. Um, I think a lot of companies get really scared about that. And rightfully so because there's been plenty of outrage for various tech companies, Uber, for example,

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
around security vulnerabilities and how it was handled and how it was announced or not announced and all that

Justin Gardner (@rhynorater):
Yeah,

Joel Margolis (teknogeek):
kind of stuff. Um,

Justin Gardner (@rhynorater):
can take the stock price and all that sort of thing.

Joel Margolis (teknogeek):
Right. And so like, I think that is a real risk there. Now the other side is like, okay, I want to get paid. Great. Report it to the Bug Vanny program. I want to disclose it. Well, that's up to the program. Um, I see a lot of researchers who will report stuff and then get frustrated when they aren't allowed to disclose or they, you know, disclose anyways, or, or whatever, like here's the reality of it. When you submit a bug through a bug bounty program, you agree to the bug bounty program terms and policy. That states that you can't disclose stuff without the permission of the program.

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
If you disclose stuff without the permission of the program, you are violating the program terms, which number one, often excludes you from safe hardware protections, and number two,

Justin Gardner (@rhynorater):
RIP.

Joel Margolis (teknogeek):
sets you up for a platform ban, or at least some sort of a suspension or repercussion.

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
And so that's not to say that you can't report stuff without being able to disclose it. You can always like email security at, and

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
you know, reach out to them directly, not going through the bug bounty program.

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
You're not agreeing to their program terms if you're not going through the bug bounty program.

Justin Gardner (@rhynorater):
I wonder if you could submit directly to the bug bounty program, but explicitly state in the report that I decline the terms and agreements to this. And say like, I intend to disclose this bug, but I'm also giving you a heads up. I

Joel Margolis (teknogeek):
Hmm.

Justin Gardner (@rhynorater):
don't know.

Joel Margolis (teknogeek):
Yeah, I mean maybe I think that would be a little more nuanced because there's probably

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
some stuff in the hacker one Terms of Service that says like, you know, whatever but

Justin Gardner (@rhynorater):
It's definitely a way to set you up for platform problems, for sure.

Joel Margolis (teknogeek):
Yeah, like, I just wouldn't recommend it. And like,

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
the other aspect of it is, if you really wanna disclose something and the company really doesn't, you're gonna set yourself up for a rocky relationship with that company and

Justin Gardner (@rhynorater):
Right.

Joel Margolis (teknogeek):
that program every day after that. Because

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
if you decide to go ahead and disclose and they don't want to do it, they are really not gonna wanna work with you in the future.

Justin Gardner (@rhynorater):
Right.

Joel Margolis (teknogeek):
So if you find something like, you know, some other like really bad phone and you want to report it to them or you want to like, you know, get a bounty or something. How is that company supposed to feel when they've got this researcher who's like been super aggressive with them in the past? about disclosures and now coming in with another vulnerability.

Justin Gardner (@rhynorater):
Yeah,

Joel Margolis (teknogeek):
Like,

Justin Gardner (@rhynorater):
yeah,

Joel Margolis (teknogeek):
are they going to fix it? Yeah, probably.

Justin Gardner (@rhynorater):
that doesn't feel

Joel Margolis (teknogeek):
Are they

Justin Gardner (@rhynorater):
safe.

Joel Margolis (teknogeek):
going to want to like, you know, it really sets a bad precedent. I think for that type of, for the type of relationship that you want to have with the program, which is like an amicable working relationship where

Justin Gardner (@rhynorater):
Hmm

Joel Margolis (teknogeek):
you can

Justin Gardner (@rhynorater):
professional.

Joel Margolis (teknogeek):
secure.

Justin Gardner (@rhynorater):
Yeah

Joel Margolis (teknogeek):
Yeah. You can secure things together and both be like happy and positive about it. Not be like, I want to disclose this. You can't disclose it. Or I'm mad about that. Like,

Justin Gardner (@rhynorater):
Right.

Joel Margolis (teknogeek):
you know, like at the end of the day. That's part of the bargain, right? Part

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
of the bargain is that you get paid, so

Justin Gardner (@rhynorater):
but that feels

Joel Margolis (teknogeek):
there

Justin Gardner (@rhynorater):
a little

Joel Margolis (teknogeek):
are

Justin Gardner (@rhynorater):
dirty,

Joel Margolis (teknogeek):
somebody like.

Justin Gardner (@rhynorater):
right? Like that feels a little bit dirty if you say it like that. Cause you said that's part of the bargain and it is part of the bargain and you're totally right in representing it that way. But essentially what you just said is we're paying you to keep your mouth shut. Which is like really sketchy and not the way that I really wanna see bug bounties.

Joel Margolis (teknogeek):
Yeah, and I mean like, I don't want to get too reductive here because I think that

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
you could frame... a lot of things that way, but especially

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
like any private program you could phrase it that way

Justin Gardner (@rhynorater):
Nah.

Joel Margolis (teknogeek):
and that like Oh, you don't have a public program. So like You know, I don't know or like

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
oh things

Justin Gardner (@rhynorater):
Now

Joel Margolis (teknogeek):
are

Justin Gardner (@rhynorater):
that-

Joel Margolis (teknogeek):
out of scope like if it's out of scope then like, you know, you're basically just like telling me not to look at something that's vulnerable or like I think there's a lot of reductive ways that you could take that same

Justin Gardner (@rhynorater):
Yeah,

Joel Margolis (teknogeek):
kind

Justin Gardner (@rhynorater):
that's

Joel Margolis (teknogeek):
of

Justin Gardner (@rhynorater):
true.

Joel Margolis (teknogeek):
stance

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
and While I while I see it, I think that it's a little more nuanced than that

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
Like bug bounty programs are not just designed to pay a researcher. They're designed to set a neutral environment in place so that researchers can communicate things safely with the program

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
and the program can communicate things safely with the researcher,

Justin Gardner (@rhynorater):
Right.

Joel Margolis (teknogeek):
fix it, not worry about like... the researcher leaking their payload or telling

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
other people about it

Justin Gardner (@rhynorater):
Sure.

Joel Margolis (teknogeek):
and all this other kind of stuff and in return the researcher gets paid for that you know for not telling other people about

Justin Gardner (@rhynorater):
Well,

Joel Margolis (teknogeek):
it for reporting

Justin Gardner (@rhynorater):
for

Joel Margolis (teknogeek):
it directly

Justin Gardner (@rhynorater):
reporting the

Joel Margolis (teknogeek):
for

Justin Gardner (@rhynorater):
bug

Joel Margolis (teknogeek):
not exploiting

Justin Gardner (@rhynorater):
in general,

Joel Margolis (teknogeek):
it like

Justin Gardner (@rhynorater):
right? That's kind of where I was thinking with this, is like the incentive, the keeping your mouth shut piece is a little bit more of a, of a addendum to the whole situation. The motivating factor for a bug bounty program is these researchers have done high quality security research and deserve to be rewarded for that and to be compensated for that. And so, I think that's the primary services exchange.

Joel Margolis (teknogeek):
Yeah, I mean, yes, however, part of high quality security research, in my opinion, involves how the delivery is handled.

Justin Gardner (@rhynorater):
Mmm.

Joel Margolis (teknogeek):
And so you can't just do high quality research and then start making demands about what you should be paid and how it should

Justin Gardner (@rhynorater):
Yeah,

Joel Margolis (teknogeek):
be handled

Justin Gardner (@rhynorater):
that's a good point.

Joel Margolis (teknogeek):
and how fast it should be fixed and all this kind of stuff,

Justin Gardner (@rhynorater):
That's a good

Joel Margolis (teknogeek):
right?

Justin Gardner (@rhynorater):
point.

Joel Margolis (teknogeek):
Like again, you're participating in the bug bounty program. If you

Justin Gardner (@rhynorater):
Guys,

Joel Margolis (teknogeek):
wanna

Justin Gardner (@rhynorater):
I...

Joel Margolis (teknogeek):
send stuff to security at

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
it and you wanna strong arm the program, you can by all means go ahead and try that. And they may or may not respond, but they also may or may not respond in a very nice way.

Justin Gardner (@rhynorater):
Sure. I'm, you know, to all you hackers out there, I'm trying. He's just doing good, you know? Like,

Joel Margolis (teknogeek):
Ha ha!

Justin Gardner (@rhynorater):
like, I've tried to give him a piece of our mind here, but no, these are very reasonable answers you're coming across with Joel, so I kind of put you on the spot this episode, and I will

Joel Margolis (teknogeek):
That's

Justin Gardner (@rhynorater):
continue

Joel Margolis (teknogeek):
totally fine.

Justin Gardner (@rhynorater):
to put you on the spot, but

Joel Margolis (teknogeek):
That's

Justin Gardner (@rhynorater):
uh...

Joel Margolis (teknogeek):
fine, listen, these are a lot of things that I used to feel as a hacker

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
myself. I still understand where a lot of hackers are coming from with these things,

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
but having worked on the other side, the answers and the reasons behind stuff are so much more nuanced, I think it's important that people understand it.

Justin Gardner (@rhynorater):
Yeah, yeah, me too. With that being said, let's talk about the shittiest piece of bug bounties, which is dupes, and particularly internal dupes. What the heck, dude? Like, what, so, okay, hold on. I am gonna rant a little bit about this, okay? So if you've got an internal dupe, and you have not resolved this thing, right? And it's within a longer timeframe than one would expect. You know, a couple months for you know, maybe even a high or medium, or like a couple weeks for a crit. Like, okay, that seems like something that you need to still pay the researcher for, because you're sitting on that bug, and like I have put in high quality, I'm sorry, you can tell it's a little bit fresh, okay? But you know, I put in this high quality research, you've known about this issue for a while, my time just went down the drain, and it's still violated, and it's you know, our sort of agreement that we have in place of, I provide you with high quality security research and you pay me for that. And when no other hacker had reported it, this is an internal dupe either discovered from code review or maybe like an internal pen test or something like that, just feels like crap. And how can we get accountability for that as a hacker? How can we understand when truly this is an internal dupe or whether, you know, we're getting screwed out of our bounty?

Joel Margolis (teknogeek):
Yeah, so there's a lot of nuance and angles to this. I think what I'll start with is that I'll address that last part first, which is how can I be sure that this really is an internal do? The main thing I would say is where's the incentive for the program to lie to you in the longterm? Yeah, money, but

Justin Gardner (@rhynorater):
I'm, for those

Joel Margolis (teknogeek):
yeah,

Justin Gardner (@rhynorater):
of you that are

Joel Margolis (teknogeek):
Justin's

Justin Gardner (@rhynorater):
actually

Joel Margolis (teknogeek):
rubbing his fingers together.

Justin Gardner (@rhynorater):
listening to the podcast, I'm rubbing my fingers together while I'm taking a sip of water.

Joel Margolis (teknogeek):
Yes, money, however, reputation, right? And I think reputation, from the programmer's perspective, whether or not programs actually view it this way, but the reality is that your reputation is significantly easier to damage than it is to spend money, right? And so

Justin Gardner (@rhynorater):
Mmm.

Joel Margolis (teknogeek):
you're gonna damage your reputation significantly more,

Justin Gardner (@rhynorater):
with the hackers.

Joel Margolis (teknogeek):
yeah, by like, you know. either lie, like if you'd lie to a hacker and say, oh, this

Justin Gardner (@rhynorater):
Mmm.

Joel Margolis (teknogeek):
is an internal do, but it's not instead of just paying for something, the damage to your reputation, if that comes out or whatever is so much worse than whatever you paid for that bounty,

Justin Gardner (@rhynorater):
Yeah, no one's

Joel Margolis (teknogeek):
like

Justin Gardner (@rhynorater):
ever going to

Joel Margolis (teknogeek):
people

Justin Gardner (@rhynorater):
touch

Joel Margolis (teknogeek):
are

Justin Gardner (@rhynorater):
that

Joel Margolis (teknogeek):
just,

Justin Gardner (@rhynorater):
program

Joel Margolis (teknogeek):
yeah,

Justin Gardner (@rhynorater):
with a 10

Joel Margolis (teknogeek):
right.

Justin Gardner (@rhynorater):
foot pole.

Joel Margolis (teknogeek):
Exactly. Right. And so like the incentive for a program to lie like that is just like, you know, you have to think about it. Like, why would somebody do that? And it's not that it doesn't happen, but I think that there's not a great incentive to start with for like, why would that like, what is the reason for like some random employee who's running this program to like come sign on and be like, I don't want to pay for this. I'm going to tell them

Justin Gardner (@rhynorater):
Because

Joel Margolis (teknogeek):
that

Justin Gardner (@rhynorater):
it's

Joel Margolis (teknogeek):
it

Justin Gardner (@rhynorater):
not

Joel Margolis (teknogeek):
was

Justin Gardner (@rhynorater):
like

Joel Margolis (teknogeek):
that we

Justin Gardner (@rhynorater):
it's

Joel Margolis (teknogeek):
found

Justin Gardner (@rhynorater):
their

Joel Margolis (teknogeek):
it.

Justin Gardner (@rhynorater):
money anyway, right? Like it's not like it's the employee's money. Like it's not coming out of their salary, right? You know?

Joel Margolis (teknogeek):
I mean we could get into that in the net it wasn't in the budget part,

Justin Gardner (@rhynorater):
There

Joel Margolis (teknogeek):
but

Justin Gardner (@rhynorater):
would be no

Joel Margolis (teknogeek):
I mean

Justin Gardner (@rhynorater):
KPIs associated with how much money this Bug Bounty program is paying or isn't paying, would there?

Joel Margolis (teknogeek):
I pro- budgets are very con- very complicated.

Justin Gardner (@rhynorater):
Whoo!

Joel Margolis (teknogeek):
Budgets are

Justin Gardner (@rhynorater):
We

Joel Margolis (teknogeek):
very

Justin Gardner (@rhynorater):
found

Joel Margolis (teknogeek):
complicated

Justin Gardner (@rhynorater):
a spot,

Joel Margolis (teknogeek):
things.

Justin Gardner (@rhynorater):
boy! Alright.

Joel Margolis (teknogeek):
Yeah. Budgets are very, very complicated things. Uh, the way that budgets at companies, especially the larger the company, uh, the more confusing and weird it gets around

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
who gets what money for what thing. How does that process get approved? Uh. How do you get more money? How do you get more money next year?

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
There's a lot of different aspects to budgets and that kind of stuff. And you have to remember that Bug Bounty at the end of the day, when they make a payment, that comes out of a budget that has been set aside probably at the beginning of the year for Bug Bounty. And that pool is all they have to work with. And so if they run out of money, that's a problem. And if they have too much money, that's also a problem.

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
So we're kind of jumping. ahead a little bit

Justin Gardner (@rhynorater):
We

Joel Margolis (teknogeek):
into

Justin Gardner (@rhynorater):
are jumping ahead,

Joel Margolis (teknogeek):
budgets here.

Justin Gardner (@rhynorater):
but

Joel Margolis (teknogeek):
So.

Justin Gardner (@rhynorater):
I wanted to try to corner you with that, but somehow you weaseled your way out of it.

Joel Margolis (teknogeek):
So, yeah, let's just take it back for a second. So like the main thing is that I Personally don't think that many programs really have much to gain by lying to a hacker

Justin Gardner (@rhynorater):
Sure.

Joel Margolis (teknogeek):
Especially like it's a bug bounty program Like

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
what is your point if you're lying to the main like thing that matters to you, right? Like you might as well just not have a program That's my personal take. I don't know if that's how most programs are I sure hope that they don't lie to the hackers, but you never know

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
Okay Separate from that is how do you not get your time wasted? I think that is really, really difficult.

Justin Gardner (@rhynorater):
Wow, I forgot

Joel Margolis (teknogeek):
Ha!

Justin Gardner (@rhynorater):
I asked that question. Joel's got like a, like, okay, all right, I see it. You're on it today.

Joel Margolis (teknogeek):
Yeah, I think like, so this risk is always there, right? Like if you take internal dupe, if you just take the word internal out of it and leave dupe, like that's always been a risk,

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
right? The risk has always been there, that a researcher could have reported it and you find it tomorrow and you dupe and your time is wasted. Maybe you spent two weeks and you reported it, you said, I'll report it tomorrow and you get duped, okay?

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
That risk is always there. The fact that it's found by not a security researcher, but maybe a security engineer instead, is very little change in that situation. Now, I think that a lot of programs could do a better job at documenting that and integrating it into their program. And

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
what do I mean by that? I mean, if you have

Justin Gardner (@rhynorater):
Ahem.

Joel Margolis (teknogeek):
a security vulnerability on a public facing endpoint, you could create a ticket on your bug bounty program. an internal like, you know, close it as informative or whatever. Like

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
I don't, there are ways to handle that, but, or import it. And if you need to reference it from like a public report that later comes in, you have an actual report ID that you can

Justin Gardner (@rhynorater):
with

Joel Margolis (teknogeek):
reference

Justin Gardner (@rhynorater):
a timestamp

Joel Margolis (teknogeek):
it to.

Justin Gardner (@rhynorater):
and that sort of thing, yeah.

Joel Margolis (teknogeek):
Yeah. The other thing I think that you could do that's probably honestly easier is just be transparent. And this I think goes a long way in a lot of different areas, but I'll talk about it right now, which is that you should be defaulting to transparency with the researcher. Number

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
one, oftentimes they're bound by an NDA. Number two, they're bound by your program terms. Number three,

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
they're bound by a privacy policy on HackerOne. Like, there are a lot of aspects where you can have some level of comfort with the researcher and just be transparent with them about what's going on. And that applies in a lot of different aspects. For this aspect, that applies in that if there's an internal dupe. take a screenshot of the internal ticket and

Justin Gardner (@rhynorater):
Right.

Joel Margolis (teknogeek):
send it to them. You don't have to include all the information. You could redact stuff, but just

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
show them that it's real. Show them that it's a thing and that they missed it by three weeks or whatever. And I also think to that point, as you mentioned, what happens if it opened for a year or whatever? Programs should be taking accountability for that.

Justin Gardner (@rhynorater):
Mmm, yeah,

Joel Margolis (teknogeek):
I don't

Justin Gardner (@rhynorater):
I totally

Joel Margolis (teknogeek):
think it's

Justin Gardner (@rhynorater):
agree.

Joel Margolis (teknogeek):
acceptable if you have a high or critical, especially severity vulnerability that is open for a long period of time. That means your security team is not doing their job. They should be getting this fixed. This should be a priority for them. It should be a priority for the engineers. I understand that there's lots of work culture and all sorts of other things that could play into that relationship between security and engineering. It does not matter. If you're going to have a bug bounty program, that means you have to take security seriously.

Justin Gardner (@rhynorater):
Preach.

Joel Margolis (teknogeek):
to fix your stuff in a reasonable period of time, or you have to explain that to your researchers and say, sorry, I know this has been open for a year, but it's still not fixed yet. And

Justin Gardner (@rhynorater):
Mmm.

Joel Margolis (teknogeek):
then you have to be willing to own that and answer as to why not, right?

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
And I think that's just a reality that a lot of programs don't want to accept, is that sometimes stuff that should be high priority isn't prioritized.

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
And... You know, that sucks. So yeah, I agree with you and I'll also take like a program like advocate side here, which is that like, internal dupes are real, they do happen. However, programs can be a lot better about being transparent. They can provide more information about the dupe itself, show that it's, you know, open, that it's being worked on, whatever. And if stuff is open for too long, you have to take security seriously and realize that A real attacker would go and exploit that if you had that open for a year. And there's no excuse for it to not be fixed.

Justin Gardner (@rhynorater):
Whew, amen brother.

Joel Margolis (teknogeek):
I hope that

Justin Gardner (@rhynorater):
The work

Joel Margolis (teknogeek):
answered

Justin Gardner (@rhynorater):
of the

Joel Margolis (teknogeek):
things.

Justin Gardner (@rhynorater):
Lord is being done here. Dude, that was fire, dude. Good shit, okay. Was supposed to disagree with you and debate that at all, but to be honest, it was pretty on fleek. And I really appreciated the point about them taking responsibility for, and I think that's a little bit what we want with the transparency, right? Like if we can get a ticket showing, hey, you missed it by two days. then that's like, well, drat. I still hate that for me, but it's not the end of the world. But if I see a ticket that says that this problem has been around for six months or something, then I'm gonna be a little bit salty. When they just say, hey, sorry, internal dupe, boop, then that makes me feel a little bit like they're trying to hide something and that this bug's been around for a while and this program is just a... you know, pilot,

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
you know, not

Joel Margolis (teknogeek):
And like,

Justin Gardner (@rhynorater):
goodness.

Joel Margolis (teknogeek):
you just have to remember that like every company is different. The amount

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
of resources that they have both on the engineering side and the security side are different per company. And so there's this thing called SLAs. I'm not going to explain it. I feel like you can look up what an SLA is, a

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
service level agreement. Okay. It basically just means you have this much time to fix this thing or deal with this thing. Right.

Justin Gardner (@rhynorater):
in

Joel Margolis (teknogeek):
That's like time to first response, all that kind

Justin Gardner (@rhynorater):
right

Joel Margolis (teknogeek):
of stuff. Those are SLAs. Okay. Every company has different SLAs for tickets and especially security tickets. Some don't even have SLAs clearly defined for security tickets. And so if something's been open

Justin Gardner (@rhynorater):
oof

Joel Margolis (teknogeek):
for six months, that could be the normal for the company, okay? And if that's the normal, there's really not

Justin Gardner (@rhynorater):
That's

Joel Margolis (teknogeek):
much you can

Justin Gardner (@rhynorater):
not

Joel Margolis (teknogeek):
do as

Justin Gardner (@rhynorater):
good.

Joel Margolis (teknogeek):
a researcher about that. No, it's not good, but you can't be like, ah, fix this faster because... Do you own that company? Are you a C

Justin Gardner (@rhynorater):
Yeah,

Joel Margolis (teknogeek):
level on their,

Justin Gardner (@rhynorater):
no,

Joel Margolis (teknogeek):
like,

Justin Gardner (@rhynorater):
that's true. Yeah.

Joel Margolis (teknogeek):
you know, the secure, one, the security team only has so much leverage and two, like you're asking for large structural changes to how stuff gets done. And yeah, it would be correct and better and awesome to see it get fixed faster. But if that's not how they prioritize security and engineering, then that's not how they prioritize security and engineering. That's,

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
that is what it is.

Justin Gardner (@rhynorater):
And I feel like some of this is also a little bit on the platforms because like you, whenever you. As a platform, when you bring somebody on for a bug bounty program, you have two customers. You have the company and you have the hacker, right? And if you're gonna bring on a client, right, a customer, a company, and then they're gonna provide really poor service to your other client, that's not good. So I think there needs to be some auditing in place, and I'm sure there is at some levels in some organizations, but I think there really needs to be some sort of standard SLA assurance or something like that where hacker one or bug crowd or integrity says Yes bugs Even if they don't always hit it They should be shooting for bugs to be fixed in a certain amount of time, right? And you sort of see that on the program policies, but those numbers are very You know, I can't come up with the word and the Japanese word just keeps on coming into my head But they're very like not specific and not stuck to You know,

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
and so I would like to see a little bit more adherence around that because that's very important to the hackers

Joel Margolis (teknogeek):
Yeah, I'd say like around that, it's very difficult because

Justin Gardner (@rhynorater):
it fluctuates too.

Joel Margolis (teknogeek):
yeah, I mean, the reality is that HackerOne is a vendor

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
for a company, right? From a company perspective, HackerOne is a vendor. You're paying HackerOne for their product,

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
hackerone.com or you know, whatever, generic, but

Justin Gardner (@rhynorater):
or bug crowd or integrity,

Joel Margolis (teknogeek):
we're

Justin Gardner (@rhynorater):
yeah.

Joel Margolis (teknogeek):
just gonna use HackerOne in place

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
here, right? you're paying to use that platform. And in return, you get access to the inbox and maybe you get access to triage services.

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
And the reality is that they, you're not paying them for advice on how to run your security program.

Justin Gardner (@rhynorater):
Okay, that's a problem though, I think, if that is the case. Is that truly the case?

Joel Margolis (teknogeek):
I mean, that's not what you're paying for them. You know, you're not paying them to tell you how to run your, like, they can give you advice, but it would be unsolicited. Right. And so that is like,

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
I think it would be better if they were able to like meaningfully give advice to companies. But I think it's not really their place to be doing that because that's not what they're being paid for. And so not only is it an overstep of their services, but it's like, not really their concern, like their concern is to provide an excellent bug bounty platform that helps the company communicate with researchers and receive vulnerabilities and

Justin Gardner (@rhynorater):
Yeah,

Joel Margolis (teknogeek):
all that

Justin Gardner (@rhynorater):
but

Joel Margolis (teknogeek):
kind of stuff.

Justin Gardner (@rhynorater):
you can't provide an excellent bug bounty platform without it being a two-way street, right? So if you're not ensuring that the program is providing a quality experience to the hackers, then you can't, then the platform is going down,

Joel Margolis (teknogeek):
Sure,

Justin Gardner (@rhynorater):
right? You

Joel Margolis (teknogeek):
but

Justin Gardner (@rhynorater):
see what

Joel Margolis (teknogeek):
again,

Justin Gardner (@rhynorater):
I'm saying?

Joel Margolis (teknogeek):
like what is HackerOne gonna do? Say like, oh, in order to sign this contract, you have to provide us that, you know, you're gonna adjust your security SLAs and do all the, like these things would

Justin Gardner (@rhynorater):
Yeah,

Joel Margolis (teknogeek):
be blockers that,

Justin Gardner (@rhynorater):
that'd be great.

Joel Margolis (teknogeek):
okay, well, I mean, they would just lose business,

Justin Gardner (@rhynorater):
That's

Joel Margolis (teknogeek):
right?

Justin Gardner (@rhynorater):
true. And

Joel Margolis (teknogeek):
That's

Justin Gardner (@rhynorater):
it's a

Joel Margolis (teknogeek):
the

Justin Gardner (@rhynorater):
company,

Joel Margolis (teknogeek):
reality is,

Justin Gardner (@rhynorater):
you know, yeah.

Joel Margolis (teknogeek):
yeah, the company will be like, okay, we're just gonna go to Bugcrowd who doesn't require that,

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
right? Or like, whatever, right? Like the reality is that I think that would be too much of a blocker. when the goal is to just get companies to adopt security more by having

Justin Gardner (@rhynorater):
Hmm.

Joel Margolis (teknogeek):
a bug binding program and having a way to receive vulnerability reports and to make it a requirement that they have to like vamp up and buff their

Justin Gardner (@rhynorater):
Hmm.

Joel Margolis (teknogeek):
whole security program in order to receive reports. That's really difficult. And that's a really hard sell.

Justin Gardner (@rhynorater):
Yeah, I don't know, man. I would like to see that sell. And maybe this is a, maybe something that HackerOne or Bug Crowder Integrity could run with and say like, these are HackerOne vetted programs or something like that where, you know, and they have some sort of extra symbol, you know, sort of like they have for some of the other things that I'm not even sure I can mention exist in HackerOne, but

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
like

Joel Margolis (teknogeek):
I mean,

Justin Gardner (@rhynorater):
for the private

Joel Margolis (teknogeek):
I

Justin Gardner (@rhynorater):
programs,

Joel Margolis (teknogeek):
mentioned this,

Justin Gardner (@rhynorater):
you know, yeah.

Joel Margolis (teknogeek):
I actually had a discussion with them, not related to work, from like a hacker side,

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
I was asked as a hacker to go

Justin Gardner (@rhynorater):
Whoa,

Joel Margolis (teknogeek):
speak

Justin Gardner (@rhynorater):
whoa,

Joel Margolis (teknogeek):
with

Justin Gardner (@rhynorater):
Joel. You're

Joel Margolis (teknogeek):
the

Justin Gardner (@rhynorater):
not

Joel Margolis (teknogeek):
H1

Justin Gardner (@rhynorater):
a hacker

Joel Margolis (teknogeek):
sales

Justin Gardner (@rhynorater):
today.

Joel Margolis (teknogeek):
team.

Justin Gardner (@rhynorater):
You're, I'm sorry, continue, continue.

Joel Margolis (teknogeek):
But like one of the things that I suggested was like, if you have some knowledge about the tech stack that's being used, why not provide security recommendations for that tech stack, right? If

Justin Gardner (@rhynorater):
Mmm.

Joel Margolis (teknogeek):
you know about stuff. Or you know about trends, like that's an easy win for your customer to like give them extra value for the product. And maybe you

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
later turn that into an extra paid feature. I don't know,

Justin Gardner (@rhynorater):
It's true.

Joel Margolis (teknogeek):
I don't care. But like, I think that there are certainly things that they could do there. That's not quite going as far as saying, here's what you should have for your SLAs and all that kind of stuff. Cause I

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
think that is, I mean.

Justin Gardner (@rhynorater):
That's a very intense decision as well, I acknowledge. And in order to hit those SLA's, you may even have to bring on additional developers, right? Which is a

Joel Margolis (teknogeek):
Right. Yeah,

Justin Gardner (@rhynorater):
six

Joel Margolis (teknogeek):
there's so

Justin Gardner (@rhynorater):
figure

Joel Margolis (teknogeek):
many more...

Justin Gardner (@rhynorater):
a year at least

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
decision, you know? And so there is a lot of nuance to that, and I guess I can argue for the ideal from this side as the hacker, but, I just feel like a book bounty. program is a more mature security decision, you know, and a VDP, everyone should have a VDP. You can be throwing VDPs out left and right. But I think about

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
bounty

Joel Margolis (teknogeek):
I mean,

Justin Gardner (@rhynorater):
program is a little bit,

Joel Margolis (teknogeek):
yeah.

Justin Gardner (@rhynorater):
is a little bit more of a mature security program decision. And by that point, you should have a grip on this.

Joel Margolis (teknogeek):
Yes, I totally agree. I think hacker one would, uh... enthusiastically disagree

Justin Gardner (@rhynorater):
Hahaha

Joel Margolis (teknogeek):
with you. Just based on the fact that it's their, their whole business to

Justin Gardner (@rhynorater):
Oh

Joel Margolis (teknogeek):
create and start more programs,

Justin Gardner (@rhynorater):
man.

Joel Margolis (teknogeek):
paying

Justin Gardner (@rhynorater):
Well

Joel Margolis (teknogeek):
programs,

Justin Gardner (@rhynorater):
that's why

Joel Margolis (teknogeek):
but

Justin Gardner (@rhynorater):
we

Joel Margolis (teknogeek):
yeah.

Justin Gardner (@rhynorater):
can say whatever the heck we want on critical thinking.

Joel Margolis (teknogeek):
Yes.

Justin Gardner (@rhynorater):
That's good. Alright,

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
so alright. I have CVSS as the next topic. Little bit of a loaded topic. What do you think? Skip it and move on to budget or do we want to... Would

Joel Margolis (teknogeek):
Sure,

Justin Gardner (@rhynorater):
you like

Joel Margolis (teknogeek):
I mean

Justin Gardner (@rhynorater):
me to yell at you

Joel Margolis (teknogeek):
we

Justin Gardner (@rhynorater):
a

Joel Margolis (teknogeek):
could

Justin Gardner (@rhynorater):
little

Joel Margolis (teknogeek):
cover

Justin Gardner (@rhynorater):
bit

Joel Margolis (teknogeek):
it

Justin Gardner (@rhynorater):
here?

Joel Margolis (teknogeek):
directly.

Justin Gardner (@rhynorater):
Alright.

Joel Margolis (teknogeek):
I mean sure, like here's my take. Maybe I'll

Justin Gardner (@rhynorater):
No, no,

Joel Margolis (teknogeek):
give you my take first. Okay.

Justin Gardner (@rhynorater):
no

Joel Margolis (teknogeek):
I don't think CVSS is very useful. That's it. Like

Justin Gardner (@rhynorater):
Okay.

Joel Margolis (teknogeek):
it is a factor that you can consider. I don't think it should be the only factor you consider.

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
And I don't think it's a perfect system. I don't think any system like this, whether you wanna be VISS, whether you wanna

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
be CVSS, whether you wanna be ABCD,

Justin Gardner (@rhynorater):
Please

Joel Margolis (teknogeek):
I don't

Justin Gardner (@rhynorater):
don't

Joel Margolis (teknogeek):
give

Justin Gardner (@rhynorater):
eat, okay good,

Joel Margolis (teknogeek):
a

Justin Gardner (@rhynorater):
yeah.

Joel Margolis (teknogeek):
flying. You know,

Justin Gardner (@rhynorater):
I,

Joel Margolis (teknogeek):
I don't care. Like

Justin Gardner (@rhynorater):
okay.

Joel Margolis (teknogeek):
fill in the blank. Right. Like. All of these systems are trying to solve a problem that is incredibly difficult to solve, which is

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
that you're trying to take a case by case scenario that is truly impact dependent and genera, fire it into a set of factors that you can apply globally, and that just doesn't really work. Um, I, we've had some really interesting discussions, me and you, uh,

Justin Gardner (@rhynorater):
Mm, yeah.

Joel Margolis (teknogeek):
with. programs about this and like from a optimization perspective, every program wants this because it makes payouts easier, it makes payouts more consistent, it makes processing bugs

Justin Gardner (@rhynorater):
streamlines

Joel Margolis (teknogeek):
and handling

Justin Gardner (@rhynorater):
everything.

Joel Margolis (teknogeek):
all that kind of, it streamlines everything. Yeah, okay, that's great. However, you can't, you can't do that. Like you can't just say like, oh, if it checks these boxes, then it is this bounty or it is this bad because some cases, yes, you can do that. many cases you cannot. Yeah.

Justin Gardner (@rhynorater):
Okay, so lots to unpack there, lots of coming up with a problem and not coming up with a solution. And I'm actually gonna take a little bit of a controversial position here that I am slightly, slightly truly believing, which is that CVSS is like, so the programs that really stick to CVSS, it works well. You know, like if you look at Shopify and you look at PayPal and some of the other ones that really stick to it like glue, I think that rarely are the, and maybe it's just because those programs have really good bounties, which is also another thing, right? Those programs pay very high bounties. So, you know, you could get your bug rated at a medium 6.8 and still be walking out there with, you know, well into the four figures, you know, range. Yeah, I think there doesn't necessarily have to be a system that makes it all streamlined and stuff, but having some sort of baseline for an attacker to know, and maybe this is just a piece in your policy where it's like, hey, we really prioritize user data, and if you can get access to user data, we are very, very interested in that, or we don't really care as much about user data as we care about you getting access to our source code, and if you can get access to our source code, very bad. That sort of direction is really helpful for me in understanding the threat model of a company.

Joel Margolis (teknogeek):
Yeah, no, I think that's a great point. Like

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
again, generally I air towards trying to pay, I think programs should be paying by impact.

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
Like I think that impact is really what matters. It's not about like, I got an XSS, that's great. What can you do with it? Like, can you steal my data? Can you take over my account? Can you... only pop an alert, like what is the actual impact of this? Because if it's nothing, then why am I paying, you know, a flat bounty for this? Or why am I paying what I paid for the last XSS that could do an account takeover on this XSS, which can do nothing, right? And I think that type of information is really important. And what you said about like... if you really stick to CVSS, then it works well. I think it works well in the sense that you have a consistent program, right? So what that means is that one, your process is streamlined and two, your process is consistent. So an XSS is gonna grade the same, is gonna pay the same all the time. And if that's how you wanna handle your security program and you say, we've got this much money for bug bounty and we're just gonna, you know, we're gonna pay 5K for every XSS, we're gonna pay 2K for every sub-domain takeover, that's fine, but... You have to like budget and account for that and just be willing to, regardless of the impact, you have to be willing to press that button and pay the bounty

Justin Gardner (@rhynorater):
Hmm.

Joel Margolis (teknogeek):
at the amount. Right.

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
And a lot of programs, I think, don't look at it that way,

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
which is fair. Like I don't look at it that way. We don't use CVSS and I don't think... CVSS is a great solution because

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
of what I said, like there's so much nuance in impact. And so

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
what you pointed out about like, what does the program care about? This is something that I see almost no programs do well. And like this is something that I think should be addressed from a probably from a platform perspective as well. But there needs to be a better way for programs to explain to researchers what their security threat model looks like. And What that means is as a security team, when you report a bug to me, how do I think about it? What are the things that I'm looking for? How am I grading the impact,

Justin Gardner (@rhynorater):
Amen.

Joel Margolis (teknogeek):
right? And so there's not a great way to communicate that from the program side to the researcher side.

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
And especially for programs that don't use CVSS, there's like... Basically zero guidance, right? And

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
so those are the programs where you submit something and you're like, this is a crit for sure and then they regrade it to a

Justin Gardner (@rhynorater):
below.

Joel Margolis (teknogeek):
medium

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
and like, you know, you're like what just happened, right and oftentimes it may be one of you know, the early reports to that program that you've you know, you're still trying to feel it out and that takes literal like multiple reports to start to feel out manually what does the program care about, how do they respond to this type of stuff, like do they care about this, do they not care about this? Like understanding that stuff in a trial and error method really sucks from a researcher perspective.

Justin Gardner (@rhynorater):
Hmm.

Joel Margolis (teknogeek):
And so I think programs could definitely do a better job in saying. we care about user data, specifically even maybe we care about these user data fields. Now that second question right there, you'll have programs who will immediately say no to that just because of legal and comms. Like they don't want to point out explicitly

Justin Gardner (@rhynorater):
They don't want to put

Joel Margolis (teknogeek):
these are

Justin Gardner (@rhynorater):
targets

Joel Margolis (teknogeek):
sensitive. Yeah,

Justin Gardner (@rhynorater):
on specific

Joel Margolis (teknogeek):
exactly.

Justin Gardner (@rhynorater):
stuff. Yeah.

Joel Margolis (teknogeek):
Right. And so like, I get that. But like, that's fine. Like you don't have to go that far. Just say like, we care about user data and we would care if you can do, you know, X, Y, and Z. The other side that I see, by the way, is way too extreme in terms of examples. Well, where, like I've seen a crypto program, for example, where they will say, we consider it a crit if, and then they have four bullet points with very specific scenarios laid out of detailed, oh, you can execute x, y, and z through a specific, I'm like,

Justin Gardner (@rhynorater):
Right. This function

Joel Margolis (teknogeek):
what?

Justin Gardner (@rhynorater):
never returns a negative number, then we're

Joel Margolis (teknogeek):
Yeah, like.

Justin Gardner (@rhynorater):
screwed, you know?

Joel Margolis (teknogeek):
I'm just gonna go test this one specific use case that I know your security team already knows isn't vulnerable. You know what I mean? That's not helpful either.

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
So I think being more generic about the things that you care about, not the specific scenarios that you care about,

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
would definitely help.

Justin Gardner (@rhynorater):
For sure.

Joel Margolis (teknogeek):
And then somehow figuring out, I don't have a great solution for this either, but somehow figuring out some way to explain the way that your security team is thinking about bugs when they come in. whether

Justin Gardner (@rhynorater):
Hmm. Well,

Joel Margolis (teknogeek):
that

Justin Gardner (@rhynorater):
danger.

Joel Margolis (teknogeek):
be, you know, is it on the main domain? Is, does it have access to cookies?

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
Can it talk with the main domain? Are there ever cookies scoped to the, like some of these things you can figure out as a researcher, some of these things the program needs to provide.

Justin Gardner (@rhynorater):
Dude, like, that was great. And I think we need to start a critical thinking bug bounty program consultancy

Joel Margolis (teknogeek):
Hahaha

Justin Gardner (@rhynorater):
based off of this conversation, because there's a lot of high quality stuff in there. And I think a lot of lessons you've learned as being on the program side and on the hacker side in extreme ways on both, right? Managing some of the biggest bug bounty programs and also, or contributing to some of the biggest bug bounty programs and also hacking on, you know, the live hacking events in the, you know. cream of the crop scene there. So that's really cool. And I think CVSS definitely has its flaws, but for me that consistency that you mentioned, that's kind of what I wanna see. Because if I can see the consistency, if I know what the system is, then I can work within the system, right?

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
That's our whole thing as hackers, is working within the system and making our goals get accomplished, right? And we talked about this a little bit with Inti, but like... the ethical boundaries of sort of gaming the vulnerability, you know, rating system. But if at the end of the day, it truly, if the securities team's priorities are truly aligned with their vulnerability rating system, then it should be a dub for all players.

Joel Margolis (teknogeek):
Yeah, and like, I'm definitely winging this take right here, which

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
is that I think CVSS isn't the worst place to like start. And what I mean by that is like, calculate the CVSS, and then if that doesn't line up with your security team's threat model, try to figure out why.

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
And those are probably your mitigating factors,

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
right? And what I mean by that is like, okay, if something is a crit, but it's actually a medium because blank. Well, that's why.

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
CVSS can't really account for that easily. Yeah, maybe it's like attack complexity high or so, but like, it's not, right? Like, again, like if you have an XSS that can do nothing versus an XSS that can do an account takeover, those are two very, very different things. And

Justin Gardner (@rhynorater):
Dairy.

Joel Margolis (teknogeek):
attack complexity or privileges required or whatever, like those don't really account for what's probably causing that change in severity, right? And

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
so I think, be again, being transparent with the researcher, describing what has changed that has caused it to not be a critical, but to be actually a medium or whatever, is really, really important for the program perspective and to help build those relationships that lead to fruitful security research from your hacker piece, right?

Justin Gardner (@rhynorater):
Yeah. Good stuff, man. We are an hour and one minute in. I've had a little bit of a draining day, so that steam that was coming out earlier is...

Joel Margolis (teknogeek):
I'm still going it just

Justin Gardner (@rhynorater):
Yeah,

Joel Margolis (teknogeek):
it just read

Justin Gardner (@rhynorater):
you

Joel Margolis (teknogeek):
just

Justin Gardner (@rhynorater):
still

Joel Margolis (teknogeek):
read

Justin Gardner (@rhynorater):
got

Joel Margolis (teknogeek):
the

Justin Gardner (@rhynorater):
it.

Joel Margolis (teknogeek):
math

Justin Gardner (@rhynorater):
All

Joel Margolis (teknogeek):
you wrote them

Justin Gardner (@rhynorater):
right,

Joel Margolis (teknogeek):
and

Justin Gardner (@rhynorater):
yeah, all right. Well, okay, I'll just read them as I wrote them. Let me just hit you with this one, big boy.

Joel Margolis (teknogeek):
Alright.

Justin Gardner (@rhynorater):
regarding bounty payments and budgets, it's not your money. So why are you so freaking stingy with it?

Joel Margolis (teknogeek):
Yeah, okay, so as I mentioned, bounties or sorry, budgets are very complex topics. Okay. I, let's see how generically I can talk about this. Essentially, you have to imagine a budget for a company as a giant pool of money that starts from the very, very top of that company and every single layer of management you go down, you chop another piece off of it, okay? And

Justin Gardner (@rhynorater):
Sorry,

Joel Margolis (teknogeek):
so.

Justin Gardner (@rhynorater):
as soon as he said giant pool of money, I just like, the

Joel Margolis (teknogeek):
Oh,

Justin Gardner (@rhynorater):
eyes went big.

Joel Margolis (teknogeek):
no.

Justin Gardner (@rhynorater):
Like I'm like, yeah, that's what I want.

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
No.

Joel Margolis (teknogeek):
right. Justin wants the entire,

Justin Gardner (@rhynorater):
Right.

Joel Margolis (teknogeek):
he'll just take the entire

Justin Gardner (@rhynorater):
That

Joel Margolis (teknogeek):
budget

Justin Gardner (@rhynorater):
budget,

Joel Margolis (teknogeek):
for the whole

Justin Gardner (@rhynorater):
I'll

Joel Margolis (teknogeek):
company

Justin Gardner (@rhynorater):
take that.

Joel Margolis (teknogeek):
for the year.

Justin Gardner (@rhynorater):
No,

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
okay, sorry, go ahead.

Joel Margolis (teknogeek):
And so the reality is that like, money is complicated. And

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
when you make a payment, like I said, out of a Bug Bounty program, that comes out of the Bug Bounty budget pool, which is part of the greater security team budget.

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
which is part of the probably engineering team budget, which is part of the yearly budget for the entire company. And when anything starts to shift, whether you go over or under that, then lots of other factors come involved, right? It's like, oh, there's extra money here, let's take that and spend it on a security tool. Or let's... Oh, you don't you didn't use all that money. All right Well that other team is going to get more money because they needed it last year And so there's like all sorts of like weird Meta aspects to paying a bounty and to where that money's coming from that Yes, it's not your money, but it kind of is your money because it's your team's money.

Justin Gardner (@rhynorater):
Mmm.

Joel Margolis (teknogeek):
It's you know, it's the money that you have for next year or it's the tooling that you're planning to buy in six months. Or like,

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
there are a lot of other aspects where it kind of is your money. It's your money in the sense that you're, you have like ownership over it, but it's not your money, right?

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
And so I think

Justin Gardner (@rhynorater):
You're

Joel Margolis (teknogeek):
that's

Justin Gardner (@rhynorater):
a steward

Joel Margolis (teknogeek):
where,

Justin Gardner (@rhynorater):
of that money. You're supposed

Joel Margolis (teknogeek):
yeah.

Justin Gardner (@rhynorater):
to be using this money for the greater purpose of the organization that you're working with. Makes sense.

Joel Margolis (teknogeek):
Yeah, yeah. And I think very few companies are able to keep up with the type of pace that gets set by live hacking event programs

Justin Gardner (@rhynorater):
Right,

Joel Margolis (teknogeek):
specifically.

Justin Gardner (@rhynorater):
yeah.

Joel Margolis (teknogeek):
I think it really sets an unfair bar in terms of what people should be paying and what's considered a reasonable bounty.

Justin Gardner (@rhynorater):
Those are just

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
blank checks sometimes to, you know, to the security team.

Joel Margolis (teknogeek):
yeah.

Justin Gardner (@rhynorater):
So, yeah.

Joel Margolis (teknogeek):
I mean, it's like, you know, you see a program that pays multiple millions of dollars at a single event. That is many times the full security budget for plenty of companies.

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
And like the reality is that not every company can do that. Not every company can keep up with that. And so that is also part of where the stinginess comes in, which is like, is this something that we actually care about? If I'm looking at my budget and what I'm supposed to be paying for this, is this something that's worth paying? Say I have $50,000 in my budget. Is this worth paying 20% for a 10K crit

Justin Gardner (@rhynorater):
Right.

Joel Margolis (teknogeek):
of my whole budget for who knows how like, you know, what if it's February and you got 50K in your budget and you just got a 10K

Justin Gardner (@rhynorater):
If you got

Joel Margolis (teknogeek):
crit.

Justin Gardner (@rhynorater):
50k in your budget, I don't know, like...

Joel Margolis (teknogeek):
And it's like, oh, okay, I'm two months in and I just blew 20% of my budget. Like,

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
what do I, you know, what happens if this happens again in two weeks, right? And so I think there's a lot of aspects where programs start to get stingy there are other aspects involved around like, where's the money gonna come from? Yeah, I

Justin Gardner (@rhynorater):
How's the company

Joel Margolis (teknogeek):
mean,

Justin Gardner (@rhynorater):
doing in general?

Joel Margolis (teknogeek):
there's tons of other things, but yeah, a lot of it is just really comes down to budgets are weird and confusing, and getting more money or having extra money is like a weird balance, and every company kinda handles that differently. A lot of times it's not received well,

Justin Gardner (@rhynorater):
Hmm.

Joel Margolis (teknogeek):
in either case. So.

Justin Gardner (@rhynorater):
But on the flip side of that, the more bounties you pay, the more budget you may get allocated the next year.

Joel Margolis (teknogeek):
Yeah, man, I have a really...

Justin Gardner (@rhynorater):
I can really help you out here, you know? Like, I could, I could submit, you could pay him really good and then you'll get more money next year.

Joel Margolis (teknogeek):
This is not gonna be a popular take, but I think

Justin Gardner (@rhynorater):
Okay.

Joel Margolis (teknogeek):
that programs that pay a lot of money are really just messing it all up.

Justin Gardner (@rhynorater):
What?

Joel Margolis (teknogeek):
Like,

Justin Gardner (@rhynorater):
Oh!

Joel Margolis (teknogeek):
here's the thing, here's the thing, okay? As a bug bounty researcher, would I like to get lots of bounties? Absolutely, every day of the week. As a program, would

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
I want to be paying millions of dollars every single year because my security team is not... finding those things and not fixing those things and not detecting those things and not securing from the inside, but putting all this emphasis in money, literally just like spending money because we're paying other researchers to find these things that our team should be proactively trying to fix and find.

Justin Gardner (@rhynorater):
I don't

Joel Margolis (teknogeek):
Right.

Justin Gardner (@rhynorater):
know, man.

Joel Margolis (teknogeek):
Like

Justin Gardner (@rhynorater):
I think, and I know you said it's a hot take or whatever, and it is, oh, for sure it's a hot take. But let's say this, okay? Let's say you spend $500, or, $500, $500,000 on your

Joel Margolis (teknogeek):
That's

Justin Gardner (@rhynorater):
security,

Joel Margolis (teknogeek):
a big difference.

Justin Gardner (@rhynorater):
I know, I'm sorry, $500,000 on your bug bounty reports for this year, okay? Let's say what? average bounty of 2K. So you're resolving 250 vulnerabilities there, roughly, right? If you were to hire an employee, let's say a Silicon Valley engineer, AppSec employee, right? Let's say you can hire, let's be generous, and let's say we can hire two of them for 500,000, which I don't think you can. What are the chances that they will produce 125 valid vulnerabilities each in one year on some of the hard-end, you know, some very hard-end scope. I feel like that's pretty unlikely.

Joel Margolis (teknogeek):
For the math here by the way, 52 weeks, 5 days a week is 250 days a year. So you're asking for a bug every other day.

Justin Gardner (@rhynorater):
Well, no one's working 52 weeks a year, though,

Joel Margolis (teknogeek):
Here's the thing.

Justin Gardner (@rhynorater):
as a Silicon Valley engineer, you know, just to be clear, but,

Joel Margolis (teknogeek):
Sure,

Justin Gardner (@rhynorater):
uh...

Joel Margolis (teknogeek):
sure, but you know, they're also not, they might find two bugs

Justin Gardner (@rhynorater):
Maybe

Joel Margolis (teknogeek):
in a day.

Justin Gardner (@rhynorater):
50, but yeah.

Joel Margolis (teknogeek):
Yeah, whatever. Like I think the reality is that the breakeven is pretty close. But if you, again, like I defer back to the fact that if you have a program that is consistently paying insane amounts of bounties and that number's not going down.

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
That means either you're raising your bounties

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
and finding less or you're not doing your job. That's how I see it. I like,

Justin Gardner (@rhynorater):
Oof.

Joel Margolis (teknogeek):
you're just not like, a

Justin Gardner (@rhynorater):
That

Joel Margolis (teknogeek):
security

Justin Gardner (@rhynorater):
is a take.

Joel Margolis (teknogeek):
team should be paying less bounties, right? Like I don't understand how your engineering team could be outpacing security so badly that you're paying more money. And also I don't see how that's a good thing.

Justin Gardner (@rhynorater):
It's kind

Joel Margolis (teknogeek):
Like

Justin Gardner (@rhynorater):
of based

Joel Margolis (teknogeek):
I...

Justin Gardner (@rhynorater):
though, not gonna lie.

Joel Margolis (teknogeek):
Cough cough

Justin Gardner (@rhynorater):
At the end of the day, from a business perspective, unless you're growing extremely fast and the number of engineers that you're employing to write code that have mistakes in it is increasing very fast, over time, security is a cost, you know? And that

Joel Margolis (teknogeek):
Yes.

Justin Gardner (@rhynorater):
cost ideally should be going down as you understand what mistakes you're company is making over time. Yeah,

Joel Margolis (teknogeek):
Yeah. Or it goes up

Justin Gardner (@rhynorater):
I don't

Joel Margolis (teknogeek):
because

Justin Gardner (@rhynorater):
like

Joel Margolis (teknogeek):
of the

Justin Gardner (@rhynorater):
it,

Joel Margolis (teknogeek):
value.

Justin Gardner (@rhynorater):
but I kind of agree

Joel Margolis (teknogeek):
Like,

Justin Gardner (@rhynorater):
with you. Yeah.

Joel Margolis (teknogeek):
yeah, like I just don't think that like, like I see so many programs that are like boasting like, oh, we paid so much money this year. Yeah, man. That's awesome. But like, why? Like what, what is happening internally? Like what is your security team doing?

Justin Gardner (@rhynorater):
Well, okay,

Joel Margolis (teknogeek):
Because

Justin Gardner (@rhynorater):
let

Joel Margolis (teknogeek):
they

Justin Gardner (@rhynorater):
me div-

Joel Margolis (teknogeek):
should be helping like cut this back so much.

Justin Gardner (@rhynorater):
Let me devil's advocate that though. The devil's advocate of that is that you're resolving actual vulnerabilities and you're finding out these problems. Every single problem that is being reported and paid for is a valid vulnerability and that's the beauty of bug bounty, right? So

Joel Margolis (teknogeek):
Sure.

Justin Gardner (@rhynorater):
if you're paying out more money, it doesn't, I guess you could try to trace it all the way back down to the cost, but at the point you're paying it out, it's already a sunken cost. you've already have that vulnerability on your platform. And at any given point, if you could pay the money to resolve the vulnerability, that's the choice you're making because of risk, right?

Joel Margolis (teknogeek):
Um, yes, but if the engineer finds it, like if your security team finds that and fixes it, and then also simultaneously fixes 10 other bugs, not only have they saved you

Justin Gardner (@rhynorater):
Mmm.

Joel Margolis (teknogeek):
so much more money,

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
but also that's what they're getting paid to do. You

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
know, like on top of paying bounties, you're paying security engineers to run the security team, maintain the program, like do day to day stuff. And also you're paying bounties, right? Like

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
Imagine if you could take the $7 million that you pay in your bounties and spend one and a half on salaries and

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
then keep $6 million to do whatever you like.

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
I don't know. Like it's just a really hard concept for me to understand why when I see companies bragging about the fact that they're paying super large amount of bounties every year. I'm like, yeah, maybe for like a year or two that's. a good thing to see, but after that, you should be paying nowhere even close to that because your security team should be learning from this. They should

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
be making changes. They should be putting stuff in place that's going to like, oh, this happened because XYZ. Let's figure out how to detect that so it doesn't happen again.

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
This is what, from a security team perspective, this is what we do. And when I see companies who are like, oh, we paid $5 million last year. Before that, we paid $2 million. I'm like... What? Like...

Justin Gardner (@rhynorater):
Yeah, yeah. I see it, I see it. I don't love it, but I see it. And

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
it sort

Joel Margolis (teknogeek):
I

Justin Gardner (@rhynorater):
of

Joel Margolis (teknogeek):
don't

Justin Gardner (@rhynorater):
makes

Joel Margolis (teknogeek):
know.

Justin Gardner (@rhynorater):
sense. And I think that is one of the things that's tricky about bookbinding as well, is you've got to study the bugs you're getting. And you really, to be honest, you need at least, you know, maybe, depending on the volume of your program. But a lot of programs need a dedicated asset, a dedicated person to that because it is so much research. And every single time you look at a bug bounty report, you're not seeing all of the layers of research that went into getting that bug bounty report out there. So there's a lot of reverse engineering that needs to happen from the program side. And I was privy

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
to a conversation lately, thankfully somebody came up to me and told me about how one of my reports was dealt with within their internal organization where After I submitted this report, their internal red team is actually weaponizing these reports. They were able to use them to accomplish a couple tasks, a couple objectives that they had for the red team goals. That was really cool to hear. I think that's the correct approach because you know that the internal red team, the internal security team is learning from those reports. we'll make sure those sort of mistakes don't happen again.

Joel Margolis (teknogeek):
Yeah, dude, absolutely. I like when I see programs like not taking learnings away, which is usually pretty evident based

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
on how they're paying stuff and how much they're paying. It's really frustrating because I'm like, dude, like. Your company is like burning money, like

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
literally just like spending money, like hand over fist to pay researchers, to pay the engineers, like all this stuff. And you could save, like, this is an easy justification. Like, oh, we could save a bunch of money if we hire five new people and had them specifically focus on stuff that's reported by the bug running program to like put preventative measures in place. Like easy,

Justin Gardner (@rhynorater):
Yeah,

Joel Margolis (teknogeek):
right?

Justin Gardner (@rhynorater):
you

Joel Margolis (teknogeek):
Like,

Justin Gardner (@rhynorater):
could probably save

Joel Margolis (teknogeek):
yeah.

Justin Gardner (@rhynorater):
the bounties that you're paying, you know, that you're paying,

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
you know, just by doing

Joel Margolis (teknogeek):
exactly.

Justin Gardner (@rhynorater):
that. Yeah,

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
it pays

Joel Margolis (teknogeek):
so

Justin Gardner (@rhynorater):
for itself.

Joel Margolis (teknogeek):
I don't know.

Justin Gardner (@rhynorater):
Okay, all

Joel Margolis (teknogeek):
That's

Justin Gardner (@rhynorater):
right.

Joel Margolis (teknogeek):
my take on bounties and budgets.

Justin Gardner (@rhynorater):
Next question. Why on earth are you not paying me a bounty at Triage?

Joel Margolis (teknogeek):
I don't know man, honestly, fun fact, did you know this? I didn't know this. Hacker one triage is supposed to retest bugs before they're resolved.

Justin Gardner (@rhynorater):
Really? Huh.

Joel Margolis (teknogeek):
Yeah, says

Justin Gardner (@rhynorater):
That's interesting.

Joel Margolis (teknogeek):
that in the HackerOne docs.

Justin Gardner (@rhynorater):
Huh.

Joel Margolis (teknogeek):
Yeah, go ahead. Google HackerOne retesting. First result, it says right here, note for response programs using HackerOne's triage services, the triage team will retest the vulnerabilities to verify the fixes instead of hackers.

Justin Gardner (@rhynorater):
Interesting. Huh.

Joel Margolis (teknogeek):
I'm not going to lie from a both a hacker and a program perspective. I have never seen that happen.

Justin Gardner (@rhynorater):
Wow.

Joel Margolis (teknogeek):
I'm just going to put that out there.

Justin Gardner (@rhynorater):
That's pretty odd, man. I've

Joel Margolis (teknogeek):
Have

Justin Gardner (@rhynorater):
not,

Joel Margolis (teknogeek):
you ever?

Justin Gardner (@rhynorater):
I've

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
not.

Joel Margolis (teknogeek):
no, me neither.

Justin Gardner (@rhynorater):
I've seen triagers escalate my bugs before and that mostly occurs at triage time. But I haven't seen them retest it. Okay, but retesting, hold on. We've got a section, Joel, for retesting. So we'll come to the retesting thing. What I wanna

Joel Margolis (teknogeek):
Is

Justin Gardner (@rhynorater):
know

Joel Margolis (teknogeek):
that not where we're

Justin Gardner (@rhynorater):
is,

Joel Margolis (teknogeek):
at

Justin Gardner (@rhynorater):
no,

Joel Margolis (teknogeek):
right now?

Justin Gardner (@rhynorater):
no. The question is, why are you not issuing a bounty at triage for...

Joel Margolis (teknogeek):
Oh, at triage.

Justin Gardner (@rhynorater):
Yeah, at triage, not at retest, yeah.

Joel Margolis (teknogeek):
Oh, okay. I mean, yeah, so

Justin Gardner (@rhynorater):
Okay, this is-

Joel Margolis (teknogeek):
I

Justin Gardner (@rhynorater):
that's

Joel Margolis (teknogeek):
think...

Justin Gardner (@rhynorater):
a little bit more pointed. Okay, I'm gonna make it less pointed. No, no, I'm gonna leave it. I'm

Joel Margolis (teknogeek):
No,

Justin Gardner (@rhynorater):
gonna-

Joel Margolis (teknogeek):
no,

Justin Gardner (@rhynorater):
I'm

Joel Margolis (teknogeek):
just

Justin Gardner (@rhynorater):
gonna

Joel Margolis (teknogeek):
short,

Justin Gardner (@rhynorater):
leave

Joel Margolis (teknogeek):
short.

Justin Gardner (@rhynorater):
it pointed. I'm gonna

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
leave it pointed.

Joel Margolis (teknogeek):
why not pay at triage?

Justin Gardner (@rhynorater):
Sure.

Joel Margolis (teknogeek):
I mean, I think you probably should pay minimum bounty at triage,

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
at least. The hard part about paying at triage is that oftentimes you don't know what the full impact is until you can do some further investigation. That's where it gets hard to pay a full bounty at triage. However, like I said, I think... should probably be trying to pay minimum bounty at triage. Meaning,

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
you know that this is at the very minimum X severity, medium, high, whatever, low. Okay, pay that bounty. You know you're gonna pay at least that much. Pay some money, show them that you appreciate the report that it's being worked on, give them some crumbs, okay?

Justin Gardner (@rhynorater):
Yeah,

Joel Margolis (teknogeek):
Like figure out the impact

Justin Gardner (@rhynorater):
okay,

Joel Margolis (teknogeek):
and then

Justin Gardner (@rhynorater):
so

Joel Margolis (teknogeek):
pay

Justin Gardner (@rhynorater):
we've

Joel Margolis (teknogeek):
the

Justin Gardner (@rhynorater):
all

Joel Margolis (teknogeek):
rest.

Justin Gardner (@rhynorater):
heard this argument, right? But like why, so I feel like there's two things here, right? There's the internal technical assessment, right, that will tell you exactly what the impact of that bug is, you know, assuming you do a thorough investigation and don't miss anything, right? But then there's also the triage assessment, which is just like, oh no, that's bad, I don't want that happening to my organization. Right, so like, you know, you go to the report, you click on the link, the alert box pops up. We can all acknowledge that normally that's bad. Right, you know,

Joel Margolis (teknogeek):
Sure.

Justin Gardner (@rhynorater):
so I feel like at that point, you triage the bug, pay the lowest bounty. And I mean, I just wonder, like, do you have any insight into why we don't see this more often, because I feel like this is a pretty easy thing to do.

Joel Margolis (teknogeek):
No, I mean honestly this is one of those things that like seems like a no-brainer to me.

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
I don't... I have yet to get a really good understanding why programs don't do that.

Justin Gardner (@rhynorater):
Okay.

Joel Margolis (teknogeek):
I think it really just boils down to the nuance, which is that they don't want to overpay for something accidentally,

Justin Gardner (@rhynorater):
Mm-hmm. Mm.

Joel Margolis (teknogeek):
because if the minimum is so high that it's maybe higher than a maximum for something else. right,

Justin Gardner (@rhynorater):
Sure,

Joel Margolis (teknogeek):
then

Justin Gardner (@rhynorater):
and we

Joel Margolis (teknogeek):
they

Justin Gardner (@rhynorater):
do see that sometimes.

Joel Margolis (teknogeek):
don't want to make that call. And that's fair, but I think like you should be doing your best to try and pay the minimum acceptable bounty that you like know for sure this is going to be at least as much. And you know, I, yeah, I just think program should be doing that.

Justin Gardner (@rhynorater):
Well, you know, that's not the answer I wanted to hear, Joel. I wanted to hear some

Joel Margolis (teknogeek):
I'm sorry.

Justin Gardner (@rhynorater):
reason,

Joel Margolis (teknogeek):
Like

Justin Gardner (@rhynorater):
like, I mean, I...

Joel Margolis (teknogeek):
I, I cannot fathom why a program would, would be like, this is worth at least as much, but we're going to wait

Justin Gardner (@rhynorater):
All right, yeah,

Joel Margolis (teknogeek):
until it's fixed.

Justin Gardner (@rhynorater):
I mean,

Joel Margolis (teknogeek):
I don't know.

Justin Gardner (@rhynorater):
that makes sense. That makes sense.

Joel Margolis (teknogeek):
Like you're gonna pay them one way or another.

Justin Gardner (@rhynorater):
Frustrating.

Joel Margolis (teknogeek):
It's like, I don't know.

Justin Gardner (@rhynorater):
It's frustrating. All right, so let's talk about the retesting. So HackerOne

Joel Margolis (teknogeek):
Sure.

Justin Gardner (@rhynorater):
is supposed to do, apparently if you're

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
using HackerOne retest services, HackerOne is supposed to retest.

Joel Margolis (teknogeek):
No, no, if you have H1 triage,

Justin Gardner (@rhynorater):
Yeah, I mean, if you're

Joel Margolis (teknogeek):
according

Justin Gardner (@rhynorater):
managed

Joel Margolis (teknogeek):
to the

Justin Gardner (@rhynorater):
triage

Joel Margolis (teknogeek):
H1 docs,

Justin Gardner (@rhynorater):
services,

Joel Margolis (teknogeek):
it says

Justin Gardner (@rhynorater):
yeah.

Joel Margolis (teknogeek):
if you're using Hacker One's triage services, the triage team is supposed to retest the vulnerability

Justin Gardner (@rhynorater):
Hmm

Joel Margolis (teknogeek):
to verify the fix instead of hackers. Now, again, I, in my experience in hacking, both as a researcher and as a program operator, I've never seen H1 triage do that. Not to the best of my knowledge.

Justin Gardner (@rhynorater):
Yeah, I think

Joel Margolis (teknogeek):
It's

Justin Gardner (@rhynorater):
they

Joel Margolis (teknogeek):
possible

Justin Gardner (@rhynorater):
need to pull

Joel Margolis (teknogeek):
that it may

Justin Gardner (@rhynorater):
that

Joel Margolis (teknogeek):
have happened.

Justin Gardner (@rhynorater):
out of those dogs.

Joel Margolis (teknogeek):
Everybody that I've talked to has been like, Really?

Justin Gardner (@rhynorater):
That is

Joel Margolis (teknogeek):
So,

Justin Gardner (@rhynorater):
mega workload increase for them, for sure.

Joel Margolis (teknogeek):
yes, yeah. And like TreeRider already has had enough thing, but yeah, I don't, I don't know. I think the whole like, should we pay retesting? The fact that it's a feature there and it's like literally $50, just do it.

Justin Gardner (@rhynorater):
Just pay it, yeah, just pay it, right?

Joel Margolis (teknogeek):
Like, I think if you're getting a $5,000 bounty, me paying you $50 more shouldn't matter to you as a researcher, personally. Like, I'm like, what is the... what is the...

Justin Gardner (@rhynorater):
To be,

Joel Margolis (teknogeek):
you

Justin Gardner (@rhynorater):
yeah,

Joel Margolis (teknogeek):
know, whatever.

Justin Gardner (@rhynorater):
to be honest, the $50 isn't even really, to be honest, the $50 isn't really that motivating in general even. Like, I feel like if you want, especially when programs take so freaking long to resolve the bug. Like, if you resolve the bug in 24 hours and you ask me to retest, I'm gonna retest for you, cause I appreciate you. You know,

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
like, cause that's good shit, right? And I wanna encourage that. But if it's three months down the line, I got pinged the other day to do a retest of a bug that I reported in January. And I said,

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
actually I didn't say anything, because I was like, I'm not even gonna respond to that, because there's no way that I'm about to come back to this report and reread the whole report and go set up the whole thing that I need to do and re-figure out where that bug was and get back in that flow state to be able to actually test

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
this without any

Joel Margolis (teknogeek):
well,

Justin Gardner (@rhynorater):
sort

Joel Margolis (teknogeek):
and I mean,

Justin Gardner (@rhynorater):
of

Joel Margolis (teknogeek):
that's

Justin Gardner (@rhynorater):
incentivization.

Joel Margolis (teknogeek):
why like random people get invited to perform retests when you don't accept it and

Justin Gardner (@rhynorater):
That's

Joel Margolis (teknogeek):
like

Justin Gardner (@rhynorater):
bullshit

Joel Margolis (teknogeek):
That

Justin Gardner (@rhynorater):
by the way, that is absolutely

Joel Margolis (teknogeek):
yeah

Justin Gardner (@rhynorater):
not cool.

Joel Margolis (teknogeek):
like that definitely feels like a little bit of a weird thing in terms of like my bug being

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
leaked and whatever but nonetheless, um, you know, I think Paid retesting should be the normal. I think even if you don't pay someone to retest, doing that retest form should be part of the normal flow. Because from a program perspective, the hacker is prompted to say whether or not it was fixed and whether or not they found any bypasses. And that's really, I think, why programs ask researchers to do retesting in the first place,

Justin Gardner (@rhynorater):
Mm-mm.

Joel Margolis (teknogeek):
is because they don't want to say, okay, this is fixed. And then tomorrow get a new report with a bypass when the researcher Like it's not, it's not really good faith, right? The reality is that like,

Justin Gardner (@rhynorater):
Ah.

Joel Margolis (teknogeek):
you know, like if I ask you to fix something

Justin Gardner (@rhynorater):
I

Joel Margolis (teknogeek):
and

Justin Gardner (@rhynorater):
don't

Joel Margolis (teknogeek):
you

Justin Gardner (@rhynorater):
know

Joel Margolis (teknogeek):
find

Justin Gardner (@rhynorater):
about

Joel Margolis (teknogeek):
a

Justin Gardner (@rhynorater):
that.

Joel Margolis (teknogeek):
bypass, and this is, I think is like, the program should probably be paying like a bonus

Justin Gardner (@rhynorater):
Mmm,

Joel Margolis (teknogeek):
if you find a bypass

Justin Gardner (@rhynorater):
I could

Joel Margolis (teknogeek):
or

Justin Gardner (@rhynorater):
do

Joel Margolis (teknogeek):
something

Justin Gardner (@rhynorater):
that.

Joel Margolis (teknogeek):
like that, right? Like the security team should also be checking it on their half. It shouldn't just be like, oh, the researcher is now gonna be an engineer and verify this on our behalf.

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
Like you should probably check it, then you should have the researcher check it, and then if the researcher finds a bypass, they should get a bonus.

Justin Gardner (@rhynorater):
Hmm. Yeah. I agree. I think bypasses don't necessarily need to have 100% of the bounty paid again, unless it is, I guess, unless some time has passed and unless that Reese that, that component has changed or if the hacker had to be in a different, you know, sort of flow state, you know, it's,

Joel Margolis (teknogeek):
Like the way I view it from a security

Justin Gardner (@rhynorater):
it's

Joel Margolis (teknogeek):
perspective

Justin Gardner (@rhynorater):
tricky.

Joel Margolis (teknogeek):
is that it's the same problem, it's just an incomplete fix. And so asking to get paid.

Justin Gardner (@rhynorater):
but it's still a vuln,

Joel Margolis (teknogeek):
before

Justin Gardner (@rhynorater):
right?

Joel Margolis (teknogeek):
it's

Justin Gardner (@rhynorater):
You

Joel Margolis (teknogeek):
the,

Justin Gardner (@rhynorater):
can't

Joel Margolis (teknogeek):
yeah.

Justin Gardner (@rhynorater):
say that you pay versus off of impact, Joel, and then say that it's the same vuln and it, and, you know, pay for less.

Joel Margolis (teknogeek):
I mean it is the same fault. It's again. It's like

Justin Gardner (@rhynorater):
but it's still a vuln and it's still vulnerable and it's still on your attack surface. This

Joel Margolis (teknogeek):
Sure,

Justin Gardner (@rhynorater):
goes back

Joel Margolis (teknogeek):
but...

Justin Gardner (@rhynorater):
to the same conversation that we had at the beginning of the episode, you know, about Zero Days is like, if the vuln is there and it's vulnerable and you're not actively doing anything about it and I make you do something about it, then we've completed that transaction that Bug Bounty is built upon, no?

Joel Margolis (teknogeek):
Sure, okay, but here's the thing. If it's the same problem, like this is an ongoing issue, this is not like a new attack vector.

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
This is like, ah, your fix is incomplete. Then that doesn't deserve a new bounty, in my opinion. Like if this is the same root cause and something was incorrectly fixed, we're talking about bypasses versus new vulnerabilities here. It's not like oh there was six months and they regressed or

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
they made like a code change that reintroduced this vulnerability

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
we're talking about the same vulnerability that was not properly fixed and You the researcher found a bypass. I'm happy to give you a bonus for that. I'm not gonna pay you the same bounty

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
I'm not gonna pay you 2x because

Justin Gardner (@rhynorater):
Well, I guess it sort of comes

Joel Margolis (teknogeek):
you know

Justin Gardner (@rhynorater):
down to what the con...

Joel Margolis (teknogeek):
Like there's not

Justin Gardner (@rhynorater):
Yeah. You blipped for a

Joel Margolis (teknogeek):
Oh.

Justin Gardner (@rhynorater):
second, are you there? Can you hear me?

Joel Margolis (teknogeek):
There you are.

Justin Gardner (@rhynorater):
All right. I guess it sort of comes down to what we define this sort of researcher program relationship to be. Because in my eyes, as only a bug bounty hunter, right, this is my perspective, my job is to bring you valid volumes on your attack surface so that you can fix them and make it better. and secure your thing, right? I'm not in charge of telling you how to fix it. I'm not in charge of telling you, you know, I'm not in charge of anything else. I just bring you the vuln, right? And if you introduce a fix that it's not a complete fix, then I do my job again and I bring you the vuln, you know? And I see, like, if it's something like, let's say for example, something like, let's say an SSRF, right? And the fix is like blacklisting, you know, specific hosts or whatever, and you bypass it by using like a octal encoded IP address or something like that, right? That is a scenario where I have brought additional piece of knowledge to the table. And to be, just to be clear, I'm devil's advocate getting a bit here, I don't mind the bonus. I think the bonus is sort of fine unless I've put a substantial amount of work into bypassing that. And I think the researcher has to be a little bit transparent about that. But we need to be a little bit more clear, I think, about what the transaction is in Bug Bounty. Because at the end of the day, the vulnerability that you fixed as you perceived it was a vulnerability of this SSRF can attack a specific host in this scenario. and I just brought to you that exact same scenario again, after you had already taken that issue off your to-do list, you're no longer actively aware of this vulnerability, and I just brought you the exact same thing again.

Joel Margolis (teknogeek):
Yeah. So here's the thing. I think a lot of teams will view this as the same problem. And I think rightfully so, because it's going to be all the same attack vector is going to be the same fixed location. It's going to be everything. It's just that the fix is different and it's not that it's a new vulnerability. It's the same vulnerability in a different form. Um, and in the same way that, uh, the same vulnerability on multiple hosts. May or may not get paid. as multiple vulnerabilities, you know, they might pay a bonus because it affects multiple things, but they probably, depending on the program, probably won't be paying you that bug times that many hosts because it's going to be like a single fix or it's going to be like a one single, you know, usually it's like, where's the fix? And from a program perspective, like, let me walk you through it. You submit a bug to me. We fix it. We resolve it, we pay it. Tomorrow, you submit the same exact vulnerability to me except with octal encoding. That really rubs

Justin Gardner (@rhynorater):
Okay.

Joel Margolis (teknogeek):
me the wrong way. Okay, like from a program perspective, like

Justin Gardner (@rhynorater):
So...

Joel Margolis (teknogeek):
that is enough to like really set things off in terms of one, is this even something valid? Like the program could close that as a dupe of your other report and reopen. Like I think like.

Justin Gardner (@rhynorater):
Not if it's already resolved, not if the report is already resolved,

Joel Margolis (teknogeek):
Well, right.

Justin Gardner (@rhynorater):
Joel.

Joel Margolis (teknogeek):
And so this is the thing is like, if, if their pork gets resolved, right. And I think, again, this is why a lot of programs will go back and say, Hey, can you verify the fix and let us know if you find a bypass, because if I say, Hey, can you verify this and you say, yep, looks fixed to me, and then tomorrow you report a bypass to the bone, you were lying to me.

Justin Gardner (@rhynorater):
Well, okay, whoa, whoa. You know, okay, well we do have incentive to lie actually because we get more money,

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
you know? You know?

Joel Margolis (teknogeek):
yes, right? Like the incentive here is that like, oh yeah, that's fixed. Oh, here's a bypass to it. Oh, yeah, that's fixed. Oh, here's another bypass, right? And keep farming that. And I totally see the aspect, like yeah, you gotta make a living. However, this is the same bug, right? This is not like, oh, this is a novel like,

Justin Gardner (@rhynorater):
Okay,

Joel Margolis (teknogeek):
you know.

Justin Gardner (@rhynorater):
but here's the thing, Joel, it's not the same freaking bug because it's not on your, like, why do bugs

Joel Margolis (teknogeek):
Well,

Justin Gardner (@rhynorater):
happen?

Joel Margolis (teknogeek):
and this

Justin Gardner (@rhynorater):
Listen,

Joel Margolis (teknogeek):
is why retesting exists.

Justin Gardner (@rhynorater):
this is why bugs happen in the first place, is some developer somewhere doesn't know enough, and not to say that they need to know everything, it's okay, everyone makes mistakes, it's fine, but at the end of the day, they don't know enough about the technology they're using to implement it properly. Right. That's the TLDR

Joel Margolis (teknogeek):
Okay.

Justin Gardner (@rhynorater):
of the situation. And, and that same situation happens again, when we, we have a sort of a retest to new bugs sort of situation, which is we've identified a vulnerability. They've fixed the vulnerability to the extent of their knowledge. Right. And then at the end, and at the end of the day, that is off the to do list. No one's thinking about it. No one's thinking like, you know, this is a, is a valid, you know, thing that we, maybe we need to go back and like, you know, workshop it. No, no, no And then the attacker highlights that exact same sort of problem again, where it's like you clearly aren't understanding the technology you're working with here to the extent that you need to protect your users, right? And I don't know, it just feels like the... You know what? I think we're going around in circles here. I understand where this

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
is gonna go. I actually

Joel Margolis (teknogeek):
I mean...

Justin Gardner (@rhynorater):
have sort of talked myself into a little bit of a different position than I had at the beginning of this argument, I think, which is that if our job is not actually to advise you on how to fix this vulnerability, and sometimes it is, and sometimes it isn't, right? Then I think retests need to be new, either more strongly compensated than the 50 bucks that HackerOne offers, or they need to be new bucks.

Joel Margolis (teknogeek):
Yeah, like again, I think that the program should be fairly compensating. Right. And if that means like, if you find a unique, like a novel bypass, like I should be at least giving you that $50 for the retest,

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
right? Like

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
the reality is that like, as I mentioned, this should not be like, you should not be the security team. Right? The security team should be verifying this themselves. In some cases, H1 Triage should also be verifying this themselves. Like there should be other layers of verification in place before this gets to the researcher. Not only does it make the security team look bad if the researcher finds a bypass so they're fixed, but it also means that you didn't do your job well.

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
And so, like, I think if it gets that far, like there should be at least some sort of compensation for the time and the effort, but I don't see it as a totally unique new vulnerability that deserves a fresh

Justin Gardner (@rhynorater):
Yeah,

Joel Margolis (teknogeek):
new payout.

Justin Gardner (@rhynorater):
here's an interesting thing is that as a hacker, you can like say with the SSRF scenario, right? Let's say I put in a new host, you know, you hit that host, okay, that whole thing goes to the thing. And I know in the beginning, you know what, I bet this uses octal encoding too, and I bet because there's already a whitelist, they're just gonna fix the whitelist, right? Or whatever, right? Or the, I'm sorry, the blacklist. And then you withhold that information from the original report. Right. That's, that is, that is tricky. That is dishonest and not something that I would

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
do. Right. But I also feel like there's a flip side of this, which is that programs also have this sort of trickiness when it comes to impact assessment, because you can look at your organization and say, okay, wow, this guy found an XSS on this one host that actually affects 1500 hosts. Right. And then pay them out for the specific. you know, the one bug, right? Because they didn't know, right? So there are things that both parties

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
can bring to the table that will portray honesty, that will maintain their, you know, ethical integrity in this exchange.

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
And I think that's a valuable part of building that program researcher relationship. I think this is also the case

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
with retesting, where in a retest, I will do a retest if I'm interested in building a good relationship with the program, and they've sort of paid that same respect forward, right? And so I think it can be a tool just like anything else to build the relationships that Bug Bounty is really built upon.

Joel Margolis (teknogeek):
Yeah. And I think like that point you made is like, I think it's a kind of a good way to think about it. It's like, okay, if, if I have two different ways to exploit this octal and normal encoding, for example,

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
if I submit them both at the same time, almost certainly one would be duped on the other one.

Justin Gardner (@rhynorater):
Absolutely.

Joel Margolis (teknogeek):
If I report one, wait for it to be fixed and report the other one. probably the same thing should happen to some extent. Now, if I report them both in the same report, does that mean I should get extra money? Maybe, maybe not. I think when it comes specifically down to bypassing a fix, that's when extra money should start to come into play. Where it's, we fixed this, no you didn't.

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
If I was a different researcher, this would be a new report.

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
And that I think is where it gets to be fair, where you can start to throw extra money in there. But simply as like, if you know that these two things are both possible, and then only one of them gets fixed, I think you are somewhat obligated to tell them, like, alright, by the way, you know. And I think the program, in good faith, should pay for that,

Justin Gardner (@rhynorater):
Yeah,

Joel Margolis (teknogeek):
to some extent. Yeah, I

Justin Gardner (@rhynorater):
it's

Joel Margolis (teknogeek):
don't

Justin Gardner (@rhynorater):
an

Joel Margolis (teknogeek):
know.

Justin Gardner (@rhynorater):
interesting conversation, man.

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
I think unexpectedly that actually

Joel Margolis (teknogeek):
It's a complicated

Justin Gardner (@rhynorater):
got a

Joel Margolis (teknogeek):
topic.

Justin Gardner (@rhynorater):
little bit, that actually got a little bit tricky, a little

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
bit

Joel Margolis (teknogeek):
for

Justin Gardner (@rhynorater):
heated

Joel Margolis (teknogeek):
sure.

Justin Gardner (@rhynorater):
there. And yeah, I think there's definitely a lot of ethics that comes into Buck Bounty that's under talked about. We should do an episode on that. And I think

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
I wanna say it was Jew Bob's or. Yeah, I think it was Jubeobs that commented on the Twitter post where we were talking about topics. So that would be a good topic. I totally agree. We'll have to think about

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
that.

Joel Margolis (teknogeek):
Yeah, for sure.

Justin Gardner (@rhynorater):
OK, so we are at, like, what? An hour 30? You doing OK? You

Joel Margolis (teknogeek):
I don't

Justin Gardner (@rhynorater):
got a couple

Joel Margolis (teknogeek):
know.

Justin Gardner (@rhynorater):
more minutes?

Joel Margolis (teknogeek):
Yeah, I'm fine.

Justin Gardner (@rhynorater):
OK.

Joel Margolis (teknogeek):
Yeah

Justin Gardner (@rhynorater):
Let me see which one of these are going to hit the hardest, because we're not going to hit all these. Um. I guess we sort of talked about the sort of systemic issue piece when we talked about, you know, in this last conversation as well. From a program perspective, from your experiences, do you think it is dishonest to withhold specific reports for any given period of time?

Joel Margolis (teknogeek):
depends on the severity.

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
If it's a crit. Yeah, probably unethical.

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
Like, I think it depends on two things. Well, I mean one thing really, like severity, which should

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
be based on impact. Like, the reality is that the reason that a bug vending program exists and the reason that you participate in a bug vending program is to try and increase the security of that company in a ethical and profitable way for the researcher. And for the company, right?

Justin Gardner (@rhynorater):
Hmm.

Joel Margolis (teknogeek):
They're saving money,

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
they're, you know, whatever. If you start to hold stuff, I mean, there's a lot of tangential stuff you can get into, right? Like, why are you holding it? Is it for a live hacking event? Is it because, whatever, right?

Justin Gardner (@rhynorater):
Don't at me, bro. What are you doing? Ha ha ha.

Joel Margolis (teknogeek):
Like, it depends. I know a lot of researchers, myself included, will hold bugs, like,

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
you know, for like a week or two. you know, depending,

Justin Gardner (@rhynorater):
The

Joel Margolis (teknogeek):
right?

Justin Gardner (@rhynorater):
programs

Joel Margolis (teknogeek):
I think that's

Justin Gardner (@rhynorater):
know

Joel Margolis (teknogeek):
fine.

Justin Gardner (@rhynorater):
this, just to be clear. You know, we're

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
not they know this. Yeah.

Joel Margolis (teknogeek):
Yeah, like, you know, the thing is that like, researchers start hacking before the submission window gets open and they might have stuff stocked up that they're ready to submit because they want to submit it through the special event program or whatever, like, or they want to use it for like a later bug.

Justin Gardner (@rhynorater):
chain or something, yeah.

Joel Margolis (teknogeek):
I think the ethics... of that are really complicated. From a program perspective, obviously we wanna hear about everything, but the reality is that not everything needs to be heard and

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
reported. If it's an open redirect, that is something that's a lot easier to make that decision for. You know what I mean? If it's something that's like... Mostly a non-issue unless it's used in another circumstance and chained together with something then I think that's fine like I understand that like You know That may not be the best case. However, I think it's still something that the company should eventually hear about

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
Like if it's like an IDOR that or like an endpoint that leaks a lot of IDs that you can then use for an IDOR The company should probably hear about that to some, like

Justin Gardner (@rhynorater):
Really?

Joel Margolis (teknogeek):
in one way or another.

Justin Gardner (@rhynorater):
That's interesting.

Joel Margolis (teknogeek):
Either through your report or either, if you chain it, like, you know, put them both in your report and then reference that report in future reports. Like

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
if you have another IDOR that uses that same type of ID, be like, as I showed in...

Justin Gardner (@rhynorater):
That's

Joel Margolis (teknogeek):
report

Justin Gardner (@rhynorater):
a mature

Joel Margolis (teknogeek):
number,

Justin Gardner (@rhynorater):
way of approaching

Joel Margolis (teknogeek):
whatever.

Justin Gardner (@rhynorater):
it.

Joel Margolis (teknogeek):
I think that there are ways that you can disclose stuff without losing your bug, you know what I mean? You can prove that these things exist and these things are possible, and that makes it a lot easier to make an impact case for your bug.

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
The same way that if you have one bug that shows that cookies are scoped to every subdomain on on a root domain, right? And then later you find another XSS on a subdomain of the root domain. You can use that as a case. I mean, you should use that and say, as you know, look, this happens. There are cookies, like sensitive cookies that have been previously scoped to main domains. And, you know, it's a plausible scenario that it could happen,

Justin Gardner (@rhynorater):
Yeah,

Joel Margolis (teknogeek):
right?

Justin Gardner (@rhynorater):
I feel

Joel Margolis (teknogeek):
I

Justin Gardner (@rhynorater):
like

Joel Margolis (teknogeek):
think

Justin Gardner (@rhynorater):
that

Joel Margolis (teknogeek):
like...

Justin Gardner (@rhynorater):
kind of goes against the POC or GTFO piece of bug bounty though, you know? Like, I mean, I guess you can say that

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
from one, you know, from one or, you know, X many

Joel Margolis (teknogeek):
Yeah, I mean like

Justin Gardner (@rhynorater):
programmers

Joel Margolis (teknogeek):
don't get upset

Justin Gardner (@rhynorater):
perspective.

Joel Margolis (teknogeek):
if the if the program says, okay. Yes, but

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
it's not exploitable

Justin Gardner (@rhynorater):
Well,

Joel Margolis (teknogeek):
right now

Justin Gardner (@rhynorater):
that's a good point is like, you know, if you do report it and they close it as an informative, you know, or whatever,

Joel Margolis (teknogeek):
Well,

Justin Gardner (@rhynorater):
you know.

Joel Margolis (teknogeek):
I don't think they should call it informative. I think that they should, that should be a mitigating factor, but I think it should absolutely be something that's relevant to like, okay, yeah, this was like, six months ago, this would have been exploitable, right?

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
And it's a separate bug than whatever. Like, I think that's reasonable. Like when we look at impact

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
internally, a lot of times we'll ask, was this ever possible? Like, Was,

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
was there ever a case where cookies could be scoped to

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
the sub-domain accidentally or globally, and then you could get an ATO with an XSS? Like,

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
you know, if that's something that's happened and it is possible within the bounds of the infrastructure that's being used, then that's something that should be taken as a serious consideration as a possibility of something that could happen or exist somewhere else, right? Like. The fact that pattern ever was there means that pattern can be there again. And so I think like that's, that's something that should be kept in mind.

Justin Gardner (@rhynorater):
So let.

Joel Margolis (teknogeek):
Again, like don't get a mess, like don't, don't get all frustrated if the, if the program's like, okay, well yes, but this isn't actively exploitable right now. So we're going to put a mitigating factor

Justin Gardner (@rhynorater):
All right,

Joel Margolis (teknogeek):
on that.

Justin Gardner (@rhynorater):
so what you're describing here is. I guess the most communicative or most respectful way to interact with the program in that regard, which is like, let's say we've got the IDOR ID generator, right? We report that to them. They don't accept that as a bug. We're not upset about that, but they know that exists, right? And they're able to use that in their impact assessments. And then the next time we find a bug where we actually utilize this, then we reference it and say, here's the, you know, that's the... That's the report that

Joel Margolis (teknogeek):
Yeah, I mean,

Justin Gardner (@rhynorater):
I would

Joel Margolis (teknogeek):
and

Justin Gardner (@rhynorater):
use.

Joel Margolis (teknogeek):
again, like personally, like as a researcher, I probably wouldn't just report that by itself

Justin Gardner (@rhynorater):
Same. 100%.

Joel Margolis (teknogeek):
unless it was like, if it's just IDs, like, again, very low security impact here.

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
But if I had another IDOR, I might report those two things together.

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
And then again, if I find a third IDOR later down the

Justin Gardner (@rhynorater):
reference

Joel Margolis (teknogeek):
line that

Justin Gardner (@rhynorater):
that

Joel Margolis (teknogeek):
uses

Justin Gardner (@rhynorater):
other

Joel Margolis (teknogeek):
that

Justin Gardner (@rhynorater):
report.

Joel Margolis (teknogeek):
same, right, then I can say, oh, okay. You know, as I showed in this other report, it's possible to leak these IDs. Like maybe I had them all stored as an attacker or I web archive them or whatever. Like I don't really think you have to get that in depth about like,

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
oh, like this, this is how an attacker could have gotten it. It's like, I showed that it was possible to get them. Right.

Justin Gardner (@rhynorater):
Okay. So yes, that makes sense. I'm going to jump into the timeline section here because I've actually got a couple pointed questions that I'd like to ask and get your response on. The, the first one that I'm just going to drop is like, I'm curious what an actual root cause analysis looks like. from a program's perspective and an impact assessment. And why the heck does it take so damn long sometimes? I know there's a lot of code, right? But if you dropped me into a production code base with grep, you know, and I have a unique parameter name or whatever, I would expect to be able to find the code that is responsible for this thing within an hour max. Like, am

Joel Margolis (teknogeek):
Sure.

Justin Gardner (@rhynorater):
I crazy? Like, you know.

Joel Margolis (teknogeek):
No, that's totally doable, but what does that give you?

Justin Gardner (@rhynorater):
That gives me the piece of

Joel Margolis (teknogeek):
Yeah, I

Justin Gardner (@rhynorater):
code

Joel Margolis (teknogeek):
know where

Justin Gardner (@rhynorater):
that's,

Joel Margolis (teknogeek):
it happens,

Justin Gardner (@rhynorater):
yeah.

Joel Margolis (teknogeek):
but like, does that tell me what it can do? Does that give me the impact of what is capable from there? That's where it takes more nuanced understanding. So finding where it actually exists, like where the actual root cause is, that is generally pretty easily.

Justin Gardner (@rhynorater):
That's

Joel Margolis (teknogeek):
It

Justin Gardner (@rhynorater):
the

Joel Margolis (teknogeek):
depends

Justin Gardner (@rhynorater):
easy part.

Joel Margolis (teknogeek):
on, yeah, it generally depends on the org. Like some orgs are a lot more organized than others. I'd say that where I work, it's very straightforward. It's

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
pretty, you know, yeah, very easy to like see, oh, okay, this endpoint points to this service,

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
which is handled here,

Justin Gardner (@rhynorater):
Sure.

Joel Margolis (teknogeek):
this line of code, right? Like easy. Other companies, probably not as easy. It depends on the company, it depends on how like well structured they are. But the hard part is like, Okay, like what could you do with this? And that's where it takes knowledge from like a security engineer or somebody to, or maybe just even a normal engineer who understands what the rest of the infrastructure looks like. What can that service talk to? Uh, what access, what data access does it have?

Justin Gardner (@rhynorater):
So for each bug,

Joel Margolis (teknogeek):
Have

Justin Gardner (@rhynorater):
are you

Joel Margolis (teknogeek):
historically

Justin Gardner (@rhynorater):
having a

Joel Margolis (teknogeek):
a

Justin Gardner (@rhynorater):
meeting

Joel Margolis (teknogeek):
bug bit.

Justin Gardner (@rhynorater):
with these engineers? Are you sending an email? Is it a message? I mean, what is...

Joel Margolis (teknogeek):
Yeah, I mean, it's like a usually it's a message so like Typically, here's what happens with the report and this is again generic. This is not like

Justin Gardner (@rhynorater):
Mm-hmm, sure.

Joel Margolis (teknogeek):
Specific to

Justin Gardner (@rhynorater):
Yes,

Joel Margolis (teknogeek):
either program

Justin Gardner (@rhynorater):
just

Joel Margolis (teknogeek):
but like

Justin Gardner (@rhynorater):
roughly.

Joel Margolis (teknogeek):
I'd say like most of the time Report comes in gets validated by either the team or triage or some it gets triaged somehow, right? In one way or another then that ticket that report gets turned into an internal ticket which gets assigned to an engineer sometimes with a root cause, sometimes with a fix, usually one or both. And then from there, it's up to the engineer and their team to fix it. And that priority is generally up to the engineering team, not up to the security team. Sometimes it's up to the security team. Sometimes there are agreements that say like, security tickets have this SLA, they need to be fixed in this amount of time. there's as much priority. If it doesn't get prioritized, then we'll have discussions with leadership or whatever,

Justin Gardner (@rhynorater):
It's

Joel Margolis (teknogeek):
right?

Justin Gardner (@rhynorater):
crazy to me

Joel Margolis (teknogeek):
But...

Justin Gardner (@rhynorater):
that you guys have such segments in these big organizations where it's like, all right, when I hear SLA, I think external vendor agrees to do XYZ by external time. But you're literally talking to somebody in your own company and saying,

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
hey, we have an SLA agreement where you gotta fix this by this

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
time. It's really, that's very

Joel Margolis (teknogeek):
Well,

Justin Gardner (@rhynorater):
odd.

Joel Margolis (teknogeek):
and what you have to realize is that the engineering team is not just sitting around waiting for the security team to

Justin Gardner (@rhynorater):
No.

Joel Margolis (teknogeek):
be like, hey, we got something for you

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
to fix, right? Every single day while the security team is handling other bugs and all that kind of stuff, the engineering team is being told by the product team, oh, here's a new feature we want you to write, and can you implement this, and can you

Justin Gardner (@rhynorater):
Right.

Joel Margolis (teknogeek):
fix this bug in non-security

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
bug,

Justin Gardner (@rhynorater):
Security

Joel Margolis (teknogeek):
just a

Justin Gardner (@rhynorater):
is

Joel Margolis (teknogeek):
normal...

Justin Gardner (@rhynorater):
not their thing. Yeah.

Joel Margolis (teknogeek):
logical bug, right? Like, can you add this feature? Can you, whatever, like all these other things that relate to like their day-to-day engineering job, and then a new thing comes into that system and is like, Hey, make room for

Justin Gardner (@rhynorater):
I'm

Joel Margolis (teknogeek):
me.

Justin Gardner (@rhynorater):
important, yeah.

Joel Margolis (teknogeek):
Yes.

Justin Gardner (@rhynorater):
And it doesn't even, and I'm sure these devs are aligned with the vision, you know, sometimes too, of the products iteration, you know, of the products development. And this can kind of seem, I bet 99% of the time, this feels like a blocker. And this feels like just this annoying little task they gotta get out of the way

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
to move on with their actual features that they're developing.

Joel Margolis (teknogeek):
Yeah. And so I think this is where like a lot of nuance falls in between like the relationship between security teams and engineering teams and how to like promote security as like a core value,

Justin Gardner (@rhynorater):
Mmm.

Joel Margolis (teknogeek):
because it's, it can be very difficult to get engineers to give a shit about security

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
issues because it's exactly what you're describing, which is that like, there's a new ticket that just like popped up out of nowhere that is now increasing their workload

Justin Gardner (@rhynorater):
Yeah,

Joel Margolis (teknogeek):
and is now a thing that they have to

Justin Gardner (@rhynorater):
popped

Joel Margolis (teknogeek):
take care

Justin Gardner (@rhynorater):
right onto

Joel Margolis (teknogeek):
of

Justin Gardner (@rhynorater):
my

Joel Margolis (teknogeek):
that they

Justin Gardner (@rhynorater):
Kanban

Joel Margolis (teknogeek):
weren't.

Justin Gardner (@rhynorater):
board or whatever, you know,

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
like,

Joel Margolis (teknogeek):
it's like not they weren't planning on this

Justin Gardner (@rhynorater):
right.

Joel Margolis (teknogeek):
like this wasn't something that was on their radar It wasn't in their sprint It wasn't in their planned amount of work and now they have this extra thing that they have to deal with

Justin Gardner (@rhynorater):
Hmm

Joel Margolis (teknogeek):
And that goes all the way up the chain. It goes to the man their manager it goes to the product team like everybody now has to account for this extra load within the system

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
and It really depends like on how the teams view security as like, it should be like a helpful thing, right? Like this is to like stop, like prevent issues, everybody

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
should be owning security. And so like, there's a lot of nuanced aspects to getting the engineering teams and not just engineering, but all the teams to take security seriously and own it. And that can be difficult. And if you have a team who gives a shit about security and like they respect it, and when a security issue comes in, they take it seriously, that's... the best type of relationship. That's typically where you see stuff get fixed very fast,

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
like they're responsive, all that kind of stuff.

Justin Gardner (@rhynorater):
Man, I.

Joel Margolis (teknogeek):
But if it doesn't, you just have to realize that like, you and your issue are battling against. So many other things. You're battling against the product team who tells that engineering team what to do. You're battling against the C levels and the management teams who are trying to drive the business

Justin Gardner (@rhynorater):
Yeah,

Joel Margolis (teknogeek):
in

Justin Gardner (@rhynorater):
yeah,

Joel Margolis (teknogeek):
a

Justin Gardner (@rhynorater):
yeah.

Joel Margolis (teknogeek):
certain direction. Like there are so many other things that the security team, as much as they'd like to have number one priority and be the most important thing, just can't. Yeah.

Justin Gardner (@rhynorater):
I wonder how we as Bug Bounty Hunters, or we even as external consultants to an organization could help foster that. sort of relationship with the devs. Like, I guess I'm sure that's a challenge and one of the challenges that I've not been exposed to as I'm not a CISO or even an AppSec engineer. But that seems like a good investment of time is like having your security champions or whatever in your various dev product teams and stuff like that are really like you've invested in somehow making them appreciate security and champion it within their, little block, their little team.

Joel Margolis (teknogeek):
Yeah, absolutely. I like, there are various ways you can do that. I think one really great way for bug bounty, especially, is any valid bug bounty report is an external vulnerability report, right? Somebody outside the company found it. It's a valid vulnerability, and it affects your company with no special permissions. It affects it externally, right? That's a huge,

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
like, thing that you can use to advocate and say, like, an external security researcher found this vuln on our site.

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
It is currently actively exposed externally. we need to

Justin Gardner (@rhynorater):
That's

Joel Margolis (teknogeek):
fix this

Justin Gardner (@rhynorater):
a big

Joel Margolis (teknogeek):
and

Justin Gardner (@rhynorater):
deal.

Joel Margolis (teknogeek):
prioritize fixing it,

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
right? Like that's a big deal. And I think that you can at least use that to help drive the importance of it. And then you can use that again as backup when you're like. Training your engineers so like one of the things that we'll do internally is we'll take a lot of these things that we notice Like systemic problems or like core problems that are popping up over and over again And we'll try and turn that into trainings that then every single new hire engineer gets put through this security training that walks them through like how the security process

Justin Gardner (@rhynorater):
Yeah,

Joel Margolis (teknogeek):
should work

Justin Gardner (@rhynorater):
but it's a training,

Joel Margolis (teknogeek):
and Then

Justin Gardner (@rhynorater):
you know like

Joel Margolis (teknogeek):
but this is like day one, okay,

Justin Gardner (@rhynorater):
Yeah, but like, so is

Joel Margolis (teknogeek):
and it

Justin Gardner (@rhynorater):
everything.

Joel Margolis (teknogeek):
trust me it works

Justin Gardner (@rhynorater):
Okay.

Joel Margolis (teknogeek):
I promise we wouldn't do it if it didn't work.

Justin Gardner (@rhynorater):
Okay, all right.

Joel Margolis (teknogeek):
It really, it helps a lot. It helps people understand that like security matters and is taken seriously. It shows them from day one that security is not just like a background process,

Justin Gardner (@rhynorater):
Hmm.

Joel Margolis (teknogeek):
but it is part of the process.

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
And that when they wanna like expose a new endpoint, security has to know and

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
security has to review that. And like that is like a real core part of the engineering process. And it states it as a fact instead of an option.

Justin Gardner (@rhynorater):
That is important and I'm sure that that's an intentional cultural thing that you've got to foster. My next question here on the list is why the frick does it take so long? Aren't you guys using

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
Agile? Can't you

Joel Margolis (teknogeek):
sure.

Justin Gardner (@rhynorater):
just slip it into the next sprint and it be fixed within

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
the week?

Joel Margolis (teknogeek):
If it were up to me, absolutely. Yes.

Justin Gardner (@rhynorater):
But

Joel Margolis (teknogeek):
But

Justin Gardner (@rhynorater):
that's

Joel Margolis (teknogeek):
again,

Justin Gardner (@rhynorater):
a company

Joel Margolis (teknogeek):
as I described,

Justin Gardner (@rhynorater):
culture problem.

Joel Margolis (teknogeek):
yeah. Yep. It's a company culture problem. It's a you're fighting with all these other teams and all these other organizations and all these other tickets and all these other everything. Like, again, if it was just security and three engineers, I'm sure the relationship would be such that like, we'd be like, hey, can you fix this? And they'd be like, sure. And then they'd fix it. But there are so many other players that are invisible to the researcher and so many other like, you know, just communication and organizational type problems that are really hard to get around that even if the security team knows in their heart that this is an important thing, it doesn't always... get received that same way by the engineering team. And so the best thing that you can do is work in the background to try and make security more valuable and have more weight to it when they say, hey, we have a vulnerability. Like figure out how to get the engineering team to understand that, oh, this is an external vulnerability. When the security team comes to me with something that that's... I need to fix this. This is important for the company. It's important for our users. It's important for our data, whatever

Justin Gardner (@rhynorater):
Yeah, I have a hard time, and maybe it doesn't come across to the devs this way, but I have a hard time reconciling that they have two tickets. One is change the CSS styling on this button, and the other one is prevent mass data leakage of all your users, right? And

Joel Margolis (teknogeek):
You

Justin Gardner (@rhynorater):
somehow,

Joel Margolis (teknogeek):
got security brain bro.

Justin Gardner (@rhynorater):
I

Joel Margolis (teknogeek):
Yeah

Justin Gardner (@rhynorater):
mean, do I? I mean, is that really how it is?

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
Sorry, I just slapped

Joel Margolis (teknogeek):
yeah,

Justin Gardner (@rhynorater):
my mic with my chin,

Joel Margolis (teknogeek):
listen

Justin Gardner (@rhynorater):
but.

Joel Margolis (teknogeek):
like no like as a security person, of course like security is number one for me I like absolutely security should be number one. It should be the number one priority. It should take precedent over everything

Justin Gardner (@rhynorater):
Well,

Joel Margolis (teknogeek):
but

Justin Gardner (@rhynorater):
yeah,

Joel Margolis (teknogeek):
It's not my decision.

Justin Gardner (@rhynorater):
we don't have to

Joel Margolis (teknogeek):
Yeah

Justin Gardner (@rhynorater):
debate that, that's fine. Okay, last couple questions. Do you ever reread dupe slash informative reports that have been set that way by HackerOne triagers or is that just like a, is that like something that would ideally happen but like just very rarely happens when you actually do something with it?

Joel Margolis (teknogeek):
So I try to yes, I try to read through like I try to watch everything that they're doing.

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
I'm not gonna lie um, I hope that this doesn't portray h1 triage in a negative or unfavorable light uh, the reality is that they have a very difficult job and

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
that very difficult job is trying to manage and validate reports across thousands of companies, hundreds, two thousands of companies. I don't know how many programs each TreeAzure is responsible

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
for, but I know that I see a lot of the same names across a lot of

Justin Gardner (@rhynorater):
Oh yeah.

Joel Margolis (teknogeek):
different programs, right? So the reality is that they're dealing with a lot of different things and they're trying to do that at scale, like many, many times a day. And that is a difficult job to do. It's a difficult job to do it well. And it's a difficult job to do it to this standard that, and like somebody who works for that company would be like, Oh, that's how I would do it. Right. And everybody's standard is different. This is why external triage services exist. Like once not provided by the platform, but like a cohort of people who you pay

Justin Gardner (@rhynorater):
Sure.

Joel Margolis (teknogeek):
like NCC group

Justin Gardner (@rhynorater):
Right.

Joel Margolis (teknogeek):
to do your triage for you and those. that type of stuff, that's because it's much more nuanced. It's like, oh, I want you to like handle this in a certain way and I want you to like do this, these specific things. Like you're gonna pay extra for that. You're

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
not gonna,

Justin Gardner (@rhynorater):
Oh, yeah.

Joel Margolis (teknogeek):
when you pay for H1 triage, you're not paying for three specific people to just be like your triagers. Maybe if you're paying the millions of dollars a year, you'll get

Justin Gardner (@rhynorater):
Hahaha.

Joel Margolis (teknogeek):
that, but most programs are not gonna be getting

Justin Gardner (@rhynorater):
Right,

Joel Margolis (teknogeek):
that,

Justin Gardner (@rhynorater):
right,

Joel Margolis (teknogeek):
okay?

Justin Gardner (@rhynorater):
right.

Joel Margolis (teknogeek):
Like the reality is that like 99% of programs are not gonna be getting that level of triage experience

Justin Gardner (@rhynorater):
share.

Joel Margolis (teknogeek):
like, oh this is my triage team who handles

Justin Gardner (@rhynorater):
Right.

Joel Margolis (teknogeek):
things in this specific way and I get to tell them what to do.

Justin Gardner (@rhynorater):
In your experience, do the same H1 triagers work with your program on a regular basis? Or is that, you know...

Joel Margolis (teknogeek):
Uh, it's like the same handful of them.

Justin Gardner (@rhynorater):
Sure.

Joel Margolis (teknogeek):
I'd be lying if I said that there weren't specific instances where we've like had one thing with like a specific triager and had something had to be reassigned.

Justin Gardner (@rhynorater):
sure.

Joel Margolis (teknogeek):
But like, often from what I can tell it's who's online in this time zone and.

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
Like who, you know, who's online, who. who's triaging right now.

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
And sometimes it's the same people, sometimes it's not. I think not every triager has access to every program. So there is going to be some cohort that is responsible for your program, but it's gonna vary by time zone and like what time the report comes in and whatever. So yeah, because of all that, because there's so much like, they have to do so many things, they like, nobody's gonna be perfect. And so... for my programs, I'm gonna look and I'm gonna see like, did they mess anything up? I'm not gonna do that 100% of the time because

Justin Gardner (@rhynorater):
That's

Joel Margolis (teknogeek):
I wanna

Justin Gardner (@rhynorater):
cool.

Joel Margolis (teknogeek):
trust them. And

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
like, there's just way too much overhead. There's no point paying for triage. You're

Justin Gardner (@rhynorater):
Right.

Joel Margolis (teknogeek):
just gonna do it yourself. But, you know, I think like most

Justin Gardner (@rhynorater):
But

Joel Margolis (teknogeek):
of the time they're pretty good.

Justin Gardner (@rhynorater):
if you see something that looks weird, that's getting closed by a triager, what do you say, okay, let me put this a little bit differently. In the experiences you've had, dealing, you know, talking to other program managers in your own programs, in your friends that are managing programs, What do you think the chances are that if a valid bug was closed by HackerOne Triage, what do you think the chance is that the program will catch it?

Joel Margolis (teknogeek):
I'm not gonna put a number on it because

Justin Gardner (@rhynorater):
Sure.

Joel Margolis (teknogeek):
I don't want to.

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
But I'd say it's pretty good that it would get caught.

Justin Gardner (@rhynorater):
Hmm.

Joel Margolis (teknogeek):
It really depends.

Justin Gardner (@rhynorater):
That was not the answer I was expecting, to be honest, when I wrote this question.

Joel Margolis (teknogeek):
If something gets closed, it really depends on the program. And it's

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
gonna depend on

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
who's managing it and how much attention they're paying to it and how much of a fuck they give.

Justin Gardner (@rhynorater):
Yeah. Yeah.

Joel Margolis (teknogeek):
The reality is that, as I said, a lot of programs will be like, why are we paying for H1Triage if we're gonna look through this ourselves?

Justin Gardner (@rhynorater):
Right.

Joel Margolis (teknogeek):
And that's valid. Like, why are you paying for H1Triage if you're going to go look through it yourself, right? But I think it's also your responsibility as a program manager to, if you see something that isn't in line with the standard spam reports you're getting. And anybody who manages a program kind of knows what I'm saying, which is basically that like every program has a certain cohort of things that are very common.

Justin Gardner (@rhynorater):
very

Joel Margolis (teknogeek):
Like,

Justin Gardner (@rhynorater):
clearly just something

Joel Margolis (teknogeek):
yes,

Justin Gardner (@rhynorater):
you're

Joel Margolis (teknogeek):
very

Justin Gardner (@rhynorater):
gonna get

Joel Margolis (teknogeek):
clearly,

Justin Gardner (@rhynorater):
spammed

Joel Margolis (teknogeek):
like

Justin Gardner (@rhynorater):
on

Joel Margolis (teknogeek):
this

Justin Gardner (@rhynorater):
all the

Joel Margolis (teknogeek):
is.

Justin Gardner (@rhynorater):
time, sure.

Joel Margolis (teknogeek):
not something that we care about, or this is not even a security issue, or whatever. And the reality is that when you see stuff outside of that, it's very obvious.

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
And so even if it is, it's immediately marked as informative or duplicate, you're probably gonna look at it because you're gonna be like, huh, that's something different. Like that's not just what I'm normally used to seeing.

Justin Gardner (@rhynorater):
Sure.

Joel Margolis (teknogeek):
Now I think if you get a lot of submissions, that can be harder to identify. And I think the more submissions that you get, the harder you start to lean on

Justin Gardner (@rhynorater):
public

Joel Margolis (teknogeek):
triage

Justin Gardner (@rhynorater):
version

Joel Margolis (teknogeek):
to...

Justin Gardner (@rhynorater):
program versus private program, that sort of thing.

Joel Margolis (teknogeek):
Yeah, like the harder you're gonna lean on triage to basically like be that helping hand to

Justin Gardner (@rhynorater):
that first

Joel Margolis (teknogeek):
filter

Justin Gardner (@rhynorater):
line.

Joel Margolis (teknogeek):
stuff out. Like the whole reason that triage exists is, oh, we don't have to look at a report in a new state, we look at a report in a triage state.

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
Like I don't have to say, oh, this is valid. This isn't valid. This is valid because x, y, like triage should be like, okay, you're getting a bug because we reproduced it and it's valid and you

Justin Gardner (@rhynorater):
Mm.

Joel Margolis (teknogeek):
can turn that into a ticket. Right?

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
Well, that's reassuring to hear the fact that there's a decent chance that if something was slipping through the crack, and like you said, it depends on the program for sure, but if you're submitting a valid bug to a program that is not just getting slaughtered with reports all the time, there's a pretty decent chance that it'll get caught even if

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
it gets miscategorized.

Joel Margolis (teknogeek):
But this totally depends on like who's running it.

Justin Gardner (@rhynorater):
Yeah,

Joel Margolis (teknogeek):
You know,

Justin Gardner (@rhynorater):
it

Joel Margolis (teknogeek):
like

Justin Gardner (@rhynorater):
does.

Joel Margolis (teknogeek):
how, how much attention are they paying?

Justin Gardner (@rhynorater):
Um, internal activity on a report. What does that mean? What, what, why am

Joel Margolis (teknogeek):
What

Justin Gardner (@rhynorater):
I

Joel Margolis (teknogeek):
does

Justin Gardner (@rhynorater):
not

Joel Margolis (teknogeek):
that

Justin Gardner (@rhynorater):
seeing

Joel Margolis (teknogeek):
mean?

Justin Gardner (@rhynorater):
this? Like, what, like,

Joel Margolis (teknogeek):
It's really funny to see this as a question because

Justin Gardner (@rhynorater):
like.

Joel Margolis (teknogeek):
like This is totally normal stuff from like a program management. Okay, so basically when you leave a comment on

Justin Gardner (@rhynorater):
Ha

Joel Margolis (teknogeek):
a report

Justin Gardner (@rhynorater):
ha!

Joel Margolis (teknogeek):
There's like you leave a comment. That's it You know, you just like type something up you press post and it goes into your bug report Well, you probably don't realize is that there's like hidden internal communications that happen also on your report that you don't see

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
so like When a ticket, for example, if you link it to an internal ticket, that is internal activity. We can type out comments that are only seen by the triage team

Justin Gardner (@rhynorater):
Right.

Joel Margolis (teknogeek):
and it has like a red background. Like you don't, it's

Justin Gardner (@rhynorater):
So,

Joel Margolis (teknogeek):
marked

Justin Gardner (@rhynorater):
but

Joel Margolis (teknogeek):
as

Justin Gardner (@rhynorater):
like,

Joel Margolis (teknogeek):
like this is a hidden...

Justin Gardner (@rhynorater):
okay, so is internal activity a good indicator that the report is actually being worked on or is there like, this ticket hasn't been updated in 23 days sort of notifications that go on there

Joel Margolis (teknogeek):
Oh. No,

Justin Gardner (@rhynorater):
that

Joel Margolis (teknogeek):
that

Justin Gardner (@rhynorater):
would

Joel Margolis (teknogeek):
definitely

Justin Gardner (@rhynorater):
trigger that?

Joel Margolis (teknogeek):
means that something's happening. Like

Justin Gardner (@rhynorater):
Okay.

Joel Margolis (teknogeek):
if there's internal communications, that means that the triage team and the internal team are having discussions on your ticket privately.

Justin Gardner (@rhynorater):
Interesting.

Joel Margolis (teknogeek):
And they want that to be tied to the ticket.

Justin Gardner (@rhynorater):
Why don't you just do it in front of me so I can

Joel Margolis (teknogeek):
Because,

Justin Gardner (@rhynorater):
see what's going on?

Joel Margolis (teknogeek):
because oftentimes it's not relevant for the researcher to know these things or it'll be like For example triage will say oh, we're having trouble validating this Internal team can you help

Justin Gardner (@rhynorater):
Sure.

Joel Margolis (teknogeek):
and the internal team will be like, oh, you know, he You know x y and z like we'll give you

Justin Gardner (@rhynorater):
No.

Joel Margolis (teknogeek):
I mean no like

Justin Gardner (@rhynorater):
Do your

Joel Margolis (teknogeek):
yeah,

Justin Gardner (@rhynorater):
job.

Joel Margolis (teknogeek):
I don't know but

Justin Gardner (@rhynorater):
And

Joel Margolis (teknogeek):
yeah,

Justin Gardner (@rhynorater):
retest

Joel Margolis (teknogeek):
but like

Justin Gardner (@rhynorater):
it when

Joel Margolis (teknogeek):
basically

Justin Gardner (@rhynorater):
we're done.

Joel Margolis (teknogeek):
It's like you and the, it's, it's the internal teams, whether that's, you know, the people running the program or the triage team or whatever, having communications, talking, trying to figure out, you know, if something is stuck, that usually means that there's like something weird going on.

Justin Gardner (@rhynorater):
Gotcha.

Joel Margolis (teknogeek):
But internal activity is real, is real. Yes.

Justin Gardner (@rhynorater):
Okay,

Joel Margolis (teknogeek):
There are real conversations.

Justin Gardner (@rhynorater):
all right, good to

Joel Margolis (teknogeek):
Yes.

Justin Gardner (@rhynorater):
know. And I'm sure there are caveats to that. I'm sure some programs have internal automation or whatever that will comment on the ticket and say like, this ticket hasn't been bloody blah and bloody blah, or whatever. But most of the time that's gonna be actual activity. Good to

Joel Margolis (teknogeek):
Yeah, it's

Justin Gardner (@rhynorater):
know.

Joel Margolis (teknogeek):
a

Justin Gardner (@rhynorater):
Okay,

Joel Margolis (teknogeek):
good sign that people are alive and doing things.

Justin Gardner (@rhynorater):
that's good. No need to ask for a followup then if I see internal activity happening.

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
All right, so two... Two questions, one you can opt out of, the other one you cannot opt out of.

Joel Margolis (teknogeek):
I already know which one I'm opting out of. Hahaha.

Justin Gardner (@rhynorater):
Are you opting out of it? How much does it cost to run a bug bounty program, Joel? Like, come on, you gotta tell me something about that.

Joel Margolis (teknogeek):
I definitely can't tell you hard numbers on anything. However,

Justin Gardner (@rhynorater):
Okay.

Joel Margolis (teknogeek):
what I can tell you is that there are different payment models depending on each company. I definitely, almost definitely cannot tell you what the per program, I'll tell you off air. How about that?

Justin Gardner (@rhynorater):
Okay, alright, yeah,

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
maybe,

Joel Margolis (teknogeek):
I'll

Justin Gardner (@rhynorater):
maybe

Joel Margolis (teknogeek):
tell you

Justin Gardner (@rhynorater):
we'll...

Joel Margolis (teknogeek):
off air. It is definitely tens of thousands of dollars a year,

Justin Gardner (@rhynorater):
Okay,

Joel Margolis (teknogeek):
minimum,

Justin Gardner (@rhynorater):
minimum.

Joel Margolis (teknogeek):
and. And then you also have like bounty budget on top of that. So

Justin Gardner (@rhynorater):
Hmm

Joel Margolis (teknogeek):
there's a fee for the program itself, there's a fee for triage services, there's a, if you decide to do that, there's a fee for, that's like dollar for dollar or more for when you pay bounties. So if I pay a $5,000 bounty to you, that comes out of my bounty pool, which is

Justin Gardner (@rhynorater):
Sure.

Joel Margolis (teknogeek):
separate from the amount of money that I'm paying HackerOne

Justin Gardner (@rhynorater):
Different.

Joel Margolis (teknogeek):
or whoever,

Justin Gardner (@rhynorater):
That makes sense.

Joel Margolis (teknogeek):
just

Justin Gardner (@rhynorater):
Okay.

Joel Margolis (teknogeek):
to run my program.

Justin Gardner (@rhynorater):
Right.

Joel Margolis (teknogeek):
So I pay a yearly contract. to platform

Justin Gardner (@rhynorater):
Sure.

Joel Margolis (teknogeek):
for whatever, maybe triage, the program itself, et cetera. And then in addition to that, there's a separate amount of money that all the bounties come out of. And if that amount runs low, then you make another deposit.

Justin Gardner (@rhynorater):
Sure, okay, gotcha. So there's, and then whenever you pay a bounty, you can potentially, you know, platform also take some of the top off that, or, you know, some amount of that also gets paid to the platform in some scenarios.

Joel Margolis (teknogeek):
Yeah, that's where it varies.

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
I believe the newer contracting models,

Justin Gardner (@rhynorater):
They don't do that.

Joel Margolis (teknogeek):
they don't do fees like that anymore.

Justin Gardner (@rhynorater):
Okay.

Joel Margolis (teknogeek):
They used to do a flat 20% fee,

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
which I think is public.

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
But since then, lots of stuff has changed. It really depends on like... Usually when you renew contract, you might negotiate that with the platform

Justin Gardner (@rhynorater):
Sure.

Joel Margolis (teknogeek):
to determine like how you want to do your fee structure, should you pay fees ahead of time, like all that kind of stuff. So yeah, that varies a lot.

Justin Gardner (@rhynorater):
Saw it. Last question. This has been a absolute freaking marathon of an episode. I have not even done that much talking and I am exhausted. I can't imagine how you must feel. But talk to me about LHEs. Are they with live hacking events, for those of you that aren't familiar with the acronym. How does that feel from the program side? Is that stressful? Is that fun? Is that all of the above?

Joel Margolis (teknogeek):
Yeah, all the above, all the above. Um, I run a couple live hacking events, um, while I was at Uber.

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
They are, they don't call it a war room for nothing. Okay. Like,

Justin Gardner (@rhynorater):
Pfft

Joel Margolis (teknogeek):
uh,

Justin Gardner (@rhynorater):
Ha ha

Joel Margolis (teknogeek):
yeah, it's, um, It's chaos in there, I'm not gonna lie. Okay, so basically here's the picture. All right, you've got at least a couple people from the security team in there. You've probably got at least a couple people from the engineering team or people on call remotely who can help answer questions and direct tickets in the right direction. And then you've got the triage team and the H1 team in there as well. So triage is helping like validate stuff in real time, like log in, test stuff. Like if they need something from the security team for help retesting or whatever, they're there. Security team is there to like look at all the bugs, address criticality, address impact, address validity. Um, see if this is stuff that needs to be fixed, like now,

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
or can wait until after the event. Um, figuring out payouts. That's like a huge part of it is like. That security team meeting trying to decide impact on like 10 bugs every hour or

Justin Gardner (@rhynorater):
Yeah,

Joel Margolis (teknogeek):
more

Justin Gardner (@rhynorater):
that's like

Joel Margolis (teknogeek):
Like a

Justin Gardner (@rhynorater):
a weekly meeting

Joel Margolis (teknogeek):
minimum.

Justin Gardner (@rhynorater):
that happens, but

Joel Margolis (teknogeek):
Yeah

Justin Gardner (@rhynorater):
actually every hour.

Joel Margolis (teknogeek):
Yeah, it's like you're sitting down. You're like, okay, what about this one? Oh, what could they do? Oh, they could do this this. Okay. Well, that's you know, the severity blah blah. Okay, let's pay it out this next

Justin Gardner (@rhynorater):
Yeah,

Joel Margolis (teknogeek):
like

Justin Gardner (@rhynorater):
that's intense, man. And you're

Joel Margolis (teknogeek):
Yeah

Justin Gardner (@rhynorater):
making big money decisions on the fly like that. That's nuts.

Joel Margolis (teknogeek):
Yes, yeah, so definitely stressful, definitely a lot of fun, I think,

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
like, in hindsight, it's such a great experience to

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
get to have that real, like, face-to-face interaction with the researcher

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
and build those relationships. There's so much... like partnership building that you can do in just like that one day of hacking between like top hackers and your program that like build like life long like sorts of relationships with those hackers now want to come hack on your program. They have a face that they assign when they submit a report

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
and they have like a real, you know, there's a real

Justin Gardner (@rhynorater):
It's

Joel Margolis (teknogeek):
tie

Justin Gardner (@rhynorater):
super

Joel Margolis (teknogeek):
there.

Justin Gardner (@rhynorater):
valuable.

Joel Margolis (teknogeek):
Like. It adds a lot. Yes, they're stressful. Yes, they're expensive. Yes, it's chaos, but is it worth it? Yeah, I would certainly say so. I think it depends on the company. Not every company can run a live hacking event. There are certainly tons of companies who should not run a live hacking event.

Justin Gardner (@rhynorater):
Oh yeah, for sure.

Joel Margolis (teknogeek):
Yeah, like you don't have the scope for it. You don't have the bounty for it. You don't have the team for it. You don't have, like, yeah. Like I think, I was telling this to the hacker one team. I feel like honestly one of the requirements for running a live hacking event should be that one of your people at least has to come on site and just sit in the war room and watch what a live hacking event looks like

Justin Gardner (@rhynorater):
of somebody

Joel Margolis (teknogeek):
from the

Justin Gardner (@rhynorater):
else's

Joel Margolis (teknogeek):
other side.

Justin Gardner (@rhynorater):
live hacking event.

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
Ah,

Joel Margolis (teknogeek):
and just be like, oh shit, okay, this is what

Justin Gardner (@rhynorater):
this

Joel Margolis (teknogeek):
we gotta

Justin Gardner (@rhynorater):
is

Joel Margolis (teknogeek):
do.

Justin Gardner (@rhynorater):
gonna

Joel Margolis (teknogeek):
Yeah. Yeah,

Justin Gardner (@rhynorater):
rock my world.

Joel Margolis (teknogeek):
like I feel like they need some perspective because so many companies will come in and be like, we wanna run a live hacking event and I think. I mean, if I worked at a hackathon, my reaction would be, huh, what you do?

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
Really, are you sure?

Justin Gardner (@rhynorater):
Are you sure? Yeah.

Joel Margolis (teknogeek):
Yeah, like we'll take your money, but are you sure?

Justin Gardner (@rhynorater):
That's a pretty, yeah, like, like live hacking event training or something

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
like that could be

Joel Margolis (teknogeek):
because

Justin Gardner (@rhynorater):
really

Joel Margolis (teknogeek):
like

Justin Gardner (@rhynorater):
a thing

Joel Margolis (teknogeek):
there's

Justin Gardner (@rhynorater):
as well.

Joel Margolis (teknogeek):
so much like you want it to be a good experience for the researchers. You

Justin Gardner (@rhynorater):
Mm-hmm.

Joel Margolis (teknogeek):
want it to be a good experience for the company. You want it to be good experience. Like overall you want like good bounty payouts. You want everybody to walk away happy swag, like all these things. There's so many moving parts, so many aspects that like, there's a reason why you see a lot of the same companies running life hacking events over and over again, because they have the money, they have the scope, they have the

Justin Gardner (@rhynorater):
That's a pretty,

Joel Margolis (teknogeek):
everything

Justin Gardner (@rhynorater):
yeah, the stars

Joel Margolis (teknogeek):
that

Justin Gardner (@rhynorater):
really got

Joel Margolis (teknogeek):
can

Justin Gardner (@rhynorater):
online

Joel Margolis (teknogeek):
make it happen.

Justin Gardner (@rhynorater):
for it to happen.

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
Wow, dude. All right, Joel, that was a marathon of an episode. I think this is the longest episode we've ever done. We definitely went down some rabbit holes. We definitely,

Joel Margolis (teknogeek):
Ha ha.

Justin Gardner (@rhynorater):
you know, went off on each other a little bit there. I think overall you did a great job answering all these questions, and I'm gonna go take a nap now.

Joel Margolis (teknogeek):
Hahaha

Justin Gardner (@rhynorater):
But for those... For those few of you that are still with us at the end of this episode, one, you rock. Two,

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
check out the new website, check out the newsletter that's there, go ahead and subscribe to those. And also, we have a Kaido referral code, which we've mentioned on Twitter, but we haven't really made mega public in an episode before. We'll probably announce it at the beginning of next week's episode as well. But... For those of you that are still with us, you can go ahead and use that and get 10% off of your Kydo subscription, which we would appreciate, and you would also appreciate, because that

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
saves you money.

Joel Margolis (teknogeek):
Do you know what that code is off the top of your head

Justin Gardner (@rhynorater):
It

Joel Margolis (teknogeek):
or

Justin Gardner (@rhynorater):
is,

Joel Margolis (teknogeek):
will it be

Justin Gardner (@rhynorater):
oh yes,

Joel Margolis (teknogeek):
down

Justin Gardner (@rhynorater):
that's

Joel Margolis (teknogeek):
below?

Justin Gardner (@rhynorater):
a good point. I should actually say what it is. It's the CTBB podcast. It's our handle on Twitter, Critical Thinking Bug Bounty podcast, and that will get you 10% off on your Kydo subscription.

Joel Margolis (teknogeek):
Awesome.

Justin Gardner (@rhynorater):
Um,

Joel Margolis (teknogeek):
That's

Justin Gardner (@rhynorater):
Joel,

Joel Margolis (teknogeek):
kind of C A I D

Justin Gardner (@rhynorater):
yes,

Joel Margolis (teknogeek):
O not K.

Justin Gardner (@rhynorater):
every time we mentioned

Joel Margolis (teknogeek):
Yes.

Justin Gardner (@rhynorater):
it on the pod, everyone's like, what's a cat?

Joel Margolis (teknogeek):
C

Justin Gardner (@rhynorater):
What's

Joel Margolis (teknogeek):
A I D O

Justin Gardner (@rhynorater):
what's

Joel Margolis (teknogeek):
dot

Justin Gardner (@rhynorater):
a,

Joel Margolis (teknogeek):
I

Justin Gardner (@rhynorater):
what's

Joel Margolis (teknogeek):
O.

Justin Gardner (@rhynorater):
a key?

Joel Margolis (teknogeek):
Yes.

Justin Gardner (@rhynorater):
A dough, you know, like, no. Um,

Joel Margolis (teknogeek):
Okay.

Justin Gardner (@rhynorater):
so, all right, Joel, you got anything before I fall asleep and go take a nap from

Joel Margolis (teknogeek):
Uh...

Justin Gardner (@rhynorater):
exhaustion? Cause that, that is a lot of

Joel Margolis (teknogeek):
I think

Justin Gardner (@rhynorater):
talking.

Joel Margolis (teknogeek):
that's it. I'm glad I've been chugging

Justin Gardner (@rhynorater):
You've

Joel Margolis (teknogeek):
caffeine

Justin Gardner (@rhynorater):
been,

Joel Margolis (teknogeek):
this whole

Justin Gardner (@rhynorater):
you've

Joel Margolis (teknogeek):
time

Justin Gardner (@rhynorater):
been

Joel Margolis (teknogeek):
because...

Justin Gardner (@rhynorater):
chugging me. Yeah. You were, you're like, I, you know, towards the end, you can kind of see me like, you know, lower the desk and then

Joel Margolis (teknogeek):
Yeah,

Justin Gardner (@rhynorater):
get

Joel Margolis (teknogeek):
so

Justin Gardner (@rhynorater):
like, start

Joel Margolis (teknogeek):
slowly

Justin Gardner (@rhynorater):
like,

Joel Margolis (teknogeek):
slumping

Justin Gardner (@rhynorater):
like

Joel Margolis (teknogeek):
into your chair. Hahaha.

Justin Gardner (@rhynorater):
Well, it doesn't help either because I had like a two hour meeting this morning and then

Joel Margolis (teknogeek):
Oh man.

Justin Gardner (@rhynorater):
my buddy came over and we just chatted for a little while, so like two hours. And you know, like I'm not really, I'm not an extrovert, I'm an introvert, so this sort of

Joel Margolis (teknogeek):
Yeah.

Justin Gardner (@rhynorater):
long form talking takes a toll on me, you know, and so I'm at like minimum energy right now. But we made it.

Joel Margolis (teknogeek):
Yeah dude, well yeah thanks for having this discussion.

Justin Gardner (@rhynorater):
Yeah.

Joel Margolis (teknogeek):
I hope that I helped to answer some of the mysteries behind

Justin Gardner (@rhynorater):
You did indeed.

Joel Margolis (teknogeek):
Bug Bounty and reports and what the hell's going on with my ticket and all that kind of stuff. And yeah, if there's more questions that come after this, we can certainly do

Justin Gardner (@rhynorater):
for sure.

Joel Margolis (teknogeek):
a follow up episode or something. All

Justin Gardner (@rhynorater):
All right, GG, man. That's the pod.

Joel Margolis (teknogeek):
right, that's fun. Catch you later.

Justin Gardner (@rhynorater):
Holy moly man.