Episode 67: VDPs & Accidental Program VS Hacker Debate Part 2

Justin Gardner (@rhynorater) (00:00.)
Yeah, let's just go ahead and start it. And OK, you were saying you said when you're using Kaido, we're just going to get this on air. When you're using Kaido, what's going on?

Joel Margolis (teknogeek) (00:06.591)
Sure.

Yeah, so I still, I'm still going back and forth between Kaido and burp. I've gotten into some like more complex workflows recently, like more technical type of like binary content and like weird protocols and stuff. And I found that it's just these things don't work as well in Kaido. Like Kaido works really well for just sort of like standard type of HTTP work, right? Like plain text or plain text requests, plain text responses.

Justin Gardner (@rhynorater) (00:22.484)
Mmm.

Justin Gardner (@rhynorater) (00:34.196)
Mm -mm.

Joel Margolis (teknogeek) (00:39.007)
And when you start to dive into more of the binary aspects, for example, there's no hex editor. That's a great example. So I was working with, I want to say protobuf or something. Selecting bytes in protobuf and relying on the system clipboard to be copying those bytes correctly. It's just like, I don't trust it and I don't know how to verify it and all that kind of stuff. So I just end up going to BERT for those types of things. And there are certain things. Go ahead.

Justin Gardner (@rhynorater) (00:44.18)
Yeah, that's a big problem.

Justin Gardner (@rhynorater) (01:03.636)
Yeah, yeah, I had the same thing the other day with that. Like, I was, and that's one of the topics for today, was I was taking a look at some of the code behind AirChat, and they used gRPC on the backend for their API, and it's binary based. So I was like, uh, and there's so many characters that I couldn't even see, and I was like, I don't even know how this is working, this is kind of magic, and then I put it into burp and saw, like, turn on hex mode, and I saw a bunch of bytes that I was kind of messing up, which is why I was getting bad responses, so.

Joel Margolis (teknogeek) (01:23.775)
Yeah.

Justin Gardner (@rhynorater) (01:33.78)
Yeah, the hex editor, it's a big thing for those proto buff or GRPC based targets.

Joel Margolis (teknogeek) (01:40.671)
Yeah, for sure. And like, it's not, I'm sure you can make Kaido do it with workflows and custom commands and all that kind of stuff, but that, when I'm using a proxy, that's not the type of stuff I want to be fighting with at this time. Right. It needs to like work with my flow because if I'm spending a lot of time trying to make Kaido work like burp, then that's taking away from the time that I can spend hacking. And it's just like not an efficient use of my time. And I will rabbit hole into that and spend like four hours making Kaido.

Justin Gardner (@rhynorater) (01:45.652)
Mm -hmm.

Justin Gardner (@rhynorater) (01:53.332)
It needs to be seamless.

Joel Margolis (teknogeek) (02:09.119)
do this one thing the same as burp and then I'll be like, wait, what was I doing again? And it's, you know, I've just found that I have to kind of, um, uh, you know, if I get into those situations, just, you know, call, call it and say, okay, kind of not there yet. Switch to burp and then maybe I'll switch back or, you know, kind of depends on the workflow or the target.

Justin Gardner (@rhynorater) (02:24.436)
Mm -hmm.

Justin Gardner (@rhynorater) (02:29.588)
Yeah, I think it's a maturity thing too. You know, Burp has been around for so long there and they have a really developed plugin system where you can, you know, import like black box, protobuf and all sorts of plugins for these various systems that people have rabbit hold on and built just like you were talking about for Burp. So I think as Kaido continues to mature, you know, we'll see that stuff as well. And we are seeing it already, you know, there's new plugins getting released for Kaido all the time.

Shout out to Monke for, was it Kaido Pets and Kaido Nerds type? Did you see Kaido Nerds type?

Joel Margolis (teknogeek) (03:03.007)
Yeah, I saw that. I didn't see kind of nerds there, but I - or actually yes, I did. Yes, I did. Okay. Yeah, yeah.

Justin Gardner (@rhynorater) (03:07.732)
Yeah, yeah. So essentially, this is based off of the conversations we've had on the pod. But essentially, it adds a right click, send to Joel, or right click, send to Justin interface on Kaido, which I think is super clutch. And that's something that I've kind of been lobbying for as well. For those of you listening, I'm an advisor on Kaido's board. And one of the things I've been kind of lobbying for,

within the Kydo ecosystem are sort of collaboration -based features. I think that would be really huge. So I'm glad to see some pseudo -collaboration features there coming in from the plugin context. And yeah, let us know if you, for those of you listening, let us know if you'd be interested in something like that, because it might be able to push that along through the system if we have any big promoters of that. So yeah.

Joel Margolis (teknogeek) (04:01.375)
Yeah, absolutely. Should we talk about this, this Nagley thread by the way, or? Okay, okay, so I don't even remember if I saw this like on my feed or somebody sent it to me or whatever, but Nagley posted this big long thread this morning, right at 6 a .m. Totally not scheduled.

Justin Gardner (@rhynorater) (04:04.948)
Dude, this is gonna open up a can of worms. Yeah, let's do it, let's do it.

Justin Gardner (@rhynorater) (04:24.692)
Yeah.

Joel Margolis (teknogeek) (04:25.311)
But it was basically just a rundown of, you know, sort of a brain dump from his thoughts and opinions about VDPs, Vulnerability Disclosure Programs, right? Not bug bounty programs, but the non -paying programs. And sort of, you know, why he feels like, you know, they're really negative for the platforms, why they're negative for hunters, you know, what sort of load they create and whether or not it's even worth having these sort of things. So real quick.

Let's just sort of go over the general claims, right? So first of all, VDPs are worse for bug bounty hunters, worse for paid bug bounty programs. They're free labor. You can only find bugs on VDPs, but you can't pivot that into bug bounty programs. The leaderboard stats get skewed from VDPs, and there are these same scope.

Justin Gardner (@rhynorater) (05:19.828)
Mmm.

Joel Margolis (teknogeek) (05:23.615)
programs where somebody has a VDP that's fully non -paying and then they have a private bug bounty program with the same scope or a limited subset of that scope that pays, but they don't pay for everything. So.

Justin Gardner (@rhynorater) (05:27.412)
Mm.

Justin Gardner (@rhynorater) (05:34.164)
Not a big fan of that last one, man. That last one kind of gets my goat a little bit, not gonna lie.

Joel Margolis (teknogeek) (05:39.167)
Yeah, so I like personally, I rarely ever interacted with VDPs, but I will say my general stance on VDPs is fairly similar to Nagly's. I don't think that VDPs are really a good use of resources for a lot of companies, especially the ones that are primarily using them. And generally speaking, I don't know about so much about like, you know, the effects on other programs and on hackers and all that kind of stuff. We can, we can dive into that for sure.

Justin Gardner (@rhynorater) (05:52.756)
Mm -hmm.

Mm.

Joel Margolis (teknogeek) (06:06.687)
But I definitely think that a lot of the companies that are using VDPs shouldn't be using VDPs. And what I mean by that is, I mean, let's call it spade and spade. I think the greatest example of this in history is the Red Bull program. Okay. Or maybe the Department of Defense. I don't know. There's a lot of good examples.

Justin Gardner (@rhynorater) (06:13.14)
Mm.

Justin Gardner (@rhynorater) (06:22.132)
Wow, okay.

Justin Gardner (@rhynorater) (06:27.22)
Department of Defense, great example, Ford, another one, massive company. Yeah, there's so many big companies like that.

Joel Margolis (teknogeek) (06:31.327)
sure. I mean, there's too many good examples. Right, right. Yeah, almost too many good examples to even be like, here's the best example, right? Which is kind of goes and talks about the problem, doesn't it? You have these huge, huge, huge, huge, multi -billion dollar companies. Okay. Real quick, we didn't plan this. Ford market cap. Ford's market cap is $49 .15 billion.

Justin Gardner (@rhynorater) (06:41.492)
Mm -hmm.

Justin Gardner (@rhynorater) (06:58.58)
Oh my gosh.

Joel Margolis (teknogeek) (07:00.159)
Red Bull Market Cap.

Red Bull's market cap is unknown because I think they're private, but somewhere around like $10 billion at least. Am I reading that right?

Justin Gardner (@rhynorater) (07:14.388)
And look at this dude, Ford's annual gross profit for 2023 was 25 billion. So they're operating at like 50 % profit margins. So there's plenty of space there for paying researchers 10 grand to find a crit.

Joel Margolis (teknogeek) (07:31.231)
Yeah. And I will say, right, it's not always about the money. I think anybody who has worked at a company has dealt with finance and has seen sort of the reality of, you know, just because a company makes revenue doesn't mean that that revenue is accessible or usable by whatever. That shouldn't stop you from like emphasizing your security program, right? Like if you are a $50 billion company, you should have at least a little bit of money where your mouth is on your security team, right?

Justin Gardner (@rhynorater) (07:51.668)
Mm -hmm.

Justin Gardner (@rhynorater) (07:58.612)
It's an investment too. I feel like most of these companies as well, it's nice to be able to say, if you do get popped, for example, right? It's nice to be able to say, well, we've been running this bug bounty program and we've been doing this and that and the other thing. To at least show that you're working hard to take your security seriously. And I just feel like VDP doesn't do that.

Joel Margolis (teknogeek) (08:22.975)
Yeah, now that being said, I think there are certainly companies and candidates that fit into a category where VDPs make sense. You've got smaller companies, companies that don't have this kind of funding, maybe open source projects, like the internet bug bounty stuff, and even them, they offer bounties, right? I think that there are certainly cases that can be made for nonprofits or whatever who can fit into this category of we want to be receiving vulnerabilities, we don't have the kind of budget and money,

to pay huge amounts, but we want to recognize in another way. Are VDPs the solution for that? I don't know. Again, I think VDPs are a really crude way of solving this problem where you want to recognize hackers for their work, but you don't want to do it with money directly. And I think what...

Justin Gardner (@rhynorater) (09:13.748)
or necessarily incentivize hacking on your program. Yeah.

Joel Margolis (teknogeek) (09:17.087)
Right, or incentivize it at all, right? Because it's like points. It's magic internet points that are up to the platform, not even up to like you as the company. Like there's no skin in the game whatsoever for Ford to give out 50 reputation points, right? It's literally a click of a button. No money, no nothing, no anything. And that's not a huge incentive for the program to do things right. That's not a huge incentive for the researchers to even submit stuff. Right? If you find a vulnerability, a really critical vulnerability on a program that has a

Justin Gardner (@rhynorater) (09:30.26)
Absolutely.

Joel Margolis (teknogeek) (09:46.783)
VDP, where's your incentive to report it directly to that company so that you can get a pat on the back and maybe a t -shirt or something, right? Like I think that it's a really again, it's a crude way of sort of solving this problem and If we're gonna allow it to exist within the space of bug bounty We should probably be limiting limiting it to certain factors, right? Like what's your total revenue? Are you a nonprofit? Like do you have?

Justin Gardner (@rhynorater) (10:12.884)
Mmm.

Joel Margolis (teknogeek) (10:13.183)
Any sort of like extra cash flow, like how many employees do you have? Like, I don't know. I'm sure the platforms would probably be better at creating those sort of parameters. But I think that VDPs are really starting to take up a space where they shouldn't be. And every single hacker I talk to who has interacted with a VDP that should be paying has the same exact reaction. It's like, why isn't this company paying? They have the money to pay it, right? Red Bull, Ford, the DoD. I mean, the DoD is a little bit of an edge case, I think, but you know,

Justin Gardner (@rhynorater) (10:34.932)
Yeah. I think it's an interesting...

Joel Margolis (teknogeek) (10:42.303)
I hate to pick on these two companies because I know that there are other ones that I'm just like not thinking of at the moment, but I will say IBM, sure, right? Like there are so many large companies that make a ton of money and could afford to pay and choose not to, right?

Justin Gardner (@rhynorater) (10:46.836)
IBM.

Justin Gardner (@rhynorater) (10:58.164)
Yeah, yeah, I think one of the top points that he made in this tweet that he did was the fact that bug crowd has dropped incentivization for VDPs and HackerOne, I remember them sort of speaking out like, yeah, yeah, that makes sense, we'll move in that direction, but they haven't. And I think that's one of the things we're seeing with HackerOne a little bit too is this sort of like companyization of HackerOne.

where it's like, okay, how is this, you know, and not to say that bug -crowded integrity and all the other ones aren't thinking about their margins or whatever, but I imagine VDPs and having active VDPs is a really good onboarding place for HackerOne. Like, hey, we've already got a VDP set up, we get to talk to you every couple months or whatever to check in on the VDP or give you any findings and stuff.

It's enough touch points where it's like, okay, sales becomes much easier when you've got a VDP. I guarantee you there's some dashboard, some metric inside of Hacker One sales presentations where it's like, you know, 28 % of VDPs convert to actual bug bounty customers or something like that after a certain period of time. I don't necessarily blame them for, I do blame them. I really think it should not be incentivized and I also hate the fact that there are private VDPs.

That's something that for those of you that don't know, you can turn that off in your settings on HackerOne and your invite preferences. Pretty much all the top hackers and all of the mentees of the top hackers have this turned off. It's one of the first things you do when you set up your HackerOne account. So I would definitely recommend turning those off. And yeah, HackerOne needs to get rid of those because it's just bad for the ecosystem.

Joel Margolis (teknogeek) (12:43.295)
Yeah, and somebody actually mentioned in a reply here, apparently, and I haven't verified this, but YesWeHack has no VDPs at all. So some of the platforms have taken a really, really hard line approach with this where they're not having any VDPs whatsoever. I think the bottom line is that not only from a platform perspective, but from a program perspective, you should be paying for security vulnerabilities.

Justin Gardner (@rhynorater) (12:51.284)
Oh, really? Interesting.

Joel Margolis (teknogeek) (13:12.606)
even if it's a little bit of money, right? That's almost a separate discussion of if your bounty table is too low and if you should be raising it. But the fact that there are these companies that are not paying at all really says a lot about sort of how they view external security research and how they handle that internally, right? Like if you had a really serious vulnerability, there's no proper formal route to get a hold of their security team. Maybe you could email them or, you know, I guess you could submit it through their VDP, but...

Justin Gardner (@rhynorater) (13:12.628)
Mm -hmm.

Justin Gardner (@rhynorater) (13:28.82)
Mm.

Joel Margolis (teknogeek) (13:42.591)
You know, it really just says a lot about sort of where the priorities are for these companies, especially the really, really big ones that aren't paying. So I would love to see a more hardline stance on this from the platforms.

Justin Gardner (@rhynorater) (13:51.988)
What? Yeah.

What is your thought on his metric of like, okay, there should be roughly, and these are rough calculations, and I tagged Yobur to see if we get actual numbers on this, but if there's a similar number of submissions to VDPs as there are bug bounty programs, you're spending, your triage is spending half of your time triaging VDP bugs. And that is a lot. That is a lot of overhead. So, I mean, what do you think about that?

Joel Margolis (teknogeek) (14:25.599)
Yeah, so I will say, I don't know how that works. I assume that HackerOne has separate triage teams for paying customers than they do for VDP customers. So it's hard for me to make a claim about whether or not that extra triage load and the triage burnout and the excess report load and all that kind of stuff, it's really hard to say whether or not that's valid. That being said, if,

tomorrow there was a switch that was flipped and these VDP programs were now private programs, not only would they pay the same amount for triage services, but, well actually I don't know, I mean they must pay for triage services even as a VDP, right? So they're probably gonna say they pay the same amount for triage services, maybe a little bit more because it's dedicated, question mark, I'm not sure. But really it's the same triage load, right?

Justin Gardner (@rhynorater) (15:07.764)
They must, yeah.

Justin Gardner (@rhynorater) (15:23.124)
Mm.

Joel Margolis (teknogeek) (15:23.135)
you still have the same number of triage people who are gonna be dealing with the same number of reports and all that kind of stuff. It's really just maybe more reports and more money overall. So.

Justin Gardner (@rhynorater) (15:33.332)
It's a private program though now, so the volume isn't going to be quite as much though.

Joel Margolis (teknogeek) (15:38.111)
Well, they could be a public paying program, right?

Justin Gardner (@rhynorater) (15:40.147)
They could be, yeah, they could. But going from a VDP to a public paying program is a pretty ballsy move. I think that the best practice is like go from VDP to Buck Bounty, private Buck Bounty, run that for a year or two, clean up your tax surface, and then jump right into a public program.

Joel Margolis (teknogeek) (15:53.023)
Yeah.

Joel Margolis (teknogeek) (15:56.735)
So this gets to one of the other problem points, doesn't it? Because the cat is now out of the bag. And so you've created an issue where you have a public VDP, but you want to start paying for things, but you don't want to go zero to a hundred. So what do you do? You create a private paying bug bounty program with the same scope. And you start to pay researchers for those vulnerabilities who are invited. And you run that for a couple of years and eventually.

You either phase it over to public or what they probably do is they see, oh, wow, we're still getting a lot of reports on this public VDP and we're getting a lot of reports on this private bug bounty program. Why would we ever merge the two? Because now we're going to pay a lot more money. And they just never do.

Justin Gardner (@rhynorater) (16:38.324)
Right. Well, yeah, and here's the thing. I spoke with somebody who runs a public VDP and a private program, a public money program, and I said, why are you doing that? And they said something that was kind of interesting. They said that they actually pay the VDP submissions if they meet a certain threshold of report. So the lows and the lower mediums, they will...

let sit in the VDP and just accept the report through the VDP. The mediums and above, the mediums, the upper mediums are sort of a judgment call. Do we bring them into the private program, pay them for it, or do we just accept it and invite them to the private program? And then highs and crits all get moved to the bug bounty program and paid. And so I was like.

Okay, you know, that's kind of interesting. So it's sort of like an onboarding mechanism for the people that are using your app, notice a bug and submit it, and then you're not robbing them. Yeah. Okay.

Joel Margolis (teknogeek) (17:42.847)
Okay, so let me pick your brain about that a little bit. So first of all, do all the submissions go through the VDP or are some people on both and they just...

Justin Gardner (@rhynorater) (17:50.964)
Yeah, you know, everybody who's in the private program just submits to the private program.

Joel Margolis (teknogeek) (17:56.383)
Okay, and so anything that's, you know, medium, high end of medium or above and should be in the main program gets moved from, it came in through VDP, gets moved to the main program and then, okay.

Justin Gardner (@rhynorater) (18:06.388)
Exactly, and then they pay it off, they pay the researcher for that submission even though they submitted to a VDP. Which I think is kind of a good idea, right? Like, to be honest, I'd be pretty happy if for some reason I submitted a bug to a VDP. In some world, Justin submitted a bug to a VDP, and it got moved into the bug bounty program and I got paid for it, I would have a ton of brand affinity for that target, right?

Joel Margolis (teknogeek) (18:13.119)
Okay, so how do you feel about that from -

Joel Margolis (teknogeek) (18:32.863)
Yeah. Okay. So, so how, as a hacker, how do you feel about that process of not only having sort of two places to report, but...

getting paid, like regardless of the VDP existence, like if you submit a good bug, you're still going to get paid. Like does, does that still annoy you in the same way that they have a VDP?

Justin Gardner (@rhynorater) (18:55.476)
You know, so I said in the beginning, I really don't like that structure. And I think what I don't like about it is the fact that if you had known somebody who's in the public program, or the private program, then, and you sent them a DM saying, hey, I got a bug, can you submit it? You know, and they submit it to the program, then it's like, okay, so why can't I just submit it to the program? Or like, you know, you DM someone and say, hey, you know, I know this program's got like a private program, can I get added to it or whatever, I found a bug there.

And you can't really submit a bug to the VDP and be like, hey, I've got a bug, but I know you've got a bug bounty program, so give me access to that. Because that just feels extortiony, right? So it's like, I don't know. The times when they don't pay the researcher is not great, but the times when they do pay the researcher anyway, I'm actually not super opposed to that structure. I don't know. Maybe that's off base, but. Mm -hmm.

Joel Margolis (teknogeek) (19:49.599)
Okay, so here's a follow -up. Let's say they had a public paying bug bounty program. They merged the two, but they had the same policy. Lows and anything under 5 .5 or whatever, CVSS, zero bounty. They'll accept everything, but they'll only get bounties for things that are medium and above.

Justin Gardner (@rhynorater) (19:56.692)
Mm -hmm.

Justin Gardner (@rhynorater) (20:02.035)
Mm -hmm.

Justin Gardner (@rhynorater) (20:14.612)
Yeah, I mean, I don't love it, to be honest with you. You know, it's the same thing, but I don't love it, you know? Feels bad.

Joel Margolis (teknogeek) (20:18.751)
Okay.

Okay. Why? Because they might as well just pay for it all if they're going to pay for some of it or?

Justin Gardner (@rhynorater) (20:26.1)
It's just not competitive at that point. It's like, why would I spend time on this? Because there's plenty of programs that pay for lows and mediums, and you know I'm a big medium advocate. I love a good medium, man. I love a good medium. So I don't know.

Joel Margolis (teknogeek) (20:39.295)
Okay, what if the incentive is like high enough? Like, let's say like, you know, lows and mediums, hypothetically, flat payout for lows and mediums. Let's say it's like 500 bucks and then like high ends of mediums and highs started like 5K.

Justin Gardner (@rhynorater) (20:55.572)
You know, if they're paying 500 for a medium and this is a newer program, you know, it's not the worst. My standard is 750 for a medium. It's moving towards 1K for a medium, actually. But, yeah, I don't know, man. I feel like when you pulled that, you know, hit me with that reverse or no card there, and I was like, oh, wait a second, I would hate that if it was a public bounty program with no.

lows and medium bounties, it kind of opens the eyes. I just think the VDP is like, I think it makes sense. That structure makes sense for people that don't want to get overwhelmed, but also want to pay researchers what they're due. And so that's kind of why I like that approach. Does that make sense?

Joel Margolis (teknogeek) (21:41.247)
Yeah, I think it's fair enough. I was just kind of curious from the full -time hacker's perspective of what, you know, because it's basically the same thing, right? It's basically the same thing because basically they're still not paying for loans and whatever, but they're just being transparent that if something higher comes in, they're gonna pay for it. But yeah, I think it's.

Justin Gardner (@rhynorater) (21:46.9)
Mm. I would never look at this. Yeah.

Justin Gardner (@rhynorater) (21:58.612)
But I don't know, man, there's a different thing because the full -time bug bounty hunter, this is a different conversation because most companies aren't gonna be on our radar. The only people that are gonna be on our radar are people that are really top -level programs, right? Because we're gonna be working with them long -term, we're gonna have a committed relationship with that program. And if you're not ready to pay up for that,

Like I've been talking to some programs recently, I've said, hey, look, I love your program. I use your product all the time, you know, normally, just as a part of my life. I would love to hack you because I know this product inside, outside, upside down, and I've got all my data in it, and I've got everything configured, right? Be awesome. Your boundaries aren't there. You know, I can't do it because of this, this, this, you know? And it's just, it's a shame, but it's also where some programs are at maturity -wise is they...

they can only pay to incentivize mid -grade hackers, and that's fine.

Joel Margolis (teknogeek) (23:02.015)
Okay, now here's, this is gonna get a little deep. There is a pro, yeah, sorry.

Justin Gardner (@rhynorater) (23:05.908)
We're spending longer than I anticipated on this target, but you know, I'm gonna let you cook because you're a program manager, so this is like...

Joel Margolis (teknogeek) (23:12.255)
Yeah, no, I think this is a good conversation. So I will say like, there's definitely a weird balance coming. I've noticed, and I don't think many people have noticed it, but I certainly have noticed it between like. Revenue budgets, inflation. And bounty tables. Okay. And you could probably already sort of piece those things together, but for one, the economy is like doing all sorts of weird, weird things. Right.

Weird, weird things right now. Inflation is still through the roof and continuing to go up at like a really crazy rate. Bounty tables are also inflating, in my opinion, quite significantly. Not fast enough from the hacker's perspective, but way too fast from a program perspective. And...

Justin Gardner (@rhynorater) (23:49.108)
I haven't checked that recently. That's interesting.

Justin Gardner (@rhynorater) (23:55.092)
Not fast enough.

Justin Gardner (@rhynorater) (24:00.852)
What? Shnola, don't open up this can of worms right now. Are you kidding me?

Joel Margolis (teknogeek) (24:06.175)
Well, I mean, just think about this, OK? Think about this. Criticals, a good critical on a bug bounty program in 2019 would be 10K. Like, that's like the high end of crits. Like at most of programs paying 20K crits, OK? Like top, top programs 2019. 2024, there are plenty of companies who offer at least 50K crits, 100K crits.

even higher, especially for the crypto programs.

Justin Gardner (@rhynorater) (24:38.516)
Yeah, but that's okay. A solid crit payment right now is 20K. Like I will look at a program that has a 20K crit. If you want to get above a 20K crit, that's when you get really into the like, okay, this is on the hit list. Like I will hack this program at some point, you know? But the 50K ones, I almost, and I've gotten a couple of 50K bounties and it's great and it feels awesome, but they are hard to get the companies to pay.

Joel Margolis (teknogeek) (24:56.767)
It's true.

Justin Gardner (@rhynorater) (25:07.892)
and they are oftentimes on very specialized scope, right, which is two downsides.

Joel Margolis (teknogeek) (25:13.151)
Yeah, so I will say, I think the hard part is not so much that most people are ending up paying those critical bounties. I think it's more that because the table ends up there, the next levels have to be significantly higher, right? So high vulnerabilities starting at 10 or 15K and mediums starting at five or two and a half, right? Those are, yeah, you love to see it, but...

Justin Gardner (@rhynorater) (25:36.5)
Mm -hmm. You'd love to see it. You'd love to see it!

Joel Margolis (teknogeek) (25:41.727)
Then from like a program perspective, you make one little slip up. There's one like medium severity vulnerability and it's like thousands of dollars out of the budget. Right. And so in order to be competitive, like you said, like as a top level hacker, you see a program that's paying like.

3K, 5K, I don't know, is 5K crits enough for you to hack or not anymore?

Justin Gardner (@rhynorater) (26:03.796)
So 5K crits I will not normally hack on unless I have some other incentivization.

Joel Margolis (teknogeek) (26:11.487)
Right, so that's like still pretty good. Like that's probably the average payout for most programs that don't have a ton of budget. And you know, they're basically off the radar for a lot of the top hackers because they can't compete financially with the bounty tables that the really big programs have. And so they just can't attract that attention. And so it's just a very, very difficult situation, I think. Like I'd love to see bounty tables continue to grow and...

Justin Gardner (@rhynorater) (26:19.796)
see that.

Joel Margolis (teknogeek) (26:39.743)
companies to continue to put more and more money behind, you know, their security teams and all that kind of stuff. But then either those single slip ups or in order to stay competitive and have an edge against other programs, it's like, I mean, there's always that like classic joke, right? Of like the lemonade stand, right? Where you're like, there's two guys selling lemonade and one guy's at like 40 cents and the other guy's at 50 cents. And so then the guy next to him drops his down to 40 cents and then the guy,

Justin Gardner (@rhynorater) (26:58.42)
Mm -hmm. Mm -hmm.

Justin Gardner (@rhynorater) (27:08.404)
Mm -hmm.

Joel Margolis (teknogeek) (27:09.311)
Next to him drops us to 30 cents and then 20 cents, 10 cents. And then the guy next to him buys all of his product, puts him out of business and raises it back to 40 cents. Right. And so it's just like that kind of the same thing, but without like the acquisition costs so much where basically there's a race to the top for bounty tables where the programs who have highest bounties and the least vulnerabilities get the most attention or like close to the top and.

Justin Gardner (@rhynorater) (27:17.204)
Right, right.

Joel Margolis (teknogeek) (27:36.831)
a lot of scope, you know what I mean? Like there's a balance between scope and bounties where you're on the, yeah, Epic games. You know, you know what I'm saying? Yeah, yeah, yeah. Yeah. Other examples, Shopify, you know, but yeah, there are definitely, it's kind of like a race to the top with the bounty tables where the higher the bounty table, the more attention and sort of stuff you get.

Justin Gardner (@rhynorater) (27:38.388)
Mm -hmm. Yeah, we see that. Epic games, you know, like, yeah.

Justin Gardner (@rhynorater) (27:45.588)
Let's, let's, we gotta be careful about what we say. I know what you're saying, I know what you're saying.

Justin Gardner (@rhynorater) (27:54.9)
Yeah.

Joel Margolis (teknogeek) (28:05.311)
but then it becomes even harder for smaller programs to compete with that at all because there's so much attention being garnered from these 15K highs, 5K mediums, right? And so it's like, yeah. It is part of the game, but you know, we're also simultaneously saying, oh, you know, like, fuck VDPs, like, you should be a paying program. And then they become a paying program. And it doesn't even matter because they can't even compete, right? Like they can't even, like they can't put.

Justin Gardner (@rhynorater) (28:14.26)
How it is though, man. That's a part of the game. Like, you know...

Justin Gardner (@rhynorater) (28:24.5)
You know?

Justin Gardner (@rhynorater) (28:28.692)
And they were like, screw you, we don't have...

Joel Margolis (teknogeek) (28:34.527)
20k crits down. So like yeah, it's it's a definitely a challenging

Justin Gardner (@rhynorater) (28:36.244)
No, but at least you're gonna get the mid -tier hackers though. And there are some very, very, very talented mid -tier hackers. And this whole hacker tier system that I'm talking about is totally arbitrary and essentially a figment of our imagination. Every hacker has their unique eyes and are gonna be able to find something that other hackers can't find. We see that every single live hacking event. But.

Yeah, I mean, you're still gonna attract some hackers that have lower standards. Dude, if I lived in Argentina, if I lived in India, if I lived in some of these places where the amount of money you need to spend to have an awesome life is so much less, then I would absolutely hack on a 5K crit program, because it's like, okay, that 5K crit results in this much spend for me, which is the lifestyle that I wanna live, and there are tons, I know many, many, many.

Joel Margolis (teknogeek) (29:24.703)
Yeah.

Justin Gardner (@rhynorater) (29:34.58)
hackers that are very skilled doing the digital nomad thing where if they get 5k they're set for, you know, two months. And it's like...

Joel Margolis (teknogeek) (29:41.599)
Yeah, I will say I think that's probably one of the hugest life -changing aspects of Bug Bounty. It's always been life -changing amounts of money to some extent, depending on where you are. But I think especially in countries that are a lot more, their economies are struggling a lot more, they're a lot less developed and stuff.

Justin Gardner (@rhynorater) (29:49.012)
Mm -hmm.

Justin Gardner (@rhynorater) (29:52.948)
Mm -hmm.

Justin Gardner (@rhynorater) (30:02.932)
Hmm.

Joel Margolis (teknogeek) (30:05.599)
what we would call a small bounty, right, like $500 or something. I mean, walk up to anybody and give them $500 outside of the US and even in the US, right? It's like, it's a huge amount of money, right? And so I think bug bounty is like doing a ton in that area, but there's still too many BDPs, right? Like you don't wanna be taking somebody who has that kind of talent and is struggling and then abusing that.

Justin Gardner (@rhynorater) (30:13.204)
Mm -hmm. Yeah, yeah. That's true.

Joel Margolis (teknogeek) (30:30.207)
Like their time and their energy and stuff and not giving them money, even if it's a small amount of money that's going to change their life, right? Like give them the $150 or whatever that, that you were going to spend on team lunch or whatever the fuck. And like, you know, like change somebody's life instead of having a VDP. That's, that's my call to action.

Justin Gardner (@rhynorater) (30:37.716)
Mm -hmm.

Justin Gardner (@rhynorater) (30:41.556)
Yeah.

Justin Gardner (@rhynorater) (30:45.012)
Yeah, yeah, yeah, yeah, I agree, man. No, it's good, it's good shit. And it's funny, you know, I talked to somebody that got actually a massive bounty and lives in one of these countries that it's like, this is literally like hitting the lottery. You know, it's like, you know, your life has changed forever. Your family's life has changed forever. It's like, it's crazy. So really exciting for people living in those regions of the world. And...

Joel Margolis (teknogeek) (30:58.527)
Yeah.

Justin Gardner (@rhynorater) (31:10.452)
Yeah, for anybody who's willing to make that lifestyle switch to go and live in those countries because, you know, lots of awesome opportunities there. I'm looking at, no, no, you're good, man. I'm looking at your program here, man. You got the Tinder program, 10K highs, you know, 2K mediums at minimum, 10 to 20K, all right, all right, I see you. I'm sorry, man, I see the thing is I don't use Tinder.

Joel Margolis (teknogeek) (31:20.447)
Sorry, I didn't mean to derail that too much.

Joel Margolis (teknogeek) (31:31.359)
I'm just saying, um, I'm just saying what do I need to do to compete?

Justin Gardner (@rhynorater) (31:37.908)
Like if I was a single dude that uses Tinder, which there's two big conditions that are not fulfilled here, then I would absolutely hack on it. But I don't, so that's really the reason.

Joel Margolis (teknogeek) (31:41.855)
Yeah.

Joel Margolis (teknogeek) (31:51.999)
Yeah, well, all I'm saying is I think we both known that sometimes those barriers make for great programs to hack on because it deters a lot of people. So even if you are a non -single person who doesn't use Tinder, if you're willing to get on Tinder and use Tinder and try and hack it, you know, you'll have an edge on other people, I think. Yes, honey, I, yeah, yeah.

Justin Gardner (@rhynorater) (32:13.012)
just make sure you tell your wife in advance. She's gonna get some weird DMs from your single friends like, um, Justin's trying to, oh that's great.

Joel Margolis (teknogeek) (32:21.631)
Yeah, yeah. Okay, so we actually have another kind of like program hacker debate topic here, don't we?

Justin Gardner (@rhynorater) (32:28.788)
Okay, hold up, hold up, hold up. I actually, I'm sorry, I'm not done with this Nogli tweet. Nogli, you screwed us here, man, because now we're talking 30 minutes into this podcast, still on the same thing. What do you think about the leaderboard thing? Like, I thought that was, I think that is one of the things that needs to change, absolutely. It's like, you know, this whole gamification piece of Bug Bounty is awesome and is a major, major, major motivator for a lot of the hackers. And...

Joel Margolis (teknogeek) (32:33.119)
Oh, okay.

Justin Gardner (@rhynorater) (32:57.94)
hackers that spend times on hard, like hackers like myself and others that spend time specifically looking for the most hardened programs they can find, you know? And try to hack on these programs are not seeing as good of a return or as good of placement on these leader boards, or even people that just hack bug -binding programs in general. Because of the presence of VDPs, and somebody commented, I wanna say it was.

Let me look down here. I want to say it was PQhacker from the Discord. Yeah. And said that essentially the number one hacker on Integrity right now too is a VDP only program. And I saw them kind of get into it on Twitter the other day. There's like a Twitter argument about like, you know, you found 28 bugs or whatever and I found 28 bugs this morning. You know, and it's like, who's the better hacker? And I'm like, the guy that's hacking.

the paid program for sure. So I don't know. What are your thoughts on that?

Joel Margolis (teknogeek) (33:59.967)
Yeah, so I also like gamification is fine, right? I think everything's going to get gamified. If they don't do it, some other hackers going to do it, right? Nagley's intelligence network or whatever. He's going to build his own leaderboard and whatever. But that being said, these are not the same thing, right? VDPs and by branding programs, I don't think are the same thing. And the fact that there is this.

Justin Gardner (@rhynorater) (34:10.388)
Mm -hmm.

Yeah, exactly.

Joel Margolis (teknogeek) (34:28.511)
alternative method of earning the same points. Like it doesn't feel right to me. If we're going to have points, let's keep it what it is, right? And let's either delineate it separately, bug bounty reputation, VDP reputation, VDP signal, VDP, you know, whatever, or don't even include it entirely. Do what Bug Crowd did, remove the points aspect from it.

Justin Gardner (@rhynorater) (34:33.812)
Mm. Mm.

Joel Margolis (teknogeek) (34:55.295)
You don't get any points for any of that stuff. It doesn't affect your stats at all. If you want to be a good guy hacker and you want to submit stuff to VDPs and you want to get your practice on there, that's great, but you're not going to get any stats for that because that's not bug bounty program reflective stats. That's VDP reflective stats.

Justin Gardner (@rhynorater) (35:05.748)
Yeah.

Justin Gardner (@rhynorater) (35:10.964)
Yeah. Yeah, and you know, I'm not, I think it should be a separate leaderboard, I think, you know? I think you should get Bug Bounty rep and you should get other rep because it is a good resume builder. Yeah.

Joel Margolis (teknogeek) (35:20.831)
Yeah, like why hide it, right? Like, yeah, if you want, if you spend a bunch of time doing that, that's fine. But I think to make, to combine them is a little bit misleading, both to people who look at the leaderboards, but also to the people who are like trying to get up on the leaderboard, right? You could go hack on a bunch of VDP programs that are, have much more scope, lower barrier for entry, like easier to farm points, all that kind of stuff. And you'll see exactly what's happening. The people on the top of the leaderboards or a lot of the people who are in the top of

Justin Gardner (@rhynorater) (35:34.1)
Mm -hmm.

Joel Margolis (teknogeek) (35:49.663)
of the leaderboards are VDP hackers. They're people who are hacking on VDPs, they're earning a ton of points from VDPs, and to try and compete from a purely bug bounty program perspective, extremely, extremely difficult. Like you would have to go out of your way to probably hack on VDPs just to get points so that you can compete on the same number of reports and all that kind of stuff if you're just a bug bounty program hacker. And again, it's fine to hack on VDPs.

Justin Gardner (@rhynorater) (36:10.292)
Mm -hmm.

Joel Margolis (teknogeek) (36:17.183)
Get the points for it, that's great. But from the programs, or sorry, I keep mixing this up, from the platforms, HackerOne, BugCrad, they should be listing these things separately. Don't put somebody's 10 ,000 rep when 8 ,000 of that is from the DoD program and 2 ,000 of it is from paid programs. Put that separate so that people can see, oh, they've got a ton of rep from this. I think this actually extends even further if we're gonna talk about leaderboards. This is a complaint I've had about this as well.

Justin Gardner (@rhynorater) (36:21.62)
Hmm.

Justin Gardner (@rhynorater) (36:44.98)
Mm.

Joel Margolis (teknogeek) (36:47.295)
the program leader boards do the same exact problem, right? You go to any program, you look at their leader board, you see who's on the top of the leader board, right? Top five, top 10. Gonna have thousands and thousands of rep. But if you go and look at the thanked hackers and you look at this year, you could go back, I've done this like five years back, six years back, until I actually hit when they got 2 ,000 rep on that program, when the top hacker, and the hacker hasn't hacked on this program since before COVID.

Justin Gardner (@rhynorater) (37:11.956)
Yeah.

Justin Gardner (@rhynorater) (37:16.5)
There we go.

Joel Margolis (teknogeek) (37:17.055)
And they're still at the top of the leaderboard because they farmed the hell out of the program. When it first launched, they got a ton of rep and they haven't touched it since. And you're looking at this and you're like, wow, how am I ever going to get 4 ,000 rep on this program? And it's, it's just very hard to get like an accurate, right? It's a different game. It's a different perspective. And like, yeah, all time stats, that person really crushed the program, but so much has changed since then. They might have a different scope. They might have a different bounties. They might have a different everything.

Justin Gardner (@rhynorater) (37:29.492)
Mm.

It's a different game.

Joel Margolis (teknogeek) (37:46.175)
and you're still seeing this person as the number one hacker on this program. And I think like it's a similar aspect where it's really hard, like you can infer a lot of things. And I think a lot of people are inferring the wrong things when they see these, right? You look at the top leaderboard, you see somebody with 10 ,000 rep, you're thinking this person hacks a lot. They make a lot of bounties. They're getting a lot of really valid reports. If you were to see the breakdown of BBP versus VDP rep, it's probably going to tell a very different picture. And the same thing goes for, you know,

like the rep on the top hackers on a program. Let's look at how much rep they've gotten in the last 90 days or the last 360 days, 365 days instead of all time. And it's gonna tell a very different picture.

Justin Gardner (@rhynorater) (38:20.596)
Time boxed it.

Justin Gardner (@rhynorater) (38:28.692)
I think this is a great point for Hacker One too, is like that top hacker, they moved that to the bottom by the way, I don't know if you noticed that, but like, I don't love that. But they moved it to the bottom and it defaults to top hackers of all time. I think it should default to the last 365 days, because, and I think Hacker One would really benefit from that too, because common mistake in the beginning, everyone made it, like you mentioned, was going there and being like, oh, you know, freaking today's new has like, you know.

Joel Margolis (teknogeek) (38:35.615)
Yeah, I did notice it.

Justin Gardner (@rhynorater) (38:57.204)
a thousand rep on this program, I'm never gonna find anything. You know, one, flood mentality, we've talked about this many times, we're not gonna repeat it again. But two, he hasn't touched the program in a year or two years, and so if you do that running leaderboard, one, that incentivizes you to continue to hack on a program if you wanna maintain your leaderboard status, but also it doesn't scare away those people that are just not gonna...

look at the program if some top hacker has been looking at it in the past year or whatever. So I think it could be a really good move for HackerOne and I think it would help sort of solidify some of that, yeah.

Joel Margolis (teknogeek) (39:35.231)
Yeah. And what I'll say is just as a, as a recent example of this with the public program, Shopify, okay. You go to Shopify's all time leaderboard, their top hackers, H1 3 dash. He's got 3 ,900 rep all time. You set that to 2024. He's not even on, he's not even listed. You set that to 2023, not even listed. You set that to 2022. Now he's listed. He's got 680 rep. Right. So we're two years out. And overall year and a half, whatever. And.

Justin Gardner (@rhynorater) (39:44.34)
Mm -hmm.

Mm -hmm.

Justin Gardner (@rhynorater) (39:51.156)
Mm -hmm.

Justin Gardner (@rhynorater) (39:58.1)
Mm -hmm.

Joel Margolis (teknogeek) (40:05.983)
that this hacker hasn't even, they're still at the top of the leaderboards, right? So this, it's just not a very accurate way of sort of looking at things. And I think it's a little too broad and generalized in terms of like how a lot of stats are being shown and how the leaderboards and stuff work. And yeah, I think it would be good to see a little bit more separation and granularity on that data from.

Justin Gardner (@rhynorater) (40:09.204)
Mm -hmm.

Joel Margolis (teknogeek) (40:31.903)
just like a user's perspective of if I visit someone's profile, maybe only show the bug bounty program rep and stats by default. And if I wanna see their VDP rep and stats, there's a dropdown or split it and show them separately. Right, right, show both at the same time or yeah, whatever, right? Like there's gotta be a better way to show this that's not as confusing and not as easy to sort of game, right? Because I think...

Justin Gardner (@rhynorater) (40:40.884)
Mm.

We're half slash VDP or something like that, you know, like, hmm.

Justin Gardner (@rhynorater) (40:56.948)
Hmm.

Joel Margolis (teknogeek) (40:59.007)
A lot of this boils down to people are gaming this in a negative way that's not fair to the people who aren't gaming it.

Justin Gardner (@rhynorater) (41:06.868)
Yeah, yeah, I agree. And so, yeah, I think that leaderboard piece could be really cool. I got a little nerd sniped for a second, you know, because I looked at the hacktivity on Shopify and I see our boy Matan Bear is just crushing it out here. This man is getting bounty over bounty over bounty. So nice job, dude, rocking it. All right, cool. So you know what? I think we're going to cut Nogli's tweet off there. Nogli, thanks for 40 minutes of content this week. Jeez, dude.

Joel Margolis (teknogeek) (41:17.055)
Yeah, yup, yup.

Joel Margolis (teknogeek) (41:32.319)
40 minutes, yeah.

Justin Gardner (@rhynorater) (41:35.892)
Okay, so yeah, let's continue the whole program versus hacker debate version two. Somebody was asking about this in the Discord too, so it's good we fell into that. Let me ask you about this. So I've run into this situation a couple times in the past where there is a...

very common issue that happens across many, many endpoints on a target. And this issue is, let's say there's an issue more often than there's not an issue with this target, with this specific set of endpoints with this specific vulnerability. But there are endpoints that have the correct access controls in place.

So clearly there's some access control system in place. What do you think about?

making it like, seeing 20 reports come in for this specific thing, and realizing, hey, our current implementation is flawed, and really we should just shove this off in middleware, or something like that. But then you've gotta undo all of the code in those other functions, and then put the middleware in, and then,

So it's sort of a global fix, but also you're undoing code in functions that were not vulnerable, whereas the normal standard would be like, okay, if I changed some code in this specific function for this specific endpoint, then it's a fix for that endpoint, and that gets a unique bounty. That's the normal standard. So rather than that, you fix at the middleware level, and then you remove the previous flawed checks from the endpoints that were not vulnerable.

Justin Gardner (@rhynorater) (43:36.148)
I feel like that's sort of a sneaky way to get around it. And I feel like what would happen is if you sent in one report at a time, then they'd be like, oh, I just need to copy and paste this access control check from this function into that function. Then they fix it in that individual function. Then you send the next one. Then you send the next one. And so do you continue with this sort of incomplete or sort of flawed model? Or do you try to take a different approach? And if you learned from...

the hackers submitting 20 reports that this is a massive problem, do you pay them for all 20 reports? And then, I don't know, what are your thoughts on that? You see what I'm saying? I don't know if that was clear.

Joel Margolis (teknogeek) (44:16.991)
Yeah, yeah, yeah, yeah. Yeah, so I'll talk generically about this. So my stance is that generally these things kind of end up working themselves out almost the same way. And so typically what happened, let's run through both scenarios. The first scenario is that you have this semi -systemic issue, you find it in a lot of different endpoints, you're submitting them one at a time. As the program gets them, they pay them basically, let's say the same bounty for every single one.

And that's sort of how it goes. The other way is that you submit, or maybe there's three options. The second option is that you submit all 20 at the same time. And they, or is that, are those just the two options for submissions? Either you slow roll them or you do them all at once. Is that?

Justin Gardner (@rhynorater) (45:05.14)
So the slow roll, I think it's pretty clear that if they fix it globally from the slow roll, then it's not a global issue. It's not an issue that you should get paid for each one of them.

Justin Gardner (@rhynorater) (45:23.412)
But then, on the other hand, there is that aspect of like, okay, well we looked at the code and we found 20 other functions that you couldn't see because they're not in the JS files and we don't have any API docs leaking, you know, that was vulnerable and this is actually a massive issue. I don't know, man, it's tricky.

Joel Margolis (teknogeek) (45:23.583)
So the problem, yeah.

Joel Margolis (teknogeek) (45:38.175)
So that second, yeah, that second scenario, like if the program discovers significantly more impact that it was a systemic issue they didn't know about, I would say nine times out of 10, nine and a half times out of 10, 9 .9 times out of 10. Program is never gonna even let you know that that was a thing and they're gonna fix it. No, hell no, hell no, no, no, I mean, obviously it's in the program's best interest to pay as little money as possible.

Justin Gardner (@rhynorater) (45:46.356)
Mm -hmm.

Justin Gardner (@rhynorater) (45:55.604)
Yeah, I thought you were gonna say the opposite. I was like, do not say that. Like.

Joel Margolis (teknogeek) (46:07.871)
Right? Like that's, that's where the incentive lies. Um, uh, that's just, you know, economics, right? It's the hacker's goal to earn as much money as possible. It's the program's job to pay as little money as possible. Right? That's just how the economics work. Um, that being said, there is like ethics and ethics side here. Right? So I think like a really highly ethical and transparent program, which are very few of who go to this length.

Justin Gardner (@rhynorater) (46:12.5)
Dude, what is wrong with you, man?

Joel Margolis (teknogeek) (46:37.567)
would be like, hey, as we were just looking into this, we found that this is actually a really huge problem, so we're going to pay it out. That is rare for a lot of reasons, mainly like money, because it's literally just like lighting extra money on fire and like going to find an impact where you're going to pay more money. You know, it's it's rarely a good. Justification when you're like trying to explain why you paid 50 instead of paying like 20, you know what I mean?

Yeah, so there's that aspect to it. Like that scenario. However, I think, as I was saying, most of the time these sort of merge into the same sort of resolution because if you submit 20 of the same vulnerability, then they for sure know, okay, like there's some sort of weird systemic thing here. We need to look into this and figure out what's going on. And they're gonna fix it systemically. And most likely, I would say the program's gonna...

pay a higher bounty, but close those all as one issue because it's going to get fixed in one place. If you were to slow roll them, initially they're going to fix them one at a time. And then after like four, they're going to be like, this is probably systemic. And they're going to look into it. Well, they should look into it, find that it's systemic. And then every single report after that, they're going to say, okay, this is a known issue. We're working on fixing this. We're not going to pay them out as additional bounties.

And then it's probably close to the same bounty depending on how many reports you manage to sneak through or what they pay for it. Right. And maybe you as a researcher, you can make that case and be like, hey, you know, I actually had a bunch more of these. Like, can we agree on, you know, paying like a portion of, you know, like the total that I had found or what, you know, I don't know that that's a little more complicated. But I think generally the program, if they're doing their job well, like after

Justin Gardner (@rhynorater) (48:11.988)
Yeah, hopefully. Yeah.

Joel Margolis (teknogeek) (48:34.207)
four or five of the same type of report on like, you know, systemic thing, they should be seeing, okay, this is weird. This looks like it's systemic. Let's figure out what's going on here. And again, we're back into the incentive model here where like the program is gonna try and want to try and get on top of it. They're gonna try and figure out what's going on in the backend. They want to create tickets to make sure that they're getting ahead of this so that when more reports come in for that same issue, they can say, oh, we're actually already working on a fix here.

Justin Gardner (@rhynorater) (48:59.028)
Dude, can I just go back to your whole, like, the program is incentivized to not pay? Just, like, dude, okay, so here's what I don't get. I understand the whole ethical conundrum of, like, okay, I have been entrusted with this responsibility by my employer, and I should be giving my employer, you know, my best effort, and that sort of thing. But also, these are massive, massive companies that have, you know, massive, massive revenue and stuff.

And these researchers are doing quality research that are fixing vulnerabilities. This is not like we paid 500 grand for Nessus. You know, it's like, this is actual, here's a vuln, a valid vuln you fix, here's the money. You know, it's a pretty clear exchange and it's high value, as in you're getting something worth money for what you're paying. So I don't understand people's hesitancy to push around.

And maybe this comes from my experience, you know, not having to deal with finance, but, you know, pushing around and saying like, hey, listen, we have technical, you know, debt. We have problems that we need to fix that were taken from improper architecting and cutting edges, you know, sometimes whatever it is in code development. And that's a price we need to pay. And it's not something that you can really say.

no to because it's putting users at risk and that risk is much greater than the 2k we're gonna get. So I just, like if I were on your side, I feel like I would say I am fighting for the researcher to pay them as much as I can because they're producing value and I can see that and I can validate that and I don't have as much high hopes for other money that's being spent in the organization about whether.

you know, Jim is spending two hours a week at the freaking watering, you know, talking to people and shooting the shit and then going to his company paid lunch for, you know, two hours. It's like, maybe I'm being too pessimistic on that, but I feel like there's so much BS in so many companies, especially when you get above a certain size, where people are not actually doing anything for the money they're getting. And...

Justin Gardner (@rhynorater) (51:25.076)
Bug bounty is always gonna be a here's something worth value.

Joel Margolis (teknogeek) (51:29.599)
Yeah, so bug bounty is like a weird, it falls into a weird spot, right? Because it's both, it assigns a financial value to security problems and it puts a really hard dollar value on mistakes. And we try not to assign blame on things, but it makes it really difficult not to when you pay a $10 ,000 bounty.

And then it's like, okay.

Justin Gardner (@rhynorater) (51:58.74)
But you're not paying that. The company's paying that. I know you've explained to me how this works, where you get a specific amount of budget or whatever, but.

Joel Margolis (teknogeek) (52:02.111)
Well, okay, so think about it this way. Yeah, no, but think about it this way. No, no, no, no, no. It's not even about the budget side, right? Think about it this way, okay? You paid $10 ,000 because a security vulnerability was found. Who should have found that?

Justin Gardner (@rhynorater) (52:17.812)
Okay, I see where this is going. You know, maybe the security team or a code review or something like that, or the person that wrote it.

Joel Margolis (teknogeek) (52:22.815)
Yeah, yeah, and when should they have found it?

Justin Gardner (@rhynorater) (52:26.1)
when they were developing it.

Joel Margolis (teknogeek) (52:28.031)
Yeah, there's actually like multiple places where they probably should have found it, right? Like during review, during like pre -review, during pre -release, like during post -release. Like there's many, many different spots where like the security team probably had opportunities to identify that vulnerability and didn't. And now there's...

Justin Gardner (@rhynorater) (52:43.7)
Or you, is that what you're saying? Or the security team?

Joel Margolis (teknogeek) (52:48.511)
Well, right. Like, yeah, the security team or the mostly like the security team as a whole, we'll just say like somebody somewhere within the security team probably at some point should have had an opportunity to identify that if they didn't. The process is not great. And that like speaks to other things. Right. But there's like this sense of blame where it now there's like a monetary value on this vulnerability where somebody missed it. And now that cost a lot of money. And I think.

That makes it a little bit easier to justify to leadership in terms of like, hey, there's a real cost of security where if we don't find these vulnerabilities and we don't have the processes in place to be looking through this stuff properly, then it's gonna cost us a lot of money and vulnerabilities. But then there's also the side of,

I want to pay the researcher at least as what's fair, but ideally as much as possible. Right? And so those two things are, are like hard to balance because paying what's fair can be really tricky because maybe there's like a legacy problem. There's a lot of vulnerabilities you didn't know about. Now you, you have like a budget problem, but paying as much as possible is also like a hard budget problem because now you're like,

We want to pay as much money, we want to create as much spend on these problems as possible to try and incentivize them to be fixed. But all it really does is make finance really annoyed that you're spending so much money.

Justin Gardner (@rhynorater) (54:19.092)
You know, who cares? Who cares about annoying finance? Like, I mean, maybe this is my, this is very, probably, I will preface this by saying, this is from a very privileged position of I have not ever had to deal with corporate crap. I did consulting for two years and I've been self -employed ever since and I never am going back. But also, who cares? Because at the end of the day, it's not, like, I feel like Bug Bounty should have that, you know, that check of like, listen.

If you don't pay the money, then you're accepting the risk implicitly that your users are gonna get screwed. And that is so much more expensive. Look at the (REDACTED) hack, right? Total overall cost for that, like $110 million. (REDACTED) has run a bug bounty program for years and years and years, and they've only paid out what, like 12 million? It's a drop.

Joel Margolis (teknogeek) (54:56.735)
So.

Joel Margolis (teknogeek) (55:16.735)
Right. So I think cyber, like, you know, cybersecurity insurance and stuff has also like all of the ways of putting monetary incentives and values on parts of cybersecurity and like hacking and stuff has created direct incentive models where it's easier to justify to the company why this spend needs to exist and why it is what it is. And I think that's, that's very good. But it's still like a hard, like you still have to sell it, right?

Because at the end of the day, it's always going to be a financial burn. Where not only are you like doing your security stuff, but then you're spending extra money to like pay extra researchers. And you know, you have to somehow, you know, justify that money. Like for finance, I mean, the reality is they'll just give you less money. You know what I mean? Like it's a very privileged, it's a very privileged. Yeah, sure. Sure. They control the budget, right? They say, you know, we feel like you're spending too much money, so you need to spend less money.

Justin Gardner (@rhynorater) (56:06.164)
I mean, can they just do that? Like.

Joel Margolis (teknogeek) (56:16.479)
And then the security team basically has to figure out, okay, if we're gonna spend less money, if we wanna pay the same amount in bounties, then we need to spend less money on other things, or we have to figure out how to pay less bounties.

Justin Gardner (@rhynorater) (56:26.964)
And who does that go up through? Because how I would respond to that is finance saying, hey, we're going to pay you less money. And then I would try to get some statistics that say, OK, this less money is going to result in this amount of decreased findings or this amount of, oh, we don't have enough time to go over x amount of code if you're even talking about normal security stuff. We need one more security engineer, and we're not going to get it. So it's going to be.

you know, less amount of time looking at this code, which means X amount of vulnerabilities introduced, which means X additional risk. And then you just push that up the line and you say, okay, listen, CEO, like, would you like to, you know, experience a 12, you know, 20 % increase in cybersecurity risk, or would you like to pay an extra, you know, 200 grand or whatever it is a year to get that down to a reasonable level? And then also have that in place with, you know, get, get, this is, this is like I said,

Could be totally off. But then you're putting it back up to the finance people. It's like, OK, listen, now there's a 20 % risk of whatever cyber incident at $10 million. You run the math. You know? Like.

Joel Margolis (teknogeek) (57:39.775)
Well, yeah, so I think it's very complicated and a lot of this is just completely hypothetical speculation, but I think a lot of finance teams would probably come back and say, well, why can't you just spend less on bug bounty? Why can't you offer lower bounties? We're still incentivizing people to hack, but why can't you pay 5K crits instead of 20K crits? It's still incentivizing and putting money where our mouth is, but why do we need to spend 20K on a critical when we could pay 20K for a critical or 5K for a critical?

And, you know, that's a, that's a difficult one. Or like, if you go over budget and you need to pull money from other places and it's like, okay, where do we pull, like, where do we decide that we don't need money for so that we can spend it on book bounty or, you know, like, so it's a very complicated, like game of chess. Yeah. Yeah. Yeah.

Justin Gardner (@rhynorater) (58:25.844)
This is why I don't want your job, dude. This is why I do not want your job. Because I would get in these meetings, I'd be like, listen here, like, and finance would not like me at all.

Joel Margolis (teknogeek) (58:34.239)
No, no, I don't think so. Yeah, so I mean, like this is like the weird, like complicated side of like the back end side where, you know, when you, I think you get in a little more respect when you look at programs who have paid so much money or pay like a ton of money all the time, because when you think about it from a cashflow perspective or even just a budgeting perspective, the fact that they're able to continually get that amount of money or get like, or have those types of funds to spend on Bug Bounty says a lot about where,

the company's head is at for security, right? Like I think all of the things that you want are totally reasonable and I would like them to, but it comes from a very privileged point of view from a security perspective where it basically assumes that like all the upper leadership, including the people in finance and accounting value security to like a high level and think that security is worth like an act, like a dollar value in enough to say like, okay, yeah, we're going to put like a million dollars a year or whatever towards.

Justin Gardner (@rhynorater) (59:25.62)
It's a budget item, you know?

Joel Margolis (teknogeek) (59:31.839)
paying bounties, not even like the platform fees or anything else, just like bounties, right? And that is a line item. And so I think, you know, it just, you have to have that sort of buy -in from upper leadership and all that kind of stuff and having everybody else think that it's as important. And again, I think this is why things like cyber insurance, for example, have started to make a bit of a change where when a company has a cybersecurity insurance policy that says that they're going to, you know,

If they have a breach, they're going to have to pay out like $10 million or whatever. Well, then depending on how much you're spending on bug bounty, that's way worth it because then you don't have to fall on your cybersecurity insurance policy and pay out a bunch of money and have all these other problems. There is an actual cost benefit analysis that you can do there, but when you're like, we just want to pay more in bounties.

Like it's very hard to justify like if finance, for example, like the situation I said, finances, oh, why can't we just pay 5k crits instead of 20k crits? How do you generate the data that says, well, by paying 5k, we're going to get less submissions. We're going to like all like quantitatively, we're going to get this many less submissions and that's going to account for like this much less money, but this much more risk. And that risk equates to this much potential cybersecurity risk. And.

Justin Gardner (@rhynorater) (01:00:44.596)
Hacker one, please.

Joel Margolis (teknogeek) (01:00:54.687)
Like there's so much like math, like you might as well like legit be like an insurance actuator when you're trying to calculate this kind of stuff. And it just, it's so complicated that I think a lot of programs just take what they can get and they pay what they can and they hold onto the money that they can. And when you have programs that have a ton of budget and a ton of respect for their security team and their bug money programs, it shows in how they can spend their bounties. But it's usually pretty obvious. I don't know.

Justin Gardner (@rhynorater) (01:01:22.356)
Yeah, yeah. It does, and you need the data. That is a good point, and that takes a lot of work to get, and it's not super readily available. I will say something, though, when you mentioned insurance. The insurance piece is really interesting, because I bet after a breach, your insurance premiums go up a lot. Yeah, exactly, right?

Joel Margolis (teknogeek) (01:01:22.879)
I hope that answers that. That was a hard one to answer.

Joel Margolis (teknogeek) (01:01:49.119)
What happens when you get in a car accident? And it's your fault.

Justin Gardner (@rhynorater) (01:01:52.596)
I bet that 110 million number that a lot of people saw that number, I think for MGM. I think I'm remembering the number right. And we're like, well, that's crazy. And then other people were like, oh, it's just covered by cyber insurance. I'm like, all right, but then you're cyber. Yeah, yeah. And so, cyber insurance, that amount is gonna go up substantially. And so, I don't know. It's crazy. It's definitely a trade off.

Joel Margolis (teknogeek) (01:02:05.119)
Well, and MGM has a paying bug bounty program, right? Uh -huh.

Joel Margolis (teknogeek) (01:02:19.839)
I would love to see the calculations that they're doing. I would love to know how they're waiting. This is what actuaries do, right? They do these really complicated mathematical probabilities on all these different things happening. I'd love to know what sort of factors that they're thinking about when they're calculating that kind of cybersecurity insurance stuff, because it would be really interesting to see how they're weighing the risk, just like kind of what I talked about.

Justin Gardner (@rhynorater) (01:02:24.436)
Yeah. Mm -hmm.

Mm -hmm.

Joel Margolis (teknogeek) (01:02:48.575)
Okay, we lower our criticals so that we can pay more bounties or more individual bounties, right? In theory. But what does that do to risk? Does that increase the risk of us getting a worse vulnerability that we didn't know about not through the bug bounty program? Or does it increase the number of bugs that we're going to receive because now we can pay more or like, I don't know. Yeah, yeah, right? Like, I don't know. It's, yeah.

Justin Gardner (@rhynorater) (01:03:10.228)
And you're kind of paying the price anyway. Yeah, I see that. One more point here before we move along, okay? Because we're already an hour in and I can't spend super long today.

Justin Gardner (@rhynorater) (01:03:23.572)
This was going to the staffing versus bug bounty money conversation.

Bug Bounty doesn't take a break. Bug Bounty doesn't spend time on a useless email. Bug Bounty doesn't do any of that crap. And.

I think that should make it a much easier conversation with finance. It's like, listen, when we hire someone, we are agreeing to pay them X amount of money, regardless of their performance, regardless of their, their, you know, whatever, right? That's inherently risky because you don't know how they're gonna perform and you do your best to pick somebody who will, but it's a risk.

And you don't have that risk with book bounty because you're only ever paying for results. And I think that should make the sell much easier. Like, listen, listen, finance every single time we pay out here, it's, it's actually fixing something and you're buying something, you know, like you're getting some result. Um, so I don't know that that's just my two cents. You feel free to respond to that and then we'll move along. But yeah, it is.

Joel Margolis (teknogeek) (01:04:33.535)
It's just, you know, it's just crazy complicated calculus, right? It's like, yeah, that is true, you know, until you pay enough crits and then suddenly the math is all out of whack. So yeah, it's just one of those weird things that like, I think from a Bug Bunny hunter's perspective, it doesn't matter so much than nitty gritty, but you can tell.

sort of like how finance is when you're dealing with a lot of programs based on like how fast they pay and how much they pay and how loose they are with bounties and all that kind of stuff. Yeah.

Justin Gardner (@rhynorater) (01:05:08.436)
Yeah, all right dude, we're an hour in. I'm looking at the rest of the topics and they are not at all related to, okay actually I've got one semi -related one and then we'll cut it and we'll move the content to next week. Keep it sort of on topic with this whole hacker versus program thing. Let me ask you this. What is,

Joel Margolis (teknogeek) (01:05:15.391)
Nope! We're in deep.

Joel Margolis (teknogeek) (01:05:21.695)
Okay.

Justin Gardner (@rhynorater) (01:05:31.732)
What is your thought on?

walling off endpoints as a permanent fix and as a bandaid fix. And does it happen? And like with a WAF or like with a reverse proxy? Yeah, what are your thoughts?

Joel Margolis (teknogeek) (01:05:52.255)
Oh, it happens all the time for sure. I think the greatest example of this is like, you know, like a year ago or six months ago and all those zero days kept popping up and like Apache and Spring Boot and there were all these really core fundamental problems. The first line of fixes that you would see would be add this to your detection rules, add this, you know, block these requests that are containing these parameters and stuff, right? So that for sure happens. And I think it's a great immediate fix.

Justin Gardner (@rhynorater) (01:05:59.988)
Mm -hmm.

Joel Margolis (teknogeek) (01:06:21.759)
Cause when you have something that affects you, number one priority is making sure that you're not vulnerable. And the only way to do that is to either patch the code and deploy it or to stop requests and then patch the code and deploy it. And so I think a lot of companies, first thing they want to do is lock that down, make sure that they, you know, they can't be exploited externally. Then they can figure out, you know, what they need to change, what they need to fix, whatever, as long as it's not going to impact things.

Justin Gardner (@rhynorater) (01:06:35.828)
Mm.

Joel Margolis (teknogeek) (01:06:51.711)
and then they can go from there. I don't really see any problems with it. I definitely think from like a security response perspective, it's probably like the best and fastest thing that you can do in the moment. Oftentimes, it's just a normal bug bounty report, depending on the severity. I don't know about if it needs to be handled in that sense. But maybe. Okay, yeah, then yeah, probably like you. I think I would want the program to respond as soon as possible. Maybe you can explain a little bit more of the context.

Justin Gardner (@rhynorater) (01:07:10.1)
Let's say it's a crit.

Justin Gardner (@rhynorater) (01:07:16.564)
OK, let me ask this, because I think we've had a debate. I think this is actually in the last Hacker Program debate, where we were like, OK, we've only had a third party. Just whap it off. And you're like, just whap it off? What do you mean, just whap it off? That takes time and effort. And I'm like, just whap it off, man. It's just an nginx config, or go into Cloudflare, or whatever. But maybe I'm wrong about that. You know, like,

And I don't know if this is actually something that you're doing in your day to day or whether you've got teams associated for this, but let's say you want to WAF off an endpoint, or Nginx reverse proxy off an endpoint. Does this go through the whole... This is such a weird question. Does this go through the whole dev process? Are you just... Now I feel embarrassed asking it this way, but do you just SSH into prod and modify the Nginx config?

Joel Margolis (teknogeek) (01:08:04.991)
Yeah, so yeah, no, no, no. No, no, no. I would say at most companies that have at least 50 employees, that's never going to happen. Smaller, smaller startups, like one thing you'll notice, startups, less employees, less process, less procedure. Everything's a lot more scrappy. Everything is much more like, oh, just tell Dan SSH into the box and like, you know, whatever. Like, yeah, exactly. Like, so.

Justin Gardner (@rhynorater) (01:08:16.116)
Okay.

Justin Gardner (@rhynorater) (01:08:32.084)
Hot mix and prod.

Joel Margolis (teknogeek) (01:08:35.167)
Exactly, right? Like that type of stuff. Once you start to grow and there's a lot more procedures and processes in place, you have like, you know, people in charge and managing and stuff, those things are harder for a lot of different reasons. At its fundamental level, it's a security reason, right? That people shouldn't just be able to like SSH into the box and change that kind of stuff, right? Like that's risky. So typically there's what I think is commonly referred to as like a break glass process, where essentially in these P zero scenarios, there are

certain layers of approval, whether that's through like IT or a director or some high level manager who's in charge of this process. They might get paged on call or something like that. You know, the people who are basically in charge of these emergency response processes and there will be special approvals that can go in place to push a change like that. And usually it's not just that like one person has the ability to do that. It does need to go.

through some layers, but those layers exist for those P0, those super high critical risk cases where something needs to happen at two o 'clock in the morning.

Justin Gardner (@rhynorater) (01:09:40.628)
Yeah, it's so much, I think that's something that's really high value to understand too because I think there have been some scenarios in the past where you, I was just reading through some old reports where you can access, where you can bypass those sort of Nginx config stuff. It was on the, you know.

2023 top web hacking techniques. One of the top ones was like, here's these characters, you stick at the ends of URLs or whatever and you can get through. And I just love that. I love the idea of that. And so I think it really makes a lot of sense for researchers to pay attention to security architecture in that way. And if people are waffing off endpoints and doing Apache blocks or whatever.

Joel Margolis (teknogeek) (01:10:11.711)
Mm -hmm.

Justin Gardner (@rhynorater) (01:10:29.684)
then knowing what kind of backend stack is in place and how the front end proxy is dealing with various routes and what data will be stripped out of those URLs. That's really huge and I think something that a lot of people, a lot of bug hunters could benefit from. And I also think it's something that a lot of bug hunter, bug hunters like me, rather than enterprise, app sec people that have a better understanding of how these architectures flow.

I think people like me overlook this a lot because we don't understand, you know, okay, this is what the whole flow looks like and this is what people do in an incident response situation. They laugh it off. So, I don't know, it's interesting.

Joel Margolis (teknogeek) (01:11:12.543)
It's definitely very interesting. And I think that architecture side is really valuable. Like that's one of the big things that I've learned from like the company sort of like, you know, engineering side is every company is very different with how they structure things, but a lot of it can be very similar, especially just like the core, like aspects of how it's working, right? Like there's some edge proxy that's going to load balancer that's going to.

Justin Gardner (@rhynorater) (01:11:22.26)
Mm.

Joel Margolis (teknogeek) (01:11:38.111)
individual instances of code that is a service, right? Or whatever, like, and like that's running some deployment of the thing that an engineer wrote. That's a very common structure. Like you have services, service -based architecture, and then you have these routing and the routing happens one of a handful of ways, whether it's through like internal engine X or Kubernetes pods or Docker clusters or whatever, like, you know, they have some way of communicating between services and from the front end to the backend. And then there's some,

high -performance layer that's handling the request. Because large companies, I don't think people realize the scale of requests that they're dealing with. Large Fortune 500 companies are dealing with hundreds of millions of requests per hour, per minute. It's like lots. If you think about how many... For instance, Facebook has 2 billion active daily users or something. Just think about how many requests 2 billion people are generating.

Justin Gardner (@rhynorater) (01:12:25.428)
Wow.

Justin Gardner (@rhynorater) (01:12:32.468)
Mm -hmm.

Justin Gardner (@rhynorater) (01:12:36.116)
Yeah, and every single page load triggers JavaScript files and it's like 20 requests on a good day. It's like...

Joel Margolis (teknogeek) (01:12:39.359)
Right. It's like 10, yeah, 15 requests. Right.

Right. So like from an engineering architecture perspective, there's like, this is a huge, huge like problem to try and even solve in the first place. And somehow they have this solution that's been architected to like deal with this massive load of requests and figuring out how that works and like how that's doing that and what path your request is taking can tell you a lot about the possibilities that you have for lateral movement and different ways to attack it. And like what kinds of characters will and won't pass. And, um, you know,

where am I getting stopped at the edge layer or like some middle layer or am I getting stopped at the back end? Like where does this all sort of how does this all connect together?

Justin Gardner (@rhynorater) (01:13:17.492)
Are they implementing authorization at this middle level? And is there some way I can get around that? That data, dude, that would be so interesting to me. And I really wish company, and maybe there's not these little diagrams in your organization or whatever, but I imagine there would be for onboarding. If I got access to those, man, that would be so awesome to see, okay, this application that I'm working on. And if it was actually detailed too, because you'll often see like,

Joel Margolis (teknogeek) (01:13:21.599)
Yeah, what layer, right? Exactly.

Justin Gardner (@rhynorater) (01:13:43.892)
All right, here's a little cylinder thing and this is the database and it's like, and then there's like a line. Yeah. Yeah. I'm just like, okay, great. Thanks. You know, but like, if it was actually detailed, like, okay, coming into this, you know, NGINX is reverse proxying with a proxy pass, you know, and it's hitting this and that, if it was actually really detailed like that, that would be so helpful. It's so fascinating. And what ends up happening is, is when we get at these SSRFs, we have to figure it out, you know, and it just, ah.

Joel Margolis (teknogeek) (01:13:46.367)
There's like three boxes and it's like an external request front end back end. You're like what?

Joel Margolis (teknogeek) (01:13:59.775)
Yeah.

Joel Margolis (teknogeek) (01:14:07.359)
So what?

Justin Gardner (@rhynorater) (01:14:12.756)
It'd be really cool to actually see it.

Joel Margolis (teknogeek) (01:14:15.199)
One thing I will say is probably a good place to look for that is engineering blogs. Okay. Because a lot of these companies have engineering blogs that I hate to break it to you. Literally nobody reads this. I'm really sorry if somebody listening is like an engineer who spent like three months writing this really in depth engineering blog post for their company. And then it got because.

Justin Gardner (@rhynorater) (01:14:22.132)
Yeah.

Justin Gardner (@rhynorater) (01:14:37.012)
This is what I'm talking about. Pay the freaking bounties instead of, you know, like hiring people to write blog posts. Like...

Joel Margolis (teknogeek) (01:14:42.687)
I'm just saying, like, I see this a lot where, like, these companies, like, put this huge, in -depth, like, engineering blog post together, and I'm like, who is this for? Like, this is actually for nobody but some, like, OKR. Nobody's actually reading it. Maybe, like, you know, a handful of people read it, and the people think it's cool, think it's really cool, but, like, almost nobody actually sees it. However, from a bug bounty perspective, it's really, really interesting to hear engineers talk about their own internal infrastructure and how stuff works and how stuff communicates.

and the problem spaces that they're encountering. And like, usually they're very happy to talk about those things in these engineering blogs. So if the company has an engineering blog, I would probably go read it because there's, there's gotta be a, there's gotta be a diagram somewhere. Yeah.

Justin Gardner (@rhynorater) (01:15:23.124)
Pro tips, pro tips dude. But that's what I'm talking about man. What kind of big company is gonna win top talent when they say, hey, we're not allowing you to go write a blog on company time. If the person really wanted to do that, no, they would look like assholes and no one would go work for them.

Joel Margolis (teknogeek) (01:15:43.583)
Well, and from the company's perspective, this is promoting the security brand and like bragging about the capabilities internally and attracting talent and all that kind of stuff. So there, there is a justification. Wow.

Justin Gardner (@rhynorater) (01:15:48.884)
Exactly.

But that does nothing. It does nothing for, well, but listen, nobody reads it, right? Maybe your internal people read it and that's cool and that's great with onboarding. If it adds value there, then there's that, right? But.

Joel Margolis (teknogeek) (01:16:02.591)
For all I know, I'm an anomaly and I'm the one who's not reading it but everybody else does, I don't know.

Justin Gardner (@rhynorater) (01:16:05.268)
Yeah, everyone besides you reads these engineering blogs. But Bugbounty doesn't spend three months writing a blog post. It's such a clear exchange for money, and you're rolling the dice in so many different ways. Why don't you grab onto the one thing that's for certain? This is just my thought. All right. Anything else there? Do you want to rebuttal that, or should we close it there?

Joel Margolis (teknogeek) (01:16:33.695)
No, no, I mean, I listen. Bug bounty is a really easy way to be like, hey, we just spent this much money on a vulnerability. Balancing that with the rest of like the finance BS is really difficult, but I think it's not as difficult as a lot of companies do. And the ones who are running VDPs, because God knows why, like if you're making more than $5 million a year in revenue, you should have a bug bounty program that pays money, even if it's a small bounty.

Justin Gardner (@rhynorater) (01:17:01.076)
Yeah, yeah, yeah, I agree, man. Yeah, all right, let's wrap it there. Peace.

Joel Margolis (teknogeek) (01:17:06.623)
All right, cool. That's the pod. See you.