Episode 56: Using Data Science to win Bug Bounty

Transcript

Justin Gardner (@rhynorater) (00:00.517)
Alrighty, John, I am looking at this massive document over here to the right now, filled with like, pretty much just gold on the crossover between data science and bug bounty. Thanks so much for coming to the show, man. And thanks so much for all the effort you put into prepping for it.

Jon Colston (00:15.498)
Thanks, Jess, for having me. For those that don't know who I am, my name is John Colston. I go by the handle of Minis or My Own Eyes or Mayo. I go through all three of those. And I got into the bug bounty world about five years ago. And I've been working at it very slow since then.

Justin Gardner (@rhynorater) (00:36.693)
So dude, yeah, mayonnaise, I've always thought about this. Mayonnaise is like pretty odd name. Give us like the 30 second summary of why you chose that name. And it definitely made you a figure in the bug bounty space. We're like, who's this guy calling himself mayonnaise?

Jon Colston (00:50.172)
Ha ha ha.

Yes, so, um, it's funny, I, uh, young.

John was a huge fan of the Smashing Pumpkins and the Siamese Dream album. It was one of those life changes, right? And there's a song on there, and everybody called it Manet's, but it's spelled Manet's with one N. And then somewhere along the years I researched the meaning of it, and it's actually like a phonetic play. It's My Own Eyes instead of Manet's, the drop of the N, and My Own Eyes. And so I thought that was sort of cool. Yeah. So I said, hey, that's perfect for a hack of My Own Eyes.

Justin Gardner (@rhynorater) (01:19.509)
Mayonnaise. It's like some weird pronunciation.

Jon Colston (01:27.236)
stuff and related to something that I enjoy which is music and that's how that came together.

Justin Gardner (@rhynorater) (01:37.149)
That's awesome, man. There's so many different areas. So let me, let me back up a little bit. The reason why we wanted to bring you on the pod is, uh, time and time again, on the critical thinking podcast here, we have discussed the part of hacking.

relating to data and to taking notes on a specific target. And so many of the people that we have on are just like, yeah, I take notes in this asdf.txt file and it's really rough and there's no system to it. It's just kind of like, I write things down on this paper and I stick it in my shoe or something like that. It's all over the place. And we're like, I was thinking, okay, we've got so many takes on that perspective. Let me get...

a different perspective. And I was thinking, who is the most organized and who is the most methodical, you know, data-driven hacker that I know? And you came to mind. So what I'm hoping to get out of today is a little bit of a look into how you use data science to win bug bounties. And man, if you look at your HackerOne profile, you're winning it, man. It's great. And so I want to know about how you've done that.

Jon Colston (02:43.244)
Thanks.

Justin Gardner (@rhynorater) (02:50.961)
a specific propensity for digital marketing in your past experiences and kind of how that plays into the data science, into the hacking role. And to get that holistic perspective, I think we're gonna have to go back a little bit and hear about your career path and your career history and how you kind of got into this digital marketing and data science areas that made you the hacker you are today. So give us a little brief introduction to that, if you would.

Jon Colston (03:16.987)
Yes, so I went to Georgia Tech and I graduated.

in 1999, which was the kind of the height of the dot com boom, right? My focus was in information technology. Yeah, and went to work for a startup and was integrated as a consultant into a credit card company. Was responsible building out a lot of their information systems and integrating the technology that we built was a real-time decision engine to be able to prove customers for credit

Justin Gardner (@rhynorater) (03:28.634)
Nice, yeah, a good time.

Jon Colston (03:51.616)
stuff. Yeah, and so yeah, went through that, eventually sort of working for the credit company, did a new venture, I guess, underneath there that eventually got acquired because the whole lead generation vertical was exploding. So got into that.

Justin Gardner (@rhynorater) (03:51.901)
Wow, that's a big deal.

Jon Colston (04:14.594)
in that industry as part of an acquisition, built out several national brands. And with that, you had not television, you know, radio, but you had an incredible amount of digital media or digital advertising that would go on. And so I was responsible as all the analytics to tie everything together, make sure that every dollar spent was accountable for generating a return and worked very closely, especially with the digital marketing team.

be able to understand what levers are being pushed, how do I now provide them with the information to help them make the decisions needed to perform better and acquire customers at a lower cost and at a higher volume. So that's kind of how I got into the whole, now as far as like how that transition to bug bounty, eventually had gone on to start doing my own.

Justin Gardner (@rhynorater) (04:59.305)
Wow, yeah.

Jon Colston (05:11.058)
a business that was in a completely different vertical, still Legion, but it was in the mortgage space. We hit some walls at one point with compliance, trying to get through some legal matters. And I needed to figure out some, how to earn some income while we worked through those issues. And found a freelance opportunity in hacking. It was something that I was aware of. It was on my radar, but just was so busy doing this. Anyway, so I jumped in.

Having a background in data science and analytics and whatnot, I took that approach. I took a very methodical data processing, how do I approach this type of mentality, and then stuck hard to one program. And I believe that was probably a little bit of naivety about how the bug bounty space was back then, favoring loyalty. But yeah, that's kind of how I went into the space.

Justin Gardner (@rhynorater) (06:06.829)
Mm. Wow. Dude, that's such an interesting start.

because one of the pieces of advice that I give to most people that are trying to start Bug Bounty is like, hey, you can't really be in a situation where you're trying to enter the industry to make money because you're in some sort of situation where you don't have your finances squared down. And not only did, you know, and maybe not speaking to the finances part, but you were transitioning out of a, you know.

Jon Colston (06:29.416)
Mm-hmm.

Justin Gardner (@rhynorater) (06:40.125)
situation with your startup where you needed to find some additional income and going into the space which adds additional pressures and then coming from a non-security related field and just taking that expertise from that field and applying it to Bug Bounty. I'm not going to lie man, I would have bet against you. I would have bet against you that you wouldn't have been able to pull it off as well as you did, but you did. That's amazing.

Jon Colston (07:00.636)
Ha! S-s-

Jon Colston (07:04.126)
Yes. So, I mean, you know, working with data, whatever thing, right. And I was reading up, right, a lot of how to do about hacking APIs. Well, okay, I'm very familiar with APIs. That was not something that looked so then it was like, okay, how do you find APIs, right. And then so you go through that process. And then as you're reading and so much content was out there, it was basically saying, you know, you get, you know, get access to systems and you

Justin Gardner (@rhynorater) (07:16.434)
Yeah.

Jon Colston (07:34.)
Okay, well, they're in clean up clean up mode, right? They're one of the original advertising platforms, not the original advertising platforms that were out there. So the technology was going to be old.

Justin Gardner (@rhynorater) (07:39.015)
Mm, mm, mm.

Jon Colston (07:47.99)
doubling that up over, right, is that, okay, they're spending their time on the public-facing assets, right? So you knew if you can get into the business of business, the B2B type of applications, then you were going to have opportunities that were going to be once in a lifetime, right? I mean, you were going to have...

Justin Gardner (@rhynorater) (07:50.805)
Mm-mm.

Justin Gardner (@rhynorater) (08:01.419)
Mmm.

Justin Gardner (@rhynorater) (08:09.038)
Yeah, dude, this is-

this is such an intuitive approach, man. You're like, okay, let me just think. So if I was gonna hack, I need to find APIs. And how do I find APIs? Well, I find some sketchy advertising company. And I like that you went in your vertical as well. You say, that's another thing we talk about a lot on the pod is like, if you can hack within your vertical within the thing that you understand, within the industry that you're familiar with, whether from a hobby perspective or from a work perspective, that's gonna afford you a tremendous advantage. And that's what you did.

Jon Colston (08:31.863)
Mm-hmm.

Jon Colston (08:39.351)
So huge.

Justin Gardner (@rhynorater) (08:40.631)
So you went after Yahoo here. Anybody who has ever looked at the Yahoo leaderboard knows that you went after Yahoo. And you were finding APIs left and right, and you were trying to get your hands on these business B2B applications or some of this older stuff, right?

Jon Colston (08:57.834)
Right, exactly. And so, you know, when you find the APIs, and this goes back to the journal you were talking about, Ray, I would find those APIs, they're not vulnerable at that point in time, but they provide a wealth of information so whenever I do get access or whatever, or it helps me identify, hey, there's a system out there that I'm not aware of, I don't know what the login is, but I can start researching that.

Justin Gardner (@rhynorater) (09:04.676)
Mm-hmm. Yeah.

Justin Gardner (@rhynorater) (09:15.289)
Mm-mm.

Jon Colston (09:21.318)
So it was an effort to be able to identify the different platforms that were out there.

Justin Gardner (@rhynorater) (09:22.447)
Mm-mm.

Jon Colston (09:32.402)
which was incredible, the number of them. Yahoo and AOL had gone through all these murders and acquisitions. I mean, you had all these other bolt-on ad platforms that were out there. So it was just, it was a garden of opportunity. Yeah, yeah. And all of it was, you know, all these... A garden of opportunity that just needed to be harvested.

Justin Gardner (@rhynorater) (09:36.178)
Mm.

Justin Gardner (@rhynorater) (09:40.37)
Yeah.

Justin Gardner (@rhynorater) (09:48.661)
It's a mess. Ha ha ha. He says, I say a mess, he says a garden. Ha ha ha.

Jon Colston (09:59.857)
So yeah, so I signed up for the job.

Justin Gardner (@rhynorater) (10:00.239)
That's great, man. I love that.

Yeah, no, clearly, clearly. And so, you stuck for the beginning, you kind of stuck into your vertical there and you were looking at Yahoo, spending a lot of time on them. I guess, and nowadays, are you still looking at Yahoo primarily or have you kind of moved along? How has your methodology evolved from those early times to nowadays?

Jon Colston (10:28.45)
Yeah, so, you know, John Colson mayonnaise 2024 is completely different than 22. Right. And so back in the day, I was, I was fully committed and spent an insane amount of time focusing on Yahoo. I knew that the opportunity was limited and the boundaries were good. And so I just went.

Justin Gardner (@rhynorater) (10:37.908)
Yeah.

Justin Gardner (@rhynorater) (10:51.078)
Mm-mm.

Jon Colston (10:57.762)
so deep. And when I went going deep on applications, it wasn't necessarily going deep into the technology. It was going, you know, finding if there were any seminars in that were out there that are public education, you know, the B2B types, like what were some of the things that were being pushed out to the industry that I could then, like, okay, hey, you know, they're, they're releasing some new

Justin Gardner (@rhynorater) (11:14.616)
Mmm.

Jon Colston (11:25.026)
called immersion. Okay, and I need to figure out what that is. Where does that live? And things to that niche. So I would use that also as a lead to be able to try to find things on the platform and go from there. So yeah.

Justin Gardner (@rhynorater) (11:38.601)
Mm-hmm. And so about that, would you automate that actual part of it yourself, or was this just you kind of being plugged into the data sources for Yahoo, and what's happening with the platform and that sort of thing?

Jon Colston (11:49.986)
Both. So the automation was, it was constantly, so constantly going after and finding what are the new assets that were coming out, looking at the domain names and say, hey, this is a word I'm not familiar with. Is that a new product? So, you know, do I need to search this more? And it would dovetail into a lot of manual searching and looking for, you know, documentations that, you know, a development team would use to integrate their system into their, you know, into the ad system because they've got a bunch of stuff

Justin Gardner (@rhynorater) (11:57.779)
Mm.

Jon Colston (12:19.92)
to do to be able to determine my budgets and where I'm going to put placement and things of that nature. These companies are tied in on multiple platforms, so there's integrations that go on. So there's documentation flying around on that kind of stuff as well. All that kind of...

Justin Gardner (@rhynorater) (12:33.797)
Right, right. Wow.

Jon Colston (12:38.69)
All those things were what I was used to be able to help guide me and identify those spots. So there was, I mean, I don't know how many platforms I was able to get to and put my hands on. But yeah, that was my goal.

Justin Gardner (@rhynorater) (12:43.741)
Wow.

Justin Gardner (@rhynorater) (12:49.333)
Dude, the layer, the depth there, right? I just wanna point out, you know, the fact that you said, hey, I'm monitoring subdomains, right? Which is, you know, something that's fairly normal in the bug bounty industry, it's something we do. And then he says...

And there was a word I didn't know. This is the level of granularity that I feel like a lot of people don't realize that you need in Bug Bounty. Because when you look at the list of 20,000 subdomains or more that you get back from running a subdomain enumeration scan on Yahoo, it can be overwhelming. But then John says, hey, there's actually a new subdomain that just popped up.

Jon Colston (13:04.994)
Yeah.

Justin Gardner (@rhynorater) (13:27.581)
with a word that I don't know in it. Let me go and research what's happening with this company. Let me try to figure out what transitions are happening at this current time with the company and dive deep into that. So that's amazing, man. So you're, yeah.

Jon Colston (13:40.163)
Yeah, because it can expose a new product, a new platform, a new technology. So yeah, understanding what the domain names provide you gives you a hint of what's going on behind the scenes.

Justin Gardner (@rhynorater) (13:44.841)
Yeah.

Justin Gardner (@rhynorater) (13:51.466)
Yeah.

So there's the new stuff that's happening, the new transitions that are happening with the company, there's the old, old stuff that you're getting access to, these antiquated applications. You've said in the notes here somewhere that it was stuff that was sort of pre-security life cycle. Security wasn't even a thought really at the time when some of these applications were being developed because it was so long ago. And so getting access to those core applications is important, but that's tricky, right? Getting access to some of these

Jon Colston (14:08.984)
Mm-hmm.

Justin Gardner (@rhynorater) (14:24.9)
more niche platforms. So, I mean, were you mostly hacking from an unauthenticated perspective? Did you, how did you get your hands on credentials, or how did you actually hack these platforms?

Jon Colston (14:36.81)
So yeah, so ultimately you had to get, well.

many percent of the time you had to get credentials. So a lot of the automation hacking would build a portfolio of documents that you would then silo into, okay, this is probably ALL1, or this is DSP, or something. So you had the wealth of the documentation, and then you'd find... So the typical way that you go about...

Justin Gardner (@rhynorater) (14:44.84)
Mm-hmm, yeah.

Justin Gardner (@rhynorater) (14:57.579)
Mm-hmm.

Jon Colston (15:10.51)
is, okay, can I find a place to brute force? Okay, that's limited. But even with the limitations, you know, being on the business side of things.

Justin Gardner (@rhynorater) (15:14.815)
Mm-hmm.

Jon Colston (15:21.422)
and this is a good tip to take back, right? If you're on a hacking a B2B system and you know that at some point, they're going to have demo accounts or test accounts so that they can share with other businesses, okay? So if you're able to identify in some capacity that, okay, it's demo at blah, yahoo.com or whatever, and you'll find some screen or some user dump, I don't know,

Justin Gardner (@rhynorater) (15:25.318)
Mm.

Justin Gardner (@rhynorater) (15:35.3)
Yeah.

Yeah.

Jon Colston (15:51.436)
okay, that user is meant to be shared and therefore.

Justin Gardner (@rhynorater) (15:52.177)
Mm, mm, mm.

Jon Colston (15:57.426)
That password is probably going to be very simplistic and a standard brute force password tracker will go after it. Right? So that's one way. Another way, especially on the B2B, is that if you can figure out who are the users, like which businesses have used the platform, and you're able to assemble that and monitor

Justin Gardner (@rhynorater) (16:02.292)
Yeah.

Justin Gardner (@rhynorater) (16:20.241)
Mm.

Jon Colston (16:26.514)
which of those companies go out of business? And then buy their domain, buy their domain, and then you can approach. So there was a fortunate, you know, I was able to download a list, and then I memorized every user in that list and all the domains, because you can't save it. So I memorized all of them, right? And, you know, and then, you know, I...

Justin Gardner (@rhynorater) (16:31.127)
Ah, interesting. Hmm.

Justin Gardner (@rhynorater) (16:48.544)
That's great. I love it. Yeah.

Yeah.

Jon Colston (16:55.134)
turn and see, okay, is that GoDaddy has a great tool for that. Is this domain available? Oh, it is. Okay. Let me register that domain and create an email address. Okay. And then now let me, let me do a password reset. So now two factor authentication is kind of taken over that. Right. And, and if you, even if you get it, even if you get, if you brute force an account or you.

Justin Gardner (@rhynorater) (17:06.641)
Let me try to do some password reset stuff. No way.

Justin Gardner (@rhynorater) (17:16.625)
Mm-mm.

Jon Colston (17:23.246)
I'll do some time at Caltech over there. I'm telling you, nine times out of 10, you're going to get a hard stop from the company. I'm like, yup, okay, great.

The bug in's there. I do recommend saying hey, once I get in, I didn't poke around much, but I'd like to see what I can do in the next x hours. Would that be possible? You might get a reprieve and permission to go in. But ultimately I think that if you can figure out a way to overcome whatever barriers it is to get an account in your own credentials, then they can't stop you. They can't stop you.

Justin Gardner (@rhynorater) (17:45.085)
Mm. Mm-hmm.

Justin Gardner (@rhynorater) (17:58.057)
Mm.

Jon Colston (18:00.626)
So I highly recommend going that route.

Justin Gardner (@rhynorater) (18:01.624)
Yeah, that's amazing.

So you can also, so that's ways that we can sort of compromise accounts with trying to brute force some stuff or be able to do password resets on people whose domains have expired. And then getting your own account, that's a whole different beast, right? Because you have to, in those scenarios, you've kind of got to be able to walk the walk and talk the talk to say, hey, yeah, I know about this product, yeah, I'm interested in using this product for my business, that sort of thing, and getting that all set up. Have you had experience with that?

Jon Colston (18:19.201)
Oh yeah.

Jon Colston (18:34.654)
Yes, so I have tried many different times, failed quite a bit whenever you're trying to pretend to be a business. I think my favorite story tell is obtaining access to the Taiwan assets of Yahoo. Because those assets required a physical presence within the country and a phone number.

and so and maybe even an additional ID. But there was a huge barrier unless you were in Taiwan, you were very limited. So I threw several attempts, the first three or four failed, but to finally figure out the winning methodology is that I posted a job on Upwork and I was looking for an English translator.

Justin Gardner (@rhynorater) (19:08.753)
Wow. Yeah.

Jon Colston (19:31.154)
And it was, you had to be, where I felt is that I would like try to hire them on and then I'm telling them what I would do and they were like, no, I'm not going to hack, no and they would go away. So in the job posting I said,

Justin Gardner (@rhynorater) (19:31.995)
No way.

Jon Colston (19:44.958)
lifetime opportunity, ride shotgun with the hackers, we blah, and go after. And yeah, the response board just lit up. Like it was like immediately slow down, slow down. And so I was able to find this wonderful lady named Lexi in Taiwan. She had a family member that actually had, you know.

Justin Gardner (@rhynorater) (19:50.261)
Dude, what? That's amazing.

Jon Colston (20:13.318)
the mall and store and all that had those accounts. She was enough familiar. So we're like, all right, let's go. And we registered and it was like, you know what? Yeah. We went through the whole process and registered and she was in my eyes and ears and got me in. And yeah, no, it was right timing for a, for a live hacking event. And it was just, it was, everything was great. But yeah.

Justin Gardner (@rhynorater) (20:23.044)
No way.

Justin Gardner (@rhynorater) (20:34.613)
Dude.

Justin Gardner (@rhynorater) (20:38.429)
That's amazing. So this, you know, you went through those, you jumped through those hoops, and then you've got yourself an account, and now you've got access to the scope that nobody has access to. And you've got creds, and you've got all these old APIs, and you know, beginning of the marketing era tools that you've got them all to yourself. Wow, that is crazy, man. So, hmm.

Jon Colston (20:43.021)
Hmm?

Jon Colston (20:48.946)
Exactly, exactly, exactly.

Jon Colston (20:58.238)
Exactly, exactly. Yeah, because I mean, I'll be always, I mean, I'll always submit like, I mean, my skill as a, as a hacker is, is secondary, right? Compared to like, when I watch

When you guys talk about, oh yeah, we inserted a hyperlink with this tag and then we redirected and then we did, then we somehow got a dump in which we had the creds for the entire company on AWS. I mean, okay, fantastic. I'm not going to go there. That's not my skill set. But if I can get to assets that, you know, that, or that are in line, more my skill set, then I'm going to, that's what I'm going to do. So that was, that's how I went after things.

Justin Gardner (@rhynorater) (21:19.73)
Shit. Hahaha.

Justin Gardner (@rhynorater) (21:43.113)
That's great man and this whole this whole concept of going deeper and deeper and deeper than people People think you should go or people think you have to go to find these various assets and get access to these systems and Understand the company at a deeper level. That's one of the core things That's really important in bug bounty and you've kind of nailed it. So we've talked a good bit about that I want to swing back around to the so we're gonna go two places here We're gonna go the data

data science and how you use data science to, more specifically, granularly, how you're using that in your recon flow, how you're using it to predict vulnerabilities and that sort of thing. And then I also want to talk about specific, because clearly you have a lot of industry expertise on digital marketing. I want to talk about digital marketing platforms in particular in some sort of, I guess, hands on the keyboard or looking at the data, tips and tricks for how to do that. So let's start with data science.

Obviously, for me, let me just kind of give a little brain dump of what I'm thinking here. So when I think data science plus bug bounty, I'm thinking...

Mostly from a recon perspective, but if you look at your hacker one profile, you're not just doing recon You're finding a lot of vulns. So I'm wondering how much of your methodology with that uses data science is pertaining to reconnaissance and how much of it actually gives you a tangible vulnerability and And then I guess you can kind of talk about either one of those recon and actual vulnerability itself

Jon Colston (23:18.558)
Yes, so figuring out how to best approach this and explain it. So data science.

Justin Gardner (@rhynorater) (23:25.036)
Mm.

Jon Colston (23:31.822)
I use it as a construct. It's not a one and done type thing, applying it to word list. It's applying it to everything I do. And so to do that, I go under the mantra, Peter Drucker, he's a management philosopher, he came up with this saying is what is not measured is not managed, right?

Justin Gardner (@rhynorater) (23:40.125)
Right, right.

Justin Gardner (@rhynorater) (23:44.627)
Mm.

Justin Gardner (@rhynorater) (24:00.306)
Mm-mm.

Jon Colston (24:00.334)
And so when you put everything into that framework is that you wanna measure everything. And because if you're able to measure it, then you're able to optimize it. So to do that in Bug Bounty, most of us, whenever I watch hackers use the tools, they're a one-endoner, right? It's not built into some.

Justin Gardner (@rhynorater) (24:08.806)
Hmm.

Justin Gardner (@rhynorater) (24:12.85)
Mm-mm.

Justin Gardner (@rhynorater) (24:27.196)
Mm.

Jon Colston (24:27.57)
system that's intended to keep track of performance that you can then later go back at a future date and make modifications and do your optimization. So I built a system that, look, whenever I run a fuzz on a target, right, it's going to log everything that I do. So I have wrappers around all my tools and I call that worker with this and then make sure that all that information goes into.

Justin Gardner (@rhynorater) (24:49.346)
Hmm.

Jon Colston (24:56.023)
database but it's flat files that I can later use to execute. Okay.

Justin Gardner (@rhynorater) (24:58.937)
Okay, I'm gonna pause you right there because there's just so much to unwind just right there. Okay, so first step is we need to measure everything.

so that we can optimize everything, so that we can perform better as bug bounty hunters. In order to do that, what you've done is you've wrapped all of your tools with some sort of, I guess some sort of script or something that is going to be taking the output of that tool and allowing you to retrieve the output later if you need to, or can you just talk a little bit, it's going into flat files, you know, you say a database, but it's going into flat files, and then you're searching those flat files for patterns that you've identified, is that the idea?

Jon Colston (25:11.428)
Yes.

Jon Colston (25:39.054)
So, okay, so yeah, it's the next level. So let's just take, we have one file that we're fuzzing, right? And it's a generic file and you're fuzzing it on all domains, right? So let's take a look at all the ways that we can measure that, right? So we have a domain, right? We know what the program is.

Justin Gardner (@rhynorater) (25:49.69)
Mm-hmm. Mm.

Justin Gardner (@rhynorater) (25:53.873)
Mm-hmm, sure. Mm-hmm.

Justin Gardner (@rhynorater) (26:00.245)
Mm-hmm.

Jon Colston (26:01.238)
We know the subdomain of that program or whatever the root is, right? Those are two pieces of traceability that you need to have, right? So you not only domain, subdomain and program, right? Then you give it the file. How big is that file? What is the purpose of that file and can I track it so I know the identity of it, right? So how many records in, right?

Justin Gardner (@rhynorater) (26:04.693)
Mm-hmm. Right.

Yeah, yeah.

Jon Colston (26:24.47)
How many records out? How many successful hits did it have? And your definition of successful hit could be different between people, whatever, right? So now I have a hit rate for that file by that program, by that route.

Justin Gardner (@rhynorater) (26:34.437)
Mm-mm-mm.

Jon Colston (26:39.806)
Okay, so now I can start seeing, okay, this particular file is going to have a performance of X on here. So now whenever you start looking at the bigger picture things, I have a hit rate of X percent here and Y percent here. Why is that? Right? Okay, maybe this, then you start getting to the topics of relevancy and all that stuff. Right? So that's a very generic. Yes.

Justin Gardner (@rhynorater) (26:41.502)
Wow.

Justin Gardner (@rhynorater) (26:55.654)
Mm-hmm.

Justin Gardner (@rhynorater) (27:01.717)
Ah, so you're correlating. I can see, I am starting to see it here. So you say, okay, I ran this across this specific set of subdomains. I'd expect I get this specific hit rate. I've run it across this subdomains. And wow, this hit rate is specifically, drastically different. What is, am I doing something wrong here? Does this file just not exist on these subdomains? But they're in.

they should exist, so maybe a certain path is mapped wrong or incorrectly or differently or something like that. And wow, okay, interesting.

Jon Colston (27:31.11)
So, yeah. So, applying it to flat files, you're not gonna have that much of a difference. But whenever you start getting into your specific files, okay, so all my files have.

Justin Gardner (@rhynorater) (27:39.645)
Sure. Mm-hmm.

Jon Colston (27:47.366)
a unique purpose, like I have a purpose. This file is meant to find APIs. This file is meant to find endpoints that reveal directory structures or additional endpoints within the server.

this file is meant for find upload features. I don't, you have, right? You'll quickly be able to start finding things accordingly by program, by the actual route within the program and be able to see what you're doing well and what you're doing not, right? If you can identify, okay, here are my top performers. Here are my 10 top performers. Why is it performing well? What is happening here?

Justin Gardner (@rhynorater) (28:02.565)
Mm, mm, mm.

Justin Gardner (@rhynorater) (28:23.613)
Mm.

Jon Colston (28:28.978)
that I can then. So where I'm going is whenever you take a look and you just look at your standard, your output of, how do you measure it? How do you know you're improving on your fuzzing? How do you know that your word list is applicable or relevant to this file or to this program or this program or this program? So now once you build a, and this is a bit of an overkill, and the only reason why I do this is because... No.

Justin Gardner (@rhynorater) (28:42.654)
Mm.

Justin Gardner (@rhynorater) (28:56.37)
Hey, you said it not me, man. I don't... That's great.

Jon Colston (28:59.006)
It's an overkill and what I only hope is that for people, for the audience that's listening here, is that they can take a little tidbit, so you don't have to build out this massive system, but being able to understand how to put together a construct so you can start measuring performance across different programs, different word lists, different sections, so how's your crawler doing?

Justin Gardner (@rhynorater) (29:23.138)
Mm-mm.

Jon Colston (29:27.798)
you know, how's your link finder doing? You know, why is it, clearly I'm missing out on some keywords for this program because its hit rate is very low. What's unique about it? Let me try to spend some extra time fuzzing on it to see if there's, okay, there's a different path to, you know, the files that I normally use to be able to determine like a directory structure. So it kind of identifies areas where you can focus and then what not.

Justin Gardner (@rhynorater) (29:51.209)
Hmm.

Justin Gardner (@rhynorater) (29:55.421)
So you're assigning performance, you're measuring, even down to like the path in the directory level and creating lists and analytics that are telling you about the efficiency of specific, like directory brute forcing list entries, that granularly. Wow.

Jon Colston (30:14.662)
It's just, it's, yeah, and where it goes in is, okay, so I have, so I have, so as, I have workflows, right, and so my, one of my main workflows goes through a series, and it looks something like this. It's like a quick hit fuzz of like.

things that's going to reveal directory structure on, right? And some of the API, but a lot of it's like robots.txt, asset manifest, things of that nature, right? Okay, it goes in, run a crawler, run link finder. Okay, now, you know, words that are relevant to the host are generated, okay.

Justin Gardner (@rhynorater) (30:38.975)
Mm-hmm.

Justin Gardner (@rhynorater) (30:43.819)
Mm-hmm. Sure.

Jon Colston (30:56.802)
put those words into a recipe ingredient type mix, run that through, do I get additional hits? Okay, so I'm stepping through a series of, I don't know, 30 different sub-processes within this workflow. And I need to be able to manage it, I need to have KPIs on each one. So all the way back to, okay, is part of this program is looking from this sub-domain that is...

Justin Gardner (@rhynorater) (31:02.516)
Mm.

Jon Colston (31:25.07)
This is the actual sub domain, but this was the root domain. Okay, it goes through, all right, here are the performance of the 30 steps of this processes, the results, okay. Now I can aggregate that up. I can now say, okay, this program or this root, or, you know, with the performances of the hit rates and whatnot. That's just in like content discovery. You go back and you start doing a different analysis from a standpoint of,

Justin Gardner (@rhynorater) (31:49.599)
Wow.

Jon Colston (31:54.49)
you know, did I get a bug, right? And then trace it back. So as soon as I get a bug, I'm going back. I'm going back, tracing through all the processes, all the way back to how did I find that? What was the source? Like, was this part of a pull from security trails? Was it like a CIDR scan? What it was, right? And so by putting all this together, you start.

Justin Gardner (@rhynorater) (32:05.066)
Mm.

Justin Gardner (@rhynorater) (32:11.337)
Yeah.

Yeah, yeah, yeah.

Jon Colston (32:17.318)
understanding. Now this is very what falls into the automationist realm. I heard that being kicked around and right. Right. And so there's tools that can like this concepts can be applied to people that not necessarily are wanting to build out full automationists.

Justin Gardner (@rhynorater) (32:23.613)
Mmm. Yeah. Yeah, yeah, for sure. 100%.

Jon Colston (32:35.67)
You can build out, you can use this concept to build out, okay, I'm gonna do a new keyword discovery tool, right? Because I need to add and grow my word list. Okay, well, let's use that.

Justin Gardner (@rhynorater) (32:36.323)
Mm.

Justin Gardner (@rhynorater) (32:43.358)
Yeah.

So I want to hear about that. And I want to also go back to what you were saying before about tracing the vulnerability. Now you found the vulnerability. Let me see what.

parts of my process led to me finding that endpoint, which we found to that vulnerability. But first I wanna get a little bit into the weeds on how exactly you're doing this. Because I was like, when I hear you talk about statistical analysis on paths and stuff like that, I'm like, wow, that sounds really cool. I should have all of my word lists sorted by, hey, this is the ones that are gonna hit first, statistically across a big set of data. But I'm a little bit.

Like when I start thinking about how I'm gonna program that and how I'm gonna do that, it gets a little bit fuzzy for me. So I mean, obviously, how are you, you said you mentioned flat files. You said database and then you're like, oh, well, it's actually flat files. I mean, are you computing efficiency for these specific paths and then like storing them in a flat file in like CSV format and then like dumping it in after you've sorted it with, I don't know, how are you doing this?

Jon Colston (33:30.542)
Sure.

Jon Colston (33:37.334)
Yeah.

Jon Colston (33:53.442)
Yeah, yeah, it's not quite as automated as what you're going. It's just collecting the data that leads me to the information or that leads me to where I can start diving. So let's say this process that has X number of sub processes, right?

Justin Gardner (@rhynorater) (33:58.284)
Mm.

Justin Gardner (@rhynorater) (34:02.665)
Mm-hmm.

Jon Colston (34:12.166)
I'm going to like I know which each sub process does. So I know the word list that it uses, but I'm going to count in, okay, how many in, how many records in, how many hits, right? And I'm gonna dump that to a KPI log, right? So for this sub process, this is the in, this out, and this is the time duration.

Justin Gardner (@rhynorater) (34:12.426)
Mm-hmm.

Justin Gardner (@rhynorater) (34:27.007)
Mm-hmm.

Mm-hmm. Okay.

Jon Colston (34:34.902)
Because whenever you, where I'm going with this is that I want, at the end of the day, I want to sit down, have a cup of coffee, spend two hours writing reports, and then go play golf. That's what I want to do. But that's the end goal.

Justin Gardner (@rhynorater) (34:35.86)
Okay, that's important too.

Mm-hmm.

Justin Gardner (@rhynorater) (34:48.137)
That's that's amazing. I love that.

Jon Colston (34:52.382)
I'm not trying to be like, I'm not trying to be like, oh, I call rate this great attack vector and like I am what I am. I call this practice retirement right now. I'm figuring out what I want to do. But I would, I am, you know, I'm, I'm living the good life. I'm taking my intensity down a bit. And if I can have these systems generate a list of what I call opportunities, I'm basically

Justin Gardner (@rhynorater) (35:04.036)
Yeah.

Justin Gardner (@rhynorater) (35:12.444)
Yeah.

Justin Gardner (@rhynorater) (35:17.822)
Mm-hmm.

Jon Colston (35:18.594)
I'm building a lead generation platform for hackers. It's gonna give me leads at the end of the day. There's gonna be opportunities and I'm gonna. That's a great way to call it. Yeah. Lead gen. Yeah, yeah, exactly. That's exactly what I'm trying to build.

Justin Gardner (@rhynorater) (35:22.162)
Mm-hmm. Dude, that's a great way to call it right there. It's Lee Jen for Volms. Wow, that's clutch.

Jon Colston (35:31.082)
I'm not doing, it's not vulnerability scanning. While that is part of it, you can run Nuclei all day long and find stuff there. But this is stuff that is tailored just to me, right? And so let's say there's another hacker that I eventually partner up with that is like, oh, my A game is SQL injection. Okay, what are your flags? Let me program it in. Here's your feed, right? And so that's kind of where I'm going with it. So.

Justin Gardner (@rhynorater) (35:38.739)
Mm-hmm.

Justin Gardner (@rhynorater) (35:44.39)
Mm-mm.

Justin Gardner (@rhynorater) (35:52.405)
Mm-hmm.

Justin Gardner (@rhynorater) (35:58.897)
Wow, so okay, but your key performance indicators there is your inputs, you're saying, okay, I'm fuzzing this file, this file of, let's say you mentioned quick hits before, right, we've got like quickhits.txt or whatever, and we fuzz that on a target, and then we say, okay, it took this much time, there's this many hits, and this many, you know.

Jon Colston (36:02.86)
Yep, sorry.

Jon Colston (36:15.638)
Great.

Jon Colston (36:22.622)
Mm-hmm. Yep.

Justin Gardner (@rhynorater) (36:27.349)
Yeah, I guess this many entries in the thing. So you can say, okay, you know, my success rate on this specific word list is X percent. And then, but you're probably not breaking it down further than that and saying, okay, well let me take the 6% that hit. No, okay, because that's just, it gets a little bit crazy because then at that point, you'd have to actually have a database. Yeah.

Jon Colston (36:39.112)
No, no, no.

Eventually, eventually, eventually you do. Yeah, well, eventually you do from a standpoint of, okay, you know, what was in contained in the file, you know, what your hits were, you know, which like, so you could filter down and say, okay, just give me just the hits that have happened and put it in, right? You it's more of making sure that you're not going completely out of rounds. Okay. Did that file run in 20 seconds? Okay. I'm fine. I mean, it's just the initial stage. I don't want it to be. Um, I don't want it to be a, be a hog, but.

Justin Gardner (@rhynorater) (36:53.221)
Right. Oh, yeah. That's true.

Mm-hmm.

Justin Gardner (@rhynorater) (37:05.983)
Mm-hmm.

Mm-hmm.

Jon Colston (37:10.758)
As you go through my process, I'm bolting on and adding new stuff and I'm building word list on the fly dynamically based upon what's been. So what words are in the domain. Okay, those need to be fuzz and be fuzz in a certain pattern. And what you know words, you know, X and a link finder find out that are say, hey, these are relevant to the target. Okay, those need to put in some type of Recipe ingredient combination that can then go into be fuzz, right. So, you know,

Justin Gardner (@rhynorater) (37:19.323)
Mm-hmm.

Justin Gardner (@rhynorater) (37:25.723)
Mm-hmm.

Justin Gardner (@rhynorater) (37:30.772)
Mm.

Justin Gardner (@rhynorater) (37:39.878)
Mm.

Jon Colston (37:41.674)
Let's say you're going through and you see an outlier and all of a sudden that, okay, there you found this one step of your process where all of a sudden there were five million keywords that were like, whoa, wait, what happened here, right? Now you're able to go in and not that from an optimization standpoint, say, okay, what specifically occurred here because that's messed up. I don't ever want that to happen. Or you can say, hey, this particular recipe in my keyword

Justin Gardner (@rhynorater) (38:06.509)
Right.

Jon Colston (38:13.23)
is working really well. Okay. How can I find other variants of that I can then add in? So always it's V2, Word, internal. I don't know what it... Okay, well now let's go in and find V3, V4, V5, and V1.

Justin Gardner (@rhynorater) (38:20.461)
Mmm.

Justin Gardner (@rhynorater) (38:29.153)
Dude, that's so data driven. I love that because it's like, one of the things that I often do is I'll get excited about that and I'll get all the data and I'll get it in a database and I'll be like, all right, I got all the data, you know? And then I won't even necessarily go back and be like, okay, well, what paths really have been performing well across this whole.

Jon Colston (38:31.722)
Yeah.

Jon Colston (38:41.005)
Right.

Justin Gardner (@rhynorater) (38:49.361)
you know, this whole target, and what can I extrapolate from that? It's all about extrapolation in these sort of contexts too. It's like looking at the things that you've done that have succeeded and saying, hey, why did these succeed, and how can I replicate that? And actually, I mentioned a couple times on the pod before that I was really into Recon back in the day, and I implemented something similar to this, where I would, even just parsing the words from the domain name, nothing else, just the domain name,

Jon Colston (39:18.418)
Mm-hmm.

Justin Gardner (@rhynorater) (39:19.355)
using those words to enumerate other domain names and this whole concept that we were talking about off air of recipes and ingredients and that sort of thing. And man, I was blown away by the results, man. There's just a ton of stuff you'll miss because it's not in security trails, it's not in any of this other stuff, but it is there when you start mixing, you know, mixing and matching all these words together. So.

Jon Colston (39:28.938)
ingredients.

Jon Colston (39:44.364)
Hehehe

Justin Gardner (@rhynorater) (39:45.849)
Two questions with that. One, how do you get your ingredients? You mentioned that you're using words related to the target. You're pulling it from various tools. Let's say you've got a domain. It's all words just smushed together. How do you tell where the word starts and begins?

Jon Colston (39:49.654)
Mm-hmm.

Jon Colston (40:02.616)
Okay, well, so one is that...

Jon Colston (40:10.362)
Data science is used for probability. It's not to get 100%. So if, let's say there's a domain that is like 36 characters long and there's no dashes and dots, okay, that's a one-off. How many other domains are you ever gonna see that like that again? So just stay awash, hey, let that one go. It's not worth my time to find that. So yeah, so I break down the domains, subdomains. So actually, let's start back.

Justin Gardner (@rhynorater) (40:14.853)
Right.

Justin Gardner (@rhynorater) (40:21.716)
Mm.

Justin Gardner (@rhynorater) (40:26.17)
Yeah, rare.

Justin Gardner (@rhynorater) (40:30.437)
Mmm.

Jon Colston (40:37.922)
We've got all of our different data repositories that we're pulling from, all that information comes in. All right, you have ones that resolve, ones that don't resolve. Just because they don't resolve doesn't mean they don't have value. Make sure you store the ones that don't resolve. It could have, you don't know. It can resolve in the future. So make sure you...

Justin Gardner (@rhynorater) (40:42.522)
Mm-hmm.

Justin Gardner (@rhynorater) (40:50.813)
some that resolved in the past, you know, yeah, exactly. Yeah.

it could resolve on the back end, you know?

Jon Colston (40:58.762)
Yeah, still, yes, you know, you can still keep trying to resolve it over the next 30 days or so, whatever, right? Okay, so now you got this information, right? And what you start looking for is frequency, right? Frequency of words. So you break it down. So let's do a quick example. You got, I don't know, 10,000 subdomains. You break it down by the dashes and dots. And sometimes you can dash it together, whatever.

Justin Gardner (@rhynorater) (41:14.046)
Hmm.

Justin Gardner (@rhynorater) (41:22.011)
Mm-hmm.

Jon Colston (41:28.638)
And then you do a frequency. Do a frequency of what is showing up. So you're gonna see API, you can see admin, you can see internal, you can see blah, blah. And then what you're gonna realize is, yeah. So as you go through and let's say you take the top 100 words, or anything, you say, okay, where does this fall into? Okay, this is related to API. Okay, so this is an API ingredient. So API, API one, YQL, blah, blah.

Justin Gardner (@rhynorater) (41:29.412)
Mm-hmm.

Justin Gardner (@rhynorater) (41:36.509)
Yeah. Company name. Yeah.

Justin Gardner (@rhynorater) (41:54.557)
Mm, mm, mm. And you're manually doing this.

Jon Colston (41:57.334)
those are all ingredients, you have to, right? I mean, there has to be some type of, and yeah, I mean, so you're grouping, and you'll find that there's not crazy amounts of, you might get into specifics of, you know, there might be a particular company that puts their co-location in the mix, right? And yeah, I mean, and so,

Justin Gardner (@rhynorater) (42:02.001)
Yeah, in the beginning for sure, yeah.

Justin Gardner (@rhynorater) (42:22.611)
Right, right.

Jon Colston (42:26.222)
The nuance of that and being able to go through and understand, but you put together a list of ingredients, a word list of each ingredient as you go through and the word list will tell you what the ingredients are based on the frequency of what's been shown up in the past. Alright, so now you've got the ingredients. Now build a loop that replaces those words in your 10,000 domains.

with the ingredient name, right? And then now you start, okay, resummarize that up and then now you start to see, okay, and if at the end, if you haven't fully identified all the things, just have it insert a generic word, like this word goes here, right? So that way you can then start summarizing up and then say, okay, what are my recipes? All right, so it's, you know,

Justin Gardner (@rhynorater) (43:16.642)
Mm, mm, mm.

Jon Colston (43:22.334)
API, you know, internal, some word. Yeah, environment, yes, blah, right? And you'll say, oh, there's my, here's 20 recipes that may be specific to that program. Maybe very generic for all programs. I mean, it depends on what you start with, right? And so now, so now,

Justin Gardner (@rhynorater) (43:29.117)
version number, word, integer, yeah.

Justin Gardner (@rhynorater) (43:44.989)
Wow, yeah, and you see this a lot with stuff that across a lot of companies and you also see stuff like.

like AWS region and like you mentioned, you know, prod, QA, that sort of thing all over the place. And yeah, these could be relevant specifically to the company or these could be across all companies. It's just a human thing. So correlating those as well, that's great. So you've got your ingredients, you've broken those down by the dots and the dashes, haven't done any word splitting, just mostly done it that way. And then you've identified and you've grouped your words. So you said, okay, API words,

would be API one, API two, that sort of thing, or maybe it's like backend API, back API, so something like that, and then you've got those sort of groups, and then you do all of the combinations of all of them, resolve all of them, and then that tells you.

Jon Colston (44:28.644)
video.

Jon Colston (44:34.623)
Sure.

Justin Gardner (@rhynorater) (44:45.717)
I mean, obviously, you'll definitely identify some of the additional hosts that you didn't have that were based off of those patterns. But I'm wondering what other takeaways as well you have from that, where you can say, okay, this specific word is particularly high frequency or high performing. How can I replicate this success elsewhere?

Jon Colston (45:07.678)
Yeah, so are you talking about using that same methodology in content discovery? Because I do that as well.

Justin Gardner (@rhynorater) (45:14.841)
Yeah, well, absolutely. There's definitely that piece. And so you've got a list of ingredients and recipes, and you've enumerated a bunch of subdomains. And now we're going an extra layer down. And so you're using this as well for content discovery, yeah?

Jon Colston (45:18.295)
Yeah.

Jon Colston (45:30.57)
Yeah, absolutely, absolutely. So let's talk about content discovery, right? Okay, so let's say, all right, so you're in your fuzzing stage, right? And I mentioned there's 30 processes in mind, and that this is a number, I don't know exactly how many there are. All right, so it goes down, now we're going to start looking for words that are relevant, because relevant is a very important aspect of finding content.

Justin Gardner (@rhynorater) (45:36.892)
Mm.

Justin Gardner (@rhynorater) (45:43.802)
Mm, mm, mm. Mm.

Jon Colston (45:57.694)
Let's take whatever's relevant. All right, so we know that the domain words are relevant because it provides information. You're gonna find three-liter acronyms, you're gonna find products, you're gonna find things of that nature that are listed in there, all right? So now through your history, so now you've got years worth of fuzz history that you've now captured over time, right? You kind of say, okay, what are some of the paths that you commonly see whenever

Justin Gardner (@rhynorater) (46:07.295)
Mmm.

Justin Gardner (@rhynorater) (46:20.597)
Mm.

Jon Colston (46:31.254)
a word is listed in domain, how does it apply to finding APIs? So you take a look at it, you're like, okay, I've noticed now that it takes a word from the domain, and I do doma

Jon Colston (46:49.994)
or array, right? And so you start adding in and start playing around with, okay, how do I take the information that I have and then how do I then apply it extremely relevant and abbreviated so it's quick but highly.

Justin Gardner (@rhynorater) (46:50.033)
Right, right.

Justin Gardner (@rhynorater) (47:05.033)
Dude, I love this. So then there's an extra layer of this here, okay? So now we say, okay, what do I know about domains that have this recipe? Or what do I know about domains that have an API group keyword in it?

And then you can cross reference, because some other host has some weird, you know, get all URLs or way back, you know, entry on a different domain. And it's not even the same API, you know, string, but it's in the same API group. And so you take, you know, because you're correlating it with the API group directly, you're grabbing that string, that path, and you're trying it on this totally other host, the totally different other host, because you've drawn the connection between those groups.

Jon Colston (47:44.738)
Yeah.

Justin Gardner (@rhynorater) (47:49.454)
Wow, dude, this is crazy.

Jon Colston (47:52.75)
So now let's talk about the next level. And this is something I recommend for the automationists and for the non-editors. Right, right. All right, so there is this need to always look for new keywords, right? And grow your list and validate it. So when we start, right, we're given a keyword list that we download from SecOps, right? Or we, I'm gonna say jhatx because I think he's amazing. I'm gonna give you the all.txt, right?

Justin Gardner (@rhynorater) (47:58.089)
Ah. Take it to the next level from there. Okay, alright. I'm listening, John. Let's go.

Justin Gardner (@rhynorater) (48:10.107)
Mm.

Justin Gardner (@rhynorater) (48:18.501)
Yeah. Mm.

Jon Colston (48:22.406)
whenever I first started, the most frustrating thing is that there was no context, right? Every word was of equal weight when I know that some words are much more valuable than others. All right, so let's take a sample where, okay, over time we have now, if we're doing a bunch of, well, let's say you got 10,000 domains, but you know, get 10,000 domains.

Justin Gardner (@rhynorater) (48:30.638)
Mm-hmm.

Yeah.

Jon Colston (48:50.234)
Some you're interested in, some you're not, doesn't matter, okay? Still throw them in there. Right. And take whatever word list you got and create, um, some loop process, bash processes, basically say, okay, I want you to take the first thousand of that list and I want you to fuzz these 10,000 dummies and I want you to dump the results.

and then once you go back and get the next thousand, and then the next thousand, and the next thousand, and the next thousand, and you're just dumping, dumping the results, dumping the results, dumping the rolls, right? Now you've got something that provides you with some, what are my hit rates for each one of those words, right? Now I took a list of...

two million, right, and now I've scrubbed out, and now I've got, you know, these with, you know, a thousand hits, and I've got these with two hits, right? You now start to build weight. And then, whenever you start taking a look at weight, and then you can say, okay, you did that first, you know.

Justin Gardner (@rhynorater) (49:46.534)
Mmm, mmm.

Jon Colston (49:53.654)
two million that you got from SecOps, maybe you go get a bunch of stuff from Asset Notes, it comes in, but you're continually feeding this machine that's just, okay, DDoop, have I ever fuzzed this before? Yes or no? Okay, no, I haven't, okay, just toss it in, and it's just grinding in the background. Just set up a server that does nothing but this, right? And then put a reminder on your calendar so you can put in another two million, or whatever, right? Or you're finding some patterns,

Justin Gardner (@rhynorater) (49:59.55)
Mm.

Justin Gardner (@rhynorater) (50:08.22)
Mm, mm.

Jon Colston (50:24.148)
and you're like, okay, let's see what happens if I do config v2, right, and then put, and then do every iteration of that. It just sits and runs in the background on, you know, and then you just occasionally go in and it's like, oh, this is interesting. This is something that has a high relative rate. I need to start looking into this more and then starting adding that into. So if I can, before I start adding new keywords into my production files, I wanna know that they're relevant, right? I just don't wanna throw fudge at the wall and see what sticks.

Justin Gardner (@rhynorater) (50:29.701)
Mm-hmm.

Justin Gardner (@rhynorater) (50:41.074)
Yeah.

Jon Colston (50:53.61)
do that outside. I want to protect my core process and then go from there.

Justin Gardner (@rhynorater) (50:55.229)
Wow. Yeah. That's great. So then we're waiting at that level too. And wow, this is just, you know, I knew that your process here was pretty in depth. This is a lot deeper than I thought actually. So I'm very impressed. And there's, I'm sure there's.

so much more we could dive into with regards to like where you're getting all of these data sources and how you're actually going that deep to correlate all these. But I want to do two things. One I want to...

emphasize what you said just a moment ago, which was you went through and you categorized what words belong to what conceptual group. And I think this is something that a lot of hackers would have a little bit of a stigma against because it's like...

Jon Colston (51:42.604)
Okay.

Justin Gardner (@rhynorater) (51:48.769)
I'm going to go through here and I'm going to manually say, okay, this belongs to this group, this belongs to this group, this belongs to this group. It's a manual process, especially automationists in general don't love manual processes, but that specific thing that you did requires a human brain and it requires you to be able to look at it and say, oh, this is...

Jon Colston (51:58.284)
Mm-hmm.

Jon Colston (52:06.763)
Mm-hmm.

Justin Gardner (@rhynorater) (52:10.257)
and sort of empathize with the developer and say, that's what they were thinking when they did this word. These two ideas are drawn together by nature of, how words work and that sort of thing. And so using that extra little bit of human touch, I think is an extra step that a lot of people don't go and is really necessary in the data science world, that whole categorization piece, right?

Jon Colston (52:21.646)
Great.

Justin Gardner (@rhynorater) (52:36.806)
That's very cool. Is there any other scenarios like that you could explain to us and then we'll talk about data sources and where you're getting, you know, like security trails and that sort of thing?

Jon Colston (52:48.446)
As far as like manual classification, I mean there's, oh, I mean it's so built into my processes. I'm trying to think. There's so many manual things that I do, right? I mean you can't automate everything. You want to be able to look at stuff, especially asking questions.

Justin Gardner (@rhynorater) (52:52.081)
Yeah.

Justin Gardner (@rhynorater) (52:58.706)
Yeah.

Justin Gardner (@rhynorater) (53:07.697)
Let, talk to me about your manual processes then. This will be great because, you know, someone who does lean a lot into automation, I wanna know what you do manually.

Jon Colston (53:15.406)
I want to know what you do manually. Yes, so my manual stuff is now that I've got a system that collects all the data, I ask questions, right? And the manual stuff is investigating, like I know what I'm monitoring, but what else should I be monitoring? What else can I find? So if I can spot a trend with like, you know...

Justin Gardner (@rhynorater) (53:23.457)
Mm-hmm.

Jon Colston (53:38.618)
some type of directory structure, right, that has a decent hit rate, okay? Let me come up with a way to create 100,000 variants of that, run it through, and see if I get any hits, make it to the production. Hell, and I might find four bugs while I'm doing that, just in the discovery process, right? That's a, that's manual. The process is doing what it's doing, but the concept of...

Justin Gardner (@rhynorater) (53:50.077)
Mmm.

Mmm.

Justin Gardner (@rhynorater) (53:57.693)
Right. That's great. So let's take that example really quick and say, okay, walk me through that. So for you, what that looks like is I'm going to my, logging into my server and I'm saying, hey, what paths have the highest hit rate?

Jon Colston (54:16.794)
Yeah, so yeah, I mean you can kind of go through that, but as you're going through data, you'll see stuff that sticks out. Like you'll see, if you're doing like API, all of a sudden you start seeing with all this crawl and whatnot, you're starting to see swagger with variables on it. Well, that's interesting. How many other swagger with variables do I have on it? Oh, right, can I hit it without the variable? Well, I can't. Okay, so how many swagger am I missing? Okay.

Justin Gardner (@rhynorater) (54:33.762)
Mm-hmm. Mm.

Jon Colston (54:46.438)
And it leads you down a path. It's like, okay, well, shit, I need to figure out what variables I did. Okay, can I go to big data query? Can I go to asset note? Can I find examples of this in any type of live, right? What are some examples? Okay, it's group one, group two. Okay, that's a trend. Okay, so let me do a bunch of different tests and that's gonna go out.

Justin Gardner (@rhynorater) (54:54.223)
Yeah.

Justin Gardner (@rhynorater) (55:06.252)
Mm.

Jon Colston (55:11.642)
against all those domains and see if I can pick up something like something that I missed or pick up on something that is being left out. Maybe it's like, oh man, here, my variable value should be part of the word list that gets dumped out from xml link finder. Okay, oh, what if I add that in, right? And our, you know, so...

Justin Gardner (@rhynorater) (55:27.947)
Mm.

Jon Colston (55:34.066)
The manual process is getting in there, getting your hands dirty and asking the questions and being able to pull out the data. You don't know what you don't know. That's where most of my manual aspect is from.

Justin Gardner (@rhynorater) (55:40.755)
Yeah.

Justin Gardner (@rhynorater) (55:45.393)
And for that, are you mostly looking at, are you picking a random host and being like, okay, what questions can I ask about this host? Or are you looking at the data from an overall perspective and saying, okay, this host has a 12% higher hit rate than other hosts, so I'm gonna look at this one, or maybe I'm gonna look at the one that doesn't have a very high hit rate because something's weird there, or how do you decide?

Jon Colston (55:56.471)
Yeah.

Jon Colston (56:12.788)
Yeah.

So yes, so actually what happens, so most of those hit rate stuff for the most part is gonna be more operational, make sure I stay within line and I'm not going outside. My target is four minutes, five minutes at max on a, if I can get through 3000 targets on one server a day, fantastic, right? That's the dialed into that, right? So what happens for me is, I go through all this automation process, blah, we'll talk about the conversion funnel, but anyway, at the output, put at the conversion funnel at the very end

Justin Gardner (@rhynorater) (56:21.377)
Mm.

Mm.

Justin Gardner (@rhynorater) (56:35.162)
Mm.

Justin Gardner (@rhynorater) (56:41.566)
Mm.

Jon Colston (56:43.572)
is a log file for me that basically has said what it's discovered. The endpoints, it tells me whether what method was used, git push pull, blah, blah. It tells me git post git pull, git post patch put. Yeah.

Justin Gardner (@rhynorater) (57:02.929)
Mm. Ah, okay, I see, I see, I see, yes. Heh.

Jon Colston (57:07.65)
what method was used, what stage of the process it found. So if it's a 403 on a crawl, that's gonna be more interesting to me than a 403 on a fuss, because it's much more relevant. Right, so now I have a little bit of information, okay, if I'm gonna spend some time doing a 403 bypass, I'm gonna do it on the crawl versus, you know, so I'm gonna wait that accordingly.

Justin Gardner (@rhynorater) (57:19.165)
Hmm. Yeah.

Jon Colston (57:30.294)
Right. And then, and then I'm looking at all the endpoints, right. I'm looking at all the endpoints and I'll see something that I had not expected to see as something that comes out of like, interesting. I've never seen this before. Or if I am diving, like is it host, it's interesting and I'm diving in and I'm, you know, crawling in and I'm, you know, messing around with burp and I'll find something interesting on that. I'm like, Hey, I need to look at this. I have all the, excuse me, I have all the data historically stored. Now I can then.

Justin Gardner (@rhynorater) (57:31.625)
Very cool.

Jon Colston (58:00.266)
Now that identified as a question that needs to be asked, let me go answer that question with the data that I put in.

Justin Gardner (@rhynorater) (58:08.189)
Wow, man, the breadth of it all is a little overwhelming to me as somebody who doesn't really do data science on all this. So then you get this question and then you say, OK, yeah. Yeah.

Jon Colston (58:19.493)
Right. Does that make sense? Because I want to make sure that point gets through.

Justin Gardner (@rhynorater) (58:23.721)
No, it does. So you're looking at a specific host, you're saying, okay, I'm waiting things like crawling versus hitting various paths. I'm waiting probably status codes and waiting all these different attributes of the domain, method used, get post put, whatever. And then you're looking at the interesting things from there. That all makes sense. And then you

manually looking through it, at the end of the day you've actually got to do the hacking, right? You've got to read and you've got to put together the attack vector in your brain. And then, you know, there's this, you're looking at all this in the log file. And so, you know, then you go back to ask the data a question about x, y, z thing. And I guess, I imagine this doesn't take a normal...

Jon Colston (59:13.995)
Yes.

Justin Gardner (@rhynorater) (59:17.653)
shape, right? And this is the sort of thing that's a little bit hard for me to wrap my head around. But you know, you've got some something like, hey, where do I see 403s or something like that? Should I pay attention to 403s on the specific endpoint? How I mean, are you just running a grep across all of your, you know, 100 terabytes of files to try to get data on that question? Or what kind of common forms does this ask the data a question take?

Jon Colston (59:21.324)
Okay.

Jon Colston (59:46.206)
Sure. So I was using like a standpoint of I've like with a swagger that has a variable, like it's a 200 and without the variable, I'm not getting anything. I'm like, okay, that's something. Right? That is something that I need to be aware of. All right. One, let me go back and see if I've ever seen.

Justin Gardner (@rhynorater) (59:53.629)
Mm. Yeah, perfect. Let's go back to that. Yeah.

Justin Gardner (@rhynorater) (59:58.994)
Mm.

Mm. Absolutely.

Jon Colston (01:00:10.782)
a variable on a swagger before and then if I have what's the frequency of it? Okay, I have it. Okay, what if now I then create a test fuzz that does nothing but fuzz for different variables, right, values and see if I can get a hit on all this, right? I've learned this. I've learned that this is now a new way to extract data. How can that I apply it to all the data that I have that's very relevant, right?

Justin Gardner (@rhynorater) (01:00:14.302)
Hmm.

Justin Gardner (@rhynorater) (01:00:17.641)
Mm-hmm.

Justin Gardner (@rhynorater) (01:00:30.751)
Mm.

Justin Gardner (@rhynorater) (01:00:39.749)
Mmm, mmm.

Jon Colston (01:00:39.842)
how do I do it? So now, now then I'm okay, we extract all the whatever swagger.json's and then okay I'm going to put together and craft a specific test.

Justin Gardner (@rhynorater) (01:00:46.923)
Mmm.

Jon Colston (01:00:49.494)
But I get measuring and say, okay, I'm gonna put in 50,000 variable names and run this through. Am I getting hits? Oh, dang, it hit on two. Okay, it might not be a vulnerability. But now in my head, I'm like, okay, I need to be aware of this and I need to be able to then update my processes to at least take into consideration. And then if it's a new, it's a new segment then, okay, now I'm starting to measure that. Okay, how much time am I spending on this? What's my hit rate for this new test that I've created that is

specific thing.

Justin Gardner (@rhynorater) (01:01:21.777)
Wow, okay, you've said all of that in different parts. Now the picture's getting much clearer. So you're saying, okay, I've got variables on the swagger file, I'm gonna scan across all these things to see where have I ever seen variables on a swagger file before, and you're updating your process to maybe even weight that higher in the future and say, okay, if I've got something like a swagger file and then that's xyz weighting, and then.

If it has variables, oh, that's a little bit more interesting because I'm getting more deep. I'm getting more information out of it.

Jon Colston (01:01:53.762)
But it also tells you how complete of an assessment you're doing, right? And so, leave your existing program going, don't modify it, and keep it going. It's all about testing, right? You're all about trying to test and find new winners. So you put together a hypothetical test, you're saying, okay, let's see if I do this, right? And I started with 10,000.

Justin Gardner (@rhynorater) (01:01:59.646)
Mmm.

Justin Gardner (@rhynorater) (01:02:06.012)
Mm.

Justin Gardner (@rhynorater) (01:02:11.292)
Mm.

Jon Colston (01:02:19.274)
10,000 different variable names in my test. I only got a hit on, you know, five and it was always group one, group two, group three, group four. Okay, well that may be an industry standard, it's just groups. So all I need to do is those 10 names and put it into the, into the, into the process. And now I've got coverage, right? I've got a process that takes one second to run and, you know, it's going to find incrementally more than I did the day before.

Justin Gardner (@rhynorater) (01:02:27.862)
Mmm, yeah.

Justin Gardner (@rhynorater) (01:02:46.697)
That's pretty lit, dude. And I think reintegrating all of that too is really, is really, it's a challenge from a programming perspective. And one of the things I didn't love about being an automationist back in the day was this concept of, I'm spending so much time programming and it results in cool bugs. But I like to spend more time staring at other people's code and wondering how to break it than writing my own crappy code that's gonna break. And so that.

Jon Colston (01:02:48.458)
Hahaha

Jon Colston (01:03:03.639)
Mm-hmm.

Jon Colston (01:03:09.515)
Yeah.

Jon Colston (01:03:13.282)
Right, right, right. No, it was a headache. It's a headache and whatever. Yeah. And most of them, you know, a lot of, I would say actually this month, majority of my time has been trying to tinker around and get my server, one of my servers that broke back up and running, right? No, that's a pain. But whenever it does run, right, I sit down in the four hours, pull out eight bugs. Okay, great. So now if I can do that every single day, but I just got to get it back. And thank goodness for AI,

Justin Gardner (@rhynorater) (01:03:26.121)
Yeah.

Justin Gardner (@rhynorater) (01:03:29.896)
It is.

Justin Gardner (@rhynorater) (01:03:41.241)
Yeah.

Jon Colston (01:03:43.116)
That's a pain, but now with AI, shoot, I need this, and this. Thank you very much. Copy, paste.

Justin Gardner (@rhynorater) (01:03:49.625)
And that's amazing, and I want to hear about, so obviously you're using it from a coding perspective. And I hate to even ask this question, because we normally try to stay away from like, what programming language are you using, you know, sort of questions, but I mean, what are you doing with all this? Because this is, I mean, if you're doing all this in like bash, then it's going to be insane to maintain.

Jon Colston (01:03:54.296)
Mm.

Jon Colston (01:04:02.667)
Yeah.

Jon Colston (01:04:10.654)
It is, it is bad. I mean, it is all in bash, but it's all like, um, no, no. So it's, it's workflow type of thing. Right. And then, and in bash and you know, I just basically, you know, if then, okay, did the pass this step, if then, it's if then steps all the way through, right. And then something come out. Okay. Then, then, then simple, simple. You only focus on one small segment each time. Right. Um, right.

Justin Gardner (@rhynorater) (01:04:14.497)
Dude, you're a beast, man! I don't understand how you do this!

Justin Gardner (@rhynorater) (01:04:29.46)
Yeah.

Justin Gardner (@rhynorater) (01:04:36.337)
You've got to compartmentalize. That makes sense. Yeah.

Jon Colston (01:04:38.55)
Very, very calm department. And if somebody would look at my code and like, you are nuts, you're not a programmer. I'm like, you're absolutely right, I'm not a program, but it works and this is how I've got it done, right? I was able to do it, it goes. So, but yeah, so, Bash is a very simple scripting language and that's very easy for me to understand. If I get into some type of object oriented, I'm gonna be just lost. It's just not, I'm gonna go set.

Justin Gardner (@rhynorater) (01:04:44.339)
Ha!

Justin Gardner (@rhynorater) (01:04:49.125)
Wow, dude.

Justin Gardner (@rhynorater) (01:04:57.861)
It is, yeah.

Justin Gardner (@rhynorater) (01:05:03.826)
Wow, dude.

That's legendary. And I think it probably really speaks to the organization of the system as well, having it compartmentalized out to different pieces. So that's really cool. I want to go towards the working backwards from Avon and learning how to do that. But before that, I want to talk about your data sources. So you mentioned before, you're hitting, you've got these various curated lists that you've got. You mentioned a crawler. You mentioned security tricks.

any other data sources you want to kind of shout out and share with the people.

Jon Colston (01:05:40.126)
Yeah, so whenever you're talking about paid sources, security trail seems to be top spot. I think that I do, yeah, maybe, I'm just going off. When we start taking a look at all the different sources you can get from ZoomEye, to Shodan, to Binary Edge, right? I've tested them all and I've integrated them all. Some I've cut.

Justin Gardner (@rhynorater) (01:05:46.249)
Mm. Yeah. I agree.

Justin Gardner (@rhynorater) (01:06:02.537)
Hmm.

Justin Gardner (@rhynorater) (01:06:08.829)
Mm-hmm.

Jon Colston (01:06:11.111)
But yeah, so I'm doing all that. So those are the data sets. Some are free, some are not, right?

Justin Gardner (@rhynorater) (01:06:17.723)
Mm-hmm.

Jon Colston (01:06:18.346)
So let's go back to the, so when we were talking about this, let's talk about how to incorporate scope management in this whole data management or data science thing, right? I prioritize, so I have a scope management tool, which is basically flat file that has program, sub-domain or CDIR, blah, whatever the target type, whatever target is. Is it in scope, what's the platform, what processes I need to include or not, okay?

those processes are for pulling data, I can assign it a value of one through whatever. One is daily or multiple times a day, two is I can stage it. So I'm using data science also to tell me but the frequency in which I should pull from these locations. So then I'm not wasting money on, I know it's overkill at times. But with the data is expensive though.

Justin Gardner (@rhynorater) (01:07:13.433)
That's crazy. It is a little over engineered, I'm not gonna lie, but it's very efficient. It is, man, it is. And querying an API all the time is gonna cost a ton, especially if you're going for these paid data sources.

Jon Colston (01:07:19.019)
It's expensive.

Jon Colston (01:07:26.718)
Yeah, so I mean, I've, you know, between data and servers, you know, I'm pushing close to $1,500 a month, right? That's a mortgage payment for some people.

Justin Gardner (@rhynorater) (01:07:33.169)
Wow, I was gonna ask that, yeah, $1,500 a month. I mean, and that doesn't, you know, a lot of people would wince at that. I know friends that are in this community that are like, ah, $10 a month for a server? I'm like, come on. But having this whole access to this whole data service is massively advantageous. Exactly, exactly. And the other thing with this too, from the automation side, is like...

Jon Colston (01:07:45.16)
Hahaha.

Jon Colston (01:07:48.97)
One bug. One bug takes care of it. Right? Yeah.

Justin Gardner (@rhynorater) (01:07:57.657)
We've been talking with people on the podcast lately about the sort of trade-offs that we've seen over the past couple of years of less people focusing on Recon and more people focusing on sort of diving deep and knowing the application. Not that those are mutually exclusive, but the one thing that I miss about automation stuff is like, and I think to be honest, for me, the money sort of evened out either way. But the one thing that I miss about the automation is like, ah.

Jon Colston (01:08:09.47)
Yeah.

Justin Gardner (@rhynorater) (01:08:24.081)
I'm gonna, like you mentioned, wake up in the morning, get my cup of coffee, sit down, and then it's all doing it. And if I go to the golf course, then it's still working for me, and I don't have to stress, like, oh, I'm not looking for bugs right now. So I do think while there is stressors of keeping your servers up, keeping your code maintained, there's a lot of relief that comes as well from knowing that something's constantly working, something efficient, something that you've built.

Jon Colston (01:08:33.73)
Yeah.

Jon Colston (01:08:36.896)
Exactly.

Jon Colston (01:08:48.308)
Yeah.

Justin Gardner (@rhynorater) (01:08:49.781)
is constantly working in your favor. So that's definitely something to be considered for the people that are looking to have a little bit more of a relaxed lifestyle, kind of like you mentioned, of like pre-retirement sort of vibes. That could definitely be a great solution.

Jon Colston (01:08:57.388)
Mm-hmm.

Hehehe

Jon Colston (01:09:03.986)
Yeah, if I put a little commentary to that is that I will say this like whenever I was going crazy deep dive And doing the things that I was doing that was so much that was rewarded so much more right though now I Lost years of my life doing it. There's a price to pay for that

Justin Gardner (@rhynorater) (01:09:07.283)
Mm.

Justin Gardner (@rhynorater) (01:09:11.627)
Mm, mm.

Justin Gardner (@rhynorater) (01:09:16.658)
Yeah.

Justin Gardner (@rhynorater) (01:09:20.705)
Yeah, I'm sure you did.

Jon Colston (01:09:23.106)
But hands down, that was much, much more profitable. And I decided to go with this way because I needed to reinvent my relationship with bug bounty. I needed to chill out. I needed to get to another stage of life. So I've gone this automation-ish route. It identifies properties. It's not gonna find bugs per se. It's gonna find the opportunities to find bugs. And to the point of that,

Justin Gardner (@rhynorater) (01:09:29.889)
Mm.

Justin Gardner (@rhynorater) (01:09:38.241)
Mm-hmm.

Justin Gardner (@rhynorater) (01:09:42.561)
Mm.

Justin Gardner (@rhynorater) (01:09:47.836)
Mm.

Jon Colston (01:09:53.14)
Justin is that I love to create, right? I love creating things and hacking is destroying, automation is creating. So a lot of us kind of, hey, I want to build something and I have, because it is destroyed. Because there's a thing, right? You find a bug, you submit it, what residual value does it have? It's done, it's gone. There's nothing else that like...

Justin Gardner (@rhynorater) (01:10:03.045)
Hahaha

Justin Gardner (@rhynorater) (01:10:06.791)
hacking is destroying yeah it is

Justin Gardner (@rhynorater) (01:10:17.222)
It's gone.

Jon Colston (01:10:21.222)
There's no more return. It's not like an annuity and then like, you know, every year you're gonna get paid in addition like I'm in a value of it. No, it's done. So, you know, investing into something that helps you out and you know, I don't know maybe it's just to make me feel good. I don't know, but I enjoy tinkering and obviously I'm nuts about this data stuff. So...

Justin Gardner (@rhynorater) (01:10:25.95)
Yeah.

Justin Gardner (@rhynorater) (01:10:40.192)
Nah, I totally...

Justin Gardner (@rhynorater) (01:10:45.073)
Yeah, that's very clear, not gonna lie, John. That's awesome. Great, so we covered the data sources a little bit. Clearly you're not shying away from paying for data sources. You're clearly trying to get a lot of coverage, pulling from lots of different places and then sort of determining what is the best frequency to do with that. So bringing it back around to the...

Jon Colston (01:10:57.054)
Mm.

Justin Gardner (@rhynorater) (01:11:13.849)
how do we identify the source for our vulnerability? Can you tell me a bit about that system? So you've, let's say you've just found a vulnerability, how are you working back and letting that inform what automation stuff you'll do in the future?

Jon Colston (01:11:20.085)
Okay.

Jon Colston (01:11:26.306)
Mm-hmm.

Jon Colston (01:11:31.986)
Yeah, okay, so I get a vulnerability. I submit it. All right. First thing, first thing, is it a duplicate or is it, is it new?

Justin Gardner (@rhynorater) (01:11:36.811)
Mm.

Justin Gardner (@rhynorater) (01:11:42.649)
It's a new one. It's a new one. I like this though. This is good.

Jon Colston (01:11:44.262)
Okay, right, because of the duplicate, I'm like, shit, why was I not the first? Right, so it's a new one, okay, I go through, right? So I trace through, I see, okay, how my content discovery worked, what did it find, whenever I, compared to what did I do to actually discover the bug?

Justin Gardner (@rhynorater) (01:11:47.878)
Uh-huh.

Justin Gardner (@rhynorater) (01:11:51.847)
Right.

Jon Colston (01:12:06.538)
Right? So it may have alerted me that, hey, there's, you need to look at this. And then I dive into it and I'm like, Oh, I'm putting this piece with this piece and then fully formulating what the bug is. I then ask, okay, is that something that could be added? Or was this just something that I just need to make sure that I test whenever I, you know, this is something that was a one-off. So that's feedback there. Then I go back and I say, okay,

Justin Gardner (@rhynorater) (01:12:08.124)
Mm, mm.

Justin Gardner (@rhynorater) (01:12:29.329)
Mm.

Jon Colston (01:12:33.734)
when did it enter my content and how long after, how long did it take from discovery and resolution to it to get to content? So I'm taking a look at the time differential, like was my queue too long? Was my queue, you know, it took five hours? Or like, or do I need to spin up another server to clear out that queue up, right? So I'm looking at the time.

Justin Gardner (@rhynorater) (01:12:56.402)
Mm-mm.

Jon Colston (01:12:58.21)
Then I'm looking all the way back and saying, okay, what was my data source? Exactly. Yeah, even and I've got BeBot and AMAS running on their separate servers, straight, they're doing their own thing.

Justin Gardner (@rhynorater) (01:13:05.829)
And that's going to be like security trails, show Dan, you know, see IDR ranges, that sort of thing.

Jon Colston (01:13:18.282)
I'm not doing the brute forcing right now because I just got so much whenever I do run, I got so many targets coming in until it works. I just don't have, I'm choking. So then I traced back through and all the way back. Okay.

Justin Gardner (@rhynorater) (01:13:23.838)
Mm. Sure.

Justin Gardner (@rhynorater) (01:13:30.831)
Yeah, yeah.

Jon Colston (01:13:38.514)
It came from security trials. Okay, whenever it was brought, what time of day did I find it? Did I find it in the morning? Did I find it midday? Did I find it in the afternoon? Did it resolve the first time? Like sometimes I find that I get a file and it takes, it's not till the second day before it resolves and gets to my system. I'm like, okay, what happened there? Was that an operational issue? Was it down, right? So I'm trying to time crunch down and find areas that, okay, where did the gaps?

Justin Gardner (@rhynorater) (01:14:10.069)
Hmm. Yeah. So do you have like a specific timeline that you want to see there? Like I want to see, okay, it made it through my whole pipeline in one day. And I found the volume and that's, that's the happy path. Is that, I mean, what kind of metrics do you have for that?

Jon Colston (01:14:10.65)
And, yeah, but to the point of...

Jon Colston (01:14:26.49)
Yeah. So basically my goal is within 18 hours of discovery that it's in my output. Right? And I see it, and it varies. And that varies because that's on a Monday. A Saturday and Sunday is gonna be completely different. Because your number of targets is gonna be different. And holidays, holiday weeks, they're different.

Justin Gardner (@rhynorater) (01:14:34.525)
That's great, man. That's awesome. I love how you have a specific number on the top of your head.

Jon Colston (01:14:50.622)
So all those things taken, like you can see, now I look for those things because when we were doing advertising, I mean, we had to know that viewing habits are gonna be different on the weekends or on a holiday, right? But same thing with work productivity and releasing new assets, it's the same thing. You know, your dev teams are, they're gonna adjust schedules based upon the upcoming holidays. So you know.

Justin Gardner (@rhynorater) (01:14:51.615)
Mm.

Justin Gardner (@rhynorater) (01:15:03.205)
Mm-hmm.

Justin Gardner (@rhynorater) (01:15:18.386)
Absolutely.

Jon Colston (01:15:19.39)
You know the upcoming, you know, weekly into a holiday, it's gonna be lightweight. And not to be weekends, same thing, you're gonna see less stuff come up. So.

Justin Gardner (@rhynorater) (01:15:24.934)
Mm-mm.

Justin Gardner (@rhynorater) (01:15:30.098)
Yeah.

That makes sense. So just to summarize that, because all of these processes get pretty intense. You found your vulnerability. It's new. You're going back. You're looking at how long it takes to go through your pipeline to get to the point where it's like, hey, John, this is something you should look at, you know, that sort of thing. You're trying to figure out what's happening there. And then do you have data on like, hey, 90% of my vulnerabilities are coming from CIDR

this target, like maybe that's something I should over-leve- you know, spend more time looking into. Do you- have you done that sort of thing before? Is that something you do on a regular basis? Hmm.

Jon Colston (01:16:05.012)
Mm-hmm.

Yeah, I do that. Oh, one off. I mean, yeah, I do have one off. I mean, it's not like I got a hundred bugs that I'm like all of a sudden looking through and finding. But yeah, I do see. Like, I mean, like the indicators of, okay, what's my hit rate for a CIDR? What, I mean, how many end targets, how many IPs that I scan? Is it really, can I prioritize accordingly?

Justin Gardner (@rhynorater) (01:16:21.692)
Mm.

Justin Gardner (@rhynorater) (01:16:28.614)
Do you, so I guess in order to do that, you'd have to log all of that as well and say, okay, yeah, is there some thing you have that says, okay, I found a vulnerability on this endpoint and it's associated with this asset and it's associated with this path and this data source and this sort of thing. So do you have like a database or a file that's full of that information?

Jon Colston (01:16:49.962)
Uh, no, I should. You think I would. I don't have that. I mean, um, because usually I do it a one-off. When you talk about a manual thing, that's what I'm going to manually go through. Right? I know the indicators, right? I'm more, um, to be completely, um, you know, managing and then yes, you would have that, but I don't think we're on that. I think that even goes, that's one level. You've identified a level before low. You found the good.

Justin Gardner (@rhynorater) (01:16:54.556)
Mm.

Justin Gardner (@rhynorater) (01:16:59.689)
Gotcha. Mm.

Justin Gardner (@rhynorater) (01:17:08.582)
Mm.

Justin Gardner (@rhynorater) (01:17:13.521)
Wow, dude, I found the bottom. I found the bottom of the data science seek.

Jon Colston (01:17:19.11)
Yeah, exactly. The bottom of the rabbit hole.

Justin Gardner (@rhynorater) (01:17:23.773)
That's very cool, man. There's so many possibilities you can do with all that data too. So then taking it that extra level and correlating like, okay, here's the vulnerability. What's the recipe for this vulnerability? Well, it's an API grouped subdomain on a showdan source domain with a path that has.

Jon Colston (01:17:28.948)
Oh yeah, yeah.

Justin Gardner (@rhynorater) (01:17:48.141)
a word from the domain name or something like that. Even something like that would be able to just be like, oh man, that'd be mind boggling to see all of the data you could pull out of there. I love that. So I think you covered it a little bit. We talked about it briefly in the past, but I think this is a good transition into the digital marketing arena because I've got some questions about that as well. How do you apply conversion funnels?

Jon Colston (01:17:52.011)
Ray.

Jon Colston (01:17:56.898)
I'm gonna go.

Jon Colston (01:18:12.343)
Okay.

Justin Gardner (@rhynorater) (01:18:17.315)
funnel that you mentioned before to this bug bounty thing. Is that the thing you just described before or is that something different?

Jon Colston (01:18:18.704)
Mm-hmm.

Jon Colston (01:18:22.976)
It-

No, it is, it's kind of, so in digital marketing and direct response marketing, in a lot of operations, there's a measurement that's called a conversion funnel. Basically, you have X number coming in and you're whittling down to a certain end point. So in bug bounty, right, you start with total targets in and you're gonna do some filtering along the way and you're gonna do some processing. And then at the end, you hope that what comes out at the end is going to be what you're highly focused on or what you want to.

Justin Gardner (@rhynorater) (01:18:29.747)
Yeah.

Justin Gardner (@rhynorater) (01:18:33.747)
Hmm.

Jon Colston (01:18:54.48)
to gain, right? So in conversion funnels, we used to think about how many, how many impressions did we get? How many, how many clicks did it lead to? How many form fields did it lead out to? How many customers did it lead to be monetized? What was it? And then you go through and you plug all those conversion funnels all over the place and be able to say, okay, this is working, this is not, right? Same thing here. So in Bug Bounty is like all my assets in, right? And then they go through, did it resolve, did it...

Justin Gardner (@rhynorater) (01:18:56.357)
Mm-mm.

Justin Gardner (@rhynorater) (01:19:10.865)
Mmm.

Jon Colston (01:19:23.618)
you know, did they, you know, eventually did, did I get to a bug, right? And you're looking at all the different sources. So, okay. You know, this particular source, um, is always the one that find, you know, if I'm finding something that is always security trails, okay. Well, that tells me something, right? Um, that security trail funnel is important. Or, you know, if it's a CIDR scan, okay.

Justin Gardner (@rhynorater) (01:19:26.011)
Uhhh

Yeah.

Justin Gardner (@rhynorater) (01:19:42.653)
Mm.

Mm-hmm.

Jon Colston (01:19:47.734)
which clouds are coming from, which, you know, is there a certain region that's more valuable than others? Things of that nature. So you get conversion funnels all over the place and you're filtering it down. Does that make sense? Yeah.

Justin Gardner (@rhynorater) (01:20:01.088)
Wow.

That's awesome, man. So yeah, that's a technique that's borrowed from marketing, from the digital marketing world, and then we're applying this to assets in the scenario, rather than users in a traditional sales conversion funnel. And say, okay, where along this funnel, the user comes in, they can land on the website, did they fill out the form? No, okay, so there's some sort of fall off between the user landing on the site and then filling out the form. And then, you know, in our scenario, there's an asset coming in, there's some sort of fall off.

Jon Colston (01:20:07.243)
Mm-hmm.

Jon Colston (01:20:23.662)
Mm-hmm.

Jon Colston (01:20:29.183)
What marketing recipe can I use on that group? Right? Do I need to change the color to the button to be red or purple or green? Right? That's where marketing goes into it, right?

Justin Gardner (@rhynorater) (01:20:32.784)
Yeah, exactly. Yeah, marketing man. It's great. And in this scenario, it's, yeah, go ahead.

Jon Colston (01:20:44.382)
Obviously marketing is another name for hacking. I mean, instead of hacking, I mean, hacking relates to computers, marketing relates to people, right? You're hacking people. Yeah, go watch Social Dilemma and then put your mindset on of like, okay, hacking people. Get people to change their mind without them knowing. Have you seen that movie yet?

Justin Gardner (@rhynorater) (01:20:49.289)
Mm.

Justin Gardner (@rhynorater) (01:20:53.893)
Yeah. Wow.

Justin Gardner (@rhynorater) (01:21:02.553)
Social Dilemma. I have not seen it yet. That seems fascinating, so I'm definitely gonna, I'm gonna have to check that out after this. All right, let me get to, we have so many things in this document here that I need to pick your brain on. So I guess we'll switch over now towards the digital marketing area.

Jon Colston (01:21:09.106)
Oh yeah. Yeah, no.

Justin Gardner (@rhynorater) (01:21:26.521)
And we've sort of talked about the concept of conversion funnels with people coming in versus assets coming in and that sort of thing. I'm wondering if there's any tips that you have as clearly an industry expert on digital marketing and that sort of thing. What kind of things do people need to know when they're looking at a digital marketing platform from an attacker's perspective? Because there's a lot of...

lingo, marketing concepts, that sort of thing, what data's important, what data's not important, that people need to understand for them to be able to have that sort of subject matter expert approach to attacking a marketing platform. So can you give us like, I know it's a lot, because I know I'm asking you to just like condense your whole work history and expertise in digital marketing into a thing, but what kind of things do we need to know where we're gonna get the most value out of it?

Jon Colston (01:22:13.332)
No.

Jon Colston (01:22:19.854)
So the things you need to know is with hacking any target, right? You need to know the industries ins and outs, the operations. You need to know the terminology. You need to know what's important to the users, what's important to the business, right? You need to understand the ecosystem of how that works. Now, whenever you get into there, right, then you can start looking for things that are of high value that might be disguised. And

Justin Gardner (@rhynorater) (01:22:29.441)
Mm.

Justin Gardner (@rhynorater) (01:22:36.642)
Mm.

Justin Gardner (@rhynorater) (01:22:40.076)
Mm.

Jon Colston (01:22:50.147)
I guess before I get to that point, I would like to say is that with every ad platform, there is tons and tons and tons of documentation. There's a lot of functionality from the point of ad content creation to delivery to bidding to the result sets. There's a lot that goes on into that. So knowing all that, there's a lot of functionality to take a look at. Now.

Justin Gardner (@rhynorater) (01:23:00.373)
Mm.

Justin Gardner (@rhynorater) (01:23:16.993)
Mm.

Jon Colston (01:23:17.842)
By being somebody that has a knowledge of, hey, I was once a digital advertiser, I know what's important to me. And I'm kind of a subject matter expert, systems design, know how things should work type thing. You combine those two, you take a step up and you start taking a look at these platforms.

Justin Gardner (@rhynorater) (01:23:42.293)
Mmm.

Jon Colston (01:23:43.398)
you can kind of see some areas that data shouldn't be at. But highly sensitive.

Justin Gardner (@rhynorater) (01:23:53.553)
Yeah, yeah, that makes sense. So I guess without, you know, we walk the line here on critical thinking quite often of like, give us your secret sauce and tell us about all of these critical vulnerabilities you found on all of these companies that we don't, sometimes we don't explicitly name, sometimes we do explicitly name, you know, but a lot of times people know what we're talking about. So I guess with that, with that,

line in mind there. What kind of data in those sort of environments should people be looking to leak or what kind of data is important in those sort of applications? Because all the ads are getting published anyway, right? That's one thing that I've struggled with. Obviously you've got personal information of the people that have accounts on the platforms, but it's a highly public sort of concept and somehow you've been able to extract...

Jon Colston (01:24:34.014)
Yes, so in the...

Right.

Jon Colston (01:24:42.51)
Yeah, yeah, yeah.

Justin Gardner (@rhynorater) (01:24:49.573)
a ton of money from these platforms, so what data are you getting?

Jon Colston (01:24:51.619)
So, so as...

So as a consumer, yeah, it's a very public thing, right? Just being served ads. But so as an advertiser, right, you're in direct competition with every other advertiser, right, that real estate space comes at a price and you're trying to get the best real estate space for the lowest price, right? And it's typically done through blind auction. There's some public auctions out there now. The blind auction.

Justin Gardner (@rhynorater) (01:24:58.528)
Mmm, yeah.

Justin Gardner (@rhynorater) (01:25:03.856)
Hmm.

Mm.

Justin Gardner (@rhynorater) (01:25:14.482)
Mm.

Justin Gardner (@rhynorater) (01:25:21.513)
What is a blind option?

Jon Colston (01:25:23.29)
Auction, sorry, that's my southern accent. Auction, A-U-C, T-I-O-N, auction.

Justin Gardner (@rhynorater) (01:25:26.537)
No, no. Yeah, but what is a blind auction versus a not blind auction?

Jon Colston (01:25:31.914)
So blind auction is whenever you're bidding on keywords, you're saying I'm willing to spend $4.37 on this keyword, it's a bit of a phrase match, blah, for this demographic and yada, right? Now, that's my bid. Now if I'm a dollar over, advertiser B, I'm paying way too much, right? There's a lot of efficiency, or the inefficiency is very profitable for these companies. All right.

Justin Gardner (@rhynorater) (01:25:35.996)
Mm.

Justin Gardner (@rhynorater) (01:25:43.43)
Sure, sure.

Justin Gardner (@rhynorater) (01:25:52.571)
Ugh.

Justin Gardner (@rhynorater) (01:25:59.428)
Interesting.

Jon Colston (01:26:00.458)
And right because you're in competition, you're going to be like, okay, what is if I'm Toyota, I'm like, what's Ford doing? Like how are what is what's their strategy? How much are they spending? Why? Yeah, exactly. What are they doing? Right. Okay. Auction. All right. So, so there's a lot of there's a lot of data goes around that is highly sensitive whenever is viewed from the standpoint of a customer or an advertiser's KPI.

Justin Gardner (@rhynorater) (01:26:07.893)
Mm-hmm. How much are they paying in the blind option? Yeah, option, I'm saying option. Auction, yeah, that makes sense.

Justin Gardner (@rhynorater) (01:26:28.693)
Interesting. So in those scenarios, you're kind of wanting to pull data on how much people are paying for advertising and how much money they're putting into specific keywords or specific phrases or specific ads within that environment. Because maybe the ad content, you know, I guess the ad content is important because you have to be able to know the keyword, but the ad content is going to get shown to the user anyway.

Jon Colston (01:26:29.471)
key performance indicator.

Jon Colston (01:26:42.419)
Mm-hmm.

Jon Colston (01:26:49.336)
Mm-hmm.

Great.

Justin Gardner (@rhynorater) (01:26:58.487)
working and what their spend is on those specific things. So you can butt right up to that a little bit, right?

Jon Colston (01:26:59.607)
Mm-hmm.

Jon Colston (01:27:03.934)
Yeah, now I would say this caveat right now, if you were to go to Yahoo right now and look in there, you will find costs per, you know, bid, or you know, and that's because of

Justin Gardner (@rhynorater) (01:27:08.519)
Mm.

Justin Gardner (@rhynorater) (01:27:16.754)
Mm.

Jon Colston (01:27:17.178)
The section that's leaking that information right now is a public auction. It's pubs. So I'm save everybody the pain and agony of going, Oh, I found right now. So be careful on that. Okay. But there are, there are, there were times whenever it's a private art, a blind auction and you're able to strip and maybe you'll find, um, you find some KPI data that shouldn't be in there and, um, whenever it is, uh, an issue that is, um,

Justin Gardner (@rhynorater) (01:27:21.341)
Mm. Ah.

Justin Gardner (@rhynorater) (01:27:32.519)
Mm.

Jon Colston (01:27:44.702)
I guess persistent in the overall design of all your ad display operations. You then have a bug that gets rolled out everywhere.

Justin Gardner (@rhynorater) (01:27:56.389)
I see, I see, okay, that makes a lot of sense. And I imagine also, maybe tell me if I'm on base or off base, I've done a little bit of digital marketing myself, that being able to tell the contents of an ad before that ad runs, would that be something that has...

high impact to a marketer to say like, okay, you know, Ford is running an ad that says, you know, Toyota just break down more than, you know, Ford's or something like that. Having that information in advance, would that be valuable to Toyota or is that mostly just as is going to be public information anyway, it doesn't really quite matter.

Jon Colston (01:28:26.1)
Okay.

Jon Colston (01:28:37.344)
You know what, I mean, you can make the case because let's say I'm going to put a Blitz campaign on for the holidays, right? I don't want my competition to know what my marketing message is. This is part of my strategy, right? I don't know. They shouldn't know anything that I'm doing and any hint that destroys the efficiency of my... So yeah, I'd be mad as hell if that was a leak as an advertiser.

Justin Gardner (@rhynorater) (01:28:42.049)
Mm.

Justin Gardner (@rhynorater) (01:28:46.965)
Mm-mm-mm.

Exactly. Yeah.

Justin Gardner (@rhynorater) (01:29:04.221)
Yeah, interesting. I've got some ad platforms that I put in the doc that I'm not gonna mention explicitly, but that I've got some stuff on that I need to go back and revisit, I think, after this conversation. And maybe I'll hook you in.

Jon Colston (01:29:05.426)
So, yeah, I think if you...

Jon Colston (01:29:17.655)
I mean

There is, I mean, when you're talking about an aorta, is a cut throat type of situation. I mean, there's a lot of...

Justin Gardner (@rhynorater) (01:29:27.324)
Yeah.

Jon Colston (01:29:30.578)
Spy food I remember using and you're trying to do anything and everything to find out what your competitors keywords are what they're doing What ads what their performance was we use all kind of data aggregators to be able to They were basically crawling and screenshot and doing all that kind of stuff and providing reports. So it's Yeah, it's a competitive world. And so if you can find some information like that Yeah, no, and then I would definitely spin it up as like, you know, you're bread and butter your industry

Justin Gardner (@rhynorater) (01:29:35.022)
Mm-hmm.

Justin Gardner (@rhynorater) (01:29:52.657)
It's a cutthroat industry. Yeah.

Jon Colston (01:30:00.352)
make money from ad sales, right? And your core moneymaker is the thing that is at risk? No. Hi. Definitely.

Justin Gardner (@rhynorater) (01:30:03.26)
Mm.

Justin Gardner (@rhynorater) (01:30:09.437)
That's Ben. Yeah. I'd love to see you. I need to just invite you in to negotiate for me. Your bread and butter. Your bread and butter is on the line here. No. Hi. I love that. That's hi.

Jon Colston (01:30:13.204)
Yeah.

Jon Colston (01:30:21.703)
Right? Right, right. No, no, no. Yeah.

Justin Gardner (@rhynorater) (01:30:26.873)
End of story. That's great, man. So one of the things that's a part of your sort of hacker signature that I remember you from in the live hacking events is this whole concept of a Moab, a mother of all bugs, that I've heard you talk about, and there's a lot of chatter going around in the live hacking event scene when you were crushing it about John having some sort of secret weapon of a bug.

Jon Colston (01:30:42.338)
Yeah, right.

Justin Gardner (@rhynorater) (01:30:56.847)
Golden Goose, Moab, whatever you want to call it.

Jon Colston (01:30:58.461)
Alright.

Justin Gardner (@rhynorater) (01:31:00.973)
I'm wondering, you know, obviously in one or two live hacking events, you've had something like that that's just blown up and just you've found a ton of vulnerabilities with it. I'm wondering how often you find stuff like this where it's like very system. You had some criteria here that you mentioned in the interview. You said no one else knows to look for it. It can be found on different hosts, functionality, endpoint, making it ceasefire resistant, and it's a rating of higher critical impact.

Jon Colston (01:31:06.538)
Right.

Jon Colston (01:31:19.624)
Right.

Jon Colston (01:31:29.467)
Right. Yeah.

Justin Gardner (@rhynorater) (01:31:30.245)
That's what you've kind of defined. And can you talk to us a little bit about your experiences with those and how often you're finding them, what you're doing to find them? Yeah.

Jon Colston (01:31:39.226)
All right, so the one that we were alluding to earlier about finding, you know, you find information or data that you can extract information from in a place that it shouldn't be.

Justin Gardner (@rhynorater) (01:31:51.938)
Mm.

Jon Colston (01:31:55.11)
and it's everywhere, that is a Moab, right? That was a key fundamental design issue that was persistent and it was everywhere, right? It was across, think about every single place that an advertisement was displayed, it's, that's a big buck.

Justin Gardner (@rhynorater) (01:31:58.526)
Mm-hmm.

Justin Gardner (@rhynorater) (01:32:03.667)
Mm.

Jon Colston (01:32:10.766)
All right, so other ones, nothing has been like a Moab like that before, it was just one issue and it was persistent everywhere. I've been able to get into like a system and then be able to do a string of 30 bugs. That's another, that's not quite a Moab. I had a mini Moab with, in the, it didn't, it didn't, it didn't. So in,

Justin Gardner (@rhynorater) (01:32:11.106)
Mm.

Justin Gardner (@rhynorater) (01:32:21.661)
Mmm. Sure, sure, sure.

Justin Gardner (@rhynorater) (01:32:31.978)
It doesn't quite meet the measurement for a Moab, but it's a mini Moab. I'm like,

Jon Colston (01:32:38.682)
in that 702 that one left my first live hacking event.

Justin Gardner (@rhynorater) (01:32:38.713)
It's great.

Justin Gardner (@rhynorater) (01:32:42.37)
Mmm.

Yeah.

Jon Colston (01:32:46.226)
I stumbled upon the YQL servers. Everybody knew what they were at the time. You could interact with them, some queries for the most part, but you didn't know exactly the basics. Well, there was probably, I don't know, 30 of those servers. I'm just making that up. I don't know what the exact number is. Let's say there was 30 of those servers. And I found one of those servers that had...

Justin Gardner (@rhynorater) (01:32:52.695)
Mm.

Justin Gardner (@rhynorater) (01:33:03.769)
Sure, sure.

Mm.

Jon Colston (01:33:11.582)
an API document. It was the only one that had an API document. And that API document gave me an endpoint that then could apply to all 30 of those other, or 30 hosts. Yeah. And that API document led to two mediums for each host. And then, and then...

Justin Gardner (@rhynorater) (01:33:16.543)
Mmm.

Justin Gardner (@rhynorater) (01:33:36.809)
Hahaha

Jon Colston (01:33:43.586)
because of the endpoints that it would then, you know, continue to provide when she's fully down the complete path, there was other opportunities like SSF all over the place. So then there was another couple bugs here and there. So the end of the day, right, you had one API document on one host.

that gave you enough information to pull 75, 80 bugs. I don't, I can't remember how many it was. But it was from one endpoint, right? It rolled out. So it was something that applied to everything. Yeah, great time, great place. Yeah. It's a good thing you're not doing that. Yeah.

Justin Gardner (@rhynorater) (01:34:12.245)
Wow, dude.

Justin Gardner (@rhynorater) (01:34:20.677)
Wow, freaking Yahoo man, like they rock for paying for paying that sort of thing and also for having those sort of phones. I guess in those sort of scenarios, I'm wondering do you do you automate the report because it's like that's like 75 reports you got to write that that's going to take days just to write the report.

Jon Colston (01:34:29.987)
Yeah, they...

Jon Colston (01:34:44.075)
Yeah, it was simplified copy paste. I mean, it was simple. I mean, that was for the, I mean, it was copy paste, copy paste, and then I could refer to report, blah, blah. So it was simple, right? But as far as, you know, like automation, like that was before, that was before nuclei, right? And it was before FUF. And so I would, I had Nmap and I'd create a NSC file.

Justin Gardner (@rhynorater) (01:34:47.444)
Yeah.

Justin Gardner (@rhynorater) (01:34:51.388)
Mm.

Justin Gardner (@rhynorater) (01:34:55.846)
Right, right, sure.

Justin Gardner (@rhynorater) (01:35:04.993)
Mm.

Jon Colston (01:35:12.819)
And then it would do my vulnerability sculling. Yeah, yeah, yeah. So that was my, that was my Nuclei scanner. Yeah. And I dumped it out and.

Justin Gardner (@rhynorater) (01:35:13.725)
No way! John, are you kidding me? Oh my gosh! No!

Jon Colston (01:35:24.894)
So, I mean...

Justin Gardner (@rhynorater) (01:35:25.129)
Dude, that's like... that's like a really old way of doing it there.

Jon Colston (01:35:29.294)
Yeah, but I mean, I didn't know any better. I mean, maybe there were tools available. I really didn't. But yeah, it was like somehow I jumped onto the NSC map and that was 2016, right? There's a lot more efficient ways to do it today.

Justin Gardner (@rhynorater) (01:35:44.037)
Wow, that's nuts, man. I love how we all just got trounced by custom and maps script in that scenario. That makes me feel great. Thank you for that, John.

Jon Colston (01:35:53.646)
Hahaha

Jon Colston (01:36:00.13)
But no, honestly, it was really, you found, and so yeah, you can, so yeah, the results of that, the results of that life hacking event definitely overqualified me, because you see the one endpoint that led to that. Then there was this other little marketing, or no, sorry, not marketing, but there's a weird metrics panel I've never seen before. It was at one end.

Justin Gardner (@rhynorater) (01:36:11.979)
Mm.

Justin Gardner (@rhynorater) (01:36:25.918)
Mm, analytics or metrics panel, yeah.

Jon Colston (01:36:27.854)
Yeah, it was just, it was weird and I've never seen it before or after. And it was for a hot minute everywhere. And that's another 15, 20 bugs. And then, you know, layer in a couple of virtual hosts and you're done. And bang, I look like a rock star.

Justin Gardner (@rhynorater) (01:36:33.799)
Mm.

Justin Gardner (@rhynorater) (01:36:45.245)
Yeah, you can you can say that again dude that was that was crazy so I guess so in that scenario YQL server hitting finding an API endpoint on one of them that or an API documentation on one of them That just applied to all of them and just sort of mass apply the principle kind of like what you've talked about and then also Being able to identify this metrics panel that you can spray across all of them I think I think it's clear. You know your methodology is very

I guess distribution oriented as well. You know, hey, I'm going to take this thing. What can I apply? What else can I apply this to? And do you have any specific?

Jon Colston (01:37:22.062)
Exactly.

Justin Gardner (@rhynorater) (01:37:25.341)
tips for how to avoid dupes in this scenario? Because I feel like Yahoo's a great program for this. They pay them out, each one. But some programs would say, OK, whatever metrics panel, we're going to say, this is like a systemic problem. Dupe it all back to one. Have you had those? And how do you avoid those scenarios normally?

Jon Colston (01:37:32.798)
Mm-hmm.

Jon Colston (01:37:46.675)
Yeah, I have. I can't avoid them. Yeah. So each program has its own personality. That's what you have to understand, right? And the personality is defined by the management team of that Bug Bounty program. Some are going to be.

Justin Gardner (@rhynorater) (01:37:57.053)
Mm-hmm.

Justin Gardner (@rhynorater) (01:38:03.256)
Mm.

Jon Colston (01:38:03.89)
A, that's a unique change and we'll reward you accordingly. So you can have some that are much more favorable for the researchers than others. And others are very protective of the business. They're like, no, that's the same issue. It just apply differently.

Justin Gardner (@rhynorater) (01:38:10.485)
Mm.

Justin Gardner (@rhynorater) (01:38:17.46)
Mmm.

Jon Colston (01:38:22.49)
I've had those situations on the end. It's a gut punch. It's an absolute gut punch. And yeah, you've got to be thick-skinned about it. But yeah, I don't have any way to, unless you can kind of prove specifically that it's a unique effect on each. Yeah, I can go into some details on some.

Justin Gardner (@rhynorater) (01:38:29.933)
Mm-hmm.

Jon Colston (01:38:48.746)
on a particular experience that I had that was really rough. And, you know, it's been four years and I still have a little burn on it. And I'm a thick-skinned guy, right? It doesn't get it, but that one, yeah. And it was the same thing, right? It was like, nope, it's just this one fix. I'm like, you think so, but I can't argue against you.

Justin Gardner (@rhynorater) (01:38:53.786)
Mm.

Justin Gardner (@rhynorater) (01:39:01.07)
Yeah, yeah.

Justin Gardner (@rhynorater) (01:39:11.211)
You've mentioned a burnout that occurred and you've mentioned sort of the ups and downs of Bug Bounty like you're just talking about right now. I guess as a more seasoned hunter that's been doing this for years, what do you think, what are some things you do to protect yourself nowadays from?

Jon Colston (01:39:25.902)
Hmm.

Justin Gardner (@rhynorater) (01:39:32.817)
one burnout and two disappointment because my wife always says to me I always run you know whenever I find a vulnerability this is what I do John whenever I find a vulnerability I'm like I found something and then I get up and I run in the other room and I say hey Mariah you know I found this $15,000 volume that does XYZ and she's like did they pay it and I'm like no and she's like it's not a $15,000 volume is it you gotta

Jon Colston (01:39:44.151)
run.

Jon Colston (01:39:54.512)
Hahaha!

Jon Colston (01:39:59.918)
great.

Justin Gardner (@rhynorater) (01:40:00.233)
Temper your expectations, Justin. You gotta not count your eggs before they hatch because she's seen me go from way up here to go whew, down when it's like a dupe of a 35K bug like I had earlier this last year. So do you have any advice on that or do you just ride the wave?

Jon Colston (01:40:12.905)
Oh, I said this.

Jon Colston (01:40:19.351)
You know, it's hard not to run into the other room and do a little Victor dance. I mean, that's a, that, I mean, that you're just bug hunting. You, there's a dopamine hit, right? I mean, and it doesn't necessarily, I mean, sometimes it's the money. Sometimes it's just, oh my God, I just did this. I never thought that, but there's a dopamine hit from that. You know, right?

Justin Gardner (@rhynorater) (01:40:23.941)
Yeah. Yeah, exactly.

Justin Gardner (@rhynorater) (01:40:34.15)
Mm.

Justin Gardner (@rhynorater) (01:40:41.67)
Yeah.

Jon Colston (01:40:41.674)
that's like with any drug. And that's the problem that I got into, right? I was chasing the high, chasing the high, chasing the high, chasing the high. And that's a wrong way to go and it definitely earns to burn out. But yeah, it's so hard. I guess there was a saying that we always said, it's like, no, it's just business, right? Don't take it personally, it's just business. And that's what I go back to.

Justin Gardner (@rhynorater) (01:40:47.421)
Mm.

Justin Gardner (@rhynorater) (01:41:05.925)
Mm.

Jon Colston (01:41:09.862)
eases the pain a little bit, but no, it's still, it's still rough, it's still frustrating that second place is first loser.

Justin Gardner (@rhynorater) (01:41:19.477)
Mm. It's true, especially in, you know, dupe scenarios and stuff like that, where it's like, you know, Ah, I was just a little bit late.

Jon Colston (01:41:23.362)
Yeah. Do you do, do you do dupe analysis? I mean, do you go, I'm sorry to talk over you, but do you do dupe analysis? Like, no.

Justin Gardner (@rhynorater) (01:41:30.373)
I l- No.

Yeah, I do. It hurts a little bit to go back and say, and it kind of distracts me a little bit from the pain of the dupes too. You're like, all right, why did this get dupes? Why am I getting duped on this? And so you go back and you look, and sometimes you can draw a correlation when you've got monitoring stuff in place, and you can say, okay, I didn't react to this alert fast enough, especially when it's the stuff that's real tight. I duped by a couple hours or something. But most of the time for me, it's like, okay,

Jon Colston (01:41:42.603)
Right.

Justin Gardner (@rhynorater) (01:42:03.355)
this specific thing and somebody, you know, beat me to it by a little bit. That's just how it is. That's that's how the cards lie. But then when there's scenarios where it's like, ah, this bug has been open for six months and it's still not resolved, you know, that that's a real hard sell for me to go back to that program. Like, like I, and I tell, I tell them that in the report too. I'm like, Hey guys, I noticed that the report ID that I'm due to on this one is over six months old.

Jon Colston (01:42:12.747)
Yeah.

Jon Colston (01:42:23.426)
You're right.

Justin Gardner (@rhynorater) (01:42:32.485)
it's a crit, why is this happening? And I'll specifically request an explanation or some sort of response for that because I feel like that's a little bit of a violation of hacker trust in that scenario because it's like we are putting in our time for a results-based.

Jon Colston (01:42:35.063)
Yeah.

Justin Gardner (@rhynorater) (01:42:50.697)
thing, right? And we provided you a result. We provided you with a vulnerability. And if you can't protect our time with that by getting these resolved at a decent pace, then our ROI for this program goes down substantially. Yeah, so that's kind of where I'm at with DUP analysis. I don't know.

Jon Colston (01:42:50.978)
Yeah.

Jon Colston (01:43:08.51)
Yeah, no, to your point, like I always try to, you know, if I have some type of question. Sometimes I know a dupe. Like I'm putting in the report, like this, 100% this is a dupe. But sometimes if I don't understand, even if I know it's going to be a dupe, I like, how far am I behind? I'll ask for the report ID. I'm like, you know, 12 hours. I'm like, yeah, you know, let's see what happens here. But yeah. I was trying to think there was something else.

Justin Gardner (@rhynorater) (01:43:16.595)
Yeah.

Yep.

Justin Gardner (@rhynorater) (01:43:29.542)
Mm, yeah. Yeah. I imagine definitely in the, yeah, please ask more questions. I love it when guests do that. Especially in the scenarios when you're.

Jon Colston (01:43:37.059)
Yeah.

Justin Gardner (@rhynorater) (01:43:41.409)
in an automation flow. I remember we talked about it on the episode with Sean Yeoh from Asset Note. There was a time where him and I were just going back and forth and back and forth and back and forth with these takeovers. And I was like, dang it, Sean's beating me. Yes, I'm beating Sean. Dang it, Sean's beating me. It's just back and forth. It's a lot of fun, man. It's entertaining, but man, does it sting when you say, ah man, Sean made $10,000 today from something that I could have done

I just written my code a little bit better, you know? So, yeah.

Jon Colston (01:44:13.697)
Right, exactly.

Justin Gardner (@rhynorater) (01:44:17.073)
Yeah, it's a lot, man. I'm looking at this document. You know, we still have so much that we could cover on here. And we've had so much good content so far already. Unfortunately, we've got to bring it to a close because I do have a stop. But John, thank you so much for coming on, man. Is there anything else you want to shout out or you want to discuss in the last couple minutes here?

Jon Colston (01:44:41.854)
No, I just want to say thank you so much for having me on and for the content creators in this space, hats off to you. Oh my gosh, I know that's a burden of love but it's so greatly appreciated and what you guys do is fantastic so keep up the good work and be tuning in.

Justin Gardner (@rhynorater) (01:44:49.644)
Mm.

Justin Gardner (@rhynorater) (01:44:56.617)
Mm.

Justin Gardner (@rhynorater) (01:45:01.477)
That's awesome. Thank you so much, John. Yeah, I really do appreciate that. It is a lot of work, but you get a lot of love out of it, too, from people like you, and also you coming on the show and stuff like that. It really eases the load, because all I've got to do is sit here and be in, you know, dump my brain of, like, all right, how does he do it, you know? And then get the answers. So it's a great experience for me as well.

Alrighty, I think with that we'll close. Once again, John, Mr. My own eyes, Mr. Mayonnaise, thanks for coming on and we'll see you on the HackerBorn Leaderboards.

Jon Colston (01:45:37.204)
Yeah.

Jon Colston (01:45:41.33)
Sounds great, thank you very much. Bye.

Justin Gardner (@rhynorater) (01:45:43.829)
Peace.

Episode 56: Using Data Science to win Bug Bounty - Mayonaise (aka Jon Colston)

Jon Colston

Listen On

Recent Episodes