Episode 85: Practical Applications of DEFCON 32 Web Research

Joel Margolis (teknogeek) (00:00.056)
Jumpy jump.

Justin Gardner (@rhynorater) (00:01.109)
Yo dude, alright, check this out. Check this out, I got a good way to start this episode. Boom! You see this boy right here? This is a Space Raccoon pillow. This is wedding swag. So there was a competition, shout out to my boy Thilo who gave this to me. There's a competition at Space Raccoon's wedding. For those of you that are listening to a podcast in audio, like a total psychopath, I am holding up a pillow of Space Raccoon, one of my favorite hackers out there.

Joel Margolis (teknogeek) (00:04.97)
Okay. Space harcoon pillow? Dude. Dude, that... Is that wedding swag? That's crazy.

Justin Gardner (@rhynorater) (00:30.353)
And at his wedding, there was a competition of like, you know, how well do you know, Spacer Koon or whatever, not Spacer Koon, you know, his name's Eugene, right? But I always call him Spacer Koon. And there's like this whole thing. And unfortunately, like our team didn't win it perfectly because it's like, they also showed like vacations that they'd been on and been like, what city is this? I'm like, I don't know.

Joel Margolis (teknogeek) (00:40.118)
You

Joel Margolis (teknogeek) (00:54.658)
Dude, we were like GeoGuessr pros for like two weeks.

Justin Gardner (@rhynorater) (00:56.893)
I know dude, and I gotta give a shout out to one of the people on my team because she perfectly, she's like, that is the mountains in Italy. And I'm like, that is not the mountains in Italy. Like we were going Switzerland, we're going France, and we did, and she was right, it was Italy. So it's my bad.

Joel Margolis (teknogeek) (01:13.004)
Dude, okay, I cannot make this up. I cannot make this up. Like, literally, I cannot make this up, okay? I use a Chrome extension. It's called, what is it called? Moment or something? Momentum. I use a Chrome extension. It's called Momentum, okay? It's like, when you open a new tab, it shows you the time and stuff. The background picture is...

Justin Gardner (@rhynorater) (01:29.601)
Dude, you can't tell people what Chrome extensions you use, man. Now they're going to be in your browser. OK.

Joel Margolis (teknogeek) (01:43.35)
a lovely mountain range and at the bottom it tells me where it's located which is in Italy

Justin Gardner (@rhynorater) (01:50.347)
Where is it? my gosh, dude, given that, dude, what was the name of that range? I wanna say, no, it's not the Cologne, that's France. What is the mountain range in Italy that you're seeing? Maybe it's the same one.

Joel Margolis (teknogeek) (01:56.235)
Is it?

Joel Margolis (teknogeek) (02:03.958)
Let's see, it's in Gieler, wow, I do not know to pronounce this. The Giesler Peaks, Odal Grupa.

Justin Gardner (@rhynorater) (02:08.511)
Yes, dude, yeah, exactly. Dude, okay, what the heck? That might have been it, that's crazy. It was definitely something that started with a G. I can't remember right now, but that's Quintet. Anyway, I just had to flex this swag on the people because there are very few people in the world that have a space raccoon pillow. And I literally went up to Thilo and the guys in the team that won this afterwards, was like, listen guys, I need that. You'll understand, like.

I really need that. Like this is serious bragging rights for me in the security world. And they're like, all right, sure, Justin, whatever, take it. know, like, dude, I'm going to find somewhere to put it. I kind of want to like put it right by this monitor here or something like that. But yeah, shout out to my boy Thilo. I tried to bribe him with like a hundred dollars. I was like, now I'll give you a hundred dollars USD. And he's like, dude, just take it. Like, I'm like, really? Like, so, all right.

Joel Margolis (teknogeek) (02:44.952)
All right, you need to put a shelf on your back wall and hang it on there.

Joel Margolis (teknogeek) (02:52.834)
Yeah.

Joel Margolis (teknogeek) (03:02.101)
That's awesome.

Justin Gardner (@rhynorater) (03:05.151)
Alright, so what are we doing today? We are discussing DEFCON research.

Joel Margolis (teknogeek) (03:13.112)
Defcon, Defcon, not Blackhead, only Defcon. Well, okay, okay. But yeah, security summer camp research episode. I mean, there was a ton. mean, at this point it's impossible to keep up. I feel like maybe like, yeah, maybe like five, 10 years ago, you go to Defcon, you could probably see a lot of the talks.

Justin Gardner (@rhynorater) (03:14.507)
This is, well, mean, yeah, mean, no, no, I mean, there is some black cat in there, you know. Yeah. This is, it's...

Justin Gardner (@rhynorater) (03:24.551)
Exactly and there's a lot dude Yeah

Justin Gardner (@rhynorater) (03:31.861)
Yeah, but we're gonna do our best to help you guys.

Justin Gardner (@rhynorater) (03:39.915)
Mm

Joel Margolis (teknogeek) (03:40.596)
Now it's so difficult to see like I mean there's like three four tracks right plus the village tracks plus so I mean this just Yeah, yeah, the bug running village was awesome. There's like a lot of really good stuff going on So we met a bunch of people at the bug bounty village, which was really cool. We met people at

Justin Gardner (@rhynorater) (03:46.933)
Mm

And the villages were fire this year,

Justin Gardner (@rhynorater) (03:58.101)
Yeah, shout out to the squad we met over there. That was great. Dude, this is my first time in Vegas as a content creator and it rocks, man. People are always constantly coming up to you, be like, hey, it's you. And I'm like, yeah, who are you? And they're like, from the pod. was like, yeah.

Joel Margolis (teknogeek) (04:14.484)
Did I literally I think I had a dream about that last night where somebody was like yo, techno geek I listened to the pod and I was like, hello

Justin Gardner (@rhynorater) (04:20.614)
Yeah, dude, it's we need to refine our skills on that a little bit like I feel like I was not prepared going into Vegas for that and and I I need to like learn how to like carry the conversation and like, you know have that interaction. I think it went fine in general. But you know, if I had a weird interaction with you guys, sorry about that. Like I wasn't really expecting all that. So

Joel Margolis (teknogeek) (04:40.706)
Yep.

Yeah, you get to watch a edited version of one hour of us being probably as normal as possible. Yeah. Yeah. Yeah.

Justin Gardner (@rhynorater) (04:49.493)
Dude, I don't know, man. We've been goofy as heck the past couple weeks. yeah. But yeah, shout out to all y 'all that made us feel very special and loved at DEF CON. Really appreciate that. That was great, guys.

Joel Margolis (teknogeek) (04:58.646)
Yeah, very cool. Very cool meeting everybody. We gave out a bunch of stickers. I think we need to print like five times as many next time.

Justin Gardner (@rhynorater) (05:05.313)
Yeah, we ordered like what? 150, 300 stickers or something like that. I think two batches of 150. This is gonna be plenty.

Joel Margolis (teknogeek) (05:09.88)
Like yeah, we were like, this is gonna be tons and in like five minutes I had like ten left. I was like, okay

Justin Gardner (@rhynorater) (05:16.723)
Yeah, they were gone fast and we had to start rationing them. We're like, all right, we're to take one pack per day per person. You know, we had

Joel Margolis (teknogeek) (05:21.65)
Yeah, I had to take some for myself because I was like I'm gonna run out of these before

Justin Gardner (@rhynorater) (05:25.793)
That's funny. Yeah, we get to the end. We're like, my laptop doesn't have a CTBB sticker on it. Like, that's funny, dude. Yeah, we also gave out a bunch of swag. We do have the swag store live, which is great. So we've got that over at ctbb .show slash swag. So check that out if you guys want, if you didn't get one at Vegas. And the critical thinkers, of course, get a very steep discount on the swag. So

If you're a critical thinker in the discord, then check out those that discount code that's in the exclusive content channel And with that Joel, I think we move into the content we I have selected I have selected three four Write -ups that is pretty much gonna be all we have time for today and we actually may need to come back and do like another episode after all the videos air and stuff like that because

The ones that we chose for this episode, essentially all the Portswicker team, and then Orange, who we can't ignore. They did really nice text write -ups as well of their research, which I really appreciated. But I'm sure there's a lot of people that just did videos, or just did the presentation and didn't do a text version as well. So I definitely am looking forward to when everything comes out where we can go and watch back some of the other research.

Joel Margolis (teknogeek) (06:32.189)
The Forza team and Orange, yes.

Joel Margolis (teknogeek) (06:52.288)
Yeah, absolutely. I know a lot of stuff was live streamed, stuff might... Certain things might already be live. I haven't checked to see... Yeah. Yeah, I haven't checked to see how many talks are like split into their own videos and stuff online yet, or how they're gonna do all that, but I think they're usually pretty quick with it. So I would imagine... What was that reaction?

Justin Gardner (@rhynorater) (06:58.122)
yeah, that's true. They did livestream it this year.

Justin Gardner (@rhynorater) (07:05.505)
Mmm.

Justin Gardner (@rhynorater) (07:13.569)
I opened up the Defcon YouTube channel and it goes, super loud, like extremely loud. I think maybe your mic is down low, so I've got you turned up. And then I went to this page and it was like super loud.

Joel Margolis (teknogeek) (07:28.605)
man, I'll turn myself up a little bit right now. Sorry. Sorry to our audio editor.

Justin Gardner (@rhynorater) (07:30.611)
Okay, now I'll turn you down. Yeah, sorry, sorry Richard. Okay, gotcha. Yeah, I mean, it seems like there are some videos up here, so that's interesting. We'll have to go back and try to view some of those in prep for next week. But it is heavy, man. The prep is heavy for these Def Con episodes, but it's worth it because, yeah.

Joel Margolis (teknogeek) (07:52.236)
Yeah, I mean, it's like a full -time job, right? Like I think in order to educate people on a topic like this, you know, either you or them has to read through the entire thing, learn it, understand it, and then be able to explain it. So it's definitely, it's a lot of like content consumption to be able to like understand the research. Yeah.

Justin Gardner (@rhynorater) (08:08.906)
Mm

Yeah, yeah, for sure. Okay, I'll start us off with this first one. Of course, we're gonna start off with the man, the myth, the legend, James Kettle, in his research this year. This was on, this is entitled Listen to the Whispers Web. Okay, well also, hold on, let me just go back for a second. I promise you we will get to the research, but the idea behind this episode is for us to go through this research and take the pieces out that you need to know about and you can practically integrate into your hacking methodology and then talk about those.

I would also recommend that you go and read all these research pieces, papers and stuff. But if you're just listening to the podcast, the hope is that your hacker brain should be primed so that when you see something that relates to this, you might think, okay, maybe I need to go back and reread that research or something like that. Or maybe you get some nice takeaways that you can just integrate into your methodology as a freebie rather than having to go and read all this research. that's the thought for this. And with that, listen to the Whispers Web Timing Attacks by James Kettle.

This research I think was really interesting. think my take on this is that it's not normal James quality in my opinion. I think James normally has some stuff that is extremely internet breaking and massive. And I think this is very useful. I think this is a useful tool. And he even says this throughout the write up. says, if you ignore timing, you're going to miss out.

but if you focus too much on it, then you also miss out. So I think this is less of a internet breaking technique like we've seen in past years and more of you just gotta kinda know how to utilize these timing attacks in the way that he describes here so that you can exploit some really fringe situations in black box scenarios.

Joel Margolis (teknogeek) (09:42.209)
Yeah.

Joel Margolis (teknogeek) (09:59.926)
Yeah, absolutely. I think one really interesting thing about not just, I mean, this isn't exclusive to James's research, but a lot of this sort of very specific, specifically scoped research, like these timing attacks stuff that he's been working on for the last however many years is what you'll find is that the large organizations who end up being at a certain point super secure, that's probably where this is going to start to come into play.

Justin Gardner (@rhynorater) (10:15.777)
Mm

Joel Margolis (teknogeek) (10:29.776)
And the ones who have secured almost every attack vector, except for these super weird like niche, like rent, like architecture specific, like, yeah, like that's where, and it's going to be super critical, right? But it's that, that sort of research like has to be like very targeted and fine point for it to come out. And I think especially, you know, with the previous research that he's done, that's helped us secure so many other of the attack vectors and so much of the stack as a whole that.

Justin Gardner (@rhynorater) (10:34.805)
Mm -hmm.

Fringe timing stuff, yeah.

Justin Gardner (@rhynorater) (10:42.155)
Mm

Justin Gardner (@rhynorater) (10:48.523)
Mm

Joel Margolis (teknogeek) (10:59.152)
you know, just naturally this is going to start to be like more specific and more like harder to exploit and all that kind of stuff.

Justin Gardner (@rhynorater) (11:06.773)
Yeah, and I think he says in the write up as well, these things are everywhere, these timing attacks, but I think you're right in that the only places where they really need to be exploited are in these really, really like super fringe scenarios. And that being said, there is some really cool techniques that we'll talk about from this that I definitely think you should know about that are really amazing and applicable in a lot of ways.

But the actual exploitation of this at large, I think already requires there to be some sort of sketchy functionality. And then you are taking that and building upon it and using timing attacks to like gain more information about what's going on and that sort of thing. So I think that's really cool. I just want to clarify something I said, like this is 100 % James quality research. Like the research is very thoroughly done, very well documented, very...

thorough technique like clearly this guy is a master researcher, right? I just I just you know the stuff that I'm used to seeing from him is like shit, if I do not implement this right now, then I'm gonna be missing out big time, right? And I think this is like we said more of an exploitation technique. So With that let me let me kind of go into some of the stuff that he he did here. That was pretty cool one of the things that came out of this was dual packet sync as a as a methodology to

sort of be close to that single packet attack and improve upon the single packet attack that he had been using before. And essentially what this new technique ensures is that the HTTP server doesn't start processing the HTTP headers before it gets the body. Because how single packet attack normally works is you would just sync the last piece of it in one packet and that would cause everything to get processed at once.

But what he figured out was that some servers were actually starting to process the HTTP headers before, even before they got the rest of the last little data frame from the body. And using this technique, he was able to make it all sync together so that everything, including the headers, get processed at the exact same time, which is pivotal for this sort of very, very granular type of timing attack that he talks about in this paper, because it's like, I mean, in one of the scenarios,

Joel Margolis (teknogeek) (13:05.932)
Hmm.

Justin Gardner (@rhynorater) (13:28.417)
He was exploiting like a point to, okay, let me think, a point to, yeah, you could call it nanoseconds, but I'm gonna say like 0 .2 millisecond difference in some of the stuff. Like he was just very consistently seeing that this one takes just tiny, tiny bit longer, which is just super badass. And I totally agree that I thought this level of timing attack was not possible, but after seeing the way that he implements it in here with getting the server to process it exactly at the right time.

Joel Margolis (teknogeek) (13:31.67)
It's like microseconds or something, right?

Justin Gardner (@rhynorater) (13:58.606)
I think it's pretty well solved. I think he's made timing attacks practical with these tools.

Joel Margolis (teknogeek) (14:07.094)
Yeah, I mean, it's really cool. think this has made me think a bit about like the efficiency aspects that go into a lot of software, especially for like large organizations. When you think about like, you know, Google, for example, like think about how many requests per second Google has to process. And not only that, not just process, but they have to be able to like run it through.

Justin Gardner (@rhynorater) (14:22.762)
Mm

Joel Margolis (teknogeek) (14:31.818)
other software stack and like have middleware and routing and all this kind of stuff. And they need that to happen in like two milliseconds. Right. And so there are certain engineering decisions that have to be made in order to make a piece of software that can operate at that speed and doing stuff like parsing the headers, like as the packets are coming in are like the decisions that lead to these kinds of vulnerabilities. Right. So I think.

Justin Gardner (@rhynorater) (14:33.622)
Yeah.

Justin Gardner (@rhynorater) (14:38.773)
Mm -hmm.

Justin Gardner (@rhynorater) (14:56.747)
Yeah.

Joel Margolis (teknogeek) (14:59.384)
That could probably be a really interesting research point if somebody wants to like go deep dive something like envoy for example, that's like a large -scale like edge routing software that's used by a lot of organizations, I wouldn't be surprised if there is something similar there because Efficiency like is key when it comes to processing requests like this

Justin Gardner (@rhynorater) (15:02.731)
Mm.

Justin Gardner (@rhynorater) (15:19.349)
Yeah, yeah. And I mean, at scale, if you can cut off just a little bit of time, you know, in server load, it has a massive impact. And some of those things will cause vulnerabilities inevitably. the other, the other, so two more points here that I, that I wanted to hit before we talk about some of the other like more practical vulnerability stuff that he sort of coined new terms for. One, he used a really cool technique in here.

that I think is a little bit odd because of the way that it is. But I think it's a cool thing to know about nonetheless, which is he was trying to detect reverse proxies using these timing attacks. And he was able to detect whether DNS resolution, like blocking DNS resolution was occurring by using a DNS record that has an over long domain segment. Or what is the?

What is the actual technical term? I didn't write it down. But essentially, know, domain label, I believe. So like one of the segments of the domain, right? If you make that, I want to say 65, maybe it's 60, yeah, greater than 64. So if you make it 65 characters, then it won't even attempt resolution. It'll say, hey, that's not a valid domain. Like, I'm not going to touch that, right? So if you send one that has like 63 or 64, and then you see a certain response time and you see one that says 65, and then it's a shorter response time, then...

you can say with pretty good certainty that that domain or that DNS resolution is not occurring or that is occurring and that isn't occurring afterwards, right? So you can tell that differential and then use that to determine whether you're dealing with something that's actually resolving that host and might be a reverse proxy or is not resolving it.

Joel Margolis (teknogeek) (16:54.423)
Huh.

Joel Margolis (teknogeek) (17:08.408)
It's such cool research because it's like invisible fingerprinting where it's just like, you know, it's like the next level of error message, like Googling error messages. You know, it's like, this responded in 60 milliseconds. It's Apache.

Justin Gardner (@rhynorater) (17:18.934)
Mm

Justin Gardner (@rhynorater) (17:25.235)
Yeah, dude, is really crazy. And I also, he uses a really awesome technique as well. And I think this kind of goes into how versatile you should really be as a hacker and how various areas of expertise can apply to hacking in a really good way. He uses some statistical analysis on all of these to make sure that he removes.

any sort of confounding variables that might be occurring with like spikes in server load and that sort of thing by just using the lower quartile of response times. And he found that that gives him the best picture. And that's not like some crazy statistical technique, right? Like, I'm like charting it and then doing this and standard deviation or whatever. But it is like a pretty useful piece. And I think having that in my brain in knowing like, okay, well, if I wanna, you know,

reduce volatility in these timing attacks, I should use the bottom quartile of responses because anything longer than that, something else is happening clearly in the server, right? Because that bottom quartile is gonna represent the fastest that it's being processed. And so, yeah, just kind of having that in my head and knowing that, I think is one of those great little like boop takeaways from the paper of like, okay, if I am doing timing attacks, that's how I should approach it.

Joel Margolis (teknogeek) (18:49.43)
Yeah, I mean it's such a weird thing when you think about just that like, almost like seemingly randomly, right? Some requests will just like go slower or not work because server load, like something else is going on. it's just like, it's very, you know, we like to think that computers are super, super consistent, but no, they definitely aren't because there's like so many other external factors that can affect stuff, which makes this even.

Justin Gardner (@rhynorater) (18:59.071)
Yeah. Server load or whatever, right? Network jitter.

Joel Margolis (teknogeek) (19:18.145)
Just a little bit more impressive again because like the precision that was being done down to like the nanosecond on Some of these attacks. It's just like crazy crazy

Justin Gardner (@rhynorater) (19:26.133)
Mm -hmm. It really is. It really is. Okay. So the other thing that he had here was he coined a new term, which is called scoped SSRF. And so this is a new variant of SSRF, sort of. And this occurs essentially when you are only allowed to perform an SSRF to hosts that are in a certain subdomain. So let's say, you know, I'm attacking Google. I can only hit

you know, whatever .google .com, right? And he coined this term in the, excuse me, in the section of the paper where he was talking about how to discover these sort of reverse proxies, right? Which is very similar to the research he did many, many years ago on host header stuff and finding crazy, crazy hosts that you can access via host headers. And he differentiated it.

a reverse proxy from vhost scanning, is vhost is just when other hosts are using that same server. And he differentiated and said, you know, this is sort of a different scenario with reverse proxies because you've actually got a server that is a reverse proxy that is reaching out to a different server on the internal network based off of the host header that you're providing. And one of the cool techniques that he did over this whole piece of research was

essentially building out a good way to identify these reverse proxies pretty consistently and and that that is the architectural decision that they're making using that sort of DNS trick and some other timing stuff to identify these reverse proxies in the organization and then hitting those reverse proxies a little bit harder with like vhost scanning or Not really vhost scanning but like host header scanning right to see if you can pivot into different parts of the network And I thought that was a really elegant solution. I think that's something that I could easily see like asset note or like

Joel Margolis (teknogeek) (21:05.901)
Mm.

Justin Gardner (@rhynorater) (21:17.749)
some of those people that are doing very wide scanning and really wanting good understanding of network topology of their targets, implementing that in like a microservice or whatever and just saying, okay, well, XYZ organization has three reverse proxies. They're here, here, and here, because now we've got a more actionable way of actually testing this. So that was a big one. I think that was a big one, a big takeaway from the research and absolutely very cool.

Joel Margolis (teknogeek) (21:38.689)
Yeah.

Joel Margolis (teknogeek) (21:44.096)
Yeah, super cool. I didn't see whether or not he called it out, but I am really curious like what the root cause of that. That's like the scoped SSRF stuff is in terms of like, because I know I mentioned this is sort of like misconfiguration of reverse proxy, but you know, from like an app six side, it's always nice to know like, what can you actually fix? Like, how do you fix like, yeah.

Justin Gardner (@rhynorater) (21:56.725)
Well, yeah.

Justin Gardner (@rhynorater) (22:01.013)
Mm

Justin Gardner (@rhynorater) (22:07.361)
Joel's brain is spinning like, reverse, do I have a reverse proxy anyway? Yeah, so essentially what it is is when the reverse proxy is configured to hit any like star dot whatever dot com, right? And then it's, you know, you can pivot into the internal network and abuse that firewall or that, what is that DMZ, that DMZ position, right? You know, where that.

Joel Margolis (teknogeek) (22:30.281)
okay.

Justin Gardner (@rhynorater) (22:33.281)
reverse proxies on the internet and in the internal network and sort of allows for internal pivoting. But yeah, and then he does that, right, of course, via that timing attack plus that DNS resolution piece of like, okay, if I see a very significant statistical difference between the response of a 64 character domain label and a 65, then it's a very good chance that there is some DNS resolution happening that is taking, you know, 100 milliseconds or whatever.

Joel Margolis (teknogeek) (22:34.444)
Yeah. Yeah.

Justin Gardner (@rhynorater) (23:02.633)
And that's why we're seeing that difference.

Joel Margolis (teknogeek) (23:05.944)
Yeah, yeah, totally. mean, it's really crazy. mean, and again, it's just like, it took 60 milliseconds. It's Apache. Like, it's like, it's such like, it's like this invisible, like, fingerprinting where, like, what's wrong with it? Like, nothing's wrong with it. It's just how it behaves, right? Yeah.

Justin Gardner (@rhynorater) (23:13.631)
Yeah.

Justin Gardner (@rhynorater) (23:24.671)
Yeah. Yeah. And he says that throughout the paper. He's like, yeah, there's not really like a bunch of great defenses for this, you know, because it's just how stuff works, you know? Sleep. Yeah. Sleep like three.

Joel Margolis (teknogeek) (23:34.488)
just start adding random response delays to every single, it's like the anti -efficiency, like, just like, okay, we're gonna add random amounts of delays so they can't fingerprint what our systems are.

Justin Gardner (@rhynorater) (23:46.221)
My gosh so bad, dude, but those are the worst kind of things right like if there's literally no fix to this like What are you gonna do it? You know? so Okay, last little piece I was talking with Franz and actually I think Probably this episode will air before the one that that we recorded with Franz so You guys will hear more about this. I think next week

Joel Margolis (teknogeek) (23:54.113)
Yeah. Yeah.

Justin Gardner (@rhynorater) (24:13.767)
if the ordering goes right, we'll see. It depends on how long we run today. Franz was recently talking about JSON injection with me, and I thought that was really interesting because I don't really hear a lot about JSON injection. And then James also mentioned JSON injection in this research. And it just starts getting you thinking, Like two of the most respected hackers that I know of, right, are both talking about this at the same time, and it's not something that I hear about a lot. That's probably something that should

Joel Margolis (teknogeek) (24:17.559)
hahaha

Justin Gardner (@rhynorater) (24:43.679)
should trigger some thoughts, right? So I don't really know what that looks like yet. Franz seemed to insist in that episode that there are scenarios where people are essentially building JSON blobs with string concat, and you're able to actually add other attributes and stuff like that into that JSON blob. I have not seen that very often. But he insists that it's around. And then James is also mentioning something like this in his research.

Joel Margolis (teknogeek) (24:47.309)
Yeah.

Justin Gardner (@rhynorater) (25:12.009)
So I just wanted to put that out there for the people, know, the curious critical thinking listeners out there that will also poke and think and, you know, start identifying patterns in their targets because, yeah, two big names are really interested in that right now.

Joel Margolis (teknogeek) (25:27.476)
Yeah, mean, for sure. And just like, I'll say like injection in general, I feel is making a comeback. You know, people used to think of it as like SQL injection, but it's basically, I mean, it's the same thing. It's just in all the other technologies, right? So, okay, you're building a JSON object. Yep. JSON object injection, right? You're concatenating a string. You can escape that property with like an ending quote and then start to add new properties in. GraphQL injection, like again, like

Justin Gardner (@rhynorater) (25:35.093)
Mmm.

Justin Gardner (@rhynorater) (25:42.283)
Mm

Joel Margolis (teknogeek) (25:56.908)
building queries with string concatenation, like string concatenation and just inserting strings without sanitizing them or without having like a proper flow, right? Like the traditional solution for SQL injection is prepared statements, which like pre calculates a path. So you have to figure out like, is the, that, that, what is the, the, the prepared statement for Jason, right? Like you build the object and then you specifically like go attribute over I attribute or something, right? Like, you know, there, there are security ways to do it, but

I think in a lot of places people have gotten lax because there's not a traditional security risk associated with it. And people go, mean, what's wrong with concatenating a string in a JSON object? Like what's going to happen? Well, you know, maybe you're able to pass that into a function that then interprets those as parameters. And, you know, you're able to override parameters or add additional options that weren't, you know, published.

Justin Gardner (@rhynorater) (26:45.333)
Yeah, without, you know, talking too much about Franz's research for next episode, like that's exactly what he was talking about. Like being able to inject in these sort of trusted JSON blobs and then be able to overwrite specific pieces, specific attributes because of like deduplication of attributes and that sort of thing. Very interesting stuff, very interesting stuff. I'll definitely be paying more attention to scenarios where I think that could be occurring.

Joel Margolis (teknogeek) (27:09.356)
Yeah. Yeah. Very cool.

Justin Gardner (@rhynorater) (27:15.165)
And we'll see where that goes over the next couple of years. All right. That's all I had for James's research. Do you have anything else on that?

Joel Margolis (teknogeek) (27:23.34)
I don't think so. mean, hopefully we did it justice. I mean, it's definitely worth giving it a read. So if you're unsure about anything that we said or wanted some more clarification, go check out the article on the Portswager research page.

Justin Gardner (@rhynorater) (27:39.157)
Yeah, absolutely. I know James listens. So James, I'm sorry if I messed something up, man. Like, forgive me, but master of security research. but I think, I think it was good. All right. Where you want to go next? Cause we're pretty much hitting the, let me just pause and also say like the port swagger team crushed it this year with research. And, and really like, we're going to spend pretty much this whole time talking. I should probably title this episode like port swagger research.

Joel Margolis (teknogeek) (27:43.65)
Ha ha.

Joel Margolis (teknogeek) (27:48.596)
You

Joel Margolis (teknogeek) (28:00.321)
Yeah.

Justin Gardner (@rhynorater) (28:08.708)
know, brain dump or something like that because like it's pretty much all portuguese research and then we're going to talk a little bit about orange at the end.

Joel Margolis (teknogeek) (28:09.911)
you

Joel Margolis (teknogeek) (28:16.256)
Yeah, I mean, I don't know how many talks they submitted, but I know they had at least these three that got accepted. there's, I mean, it just goes to show that there's a lot of really awesome research going on over over at Portswinger, right?

Justin Gardner (@rhynorater) (28:20.884)
Yeah.

Very impressive.

Justin Gardner (@rhynorater) (28:29.769)
Yeah. Where do you want to go next? want to do the gotta, gotta cash them all or do you want to do the splitting the email at them?

Joel Margolis (teknogeek) (28:35.032)
as it is for you know i don't know if you're a cool

Justin Gardner (@rhynorater) (28:39.219)
Alright, cool. You want to take that one or should I?

Joel Margolis (teknogeek) (28:42.424)
Well, let's take it together. So this research was from Gareth Hayes, who we talked about, an inventor of Hackverter, inventor of Shazer, which I think just released. You can now use your friends' browsers to like fuzz your JavaScript payloads or something. I don't know. He tweeted something out about that. basically this research all centers around

Justin Gardner (@rhynorater) (28:48.554)
Mm

Justin Gardner (@rhynorater) (28:53.034)
Shazzer.

Justin Gardner (@rhynorater) (29:01.259)
Pretty awesome. It's pretty cool. Yeah.

Joel Margolis (teknogeek) (29:08.696)
the email format RFC, which is what RFC 2822, I think is what it says. So basically it just talks about like, know, RFCs are really interesting. And I think probably at least once a year, there's some sort of talk at either DEF CON or Black Hat about an RFC that is not being interpreted correctly or is being interpreted too correctly. Or maybe it was written in a way such that there are pitfalls that nobody really thinks about.

Justin Gardner (@rhynorater) (29:13.473)
Mm

Joel Margolis (teknogeek) (29:38.04)
And so I think email address parsing is definitely, probably one of the bigger ones, right? I mean, this has been talked about for a while. I think I want to say Zshana was the one who really made that popular initially of putting XSS payloads in an email.

Justin Gardner (@rhynorater) (29:46.175)
Mm -hmm. Yeah.

Justin Gardner (@rhynorater) (29:54.401)
I saw him talk about it, but for me the first interaction with this was Inti's research on it back in the day, and then with Ticket Trick and a bunch of other stuff like that. then I know Rojin also took it for a little while and did some stuff with it. So there's been a lot of people that have poked at this email RFC, and Gareth comes in and blows our minds with more of that.

something says that there's probably more to be discovered in the email research realm.

Joel Margolis (teknogeek) (30:28.598)
Yeah, yeah, I mean if if nothing else this I know that he had to update this article actually afterwards and he was like, yeah, I had some extra Some bonus material that I added to the end of this that were like some more injection payloads that were like worked in more specific scenarios and like Yeah, I mean, it's it's really crazy research. So what is it? so basically there a lot of it boils down to the fact that there are

certain characters that you can, that are valid in an email address, especially in the like first part of the email address before the at symbol that end up either being transformed or interpreted or parsed in a way that allows you to do way more than just provide an email. So oftentimes, you know, a lot of this was ending up as like SMTP injection, changing like the SMTP directives or commands that were being put in like a raw email, like

text or command or whatever that's getting sent to the email server. So yeah, I mean, it's really, really cool research. think one of the coolest things was this whole like equals question mark encoding thing. Yeah, yeah, yeah. RFC 2047, another RFC. Very good, very good. It's called encoded word. Yeah, and it's like this encoding system that allows you to write

Justin Gardner (@rhynorater) (31:28.224)
Yeah.

Justin Gardner (@rhynorater) (31:40.821)
Yeah, dude, I freaking loved that part. Encoded word or whatever, right?

Joel Margolis (teknogeek) (31:56.312)
Encoded characters is like hex and it's like so foreign because if you look at this screenshot You're like what what the heck is this like I've never this is a thing like what what is this? What is this thing? And yeah, apparently it is a thing you can like send hex encoded characters like an email address in like this raw format that like tells it Here's the character set and here's like the data and then it ends up decoding it Somewhere

Justin Gardner (@rhynorater) (32:05.92)
Mm

Joel Margolis (teknogeek) (32:25.58)
I don't really know where.

Justin Gardner (@rhynorater) (32:25.601)
Yeah, I'm gonna actually gonna paste this image into the doc really quick and make a note for Richie. Add this to the screen if possible. So for any of you watching on YouTube, hopefully that will be up on the screen right now. But essentially, Joel, when I saw this, I was super triggered because I did a DEF CON talk back in 2022 with Sam Erb, essentially an exploitation story of how we popped this enterprise software, but it had some crazy parts to it. And one of the crazy parts was,

this sort of encoding that you see right here, which starts with like equals question mark, and then you provide like the encoding type, and then you do another question mark, and then you just essentially start writing the text you want with equal sign and then the hex value, equal sign, hex value, equal sign, hex value. This also works in HTTP headers, man. So you can literally, yeah, hold on, let me link you to this presentation. like, because I saw this and I was was in prep for this episode.

Joel Margolis (teknogeek) (33:15.158)
We might...

you

Justin Gardner (@rhynorater) (33:23.157)
And I was super triggered because I was like, my gosh, why didn't I look deeper into this? And I sent a message to Gareth and I was like, dude, have you seen this? And he's like, whoa, didn't, like that can be done in HTTP headers as well. So there's definitely some cool applications for smuggling with this. I just sent it to you on Discord. But this type of encoding with the equal sign and the question mark, yeah, exactly, and Sherrypie is a common way, it seems, for various.

Joel Margolis (teknogeek) (33:26.966)
Wow.

Joel Margolis (teknogeek) (33:43.092)
my god and cherry pie? No way.

Justin Gardner (@rhynorater) (33:51.775)
internet systems, especially the stuff that was created way back in the day, like original HTTP protocol and original SMTP, to encode values that may not have a super clean UTF -8 or ASCII representation, right? And so he's using that again here to essentially get these emails to have sort of malicious characters in them, which then get mis -parsed by either the application layer or by more impactfully.

via the actual SMTP server that's sending all this stuff out, which is genius, really.

Joel Margolis (teknogeek) (34:26.456)
Dude, it's really, really interesting that that is specifically in the code. I actually went and I looked too, and it's still there. I don't know when you made this slide, it sure looks... Wait, did they fix something or... okay. I was gonna say, yeah, because I think it still does that.

Justin Gardner (@rhynorater) (34:34.741)
Yeah.

Justin Gardner (@rhynorater) (34:38.251)
Yeah.

It's still, no, no, there's not, you know, cause I hit up Cherypy and Cherypy was like, no. And then Nginx was like, that's Cherypy's problem. then they just, so it's one of those, you know, mismatches again, where it's like Nginx doesn't see it, doesn't parse this sort of, what is it? ISO 8859 -1.

Joel Margolis (teknogeek) (34:49.739)
No.

Joel Margolis (teknogeek) (35:05.918)
Yeah, I they literally use decode header from email .header.

Justin Gardner (@rhynorater) (35:09.727)
Yeah, yeah. So that's not fixed in ShareAPY and there's probably other HTTP servers that also support this ISO8859 -1 encoding type. And then there's also this sort of same sort of thing in emails apparently, where a valid email can have like a bunch of question marks and a bunch of like hex encoding. And then when the email actually gets sent, you know, it'll get sent to whatever the ASCII representation of that is.

I'll give our listeners the benefit of the doubt here that they can maybe visualize this. Okay guys, listen closely. this email looks like equal sign question mark, UTF8 question mark, Q question mark, and the ASCII representation in hex. So for example, ABC would be 61, 62, 63. So it's equals 61 equals 62 equals 63 at whatever .com.

And then when the email actually gets sent, it gets sent to abc at whatever .com. and so it's, it's really odd. and I think this is the main, I think you successfully identified Joel, like this is the main takeaway, right? Like from this whole thing. Cause if you look at all of the exploits that he has in, in that, that write up, all of them take advantage of encoded word. So a really, a really easy probe that you can throw out there really quick is just have your email in encoded word and.

Send that in and if you get the email then okay there might be something a little bit easier to To you know, this is an easy way for you to identify that this is something you need to look at a little bit deeper

Joel Margolis (teknogeek) (36:48.896)
Right. And I think one of the big ways that he was able to take advantage of this was around like auto invite stuff as well. Right. So like the, the, this is basically like, yeah, it's like in encoding bypass for like email, especially, which is then used in a lot of different places. like, save a work Slack and anybody who signs up for that Slack with the company domain email gets automatically added to the workspace.

Justin Gardner (@rhynorater) (36:57.845)
Yeah, which is everywhere.

Joel Margolis (teknogeek) (37:18.828)
Well, you encode your email and encoded word and the front end goes out, this is a different domain. And then the email system interprets it completely different and it ends up going and like registering it, your account to a restricted domain that you might not have access to. And now you have access to this company Slack that you shouldn't have access to. And I think that this, I don't think Slack was actually a specific, I think GitHub or something, what was it?

Justin Gardner (@rhynorater) (37:27.146)
Mm

Justin Gardner (@rhynorater) (37:46.497)
Dude, well, hold up, Joel, hold up, because, hold up, like, no, no, don't say anything, stop saying words, man. Like, I literally, while we're writing this, I like wrote down a couple targets that I know do this, and I'm like, because literally, I get back from Defcon, I was sick, then I like, and had some stuff going on for a week, and now this is my first week back from Defcon, and the first thing I do is go read all the research and like prep for this episode, and I'm finally done with that, and now I'm ready to do some freaking hacking, so don't say any of the targets that have this, because I'm gonna go try it right after this.

Joel Margolis (teknogeek) (37:48.354)
Cloudflare? I'm going.

Justin Gardner (@rhynorater) (38:16.673)
And yeah, we'll see. I'll keep you guys updated if I make any money off of it. if you know any, but you're right, man. Like any target that has this sort of implicit trust of email domain, we need to try this on, like 100%.

Joel Margolis (teknogeek) (38:18.648)
Yeah, but yeah

Justin Gardner (@rhynorater) (38:33.195)
Joel, Joel stop it, please, God. Okay, fine, whatever, I won't go after that one. All right, sweet, so let's see. Lots of other takeaways from this one, I'll take the next one here. So one of the ways that he, so encoded word isn't going to solve your problem. There has to be encoded word plus some other quirk, right?

Joel Margolis (teknogeek) (38:36.76)
You

Justin Gardner (@rhynorater) (39:01.055)
So there's a couple of ones that he mentioned here. And one of them we see is called UUCP. One of them is called source routing. One of them is called Unicode Overflow. So I'll start at the top. Unix to Unix copy UUCP is a super whack protocol. And this protocol essentially allows you to provide a, this is my TLDR of the situation, you know, having read over it. Essentially this allows you to provide address, an address for this SMTP interaction.

in this Unix to Unix copy address format, which has the domain before the actual user information, which is the inverse of email, right? The domain goes at the end for email. So essentially what he, the way he discovered this, and I love how humble he is about the way he describes the way that he found this, cause he's like, yeah, I just like jammed a bunch of characters into this thing. And all of a sudden I saw that I was getting like an invalid host resolution. and apparently he put a exclamation point character.

and backslash escaped the at symbol just by chance in this setup. And then he like had the good sense to like go and actually figure out why that was happening and then stumbled upon this vulnerability. But essentially this protocol allows you to specify a destination for this SMTP interaction that is not in the traditional format. And you can specify the domain first, then a exclamation point that is your delimiter for the domain and then a bunch of other information.

and then use a backslash to escape the at sign so that it isn't interpreted as an email address. So the front end application layer might think that this is an email address ending in, you know, at example .com, but actually this is a UUCP address that is, that's domain is actually what is before the exclamation point, which is just sick. Like absolutely sick.

Joel Margolis (teknogeek) (40:55.872)
It's just like what it's like such a weird weird thing and it makes me wonder now like Maybe I need to go pick the brain of like somebody who like used Linux in like 1985 and be like hey like Tell me about like all the annoying things you had to work around and tell me how that worked Yeah, yeah

Justin Gardner (@rhynorater) (41:07.346)
Exactly, man.

Justin Gardner (@rhynorater) (41:12.913)
Yeah. What is this UUCP thing? Like super, super cool.

Joel Margolis (teknogeek) (41:19.054)
Just need to ask them like do you know what you UCP is and if they know what it is then I'm just gonna be like okay I have like eight more questions for you

Justin Gardner (@rhynorater) (41:24.607)
Yeah, exactly. That's like the delimiter question. Like, okay, yes. Okay, sit down. You know, that's great. Yeah. So I want to say that this UUCP thing was enabled by default on like post. What was it? Postfix? that? Yeah, dude. It's like, it's pretty bad that this sort of stuff. It's like, what is going on here?

Joel Margolis (teknogeek) (41:43.99)
I think it was post -fix, yeah.

Joel Margolis (teknogeek) (41:48.278)
Yeah, does. Yeah, it was post -fix.

Justin Gardner (@rhynorater) (41:52.673)
that this is still a functional protocol. SMTP seems so disjointed from a lot of the stuff that, when I think about the application layer, I don't really include the SMTP server in that, right? It's different.

Joel Margolis (teknogeek) (42:08.576)
Yeah, I mean, it's one of those technologies that is extremely old. mean, like literally, like I remember like taking like a intro to networking class and like opening Netcat and you connect to an SMTP server and you go like EHLO and it just like replies back, you know, like it's completely text -based protocol. like very, very like simple. It's similar to HTTP in that sense, right? So it's, yeah.

Justin Gardner (@rhynorater) (42:12.779)
Yeah.

Justin Gardner (@rhynorater) (42:26.571)
Mmm.

Justin Gardner (@rhynorater) (42:32.085)
Yeah, 100%. And also, I need to correct something, The UUCP protocol was not on Postfix. Postfix has a different thing, which we'll talk about in a second, the source routes. But sendmail 8 .15 .2 is the vulnerable configuration that he was using when he discovered this. those are like the two main SMTP, you I think there's another one, but.

Joel Margolis (teknogeek) (42:52.534)
Okay interesting, he -

Joel Margolis (teknogeek) (42:56.906)
He actually does say that Postfix actually supports UUCP as well. So... Yeah, yeah. I think it was... No, no, it's actually pretty early on.

Justin Gardner (@rhynorater) (43:00.561)
yeah, he says it later, right? With the...

Justin Gardner (@rhynorater) (43:08.091)
I later found out that if you use the single parentheses trick, Postfix also supports UUCP. Okay. Yeah, you're right. send mail and mind blown, man. Send mail and Postfix both support this like ancient thing, which is pretty, pretty amazing. So that was, that was one of the main takeaways there is like there, there's a bunch of, so at a more concrete level, remember this whole UUCP trick.

Joel Margolis (teknogeek) (43:13.484)
Yeah, yeah, yeah.

Justin Gardner (@rhynorater) (43:36.981)
with the exclamation point in backslash escaping the at sign when you're dealing with anything SMTP related. And then taking this concept a little bit bigger, whenever you have sort of tangential systems to your application layer, right? Like maybe your application layer in this situation, he like shits on Ruby a lot. So maybe your application layer is in Ruby, right? And then you've got some other thing you're interacting with, like an SMTP server, like that sort of thing. You gotta think about what that second degree,

what other sort of weird configurations and what kind of other weird protocols and variants that other piece deals with. Because I think we've seen that pattern of dealing with these sort of out of band entities and them supporting some weird old shit that we can exploit.

Joel Margolis (teknogeek) (44:22.156)
Yeah, yeah, absolutely. mean, a lot of it is just like the backwards compatibility stuff and it's been baked in and never removed. And it's like, why does Cherry Pie support that encoding?

Justin Gardner (@rhynorater) (44:26.464)
Mm

Justin Gardner (@rhynorater) (44:34.571)
Who knows?

Joel Margolis (teknogeek) (44:35.701)
So like, yeah. Yeah.

Justin Gardner (@rhynorater) (44:37.227)
Yeah. Okay, next one is source routing. This one was another really cool old legacy variant and specifically within source routing, the percent hack. And essentially what this allows you to do is specify multiple domains that a specific entity should be sent, a specific SMTP interaction should communicate with in one address, which is like perfect for this sort of thing, right? Like it's just, it's perfect.

How it works is you can specify for example like ABC and then you do percent sign rhino radar comm at example comm right so the application levers layer is gonna see that as an at example comm Email address and give you access to like everything that's associated with that, you know in your organization or whatever But when that specific domain gets passed to an SMTP server that supports this sort of source routing percent hack scenario It will send the email first to

ABC at example comm then to ABC at renderator comm essentially cutting off the the at sign at the end and then replacing the the next percent sign with a an at sign and then sending the email there so the actual like validation email that might get sent out would go to like ABC at renderator comm which is perfect because then you can snag that code

Joel Margolis (teknogeek) (45:58.016)
Yeah, you know, very intuitive behavior that everybody would expect. Like, this is obviously how this is supposed to work. Yeah, like perfect user experience. Yeah.

Justin Gardner (@rhynorater) (46:01.032)
Yeah, exactly.

Justin Gardner (@rhynorater) (46:07.499)
Why does this exist? Why does this exist, man? It's crazy. I thought that was amazing though. I thought that was like literally like a dream come true scenario for hackers, right? It's like.

Joel Margolis (teknogeek) (46:17.45)
think one interesting thing about that for me is like, I'd love to know if anybody's actually legitimately using this. Like, and if so, who and why? I'm like, I'm really curious like how much of this is just like legacy garbage that's just been like, you know, kept in because of the spec or, you know, who knows what. Or if it's just like, you know, there's...

Justin Gardner (@rhynorater) (46:24.715)
Probably not.

It's time to retire, if that's the case.

Joel Margolis (teknogeek) (46:41.29)
secretly like the entire banking system is using source routing or something like for all we know. Yeah.

Justin Gardner (@rhynorater) (46:43.723)
Yeah, well that could be, I could see that actually. Yeah, for sure. Okay, so we've got, so so far we've got UUCP, we've got encoded word UUCP and source routing. Another takeaway from this one was this whole concept of Unicode overflows, which I think was this one and Punicode, which is something we're gonna talk about in a second. I think, I felt like we're like little bonuses to the actual email research. Like a lot of the stuff here that we just talked about was like.

And I found this in the RFC and this is cool. And then the Unicode stuff and the Punicode stuff are both like, wow, this is some cool like ASCII management or like Unicode to ASCII conversions and normalization that are occurring. And then we can weaponize this in the email arena, but we can also weaponize this in other areas. So listen here, listen closely here for these next couple of ones if you are not, even if you're not necessarily interested in email.

Joel Margolis (teknogeek) (47:40.354)
Yeah. Well, and also like real quick, I love that something finally is able to exploit puny code because puny code is one of those things that where, you know, you're like testing like domain registrars, you know, all these different things. Usually it's like for SRF and you you input a domain and you're trying character like bypasses and you try like, you know, some weird care set and it turns it into a puny code domain. You're like, that's so annoying. Well, there might actually be something there now.

Justin Gardner (@rhynorater) (47:41.055)
related hacking because...

Justin Gardner (@rhynorater) (47:48.533)
Yes dude. Super whack.

Justin Gardner (@rhynorater) (48:05.483)
Mmm.

Yeah, absolutely. And I'm looking, I didn't actually put a bunch of, I didn't write a bunch of notes on the whole puny code part. So maybe you can explain that one or I can take a stab at it real quick. But the Unicode piece was really interesting because the Unicode overflows, this goes into how Unicode is being processed and how, I guess even specific ASCII related stuff is being processed.

Because essentially what it would do, and he just noticed this while he was, I think, black box testing, and he was getting weird characters out of putting ASCII or Unicode into a place where the application expected ASCII. Essentially what was happening here was they were taking the Unicode code point and they were just running like mod 256 on this. So any Unicode thing that would mod to 40, right, if you...

took the code point and then ran mod on it to give you the output there. Anything that would convert to 40, including like this weird L and a bunch of other characters, would get normalized to an at sign. And then he was able to use that to like mess with some of the parsers for emails and kind of split the, the whole research is called splitting the email atom. And so the whole concept is like splitting the email domain where it shouldn't be or getting things.

delivered to invalid locations because of that. And he was able to use these Unicode overflows to do that. And I think we'll see this, 100 % we'll see this in other areas. I want to even say, file descriptor has an XSS out there somewhere on Twitter that utilized something similar to this. And it's not, just wasn't something that was on my radar actively until Gareth brought this back to my attention.

Joel Margolis (teknogeek) (49:52.662)
Yeah, absolutely. Yeah. So, so for the, for the puny code thing, very similar type of, you know, issue basically where like you could force it to decode improperly. So I guess this was a bug in the PHP IDN library, that would be used to decode puny code itself. And you could basically just generate like arbitrary characters in puny code encoding by like writing them in this certain specific format and

Justin Gardner (@rhynorater) (49:59.509)
Mm

Justin Gardner (@rhynorater) (50:04.15)
Mm.

Justin Gardner (@rhynorater) (50:07.751)
Mmm. That's right.

Joel Margolis (teknogeek) (50:22.221)
know, putting, I think it was just like two zeros before.

Justin Gardner (@rhynorater) (50:25.85)
He says, I discovered that if you use two zeros at the start, you could generate unintended characters with punicode.

Joel Margolis (teknogeek) (50:33.42)
Yeah, yeah, so with this, now you can have like valid domain, quote unquote, right? But when it gets decoded with like puny code is an encoding. It's not like a domain. I mean, it is for domains, but right, like decoding it and having it be a valid domain are two separate things, right? And so basically he was able to generate puny code valid strings that decode to invalid domains with like commas and ats.

Justin Gardner (@rhynorater) (50:51.637)
Right, right.

Joel Margolis (teknogeek) (51:00.384)
symbols and stuff like that that should never be in a puny code domain. And then when that would be used within like the SMTP protocol or like, you know, the body of an HTML.

Justin Gardner (@rhynorater) (51:06.539)
Mm

Well, in this specific scenario, in the actual application layer, when it was displaying that email on the screen for the user to see.

Joel Margolis (teknogeek) (51:13.57)
Yeah.

Joel Margolis (teknogeek) (51:16.95)
Right, exactly. Yeah. So Joomla, I believe this was, was, was the big example. So Joomla, Joomla was, you know, doing puny code decoding on your email address so that it would render properly in the web.

Justin Gardner (@rhynorater) (51:19.254)
Yeah.

Justin Gardner (@rhynorater) (51:25.92)
Mm

Joel Margolis (teknogeek) (51:30.774)
you know, again, you can inject non -valid, like invalid email characters into a valid puny code string. And so when this gets decoded, now you have essentially, X is that, well, HTML injection, which can lead to XSS. So, I think he was able to escalate it all the way to RCE, apparently. I think through template, injection or no. Okay.

Justin Gardner (@rhynorater) (51:36.865)
Mm.

Justin Gardner (@rhynorater) (51:48.713)
Yeah, no, it was through CSS injection. And it's an interesting escalation to RC because despite all of his attempts here to generate a valid XSS payload with Punicode, it seems like it is not possible to get that to actually do it. Him and Gareth and also some of the other members of the.

of course, we were a research team spent a decent amount of time, it says, kind of poking around with this and couldn't get it to generate like a valid XSS payload. But what he was able to get it to do was generate an opening style tag. And then he was able to utilize the way that that style tag was placed adjacent to another injection point that he had to essentially be able to trigger a arbitrary CSS import.

Which is... Yeah.

Joel Margolis (teknogeek) (52:43.884)
Yeah, it's like an XSS with two injection points, right? You have like one variable and you start your payload so it doesn't get fully filtered out. And the second variable, you finish your payload and there's a bunch of junk in the middle, but it doesn't really matter because you have the start and the end that builds out a full payload that actually succeeds.

Justin Gardner (@rhynorater) (52:49.953)
Mm

Justin Gardner (@rhynorater) (53:00.211)
Exactly, yeah, and so then, know, of course I love this with Gareth, right? Like he's like, all right, well we're in a CSS injection context, no problem. And then he like whips out his, hold on, like where is this thing? He's like, so we need it now to exfiltrate the CSRF token via CSS. Thankfully, there have been many good posts on this and the best way to do it is import chaining, as mentioned by Donut in Pepe Vila. And I decided to use.

Joel Margolis (teknogeek) (53:07.8)
He's like, he whips out a - yeah, the fu -

Joel Margolis (teknogeek) (53:26.282)
and Pepe Vila. Yeah.

Justin Gardner (@rhynorater) (53:29.287)
I, however, decided to use the custom or customize the tool that I already developed for my blind CSS injection research, which is just super badass. And that tool is amazing, by the way.

Joel Margolis (teknogeek) (53:35.709)
Yeah.

And it's so funny because as I was reading this I was I was thinking before I got to the end of that like paragraph I was like huh I wonder if he if that was like a sneak peek like yeah, it's here's the CSS injection tool and then behind the scenes He was like doing all this stuff, and he wrote that tool just for this and no it was is actually completely separate research That just like ended up being perfect

Justin Gardner (@rhynorater) (53:51.497)
Yeah.

Justin Gardner (@rhynorater) (53:57.883)
Yeah, he's like and I'll release that and then a couple months later I'll drop that no, so that's great and and I love seeing CSS inject injection x will explode in a really effective way here I think in the end that this would still because it's just a a CSRF token exfiltration I think that it still requires user interaction to pop this RC But you know

such as LifeMan, I think it's still in RCE, it's a user interaction. Yeah.

Joel Margolis (teknogeek) (54:26.038)
What a one click RC is better than zero click I mean wait one click RC is better than No RC. That's that's okay Wow

Justin Gardner (@rhynorater) (54:34.055)
No RCE. And I think also that there is potential to get this to zero click. We just need to sort of crack this puny code nut where essentially it just will not generate the correct characters for any sort of XSS -related payload. And he's able to get it to generate SVG, like an SVG tag. I just don't think that it can get to the point where it actually

like has the correct event handler and doesn't have junk text in between. So.

Joel Margolis (teknogeek) (55:09.004)
Yeah. And I mean, I don't know how thorough, like how deep Gareth went with this. And if he just stopped at the PHP IDN library. But one thing I remember vividly from one of Orange's earlier talks around URL parsing was the wide expanse of libraries that were tested and how many were vulnerable across like different languages, different implementations, like different commonly used, like, you know, here's this Python library, here's this Ruby library, here's this...

Justin Gardner (@rhynorater) (55:15.292)
Mm. Mm.

Justin Gardner (@rhynorater) (55:23.573)
Mm.

Justin Gardner (@rhynorater) (55:33.024)
Mm -hmm.

Joel Margolis (teknogeek) (55:38.552)
PHP library, and so I would be really curious to see how many other things are affected. Like like CherryPie, like, you know, large HTTP framework library that is used by a lot of things, like just randomly supports this word, encoded word stuff. you know, same thing, like how is the puny code library for Python doing puny code? Is it also vulnerable in the same way? And how many other, you know, web services and stuff could you affect in that way?

Justin Gardner (@rhynorater) (55:55.115)
Mm.

Justin Gardner (@rhynorater) (56:06.517)
Yeah, absolutely. And I think it's kind of odd here because the the Joomla code that's used specifically says puny code helper, colon, colon, email to UTF eight. Right. So it's like explicitly doing puny code. D D encoding, I guess. Decoding, D encoding. just said decoding. I promise English is my first language, guys. my God. Please stop. No.

Joel Margolis (teknogeek) (56:18.872)
Right

Joel Margolis (teknogeek) (56:24.641)
Right.

D -encoding.

Joel Margolis (teknogeek) (56:32.232)
Unde -encoding.

Justin Gardner (@rhynorater) (56:36.285)
And so I think this is something that will probably not happen super implicitly. I think it's probably something that'll be more implicit throughout applications, but that doesn't mean that we can't still poke for it, especially when you have source code. It might be helpful even to start looking for like, where's puny code mentioned in here? And how does that, why does it work that way? So very cool stuff there.

Joel Margolis (teknogeek) (56:59.885)
Yeah.

Super cool.

Justin Gardner (@rhynorater) (57:02.911)
I don't think there's anything else on that one, so we'll call that a wrap on splitting the email at them. Next up is, cash them all. And two.

Joel Margolis (teknogeek) (57:14.4)
Wow, might have a new tagline. We might have to just throw that in there.

Justin Gardner (@rhynorater) (57:17.749)
Dude, it's catchy, man. It's good. I really liked it. I think you said your favorite research was the email research, right?

Joel Margolis (teknogeek) (57:28.606)
Yeah, yeah, that was super cool. And I think this one is your favorite, right? Yeah.

Justin Gardner (@rhynorater) (57:31.081)
It is, it is. I think this is the one for me. And the reason for this is like, wow. One, I wasn't as familiar with, I believe it's Martin, right? It's probably, it's pronounced differently, but Martin's research. In the past, I haven't seen much from him and I was really impressed with this research. there, I think that a lot of the techniques that he talked about here were pretty known to the people that,

really know a lot about web cache deception and web cache poisoning.

Joel Margolis (teknogeek) (58:05.868)
Which, by the way, me, timing attacks and web cache poisoning are the two, they're quantum computing to me, bro. I understand how it works and stuff, but it's such a weird, deep attack scenario that you really have to be at a certain level with the application to be like, yeah, there's some, like.

Justin Gardner (@rhynorater) (58:16.853)
Really? What? Dude, we gotta talk about that.

Mmm.

Joel Margolis (teknogeek) (58:34.274)
There's some like caching stuff and then like understanding how you can actually exploit them in an effective way I was was a boy I was a boy to get but yes

Justin Gardner (@rhynorater) (58:37.801)
a different level of intimacy with the application, is that what you would say? Okay, yeah, yeah, come on, just lean into it, man. It's a catchphrase, man. Yeah, so I think this research was really good because what he did is he did the legwork. He did the legwork for everything. He went and he like, this whole research is filled with these graphics that show how different systems, Cloudflare, CloudFront, Azure,

GCP, Apache, Nginx, how all of these things process various permutations on dot segments, right? So path traversals, origin and cache level delimiters, like it's awesome. And he doesn't just say like, hey, sometimes this works. He says, this works on this, this, this, this, this, and this. And there's incompatibilities between this and this and this and this and this and this, right? And I just, I love that. Like that is also, I just want to shout out how much of a grind that is.

Because you know what he had to do was like go and spin up each one of these services and like configure that shit and that is boring stuff, right? That is not exciting security research, but he put in the work.

Joel Margolis (teknogeek) (59:37.94)
Yeah.

Joel Margolis (teknogeek) (59:46.776)
Especially when you spend like three hours spinning one of them up testing it and then it doesn't work and you just go No on your spreadsheet and you then you go to the next one Like it's not even exciting you're like, now I have 50 things I can exploit it's like okay on to the next one

Justin Gardner (@rhynorater) (59:55.121)
Exactly, exactly, man.

Justin Gardner (@rhynorater) (01:00:02.431)
Yeah, exactly. So major props to him for putting in the legwork and getting this research in place. And there were some pieces that I was not familiar with just in my own methodology in here. So props to him for that. And let's kind of talk a little bit about those, okay? So this whole thing is about practical applications of these research, right? So with...

the research from James, we're paying attention to these attacks, we're understanding how to use that domain label length with Gareth stuff, we're checking for encoded word, we're checking for some of these weird fringe things. Once we have identified that we have something funky here, or maybe if we're interested in email level authentication, and with this cache related stuff, man, there's a lot of takeaways, dude.

Fingerprinting your target will now give you so much of an insight into what is actually happening because of the research that he's done here. So first I need to make, I'm sorry, I'm gonna rant a little bit here, Joel, can I cook for a second? Dude, the apron's coming on, I got the chef hat, I'm ready to go. So in the talk he defines multiple.

Joel Margolis (teknogeek) (01:01:11.717)
Go for it. Let him cook. Let him cook.

Justin Gardner (@rhynorater) (01:01:22.911)
different types of delimiters. He talks about origin delimiters, cache delimiters, front -end delimiters, and back -end delimiters, depending on the context, right? And there's two different types of caching -related bugs. There's web cache deception, which requires user interaction, and there's web cache poisoning, which doesn't require user interaction, okay? And I think the visuals that he had in the presentation and also in the write -up on the Portsburger website, really, really good for understanding. So Joel, if you are trying to get a better understanding of those,

you know, giving that thing a twice, twice, three times over might really clear that up for you. And so the concept of origin delimiters is like the backend server, right? Like we're thinking about, we've got a caching server in front and then we've got the origin server. Those are delimiters in the path that happen only on the origin server and not on the cache server, right? So he gave a bunch of examples of these. The one that's really common that we, that a lot of us know about from the research that Orange did a couple of years back is the semicolon.

That being used in spring environments and Tomcat, think, as well, to delimit a path parameter or a matrix parameter. So the difference between that being parsed as part of the path and it not being parsed as part of the path can mean the difference between what extension appears to be in place at the caching level and at the origin level. So let's say if the caching server is caching everything that ends in .js or .css.

then we can say slash my account slash user info, semicolon, dot CSS, right? And the backend when it's processing that won't even pay attention to that, that semicolon part, because it's a parameter. It's like if you had put a question mark, right? But the front end, the caching server, sees that as a path ending in dot JS or dot CSS or whatever and says, ooh, I need to cache that. And he lists in the research,

Joel Margolis (teknogeek) (01:03:07.362)
So weird.

Justin Gardner (@rhynorater) (01:03:21.107)
like Cloudflare has, I believe it's Cloudflare. Yeah, Cloudflare has like this list of extensions. .7z, .avi, .avif, .apk, Cloudflare by default caches all of these if it ends in this. So, hmm.

Joel Margolis (teknogeek) (01:03:38.134)
Right, which I guess makes sense, right? Like from a CDN perspective, like most of those, at least the ones that you listed are like, know, media files, like you have to remember like CDNs, their business is bandwidth traffic, right? And so like the more traffic, the more that costs them. And the more they have to serve that file over and over again, the more it costs them. So their whole goal is gonna be to serve that file as few times as possible. And if it ever gets updated, updated as few times as possible.

Justin Gardner (@rhynorater) (01:04:07.637)
Yeah, absolutely. They're definitely incentivized to go down that path. So it does make sense, but it definitely causes some bones. So understanding what origin delimiters you have in your environment is pivotal to being able to trigger web cache deception and web cache poisoning, but he calls it the delimiter something different in that way. So let me just run through these really quick. So you got the semicolon for spring and like Tomcat, you've got the dot for rails, which is really interesting because I didn't know this about.

I didn't know this sort of tangentially about Rails, but I hadn't really concreted it in my brain. If you're dealing with a Rails environment, you can add dot and then whatever at the end and it'll still show you the same result in a lot of ways. And I didn't really know why that was. And the reason it is is it's looking for, it's taking that dot and it's trying to decide what view to display in Rails. And if that dot doesn't have, if that extension that you provided after the dot doesn't have a registered view,

then it will just show you the default view, which is such an amazing functionality, right? And so there are differences. This is one of the things that was in the presentation that wasn't in the actual research paper that he showed, was that if you, there are a select set of Cloudflare caching extensions that are not also valid Rails extensions. Rails won't even,

Joel Margolis (teknogeek) (01:05:35.384)
Mmm.

Justin Gardner (@rhynorater) (01:05:37.149)
like if you try to give it, I forget one of the, I think I screenshotted it or took a picture of it on my phone or whatever, but like maybe it's like .tiff or something like that, right? Rails will never serve, you know, that, but Cloudflare will always cache that, right? If that's at the extension. So Rails is gonna fall back to the default view and show you whatever it normally does while still having that .tiff extension, and then Cloudflare will be like, that's a .tiff file, so I gotta cache that.

Joel Margolis (teknogeek) (01:05:45.857)
Okay.

Joel Margolis (teknogeek) (01:06:06.16)
That's so weird. I mean, again, it's like these weird intricacies of frameworks, right? It's not even just like necessarily singular application vulnerabilities. It's like common configurations of things that are tied together like Apache and some web server behind it. like, you know, just like, right. Right, exactly. Yeah. So it's so interesting.

Justin Gardner (@rhynorater) (01:06:06.162)
It's rad, dude. It's freaking rad.

Justin Gardner (@rhynorater) (01:06:13.472)
Yeah.

Justin Gardner (@rhynorater) (01:06:24.597)
Mm

Justin Gardner (@rhynorater) (01:06:28.198)
Mm -hmm. Yeah. Reverse proxies, caching proxies, all these.

Okay, a couple more he listed. Null byte for open light speed, which I haven't seen quite as much of open light speed, but I mean, if he put it on there, it's probably decently sized. And then percent OA for Nginx rewrite rules, which has got to break a ton of things, man. Like if percent OA is truncating the path on Nginx rewrite rules, like that has got to break a ton of things. So.

Joel Margolis (teknogeek) (01:06:54.539)
you

Justin Gardner (@rhynorater) (01:06:59.937)
That wasn't one that I was super using often in my web cache deception checks. So that is definitely one that I'll be adding. And to be honest, what he recommended in that scenario is just like hit everything in ASCII range. Just give it a shot, you know? Try it encoded, try it not encoded, try it double encoded and see if there's any differential between the two. So I might just make a massive list of every ASCII character, every encoded ASCII character.

and then every double encoded ASCII character. just whenever I have a suspicion about any of this, just like throw that whole list at it and look for the outliers.

Joel Margolis (teknogeek) (01:07:33.664)
Yeah, I mean, that's what I was going to suggest, I mean, like you said, like none of this is like necessarily new research. But I think it's like a great sort of like reference to have like, you know, if you come across, you know, some some cash poisoning, then take a look at this article and go through and be like, OK, let me try this, this, this until something works. Or you can just do what you said and have that list of all the characters, fuzz everything.

Justin Gardner (@rhynorater) (01:07:38.687)
Mm

Joel Margolis (teknogeek) (01:07:59.252)
And something will work, probably. I mean, if you have a... if there's an issue there.

Justin Gardner (@rhynorater) (01:08:00.465)
Mm, yeah. Yeah, and I think there are some new pieces of research in here, just to be clear. But like this specific set, think, has been, I'm not sure if it's known publicly, but at least known within the like, people that actively exploit web cache deception and other stuff, environment, being like trading this tip with your friend or whatever, that sort of thing. But it's just nice to see it formalized, you know? And then, so.

Joel Margolis (teknogeek) (01:08:07.395)
okay. okay.

Justin Gardner (@rhynorater) (01:08:28.307)
As far as cache delimiters go, so something that the caching server thinks is a delimiter that isn't a delimiter on the backend server, there's not a lot of them besides maybe question marks sometimes if it's encoded and passing along the unencoded value. The unencoded, why do I do this? The decoded value.

Joel Margolis (teknogeek) (01:08:46.04)
D on on on D encoded

Justin Gardner (@rhynorater) (01:08:50.717)
Just, why do I do it? I don't know. But he had a great point switching from web cache deception to web cache poisoning, which is a non -user -interaction -required attack where you poison cached assets for other users, that the hashtag can really cause some havoc. And essentially the cache server seeing that as a delimiter in the backend not or vice versa, right? And since we're not having, we're not,

Joel Margolis (teknogeek) (01:08:53.766)
You

Justin Gardner (@rhynorater) (01:09:20.115)
limited by the browser, having to force the user to make a request in their browser, we can send the hashtag in an HTTP request to the server, just directly with like burp or kaito, and then see how that actually gets processed and whether the caching server discards it at the caching server level or whether it gets considered to be a part of the path at the origin level.

And there's, there's a, once again, another great graph in here that is just like blowing my mind of the, the amount of variance between whether hashtag is a delimiter in these various entities. Right? So cloud flare, no cloud front. Yes. GCP error. Azure. Yes. Imperva. No Apache error. Nginx. Yes. Open light. No. Like a spring error rails. Yes. Django. No. Like it's like everything parses this differently.

Joel Margolis (teknogeek) (01:10:12.802)
Right.

Man You know what would be awesome like follow -up research from him was would be like some sort of just like simple Docker like a repo with like 15 Docker containers in it that you just like Spin them all up and then you can just like send one request to all of them

Justin Gardner (@rhynorater) (01:10:16.002)
And so, yeah.

Justin Gardner (@rhynorater) (01:10:24.8)
my god, yeah.

Well, hold up. That is called HTTP garden. And actually I met the guy who wrote this at the Google Live hacking event in Vegas and I didn't know who he was. And I was hanging out with him and he was like, hmm.

Joel Margolis (teknogeek) (01:10:33.772)
Whoa. New tool alert.

Joel Margolis (teknogeek) (01:10:42.732)
Dude, this is sick. What the? They should be good. is a collection of ACP servers and proxies configured to be composable along with scripts to interact with them in ways that makes finding vulnerabilities much, much easier. That's awesome.

Justin Gardner (@rhynorater) (01:10:54.027)
Dude, it's awesome, man. And I was talking to him about it, and I was like, have you heard of H2B Garden or whatever? And he's like, yeah, I wrote that. And I'm like, what? Are you kidding me? You're the guy that wrote H2B Garden? Thanks, dude.

Joel Margolis (teknogeek) (01:11:01.56)
Dude, this is crazy. Like literally, literally like you just like give it a payload and like, like with the slash R slash N's and it just prints out a freaking table.

Justin Gardner (@rhynorater) (01:11:15.573)
Yeah, also dude, this guy that wrote this, Ben is his name. Let me see if I can find, yeah, Ben Callis. It was so funny, because we were like hacking together at the live hacking event or whatever and just like sharing interesting leads and stuff. And I was like, yeah, so just open up like Kydo or open up Burp or whatever. And he's like, what? And I'm like, open up those. And he's like, I don't use that. I just, I use printf and netcat. And I'm like, what? And it's so funny too, dude. If you look, if you look in the like, read me for, for HTTP garden.

Joel Margolis (teknogeek) (01:11:33.9)
Ha ha.

Joel Margolis (teknogeek) (01:11:37.752)
what? my god, what a legend.

Justin Gardner (@rhynorater) (01:11:45.523)
you see a bunch of stuff like that. Like he's clearly very used to just writing HTTP requests by hand on the command line, like a total badass. and, and so.

Joel Margolis (teknogeek) (01:11:57.606)
This is like payload. It's like get such a TV's led one by one backslash our backslash and host colon a backslash are like one -liner HTTP

Justin Gardner (@rhynorater) (01:12:02.003)
Yeah, it's like...

This is amazing. So anyway, shout out to Ben, dude you rock. Thank you for developing this tool that allows us to test a bunch of HUP servers and proxies really, really easily. Also, maybe learn how to use burp. But also, let me just say, he also told me, he's like, yeah, mean, cause like this is the ultimate level of control, right? There's no weird like parsing error and burp or like.

Joel Margolis (teknogeek) (01:12:22.24)
I

Justin Gardner (@rhynorater) (01:12:34.473)
Not that, everything you write is like you intentionally wrote that character, which is great and I can see how that would be really helpful in like HTTP request smuggling, which I think he does a lot of and other stuff like that. But man, for like testing IDORs, that sucks, you know? So yeah, very cool research there and yeah, that does exist Joel already. HTTP garden.

Joel Margolis (teknogeek) (01:12:58.404)
I will say Probably the most interesting thing I mean other than everything else But was it really caught my eye in this HTTP garden read me is under the acknowledgement section It says this material this material is based upon work supported by the the defense advanced research projects and agency DARPA under contract number HR zero zero one one nineteen C. seven six

Justin Gardner (@rhynorater) (01:13:19.221)
my gosh.

Wow, dang dude. All right, thanks DARPA. Just hooked us up with like...

Joel Margolis (teknogeek) (01:13:25.393)
Thanks DARPA. DARPA is doing HTTP server research. That's totally...

Justin Gardner (@rhynorater) (01:13:29.089)
appreciate that. Dude, apparently there's like a major DARPA AI competition or something like that. AI Cyber Challenge that's like something like $4 million is the number one prize or something like that. Yeah, it's kind of nuts. I think this is more like a, exactly. I think it's more of like a, I want to say it's more of a like company oriented thing, but still pretty baller. $4 million, that's like massive.

Joel Margolis (teknogeek) (01:13:39.896)
Dude, what? Alright, live hacking events, listen up.

Joel Margolis (teknogeek) (01:13:56.46)
That is crazy.

Justin Gardner (@rhynorater) (01:14:00.223)
All right, cool man. So let's see, where are we at? Okay, we went down a little tangent there, but yeah, hashtag really cool delimiter to use, send it not through the browser, but through your HTTP proxy and try to get something to cache incorrectly because the backend truncates the path and the front end doesn't or vice versa, right? And we see scenarios in this is hashtag a delimiter table that he created.

Joel Margolis (teknogeek) (01:14:05.654)
Sorry, yeah, DARPA contracts.

Justin Gardner (@rhynorater) (01:14:28.723)
in his write up where some origin servers will see it as a delimiter, some won't, some caching servers will see it as a delimiter, some won't. So it's like, if you can get a mismatch between the two and there are a lot of mismatches, then if anything is being cached, then you should be able to do some cool cache poisoning with this. And cache poisoning is like massively impactful. I think like.

I won't name any names or call out any scenarios, but I was at a live hacking event. was talking with a hacker that wasn't on my radar and recently came on my radar and we were talking about the Volns that he found and he's like, yeah, I found this cache poisoning that did this like essentially one of the most internet breaking things I can possibly think of. And, and he got some crazy bounties for it and all because of understanding thoroughly how to manipulate these caches. So.

Specifically cache poisoning. I find web cache deception pretty often. I've not found web cache poisoning as much. And I think that's, I'm sleeping on that personally.

Joel Margolis (teknogeek) (01:15:29.238)
Yeah, so it's I mean, this is awesome research. So definitely going to put this one in my in my bookmarks for whenever I come across some some cash poisoning and sort of like a cheat sheet.

Justin Gardner (@rhynorater) (01:15:34.496)
Yeah.

Yeah, any caching functionality really. And I'll talk about that as well here. There are some caching strategies. I love that he made this more concrete because so much of this for me was intuition and like, well, if the caching's happening here, then this, that, and the other thing. But he built out three categories of web cache poisoning strategies, okay? Caching by extension, if the server is caching by extension, right? We listed a bunch of extensions earlier that are helpful for this.

if they're caching by static paths. So essentially everything under slash static gets, gets cached. That's a common configuration, right? Or if they're caching default files. So essentially that is Cloudflare by default will cache robots .txt or like sitemap .xml or whatever, right? And so you can potentially use those default caching rules along with a path traversal or like a path truncation with the hashtag.

to make the caching server think that this is robot .txt that I'm caching, but it's actually slash user account slash information or whatever, or like even some invalid thing will dos these endpoints. So very cool stuff there. Okay, Joel, I'm sorry. One last thing on this, okay? Just give me just one more second, okay, I promise. Dot segment normalization, so path traversals. The graph that I'm looking at right now needs to be seen.

Joel Margolis (teknogeek) (01:16:49.122)
Mm.

Super cool.

Justin Gardner (@rhynorater) (01:17:04.991)
by the world, right? I put it in the doc, Joel. This is crazy, right? There is so much variance in how a path is normalized between various different web servers. This is really eye -opening to me. Like, I knew that this was happening at some level, but I hadn't seen how much variance there was. There's a ton of variance.

Joel Margolis (teknogeek) (01:17:21.592)
Yeah, I mean, it's like literally what 50 50 like.

Justin Gardner (@rhynorater) (01:17:24.097)
Yeah, it's 50 -50. It's like Cloudflare does not normalize it before it passes it through. Neither does Apache. Nginx does, IIS does, Cloudfront does, GCP doesn't, Puma doesn't, Fastly doesn't, Imperva does. It's like there's so much variance there that should be exploitable. I'm surprised I don't find more dot normalization bugs.

Joel Margolis (teknogeek) (01:17:45.1)
Yeah, I think like the

Yeah, I mean, I will say like, think like one of the biggest interesting things here for me is like just Apache versus Nginx, right? Probably, I don't know the numbers, but I would guess those are like the two most widely used web servers or like, you know, at least they're somewhere in the stack and they do it differently. They don't even, they're not even consistent with each other, right? Like of all the things that I would expect to be consistent with each other, would expect Apache and Nginx to be essentially mirrors of each other and yet there are not. And it's, I mean, it's crazy.

Justin Gardner (@rhynorater) (01:18:02.827)
For sure, for sure. Yeah.

Justin Gardner (@rhynorater) (01:18:16.353)
Yeah, and I think that might have to do with, I'm looking at this configuration here, how Nginx deals with percent 2f and stuff like that, which is an issue that I exploited really recently. essentially, Apache requires a specific configuration to decode percent 2f as a path segment, if it's given percent 2f rather than a slash, and Nginx does not. So that's exploitable in a lot of contexts. So definitely.

Keep an eye out for that. Anytime you're thinking about how you may have a specific limited path and may be able to add things onto the end and then traverse back because %2f is perceived by those servers as a path delimiter. All right, dude. That's all I had on gotta cache them all. You got anything else here? Are we moving along?

Joel Margolis (teknogeek) (01:19:02.006)
Yeah, so interesting.

Joel Margolis (teknogeek) (01:19:08.769)
You

Joel Margolis (teknogeek) (01:19:12.312)
No, no nothing else. do have a little I do have a little extra thing at the end so well Yeah, are we there are we there? Are we at the end? Should I bring it up?

Justin Gardner (@rhynorater) (01:19:22.335)
Yeah, I mean, I think I am gonna chat very briefly about Orange's write -up. Do you want me to do that now or later? Okay. Okay, so last piece of research from Defcon slash Blackhat that we're gonna have, we're gonna cover this week. So essentially this is my hit list, Portswigger research and Orange. Was this research by Orange who just decided I'm gonna, I love the quote, let's see if I can find this quote in here. He's like.

Joel Margolis (teknogeek) (01:19:27.744)
Okay. Yeah, go ahead. Go ahead. Yeah.

Justin Gardner (@rhynorater) (01:19:49.205)
He's like, yeah, you know, when I do research, I like to break a bunch of stuff, the most hardened stuff, the stuff that people use everywhere. And so I said, why don't I attack Apache? And I was like, no, he says, as you know, I always aim to challenge big targets that can impact the entire internet. Which I think, yeah, he hit the nail on the head here, right? And.

Joel Margolis (teknogeek) (01:20:10.86)
No big deal.

Justin Gardner (@rhynorater) (01:20:14.983)
I think he does a little humble flex later on in this post where he says, I am now the number one holder of Apache CVs, which is super badass in my opinion. And the research was really cool and stuff that I really couldn't believe. there was essentially, he figured out a way with a very, very common Apache configuration to just get arbitrary file system read and just dump it. And the payload is so simple too. It's like percent three F.

will truncate the path in a specific scenario and then the path will get misinterpreted from a URL to a file system file. And then like you could just like you can literally just write site .com slash, you know, var slash www slash and then just give it a full path and it'll just or slash Etsy slash password. It'll just dump it. It's like what is going on here? And so I was like when I saw this, I didn't really believe it to be honest. Like I was like, I don't think this works this way. So I like fired up my Apache server or whatever.

Joel Margolis (teknogeek) (01:21:06.168)
That's crazy.

Justin Gardner (@rhynorater) (01:21:14.823)
And of course it didn't work that way. And I, and I, and I was like, why is it working this way? And I was like, I'm patched. Cause they've already pushed it. And like, I've updated my Apache server. but yeah, like he was spitting out this error saying like, you need to have a flag, you know, unsafe percent three F or unsafe three F or whatever. And I Google that and freaking oranges open internet bug bounty, a hacker one disclosed report pops up saying like, so they fixed it with, you know,

Joel Margolis (teknogeek) (01:21:41.64)
Hahaha

Justin Gardner (@rhynorater) (01:21:43.169)
unsafe3f flag and it's just anyway really amazing research we've seen in the web cache research that nginx rewrite rules have problems with percent OA we've seen here that Apache has confusions on their rewrite rules with percent3f the question mark I bet that there are a lot of weird inconsistencies with rewrite rules in general on on web servers

That's a great area for research for anybody who's looking to do HTTP stack research.

Joel Margolis (teknogeek) (01:22:19.992)
Dude, I mean, per usual. I think, I don't know if I've told this story before. I don't know if I can tell this story. Maybe I can tell this story.

One time.

Justin Gardner (@rhynorater) (01:22:32.289)
my god, Joel, come on, don't bait us like that. You gotta give us something, no?

Joel Margolis (teknogeek) (01:22:33.581)
I'm not gonna tell that story. Let me just say this. me just say this. If you ever receive a report from Orange, it's gonna be a bad day. Yeah.

Justin Gardner (@rhynorater) (01:22:42.347)
Yeah, yeah, I believe that. I absolutely believe that. So yeah, there's lots of cool stuff in here. To be honest, I'm gonna be honest with you guys, I read this through once and I picked up some things, but I really honestly need to read it again before I feel super comfortable talking about the technical details of it. The problem with this research right now is it's a great read and you can see the methodology and like...

how he found all these things and like the impact and be like, whoa, that's nuts. The problem with it is they're patched. Thank God, right? else the internet would be breaking, right? And so, you know, a lot of the configurations that you're gonna see are gonna be patched against this. But if you can find Apache servers that haven't been updated, I think it would be very valuable for any of the recon boys out there. I know you recon boys are listening.

to essentially take a look at how this research works and try to create some flags, some fingerprinting for a server that may be vulnerable to this and then try to pop it because the configurations that are required for these exploits to work are extremely common and are even in the Apache documentation. And so I think it should be very doable to find vulnerable servers that have vulnerable configurations with this at scale.

if you get creative with your nuclei templates or your scanning environment, like your scanning modules.

Joel Margolis (teknogeek) (01:24:16.546)
Yeah, yeah, totally. Totally.

Justin Gardner (@rhynorater) (01:24:18.133)
Alright man, that's all I had on this one. I'm not gonna go too deeper, much deeper into this one, but yeah. Present 3F does weird things on Apache Rewrite, for sure.

Joel Margolis (teknogeek) (01:24:26.46)
Okay, so a little bit of extra interesting thing I got nerd sniped by you and this research I was looking at the Python Codex library the built -in library. It's called codex and sure enough they have IDNA parsing they have puny code parsing and This is not the the chosen type thing

Justin Gardner (@rhynorater) (01:24:36.036)
Lovely.

Justin Gardner (@rhynorater) (01:24:46.197)
Wait, you got, wait, hold on. This is not the trusted types thing? Joel, what is wrong with you, man? Okay, all right, fine, go. Tell me about it, tell me about it.

Joel Margolis (teknogeek) (01:24:55.024)
yes but i was looking at the source code and i think it's probably probably sneak some things through there that's what i'm thinking i think yeah it seems to be pretty i mean it's just like it yeah i mean maybe

Justin Gardner (@rhynorater) (01:25:03.103)
really? The Python codecs?

Justin Gardner (@rhynorater) (01:25:10.625)
There doesn't seem to be checks for those sort of conversion errors.

Joel Margolis (teknogeek) (01:25:13.856)
I mean, it could be just the usage and the use case because it seems like puny code is basically like a separate thing. And then there's the IDNA stuff or like domain names specifically. And IDNA will do like extra stuff to normalize the host name or whatever. But so I'm still poking at this, but yeah, I think if other languages, other implementations of the IDNA slash IDN and puny code parsing, probably.

Justin Gardner (@rhynorater) (01:25:21.243)
Mm -hmm. Mm, okay.

Justin Gardner (@rhynorater) (01:25:43.905)
Probably the wordable. Take it guys, take it and run with it. Gareth, you know, that's the thing with these, man. Like reading the research is cool in doing stuff. And I was telling myself this as I was reading it. was like, reading this research is cool. But at the end of the day, it's not gonna do me any good if I don't implement any of this. what I told myself is I need to go build kinda workflows to detect some of this caching stuff. I need to actually add it to my like brain to actually check for encoded word whenever I'm dealing with email based stuff.

Joel Margolis (teknogeek) (01:25:44.034)
Probably, probably vulnerable. Yeah.

Justin Gardner (@rhynorater) (01:26:13.721)
And I need to start paying attention more attention to timing based stuff when I'm dealing within a black box environment where I'm trying to figure out what's going on because I think there is a lot of stuff that that a lot of doors that can be opened by all of those research techniques

Joel Margolis (teknogeek) (01:26:27.64)
Yeah, and one other just last little interesting tidbit is that they actually Tell you in the in the official documentation from from Python Docs. I'll Python or if you need the IDNA 2008 standard from RFC 5891 and RFC 5895 use the third party IDNA module and they just link you to a pipe I package that some some guy made so Just keep that in mind that also

The official docs will refer you to a third party library on PyPy that who knows how that's been implemented as well. just food for thought, more attack surface. And I will certainly be taking a look at it and I would recommend other people do too. Yeah.

Justin Gardner (@rhynorater) (01:27:11.861)
Yeah, very cool. right, so did you have anything on untrusted types that you wanted to talk about this week or are we gonna push that to next week?

Joel Margolis (teknogeek) (01:27:18.268)
No, no, we gotta do that next week. I haven't done a deep enough dive on it, but I will say trusted types API, very interesting. Seems like something that could be useful or potentially very dangerous. I hadn't heard of it until recently, but like the little synopsis that they say is that it's basically a way for web developers to lock down certain parts of the DOM to avoid client -side XSS.

Justin Gardner (@rhynorater) (01:27:30.037)
Mm

Justin Gardner (@rhynorater) (01:27:38.656)
Mm

Mm.

Joel Margolis (teknogeek) (01:27:43.18)
I guess mainly specifically this seems to be for DOM. DOM exercise is my guess. Maybe Reflective would fall into this somehow, but yeah. You'd mentioned that you knew a little bit about it, but hadn't done a deep dive and I haven't done a deep enough dive. So yeah, it's very interesting. I'm surprised I hadn't heard about this.

Justin Gardner (@rhynorater) (01:28:00.179)
It is indeed. I know that FileDescriptor has a tool called untrusted types, which I think helps monitor syncs for DOM XSS that abuses trusted types, as he says in the description. All right, let's double -click into that a little bit more next week, and we'll swing back around to that, yeah? All right, sounds good. That's wrap on this episode. Peace, guys.

Joel Margolis (teknogeek) (01:28:20.226)
Awesome.

Cool, yeah, sounds good.

Yes, Peace.