Philippe Laulheret - Intro to Hardware Hacking - DEF CON 27 Conference

[0:00] >>Without further adieu, because we stay on schedule. Philippe is here uh, from uh, from Portland, by way of uh, France from what I understand. And he is going to talk to, the, talk to us about uh, breaking into embedded devices, including phones, uh, phones that you might have on your desk at work phones that might be, I'm not even gonna get into that, phones that might be around. Let's learn about embedded device hacking. Let's give Philippe a big round of applause. [Applause]. Have a good time man. >>Thank you. Thank you, everyone um, so I’m really excited to be here, it’s like a big honor and um, yes, so today we are going to talk about hardware hacking and um, I’m going to show you this introduction to it and I hope um, I can show some love about um, hardware hacking. So uh, who am I? I’m Philippe Laulheret um, I’m a senior security researcher at McAfee, um, Advanced Threat Research. So I, I do um, software and hardware phone research, and you can find me on Twitter at PHLaul. Before doing McAfee, I spent 2 years doing uh, embedded security and before that I was a C++ Dev. So I'm

[1:03] not, not coming from a um, software background actually. So you could wonder like, why should you care about um, hardware hacking in the first place? Right? Um, I can see like two main reasons uh, first is that it's actually really fun and empowering to be able to um, you know, break op, break, open a device that you would like have in your home at work wherever. And instead of considering that it's a black box, you can instead you know, like start poking at it and understanding how it works. And then in a more like pragmatic way um, if, if we consider like security in the past couple of years uh, software is becoming much and much harder to like break. Like if you try to uh, I know like, pwn like uh, whatever like um, a media device like a web server like find, finding a vuln in the web server by itself is kind of hard. With, if instead you can just like open the device, solder a couple of wires, and get like a shell from that, that's much easier. So hardware hacking is great, because it's also like a lot of uh, low hanging fruits basically. So uh, today, we're going to talk about uh, Avaya phones um, and especially like the 9. The phone picture on the right side of the slide is a 9611g model. Uh, so Avaya is like one of the biggest uh, VoIP

[2:08] um, solution provider you can find them like all over the place. Uh, Google's like 90% of the Fortune 100 company and I think that's specific model of phone started in 2006, and is going to be enough life like at the end of the year or something. And the funny um, on the bottom right of my slides there is this a funny like um, screenshots from like a page from like a really old software firmware for these phones. Where um, actually like, you can see that um, you can only download that specific firmware, if you're like a DOD customer so um, basically then, uh, see that these phones are going to a lot of different places. So uh, we're getting prior art. Uh, in hardware hacking, it’s like a lot of information nowadays is on the internet, like you, you can find a lot of blog posts, tutorials um, there’s always like the Hardware Hacking Village that you all go, go and check out. Uh, CTF as well where you can like practice, uh, like kind of like the softer side of stuff and um, on my slides, I like a bunch of links at the end uh, so you can like download the slides at later time and um, also the slides uh, I’ve like, the one I've uploaded I’ve like, more text on them. So it's like

[3:11] a more like standalone. So you can read more um, and yeah, I read that in security uh, I found like 2 RC’s in the same uh, phone uh, something like five years ago or something, um, and they like talk at RSA called Stepping P3wns I would also recommend to check out online. So uh, for the next 40 minutes um, so I'm going to talk about um, hardware hacking and use that phone as a base to like support that conversation. Um, And basically the idea of like the whole situation is um, uh, like couple of like, 12 years ago, maybe uh, via Fox and open source codes, and um, they never applied security patches to that. And um, it turns out that like me, by poking at the device, I found that and it was like, women like unnoticed for like 10 years as they were like actually like a CV that got released like, yeah 10 years ago for Defcon. So I'm going to talk about uh, how I found that like that it was like some stupid like you know God inside that firmware. But more interesting, I think is uh, talking about the whole process, you know, like, how I did it, um, and why I did what I did it Why? Oops, sorry.

[4:15] Why I did stuff I did when I did it um, you know like, magic like sharing um, the process so that. And you know, also like sharing uh, know, what if the stuff I tried didn't work or, or selection stuff that didn't work, because, you know what, what works for one device might not work for another. And having just like that mental model of what we can do and what you cannot uh, helps you a lot approaching at any kind of device. And that's, I really think why I want people to take away from that talk is more like um, learning the process of that you can check out like whatever device you have. And yeah, we want to like insist that um, if I can do it, I think everyone in this room can do it as well. As I said I’m more from like software background, and the hardware hacking is still like, you know, like um, little scary, little foreign to me, but it's in the end, it's really not that hard if you have like a good process and you know, me work with patience. So uh, how did whole project start uh. Um, uh, to be honest, I was like, you know, working at work, and there were like all these cubicles, on the, on every desk, I see this like phone like standing on, on the desk. I’m like, wait a minute, these phones are running Linux, they have you know, network access, they have like

[5:17] mics and stuff is like, how, how secure is that? You know, like, I know, you, you wonder like, what's up with it, did people consider the security of this? So um, the first step before you know even like, I still like stealing my coworkers phone and breaking it apart, you can like, and that's generally [inaudible], you can check out the FCC websites usually like any device that has Wi Fi or Bluetooth, has to be um, on the FCC website for a compliance reason. So, that's what I found. Um, That's not my phone. That's like a, also model of same brands, basically and like pretty close. So you see all these cool pictures. But more interesting is the, the inside of the phone. Uh, so we can then see it and on the left side in right, in red um, I’ve highlighted this like random debug connector, I don't know what it is, but that's interesting to see that's there. And on the right side um, there is like that, like um, in purple there is that flash memory. So that's usually where the file system of the phone is going to be stored. And the whole point of that is just to get an idea of what to expect, you know, before even approaching a real target, like building hardware and stuff. So that um, you, you know, like if it's going to be an easy target or hard target

[6:23] uh, you know, if everything was covered in black epoxy, that will be harder. So another important step is like figure, um, getting like um, online material, like um, you know, um, marketing brochure, so you have an idea of what the phone, what are phone capabilities in the ecosystem surrounding it, so that you have an idea of um, you know, like the attack surface. And you can also find like user manuals and stuff. And for instance, I found that one that was pretty cool. It's like a user, like a, advanced user manual that says, oh, if you want a serial connection to the phone, you need that funky box. And uh, yeah, which is interesting it teach, it tells us that yeah, you can find it’s serial connection and you may have had some special hardware for that, but, are we going to see that in a bit. And uh, after that, you also want to check like forms and um, so that you can find out what poor user are doing like for instance, I found like the default passwords for the phone in debug interface, because um, and system admin. So here, uh, talk uh, talking about that online. And uh, if you're lucky, you can also download if you're more of an idea of uh, what's up there. And then we'll

[7:23] cover that in a minute. Okay, so uh, what do we do now, uh, now we have an idea of like, what it might look inside. This is a hardware hacking talk. So we're going to provide volunteers and so the point is like you open the phone first. Um, and the trick is like, usually you always expect to find the same stuff inside. And uh, if you look at the labours of the components, you will see uh, you, and you Google that uh, you will find out like what it is. So for instance that's my phone, and um, you can see the centre of the CPU a, above it like the RAM, in like a lavender ish colour, like some unpopulated like header that I marked with a question mark. This, I think the same way, there are the one we were like seeing in the picture before and I, it might be JTag, I did not have to use it. So I don't know actually. Uh, in proper, we have like the Eeprom’s that’s uh, with the whole like some settings and stuff. Uh, in dark blue in the bottom of the um, of the slide is like a keyed RJ45. So I was kind of, kind of confused when I was looking at that. And I didn't know even the term for it. And the idea is just a regular, RJ45 uh, like Internet plug, but with a weird um, connector so that you cannot plug the, the wrong stuff into it. Um, and uh, more interesting

[8:26] in yellow uh, the UART’s. So it's UART is kind of like the serial console and um, we have like these experts like uh, pads. And so usually like in manufacturing or something you would have like a pogo pins that would like generally connect to the pads. And this uh, that one is actually really interesting, because you would find like serial con, you'd expect to find the serial consol on that. And on the back, we can see the NAND flash that we've seen the previous picture. And um, yeah, from that is like where the file system would be, like stored on, on the device. So okay, you have um, you, we looked at the stuff inside and the, the like, other thing that's interesting to look at is test points and debug headers. So, and like basically all this kind of stuff that on those components. And you can ask, why would you expect to find them inside? And a couple of reasons like ones like just, usually like the Dev board that, that the guys are going to use is the same as the production board. And maybe they're not, just not going to solve the other debug components. It's the same as if you are like software, and it was built with like debug symbols, for instance. And um, also like part of the manufacturing process um, you, this test points and stuff might be used for like flashing

[9:30] the device ,or making sure it's flashed properly. And sometimes, you know, like something died on um, in, in the, in the real world. And they want to like do a post mortem to understand what happened. And that's useful as well. So what kind of stuff you want to look for in that situation? You want to look at UART that we like looking at before. JTag that I'm going to, uh, cover like in a minute. And then like you can also have like random test point that are chilling there that like all over um, the all over the table, all over the board. And there might be like long test points, er, like labour like TP1, TP2, whatever. And sometimes that, that might be like not interesting to you, but sometimes you can find interesting stuff like um, you might find that like oh if you short like TP64 uh, with ground is going to reset the device, which might be actually useful in some extent. So uh, looking at UART uh, so as I said is for like serial connection. And it comes in like 3 or 4 pins. The 3 pins will be uh, ground uh, transmit and receive. And the fourth pin would be like DCC, so 3 volts, 5 volt whatever. And usually you don't care about that one, you don't even want to plug it, you don't have to. And might uh, yeah, and uh, the, the trick to know is uh, the serial

[10:36] console uh, some data at different speed. And there are only like a couple of it's, called like Baudrates. And there was only like a few uh, common baud rates values, so if you try to plug your, your system in on, into the UART, and you just see gibberish on screen, the, probably the reason is, you just have the baud rate wrong and you could just like try to cycle through like all the possible, possible one, and it would find something eventually I think. Um, JTag is another one that's um, interesting um. So that's for like hardware debugging and ways that you could, like, you know uh, debug the CPU usually, and execute like instruction by instruction and whatnot. And it, they come in many different uh, shapes, number of pins and whatnot. So some example like on the bottom of the slides uh, but actually only a couple of wires are interesting and uh, at the end of the slides I have like a link to an article about like understanding JTag better. But basically, like, if you look on the top right side uh, of my slides, there is like a Dev board and say like a nice uh, JTag Connector like plugged there. And on the other hand, like on the bottom right, it's like some uh, random device I

[11:38] was looking at, which you can see, there should be a JTag, but it's never been soldered on and so I had to solder directly onto that stuff. And the trick to remember here, if you kind of like solder your own connector, is uh, you actually may have to add like extra resistors called like pull up and pull down resistors that connects like one pin to ground or one pin to like VCC. And the point of that is, actually, I don’t know the point of that, but uh, the reason is that if you don't have them, it might not work well or it's more work most of the time, it's going to be glitchy. So um, and if you follow the instructions from like, for instance like a arm, a specification for like what resistor you need, so, you might skip them, but then you might waste time like understanding why it's a weird, like weirdly f*****d up sometimes. So I would recommend the pull up and pull down resistors. So that's just random board that my coworker was looking at uh, just to show you that what I said is actually fairly generic. Uh, on the left side, we have like UART. On the right side there like some JTag connectors. Uh, up there is the mmm, the primary there is like, the main CPU, whatever that’s under like a shield. And usually you, you do that for um, preventing um, interferences. So

[12:43] it's like a Wi Fi module, so that's why. And the good thing to know is usually the shield is grounded. So like, if you're looking for the ground on your thing, you're like, either the shield on that kind of stuff, or like the casing around the Ethernet port or whatever is also act, actually like fairly useful. So uh,now that we learn how to uh, look into stuff, we need to know what hardware you need for um, interfacing with what's inside. So I'm going to present like a bunch of uh, cool hacking tool set that's important. So the first one uh, you may want this pin or like the stickers, I don’t know if you're familiar with that um, the actually, like, what's really important is like, safety. Um, so you know, don't work with stuff that’s plugged in the wall. Uh, remember to unplug stuff working like well ventilated room, it's important. And uh, yeah, just don't be stupid when you're like uh, messing with hardware uh, but yeah. So next uh, picture from the lab uh, I'm holding a um, soldering irons. You can usually switch tip for like a fairly large tip for like big soldering and tiny ones for small soldering. I have a microscope, so that's really useful for doing uh, tiny soldering. And the trick is I, I tend to drink a lot of coffee and I shake, so

[13:47] I will, I would, I was thinking like I couldn’t do that, but actually, if you um, if you careful and the, the trick is like really try to um, you know, when you soldering, like try to hold your arm like straight and well grounded. And under the microscope, you can actually do a lot of stuff that’s, you just don’t think, you wouldn't be able to use, you would think you shouldn't be able to do it, but you actually, actually can. [Breathes out], sorry, um. Top right side um, it's more actually used for the tiny soldering and the um, mid sec top right is like flux, so it's kind of um, magic glue, magic uh, gooey sta, it’s a substance that you put on um, pins to uh, when you start doing surface mount or pads so that it works really well. So don't, never forget, flux. Uh, bottom is um, sa, like, in the same uh, size, like uh, copper wires, like a minal wire that you use, usually for making codes, you know, for like radio or whatever. Um, and the trick is that the, the wire is actually um, coated in uh, varnish so it doesn't short. And you can use it to do like really tiny point to point like soldering, and that's actually really useful. And just next to it, you have like some

[14:48] desoldering wick. So like, the idea is, if you put too much solder and you f**k it up uh, you can put the wick on top of the solder block that you messed up with and you um, put the soldering iron on top of that, it's going to get really hot, so you want to hold it through like the plastic casing and it’s going to absorb all the like extra solder. And you can clean up your stuff. Then I put a picture of like some random like uh, jumper wires that are, are, or usually, always useful to have. And on the bottom right, I have like that crazy heat uh, gun. Uh, it's just like 60 bucks at Home Depot. It's meant to like, help paint dry or something, I'm not sure, but it's actually really useful for if you’re on the chip, if you want to remove component by just like heating them and pulling them off. So uh, more cool hardware um, that's a multimeter. Um, if you're not familiar with that, it's easy to for, find voltage of chip you don't know, like what's the actual voltage, so the black goes to ground and the, with the red you uh, with the white probably select poking around to uh, measure the voltage. And there was a really cool stuff is you can set in continuity mode and when you have like connection between the two probes, it beeps. So if you do

[15:51] like crappy soldering job, as I was describing just before, you can make sure that you actually did a proper job where like everything is connected the way you think, by you know making sure that both like, when you probably, both side it actually beeps. And that's actually fairly useful uh, tool. The other one is, this is a Logic Analyzer. Uh, the name is like a Sally, um, that's a brand. And the idea is, the same as in software, you have like ways of communicating between process, like in hardware, you have like ways of communicating between chips or the CPU and the chip. And it’s uh, it's like a lot of different standards, different ways of doing stuff. But the logic analyzer like know a lot of them and it's going to convert like a bunch of signals going , going on to wires into actual data. And that's actually really useful. That would be your eyes, like when you're doing uh, debugging, you know.

[16:32] And that's like your snapshot of like some SPI communication uh, which spies with some random protocol. Uh, that's like an FTDI cable also come in many uh, serial cable, it comes in many different shapes and stuff. And the idea is just like, when you want to do like UART, you, you plug that into your laptop, and you connect the user side to like the UART port and you get like a console, it shows up as a serial port. And uh, as an example I’ve um, on the bottom right um, like a raspberry pi that’s, you can actually get to serial console and you don't need like HDMI cable or anything to interface with that. Whoo. Um, the Bus Pirate uh, also like cool device um, it does um, It, it's kind of dependent of the hardware hacking at um sorry, with um, the Sally logic analyzer, so instead of absorbing data, it's like send data, so it speaks a bunch of different protocol and you can use Python to programme it. So it's extremely useful if you want to like read flash, programme um, microcontrollers and that kind of stuff. Um, more cool stuff I covered. I was talking about JTag, that's the JTag debuggers that you, that's where you would plug actually like the Digitech port on. They

[17:38] come in many different like uh, price range and whatnot. So on the bottom right is a fly swatter it's like, probably like 30 bucks, I think. And it works okay, and above it it's like a J Link, which is more fancy. And the idea like, the more expensive the stuff is, the more, the probably the more reliable it is. And it comes with like better software and whatnot. And on the bottom right, the pink bodies are JTagulator later that Joe Grand, the same guy who made up those badges uh, made. And it's really awesome. So, if you're like that board, you don't know what it is until like a bunch of test point and you like, I bet there is some JTag somewhere in it, but you don't know where. You can just wire like all the test points to the TJagulator and is going to brute force like and try to find everything. So it's really cool. Um, those are interest, important tool is uh, Flash readers. So usually Flash content, the Flash as the file system and stuff. And um, so you want to like interface with that. And uh, it comes in two flavour, I guess, you can do like in circuit stuff. So you have like either like your little like clamp that you put on the chip or a little rectangular one um, in to clip on. Um, so like the Raspberry Pi is a good one for that. And on

[18:42] the top right, is like a flash cat, uh, it’s like useful for the rectangular one. Um, but you can also like, choose to like, actually disorder the chip and, in that case, you would have like some socket adapter. So you can clip the chip on and then you plug it into your Flash cat and whatever, and it's going to read it. And that, I will cover like why, whats, my, whats, why it's important like both in a minute. And finally, it's kind of the um, last uh, hope. So let's say uh, you know your Flash is encrypted, there is no UART , there is no firmware. So you can try to do like the more crazy stuff like, first injection and such an attack. So that's the chips whispers like more like a kind of hard, more badass device. And the idea with that is um, it, you can do stuff like um, you are wire it um, to the CPU and you're going to drop the power on the CPU for like a split second and it’s going to like mess up some instruction and you can bypass checks and stuff. That's pro, what people use for like gaming console hacking and, and um, that kind of stuff. So uh, what do we do now um, that we have an idea of what's inside the phone and um, you know um, the tools we can use. So like, I think it's three

[19:47] posts potential track, one is messing with the UART, so the serial console, and this one is uh, messing with the flash and trying to recover the firmware or something and then maybe you can just download the firmware online. And the three are like valid approach and sometimes it's good to just go from one to another to another and you know what, whatever you find like some hardware you can like go from one to another. Uh, so I went for the UART first and that's what I'm greeted at by when I connect my FTDI cable to the UART ports that I, I was doing before. So um, we can see that on the left side, the, the um, the console thing, it's like dot dot dot, usually when you see that, you can press space or escape and it's going to interrupt the booting process of the phone. And in that situation, uh, but somehow here it didn't work. And I, I will cover that why in a minute. And on the top right side of the slide um, we can see like it read something from the EEPROM and then decided to set the console to Dev null. So you don't want console to be Dev null because if that goes nowhere. But that's interesting to know that um, there is some potential there. And finally, it's like booting in next so it con, confirm us that it's a uniques port, but then uh,

[20:48] there's nothing more uh, after that. So there's some good thing, bad things. The problems are like nothing shows up after like the Linux is as we did, probably because of the Dev null situation. And when I press keys are talking and um, I'm going to like come up with like, why that happened. So at that point, I was still like, okay, I, I cannot go anywhere. So I'm going to try to recover the firmware to try to understand what's up with that. Uh, multiple ways of recovering firmwares uh, is like an easy ways like you can do it online. And actually that worked um, there is like a quick snapshot of that, uh, sometimes it’s encrypted. On our case, it wasn't. Um, but, yeah. So you can also try to sniff a firmware update uh, that’s really useful for IoT devices in general. And you can use a network tab, which is like the thing pictured on the top right of the slide. Um, or you can mirror your port on your switch or whatever. Uh, the problem with all of that is oftentimes a firmware updates going to do over like h, HTPS, or it might be, in this case, you're like stuck. And then you want to go for like a more like uh, hardware hacking stuff or like you’re going to dump a flash, for instance. And like

[21:48] even more advanced techniques that I will briefly like uh, talk about. So how do you dump a flash? There’s two ways, as I was mentioning, just before, you can do in circuit programming and um, out of circuits. So in circuit is cool, because you don't have to, you know, like remove stuff, break stuff. But a couple of issue is that um, the main idea is to read the flash, you need to power the flash. And if you, if it's like still plugged into the whole system. Uh, it, when you're like Raspberry Pi, for instance, going to try to read the flash, it's going to tr, try to power like everything else that's connected to the same like like power line. And either you know, USB Pi is not strong enough to power like a VoIP phone, which you would expect. And in that case uh, but it's just not going to work. Or maybe your Raspberry Pi is really strong and it’s going to do it, but then the problem is the CPU stop booting and it’s going to talk with the flash and that's going onto to mess up your stuff. So in that situation, you have like two choices usually, either you can try to like lift up the pin, the ball pins, so that it doesn't happen. Or you can kind of go a little more like hero approach where you want to keep the CPU in reset. So I've been successful with that many times

[22:51] or I just like starts like trying to ground like random test points, of stuff on the boards, praying that is not going to find anything. And usually at, at some point like, I'm going to notice that okay, when I do that the phone like light up, it doesn't boot, that's good. And maybe better ways you can desolder the chip with it like to be like heatgun or whatnot. You have to be careful though, because if you like pull it too hard, you might like bend pins and is going to be like useless. Um, with a good trick to know is that you can make a little like um, tinfoil hat or whatever, like on the chip, so that like when you're heating up the chip, like the stuff around, it doesn't get too hot. Er, so you don't accidentally like knock off like some like tiny like components, that would be like a pain in the butt to um, put back. And uh, bottom right is like I was, I was, I thought I would have to do that on the phone. So I wanted to practice first and then just took like some random uh, USB thumb drive and I tried. And actually, that worked really ff, nice. But the trick is to remember, I also put some flux because it helps. And uh, yeah, just don't, be careful with, when you put it off. A few other tricks that actually didn't work on this one. But I, good to know. Uh, U-Boot is the most common boot loader uh, not used here, but it's, you would find

[23:56] that on pretty much every other device. And the idea is you can also instruct a boot process of that and asked nicely you would, hey, can you load the flash in memory, and then uh, U-Boot is also able to dump in a Hex like in the screen um. The, the memory, is this way, you can just get, write into the script to do that and recover everything from the flash, which is really cool. But usually like Dev don't like letting you do that. They said they lock that approach, so, so you cannot like press space to interrupt the boot process. But there is a trick that I also call like the Yolo approach where, instead, when. The idea is like U-Boot is going to load from the flash like the, the new scanner or something. But if while it’s happening, you start like poking with, you know, like a wired is connected into ground and they started poking the NAND flash, like some of the pins, it's going to mess up like what’s uh, u-boot is reading and after a couple of tries, u-boot is going to be like, d**n that kernel is f**ked up, I don't know what to do and it’s going to give you like a shell. So that's really cool. But yeah, as I said that one we don’t work on our phone unfortunately. But uh just some cool example that I found online was like some guy actually did that. And uh, I underlined in

[24:59] uh, in the like purple on, on the bottom, like the command you need to use to like log the, the, the flash in memory. And on the right side, the secondary block of text is um, sorry, is, how uh, you will just panicking and be like, er, I don't know how to do with your kernel and give you the, the shell. So uh, how um, so a few more like tricks for um, dum, dumping the flash, so you can use JTag. So usually you have uh, if, if JTag works, you're going have to be able to um, dump the RAM. So in this case, that won't be really like dumping the flash, it will involved dumping the firmware, but, in a lot of devices it’s going to load the whole stuff in memories, so if you can dump the memory you are, you, you the same means you pretty good there. And kind of the last hope with this. That's one of my favourite about hardware hacking is, you can do crazy uh, stuff uh, and an example is a Scanlime um, that person on the internet who um, she had a um, a Wacom tablet that she wanted to dump the um, the firmware off and there was like no attack surface at all, which she realised that like when you [inaudible] USB stuff, and when you plug your USB um, onto the laptops, there is some USB exchange that I

[26:01] don't know anything off, but it sounds like USB descriptor. And if you, if you glitch the CPU just at that time, um, it's going to like, know, miss like a binary check and is going to send a way more data. And with that uh, she was able to recover like the full flash, full memory or something. So it's really impressive, but, and that's what's kind of awesome about hardware hacking. Okay, so uh, we recovered the firmware uh, in my case, just by downloading online, but, what do we do now? We want to analyse it. Um, the best tool for that is called Binwalk. And it's just basically magic dictionary that's going to like look at your binary blog of stuff and be like, mmm, halfway through it looks like there is a zip file, I'm going to try to extract it. It tries, maybe it works, maybe it doesn't depending if it was actually zip filed or not. And it does that for everything. And uh, it might also find like compressed file system such as Squashfs, JFFS2 etc. And sometimes like your um, your firmware file might just start with like Elf headers or just like random like ARM code that you could like, look at. So I haven't been work on. So that's the firmware file, I don't know it from the website.

[27:01] And once again, it's easy mode, it's just a doc file. And they already like split it up into boot 1, boot 2 and JFFS2. So like, Binwalk has nothing to do here, really, and I can just extract the all file system. And that's cool, because from there, I can look at what's on the phone, just like that. Uh, but the more interesting, use case for Binwalk is, I took like a different uh, firmware file for like, a [inaudible] phone. And this one is way more interesting, because Binwalk for, find like a way more lot, lot of stuff. And uh, I've highlighted, like you find the file system and is able to extract it. And there's also that random like my SQL stuff, which I think is a false positive, then you need to be aware that there will be a lot of false positives, so don't blindly trust what the Binwalk says. And uh, really, really funny stuff that I was looking at uh, if you run strings on that specific image um, you will find a U-Boot strings. So, that fun is using U-Boot and the cool stuff I was describing before, would actually work on that one, look. So now that you have extracted the firmware file, what do you want to look for? One of the most uh, interesting stuff is uh, finding the you, update mechanism. So assuming you have uh, encrypted firmware,

[28:04] like if you reverse engineer that you might find the keys and how to extr, to extract like subsequent uh, firmware updates without having to do like, any hardware stuff. And then you want to look for like secrets, like uh, passwords, certificates and so and so forth. Uh, originally, my goal was to look at the main binaries. So that's just the thing and deal with like the VoIP stack. Uh, that could be make sense to look at that. And um, also you want to look at the bootloader because, it may have a hidden commands, explain why all console is not working, that kind of stuff. So, remember, the point of all of that was, I wanted to get a shell on the device and um, what's the process to do that? So, the one I want to go for is, I want to fix the serial console, you know like the Dev null was not cool. Um, but also approach will be like messing with your boot arguments. Um, not working here, obviously. Uhm but you know, you can specify the console you want and the, uniscript can be marked, oh I want Bin bash as my uniscript and you have a console. Uh, and like kind of last resort stuff that I was considering as well is, you could patch the firmware. In our case the firmware is signed. So uh, that won't work unless you try to glitch it, which really hard.

[29:08] But you can also try to patch a file system. So the way Binwalk extract the JFFS2 stuff, you could modify it and then like repackage and flush it directly onto the flash. And that's usually accurate, foolproof methods like, that would work in most cases. So instead, I'm going to uh, reverse uh, um, the Bootloader. And um, for that, um, because I wanted to understand like, why Dev null, a con, the console was set to Dev null and stuff. I'm just going to cover um, in much detail and I would be come on you to check out Azeria Labs Tutorials that cover everything, which just useful here um. There is this notion in ARM of literal pool.

[29:44] Which is um, inside the code section, you would find some little chunk of data and that's usually like used to la, loading registries, um, absolute addressees, imminent values, magic offset, that kind of stuff. And uh, FYI, like in ca, in case of function calls um, the arguments are going to R0 to R3. The rooted value is in RVO. And when you like, want to return from your caller, it's, the, the return address is saved in like the LR register, that stand for like link register. Um, and the big trick, big problem here is um, we're like trying to load the boot loader. So it's uh, it's not really Elf. So if you tried to import in IDA, IDA doesn't know what to do with it and so you need to find the loading address for that stuff, so it match what, where the phone is loading it. Couple of tricks um, so none of your load key is going to be printed on screen, that's our case, sweet. Um, but otherwise, you want to look at like header file, headers in the, in the bootloader file or maybe recent vectors so that stuff for like, specifically for ARM, like the CPU needs to know uh, specific addresses where like, a recess or an intrep happening and it's like funny like um, square like,

[30:50] how it’s being set up. So if you find that and you find like what looks like an intrep vector handle or something, you can find like, ah, that must be the address. And uh, finally, the literal pool I was mentioning uh, really useful with a because it's going to contain like absolute addresses or point to strings and stuff. So uh, if, if like you have strings, and they look like all f**ked up, you probably got the wrong address. And uh, Quarkslab, uh, what a blockbuster blog, so I would recommend you checking it out. So as I said, in our case, it was easy. Um, when it boots it prints uh, the jump, whatever. And that's actually WIFI loaded.

[31:21] And um, on the bottom right side, it's like a bunch of uh, strings um, so it's a bunch of addresses that point to strings. Uh, the system snapshot from IDA and um, we can see that uh, um,it's all the strings like prints well, so that means that it's loaded properly. It's not like all messed up. Okay, so now that we have the bootloader and we can like work it with IDA, uh, we want to search for strings uh, we were looking at, you know, like that sentence like EEPROM read successful, whatever. So we can use like cross reference in IDA to find where the string is coming from and look at the function that handles that. And uh, yeah, just as a FYI, but like EEPROM and Spi, so the EEPROM that's on the board is called a Spiprom and Spi is just one of this protocol I was mentioning before. EEPROM is just uh, like a small memory, um, whatever. And the spi uh, protocol is just like 4 wires um, and the ID to CPU in command and EEPROM like reply, and uh, and that's a dump from like using the logic analyzer to have a look at what it would look like in real life. Uh, that's the data sheets of the EEPROM

[32:22] that you can find on, online, if you google the, the label on it. Um, the cool thing is in the bottom left uh, of the slide, it tells you like, the pinouts, so you how to connect stuff to it. And on the right side of the slide um, it tells you which, which, upcodes are for Spi command would send. Like for instance, if you send uh, 3 it’s going to be read, read, if you send 2, it's going to be write. So that's uh, that's something useful to know, right? Oops, sorry. Back to the um, back to the IDA and the uh, bootloader, we can see uh, the strings we were like looking for. And something is for me promising is, we can see the uh, console set to Dev null, but above that, that I could bur, 2 of the option and it’s actually saying like, oh, setting console to like UART 0 or UART 1. So we want these ones to be printed.

[33:08] Um, that's a, that's a function that actually does print that string. And uh, what it does is roughly um, cleaner, clear some bites on the stack. And then you read for me from from a magic address. And it's going to compare like bytes um, that has just been read. When if we have the good value, it’s going to actually print on the console setting, the console for uh, Dev ttyAMA0. So we want that to happen. Uh, and something you can ask me is like, oh, how did you find that read EEPROM function, you know? And there like more a couple of ways of doing that. Uh, the easy way that I've highlighted in green, that screenshot, is uh, on the bottom right, you can see some like logging that says, oh, EEPROM read fails. So I know that function is responsible for reading EEPROM. Uh, but maybe, maybe the few more I’m going to look is not uh, as uh, variables. In which case um, as I said before, like, to read from a EEPROM you know you have to send 3over SPI. So, if you find like the SPI function, and you see like sending 3, you know what it is. This whatever highlighted in yellow. But then like, how do you find a spi function in this kind of um, programme? Uh, 2 ways for that.

[34:07] Um, my guess once again, it was logged, so it was easy. But usually like, the main trick is um, you have this um, something called um, mapped uh, IO MMO mapped IO. It's this kind of like magic address values in the CPU that like if you write to them, it's actually going to send data over like you auto spi or whatever. If you're lucky, you're going to have a data sheet for the CPU. And it specifies exactly what are those addresses. If you're unlucky um, you have to guess and that's really hard. And, at the time, maybe you should like look for something else, or just you know, you're going to spend a lot of time on that. So we want to uh, patch the EEPROM now, so we can uh, build the console. The plan of action is, we know the read, write comments uh, we want to interface with the flash. So to do that um, uh sorry, the EEPROM, we, we um, we needed like a spiuh, device and that we've seen before the Bus Pirate is a perfect candidate for that. And we need to connect to the chip, uh, unfortunately,

[35:09] like it's really tiny one and so like the clip I was showing how like too big for that. You can buy online some rigs that people make um, it's like, expensive, and it's like five weeks to get them. So instead, I went kind of crazy and I tried to solder like um, little wires onto the thing. And I was like, honestly, that's never going to work, but happened to actually work. And that was like, sweet. And uh, it's all like look like cross because it’s like all the flux that remains on that. Um, and that's the code for the Bus Pirate. It's super like boilerplate uh, thing. And when you just write um, this magic values that uh, sorry, I was asked to like, replace, them with like, images. But uh, the idea is like, just after that you're greeted with console is set to uh, like your R0 and now we have a lot of more stuff uh, printed on console, which is great. But unfortunately I, I get like this a harder like, the input are still like not working, I was like trying to press space or type like, it’s as, asking for oh, you want to look as root? What’s, I type, nothing happens, like ah, what happened? Did I just cut the trace, so you cannot send data? We have the boards, we can look

[36:09] at it, um. So uh, this just like, like a zoom up on the thing and at that time, I was like, you know, looking at the UART pad and start like with my millimetre, like trying to beep out, like where it goes, you know it’s, following the continuity testing. And eventually it reached some wires, like on the right side and wires are that little hole that's connect one side of the ball to the other. So the trick is the actual like trace that connects UART to whatever uh, goes to the other side of the board and goes oh, back and forth. And it's like really, like tedious. And eventually I kind of lost where it was going.

[36:39] But it was kind of under that MOD port, uh, that's like the keyed RJ45. Um, and so um, remember like the recon side uh, recon part, um, we have that box and it’s like ah, d**n, that’s, that’s the thing. So I kind of went via like, my euro approach again where, I was holding like a, a plug, like a jumper cable in my um, the transmit of my FTDI cable. And uh, so I had like this like wire that I could poke around. And on the other side, I was like pressing keys on the console trying to see if something happened. And meanwhile, I was poking like to the pins one by one. And I think at the second one, it actually worked. And I got greeted by uh, like a root shell on the device. So that was cool um. But the trick was um, the, on the right side of like a proper setup, where I plugged my ethernet cable into the plug. So at that time, I didn't want know where the key the RJ45 was. So I was just like sanded the plastic of the ethernet cable so I could like shove it in, and yeah, it just works, so that was nice.

[37:36] Uh, now that we have a root shell is it like uh, a lot of debug strings prin, printed on screen and that's annoying. So um, the trigger was like, I want to kill the process, it's like, sha, that's like flooding my console. Uh, there’s a watchdog that’s going to reboot the device if that happens, so I highlight in the corner, like the magic commands that you can run to kill that. It’s just like a random, unique stuff. Uh, so now we can finally do some Vuln research. Um, and really what you are going to do is look at process running, poke around, look at open ports. Uh, there's just a bunch of processes, uh, new sequel stuff, but um, that’s when it’s all getting interesting. I think at some point I was tired of looking in the IDA and I was like, I’m going to poke more. And I looked at DHClient and like huh, and the really funny parts like, I run the DHClient and I see this like 2007 copyright. I was like, woah, that's not a good thing. And if you just run DHClient by itself, it's Segfaults, like that's weird. And if you like, run it the right way, they print a debug message that say oh, we modified it to work it better on our device. I’m like, huh, that's interesting. So, if we

[38:36] look back, and we look for the specific version, it turns out that, 10 years ago, actually, for DEFCON uh, Jon Oberheide um, really is like an exploit for that specific version of the DHClient, so I was like, well, that's, that's easy. Um, but we want to make sure it's actually vil, still vulnerable. So the um, the phone was like a doing like um, Stack Overflow, like while handling DHCP options. So we can compare the original source code with the patch source code because DHClient is open source. And just keep in mind, that might be like mitigation, like stack cookies, ASLR whatnot. So if we look at the vulnerable code on the left side, we can see a memcpy like highlighted in red. That’s using like data.len as a field. Or on the patched version that we place, that by 4. So the idea is like, this bug is uh subnet, uh, it's a bug in the subnet mask and link, and the subnet mask should be like 4 bytes. But, the way the HTP option works, is that you can send up to like 255 bytes, I think, and it used to just dumly trust, whatever you're sending, which is bad. And if we look in IDA like the actual like function, uh, we can see there is no stack canary and the vinner, the, the, on the

[39:39] left, on the right side, it's, it's not using that magic 4, so it's, it’s obviously like still vulnerable. So it simple to exploit stuff. Um, working on the phone os TDS, I was like, I'm going to try to set up QEMU um, if you're not familiar with QEMU it’s kind like emulation of ARM stuff or like mini of system and can do either like Userland or Full-System. Userland wouldn't work here because uh, DHCP is um, more going, try to modify an IP and stuff and that's not cool. Um, but instead, we can do, do a Full-System emulation and uh, those tutorials tell you how to like, set up a query match for that. And we just want to set up the network properly, uh, because we want to interface with the DHCP and naturally um, the um, QEMU stack would give you a digi, DHCP for yourself, so you have to be, stay like don't do that and, yeah. So I run it in DH in QEMU and after like [inaudible] fault and it's still Segfault and it’s like, huh. So it turns out that like, Avaya modified stuff that actually causing the Segfault. And um, I was going to go back too fast, because I'm running like a little of out of time. But uh, basically it's creating that weird socket, uh, it’s like

[40:42] a name socket and uh, so that’s like the main application that the VOIP stack can talk with DHClient and show options. Um, and uh, the idea’s like, that's like the main application, then we can see what it does. That's non, not super interesting. But basically on the left side, it’s like sending data to configure DHClient at startup and on the right side, I re-implemented that in Python. And uh, you can run that with like Socat, um, so, Socat is just like, kind of like Netcat in um, in more like um, more tools, so you can share that name socket and the idea, you run the Python script on the host, you on socket on the um, QEMU like binar, image, and that just work. So now we can start debugging uh, DHClients, um, you can run GDB, we can try to like, actually run the bug. I do, I couldn't compile the original proof of concepts, it was like using a really old tools and it's just after like, hella, like couple of hours like, ah f**k that, I’m tired. So instead uh, I use a Scapy based uh, exploit. Scapy is a Python library that lets you like, you know, mess with uh, network packets, like a fairly low level. And um, so now we want to like reach the um, vulnerable part, vulnerable code, so um, I

[41:47] was explaining the bug in the uh, subnet mask option. And how, the way DHCP works is like a tea, tag length value like type of stuff. So you say, Oh, this option is a uh, subnet mask, here's the length, here’s the value. And obviously it was like, it would be 4, but if you send more, it's fine. But for the DHCP specs. And so we can use Scapy for doing that. So uh, that's my, like a chunk of my Scapy script and just the payload in properly is interesting. Um, basically just sending the subnet mask option with uh, data that will be like A, A, A, A, B, B, B, B, C, C, C, C and so on and so forth. So that um, when it crashes in GDB uh, we can see like all the register we control, and we know um, exactly um, you know, if we want to af, affect which register, which chunk of the shell code we need to change.

[42:33] And we can see that we actually controlling uh, [“pc’s” so the programme console, so that's good. We know that we have um, we can exploit, we have control, control for execution. And there like a few more details that I'm going to skip over. But um, we need to sta, to craft a shell code now. Uh, unfortunately, like it's it looks like the stack is uh, you cannot execute the stack, so we will have to work it. And that's tedious. But um, cool stuff is like if you look for there is like a system calling, that binary that I think also, Avaya added. And we do control R4. So if we put us, if we can put a string in memory in an address we know, that could be a value like, system comment, and put the address in R4, we good. Um, so if you think about it, DHCP client, DHClient, sorry, receive configuration values from the server. So probably there is going to be stuff that are stored somewhere and um, it turns out that ASLR is disabled, I, sorry, I forgot to mention that. Um, but then we, the idea is like, I just some like, sending like A, A, A, A’s as like, a bunch of uh, options and then look at memory to see if I could find that string somewhere. And eventu, eventually, the domain option

[43:36] was a good one. And so um, now I can show you uh, a cool demo that's, what happened when you actually run the whole stuff. So uh, on the, um, on the left side, we have the phone that's booting. The phone is booting and the window uh, the 10 minute window is like, on the left side is a, rog DHCP client running off payload. And on the right side is um, it's a web server that uh, will provide like uh, life post exploitation kind of stuff. So we can see the packet is running. To make sure it works, I was uh, catting dev random on the uh, on the screens. That's why it's looks like noise. It's fetching data from the um, web server and and I don’t know if you can see but it’s loading like steve.data. Uh, FYI, like, Steve is actually my uh, boss. That's him. [Laughter] And uh, just then um, uh, still like ex, I don’t know if there is sound but I’m ex- filtrating audio from the main speaker. And you can see it in VLC that when I'm speaking and clapping my hands, it's actually like reading the audio on the attacker laptop. So that's pretty scary because you can turn your um, phone into um, you

[44:37] know, listening device. [Applause]. Thank you. So um, yeah, and just FYI like all the tools I use like actually already on the phone, like you're like Netcat, your like stuff to like dim the audio to like, SDN, so you, ju, just like writing some shell script, like bash script, whatever. So um, as conclusion uh, for mitigation, like, you want to make sure you monitor your network so that you don't have like bugs like sending uh, you know, like weird DHCP packets. And uh, actually like McAfee uh, firewall, like can detect that. FYI, you want to segregate your network um, to make sure that, for instance, the coffee pot is not able to like send weird DHCP packets to your phone. And make sure IT actually patch your phone, because you know, it took, it's the patch notes release like, Avaya release a patch like a month ago. So make sure it's actually patched. And uh, if you want to think about why this kind of bug happened like, tech, technical data is really hard to handle and, no [inaudible] someone in good face um. Okay. Um, somebody would face um, fogged the stuff and they forget

[45:42] about it. So um, and just as a really quick um, embedded systems are not black boxes, so I would just encourage everyone to um, look at them and uh, you can do it now as well. So if you have question uh, ask me on Twitter or whatever and thank you for your time, guys. [Applause]