As it turns out, ransomware isn’t ‘technically’ the attack vector, it's the ultimate goal. Most ransomware cases use phishing as the means of compromising a network, that’s the most popular attack vector in 2023.
Regardless, a ransomware attack seeks to encrypt or exfiltrate a company's sensitive data or proprietary information. Money is then demanded to provide either a decryption key or to not release the data into the public domain.
To address this very real threat, Mat talks to Curtis Preston (aka Mr. Backup), an cyber security expert who specialises in disaster recovery and backups. Curtis brings incredible insight and a touch of humour to an otherwise dry subject.
Stepping through the ransomware-style cybercrime from A-Z, they discuss the preventative nature of using password manager apps and multi-factor authentication before taking a deep dive into the most common forms of attack vectors for businesses in 2023 and how businesses can be better prepared in the event that they become a victim of a ransomware attack.
Where does the term ‘air gap back up’ come from and what are the modern-day options?
What are the differences between cloud-based server backup versus using a backup SAAS?
What are the pros, cons and considerations of cloud-based backups versus ‘on-premises’ backups?
If you want to understand the answers to these questions then this episode of the CyberHeroes Podcast is for you.
Importantly, they also discuss the realities for businesses who have taken the steps to consider ‘cyber’ in their Business Continuity and Disaster Recovery planning versus the repercussions for those who have not.
As always the aim is to help you, the listener better able to protect your people, profits and reputation from cyber criminals.
If you get value from this then be sure to subscribe and why not share it with someone you know who would also benefit from sharing it?
Now settle in and enjoy.
Check out Curtis Preston (aka Mr. Backup) on Twitter
Check out Curtis Preston on LinkedIn
Check out Curtis' books, including Modern Data Protection HERE
Check out the Restore It All podcast HERE
Curtis, it's great to have you here. And I thought we'd dive straight in by asking you a question. And that question is, what is the most common form of cyber-attack today?
Or based on? You know what I'm seeing, for example, there's a Verizon report that comes out every year. And they pretty much talked about ransomware. That's all anyone's used to be talking to me about lately. Ransomware is kind of it's the end result, rather than the attack itself. Ransomware isn't necessarily the attack vector, as we say. It's really sort of the symptom. That happens after the initial attack, which may be, you know, we can talk about different ways like phishing and things like that.
What are some of the more common methods of entry, let's say the actual attack part?
Yeah, so I'd say the most common method of gaining entry into an environment is via stolen credentials in some way, right?
They basically steal your username and password, and that is typically done via some type of phishing attack phishing with a PH. For those that don't know that.
And this is where, you know, they basically send you an email, they maybe make a phone call, and they pretend to be someone else. Right. And they, they lure you in, they get you, you know, they send you and I got a text today that said that my, my card has been declined at a retailer, literally that level of vagueness, right, my card has been declined at a retailer, oh, which card might it be? Right. And, you know, they get you to trust them and to give up information that they could use.
And you know, the biggest one of which, if we're looking at a direct cyber-attack, to accompany would be your username and password, they may also get that via, you know, your username and password has been stolen via some other way. And it's being advertised on the dark web as one of the 1000s that are being sold via on some list.
And the and so I would say that you know, my biggest advice for people on the front end of things. One is to have good user, you know, basically, credential management, which today means username and password.
Hopefully, at some point in the relatively near future, we will do away with usernames and passwords, but for today, that's what we have. And so I talk a lot about I'm, I'm a big believer in password management systems. And, which allows you to have a separate username and password that way, if, if someone guesses your Netflix password, it's not also going to be the same username and password you use to administer your backup server. Right?
No, absolutely. I mean, we with our community, we talk a lot and encourage strongly the use of Password Manager episodes such as we tend to talk about one password or bit Warden or the two we tend to write. Right.
Yeah, and then, of course, MFA, right MFA is not perfect multi-factor authentication. It's not perfect. In fact, it's an attack vector. That some basically what they call MFA exhaustion. So they send you the bad actor, if they get your, you know, access. They send you multiple MFA authentication requests, hoping that you get you just get tired of and go fine, fine, fine, fine.
Even by the way, I don't understand that particular method. If I'm getting, you know, 25 MFA requests, and I'm not trying to log into anything. Me, I'm going to be like, What the hell is going on? Yeah, red flag not. My response is not going to be. Yeah, sure. Go ahead. But anyway, so MFA is not perfect, but it is. It's better than nothing, right?
Yeah, absolutely. Of course. Yeah. So we certainly recommend to anyone to use MFA or two-factor authentication. Sometimes it's wherever it's available, use it as a choice of how it's going to verify our Understanding is that SMS is the least secure. The options
might be in competition with email. Email is also used as a factor. But I think SMS is definitely a very bad. Again, still better than nothing. But for sure not what you should be using, you should be using some type of app.
Yeah, yeah. Microsoft has an authentication out. There's a Google and I tend to personally use Google. But for sure, if it's available, use it as advice and use for sure, have long, complex, unique passwords and make that possible by using a password manager app.
Right, exactly. And, you know, I'm currently up to 25 characters, I think, which is what I'm using for my password length. And it's because I, you know, there's a table somewhere that shows you how, how long it will take current computers to guess your password. And I started to get uncomfortable with my old length, which was like 16, I started to get uncomfortable with the relatively short period of time that they could do that. And so I just put it up to 25, which is like, I don't know, it's like 50 billion years or something.
Which shouldn't be a problem in our lifetime. Although computing power gets faster and more effective, and that's the problem cracking software gets better and better. So yeah, staying ahead of the curve, which is obviously the game that you're in, in the game, we're in and helping our clients Debian. So these are all ways that attacks can be launched, for sure. And the ultimate aim is, in the case of ransomware attacks is trying to get trying to get what they gain access and do what how do they, then yeah, so
really, the the, they need that initial infection, right? So so they need to gain full access to some computer within your environment, right. So that could be someone's laptop. I don't think they're yet using I know, they are attacking mobile phones, but they're not using them as an attack in the data centre yet, but so they infect your laptop, or they maybe they do infect a server directly, somehow. And then from there, and this is the this is the thing, I the one thing, I think it's important for people to understand, they don't then immediately start encrypting that server, right, they have gained access to the environment, they now have, you know, essentially a backdoor into your computing environment. So what they're going to start, the first thing that they're going to do that the software is going to do, is going to reach out to what's called a command and control server. And it does this often a very common way is to use DNS. And by the way, this was this was news to me not that long ago, I didn't understand how someone would use DNS as a way to have a two way conversation. And basically, the software reaches out to a very unique, very weird looking to you and me, hostname. And you know, it's a, it's a subdomain of a subdomain of a subdomain of a subdomain. And then the the request comes back with a particular string, maybe an IP address, and then that IP address is then interpreted by the software to say, Oh, well, this is what I need to do. And most likely, if it's the first infection, what it's going to be told to do is to sort of probe around, see where else it can go see what else it can infect. And so once you're in an environment, there are unfortunately, a number of very vulnerable applications and ports. A very good example of that is called the Remote Desktop Protocol, which is or RDP for short. So this is a system that pretty much every, you know, both Windows and UNIX and Linux have, and it allows you to, to assume control of the desktop remotely, right. And many system administration things are very UI driven. And so you need to be running a, you know, a piece of software locally on the desktop. And so this allows you to do that remotely. The problem is, RDP is incredibly insecure, and incredibly easy to hack if you have direct access to it. And that's why some and I love this, some refer to RDP is ransomware deployment protocol, because it's just a way to infect your system. Right? So they might use NFS, right, the network file share or SMB, the Server Message Block, which are just file sharing protocols within Windows and UNIX. And they might also they will also, and this is where my area of expertise really comes in, they might also begin directly attempting to identify and attack your backup infrastructure. For you know, for a lot of reasons,
which is a really key topic, and so when so when we'll come to that, I mean, so a ransomware attack, that, let's say, they say they've gained access, they have control, they've sat back, they've watched what's going on, they're taking their time, they've been able to instal further malware that will go off and re try and contaminate your take control of basically, and I'm sure, I'm assuming, and I know, but we'd love to hear your take on the prevalence of I mean, these cyber criminals, that they're not stupid, and so their malware will look for backups as well, correct.
Right. Right, it will definitely look for backups. And, and at some point, right, so you know, that initial infection happens, at some point, the malware will, will have gone as far as it can go, infected as many servers as it can infect. And again, mind you, this is just sort of a stealth infection, right? Nobody, so nobody knows this is happening. Meanwhile, you got this piece of software crawling around in your infrastructure, taking control of servers, or just, you know, basically getting onto servers somehow. And then, and one of which being the backup server, right? Then at some point, it deploys itself, right? It actually activates and says, Okay, now it's time to go for something, and it's going to do one of two things. If it's an old school ransomware product, it will simply try to encrypt, you know, as much as it can, if it's managed to infect a VMware infrastructure, for example, it might actually start encrypting the VMDK is the files on the server that actually, you know, each VMware VM virtual machine is, is really represented by one or more files on the VMware server, right? The same, the same with Hyper V, the same with pretty much any virtualization product. If they want to cripple you, if they can infect a VMware server, they can encrypt your entire VMware infrastructure like that, because they're just a bunch of files, right? They do that. But they, so that's one thing they're going to try to do. And then the other thing that they're going to try to do is steal information, right? exfiltrate is the fancy word for that. Basically, they want to access your important databases, your important file shares, your you know, if you live, if you live in a country that's subject to really good privacy laws, hopefully, you don't have things like spreadsheets with important, you know, personal information, and then that can be just grabbed off of a file share, but they're gonna look for that. You know, I know, like, for example, with GDPR, that that is considered a very bad thing to do. Right. And I know, I know, Australia has done a lot and the privacy law, we're pretty much really behind and that, you know, as we often are many things, but we, but that's what that's the other thing they're going to try to do is to, to steal information.
And they want to steal that information, because they can send a really good
point, which I failed to make, they're going to basically blackmail you with that information. So, so in the end, they're going to and you might get what's actually referred to as a double extortion, attack, meaning they say give us you know, I just, you know, to steal Austin Powers $1 million. Give us $1 million, or we are we will give you your environment back. Or, and or they can say give us $1 million, or we're going to dump your customer database out into the wild. Right or we're going to reveal this product that you've been working on, you know, under wraps for a year that you know, that nobody knows about. We're going to reveal really rip Yeah, yeah, IP or, you know, here in I live in Southern California, there was a some very embarrassing communications that came out from a Los Angeles City Council meeting. And some people lost their careers as a result. And they should have, by the way that based on the stuff that they were doing and saying, but that could also be it could be just be embarrassing stuff, right? We're going to the most famous one, there was actually Sony Pictures, back in 2014. The hack, which was believed at the time to be something from Korea, they revealed all these emails of how the Sony Pictures management spoke about their talent and damaging them as a reputation. So basically, it's just it's just blackmail, it's extortion, by saying, you know, we're going to reveal this data, which will damage your company. So that that's the the newer way that that people are, you know, attacking companies.
And so the victim in this case, that's the primary victim who's been held to ransom. They have, in basic terms, two options, I guess. One, they can pay the ransom, and hope they get their data back. Or two, they can restore their systems from a backup. We hope
Yeah, exactly. Yeah, exactly. And, you know, I'll just not to be too much gloom and doom. But if you've, if data has been stolen, I don't know what choice you have. Right? If you've managed this is why I even though I specialise in backup and recovery, and that's really my my thing. I try to tell people that you should be looking, there are ways to watch for stop, you know, looking for data exfiltration going on in your environment, right? Because if you, if that happens, those two choices really come down to well, the two choices are pay the ransom. And maybe they don't publish the data, or don't pay the ransom, and they're going to publish the data and damage your company. Right. But yeah, so that is sort of separate, but I always, you know, want to bring this up, right. But yeah, the backup system, the Dr. System is crucial in this. And unfortunately, the bad actors have figured this out as well. And so what they've started doing is they've started directly attacking the backup server as part of the attacks that we talked about earlier. Right? These are smart folks. And they're like, you know, if we can find out what kind of backup software they're running, and by the way, it's not that hard to figure it out. If you know what you're doing. You know, give me a log into one of your servers, I'll figure out what backup software you're running, it's not that hard. And then I'll figure out which one is the backup server, again, not that hard. And then I just do a direct attack against it. And God forbid, you've got RDP turned on, right? Or maybe I know specific exploits, that are specific to that particular backup software package. Just within the last, you know, I don't want to name names, but very big software company, very big backup software company had an exploit that, you know, was used, actually two of them in the last, in the last two weeks, we've had two big companies where there was an exploit that was specific to their particular backup software package. And it was used to do this, right. So and they directly target the backups, again, for the same two reasons that they started in the first place. They're either want to encrypt the backup software, or delete the backups to basically take the backups out of the equation. And, or they actually see the backup server as an exfiltration point. And this one, again, is why I talk about it in the front half is you really want to make sure that you're stopping that exfiltration. So if they can gain administrative access to your backup server, they can do whatever they want, right? They can delete all the backups. They can also restore backups. They can do activities that look like normal backup and recovery activities, when in reality, what they're doing is they're restoring data for the purposes of stealing it. Right?
Sure. Yeah. So the I mean, so the only threat is not the risk of data being released, though, is it? I mean, if their systems if their data on the company servers are encrypted If they can't function, they can't operate, as usual. Right?
Right. Yeah. So either they encrypt the data and just say, Well, you know, you lost control of this database, or as I was saying earlier, they will, they can actually encrypt your server itself. What? Sometimes they don't actually encrypt. Sometimes they just basically hack your, the boot block, so that your server won't won't boot. You know, they're very resourceful and not nice people. So yeah, that's
well organised, well funded. And in Korea, many
huge both criminal and state sponsored activity, right? Yeah, North Korea, Russia, especially. These organisations are very much very well funded, very big companies with HR departments. They're really well organised. They reward ingenuity within their company as just like any other company does. And so this isn't just some random software run amok, it is there are real people on the other end of these attacks. And the attack on your company could be, you know, being directly managed, and even directly conducted by an actual human being that is well funded. And well, you know, and available. 24/7. Right.
Yeah, sure. So when it comes to backing up, then certainly a really common technique at the moment is using an air gapped backup, right. But if you could explain what that is, for the audience, that'd be great. And then we can talk about the merits, pros and cons.
So the term comes from back in the day, right? So I've been doing backups for a while. Back in the day, we, you know, we cut a tape, right? Tape was everything right? It was maybe 20 years or so ago that that, that we started to migrate over to disk. So back in the day, you cut a tape, you put a tape in a box, and you gave it to a man in a van, right? Who drove around and picked up everybody's tapes and and stored them securely. You had a gap of air between A and B, right? Which meant that you, you the worst thing could happen in your data centre, which back then we were concerned primarily about disasters and terrorist actions, right. But in this case, we would add, even if you were hacked, your entire data centre was hacked, it wouldn't hack that tape in a box in a vault. Right? Absolutely. So So we still what we now that's not really possible in most environments today, because they're not using tape or any kind of removable, removable medium. And so what we talk about today is some call it a virtual air gap or an operational air gap. And that is to make sure that at least one of the backups, at least one of the copies of your backups, or possibly the entire backup system is air gapped, meaning that it's in a completely separate administrative domain, password management, domain, computing environment, storage environment, make it as separate as possible. And so that, you know, again, the worst could happen in your computing environment, and you it wouldn't be able to affect, in fact, the backup system.
Okay, so and that could be could be cloud based, or could be a physical on site, this sort of a backup that is not connected to the network. So it's backed up and then disconnected.
That's really difficult to do. There are some who talk a little bit about that. But the challenge with that is, generally speaking, go back to humans. The more you have humans involved, the more the less reliable a system is. Key, right? Sure thing. And so what most people are doing is they do one of two things. They either have an on prem backup system, that then at the end of every backup, then sends a copy of that backup up to some kind of cloud storage. And they may even use the immutable option. So for example, most object storage providers offer the idea of turning on immutability on a particular backup which is just a fancy word for means it can't be changed. And if you turn it on, depending on how you turn it on, you can even turn it on and situate it says you You can't delete it, right? Object Storage is already pre set up like like sort of base functionality. You can't encrypt an object, right? You, you can't even change an object, you can only make a new object and delete the old object, right? It's not like a file system. So it inherently already has sort of immutability built into it. But what what these features do is they then say, well, I'm going to turn on immutability for backups in this in this bucket that we talked about an s3, the storage protocol, anything that goes into this bucket has the immutability flag turned on, and basically means even I can't delete them, even if I change my mind. And that's what that's what so that's one thing that some people are doing. There are still people by the way that are cutting tapes and handing them to a man. And and there's nothing wrong with that either. Right? I mean, the only challenge with that will be I think you're, you're missing out on a lot of modern recovery possibilities by doing that. Other than that, there's nothing wrong with that, for example, and not from a ransomware perspective. Anyway, so I was saying that either they do that they copied up into the cloud, or they use a service provider in the cloud that does their backups for them. I want to differentiate between that and just putting your backup server in the cloud. So if you have, you know, pick your favourite backup product, and then you have an Amazon, you know, VPC, virtual private cloud, and you put your your favourite backup software in the Cloud, and then you backup from your, you know, wherever it is to the cloud. That's not what I'm talking about here because that that VPC is made to be seen as if it's essentially in your data centre. So that's not really changing the security posture. Right? What I'm talking about is, you're using a cloud provider, like Dhruva, which, which is where I work, right? We're not the only one. But the were certainly the leading SaaS provider and data protection. Basically, your in your in their authentication domain. It's servers that are managed by them. Yes, in our case, they happen to be in AWS. But it's not your AWS account, it's our AWS account. And it's designed to run in AWS so that, for example, we do use s3 to store your backups. s3 when we provision a new customer, which is all done automatically. The s3 buckets that are being configured for you are automatically configured so that only our processes can write to them, right. So even if someone were to somehow get a hold of credentials, they wouldn't be able to read and write from them. The there aren't backup servers that are running the way the way our processes work is there's no backup server that's running continually for you to for someone to try to hack. And the only input into our environment is the customer's usernames and passwords. Right? Which of course, we support both internal MFA and Okta, you know, that third party MFA. To make sure that we we do that as well. So that's a way to air gap your backups. So that basically, you can take your on prem environment, do a bunch of work, to have one of your copies be air gapped, or you can use a service like Dhruva, to ensure that all your backups are air gapped?
Sure, thank you. And you mentioned, for those who are still cutting tape and person in a van ticking the box from an air gap perspective, you said they might be missing some more modern opportunities in terms of recovery process, and I'm keen to hear what some of those would be.
Yeah, so a modern disaster recovery. And by the way, disaster recovery would be a subset of a ransomware. Recovery, right? There's so much that has to go on before you kick off the actual restore part of a ransomware recovery, right. You know, I talked about typically, disaster recovery happens after a disaster after a fire after a flood and the fire in the flood is gone. It's really easy to figure out what was damaged you just like, well, those black things over there. Those are, you know, those need to be restored. And these over here that are fine. Well, they don't need to be restored in a cyber attack. You don't know any of that. And it's still ongoing, right? So all of that stuff has to be done. But the point at which you can do a disaster recovery Every, if you have a modern backup and recovery system like Dhruva, you can use the cloud, you can basically decide in in advance what you want to restore in the event of a disaster, you configure you basically you configure the types of VMs. You know, this is mainly for a virtualized type environment. So you so I'm going to this VM, you know, in, in every cloud provider, there are different sort of types of VMs that have different power, different computing and RAM and all that you make all those decisions up front, we could do a default setup for you, and then you can tweak it as necessary. And, and you decide like the recovery order, right. And then you can do test recoveries. But essentially, when you go to do a restore, you literally just push a button in a web UI. And then poof, your entire environment that you pre configured is automatically configured, automatically restored, in a matter in our case, in a matter of about 15 to 20 minutes, regardless of the size of your environment, that's just not possible. If you're using tape, right? If you're using tape, you've got to use some sort of physical world, right, you've got to use some, there are service providers that will provide you a data centre, and you're gonna, you know, first thing you're gonna do is marshal the forces and get them over to the data centre and get the tapes over the data centre. By the time you're ready to start your recovery, our customers have have already been operational for several hours. Right? So if that's okay for you, right, if that meets your you know, in the Dr. world, in the backup world, we use the terms recovery time objective recovery point objective, how long the Restore is supposed to take, how much data you're allowed to lose. If that meets your requirements, then fine, right? It's just that it probably won't. Right. So
it's about making informed decisions, isn't it? And I know that we're working closely with a number of our members to help them develop their disaster recovery plans. Right. And that's what we're talking about, we're talking about, well, one of the advantages of doing so proactively is that you're then able to make informed decisions. And the same is true here. Now, interestingly, we still have some clients who prefer to keep their data on site and back it up with their own servers. And that the most common reason for that is just an ROI discussion, the return on investment discussion. So the cost of doing that compared with the cost of using cloud based solutions. I'm interested to hear if a client was presenting that to you, what would be the discussion you'd have with them?
Happens all the time. So what I would do is I would point out to them, the normal inefficiencies of a typical on prem, backup and recovery system. And the biggest thing is that you have to design what infrastructure you need for the next two to three years. Right? You have to sort of guess what your because Because honestly, just configuring the on prem backup infrastructure is a pain. Right? It's, it's a pain in so many regards. There's everything from the process of trying to figure out how to pay for it, paying for it, the POS and the capital, purchase requests and all that stuff. And then also figuring out how big does this thing need to be? Right? And if it's going to use D dupe, which it's probably going to use D dupe, which is short for deduplication. That's going to be at best a guess. Right? You're going to guess how big the infrastructure needs to be based on how well it will D dupe, which you don't know yet you won't know every D dupe algorithm is different. Which means you don't know how well this new system that you're buying will D dupe your data until you actually buy one and put it in the data centre. But meanwhile, you have to guess how big it should be by one and, and then 90% of it is going to go unused for most of the time. Right? Meanwhile, you're powering it. You're going to pay for what you provision. You're gonna buy a you know, 500 terabyte backup server with 57 cores with you know, 20 gigabytes of memory, whatever it is, I'm just trying to, you know, put numbers up there. You're going to, but you're buying today for what you need in two to three years. And you're going to pay for all that storage powerful that storage all that time. Meanwhile, if you use the cloud provider, and also by the way, there's all the costs of securing that infrastructure. All this stuff we spent earlier talking, you know, talking about earlier. In contrast, our customers, we still have to make a guess because again, everything I said about D dupe is true of our company as well. The we're talking about data centre here, if it's if it's, if it's laptops, or cloud stuff, we just charge by the by the user. But if we're talking about data centre, it's about terabytes. And you don't have to get it right. Like if you get it wrong in a data centre, your backup system is underpowered, and it won't work with us. If you if you guessed, you're not guessing, the you're not guessing the design of the system, all you're guessing is how big the bill is going to be. Because we're gonna globally deduplicate all your data, and then store that data, and then charge you for how much we stored. And so if you get it, there's two ways it's gonna go one way or the other, you will either get high or guess low. Guessing high is actually, you know, it happens. And if you guess high, it means that you paid for some credits, because basically what we do is we charge you in advance for like a year. And then we debit against, you know, that account, right for what you actually used, right? If you guessed high, you've got some credits leftover and you roll them into your next term. If you guess low, you just renew early, right? When you guess high or low, if you guess high and your which is the most common scenario in a data centre. If you guys high in your data centre, what you've done is you've wasted potentially hundreds of 1000s of dollars. If he gets low, you've got a really underpowered system. And I've seen it where people they really misjudged how the D dupe works. And it was underpowered on day one, and they had to go and re architect the system, that just doesn't happen for us. The other thing from a cost perspective already talked about, the other thing is what happens in a restore. So in a restore, you get most people configure their backup systems for what they need for backup, they don't also add an extra power for restores. So when you go to do a restore, you get the power of whatever that box or boxes happen to be. Boom, that's it, right. And it might also be doing other things, right, you might have this really important big restore, that needs to happen. But meanwhile, you also have backups that have to happen, right? So you get however fast that boxes, that's what you get with us, you get the entire power of the cloud, right? So we can basically, our CTO likes to use the phrase darken the sky, right, as long as we the only thing we have to make sure is that the amount of bandwidth between you and us is sufficient to do the job. And what happens in a large restore, we will simply just fill that pipe as much as you allow us. And I will tell you that we get into bake offs with on prem backup people all the time. And we often beat them, and meaning restore speed. And people people are amazed are like how is that possible? It's like, well, because they're running on a x86 box, and we're running with the entire power of the cloud. And that's why
Yeah, sure. Speaking of recoveries, and that's, it's often not the main focus, certainly in the media, when they talk about there's been a cyber attack, it's a ransomware attack, and they've asked for this much money. That tends to be what gets talked about. And certainly, as we're talking about here, the restoring the systems and getting the businesses up and running again, probably isn't sexy enough for the media, but it really matters for the business owner and for their clients. And I'm interested in your experience, the difference, the impact of an attack, a successful attack, that then leads to a restoration. The difference between those companies who who have disaster recovery plans who have an incident response plan in place, they thought about it, they've planned for it, and pull it out of the drawer figuratively speaking on if they become subject to such an attack compared with those companies who do not? Can you speak to? Your just your observations? Yeah, for sure the difference is,
so Dhruva is doing a couple of ransomware recoveries a week now. Right. So not the company, but you know, of our customers. That'd be, that'd be a rough company to work out. But. And what we are generally finding, again, is that so even again, earlier, I was saying that there's a difference between having a disaster recovery and having a ransomware recovery. So we have, you know, we try to work with our customers to help them understand they need to do they need a ransomware recovery playbook. Those Those that have the playbook, they're usually back up and running very quickly. Depending on the type of the attack, it may be, it's most likely going to be days, just because of the amount of research that has to go in to figure out how big the the attack has been, how how well, it has permeated the environment. That also tends to dictate how big the recovery is. Also, whether or not they use our cloud, we have the ability to restore in the cloud, or if they're wanting to just restore back in place. That by the way, one of the beauties of restoring in the cloud is you're guaranteed a pristine infrastructure. Right? That's one of the problems is you can't if you're restoring in place, you're you don't necessarily know if you're restoring to a pristine infrastructure. But yeah, we've had many, many successful restores, and no abject failures, of which I'm aware, the what it really comes down to, and success has been those who have, you know, how well they planned for the cyber aspects of this. Right, we make the restore part relatively easy. But, you know, if they, and we have a team that works with those who are in the midst of a ransomware attack, it's just it's part of the service that we offer. And the it's pretty, pretty evident once you start working with someone that they either prepared or didn't prepare for this right. So we can see that. That's just it. Right? It's the
it's more stressful.
Exactly, exactly. The thing is, you're either the question, Are you enacting a playbook? Or are you trying to figure out what you're going to do? Right? If you're doing the former, you're just going through the you're just going through the steps that you've already previously determined? If you're to give you a really bad example of how bad it can go the other way, would be the very large ransomware attack against Rackspace in the US. This was back in December, and Rackspace that it was it was a direct attack on their exchange infrastructure. And the I don't know, I don't know what's the more depressing part of the story, the fact that the attack used a vulnerability that was patched weeks earlier that if they just patch their exchange servers in a more proactive manner that the attack would have never happened, or the fact that based on what you can observe, it was very clear that they did not have they were not following a playbook. Right? They, they're sitting around a table and they're like, Okay, what are we going to do, and they chose to move everybody over to microsoft 365. This, they were using Hosted Exchange, they chose to move everybody over to microsoft 365. And they, as a result, this was before they had restored everything, right. And because they did that, now everybody's there on the 365. And they're sending and receiving email. That's great. But what about my old email? Right? A lot of people have some really valuable stuff in their old email. Well, they had to restore that entire environment. And then figure out how to get the data from A to B, right? I Mmm. And it was two weeks before they had a fully tested plan, right? Because they were notifying people as it was going on. They were saying, hey, you know, we've we've tested this, you know that we know where we're going to work. And now we're going to start restoring customer data so that you can download your data and get your emails back. It took them two weeks to develop a plan, because they didn't have one in advance. Right. Yeah, sure. So
I'm assuming I'm assuming having a plan also helps to influence the backup design, the philosophy behind it?
Yes, yes, it's going to, I think it's definitely the disaster recovery part of the plan, right. I think the one good thing that's come out of all of this mess, right, the ransomware. And the probability that which is starting to approach, you know, one to one, the probability that you will suffer a ransomware attack is pretty much at this point close to one to one is that everyone's talking about their Dr. Plans. But back in the day, when I was directly consulting with customers on how to, you know, Psych, you really to have a good Dr. Plan. And I was getting responses like, well, if you know, you know, like, we used to use 911, right? The the terrorist attacks on 911 as a, you know, way to justify look something like 911 could happen, and they'd say, one of two things, while I'm not in Manhattan, and the chances of me getting hit by attack like that pretty low. And by the way, if I was in those towers, I would be dead. And I wouldn't have to worry about it. And I'm not making light here. This is literally what customers would tell me. Yeah. So the so when you say does it impact the backup design? And the Dr. Design? Absolutely. And the good news is that we're having these conversations now and that people are, I think, taking their Dr. More seriously, because that whole thing of well, I probably won't be hit by planes. Well, you probably you will be hit by ransomware. It's no longer if but a win. Win. So yeah,
absolutely. Even Even COVID was useful in terms of thinking about business continuity plans, and so on. That's been a very useful catalyst. I mean, it was a pain in the backside in many, many ways.
Yeah, but unfortunately, fortunately, COVID also one of them. Yeah, unfortunately, COVID also made ransomware worse, because everybody was working from home and they're nice, you know, Gui, non infrastructure. State about their infrastructure, right? Yeah, a lot of people got ransomware when they were going to Starbucks sorry, I didn't mean to malign you know, that brand, but it doesn't matter it wherever you happen to be. Sure. And, and then they bring it into the, you know, you connect via VPN to your data centre, and now you're essentially on the land and then you know, boom, right. So, there we go. Yeah, absolutely.
Curtis, I think we've, I think we're coming up on time, I appreciate it very much that we've gone through and, and really covered from A to Z see a bilingual see what they did there. He said, I totally
expected to said there, I would have understood, I'm bilingual.
But is there anything else you'd have been before we sign off or anything else you'd like to leave the audience with just any final words of wisdom?
Well, you know, in addition to getting a backup a DVR system that is fully air gapped and Dr. Friendly, right? Not just a box of tapes or even in this case, a box a disc I would I would ask you to look at to look research, this idea of helping to prevent ransomware from doing everything I mentioned in the first half of the call, right? Looked to prevent lateral movement with your within your organisation to shut down things like already P and SMP when they are not required. And you know, to to my true dream would be that you put in some kind of a Oh, and by the way, you can also do things like there are there's an entire industry called Darnit it's DDI think it's DNS. Um, the acronym is leaving me. But there's an entire industry that helps you manage your DNS and IP infrastructure. And the what they can do is they can actually use machine learning and AI to recognise that there's a command and control request going on and just sort of sinkhole it or block it. So stop that. And then also, the real dream would be to have something that is watching your typical network traffic. And then again, using machine learning or AI to say, Hey, why is this server that has never spoken publicly before? Suddenly uploading data to Russia? Right, yep. That's, I think that's the beauty of AI and ML is being able to look for patterns, and then stop them when they start being abnormal.
Yeah, absolutely. Thank you very much. And look, I appreciate your time very much and for sharing your wisdom that you've gleaned over many years of being in the world of backup and helping clients through their disaster recovery plans, and we'll we'll stay on for a little bit longer, but I'm going to say thank you again and hit the stop record button. And, and then we can keep going but I really appreciate your time. Thank you, Curtis. Anytime