--- title: The Threat Hunter's Mindset Webinar | ChaosSearch description: Learn the approach and analytical frameworks that top threat hunters use to find and neutralize hidden threats. --- Revinate leaves their ELK stack behind to find huge gains with ChaosSearch -- Read More! [Revinate leaves their ELK stack behind to find huge gains with ChaosSearch -- Read More! ](https://www.chaossearch.io/resources/customer-stories/revinate) [![ChaosSearch](https://www.chaossearch.io/hubfs/2021%20Website/logo.svg) ](https://www.chaossearch.io/) ![Gartner Cool Vendor 2023](https://www.chaossearch.io/hubfs/C2020/Logos/Gartner%20Cool%20Vendor%202023.png) [![Start Free Trial](https://no-cache.hubspot.com/cta/default/4020721/a1eeee12-14a2-41d7-a38b-11e32980b916.png)](https://cta-redirect.hubspot.com/cta/redirect/4020721/a1eeee12-14a2-41d7-a38b-11e32980b916) # The Threat Hunter's Mindset ## Learn the approach and analytical frameworks that top threat hunters use to find and neutralize hidden threats. Threat Hunting begins where most cybersecurity defense tools and techniques end: once the perimeter has been compromised. Advanced persistent threats (APTs), and the malicious actors that run them, take a “slow and low” approach, seeking to linger beneath the radar within your network, collecting intelligence without making waves, and covering their tracks along the way. But even the best hackers leave clues.   In this webinar we show: - How to adopt a Hunter's mindset, and the analytical framework that the best Threat Hunters use. - How understanding the 6 common stages of an advanced persistent attack is fundamental to tracking down your adversaries. - A breakdown of the recent SolarWinds compromise, focusing on the evidence that Threat Hunters can look for to hunt down and neutralize similar attacks. - An interactive workshop demonstrating how Threat Hunters use log analytics to conduct their hunts.     ## Interested in scheduling a brief intro call to see how ChaosSearch can accelerate your analytics? Yes - show me the calendar! ## Future-Proof Your Analytics at Scale [![Get a Demo](https://no-cache.hubspot.com/cta/default/4020721/0ae4b1b0-9ec0-4169-80cc-64fda5fd56db.png)](https://cta-redirect.hubspot.com/cta/redirect/4020721/0ae4b1b0-9ec0-4169-80cc-64fda5fd56db) ©2024, ChaosSearch®, Inc. [Legal](https://www.chaossearch.io/legal) Elasticsearch, Logstash, and Kibana are trademarks of Elasticsearch B.V., registered in the U.S. and in other countries. Elasticsearch B.V. and ChaosSearch®, Inc., are not affiliated. Equifax is a registered trademark of Equifax, Inc. Contact Us Phone: [(800) 216-0202](tel:+8002160202) Email: [teamchaos@chaossearch.io](mailto:teamchaos@chaossearch.io) Follow Us - - - - ```json { "@context" : "https://schema.org", "@type" : "VideoObject", "description" : "The Threat Hunter's Mindset", "name" : "The Threat Hunter's Mindset", "thumbnailUrl" : "https://i.vimeocdn.com/video/1093658433_100x75.jpg?r=pad", "transcript" : "TOM GRAVE: Good morning, everyone, and welcome to today's webinar, \"The Threat Hunter's Mindset,\" with ChaosSearch and ESG. I'm Tom Grave with ChaosSearch. I'll be the moderator today. Thanks so much for joining. We've got a real treat for you today with two experts in their field, and we're really excited about an interesting and interactive discussion around this important topic of threat hunting. Now, before we start with introductions and before diving in, I just want to remind everybody a few logistics. We are taking Q&A throughout the discussion. You can enter your questions in the Q&A panel there on the right. And at the end of the presentation, we'll be doing a live Q&A with the webinar attendees. We're also providing the slides and the recording for everybody that registered, so no need to furiously scribble notes or take screenshots because you're getting all the great content we're presenting here today. So without further ado, let's introduce the speakers. Jon, welcome. Thanks so much for joining us. Please introduce yourself for the audience. JON OLTSIK: Thank you, Tom. My name is Jon Oltsik. I'm a senior principal analyst and fellow at ESG. And I cover security operations, which is sort of the intersection between analytics, data, and actually doing something about that data. So nice to be here. Thanks for having me. THOMAS HAZEL: My name is Thomas Hazel. I'm the founder CTO here at ChaosSearch. And we're creating a new type of data platform that really changes the game with respect to large-scale data analysis, particularly in the SecOps space and the threat hunting analysis. TOM GRAVE: OK, thank you. I'll cover the agenda for today, starting with number one, in which Jon will explain the situational analysis, really focusing on the growing need for threat hunting in the market. Jon will also cover the threat hunting requirements that are out there, as he sees it. And that'll tee up a quick overview of ChaosSearch and the ChaosSearch data platform that Thomas will cover, specifically explaining the role that ChaosSearch can play in the overall methodology and framework of threat hunting. And then we'll get into the meat of the conversation, which is bullet number 4 here, which is adopting the threat hunter's mindset, really understanding the attack chain, the attackers' methodologies, and what the threat hunter needs to do to find and intercept an attack in progress. And we'll end with the SolarWinds example, which is a great case study from last year of a very sophisticated attack. But the interesting thing, from our point of view, is there are clues that the threat hunter can look for in similar attacks and prevent the next SolarWinds from happening, and that's what we'll cover. Finally, as I mentioned, we'll end with Q&A, so please do submit your questions along the way. And without further ado, I'll turn it back over to Jon. JON OLTSIK: Well, let me take this one, Tom and Thomas. So why do we need to adopt a threat hunter's mentality? As this slide indicates, because all of the security metrics are headed in the wrong direction. Now, I won't spend a lot of time here because I think this is patently obvious to the audience. But we've seen growth in the number of breaches, growth in the number of businesses that have been impacted. And I think that's a lot of awareness, where in the past, they may not have been aware or they may not have thought that they were a target. But now they do. The average cost of a breach continuously goes up. There's an average cost of a breach report that's done every year. I think, to me, a lot of what has to do with this is the growth in industrial ransomware that's impacting more companies. And then, Thomas, 207 days to the average time to identify a breach. I've been in security for about 20 years, and this has never gone down. So is that what you're seeing? THOMAS HAZEL: Well, absolutely. And one of the reasons why we created the solution that we did was that retention, that ability to analyze all your data is key to resolving any of these issues that we dread happen now and in the future. JON OLTSIK: Yeah, it is all about the data, and we need to have at least 207 days of data, that indicates. So there is growing interest and investment in threat hunting, really driven by the things we just talked about. Now, 82% of IT teams want to take a more proactive approach to finding and mitigating attacks. Again, it's because what we're doing-- waiting around to get an alert, triage that alert, or depending on things like signatures or heuristics, even machine learning-- that is reactive. We're getting those signals. Then we have to figure out what's gone wrong. We want to do that more proactively. And that's what threat hunting does, as you see. Threat hunting's a proactive cyber defense activity that seeks to block or mitigate threats that did not generate security alerts. So we're talking about known unknowns and unknown unknowns, and we're facing very sophisticated adversaries who know how to circumvent our defenses. So the more proactive we can be, the smarter we are about understanding what they may try to do, the better. And that's where we can find and eliminate threats before the attack. We can get in actually really early on the surveillance side and then reduce the time when a breach occurs and when it is discovered. So again, we may be finding things, finding clues, early on. And if nothing else, we can keep our eye on them. So now, how does threat hunting align with security operations goals? Well, as it turns out, it is very tightly aligned. And this is research that we did recently. So the question was, what are your top security operations objectives? So improve the detection of advanced threats-- and the key there is \"advanced threats.\" This is where people struggle. And this is where we have some hope for advanced analytics, machine learning, AI. But the more we know, the more that we can dig into the data, the better. Improve the meantime to respond-- so if we understand the threat early, we can respond early. We can fine-tune our controls. Get a better visibility of cyber risk-- we may believe we're protected. But when we do some threat hunting, we may find some gaps in our defenses. And if we do, we have the opportunity to address those gaps. And then you can see highlighted is \"improve the detection of anomalous user behavior and insider threats.\" So a threat hunter starts to get a very good understanding of what should be normal behavior. And they can communicate that to other people, so we have templates for threat hunting. And then insider threats, that is an insidious threat. That is where we can lose a lot of money very quickly because insiders know what to look for. And when they look for it, they may have access. So Thomas, this is where understanding what privileges users have, what they normally do, kind of how they look around the network. And then also, what's really critical? What are the critical systems? What are the critical data? It's really important. THOMAS HAZEL: Yeah, no, I love it. And I typically identify what you need to know, the ability to find the problems after or during when they're happening, and then recognized in the future that it may happen again and have some pattern recognition associated with it. So no, this is great information. JON OLTSIK: Yeah, you're right. It's all about pattern matching. Now, we're here to talk about threat hunting. But threat hunting is really driven by, as we talked about, the collection, processing, and analysis of massive amounts of data. And I don't use that word lightly, \"massive.\" This question is from some recent research we did on security operations. So how has the amount of data your organization collects to support its information security activity changed in the last two years? And you can see that 76% of those that we surveyed collect more data than they did in the past in the past two years. And almost a third say, we collect substantially more data to support cybersecurity operations, or analytics and operations. Is that what you're seeing, Thomas? THOMAS HAZEL: Well, one of the primary reasons why we created this company, ChaosSearch was this data growth. I think it is said by 2025, 175 zettabytes of data will be generated. If you Google what a zettabyte is, it's just off the charts. And a lot of it's this machine-generated data, this log data. So one of these key aspects was, how do you collect it? How do you manage it to ultimately solve your problems of thread hunting, and then protecting your company, your business moving forward. So not only is it growing. It's exponentially growing. So yeah, no, we see it here at ChaosSearch quite a bit. JON OLTSIK: More data, more machines, more applications, more transient-- transientness, if that's a word. But we have got remote users. We've got transient workloads in the cloud. We need to collect all that data. THOMAS HAZEL: And it seems like every day you have add to that list. So it's always this new thing, new device, et cetera. JON OLTSIK: Yeah, and not only are we collecting the data, but we are retaining the data for longer periods of time. And now, based on what we talked about, Thomas, that's good. But 52% said that they were retaining security data for longer periods of time than in the past. And interestingly, 28% said, no, but we'd like to. Well, why don't they do that? Because it's costly, because the more data you have, it may impact your query performance. So if we can solve those problems-- and I know that ChaosSearch is intending to solve those problems-- then we can retain more data, which is key to security, Thomas. THOMAS HAZEL: Yeah, and what we see is that when you know what you know, it's typically in the first hour, day, maybe even a week. But to figure out what happened is that retention, is the pre-analysis of what is going on, so that ultimately you can get that real-time reporting and alerting when you're moving forward. So there was a real limit to the cost and complexity for that scale. And if you're dealing with terabytes and up to petabytes a day of data, retention is almost not even a choice. It's too costly, too expensive. So again, as we get into what ChaosSearch is offering, we really ran at not only do we provide that real-time value to detect something is going on, but that long tail, that data retention, so you can do the postmortem of, when did they try? What was happening? Because if you don't have that, you can't actually have the future protection. JON OLTSIK: Yeah, and what I find, Thomas, is the more data you have, the longer you retain it, the smarter you get about that data because now you're poking at patterns. You understand the long tail history for retrospective investigations. And so you can fine-tune what data you actually collect. Speaking of the data pipeline, in this question, we were asking, what new benefits or net new investments would you like to make in data pipelining? So again, data pipelining is all about collecting, processing, and analyzing the data. And we can add retaining that. So 40% said building and improving a security data pipeline for keeping up with real-time data sources. This is stream processing. Things are happening instantaneously. The quicker we can analyze that data post-real-time, the faster we can detect a problem. And if we detect those problems quickly, we can address them quickly. That's the whole threat hunting mantra. 39%, adding more comprehensive analytics-- so people are really good at this. Security analysts are really good at this. But they do want that help. And machines can crunch the data and do things that people can't. Collecting and centralizing security data for more controls and sources-- Thomas, this is really apropos of what we just talked about, more data sources, more machine data. And then you see on the bottom, building and improving a security data lake for historical queries. This is the playground for threat hunters. Dump the data into a security data lake and then give me, the threat hunter, the ability to query it, to build dashboards, to really hone my processes. And then I can be much more productive. THOMAS HAZEL: This idea of this data lake, where maybe this one feed, you see that something's going on, maybe an IP address. But without testing this IP address across multiple data sources, you really can't put together that complete story. And these siloed islands of solutions is really a hindrance to solving the problem. And so from our viewpoint, a data lake philosophy is key to this threat hunting dilemma, if you will. And for our viewpoint, the great thing about a lake is data pipelines so easily stream into that central manager viewpoint. And again, we'll get into ChaosSearch, where we take those data streams, that data lake, and provide access, those insights they're looking for, both from a real-time perspective as well as historical. JON OLTSIK: So before I hand it off to Thomas, what are the requirements for threat hunting? So expertise and resources-- now, there are some guidelines. So for instance, the MITRE ATT&CK framework is a great framework for threat hunting. It gives you perspectives of tactics, techniques, and procedures. It gives you perspectives on threat actors. And it allows you to look across a kill chain and understand what a threat actor might do and whether you're covered. And the methodologies, we've got a lot of good threat hunting courses. SANS comes to mind, but there are others as well. But we do need what ChaosSearch provides, and that is a massive data repository with a wide variety of security telemetry and network logs and then long retention periods, so that we have that, like I said, that playground or that template for threat hunting, where the threat hunters can not only look at all the data, but learn how to professionalize or get their expertise around that data and their processes. And then in-depth detection and analysis capabilities-- so I'm talking about analytics, but also query and high performance query performance, dashboarding, templates, reports, anything that can really organize that much data and give me, as a threat hunter, those kinds of clues, that advantage to analysis so I can move on to the outcomes, to remediation, to fine-tuning my controls, things like that, Thomas. THOMAS HAZEL: This is why we've created ChaosSearch, was those requirements are hitting limits with the existing solutions, whether it's cost, complexity, and particularly time, where time is now essential. If you have an open door for an hour-- obviously, weeks and months-- you have a real hole in your organization. And where does it start? Where does it begin? The data. The choice to provide security is a choice of cost. You have a real problem. And so this is where ChaosSearch has come in. So what I'd like to go over what we've been building at ChaosSearch, and the capabilities and the value, particularly in this SecOps viewpoint, where more data, faster analysis, the playground that you mentioned-- I love that saying, because, in essence, the threat hunter has to have a playground, because, sure, if you know what you're looking for, that's great. But a lot of the time, most of the time, you need to look and ponder and look around the corner. Only the ability to play in that environment is you're going to find those problems. So no, I love how you say that. So at ChaosSearch, we really made the next move in data analytics. We based purely on cloud object storage. Why would we do that? Wonderfully elastic, no provisioning, no complexity of provisioning your storage. Secure-- I mean, fantastically secure. The policies controls on cloud object storage like Amazon S3 is absolutely amazing. Reliable-- cloud storage, when you have other siloed solutions, this is where you back stuff up. What if you didn't have to move it out of that data lake? You have to do the analysis right there. And again, the availability is just wonderful. So what we've done is said the user, the customer, the developer, the dev ops person, stream your log data to your cloud storage. Our servers, our Chaos service, will connect to your cloud storage. You give us read-only access to the data. And we provide analysis of that data, whether it's syslog, to Cloudflare data, to flow logs. Any type of data, we provide analysis automatically for you. So unlike traditionally, where if I have to spin up a new Elasticsearch cluster or I need to move the data to a Splunk server, et cetera, you don't have to do that with us. Just stream your data to the cloud object storage, and we take over from there. We discover what's in your cloud object storage. We'll automatically index it so that we can make fast insights quickly and efficiently. But unlike traditional systems, where you maybe have this siloed solution for your hunting and this siloed solution for your analytics, we brought all the capability to one unified platform-- Elasticsearch APIs, SQL APIs, ML APIs-- so that you can do all those use cases under one unified offering. Now, here's the key thing-- time to results. You can stream within minutes and providing analysis at scale within seconds without breaking the bank. And so the idea with ChaosSearch is you have that playground to hunt, to search, to organize, to manage. But then from there, we look like an Elastics Stack ELK offering, where you can set up dashboards, visualizations, alerting, just like you would with a traditional logging solution. But all that pain to get that into the Elastic Stack goes away. And again, we talked about retention. Almost every one of our customers store data for maybe a week if they can afford it. Particularly when it goes terabytes or petabytes and more, store it for a month. Store it for a year. You want those real-time alerting. But why did it happen weeks, months in the past? And that's a key thing. What have you seen the limitations with respect to managing that scale and the offerings that are out there? I know you know this, but data scale is the bane of threat hunters' existence. JON OLTSIK: Yeah, we're seeing massive migration to the cloud for scale. And what that encourages people to do is exactly what you're saying. It's dump all of the data into a common data lake and retain it for longer periods of time. So you're absolutely in sync with what we're seeing. THOMAS HAZEL: And it's funny. With our customers, we say, you're storing into your cloud storage today, and then you're also storing into some siloed solution. Just stop sending it out elsewhere. And again, this infinite ability to store at such a cost-effective price-- we provide 80% reduction in cost for one day's ingestion with unlimited retention. Now, when you have unlimited retention, you think of the problem completely different, where our competitors, that retention adds another vector of cost. And again, something to consider when you're looking at solutions. But we are a log analytic platform. So a common scenario is, how many logins did a device or service have? Were they failing? How do I quickly look at that? That hunt, that ability to hunt-- why didn't Elasticsearch take off in this environment? It's because you don't know what you're looking for and you want to search around to find things to ultimately provide procedures for learning and dashboards. So when you know a ChaosSearch solution, you know that you are a logging vendor without any of that pain we just talked about, that data pipelining, that ETLing, that data management. We would do that automatically for you, but we publish standard Kibana visualization. It's a well-known visualization platform to do discovery, visualization dashboards, and alerting and monitoring, the whole stack, which, again, is so key and this thread hunting movement. I know everyone has their favorite tools. Jon, I'm sure you have your tools that you think are the best. From our viewpoint, we're just going to publish open APIs so that you can use your existing tooling the way that you know and love to ultimately create dashboards. Once you know what you think you should be looking for, set up a dashboard. Set those alerting so that if this intrusion happens again, you're safe. You're in charge. Love to hear what you're hearing out in the community with respect to the tooling that they know and love, because we'll add that to our stack, because, to us, we want to solve the scale, the cost, the pain, and integrate with those toolings out there. JON OLTSIK: Yeah, Tom, it's sort of a tale of two cities. We have the people who are consolidating tools, but really large organizations, they have one of everything. And you have some center of excellence around a particular tool. So it's all about integration. And we created an architecture called the Security Operations and Analytics Platform Architecture, or SOAPA, to deal with this. And that's all about, as you said, API integration, data-level integration. One other quick point is security people aren't data engineers. They're there to prevent, detect, and respond to incidents. And so the more you can eliminate that data pipelining, that data engineering, the better. THOMAS HAZEL: Actually, that's a really great point. And I want to double down on that. Because of our platform, where we automate all that data movement, that data management, we provide what I call this data refinery, where the threat hunter can actually create their own views and lenses without having to call the data engineer, the DBA to set it up. They can play, to your point, different ways to see the data. Maybe I have five feeds that I'm looking for something. Well, if I want to add in, again, another data source, I can just go click, click, and go and add another view that increases that data source to hunt with that particular IP address or that particular string. Now the analysis person can actually self-serve through our platform. And again, to your point, if you have to set up a new data pipeline, it could take weeks and months, particularly at scale. Again, the key thing for us is to do that instantaneously, again, to solve your problem. Because you can imagine, when you realize you have a problem, now is time, right? So I thought we'd give a-- let's call it a real world/hypothetical example of what one might do from a mindset of the hacker. It's because if you think about what they're doing, you may come up with procedures and processes to address it. So a little fun, a little tongue in cheek, but we're more like the CIA, where you have to prevent the future crime, versus the FBI, where it's solving the past crime. We have to do both. But the key thing for us is that there's something that's happening. You have to address it now. It's that instantaneous. And you have to protect against that in the future. We all know this. Once you have a security breach, it is the most critical thing in the corporation at that moment. Any thoughts, Jon? JON OLTSIK: Yeah, what you're talking about is true. I equate it to the difference between maybe buying a weapon to protect yourself when someone breaks in, to surveillance cameras, intelligence gathering, things that you would look at to see if there's a crime that's in progress or in its formation. THOMAS HAZEL: Absolutely. And so I'm going to steal something from you, Jon. I know that this is actually someone else's saying. But if you know your enemy and you know yourself, you need not fear the results of 100 battles. I know you've quoted this before. This is so true. Knowing your enemy protects you. If you're in sports analogies, you try to in practice play like the opponent in the future, so you can learn and try to prevent what's going on. But we can't always do everything. But again, the key thing is to know what they might do. And this goes with data collection, data analysis. I'd love to get your viewpoints on these six steps, Jon, and what you think a classic mindset would be. JON OLTSIK: Yeah, these are the steps of the kill chain. It's used in the MITRE ATT&CK framework, but it originally came from one of the systems integrators, federal systems integrators. But your point is great, Thomas, is that we should anticipate what a threat actor will do at each of these steps, and then we can understand what to look for. And we can also understand how to prevent or detect these events much more thoroughly. So that's really the mindset of a threat hunter, is to really understand-- like the Sun Tzu quote says, put yourself in the shoes of the enemy. How would you attack your organization? And then that will give you a better idea of how to defend your organization. THOMAS HAZEL: I love it. And part of this mindset is, what data sources do you need to collect now? And that's a key thing, where a lot of times, you're like, well, that's going to take time to set up. Just start storing into your lake. If you identify what they would do, identify those sources, start saving them away, particularly with the ChaosSearch solution. Because if you don't store them, you cannot figure out what's going to go wrong. And so let's test this hypothesis. Continually assess the landscape. We know that you're not going be able to know everything all the time. But as we mentioned, think about what the attacker would do. How do they cover up? What type of machines would they want to break into? What kind of services they want to break into? And if you think like that hacker, you're going to identify, at least from my viewpoint, the data sources that you would need to monitor and ultimately alert on to make sure that whatever they're doing or trying to do is actively addressed at the moment. But again, that long-term data tail analysis is so key to this postmortem. But again, the hypothesis to us is those six steps. Love to get your thoughts, Jon. JON OLTSIK: Absolutely, Thomas. And again, I'll reference the MITRE ATT&CK framework. It gives you a taxonomy of the tactics and techniques used by adversaries. You can map them to your controls. And if you do so, then you know what data you need to gather and analyze, and you can start to imagine what dashboards you need to actually detect those steps along the way. THOMAS HAZEL: And the idea that there might be different tools for different situations-- and this is where ChaosSearch really comes in, is that when you get that data into your lake, we're going to provide you the tools, the API access, the visualizations, the machine learning modeling that is so crucial. But again, it begins with collecting and then ultimately providing controls and procedures around it. And so how do we map that to a ChaosSearch scenario? So the idea is that sysmon events are coming to your environment. Typically, our customers stream that data to cloud object storage. We take over. We [? auto ?] index that. We manage over time. So they're not worried about, oh, shoot, will my database fall over as things scale? But from there, they set up Kibana visualization to show, what is the connectivity of the sysmon information? There's so much good bits of information from there. And then from there, set up views that, hey, if somebody's trying to not only do connectivity, but trying to log into something, try to figure out, can I try this? Can I try this? Again, those are clear signs that something is going on. And we'll have a future next slide examples of where, when you see a little bit of an anomaly within the data, something might be there to look at. And you can imagine, there is so much data where if you see a log spiking just in size alone, maybe something's going on, something to look at. And then again, this unusual behavior and activity of systems-- particularly logins, network access, failures-- these are all things that in this playground, Jon, that you mentioned is so key to build up these toolings, these dashboards to, really, for every data source in the future, provide some type of analysis that you can alert on or at least visualize in the dashboard. And your thoughts, Jon. JON OLTSIK: One quick point, Thomas, is I'm really happy that the interface is Kibana, because there's an acute skill shortage out there. So if I have a new tool and I have to learn a new interface, that's going to take away from my ability to detect and prevent threats. And the fact that Kibana is already out there-- it's well-established, and the security community knows it, that's goodness. THOMAS HAZEL: Kibana is so easy to use. At the same time, it's wonderfully really powerful tooling. It's really a great tool. And when you're in the thick of it, when you're trying to hunt, it's a fantastic discovery tool. And then once you have those discoveries, set up your visualization. Set up your dashboard. Set up your learning is really a few steps. And particularly when you do that scale, you can imagine all these different data sources, where our solution allows you [? to pay ?] these views to attach that Kibana functionality, that tooling, in a really unique way, really good stuff. And so here's a good example, where if you're looking at a log, maybe it looks OK from a distance. But if you look closer, you'll see that there's something going on at a little detail. You can set up thresholds with Kibana to say, hey, if this spikes against this other metric, what's going on? And these are great signs, where post-analysis, you can find it, where you can select this and select that and do some different filters. But from that, you can actually create visualizations and triggers and alerting to say, something's happening, and jump right in. If you know Cloudflare logged, denial of service attack, it's common where you don't know where it's going to come from, but when you get that alert, you quickly can jump into the Cloudflare data, see what IP addresses are doing it, and turn it off. This is a simplistic diagram, but it's so crucial to what these day-to-day threat hunters have to do, is analyze the data. This playground, Jon, that you mentioned is so crucial to figure things out, because when you're in heat of the battle, if you have to wait for IT to set something up or maybe transfer some data to get into a view that you can analyze, time is of the essence. JON OLTSIK: Oh, definitely, Thomas. And what I like here is there's a mindset that threat hunting is too hard. This is an example where you can use simple visualizations to learn patterns and then improve your dashboards over time. So it is hard. It does require a lot of experience. But that doesn't mean that you can't gain benefits by starting with the basics, learning what to do, looking at visualizations, and then progressing. THOMAS HAZEL: So we know-- I hope everyone knows, who cares about security-- the SolarWinds attack last year. It's highly publicized. And it's really a template of what can go wrong and what you should do to not have it happen and how to resolve it moving forward. There's a lot of information on the web about this particular topic. But I'd love to have, Jon, you talk about the left side, and we can talk about the right side on how we do a log analysis. JON OLTSIK: Sure. So let me just start with the bullets at the top real quick. I say \"SolarWinds and beyond\" because this was a successful attack. And I think we'll see copycat attacks. And my fear is we'll see them to attack industry sectors like health care. So we all should be paying attention. The Orion revisions were impacted, only some Orion revisions. But they were impacted, so you should-- by now, I'm sure SolarWinds customers are into this, but others should pay attention to what happened here. There were multiple threats. And I think this cascades into third-party risk management, vendor risk management, things like that. So Thomas, those are more data points that we'll have to analyze. Interesting, so there were 29 IoCs published. And you can see where they were compromised. But there were-- actually, it started with a smaller number, and open source context blew that out by doing extrapolations. So for instance, if a .com domain was used as a command and control server, open source context looked at the .net, similar, the exact same domain, only with a .net or a .org or things like that, just to understand the nature of the attack. And then there was a DLL that was exposed. And this is particularly relevant to what we're talking about for threat hunting. That is an IoC that we should be looking for if we're SolarWinds customers. THOMAS HAZEL: And they were so sophisticated about this particular attack, where when they were sending out information, they modeled the information they were sending out in a SolarWinds-type style. They were very sophisticated of how they got in, how they expanded, but ultimately how they were sending data so that they could get the information. JON OLTSIK: Yeah, this was a nation state attack, and it was very sophisticated, as you say. But that doesn't mean that we can't do some threat hunting to find things that are indicators. So for lateral movement, as you know, that's part of the kill chain. Once I establish a beachhead, I'm going to move laterally around the network to look for things of value that I-- in a target attack, look for specific data that I want to steal. So how does that equate to what you do at ChaosSearch for threat? THOMAS HAZEL: Well, A great example about this HTTP data, well, the great thing about a log analytic tool like ours is, say, OK, what should the data look like? Hide that. Turn that off. So actually, I can't find what I'm looking for, but what should I not look for? To hone down that data, say, wait a second, I turned off all the information that I should be sending. What is left? And the ability to do that inverted analysis and then analyze the data across all those sets-- so for me, every one of those relationships across all those systems, what should be happening? And then turn that off and see what was happening. And that might be the key to the kingdom. JON OLTSIK: Yeah, and now, for execution, as you know-- excuse me, the escalation-- the attacker used a temp file to remotely execute utilities or a remote shell, very common. They similarly looked at tasks and updated existing legitimate tasks. So once again, to mimic the behavior of a privileged user or someone who has access. What can you do there? THOMAS HAZEL: No, it's funny. The anomaly detection, that's why it's so key, is that if you know your happy pattern, where the system should be behaving, and then when your logs, your data goes against that pattern, you should be setting up alerts saying, maybe there's nothing wrong here. But you should be alerted to just go look. And so the idea that you know the flow, the type, the nature of this information, and when it goes off that pattern, you should be alerted and go looking at that data. Because, again, it may be something that's a spike in just the network or usage, or something has changed. But it's a starting point of discovery to be alerted to ultimately discover. And that's why that playground is so key, is because you know something might be going on. Let me go look around. And you may discover everything's fine. But that trigger point of this common behavior to this uncommon behavior is the point to do the investigation. I guess without further ado, Jon, really, thank you for taking the time here to talk about what you see in the market, your expertise in the market. Here at ChaosSearch, we're doing some very unique work in scaling your data platform needs, and particularly the SecOps, and security we all know is so important today. So I guess we'll get into some question answers for the crew here to really ask Jon or myself any aspects of the threat hunter's mindset. ", "uploadDate" : "2021-03-25T17:21:48.000-04:00" } ```