The importance of Cloud Performance and Security Platforms
Work, education, and even many of our leisure activities have all moved on-line at an incredible pace due to current social distancing mandates. The digital backbone of the Internet and the SaaS services that drive our personal and professional lives are now foundational. Ensuring that these systems are operating optimally and securely is of paramount importance.
As each person on the planet consumes more and more bandwidth, and we are faced with new threats in this time of crisis, content delivery networks and security platforms are increasingly critical.
Netflix has had to dial down video quality in Europe due to bandwidth issues, organizations are threatened by Covid-related phishing schemes, organizations are reeling from Zoom bombings, and the world is dealing with nation state activity like the recent Russian telco Rostelecom BGP hijacking. These examples all underscore the breadth of technical challenges occurring amidst the already challenging covid-19 crisis.
Cloudflare has been continually adding to their core web application performance and security features, most recently providing safe DNS for families in its 18.104.22.168 mobile app and service. Cloudflare has a long history of being able to handle scale dealing with the largest DDOS attacks in the history of the Internet. To borrow Cloudflare’s recent messaging, they are built for this.
ChaosSearch is also built for scale, and we are happy to have Cloudflare Enterprise customers who are ingesting TB’s of Cloudflare log data daily to securely harvest actionable insights, with all the data residing in their own Amazon S3 environments.
Challenges at Scale
Dealing with daily data ingest rates of terabyte and greater scale via other log analytics platforms is extremely challenging for a number of reasons. From sizing your cluster, outlining storage requirements, estimating the number of shards and nodes, to dealing with failures, the care and feeding of an ELK stack (self-hosted or hosted) or other search platforms, can be daunting. If you are utilizing other hosted solutions, you may be offloading some of the administrative tasks, but costs and your ability to keep more than a few days of data can quickly become a problem. ChaosSearch, due to its underlying indexing technology, can index TBs or PBs of data and create an index that is a tiny fraction of the size of the source data, while providing full text search and analytics on this index. Once the index is created, there is no need for the source data to be maintained, and you can delete it or push it off to Glacier for longer term storage. ChaosSearch is a fully managed service requiring zero administration and providing the ability to index data at scales that typically choke other solutions.
Cloudflare and ChaosSearch together
Cloudflare has built-in DDoS protection to help fire-fight attacks and provide actionable insights. Adjustments to your Cloudflare web application firewall add more power to your security capabilities.
With ChaosSearch indexing your Cloudflare logs, you will be able to:
- Monitor, detect, and alert on suspicious activity or patterns.
- Quickly view and identify top IP’s, allowing you neutralize attacks as they happen.
- Monitor and set performance thresholds to ensure that end users and customers have the best customer experience possible.
- Leverage powerful full text search with automatic indexing of all fields, providing the ability to “find a needle in a haystack” at TB or PB scale.
- Integrate monitoring and alerting with other systems like Slack, PagerDuty, Jira, OpsGenie, ServiceNow via webhooks, allowing you to proactively bring the additive power of these systems and automate workflow based upon monitors that you configure in ChaosSearch.
See all of this in action…
My colleagues Tom O’Connell and Kevin Davis demonstrated all of this power of ChaosSearch and Cloudflare this week. You can replay the webcast recording here.
About the Author
Dave Armlin is VP of Customer Success and Solutions Architecture at ChaosSearch. Dave has spent his 25+ year career building, deploying, and evangelizing Enterprise and Cloud based solutions for companies globally and could not be more excited about the impact ChaosSearch can have on organizations around the globe today.