We are thrilled to be introducing a brand new podcast from the front lines of technology leadership today.
We are all dealing with data that is ever-growing in volume, velocity, and complexity; leaders across IT, Engineering, and Data stand alongside this growth, doing their best to keep up with the perpetual challenges of adopting innovative technologies, processes, and methods for managing data at scale. Most importantly, what is surfacing are the game-changing insights that have the potential to transform the entire business.
And while the top 1% of big data successes make the headlines, too many legendary stories of perseverance, innovation, and problem-solving in the world of enterprise data have gone untold - until now!
ChaosSearch is proud to announce Data Legends: Stories from the IT Trenches, a new podcast that focuses on the challenging, insightful, and often brilliant work of IT, Engineering, and Data leaders to drive digital transformation, maximize the value of their data, and stealthily shape the success of businesses around the world.
Here’s everything you need to know about our new podcast.
We’re Bringing You Stories from the IT Trenches
Each episode features a conversation with a Data Legend about their unique experiences, successes and failures, lessons learned, and battle scars earned along the way as they’ve navigated the rapidly changing IT and Engineering landscape.
Topics we’ll discuss on the podcast include:
- Site reliability engineering
- DevOps
- CloudOps
- Log analytics
- Data lakes, lake houses, data mesh, and other big data architectures
- Data democratization and literacy initiatives
- And more…
Hosted by our Chaosian Leadership Team
Our hosts include my colleagues CTO & Founder Thomas Hazel, VP of Customer Success Dave Armlin, and myself.
One of us will guide each episode, bringing a unique perspective while diving into informal, unfiltered, and fun conversations with esteemed guests. Tune in as we explore their most significant challenges and successes in the world of enterprise data.
Meet Our First Guest: Jeremy Foran, CTO, Purple Cow Internet
When it comes to building new products, there’s a fine line between which pieces of the puzzle should be owned by humans with deep domain knowledge and which aspects can or should be automated through AI.
How far can the boundary be pushed?
In our inaugural episode of Data Legends, Dave Armlin speaks with Jeremy Foran, Chief Technology Officer at Purple Cow Internet, about his new role as CTO at a fast-growing internet service provider.
Here are just a few of the topics that are discussed:
- The biggest challenges as a new CTO & how to evaluate new technologies
- Philosophies around the data lake
- Computer science and troubleshooting in the era of AI
- Determining where to introduce automation
A new CTO’s greatest challenges
Almost three months into his new role as a new CTO at the time of this interview, Jeremy opens up about his transition and challenges.
He explains how one of his recent realizations is that he can no longer “blame management” because he is management.
It is also intriguing to hear about his take on the human element of customer service. Sure, technology’s great, but it needs to solve human problems (certainly not create more, which we know can be the case).
When making decisions about how technology is used in the context of Purple Cow Internet, Jeremy emphasizes the importance of the customer experience.
Evaluating new technologies
Yes, the pandemic has changed quite a bit about how we all gather, share and discuss information and opportunities. In the rapidly evolving technology market, CTOs are constantly drinking from the firehose of new tech that may (or may not) be worth evaluating. How to determine which ones to take seriously?
In Jeremy’s case, his method of meeting people online over the recent years has served him well. It’s his network that helps him keep his finger on the pulse of new technology trends that are worth following, with compelling business cases and success stories.
There are also the front-runners who are constantly innovating. The leaders in the tech world – brands such as Amazon – are generally worth following. System and platform providers often also share useful information about challenges and solutions.
Events and recorded conversations can also prove meaningful. Keynote speeches, virtual and in-person summits, trade shows, expos, podcasts, YouTube channels, panel discussions — you name it. For Jeremy, these resources have been a gold mine of information and help to surface new innovative approaches worth investigating further.
Philosophies on activating the data lake
There is a wide range of approaches and schools of thought surrounding data lakes.
At the heart of those varying perspectives lies the sometimes painstaking decision of what data to keep and what to throw away.
As we saw with the advent of Big Data, what you need today may change tomorrow. If you don’t collect all you can today, there’s no telling how it might disadvantage your business further down the line.
Data lakes (especially those that embrace progressive new approaches with data compression) provide us all with the luxury of hindsight which, as we know, is 20/20.
The bottom line is that data lakes have removed the need to compromise on the storage of business data.
“We're 10 years down the road… Does everything start in a data lake? Does that become the first place it lands? Or is that a piece of the development lifecycle?” – Jeremy Foran
Computer science and troubleshooting in the era of AI
We’re already part of a world where AI-driven insights are becoming the norm. You may not always notice it, but you’re probably using tools that use it, at the very least.
Jeremy, without overcommitting to timelines, shared some of the things that his team is exploring, like sentiment analysis or rules-based decision-making, where machine learning could play a much greater role in the long term operation.
Troubleshooting
As Jeremy says, “Effective troubleshooting is about systematically removing assumptions.”
More often than not, really solving a problem requires getting to the heart of it to really understand its mechanics and inner workings. That’s why you need that troubleshooting protocol.
Work through a process of elimination to determine what isn’t the issue, in order to find out what the source of the issue really is.
“Many times when there's a problem that a team is trying to address, it's by overlooking something by just assuming it's not contributing to the issue.” — Jeremy Foran
Determining where to introduce automation
Like with many other decisions, this one should be data-driven too. Your data needs vary across the length and breadth of your business.
Sometimes you require real-time views of your data, such as the current value of insurance policies being underwritten within a major insurer, to manage cash flows and liquidity.
Other times, you can zoom out a little and focus on historical data and draw comparisons.
Jeremy also discusses introducing automation. This is particularly important, he says, when you need different systems to speak to each other for your operation to run smoothly, and when all the tasks must be completed by different people on your team servicing specific customers in synchronous fashion.
It’s not just about functionality, either. It’s about scaling the functionality. That’s really when you want to consider automation.
“When you're in computer science, so much of how you think about problems is doing it right. How do we build this in a way in which we're not going to be chasing ghosts in 6 months…” — Jeremy Foran
Key takeaways from Episode 1:
- Aim to solve human problems when you consider using technology.
- Build a reliable network, follow your service providers online and explore events and virtual gatherings where you can tap into a constantly-evolving knowledge base.
- Your data needs can change.
- Troubleshooting is about systematically removing assumptions.
- Automation can help you scale.
Find out more about Jeremy and Purple Cow here.
Tune In
Data Legends is streaming on Apple Podcasts, Spotify, and Google Podcasts.
You can visit the official podcast website at https://datalegendspodcast.com/ to check the latest news and updates, access past episodes, learn more about upcoming guests, and subscribe to the show, so you never miss an episode.
Additional Resources
Read the Blog: Inside DataOps: 3 Ways DevOps Analytics Can Create Better Products
Watch the Webinar: Advanced Analytics - Estimating the Total Costs of Your Cloud Analytics Platform
Check out the Guide: 2022 Gartner® Market Guide for Analytics Query Accelerators
