Amazon Web Services now offers so many data warehousing and analytics services that understanding the differences between them, let alone deciding which service to use for which use case, can be a daunting task.
Consider, for instance, AWS Redshift and AWS Athena. Both services offer overlapping functionality, in that they can be used to analyze data at scale. But ultimately, Redshift and Athena cater to distinct use cases.
Keep reading for a look at what makes Redshift and Athena similar and different, along with tips on how to decide which use cases align with which type of service.
What is AWS Redshift?
AWS Redshift is a data warehousing service hosted in the Amazon cloud. It's built on PostgreSQL, an open source database, but it's designed to offer massive scalability that would be difficult to achieve from a self-hosted PostgreSQL instance.
To use Redshift, you create what AWS calls clusters. The clusters house individual data sets, and you can run queries across multiple clusters to analyze all of your datasets at once. Because each cluster hosts its own Redshift engine, querying is relatively fast.
What is AWS Athena?
AWS Athena is a data analytics service that lets you run interactive queries against data stored in S3, the AWS object storage service. This means that Athena, which is based on the open source Presto analytics engine, can query any type of data that exists in S3 buckets, even if the data is unstructured. AWS calls Athena a serverless service because it requires no infrastructure set up or management on the part of users.
It's worth noting that Athena only supports SQL-style access to S3 data, and it doesn't provide any type of visualization or interpretation tools. Thus, Athena isn't a replacement for something like Elasticsearch. But it is useful if you are searching for specific types of data stored in S3, and you can write SQL queries for an SQL analysis that will find that data.
Main Differences between Redshift and Athena
Although Redshift and Athena both provide features for analyzing data at scale, they work in different ways. The key distinctions between Redshift and Athena include:
- Data structure: Because Redshift requires you to organize data into data sets within clusters, it works best for data that is structured. In contrast, Athena can analyze raw, unstructured data spread across S3.
- Data location: Redshift requires the data that you want to analyze to be stored inside Redshift clusters. So, if you want to analyze data, you need to move it into Redshift first. Athena is different because it can analyze data that exists in S3, without any data movement or restructuring required.
- Set up time: With Redshift, you need to wait for your clusters to initialize before you can begin running queries. This can take a significant amount of time. But with Athena, there is no waiting, because you don't have to move or initialize any data. You can just start running queries.
- Partitioning: Redshift and Athena both provide partitioning functionality (which speed up queries by limiting the amount of data that each query scans), but in different ways. Athena partitioning is more flexible and open-ended because you can define partitions based on any key. This means that it's easier to achieve high performance in Athena by optimizing partitioning.
- Pricing and cost: Although it's not necessarily the case that Athena costs less than Redshift, Athena pricing is simpler because you pay a flat fee (currently, $5 per terabyte) based on the amount of data you scan. Redshift pricing varies depending on cluster configuration, hourly run time and other factors, making it more difficult to know ahead of time what your ultimate cost will be.
Overall it's fair to say that Athena is more flexible – and, in certain ways, simpler – than Redshift. However, Redshift is more structured and deliberate in the way it handles data queries.
Example Use Cases for Redshift vs. Athena
Another way to think about the differences between Redshift and Athena is to focus on the varying use case that each service lends itself to.
Examples of use cases that are a good fit for Redshift include:
- Log analytics: Logs are an example of structured, consistent data that is easy to analyze within Redshift clusters.
- Software security analytics: Security logs are another example of structured data that you could store and query inside a Redshift database, as part of a security operations strategy.
- Business analytics: Redshift is a good solution if you need to store and analyze specific types of business data, such as financial reports or customer information. Since this data is relatively structured, you can fit it into Redshift clusters without difficulty.
Read: New Resources Reveal Benefits of ChaosSearch for FinTech Log Analytics
In contrast, common Athena use cases include:
- Querying cloud service logs: Because many AWS cloud services store log data inside S3 buckets, Athena makes it possible to query that data directly as part of cloud operations workflows.
- Performance troubleshooting: Since Athena lets you execute queries quickly and without having to prepare any data, it's handy in situations where you need to query a log file or stored inside S3 on a one-off basis – as opposed to performing systematic log analytics, in which case Redshift may be a better fit.
- Exploring S3 data: Athena can come in handy if you have large amounts of data inside S3 buckets and aren't sure exactly what that data is or how it's organized. By running Athena queries, you can gain visibility into your S3 data.
Read: How to Create an S3 Bucket with AWS CLI
Conclusion
AWS Redshift and AWS Athena are powerful tools that lend themselves to similar, but distinct, use cases. Knowing which type of service to use – or, alternatively, choosing a different solution (like ChaosSearch, which lets you search S3 data more flexibly than Athena because it does not limit you to SQL queries) – is a key step in optimizing your approach to data analytics.
Frequently asked questions
Which software are Redshift and Athena based on?
Redshift is based on PostgreSQL, an open source database. Athena is based on Presto, an open source analytics engine.
Is Athena cheaper than Redshift?
The cost of Athena depends on how much data you scan. Redshift pricing is based on your cluster configuration and how much time your cluster operates. Athena pricing is simpler and easier to predict, but not necessarily lower.
Can Redshift analyze S3 data?
Yes, but you need to initialize a Redshift cluster first, which takes time.
Are Redshift and Athena open source?
No; they are both proprietary services developed by Amazon. However, both services are based on open source software.
Additional Resources
Read the Blog: Optimize Your AWS Data Lake with Data Enrichment and Smart Pipelines
Listen to the Podcast: Making the World's AWS Bills Less Daunting
Check out the Whitepaper: Ultimate Guide to Log Analytics: 5 Criteria to Evaluate Tools
