AWS re:invent - New Storage Offerings

15 Dec 2021

During this year re:invent, Dr Swami Sivasubramanian, the VP of AWS Machine Learning, presented new features and services that will be soon available for us builders to use. Swami underlined that the current name of the game for most businesses is “Survival of the most informed”. Reinventing your business is an absolute must, as we know that keeping the status quo means falling behind quickly. Your constant reinvention needs a solid foundation to sail in the best possible direction and take a steady course. Is there a better foundation than one based on data? “Data is the underlying force that fuels the insights and the predictions that help you make better decisions and spurs innovations completely,” says Dr Swami. Your business will need to master these three areas to make the most of your data:

  • Storage & Access - fault-tolerant, scalable and highly available managed storage services with minimal administrative overhead.
  • Analytics & Visualization
  • Prediction

Dr Swami touches on those three, and I greatly encourage you to watch the entire talk. Meanwhile, this article will focus on some of the latest AWS innovations and offerings in the Storage & Access domain.

Amazon DevOps Guru for RDS

Amazon DevOps Guru is a machine learning-powered service designed to simplify applications’ operational performance and availability. Amazon DevOps Guru detects behaviours deviating from standard operating patterns, thanks to which operational issues can be identified long before they impact your customers. Now it is extended and tailored for some of the AWS Database offerings. Here are some examples of performance and operational issues that Amazon DevOps Guru for RDS would allow you to identify quickly and easily:

  • Lock scenario causing e-commerce website time-out.
  • Spike in database wait time.
  • Resource contention or exhaustion.
  • Query performance regression. 
  • Faulty database driver causing slow down on your large scale platform.

Apart from explaining the cause and detailed diagnostic information, this offering would provide recommendations on how to solve these complex database related problems and even perform automatic remediation wherever possible. AWS is making it clear that it strives to have its services as self-managed as possible, leaving you with more time to innovate. Amazon DevOps Guru for RDS also supports native integration with Amazon EventBridge and Amazon SNS service for an easy set-up notification strategy. According to the information of the AWS site, Amazon DevOps Guru for RDS currently supports Amazon Aurora MySQL-Compatible Edition and Amazon Aurora PostgreSQL-Compatible Edition. Expanded support for additional Amazon RDS database engines is expected to come later. 

 

Amazon RDS Custom

The flexibility of the managed databases that the cloud provides, such as elasticity, performance, disaster recovery and cost savings, is hard to neglect as businesses modernize. Many of these businesses also need the ability to customize their underlying databases and operating systems on which they run. This includes applying patches and enabling native features. In October this year, Amazon RDS Custom for Oracle was announced, allowing to use and take advantage of some offerings of this storage provider that was not possible with the original Amazon RDS for Oracle. Let's take a look at how the responsibility of managing various features of RDS has been changed by the introduction of the Amazon RDS Custom:

With Amazon RDS Custom, you have a lot more control which comes with a lot more responsibility. 

Dr Swami has now announced that Amazon RDS Custom will also support SQL Server applications, such as:

  • SharePoint
  • Power BI
  • Active Directory
  • Dynamics
  • PolyBase

Amazon RDS Custom supporting both Oracle and SQL Server will open up the benefits of managed databases to many more customers with legacy, packaged or custom applications.

Amazon DynamoDB Standard Infrequent Access

Most if not all of the businesses have these two requirements present on a portion of their data:

  • Store it for a more extended time.
  • Have it readily available when requested.

In the context of Amazon DynamoDB up until this point, to achieve both, your company had to choose between these two options:

  • Keep the data in Amazon DynamoDB for long periods and incur considerable costs.
  • Move it to Amazon S3 and build a query engine on top, which requires additional work and maintenance.

Whichever the choice, cutting on your storage costs was not a trivial task. Dr Swami has now announced a new feature of Amazon DynamoDB, which will make this cost vs availability problem disappear. We all know the solution from the Amazon S3 service and it is now coming to Amazon DynamoDB in the form of Standard-Infrequent Access table class. This table class offers lower storage costs than the standard tables, allowing you to save up to 60% where storage is the dominant table cost. On top of that, Amazon promises precisely the same performance, durability, scaling and availability as standard DynamoDB tables. We have to keep in mind that if we use the Standard IA table class just like we utilise the Standard class, accessing the data very frequently, in the end, we will actually incur greater costs. This is why the Standard class should still be the primary go-to option for most of your workloads. Amazon additionally gives the ability to switch between the table classes without any impact on performance. Your application code will not need any changes as well.

Summary

With the ever-evolving storage and access managed services that AWS offers, it is clear that the days of on-prem / self-hosted installations are starting to become a thing of the past. However, if you and your company are still in that group, get ready, as most likely migration to the cloud is just around the corner for you. The rich feature set they provide is simply becoming too tempting to overcome.