Migrating away from Amazon DynamoDB

Amazon DynamoDB is a fully managed NoSQL database service that runs on Amazon Web Services (AWS), designed for performance under 10 milliseconds even at scale. It supports both key–value access and document data models, allowing flexible schemas within items. DynamoDB handles data distribution and replication across multiple availability zones, offering built-in high availability and durability without user intervention. Because it is serverless and auto-scaling, developers do not need to provision or manage servers; DynamoDB handles capacity adjustments, software patching, and replication. This makes it attractive for cloud-native applications that require low-latency responses and elastic throughput with minimal operational overhead.

Typical use cases for DynamoDB include applications with unpredictable or spiky workloads and large-scale requirements. For example, it is often used in web and mobile backends such as storing user profiles, sessions, or preferences, gaming leaderboards and state tracking, IoT data ingestion, and advertising or financial platforms that require high-speed, real-time data access. Its integration with AWS Lambda and other serverless services makes it a popular choice for event-driven architectures and microservices that need low-latency data storage without the complexity of managing database servers. In essence, DynamoDB’s value proposition is to deliver fast, scalable throughput with low maintenance, so teams can start small and scale globally as demand grows.

Common reasons companies migrate away from DynamoDB

While DynamoDB scores high marks in ease of use and scalability, many organizations eventually look at alternatives due to business or technical factors, including: 

Cost management and total cost of ownership (TCO)

As data volume and throughput needs grow, DynamoDB’s usage-based pricing can lead to escalating costs, which sometimes prompts moves to cheaper self-managed or open-source databases.

Latency and performance requirements

Some use cases eventually demand lower latency or more consistent performance than DynamoDB can provide. Under typical conditions, DynamoDB reads/writes incur network calls and processing that result in ~10–20ms latency per operation. For many real-time systems, such as high-frequency trading, ultra-fast ad bidding, or user-facing decisioning, teams may need faster response times or more predictable latency distribution. If DynamoDB’s latency, or variability in worst-case scenarios, becomes a bottleneck, organizations should consider moving to a higher-performance system. Some specialized databases (or in-memory stores) deliver sub-millisecond reads by keeping indexes or data in memory and using efficient networking. Thus, when ultra-low latency is required,  teams may migrate off DynamoDB.

Scaling bottlenecks and hot partitions

Although DynamoDB is built to scale horizontally, its internal limits hinder certain access patterns. By design, each DynamoDB partition is limited to about 3,000 read capacity units and 1,000 write units per second. This means if a single partition key (i.e., a “hot key”) receives excessive traffic, it can throttle, a phenomenon known as the hot partition problem. Applications with uneven data access or extremely high update rates to a small set of keys may hit these ceilings. 

Additionally, DynamoDB items are capped at 400 KB in size, so applications needing to store larger objects or blobs per record cannot do so directly. There are workarounds, such as splitting data across items or using S3 for overflow, but they are more complex. Finally, DynamoDB’s 1 MB limit per Query/Scan page can make analytical scans or large result sets cumbersome to retrieve without pagination. When such limitations start affecting development by needing to design around item size or partition throughput, some teams opt for databases with fewer constraints. 

Operational complexity and ecosystem fit

DynamoDB is operationally simple within AWS, but that strength can turn into a weakness if your company uses other cloud service providers. As an AWS-only service, DynamoDB locks you into the AWS ecosystem, which can be a problem for multi-cloud or on-premises strategies. Organizations that want cloud provider independence or need to deploy in their own data centers for data sovereignty or latency reasons cannot use DynamoDB in those environments. 

Additionally, some teams find DynamoDB’s integration model too limiting. For example, it doesn’t support many complex queries nor stored procedures, and it requires AWS-specific tooling such as CloudWatch for monitoring, which makes it more difficult to build a custom monitoring pipeline via OpenTelemetry, Prometheus, or DataDog. 

Similarly, Amazon DynamoDB doesn't have database-level user management, and it does not manage users or passwords directly. Instead, it relies on the broader AWS ecosystem to authenticate users and applications securely, so it handles authentication primarily through AWS Identity and Access Management (IAM).  

Using DynamoDB at scale may also mean architects need to add supplementary components such as caching layers, whether it is the integrated DynamoDB Accelerator (DAX) or AWS ElastiCache/Redis, to meet performance requirements or reduce costs. The extra complexity of managing cache invalidation and multiple data stores reduces the simplicity that’s DynamoDB’s hallmark. 

Feature limitations and data model mismatches

As applications evolve, they may need capabilities that DynamoDB doesn’t have or doesn’t do well. DynamoDB is schemaless and supports ACID transactions, but it lacks the rich querying and JOINs of relational databases or some NewSQL/NoSQL peers. 

Another example is global data consistency: DynamoDB’s global tables offer multi-region replication, but only with eventual consistency, meaning that writes are eventually propagated. It cannot do multi-region strong consistency, which is a deal-breaker for financial or compliance uses that require up-to-date reads across regions. 

Additionally, DynamoDB’s secondary indexes are useful but have constraints, such as at most 20 global secondary indexes (GSI) per table, which must be defined up front, and some query patterns are hard to implement. If an application starts needing ad hoc queries, full-text search, graph traversals, or other features, one must either supplement DynamoDB with other services or migrate to a database that natively supports those queries. In short, evolving requirements for stronger consistency and richer queries can lead teams to look for a product with those functions.

In practice, the decision to leave DynamoDB is usually driven by more than one of these factors.  As one analysis summed up, most decisions to move off DynamoDB boil down to cost concerns and cloud vendor lock-in as primary motivators. Performance issues, scaling problems, and flexibility needs tend to emerge as systems grow and have more requirements. At that point, DynamoDB’s benefits, such as easy start and fully managed convenience, may be outweighed by its limitations.

DynamoDB cost structure and TCO analysis compared with alternatives

Understanding DynamoDB’s cost model is central to understanding why the business side might want to migrate. DynamoDB charges on a pay-as-you-go basis for throughput and storage. In on-demand capacity mode, you pay per request, such as read/write request units consumed, as well as data storage, data transfer, and any optional features like backups. In provisioned capacity mode, you reserve a certain throughput in RCUs/WCUs per second and pay for that provisioned capacity with auto-scaling as an option to adjust within limits. This usage-based model means that costs scale directly with your traffic patterns and data size. In other words, if your application traffic doubles, your DynamoDB bill roughly doubles as well, unless you start adding mechanisms such as caching. 

While pay-as-you-go is great for spiky workloads and avoiding paying for idle capacity, it can lead to unpredictable or high costs at scale. For example, heavy read or write workloads can incur a large number of RRU/WCUs. DynamoDB might also charge for features that some alternative databases include for free; backup/restore, cross-region replication, data export, and caching can all incur additional fees on DynamoDB. Companies often find that as their data grows and queries intensify, DynamoDB’s TCO can be more than that of running an alternative database on cloud instances or on-prem hardware, especially over time. 

Comparing TCO with alternatives requires looking at both raw infrastructure costs and operational costs. With a self-managed or open-source database such as Cassandra or MongoDB, you will pay for cloud VM instances, storage devices, and networking, typically on a fixed or reserved basis. This can be more cost-efficient at scale because you’re fully using purchased capacity. In general, DynamoDB’s model can lead to overpaying for performance headroom because you pay per request even if the underlying hardware is underutilized at times. In contrast, in self-hosted models, you pay for machines whether they are busy or idle. At very large scale or with steady, high throughput, owning or reserving capacity often saves money.

Finally, DynamoDB’s pricing model complexity, such as balancing on-demand vs. provisioned and reserved capacity deals, makes it hard to predict costs. Many companies have been surprised by unexpected DynamoDB charges, such as a surge in traffic causing a big on-demand bill, or the cost of setting up point-in-time recovery backups on large tables. Alternatives often use more straightforward licensing or infrastructure-based costs. For instance, Aerospike’s enterprise license is based primarily on data volume under management, with no direct charge per transaction or per node. This kind of model can be easier to predict and scale, essentially turning the cost into a more fixed budget item rather than an unbounded variable expense. 

Performance and scaling limitations to consider

When considering a migration, it is important to understand what’s actually causing problems and whether a new system will actually help. Some of the most pertinent performance and scaling considerations include:

Throughput and hot-key limits

DynamoDB partitions have a finite throughput limit of approximately 3,000 RCUs and 1,000 WCUs per partition by default. If your access pattern can’t be evenly distributed, such as if one customer or one item ID is extremely popular, you risk hitting throttling on that partition. Solutions include careful data modeling to distribute hot keys or using DynamoDB’s adaptive capacity, but these may not fully eliminate hot spots. 

Latency profiles

DynamoDB latency can run from under 10 milliseconds to tens of milliseconds. Under heavy load, latencies can spike if the table is throttling or if cross-region replication is involved. If your application demands consistent sub-5ms or sub-1ms latency, you’ll likely need in-memory or highly optimized databases. Aerospike, for instance, uses a smart client that interacts with the node holding the data, often with <1ms read latencies for local queries. 

Strong consistency and cross-region needs

DynamoDB offers eventual consistency by default, with an option for per-request strong consistency, but only within a single region. If your globally distributed users need the most up-to-date data, that could be a factor in migration. 

The key is to map your consistency requirements. For example, if you had issues with DynamoDB’s global tables lag, ensure the new solution either avoids multi-region writes or has a better replication mechanism. DynamoDB’s global replication is eventually consistent but might introduce seconds of lag across regions. Aerospike, by contrast, supports both active-active asynchronous replication via its Cross Datacenter Replication (XDR) feature and active-active synchronous replication with its active rack and strong consistency features. The latter can keep data centers 2500 miles apart that are part of a single logical cluster consistent in under 100 milliseconds. The Aerospike XDR capability allows even faster reads. 

Query and data model differences

DynamoDB requires careful upfront data modeling by designing primary keys and secondary indexes to satisfy known query patterns because it does not support ad hoc queries or joins. If teams find this too restrictive and need to do full table scans for certain queries or struggle with the 20 index per table limit, a new database might offer more flexibility.

Data size and access patterns

Finally, consider how the new system will handle the volume of data and traffic. DynamoDB shines with workloads up to a certain size, but datasets of terabytes to petabytes might be costly or require sharding data across tables by time to manage item limits or eliminate old data. 

Planning a DynamoDB migration 

Migrating away from DynamoDB requires planning to reduce disruption. Here is a high‑level migration plan that many organizations follow, incorporating both data and application migration steps.

Requirements and schema design

Start by gathering requirements and understanding differences in data modeling. For example, DynamoDB offers strongly consistent single‑item reads in a single region, while Aerospike’s strong consistency mode supports ACID across multiple records and survives node splits. Identify what you need from the new system:

• Do you require per‑item strong consistency or multi‑record ACID? 

• What are the expected read/write rates?

• What queries must be supported? 

Use these requirements to design the schema/data model in the target database. This may not be a one‑to‑one mapping from DynamoDB. For instance, if you used a single DynamoDB table with different entity types distinguished by a sort‑key prefix, you might now create separate sets in a single Aerospike namespace. Or if you require heavily denormalized data for DynamoDB, you might choose to normalize some of it in a relational model, or vice versa. Pay attention to data types; size limitations, such as 400 KB per item in DynamoDB vs. 8 MB per record in Aerospike; automatic 4,096-partition sharding in Aerospike, which removes the need for custom partition keys, and secondary‑indexing rules. Design and review this new schema early, and consider running a small prototype to ensure it meets functional needs.

Data export (extract) from DynamoDB

Plan how to get your existing data out of DynamoDB. AWS now offers a DynamoDB Table Export to S3 feature, which can dump the contents of a table to S3 in Parquet format without affecting live traffic, which it recommends for large tables. Alternatively, you can use the Scan operation or AWS Data Pipeline/Glue jobs to read all items. But scanning a large table directly can be slow and can consume a lot of RCUs, so if using Scan, do it in segments and consider enabling parallel scans or scheduling exports during off‑peak hours. If data size is small, you can even write a script using the AWS SDK to dump data. In any case, back up the full data set using on‑demand backups or point‑in‑time restore.

Data transformation

Once exported, transform the data into the format needed for the target database. This could include converting JSON structures, adjusting attribute names (especially if the new DB has restrictions on characters or reserved words), and splitting or merging tables if the schema has changed. If migrating to a relational schema, this step might be more involved because it requires mapping items to rows across multiple tables. For SQL‑to‑NoSQL migrations, the JSON can often be reused, but you might need to flatten some attributes or enrich the data with new keys. Write a data migration script or ETL job using something like Apache Spark or AWS Glue for this step, which reads from the export and writes to the new database in the desired format. Remember to account for secondary indexes. In other words, if you had a GSI on a field in DynamoDB and plan to index the same field in Aerospike, ensure the data is structured to allow index creation.

Initial load into the target database

Load the transformed data into the new database. Depending on the system, this could be done via bulk‑import tools or by writing directly using batch APIs. For example, MongoDB has mongoimport, Cassandra has sstableloader, and Aerospike has asloader, asrestore, or client batch‑writers. Import in stages if the data is huge, monitor the target database, and throttle the import to avoid overwhelming the cluster. Aerospike’s automatic partitioning means you don’t need to pre‑scatter keys for even load distribution. After this initial load, you’ll have a baseline of data in both the old and new systems. The new one might be slightly stale, but the next step will sync changes. 

Change data capture and sync (dual writes or streaming)

Once most of the data is loaded, address changes that occurred in DynamoDB since the export.

• Dual writes: Modify your application or middleware to write to both DynamoDB and the new database for a period of time. This ensures new data is immediately present in both. Implement idempotency and error handling so a failure on one side doesn’t corrupt data.

• Change capture/streaming: Use DynamoDB Streams to capture all changes and apply them to the new database. A small streaming app or AWS Lambda can listen to the stream and apply each change. If the new database supports Kafka or has its own streaming ingestion, such as Aerospike Connect for Kafka, route changes through that pipeline. Start the consumer at the export cut‑off to prevent gaps.

Many teams implement both strategies. Start dual‑writing at time T₀, then backfill, then use streams to reconcile any misses. The goal is to get the new database nearly in sync with DynamoDB.

Data validation

Before switching production reads fully to the new database, perform thorough validation:

• Shadow reads: Send read traffic to both systems and compare results.

• Offline checks: Compare full data dumps or run spot checks.

• Monitor latency and throughput differences to ensure the new system meets its service level agreements.

Keep doing this until any discrepancies are within acceptable limits, such as less than < 0.001 %.

Application cut‑over

Once you’re confident about data completeness and performance:

• Config/flag switch: Change the connection string or feature flag to point to the new database.

• Staged rollout: Route a small percentage of traffic first, monitor, then ramp to 100 %.

• Rollback plan: Keep dual‑writes active until the new system proves stable; if issues arise, flip traffic back to DynamoDB.

Decommission and cleanup

After a comfortable period operating solely on the new database (often a full business cycle):

1. Disable dual‑writes and stream consumption.

2. Export a final backup and archive DynamoDB tables, or delete them altogether to save money.

3. Remove unused IAM roles, alarms, and Amazon CloudWatch dashboards tied to the old tables.

Communication and incremental improvement

Keep stakeholders informed because migrations affect performance, cost, and downtime risk. With dual‑writing and careful syncing, you should be able to do this without downtime. After the cut‑over, revisit the schema to take advantage of specific features in the new database, such as Aerospike’s distributed ACID transactions and richer list/map operations, once the new system is stable.

How Aerospike supports smooth DynamoDB migrations

Aerospike offers several features and characteristics that address the common reasons for migrating away from DynamoDB, making migrations smoother and the new system more efficient:

Ultra‑low latency and high throughput

• Ultra‑low latency and high throughput: Aerospike was designed for speed.

• It uses a smart client that knows the cluster topology, so it can fetch data with one network hop to the node that owns the data partition.

• Additionally, Aerospike’s throughput per node is high; it can perform millions of read/write operations per second on a modest cluster, which means a smaller cluster can handle what might require a much larger DynamoDB capacity, or a larger cluster of other databases.

• In a recently published benchmark, Aerospike delivered 25× the throughput of DynamoDB with lower latencies.

Hybrid Memory Architecture

• Aerospike’s patented Hybrid Memory Architecture stores the index and metadata about keys in memory and the actual data on fast storage, using sub-millisecond access methods.

• You can use commodity NVMe drives, or even cloud block storage, to store large amounts of data cheaply, while just enough RAM for indexes and hot data.

• This architecture also handles high densities per node; hundreds of gigabytes to terabytes of data per node are common.

• The net effect is a lower infrastructure footprint for the same data size and workload.

• For example, Aerospike only requires a replication factor of 2 for high availability, compared with  DynamoDB’s three copies. That means you store less redundant data while still being safe.

Predictable, transparent pricing and control

• Aerospike does not charge per transaction or per query. You can use as many operations as needed as long as your cluster can handle them, without a billing surprise for each request.

Flexible data modeling

• Aerospike’s ability to hold different bins (attributes) per record (item), schemaless, like DynamoDB, allows you to denormalize or organize data flexibly.

• Aerospike supports similar data types, which usually means not much data transformation is needed. You can often take the JSON you had in DynamoDB and store it as a bin in Aerospike, or take numeric/string fields and map them to bins and even index some of them.

Consistent performance at scale

• Features such as automatic rebalancing, which is similar to Dynamo’s partition auto‑split but under your control, and a self‑managing defragmentation system mean that operational hiccups are rare.

• DynamoDB is also good at scaling, but some users find that as they push to certain levels, it becomes more complex with issues such as uneven partition distribution or dealing with global secondary index throughput.

Real‑time analytics and hybrid workloads

• Aerospike can handle a mix of workloads. It’s optimized for real‑time transactions, but some deployments also use it for light analytics or aggregations through user‑defined functions or the Aerospike query engine for secondary index queries.

Tools and expertise for migration

• While perhaps not as plug‑and‑play as Scylla’s Dynamo‑compatible API, Aerospike has tooling to help ingest data.

• For instance, Aerospike Connect for Kafka or JMS can consume DynamoDB stream data and push it into Aerospike in real-time, which can be leveraged to facilitate online migration more easily by using DynamoDB Streams as the source.

• Aerospike professional services often assist in mapping a DynamoDB data model to an optimal Aerospike design.

In short, Aerospike addresses the core pain points that lead teams to leave DynamoDB: it slashes latency to microsecond levels, cuts infrastructure costs by needing fewer nodes and not charging per operation, avoids cloud lock‑in with flexible deployment, and offers a rich‑yet‑performant data model that can stand in for key‑value or document use cases.