How Charles Schwab modernized its database architecture for real-time intraday operations
For Charles Schwab, achieving $7.7 trillion in assets required adopting a highly available distributed database architecture.
Charles Schwab is a financial giant that offers a wide spectrum of money management services. The company recognized the need for real-time data in the always-on world of securities trading. To address this mission-critical necessity, it undertook a modernization journey and adopted Aerospike as its intraday system of record.
Intraday, meaning “within the day,” holds immense importance in the securities trading sphere. It pertains to securities trading during regular business hours, as prices fluctuate throughout the day. It goes without saying that having consistent, reliable data is paramount for their customers. Day traders analyze intraday price movements to execute short-term trades, making real-time data crucial for their decisions. This blog delves into Charles Schwab’s strategy, the challenges faced in the past, and the remarkable outcome generated from a new, modern database architecture.
Charles Schwab’s database architecture modernization goals
In 2014, Charles Schwab embarked on a mission to “build better, faster, and cheaper solutions,” says Venkat Thamminana, Charles Schwab’s Managing Director of Portfolio Marketing Technology. This involved rescoping their database architecture principles to optimize their infrastructure performance.
Database architecture challenges in 2014
In 2014, Charles Schwab faced challenges with their existing database architecture. The main issues Thamminana and his team needed to address were:
Core trading applications residing on the mainframe.
Limited scalability, costly scaling processes, and lengthy development times.
Constraints on latency and throughput.
Availability issues, especially in terms of disaster recovery (DR) challenges.
Knowing that their intraday operations needed to scale with growing customer needs, Thamminana and his team settled on the following architectural guidelines.
Architectural principles
Single operational data store: Charles Schwab has numerous customer-facing applications. Instead of having multiple databases addressing or handling any single request, the objective was to have one single operational data store that housed all available data for all existing applications.
No local caching: The team was determined to avoid caching at the virtual machine (VM) level, ensuring data is stored centrally. In other words, all data must be stored in the data store, and every application should be able to access it when necessary.
Maximum code reuse: Thamminana and crew recognized a need for code efficiency, emphasizing the importance of reusing preexisting code.
CLR/JVM focus: This principle addressed moving away from multi-service architectures and handling business logic within the Common Language Runtime (CLR) or Java Virtual Machine (JVM). These are both process VMs, whereby rather than executing a whole operating system, these VMs can run a program independently of the platform environment. As Thamminana states, his team “moved away from that multi-service architecture where there will be an orchestrator calling multiple services that will add a lot of latency.”
Aerospike as an intraday system of record
One of the most important factors in establishing a real-time intraday system of record was the pursuit of data consistency. In Thamminana’s words, “We wanted to have consistency in our data, so whatever channel customers log in to, they should see only consistent data. There should not be any discrepancies between channels or calls made from the same client on the same channel, so that was the challenge we were trying to address back in 2014.”
With this as a guiding principle and adopting the Aerospike Database as their real-time intraday system of record and user profile database, Thamminana and his team had to account for several variables, namely, deciding on the appropriate migration and zone isolation strategies.
The migration strategy was two-fold, with read and write applications migrated separately. A significant bulk of the traffic derived from “read applications” is attributed to general client traffic, which typically consists of viewing their own portfolios to assess margin risks during intraday hours, for example (trading hours are open from 9:30 a.m. to 4:00 p.m., eastern time). This migration required having all the data stored in Aerospike with new, real-time refreshes as needed, including daily, intraday, or weekly refreshes.
The write applications include actual trades, cash deposits, and buying or selling transactions. For this, the Charles Schwab team posted the transactions to Aerospike in real time. It retained a DB2 as the system of record for those legacy applications, which was updated asynchronously with the relevant data.
They also needed a zone isolation strategy to reinforce data resiliency and high availability. They did this by deploying multiple data centers, making it possible for each geographic zone to have 100% of the data.
Performance transformation in 2024
Today, Charles Schwab manages a staggering $7.7 trillion in assets, offering full brokerage services through all Internet of Things (IoT) devices, with 99.99% availability during intraday hours. Utilizing Aerospike as a distributed database, Charles Schwab can handle about 150 TB in each zone, with 75 billion daily transactions with a throughput of 1.1 million reads every second and 0.45 million writes a second.
Does your database architecture need a transformation?
Charles Schwab’s strategic adoption of Aerospike as their intraday system of record marks a significant leap in their capabilities. The modernization journey has met their goals and exceeded expectations, providing a robust, scalable, high-performance infrastructure. As they continue to evolve, Charles Schwab exemplifies how embracing cutting-edge technologies can revolutionize financial operations in a dynamic market landscape.
