This article was originally published on www.gomedici.com/next-gen-nosql-databases-for-finserv-exploring-use-cases
Technology is changing the customer experience in the financial services industry. Today, customers expect to get everything done in just four clicks – this continuously increasing pressure to meet customer expectations is forcing financial institutions to innovate and introduce digital-first offerings as well as use the latest available technology capabilities to improve their back-end processes. There are many ways to do this: innovate in-house by increasing training and hiring of tech employees, buy or invest in FinTech players that increase your capabilities, or establish partnerships with technology solution providers that can solve your problem of technology limitations.
Challenges and innovations
In the landscape of growing customer expectations, systemic risk, market pressures, and regulatory forces, financial institutions are striving to evolve and establish their dominance within the FinServ to enhance shareholder values. The exponential increase in consumer/client demand has caused intense competition in the financial services industry, which has forced many financial institutions to squeeze margins while enhancing the quality of the financial products. The current sophisticated, complex, and ever-growing global economy pushes FinServ to become more innovative and disruptive with the introduction of FinTech. FinTech in FinServ can help financial institutions to adapt to today’s complex business environment and better prepare for uncertainties and challenges.
The use of FinTech is significant and potentially a game-changer within the FinServ industry due to the increasing tightening of regulation, government, and community pressure for improved transparency, stronger customers & stakeholder relationship, sharpening of process efficiency, improvement of risk management processes to deliver sustainable returns, etc. FinServ institutions are progressively employing more and more FinTech software to help accommodate this complex transformation in the global economy and compete with FinTech startups in various areas, from payment to databases like TransferWise, Stripe, Amazon Web Services (AWS), Aerospike, etc. Some view the challenges faced within FinServ as imminent since the current and future landscape of FinServ is driven by FinTech – and financial institutions that fully utilize and leverage this technology can gain the upper hand in the competition.
Database use cases and examples
Today, we want to focus on one of the most important innovations in FinServ: databases. The implementation of DevOps in databases that handle terabytes of customer data can enable tighter protection and security to the data as a result of better data and database management structure.
For example, the asset management industry uses SQL server in database management as a software solution for a car company that leases management lifecycle assets deployed on-premise to be migrated to the cloud. Another use case of a database for FIs is to provide operational data storage with high throughput and strict latency guidelines. It is important that such a database system is in place to detect and prevent fraud when approving a large number of transactions in milliseconds.
Moreover, the use of a database to assist the Internet of Things (IoT) process in a data-intensive industry like FinServ enables real-time data processing, RSS feeds, and smart grids. Supply chain logistics optimization and e-commerce personalized recommendation can be done by connecting users, products, and stakeholders through database use cases.
Overview and advantages of NoSQL Databases
One of the modern and rising popularity features and language used in databases is NoSQL. To understand NoSQL, first, we must understand what SQL is. SQL stands for Structured Query Language, is a programming language designed for managing data in relational database management systems (RDBMS). It is particularly useful when handling structured data. SQL became the standard of the International Organization for Standardization (ISO) in 1987.
NoSQL, sometimes referred to as ‘Not Only SQL,’ was designed for managing data that does not necessarily have the structure of RDBMS. Unlike SQL, where data is being structured in fixed relational columns, the data structure used by NoSQL databases such as a key-value store, graph store, and document store are different from the structure in relational databases, resulting in some operations being executed faster in NoSQL.
The key-value model is the least complex NoSQL option in which data is represented in a schema-less way that comprises indexed keys and values as used at innovators like Aerospike. The document-oriented database implements and assumes that documents encode data in some standard formatting and encodings to be its own unique key. It is a great way of storing, retrieving, and managing data in a single ‘document’ in JSON and XML, which can nest values hierarchically, e.g., MongoDB and CouchDB. Furthermore, in the graph database, data is represented as a graph with elements interconnected with numerous relations between them. This method is capable of handling much complexity and used in public transport links and road maps, e.g., Neo4j and Polyglot.
Critical features of NoSQL include it being non-relational, cluster-friendly as well as being able to deal with the heterogeneous and huge volume of data. Unlike RDBMS, where data is stored in highly structured relational databases, and data schemes are fixed and conform to specific characteristics, the NoSQL databases allow data to be stored in a form-free format, essentially removing the rigidity of RDBMS.
The significant advantage of this approach becomes apparent when a vast amount of data is involved in exhibiting an extensive range of structures from which diverse data is pulled together and analyzed to create IoT. The sophisticated analysis, flexible system, and scale of data managed in the NoSQL database prove to be very advantageous compared to SQL databases as relational databases are not designed to cope with the scale and agility challenges that modern software applications face. With its geographically distributed scale-out architecture, NoSQL is ideal when dealing with rapidly growing data sets from structured to unstructured while offering a modern and cost-effective method for data storage.
Aerospike, a leading player in this space, has continued its innovation spree in recent years. One of its key features includes being strong consistency. It is an industry-specific term for databases as opposed to the financial industry. This means that the data is not lost – you will not read a copy of the database that has data that is outdated. It is a requirement for being a true database. One would then ask, “Isn’t any NoSQL database a true database?”
The reality is because data is being read in so quickly all the time, you have to keep copies of the data because you have to keep them local. If you have a user on an app on the West Coast versus the East Coast versus Bangalore, sometimes that data needs to be synchronized – and there’s going to be a gap in time.
The strong consistency allows you to enforce waiting for the data to be replicated and so it prevents a lot of fraud or malicious behavior. This is a security mechanism, and relational databases have this feature kind of built-in.
We believe there is a clear leader among NoSQL databases with regard to strong consistency: Aerospike. Aerospike is excellent in the speed and throughput, and this has really helped the company to get into the financial services segment that demands complete accuracy without losing any of the speed or performance. It has a nominal advantage of 2%–5%, but the speed is at least an order of magnitude greater than a relational database.
Another feature that has been introduced recently is compression. Compression enables Aerospike to compress data to make it smaller. Compression has been around for quite some time; however, it is significant when you realize the speeds at which we are reading and writing data now. If you’re trying to access tens of terabytes to 100 terabytes to a petabyte of data, to compress it and then access it is pretty remarkable.
What should you expect from NoSQL?
Many of the next-gen NoSQL features used in the FinServ include real-time data analytics, fraud, and risk management. Next-generation NoSQL database used in FinServ will involve risk analytics and reporting where financial institutions need to consolidate and analyze multiple risk metrics to create a single view of exposure across asset classes to allow granular access to data at a massive scale.
The use of a NoSQL database is reference data management where data can be quickly distributed across geographies that can reduce the risk of regulatory penalties from reporting outdated information. Next, the NoSQL database enables high-speed data ingestion, and analytics combined with scale-out capability are used in market data management. The trade repository process incurs high data storage costs, which can be reduced with the use of a NoSQL database where the flexible schemas enable the integration of diverse trade data in a single database.
With the growth of Aerospike and other NoSQL technology firms, it is interesting to see how various features and innovations have been developed even further. As the FinServ industry is consistently being disrupted by various FinTechs, financial institutions place extraordinary emphasis on being fast, agile, flexible, and efficient.
Similarly, if we look at the evolution of the database system, it is clear that the rise of NoSQL database innovations can bring advantages to the database management of the financial institution. This is in congruence with the goal of being efficient and ultimately has the potential of replacing the SQL database as we knew it.