The use of In-Memory database technology is a growing trend in the database management system (DBMS) market. Although the concept of data processing in memory, the approaches and technologies adopted for both recent and existing DBMS are.
In-Memory database management systems (DBMS In-Memory) use the cache memory as the primary storage for storing, managing, and manipulating data.
Traditionally, systems store data on disks. Most of them transfer this data to memory in a cache (or buffer pool) when accessed by the system. This memory transfer makes the following accesses to the data more efficient. But these constant movements between disks and memory can lead to performance issues.
In-Memory database management systems are used primarily to improve the performance of queries and applications that access data. They can also benefit from a reduced set of instructions given the reduction in the number of activities that need to access the data (unlike access from a disk).
The NewSQL database system is another branch of the current In-Memory and NoSQL trends.
The NewSQL concept is to adopt the NoSQL-driven market trend with current database architectures, configurations, and implementations, but to support SQL, leveraging the knowledge of the huge pool. SQL developers. NewSQL DBMS systems are not all In-Memory, but many of them are.
Market Conditions for In-Memory Databases
The idea of speeding up processes using memory is not new. But what are the factors that explain the trend in favor of DBMS In-Memory?
First, the technology used is becoming more widespread and less expensive.
The amount of memory available on current servers continues to grow, many of which offer 32 terabytes (TB) of memory or more. In addition, the cost of memory continues to decline; currently, it is possible to buy 8 GB of memory (RAM) for $ 70, against several hundred dollars just a few years ago.
Although RAM is not yet as cheap as discs, its price continues to decline and make it an appropriate choice for a growing number of uses thanks to its price / performance ratio, given the potential performance gain. In-Memory DBMS compared to conventional DBMS using disks.
But the price of the material is not enough to create a trend. The need to increase speed in recent applications is contributing to the growing success of In-Memory DBMS . Businesses are creating and adopting more and more real-time Web applications that can take advantage of the high-end performance of In-Memory technology .
With improved data reliability and persistence, In-Memory DBMS now addresses the transaction processing and analytics needs of most organizations.
And with the current processing technology in memory, persistence is no longer an obstacle. By nature, memory is a volatile form of storage. If the server is no longer powered, the data in memory will be lost. The commercially available In-Memory DBMS are designed so that the data in the memory stays there even after a failure.
With improved data reliability and persistence, In-Memory now meets the processing and analytics needs of most organizations.
Types of In-Memory Databases
At first glance, it may seem easy to define the In-Memory DBMS, but the processing of data in memory has a long history and has taken different paths.
One of the first forms of data processing in memory was developed by COBOL programmers who had created tables in memory to store data that a program could access many times during execution. Of course, this system was not a database, but it was a primitive form of data access in memory adopted to increase the speed of processing.
As IT moved into the DBMS era, techniques evolved to access data from memory rather than disk.
Any database administrator who has used any type of DBMS has probably tried to make it access the data in memory rather than on disks. Database administrators must then configure the appropriate levels of memory to cache the data in buffer pools. Caching allows the data to remain in memory for subsequent access.
There is another form of in-memory data processing that uses SSDs . These disks are storage devices that work with memory chips rather than a mechanical disk, to ensure persistent storage of data.
The history of SSD goes back to the 1950s, when big mainframes and supercomputers were in existence. In the 1980s, computer management systems databases were stored on a type of basic SSDs. But until very recently, this technology was too expensive to be widely adopted. Today, the price of memory being more affordable, it is possible to create a simple form of databases in memory which consists of storing files on SSDs.
But an In-Memory DBMS today offers much more than a standard DBMS stored on a SSD.
The current In-Memory is specifically designed for memory processing. All data is stored in DRAM on a server, and all operations run in memory. All data is in memory and this data can be compressed to improve storage and access performance.
There is also another form of commonly used In-Memory which includes hybrid disk-based databases. These hybrid disks work not only with memory chips to store data, but also with hard disks.
The advantage of a hybrid DBMS is its flexibility, which makes it possible to design databases by balancing performance, cost and persistence. Many applications can benefit from the ability to access certain data quickly, while others, which are less often used, are stored on disk. Discs remain cheaper than memories, the tradeoffs that a hybrid solution makes possible appeal to many companies that have varied needs and tight budgets. Most of the leading relational database management system (RDBMS) providers add In-Memory functionality to their offering to complement existing disk storage.
In-Memory DBMS can be relational , NoSQL , NewSQL or any other type of DBMS. You can use them for the operational processing of transactions or for analytical, business intelligence (BI) applications.
Of course, each system has different features and functionality that can enable it to support operational processing better than analytical processing (or the opposite).
Advantages and disadvantages of In-Memory databases
The strong point of DBMS In-Memory is the significant gain in performance that can be achieved over disk-based systems. One can reasonably expect three to four times better performance with an In-Memory DBMS, and much more in some cases.
In-Memory offers have traditionally been well received in the integrated database market, where a small, non-invasive architecture is appreciated. But the current In-Memory market, and its many professional-level offerings, support much more than just built-in applications.
That said, if their performance is so much better, why is it that everyone has not migrated to an In-Memory DBMS?
Partly because of its cost. As we have seen above, memory always costs more than a disk, even if the price of DRAM chips decreases from year to year.
Other factors hampering adoption include lack of expertise in in-memory DBMS, existing DBMS implementations (which include some In-Memory features), and non-standard options; sometimes to get the highest speed – you may need to use an interface other than a simple SQL interface.
Finally, the size of databases was traditionally a limiting factor, but technological advances remove this constraint.
Even so, while in-memory DBMSs can handle very large databases, many still believe that the memory bases must be limited in size. There is still a lot of work to be done to inform in order to defeat this idea. There are examples of In-Memory DBMS with a capacity of up to one terabyte.
Use case of In-Memory DBMS, focus on real time
The use cases of the In-Memory DBMS are numerous and varied. Any application that can benefit from increased performance can potentially benefit from using an In-Memory DBMS.
For specific uses, these systems may be of real interest for applications with real-time data management needs, such as those used in the telecom and network management, financial markets, defense and intelligence, travel and reservations, call centers and games.
Applications that need data immediately are also good candidates, such as real-time business intelligence (BI), fraud detection, real-time analytics, and streaming data.
Additional factors to consider in your purchase evaluation
When thinking about purchasing an In-Memory DBMS, there are other things to consider in your evaluation.
Although most In-Memory DBMS offer options for dealing with the issue of data persistence, you need to pay careful attention to how data sustainability is addressed. Because the data is in memory – which is not persistent – the In-Memory DBMS must provide a way to permanently store it in persistent memory. What happens if you unplug the server?
There are different ways to manage sustainability in an In-Memory DBMS.
One of the possibilities is to log the transactions, so that periodic snapshots of the database are recorded on a non-volatile medium. If the system fails and needs to be restarted, the database can be restored to the last executed or progressed transation.
Another option is to keep extra copies, which will essentially be standby databases on a non-volatile medium. Another possibility is to use a non-volatile RAM (NVRAM), such as a RAM with a battery backup or a ferroelectric RAM (FRAM), on which data can be kept when the power is off. And, of course, hybrid In-Memory DBMSs can use disk storage for durability.
Do you really need an In-Memory DBMS?
You also need to determine if you really need an In-Memory DBMS or if another technology would not be more appropriate.
For example, if you are deploying databases to an SSD using the existing RDBMS, will you not get sufficient performance gains with less disruption to your environment?
The performance of an In-Memory DBMS will outperform a typical SSD on SSD, but it is better to test to make sure that the extra earnings justify the choice of a different technology and a new provider.
In addition, to create some database appliances, developers use In-Memory systems and technology. A database appliance must be a turnkey solution requiring little or no configuration and installation. However, a daily administration remains indispensable.
Make sure that you are familiar with the embedded DBMS technology for any appliance you purchase.