imported>Kri: added Category:In-memory database management systems using HotCat

2025-05-23T16:07:21Z

added Category:In-memory database management systems using HotCat

New page

{{Short description|Database management system that primarily relies on main memory for computer data storage}}
An '''in-memory database''' ('''IMDb''', or '''main memory database system''' ('''MMDB''') or '''memory resident database''') is a [[database management system]] that primarily relies on [[main memory]] for [[computer data storage]]. It is contrasted with database management systems that employ a [[disk storage]] mechanism. In-memory databases are faster than disk-optimized databases because disk access is slower than memory access and the internal optimization algorithms are simpler and execute fewer CPU instructions. Accessing data in memory eliminates [[seek time]] when querying the data, which provides faster and more predictable performance than disk.<ref name="Definition: in-memory database">{{cite web|title=Definition: in-memory database|url=http://whatis.techtarget.com/definition/in-memory-database|publisher=WhatIs.com|access-date=19 January 2013}}</ref><ref name="The Rise of In-Memory Databases">{{cite web|title=The Rise of In-Memory Databases|url=http://slashdot.org/topic/datacenter/the-rise-of-in-memory-databases/|publisher=Slashdot|access-date=19 January 2013|author=Michael Vizard|url-status=dead|archive-url=https://web.archive.org/web/20130201211812/http://slashdot.org/topic/datacenter/the-rise-of-in-memory-databases/|archive-date=1 February 2013}}</ref>

Applications where response time is critical, such as those running telecommunications network equipment and [[mobile advertising]] networks, often use main-memory databases.<ref>{{cite news |url=http://findarticles.com/p/articles/mi_m0EIN/is_2002_June_24/ai_87694370 |work=Business Wire |title=TeleCommunication Systems Signs up as a Reseller of TimesTen; Mobile Operators and Carriers Gain Real-Time Platform for Location-Based Services |date=2002-06-24}}</ref> IMDBs have gained much traction, especially in the [[data analytics]] space, starting in the [[2000s in science and technology|mid-2000s]] – mainly due to multi-core processors that can address large memory and due to less expensive [[random-access memory|RAM]].<ref name="Falling RAM Prices Drive In-Memory Database Surge">{{cite web|title=Falling RAM Prices Drive In-Memory Database Surge|url=http://blogs.sap.com/innovation/big-data/ram-prices-drive-in-memory-surge-020097|publisher=SAP|access-date=19 January 2013|archive-date=4 November 2013|archive-url=https://web.archive.org/web/20131104224608/http://blogs.sap.com/innovation/big-data/ram-prices-drive-in-memory-surge-020097|url-status=dead}}</ref><ref name="Rise of In-Memory Databases Impacts Wide Range of Jobs">{{cite news|title=Rise of In-Memory Databases Impacts Wide Range of Jobs|url=http://insights.dice.com/2012/07/13/the-rise-of-in-memory-databases/|newspaper=Dice.com|date=July 13, 2012}}</ref>

A potential technical hurdle with in-memory data storage is the volatility of RAM. Specifically in the event of a power loss, intentional or otherwise, data stored in [[volatile memory|volatile RAM]] is lost.<ref name="powerloss">{{cite web
| title=In-memory computing: what happens when the power goes out?
| date=18 February 2015
| url=http://www.nextgov.com/technology-news/tech-insider/2015/02/-memory-persistence-secrets-mission-critical-big-data/105249/
| access-date=March 10, 2017 }}</ref> With the introduction of [[non-volatile random-access memory]] technology, in-memory databases will be able to run at full speed and maintain data in the event of power failure.<ref>Historically, RAM was not used as a persistent data store and therefore data loss in these instances was not an issue.Whole-system Persistence with Non-volatile Memories http://research.microsoft.com/apps/pubs/default.aspx?id=160853</ref><ref>The Bleak Future of NAND Flash Memory http://research.microsoft.com/apps/pubs/default.aspx?id=162804</ref><ref>AGIGARAM NVDIMM saves data through system failure https://www.embedded.com/electronics-products/electronic-product-reviews/real-time-and-performance/4422291/AGIGARAM-NVDIMM-saves-data-through-system-failure</ref>

==ACID support==
{{See also|Atomicity, consistency, isolation, durability}}

In its simplest form, main memory databases store data on [[volatile memory]] devices. These devices lose all stored information when the device loses power or is reset. In this case, IMDBs can be said to lack support for the "durability" portion of the [[ACID]] (atomicity, consistency, isolation, durability) properties. Volatile memory-based IMDBs can, and often do, support the other three ACID properties of atomicity, consistency and isolation.

Many IMDBs have added durability via the following mechanisms:

* [[snapshot (computer storage)|Snapshot]] files, or, [[transaction checkpoint#Recovery process|checkpoint]] images, which record the state of the database at a given moment in time. The system typically generates these periodically, or at least when the IMDb does a controlled shut-down. While they give a measure of persistence to the data (in that the database does not lose everything in the case of a system crash) they only offer partial durability (as 'recent" changes will be lost). For full durability, they need supplementing with one of the following:
* [[Transaction log]]ging, which records changes to the database in a journal file and facilitates automatic recovery of an in-memory database.
* Non-Volatile DIMM ([[NVDIMM]]), a memory module that has a DRAM interface, often combined with NAND flash for the Non-Volatile data security. The first NVDIMM solutions were designed with [[supercapacitors]] instead of batteries for the backup power source. With this storage, IMDb can resume securely from its state upon reboot.
* [[Non-volatile random-access memory]] (NVRAM), usually in the form of static RAM backed up with battery power (battery RAM), or an electrically erasable programmable ROM ([[EEPROM]]). With this storage, the re-booting IMDb system can recover the data store from its last consistent state.
* [[High availability]] implementations that rely on database [[replication (computer science)|replication]], with automatic [[failover]] to an identical standby database in the event of primary database failure. To protect against loss of data in the case of a complete system crash, replication of an IMDb is normally used in addition to one or more of the mechanisms listed above.

Some IMDBs allow the database schema to specify different durability requirements for selected areas of the database{{snd}} thus, faster-changing data that can easily be regenerated or that has no meaning after a system shut-down would not need to be journaled for durability (though it would have to be replicated for high availability), whereas configuration information would be flagged as needing preservation.

==Hybrids with on-disk databases==
While storing data in-memory confers performance advantages, it is an expensive method of data storage. An approach to realising the benefits of in-memory storage while limiting its costs is to store the most frequently accessed data in-memory and the rest on disk. Since there is no hard distinction between which data should be stored in-memory and which should be stored on disk, some systems dynamically update where data is stored based on the data's usage.<ref name="intelligent-in-memory">{{cite web
| title=Teradata enters the in-memory fray, intelligently ZDNet
| website=[[ZDNet]]
| url=https://www.zdnet.com/article/teradata-enters-the-in-memory-fray-intelligently/
| access-date=July 28, 2017
}}</ref> This approach is subtly different from [[Cache (computing)|caching]], in which the most ''recently accessed'' data is cached, as opposed to the most ''frequently accessed'' data being stored in-memory.

The flexibility of hybrid approaches allow a balance to be struck between:

* performance (which is enhanced by sorting, storing and retrieving specified data entirely in memory, rather than going to disk)
* cost, because a less costly hard disk can be substituted for more memory
* [[persistence (computer science)|persistence]]
* form factor, because RAM chips cannot approach the [[memory storage density|density]] of a small hard drive

In the [[cloud computing]] industry the terms "data temperature", or "hot data" and "cold data" have emerged to describe how data is stored in this respect.<ref name="hot-cold">{{cite web
| title=What's the Diff: Hot and Cold Data Storage
| url=https://www.backblaze.com/blog/whats-the-diff-hot-and-cold-data-storage/
| access-date=July 28, 2017
}}</ref> Hot data is used to describe mission-critical data that needs to be accessed frequently while cold data describes data that is needed less often and less urgently, such as data kept for archiving or auditing purposes. Hot data should be stored in ways offering fast retrieval and modification, often accomplished by in-memory storage but not always. Cold data on the other hand can be stored in a more cost-effective way and is accepted that data access will likely be slower compared to hot data. While these descriptions are useful, "hot" and "cold" lack concrete definitions.<ref name="hot-cold"/>

Manufacturing efficiency provides another reason for selecting a combined in-memory/on-disk database system. Some device product lines, especially in [[consumer electronics]], include some units with permanent storage, and others that rely on memory for storage ([[set-top box]]es, for example). If such devices require a database system, a manufacturer can adopt a hybrid database system at lower and ''upper'' cost, and with less customization of code, rather than using separate in-memory and on-disk databases, respectively, for its disk-less and disk-based products.

The first [[database engine]] to support both in-memory and on-disk tables in a single database, WebDNA, was released in 1995.

==Storage memory==
Another variation involves large amounts of nonvolatile memory in the server, for example, flash memory chips as addressable memory rather than structured as disk arrays. A database in this form of memory combines very fast access speed with persistence over reboots and power losses.<ref>[https://www.theregister.co.uk/2013/01/30/storage_glory_days/ "Truly these are the GOLDEN YEARS of Storage."]</ref>

== Notable In-memory Databases ==

* '''SAP HANA''': This is a column-orientated in-memory database that stores data in its memory instead of keeping it on a disk. It claims to store data in columnar fashion in main memory and supports both [[online analytical processing]] (OLAP) and [[online transactional processing]] (OLTP) in the same system.<ref>{{Cite web |title=What is SAP HANA? |url=https://www.sap.com/uk/products/technology-platform/hana/what-is-sap-hana.html |access-date=2024-08-01 |website=SAP |language=English}}</ref>
* '''Oracle TimesTen:''' This is an In-Memory Database which is memory-optimized, relational database that claims to deliver microsecond response and extremely high throughput performance.<ref>{{Cite web |title=Oracle TimesTen In-Memory Database |url=https://www.oracle.com/database/technologies/related/timesten.html }}</ref>

==See also==
* [[Comparison of relational database management systems]]
* [[Distributed data store]]
* [[Embedded database]]s
* [[In-memory processing]]
* [[List of in-memory databases]]
* [[NoSQL]]
* [[NewSQL]]

==Notes==
{{Reflist|30em}}

==References==
* {{cite book|author=Jack Belzer|title=''Encyclopedia of Computer Science and Technology'' - Volume 14: Very Large Data Base Systems to Zero-Memory and Markov Information Source|date=April 1980 |publisher=Marcel Dekker Inc|isbn=978-0-8247-2214-2}}

[[Category:Computer memory]]
[[Category:Database management systems]]
[[Category:In-memory database management systems]]

In-memory database - Revision history

imported>Kri: added Category:In-memory database management systems using HotCat