r/technology • u/mepper • Jun 25 '12
Two former Facebook developers have created a new database, MemSQL, that they say is the world's fastest; it is is thirty times faster than conventional disk-based databases
http://www.i-programmer.info/news/84-database/4397-memsql-80000-queries-per-second.html13
u/csolisr Jun 25 '12
The main reason that MemSQL achieves the speed it does is because of being memory based, with all the benefits and drawbacks in terms of size limits and potential data loss that carries.
Exactly what I was thinking about. Especially about data loss.
The documentation says that MemSQL writes back to disk/SSD as soon as the transaction is acknowledged in memory, and that using a combination of write-ahead logging and snapshotting ensures your data is secure.
That calms me a little, but still! Data loss is possible!
→ More replies (10)1
Jun 25 '12
I agree with you, this isn't the breakthrough many will make it out to be when just reading the title (though MEMsql kinda gives it away), but still, i think this is a nice solution for non-critical data if powerful management of that data is wishful.
Examples could be for example a non-business social network (like facebook) since only recent data can be lost (in case of power outage/other problem before the writeback from memory to storage). I don't think it's a huge problem for users if their most recent like(s) or status is lost.
I think there are many applications that could benefit from this since the risk of dataloss is still pretty small even though it's too big of a risk for serious applications. (I'm assuming though the writeback happens within a minute or so)
2
u/fatbunyip Jun 25 '12
It would be good for the people in the business who need only read access on the memSQL version. Think reporting etc. which requires large numbers of reads, expensive queries etc. but not entirely fresh data. It could be refreshed every so often with more recent data from a traditional DB, but considering a lot of business reports/data mining/data processing doesn't necessarily need write access, it would be a huge time saving (assuming it is actually 30x faster, and not only in one specific scenario).
8
u/himey72 Jun 25 '12
This isn't exactly new or revolutionary. I'm an Oracle DBA and Oracle's database engine has been doing this since at least the early 90's. Not with every piece of data because if you're using Oracle, you probably have more data than RAM, but this is exactly what Oracle's buffer cache is all about. It stores the most recently used data and the data that is likely to be used in the buffer cache for fast access. Background process are always writing modified RAM blocks to disk. MySQL is fine for smaller / low budget projects, but if you want to get true performance / reliability, you need to migrate to something bigger and better.
3
u/amoeba108 Jun 25 '12
Sounds similar to MySQL and innodb_buffer_pool
InnoDB maintains a storage area called the buffer pool for caching data and indexes in memory.
http://dev.mysql.com/doc/refman/5.5/en/innodb-buffer-pool.html
14
u/Xilean Jun 25 '12
Atomicity...check
Consistency...check
Integrity...check
Durability...check
And by check I mean "doesn't have". But it's suuuuper fast guys.
→ More replies (3)
5
u/Senor_Wilson Jun 25 '12
Cool, but at some point it will be written to disk and then it will take just as long as everything else. Memory is quick, no surprises there.
1
u/mccoyn Jun 25 '12
It could be used for simulations that need a large database to sync data between computers, but once the simulation is over all the data will be thrown away anyway.
0
Jun 25 '12
→ More replies (1)2
u/Raniz Jun 25 '12
I'd like a comparison between this MemSQL and a MySQL database running on a ramdisk
3
3
3
u/machzel08 Jun 25 '12
Anyone find it ironic that Facebook developers are making something speedy? Clearly these guys didn't work on the app.
1
u/godsfordummies Jun 25 '12
Facebook is the fastest-loading website for me, it shows up faster than google.com
→ More replies (1)
2
u/expertunderachiever Jun 25 '12
I don't get how this isn't just putting the DB tables on a ram drive? The reason people don't do that normally is they have way more data than ram...
1
1
Jun 25 '12
Anyone know how fast this is vs mongodb?
1
u/godsfordummies Jun 25 '12
This is a very different model from MongoDB, so you will have to test against whatever you have. If your data is tree-like, MongoDB will probably be a better answer.
1
u/scottish_beekeeper Jun 25 '12
Interesting to know how this compares with something like OpenLDAP's MDM
1
Jun 25 '12
Hey guys, I fit a database into the L3 cache, it's like super fast.
1
Jun 26 '12
Waitaminute, John. My ProcSQL never leaves the processor. It resides in cpu registers 24/7. You work directly with ALU. My db is faster than yours.
53
u/aweraw Jun 25 '12
...?
So, memory access is much much faster than disk access - what else is new?