how #Stonebraker and others improved OLTP #performance in 02008 with newish designs for #databases based on "Shore"?
on 02023-07-04Stonebraker’s 02013 talk about how everything you know about #databases is wrong (because it's obsolete); instead he argues that star-schema data warehouses should use #column-stores like MonetDB and Vertica (because they’re 50–100× faster when you have 50–100 columns but only look at 4–5 attributes and always do SQL aggregates), while for OLTP databases “NewSQL” systems are “wildly faster”, about 100×. Explains column stores using Vertica as an example, explaining that segments of rows get buffered in main memory and then written out as “slabs” of compressed column “chunklets” which are later merged into larger segments (LSM-tree style, I suppose). Explains that #OLTP databases are almost never over a terabyte, which was US$30k of RAM at the time (64 gigs in each of 16 servers). “Data warehouses are measured in petabytes, but not OLTP,” but conventional databases waste a lot of time encoding and decoding it, maintaining LRU lists, doing record-level dynamic locking, latching B-tree pages, latching the lock table, and logging transaction recovery records. So H-Store and VoltDB statically partition main memory among cores and run one thread on each core to eliminate latching and locking. This makes it seem like maybe software transactional memory might be an interesting way to implement an in-memory OLTP database. Unsurprisingly #Stonebraker is very much not a fan of #eventual-consistency. He argues that replication via log shipping will inevitably result in a 3–6× slowdown over the optimal case because it commits you to writing a write-ahead log so you can ship it, so it's better to just duplicate the computation of the transaction on each replica, and similarly use "command logging" for durability. He says this requires you to use timestamp ordering (or some other deterministic scheme) rather than MVCC. #Video is loaded from YouTube misleadingly: it's 1h12m33s, not 12m33s. I guess Stonebraker must really hate Postgres for some reason because he never mentioned it once, while mentioning almost every other SQL database!
on 02023-07-02