Wes McKinney started writing #Pandas in 02008 and subsequently worked on something called Badger, and then #Ibis, Arrow, Feather, and #Parquet. Here he explains the unifying thread, but this is mostly about #Arrow.
on 02024-11-22a #dataframes system for lazy computing on data in #Arrow or HDF5 files: “Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀 (...) #Vaex uses memory mapping, zero memory copy policy and lazy computations for best performance (no memory wasted).”
on 02024-11-22