Software provider Kx Systems has released a new version of its data product, kdb+ v2.7, which allows customisable file compression and performance up to 20% faster than the previous version, according to the firm.
Kdb+ is a programme designed to allow market data to be accessed and processed quickly for trading purposes. Version 2.7 facilitates data compression for the first time, through a choice of algorithms and a range of settings aimed at reducing latency and storage requirements. The new version also introduces protection against invalid incoming data, with a data structure verification mechanism.
“As far as our clients are concerned this is a straight latency to dollars equation,” said Kx strategist Simon Garland. “Our customers wanted an efficient way of storing their data – large institutions have to store hundreds of terabytes of data, so how it's stored, and whether it's easily and quickly accessible – with minimal latency – is hugely important to them. Because there is usually a trade-off between the amount of compression vs. speed of access, we have given our clients a choice of which data to compress and how aggressively to compress it.”
Garland cites increased market data volumes resulting from greater trading activity by high-frequency trading firms and the requirement under MiFID for market participants to maintain data for up to five years as incentive for investment in data management systems such as Kdb+.