how fast is fast?

Wow... and back once again after a slight detour through the silicon/carbon composite alloy that is the jungle of finance IT...... So, back to hacking NASDAQ, march on!!

First off with all this talk of HFT and massive compute/ultra low latency/blah its a good idea to step back a bit and say... what is the shape of nasdaqs tick data rate? what kind of throughput/latency we dealing with?

The standard number in networking typically refers to bits/second either Megabit/sec or Gigabit/sec or Terrabit/sec etc that translates to (Total Bytes) / (Timebin in Seconds). Which is great if your looking at throughput e.g. downloading throttling torrentz but for trigger-happy-on-the-razors-edge-pants-on-fire trading systems such number is completely useless, as we will see shortly.

None the less lets start off nice and slow and look at some random day on nasdaq this year, specifically 2011/4/11. Each sample here is 10ms, using the peek bandwidth over a sliding window of 1second.

For reference

Bytes = Ethernet frame(12B) + IPv4 frame(20B) + UDP frame(8B) + SoupBin + ITCH length

eg  the numbers include the full protocol/framing overhead up to ethernet but not including xaui/sgmii/layer1 overhead or accounting for interframe gaps

NOTE: all graphs vertical axis is Mbit/sec

Which is kinda boring, but can clearly see the auction open(triangle thing), start of matching(first spike), nice smile/dip in activity during lunch and how it picks up again at the close.

There`s a few interesting patterns here, such as the ripples in traffic few seconds before matching starts.

... and the burst of activity just before the close which surprisingly is larger than the open

... zooming in you can see some of the more interesting behaviour / burstyness of the traffic... or perhaps a martian landscape with cacti.

... a more sharper burst

Cool pics but... pftty... it barely peeks at 300Mbit whats the big deal? the answer is - the rates are calculated over a 1second sliding window and 1second is one hell of a long time.