Tick Life

Someone recently asked "at what speed is the game played at now" on HN. To which one of the responses was "do milliseconds matter". The answer of course depends on what kind of strategy your talking about, but for the sake of argument presume he means an ultra high freq/low latency every tick/quote/trade matters style strategy.

So heres some plots to show how fast (or slow) the market moves, first up is the BBO for part of the morning session early this month on..... you guessed it MSFT

And the life of each tick, measured in nanoseconds. Life means, every new BBO on the bid or ask resets the timer, e.g. starts at 0ns

Above graph is scaled verticaly to 60Bn nanosecconds == 60secconds == 1minute. A quick look shows at most one BBO pair`s life maxes out at around 30secconds.

If we zoom in a bit more, the above plot is the same graph but vertical scaled to around 15 seconds. From here it becomes more apprent the vast majority of BBO pairs are under seconds, possibly less. 

.. and continuing the zoom, below is some half interesting zoom in. First plot is the BBO

and yellow plot is the tick life. Here you get a better feel for how long each BBO pair lasts. A few last 10sec, a few less than 1sec and son.

digging further, below is a zoom in of the small ant hill seen on the larger BBO above.

and the corresponding tick life below.

... where you can see the same 15sec or so life of the BBO peek. Ok so your saying 15sec eh? dosent sound so ultra high frequency? true until you dig further down and say, whats the lifespan of a BBO pair which has a spread of 2cents?

... (above plot) when the spread widens is circled in red. First one around 1M ns around 1 millisecond its fast but not that fast. The second one 0.1M ns, or roughtly 100microsecconds, to put that in perspective, the standard linux kernel scheduler slices at a granularity of every 50microsecconds.

... 100microsecconds is where the fun starts. 

Silly tricks

... and the long grind forward continues abeit slowly. This week found some weirdness with the exchange and problems with my strategies using end of day tick data aka non realtime paper trading. Was using slightly stale 3-4Month old data and figured it should be close enough but dam.... how much things have changed in a few months. Will post about this when ive got some spare cycles.

Im a huge fan of autonomic computing, which means hardware&software systems that do integrity checking and automatic recovery, with the obvious examples being RAID for disk and ECC for ram. Once upon a time I worked for and with (insert huge megacorps you all know) and had a fascinating discussion with one of their hardware engineers.

HW Dude: hmm thats a weird problem, whats the value of register at offset 0xbeefbabe?

Hacking Nasdaq: register 0xbeefbabe reads out to be ... 0x01234567.

(silence)

HW Dude: are you sure thats correct?

Hacking Nasdaq: ... yes

HW Dude: thats impossible, are you on crack?

.

.

.

What he was referring to is the MSB of that register was the logical OR of the other 30bits. Making a value of 0x01234567 impossible with the only correct value being  0x81234567. Moral of the story is encoding self integrity checks into everything makes it easy to catch and not waste time on dumb ass errors - the "oh woops the cable isnt plugged in, sorry" kind

... which leads us to OUCH and the 14byte identifier token. What Ive done is reserve the last byte as an integrity check such that the 32bit sum of the previous 13bytes modulo 26 is its value.

e.g.

u32 Sum = 0;

for (int i=0; i < 13; i++) Sum += P->Msg.OrderAdd.Token[i]

P->Msg.OrderAdd.Token[13] = 'A' + Sum % 26;

This way its trivial to check if the OrderToken you looking at is actually real or corrupt without any effort. So far ive caught this a few times, usually when some bit of code has gone rouge and pissing all over memory.

...and yes thats only in development, no errors in prod... yet :)