There are API's in big languages that encourage adding calendaring systems. Does anyone want me to think about this
I am interested in your ruminations because I think the most difficult thing in statistical trading is not finding edges or whatever. It is getting good synchronized data. This may not be an issue for people restricting themselves to one time zone, and one data provider with one server with consistent time stamps.
The issue is much more complex than this -- and in part it is a 'trading with perfect future knowledge' one; removal of scratched trades, re-ordering away of reporting lags, and the rest are often found in 'historical' datasets. With 'cleaner' data, different trading possibilities, EX POST, seem more obvious.
I have not yet found a broker who will take the other side of a trade against a 'replay' yesterday's data corpus -- I assume they think the temptation for me to cheat and use my perfect knowledge has something to do with it.
The data as and when you see it is all you can derive trading signals from. One can datestamp it in when it crosses your side of the communication demarcation threshold, and 'game' possible responses against a synchronized corpus; After reduction (another delay), an order may leave your plant.
One can simulate what _should have_ happened on orders leaving your plant, but this simplifying simulation usually 'assumes' that the orderbook (of your counterparties) is not affected by your entries, and that it will stand still for your order to arrive. The markets, speed of light, and data communications networks do not and have never worked that way.
I helped in the discussions underlying this piece some years ago, written in connection with a redesign of a major Air Carrier's on line reservation system, spread across the North American continent:
physics of systems design
What was wanted simply could not be obtained with the then-observed speed of light, let alone processing lags. ;)
Rule 3 -- lots of sprightly little autonomous (loosely coupled) systems can sometimes trump a (monolithic) Big Iron dinosaur is my operative belief, and a centerpiece of most of my designs.
Knowing the in-built assumptions and limits of any system is basic to developing a Godel-ian model of the formal weaknesses of a system; finding ways to get inside an OODA frontier is essential to compete effectively against an informed counterparty. This brief John Boyd [pdf] piece on this should be on any systems analyst's periodic re-reading list. The longer, Pulitzer Prize winning Godel, Escher and Bach, by Douglas Hofstadter is a staple for periodic re-reads, slowly and for contemplation, perhaps on the nightstand.
