Luna-Canis

Going to begin to write operators to read/process financial data. This link, points to some good raw data to start with.

Biggest issue so far, timestamps. I need a timestamp utility with usec granularity. The Boost stuff is all good, but it tries to be a bit too clever. I need time data that I can interpolate, use for calculations. The Boost stuff is too symbolic, as opposed to numerical.

gmtime seems to do the trick. I’m going to have to do some testing to see if I can initialize the structure and get the correct results. I’ll need that, plus another 16-bit short to count the milliseconds. The tick timestamps seem to have millisecond granularity. I think there are also methods for converting to different time zones, which could be very useful once we start dealing with exchanges in different time zones.

Looks like I already started a Time data type. It has the proper granulatiry, plus some numeric operators. If I can initialize it from an ascii date/time, I’m in business. ‘mktime’ is what I”m looking for….

I was able to get everything working by using the boost library stuff. I calculate the unix time myself, once I found the proper examples, the entire library fell into place. Timestamps are all good.

So I’ve added two new data types, ScalarArray, and ScalarArrayIn, for homogenous, and inHomogenous scalar arrays of data. The first two nodes will be a read operator, and a buffer type operator. Then a display node.

Got the timestamps fully working. Found a great reference here, that was very helpful in calibrating the timestamps, converting them from unix time, to boost::date_time, and back. Its solid now…

I can’t (rather shouldn’t) call it CHOPS. Since what we’re doing is processing arrays of scalars, I’m calling the datatype, ScalarArray, and ScalarArrayIn. ScalarArray is for homogenous data, ScalarArrayIn is for inHomogenous data, and includes an extra array of timestamp data. This is what’s going to represent the tick data, but calling it ‘TickData’, would have been too restrictive.

I’m starting the Buffer class. I guess I should call it a ScalarArrayBuffer. Like the Image Buffer, it drives the interactivity and the data flow of the entire process. I’m starting with a simple file reader. But when we start looking at real-time feeds, this node will be instrumental in behaving as a queue, pulling in more data as it becomes available.

New Tech! Is always interesting.