Luna-Canis

Long hiatus after finishing work on Land Of the Lost. Also, with Luna around, I have even less time….

Last left off at doing an overhaul on the type system. I’ve made a difference between a Type and a TypeField. As a result, I have to edit EVERY source that defines operators/functions. Including all the simd work, that’s a LOT. But its good. Its good to look at the entire work as a whole, I end up doing some large scale cleanup, and keeping the implementation somewhat consistent. As opposed to incrementally having it morph into different directions. Some duplicated data structures are also being merged.

My typing is a bit sloppier/slower. The side effect of being a user over a coder.

So the current task: Putting a script interface/interpreter on the app. Big step. I have to restructure the current grammar to allow for the language to be a list of statements, with each statement being a possible function. Currently, the grammar is a list of functions, which means that each script command would have to be wrapped up in a function. Don’t want that….

I’ve extended Type to have an array of modifiers. So we can do things like array. Its a slick way to pass types as arguments to objects. Right now, I can only see using it for arrays. I like the syntax as well.

I’ve been avoiding this, but I’m getting to the point that I have to have the script execute, which means I have to come up with some sort of virtual machine to execute it. I want it to be able to handle recursion, throw/catch, so that means having an execution stack. for/if/then/else/continue/break, the normal flow control commands.

So the commands will execute themselves, and will be able to manipulate the entire virtual machine. That’s pretty much how its going to work. The Cmd class should be under app, not smd.

UGH…. huge roadblock…. It hadn’t occured to me that I would need to be able to copy the expression trees. But yes, we DO. That means that I’m going to have to go into each Funct subclass and implement a virtual copy method. Sucks…

Had a pretty good breakthrough. The recursive copy seems to WORK. I managed to edit over 100 classes, and at least a handful of them handle it well.

Recursive arrays of arbitrary types works.

array val = { { “one”, 1.0, “two” }, “zero”, { “three” } };

parses. What I’ve also discovered is that Python does some kind of similar organization. Objects are just lists of things, and these types get passed around. If I had started with that model, I might have had a slightly different outcome for the shading/scripting languages. In any case, that functionality is doable with this implementation.

Bad side, the scriping language is highly inefficient. Just to do that one assignment, I had to create 3 copies of the array, and do the assignment 3 times. Its the overhead of requiring every operation as a node. spa::Value has always been used to carry extremely light types, with very low overhead for carrying multiple copies of objects. I should start thinking about having spa::Value that passes things by reference, not value. Expecially for things that you can’t make arbitrary copies of, like images, matrices, huge vectors, etc.

Still, big win today. Next to implement is a node class constructor object for defining operator types.

I was thinking that there are two ways to get around the efficiency issues. One is to pass everything by reference. I think I might have to have the types specify whether they are to be passed by reference or not. Anything with a non-trivial copy constructor would qualify for this. Another thing is to do lazy evaluation for copies. So if you assign multiple objects to the same value, the copy only happens when you modify one of them. Any shared pointer implementation should be able to take care of this.

Got the global ‘app’, ‘opInfo’, and ‘OpCtor’ objects working. Scripting language is kicking ass. Also added syntax that allows for calling constructors on objects are they’re defined.

Added a ‘print’ function to the grammar, that was missing for some reason. Getting script functions to work, and that means recursion.

So script functions are going to be objects that are callable. That way, they can also be passed around like data.

A mistake I made with the Types, is that the Types carry all the data within themselves. The reality is that there are (finite) amount of different types, and all the data that should be copied around is a unique identifier. So with the function type, data will definitely be non-trivial, it needs to know its argument and return types, and all the code which is a parsed Cmd object.

What I’m going to do is for the function objects to only be an identifier to be passed around, another lut is going to keep the necessary data. So these things can be passed by value, with no overhead.

Once I get this, I can start getting the virtual machine to start executing them.

UGH, I’ve come up against this RIDICULOUS msvc bug, it doesn’t setup all the virtual function table pointers for certain classes. This was happening with the Else statement class, but who knows where else this could be creeping into? I found a workaround for not using this one virtual function, but I seem to be hitting a limit/bug to the virtual function table size. And some of the virtual functions still aren’t being initialized… what a joke…

FUNCTON CALLS WORK….

RECURSIVE FUNCTION CALLS WORK….

Fuck yeah….

I’m just going to get the return values working, then maybe make it count recursively to test it, then continue with the operator stuff.

I’m having trouble getting the function inputs bindings to work properly. I know I don’t want to reparse or recopy the entire function. That’s unecessary, it would make the memory usage grow way too fast for deep recursion, and it would take too long. I should be able to get away with re-using a single function instance, and just re-binding the function inputs locally. The entire state of an execution frame should be contained in the symbol table.

So that means that VarValue nodes need to be tagged by a function command whenever they’re used to resolve an input argument. The VarValue needs a stack to bind to new local temp values, then get popped to restore them to previous values. VarValue just holds a pointer to its symbolTable, which is fortunate. To do a binding would only require updating these pointers.

Also need to make certain that variable definitions get cleaned up after they leave scope. A single variable can have multiple local instances. Don’t have a mechanism for dealing with that yet…. Function commands definitely need to have a list of any variables that resolve to their inputs. That’s step 1.

Still trying to figure out the variable binding. I think there are about 3 different ways to do this, so I think it might be better to just enumerate each one, choose one, or implement all and make it a compiler option.

• Local scope only - All variables within a function have to be either declared locally, or are arguments. No access to global variables. That would include application varaibles, might not be the best way to handle things. But functions in vex are done this way.
• Local + Global - A function can access the same variables as above, plus globally *defined* variables. That would include system wide symbols, but exclude variables defined outside the function.
• Dynamic Binding - The symbol tables would just get stacked on top of one another. Any variable not defined by the function could be queried in the calling function, an so on, until the global symbol table is reached.
• Frame access - If the same function is caling itself, then a common variable would keep getting redefined in the current frame. But what you could do it force a query a variable in the previous frame, not just the current frame. I can’t think of a use for this, (also really bad programming practice), but I think I read that its possible in scheme or lisp. If I found a formal term for this kind of addressing, I might find a use for it. So maybe ‘i’ in the previous frame could be ‘__i’. But then you have to ask why don’t you just pass in the variable as an argument? I should look this up…

A lot of this is just symbol table management. Every function needs to have pointers to its local variables to bind to a given symbol table.

Found this little gem. It formalized a lot of the definitions.

Another thing I have to do is differentiate between symbol tables that define the language, and tables that are used for global and local definitions. The language table should always be on the stack. The global/local ones will be taken on/off depending on what policy we specify above.

I got the input binding to work.