[Cs-dev] Heads Up: Compiler internals change
| Date | 2013-03-26 06:14 |
| From | Steven Yi |
| Subject | [Cs-dev] Heads Up: Compiler internals change |
| Attachments | None None |
Hi All,
Just as a heads up, I've just pushed a sizable set of commits that involve changing how the semantic checker, expression expansion, and compiler function. In general:
1) Expression expansion code refactored (expand_statement, expand_if_statment, expand_until_statement) 2) Semantic checking code refactored (verify_if_statement, verify_until_statement)
3) Expansion phase removed, integrated into semantic checking phase 4) VAR_POOL is created and used during semantic phase to lookup variables' types 5) OENTRY lookup is now only done during semantic checking 6) Semantic phase now marks up the TREE*; for opcodes, it adds an OENTRY* to the markup field, for INSTR and UDO, it sets the generated VAR_POOL to the markup field 7) Compilation phase no longer builds VAR_POOL's or does opcode lookup, but instead uses the markup data
8) TEXT* struct no longer uses opnum, replaced with oentry field that stores OENTRY* One of the big reasons for the changes were to better support arrays, as well as simplify discovery of types for variables. A few points:
1) The initial array support was coded to rename variables. This was done to accomodate to the pre-existing type detection using the initial letter of a variable to discover it's type. For example, kVar[] would get renamed as [k;Var. However, this cause a problem if there were kVar (scalar) and kVar[] (array) defined, as this should be an error of conflicting types (i.e. a var should not be added with different type). The change to the type system to define CS_VARIABLE's once and then lookup of all variables done against the VAR_POOL instead of the first-letter simplifies the change.
2) Pre-Csound 6, the Csound compiler would lookup var types only by looking at the first letter of a variable's name. This however was done in a number of places within the compiler, such that every encounter of a variable identifier would often have the type reconstructed from the name. The new code limits the type resolution of a variable identifier to just once, and the type is then maintained for the rest of the scope of semantic check/compiling.
3) The semantic checker now does the expansion of TREE* to flatten them into the final singly linked list of opcodes and labels. The reason to integrate that into the semantic checker was so that the any synthesized labels and variables could be added to the VAR_POOL and label list for checking and later use in the compiler.
4) Also, way things are expanded has slightly changed. In CS5, if an expansion was done, the nodes would get re-connected to the current list, then processing would continue from the end of the expanded code, onto the original next item. This has changed now to expand nodes, then insert and continue from the head of the newly expanded nodes. The reason for this was so that the semantic checker could verify and markup opcodes/variables statement by statement. This has the benefit of the verifier running over the expanded nodes as well, giving a sort of built-in check to the expansion code.
There's still a fair amount of cleanup/testing work to be done, but I'm pretty happy with the latest changes. Hopefully the code should be a little clearer than the pre-CS6 code.
Thanks! steven |
| Date | 2013-03-26 09:23 |
| From | Victor Lazzarini |
| Subject | Re: [Cs-dev] Heads Up: Compiler internals change |
All sounds very good and the correct way of doing things. I have a couple of points: 1) regarding 4) I see that global variable pools are still being built up in the compiler stage; I am not sure what you mean then. 2) some adjustments still need to be made, as trapped.csd does not compile now. anyway, we'll talk at 11, if you still can make it after the long night. Victor On 26 Mar 2013, at 06:14, Steven Yi wrote: > Hi All, > > Just as a heads up, I've just pushed a sizable set of commits that involve changing how the semantic checker, expression expansion, and compiler function. In general: > > 1) Expression expansion code refactored (expand_statement, expand_if_statment, expand_until_statement) > 2) Semantic checking code refactored (verify_if_statement, verify_until_statement) > 3) Expansion phase removed, integrated into semantic checking phase > 4) VAR_POOL is created and used during semantic phase to lookup variables' types > 5) OENTRY lookup is now only done during semantic checking > 6) Semantic phase now marks up the TREE*; for opcodes, it adds an OENTRY* to the markup field, for INSTR and UDO, it sets the generated VAR_POOL to the markup field > 7) Compilation phase no longer builds VAR_POOL's or does opcode lookup, but instead uses the markup data > 8) TEXT* struct no longer uses opnum, replaced with oentry field that stores OENTRY* > > One of the big reasons for the changes were to better support arrays, as well as simplify discovery of types for variables. A few points: > > 1) The initial array support was coded to rename variables. This was done to accomodate to the pre-existing type detection using the initial letter of a variable to discover it's type. For example, kVar[] would get renamed as [k;Var. However, this cause a problem if there were kVar (scalar) and kVar[] (array) defined, as this should be an error of conflicting types (i.e. a var should not be added with different type). The change to the type system to define CS_VARIABLE's once and then lookup of all variables done against the VAR_POOL instead of the first-letter simplifies the change. > > 2) Pre-Csound 6, the Csound compiler would lookup var types only by looking at the first letter of a variable's name. This however was done in a number of places within the compiler, such that every encounter of a variable identifier would often have the type reconstructed from the name. The new code limits the type resolution of a variable identifier to just once, and the type is then maintained for the rest of the scope of semantic check/compiling. > > 3) The semantic checker now does the expansion of TREE* to flatten them into the final singly linked list of opcodes and labels. The reason to integrate that into the semantic checker was so that the any synthesized labels and variables could be added to the VAR_POOL and label list for checking and later use in the compiler. > > 4) Also, way things are expanded has slightly changed. In CS5, if an expansion was done, the nodes would get re-connected to the current list, then processing would continue from the end of the expanded code, onto the original next item. This has changed now to expand nodes, then insert and continue from the head of the newly expanded nodes. The reason for this was so that the semantic checker could verify and markup opcodes/variables statement by statement. This has the benefit of the verifier running over the expanded nodes as well, giving a sort of built-in check to the expansion code. > > There's still a fair amount of cleanup/testing work to be done, but I'm pretty happy with the latest changes. Hopefully the code should be a little clearer than the pre-CS6 code. > > Thanks! > steven > > > ------------------------------------------------------------------------------ > Own the Future-Intel® Level Up Game Demo Contest 2013 > Rise to greatness in Intel's independent game demo contest. > Compete for recognition, cash, and the chance to get your game > on Steam. $5K grand prize plus 10 genre and skill prizes. > Submit your demo by 6/6/13. http://p.sf.net/sfu/intel_levelupd2d_______________________________________________ > Csound-devel mailing list > Csound-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/csound-devel Dr Victor Lazzarini Senior Lecturer Dept. of Music NUI Maynooth Ireland tel.: +353 1 708 3545 Victor dot Lazzarini AT nuim dot ie ------------------------------------------------------------------------------ Own the Future-Intel® Level Up Game Demo Contest 2013 Rise to greatness in Intel's independent game demo contest. Compete for recognition, cash, and the chance to get your game on Steam. $5K grand prize plus 10 genre and skill prizes. Submit your demo by 6/6/13. http://p.sf.net/sfu/intel_levelupd2d _______________________________________________ Csound-devel mailing list Csound-devel@lists.sourceforge.net |
| Date | 2013-03-26 10:47 |
| From | Steven Yi |
| Subject | Re: [Cs-dev] Heads Up: Compiler internals change |
| Attachments | None None |
Hi Victor, Regarding the global pool, there's still some code in csound_orc_compile.c calling addGlobalVariable, but the rest of the pool of variables found during parsing is done during semantic check time, and that pool is assigned to the engineState now. I don't know if the things that are adding to the global pool are the cause of the issue with trapped (I had been using the csdtests during development). I'll take a look at trapped after breakfast, and should be online for the chat.
Thanks! steven On Tue, Mar 26, 2013 at 9:23 AM, Victor Lazzarini <Victor.Lazzarini@nuim.ie> wrote: All sounds very good and the correct way of doing things. I have a couple of points: |
| Date | 2013-03-26 11:39 |
| From | Steven Yi |
| Subject | Re: [Cs-dev] Heads Up: Compiler internals change |
| Attachments | None None |
Hi All, The issue with compiling trapped had to do with an empty instr body. I've added csd tests for empty instr's and empty UDO's and they are now passing (and trapped is happily rendering again).
Thanks! steven On Tue, Mar 26, 2013 at 10:47 AM, Steven Yi <stevenyi@gmail.com> wrote:
|