Csound Csound-dev Csound-tekno Search About

Re: [Csnd-dev] seeking hep with flex

Date2016-07-22 18:55
Fromjpff
SubjectRe: [Csnd-dev] seeking hep with flex
I will try too explain.

In the current distributed system the score processing is monolithic
*=(in Engine/sread.c).  In particular it expands macros when reading
the score prior to sorting.  This actually restricts what macros can have
as expansions -- in the orc case tat is why there is a pre-lexer (does
macros nd comments, #include etc.  I have been working as a background
task to replace sread with flex/bison stuff which ought to give
flexibility and maintainability.

To this end there are files in Engine/ (csound_prs.lex,
csound_sco.lex, csound_so.y and scope.c.  It occurred to me a week or
so back that I should be ale to replace getscochar() by the score
pre-leer csound_prs.lex, and indeed I had this working for simple
cases.  I then hit a collection of problems exemplified by the { ... }
structure in a score.

The form "{ # foo" starts a repeated segment, repeated # times with
foo a macro counting the iteration from 0 to #-1.  So this needs to happen
before the macros are expanded -- in the pre-lexer.  Now the problems
start in earnest.   The current implementation remembers the input
pointer into the string and expands by restoring the pointer for each
iteration.  I have code tat mimics this in flex except I have totally
failed for more than 3 days to implement the  two functions I call
yytell (where am I?) and yyuntell (restore to a value from yytell).  I
have tried many things but it never moves with yyuntell.  So either

1:  I need a flex wizard; or
2:  Someone invents a different strategy; or
3:  I delete all this code and do something else

Not sure that is clear or not

Date2016-07-24 22:25
FromSteven Yi
SubjectRe: [Csnd-dev] seeking hep with flex
Hi John,

My first thought was that this might be simpler if it wasn't
considered a macro but just part of the language.  In that case, you
could just have a bison rule like:

repeated_score: "{" number name scorelist "}"

then deal with it in the grammar or in a compiler pass.

I did some further research, and I wonder about using multiple buffers:

http://flex.sourceforge.net/manual/Multiple-Input-Buffers.html#Multiple-Input-Buffers

This is hinted at from the FAQ entry:

http://flex.sourceforge.net/manual/How-can-I-expand-macros-in-the-input_003f.html#How-can-I-expand-macros-in-the-input_003f

There's also this entry about seeking on the input:

http://flex.sourceforge.net/manual/How-do-I-skip-huge-chunks-of-input-_0028tens-of-megabytes_0029-while-using-flex_003f.html#How-do-I-skip-huge-chunks-of-input-_0028tens-of-megabytes_0029-while-using-flex_003f

and using yyrestart().

Hope there's something there that's useful!
steven

On Fri, Jul 22, 2016 at 1:55 PM, jpff  wrote:
> I will try too explain.
>
> In the current distributed system the score processing is monolithic
> *=(in Engine/sread.c).  In particular it expands macros when reading
> the score prior to sorting.  This actually restricts what macros can have
> as expansions -- in the orc case tat is why there is a pre-lexer (does
> macros nd comments, #include etc.  I have been working as a background
> task to replace sread with flex/bison stuff which ought to give
> flexibility and maintainability.
>
> To this end there are files in Engine/ (csound_prs.lex,
> csound_sco.lex, csound_so.y and scope.c.  It occurred to me a week or
> so back that I should be ale to replace getscochar() by the score
> pre-leer csound_prs.lex, and indeed I had this working for simple
> cases.  I then hit a collection of problems exemplified by the { ... }
> structure in a score.
>
> The form "{ # foo" starts a repeated segment, repeated # times with
> foo a macro counting the iteration from 0 to #-1.  So this needs to happen
> before the macros are expanded -- in the pre-lexer.  Now the problems
> start in earnest.   The current implementation remembers the input
> pointer into the string and expands by restoring the pointer for each
> iteration.  I have code tat mimics this in flex except I have totally
> failed for more than 3 days to implement the  two functions I call
> yytell (where am I?) and yyuntell (restore to a value from yytell).  I
> have tried many things but it never moves with yyuntell.  So either
>
> 1:  I need a flex wizard; or
> 2:  Someone invents a different strategy; or
> 3:  I delete all this code and do something else
>
> Not sure that is clear or not