Csound Csound-dev Csound-tekno Search About

[Cs-dev] A difficult parser problem (new parser)

Date2011-09-12 08:50
Fromjohn ffitch
Subject[Cs-dev] A difficult parser problem (new parser)
This message is aimed mainly for Steven, Martin and myself, but any help
most welcome.

The Initial Problem
===================

The lexer had lines

 LABEL           [a-zA-Z_][a-zA-Z0-9_]*:
and
 {LABEL}         { *lvalp = make_label(csound, yytext); return T_LABEL; }

This has two problems.  First and trivially it does not allow spaces
before the colon, but more significantly it really messes the lexing
of
        a1 = (k1>0?a2;a3)
as the a2: becomes a single token.  There are many worse examples

The First Fix
=============

I removed the \LABEL pattern and changed the parser for statements
from
T_LABEL
to 
label S_COL

and after some other fixes it was made to work

BUT New Problem
===============

Running the tests Steven created it fails on two tests.  They both
have an example of two arguments

idur,iamp,iskiptime,iattack,irelease,irvbtime,irvbgain  passign   3

leads to the error

Reading a token: Looking up token for: 469 : idur
NamedInstrFlag: 0
Next token is token T_IDENT_I ()
Shifting token T_IDENT_I ()
Entering state 17
Reading a token: Next token is token S_COM ()
Reducing stack by rule 54 (line 483):
   $1 = token T_IDENT_I ()
-> $$ = nterm label ()
Stack now 0 32 6 82 151 205 221 233
Entering state 40
Next token is token S_COM ()
error: syntax error, unexpected S_COM, expecting S_COL (token ",") line 9: idur,iamp,iskiptime,iattack,irelease,irvbtime,irvbgain  passign   3
Error: popping nterm label ()
Stack now 0 32 6 82 151 205 221 233
Error: popping nterm statementlist ()
Stack now 0 32 6 82 151 205 221


It seems to have decided that the idur must be a label rather than a
variable

The relevant syntax lines are

ans       : ident               { $$ = $1; }
          | ans S_COM ident     { $$ = appendToTree(csound, $1, $3); }
          ;

statement : ........
          | label S_COL
                {
                    $$ = make_leaf(csound, T_LABEL, (ORCTOKEN *)$1);
                }
          .........

label : T_OPCODE    { $$ = (TREE *)$1; }
      | T_OPCODE0   { $$ = (TREE *)$1; }
      .....

ident     : T_IDENT_I   { $$ = make_leaf(csound, T_IDENT_I, (ORCTOKEN *)$1); }
          | T_IDENT_K   { $$ = make_leaf(csound, T_IDENT_K, (ORCTOKEN *)$1); }
      .....

does anyone have any suggestions?  The problems seem to arise from
labels being any word, opcode, variable or other

==John ffitch

------------------------------------------------------------------------------
Doing More with Less: The Next Generation Virtual Desktop 
What are the key obstacles that have prevented many mid-market businesses
from deploying virtual desktops?   How do next-generation virtual desktops
provide companies an easier-to-deploy, easier-to-manage and more affordable
virtual desktop model.http://www.accelacomm.com/jaw/sfnl/114/51426474/
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Date2011-09-12 21:23
FromSteven Yi
SubjectRe: [Cs-dev] A difficult parser problem (new parser)
Hi John,

I'm about to board my flight to Dublin, so I won't be able to follow
up with this properly until I get to Maynooth and settle in.  One
thing we can do is make the parsing context-dependent, much in the way
the UDO arguments are done.  So  if we're in the case after a ? is
found, we can then use a flag when looking at label to instead return
the value and the semicolon separately.  I'm not sure that's the most
elegant but it should work.

I'll check back in once I'm all setup.

Thanks!
steven


On Mon, Sep 12, 2011 at 3:50 AM, john ffitch  wrote:
> This message is aimed mainly for Steven, Martin and myself, but any help
> most welcome.
>
> The Initial Problem
> ===================
>
> The lexer had lines
>
>  LABEL           [a-zA-Z_][a-zA-Z0-9_]*:
> and
>  {LABEL}         { *lvalp = make_label(csound, yytext); return T_LABEL; }
>
> This has two problems.  First and trivially it does not allow spaces
> before the colon, but more significantly it really messes the lexing
> of
>        a1 = (k1>0?a2;a3)
> as the a2: becomes a single token.  There are many worse examples
>
> The First Fix
> =============
>
> I removed the \LABEL pattern and changed the parser for statements
> from
> T_LABEL
> to
> label S_COL
>
> and after some other fixes it was made to work
>
> BUT New Problem
> ===============
>
> Running the tests Steven created it fails on two tests.  They both
> have an example of two arguments
>
> idur,iamp,iskiptime,iattack,irelease,irvbtime,irvbgain  passign   3
>
> leads to the error
>
> Reading a token: Looking up token for: 469 : idur
> NamedInstrFlag: 0
> Next token is token T_IDENT_I ()
> Shifting token T_IDENT_I ()
> Entering state 17
> Reading a token: Next token is token S_COM ()
> Reducing stack by rule 54 (line 483):
>   $1 = token T_IDENT_I ()
> -> $$ = nterm label ()
> Stack now 0 32 6 82 151 205 221 233
> Entering state 40
> Next token is token S_COM ()
> error: syntax error, unexpected S_COM, expecting S_COL (token ",") line 9: idur,iamp,iskiptime,iattack,irelease,irvbtime,irvbgain  passign   3
> Error: popping nterm label ()
> Stack now 0 32 6 82 151 205 221 233
> Error: popping nterm statementlist ()
> Stack now 0 32 6 82 151 205 221
>
>
> It seems to have decided that the idur must be a label rather than a
> variable
>
> The relevant syntax lines are
>
> ans       : ident               { $$ = $1; }
>          | ans S_COM ident     { $$ = appendToTree(csound, $1, $3); }
>          ;
>
> statement : ........
>          | label S_COL
>                {
>                    $$ = make_leaf(csound, T_LABEL, (ORCTOKEN *)$1);
>                }
>          .........
>
> label : T_OPCODE    { $$ = (TREE *)$1; }
>      | T_OPCODE0   { $$ = (TREE *)$1; }
>      .....
>
> ident     : T_IDENT_I   { $$ = make_leaf(csound, T_IDENT_I, (ORCTOKEN *)$1); }
>          | T_IDENT_K   { $$ = make_leaf(csound, T_IDENT_K, (ORCTOKEN *)$1); }
>      .....
>
> does anyone have any suggestions?  The problems seem to arise from
> labels being any word, opcode, variable or other
>
> ==John ffitch
>
> ------------------------------------------------------------------------------
> Doing More with Less: The Next Generation Virtual Desktop
> What are the key obstacles that have prevented many mid-market businesses
> from deploying virtual desktops?   How do next-generation virtual desktops
> provide companies an easier-to-deploy, easier-to-manage and more affordable
> virtual desktop model.http://www.accelacomm.com/jaw/sfnl/114/51426474/
> _______________________________________________
> Csound-devel mailing list
> Csound-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/csound-devel
>

------------------------------------------------------------------------------
Doing More with Less: The Next Generation Virtual Desktop 
What are the key obstacles that have prevented many mid-market businesses
from deploying virtual desktops?   How do next-generation virtual desktops
provide companies an easier-to-deploy, easier-to-manage and more affordable
virtual desktop model.http://www.accelacomm.com/jaw/sfnl/114/51426474/
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net