Re: [gtk-list] Re: g_scanner funtions what for?

[Tim Janik <timj gtk org>]

On Tue, 16 Mar 1999, Andreas Tille wrote:

> On Fri, 12 Mar 1999, Tim Janik wrote:
> 
> > your modifications:
> > 
> > /* some test text to be fed into the scanner */
> > static const gchar *test_text =
> > ( "Datum: 02-16-1999\n"
> >   "Probenname: Hans_1\n"
> >   "Modulator f: 500.0\n"
> Changed to: Modulator_f
> >   "Schwingkreis f: 500.0\n"
> Changed to: Schwingkreis_f
> >   "Schwingkreis U: 5.0\n"
> Changed to: Schwingkreis_U
> >   "Temperatur: 25.0\n"
> >   "Trigger-Delay: -400\n"
> Changed to: Trigger_Delay
> >   "Punktzahl: 8192\n"
> >   "Messzeit: 2.0e-006\n" );
> > 
> > "Schwingkreis f" can't be scanned as a single token, because a GScanner
> > will skip spaces by default and not feature spaces as a valid symbol char.
> > 
> > when you create a new scanner, g_scanner_new() accepts a pointer to a
> > GScannerConfig structure that contains various default values for its
> > scanning behaviour. if you pass that structure as NULL, gscanner.c will
> > revert to a default structure, defined in gscanner.c:
> I hope to fit the requirements for the tokens so far.  Note that I
> didn't ended the lines with ';' (is this required?).

only if you require such tokens with e.g.

g_scanner_get_next_token (scanner);
if (scanner->token != ';')
   return ';';

> > also, you have to adapt parse_symbol() so it parses tokens other than float as
> > well, instead of:
> > 
> >   /* expect a valid symbol */
> >   g_scanner_get_next_token (scanner);
> >   symbol = scanner->token;
> >   if (symbol < SYMBOL_DATE ||
> >       symbol > SYMBOL_T)
> >     return G_TOKEN_SYMBOL;
> > 
> > ...
> Hmm, I've done that but my program dosn't reach this point.  It returns
> with error code for before that stuff.
> 
> I tried
> 
>   /* expect a float (ints are converted to floats on the fly) *or* a string */
>   g_scanner_get_next_token (scanner);
>   if (scanner->token != G_TOKEN_FLOAT || scanner->token != G_TOKEN_STRING )
>     return G_TOKEN_FLOAT;

andreas, look at this more closely, lets s/scanner->token/foo/,
s/G_TOKEN_FLOAT/1/ and s/G_TOKEN_STRING/2/:

 if (foo != 1 || foo != 2)
   ...

literally:
  IF foo is not equal to 1 OR foo is not equal to 2

so to get beyond this condition, you require scanner->token to be
==G_TOKEN_FLOAT *and* ==G_TOKEN_STRING at the same time, obviously impossible.

so if at all, you want
  if (scanner->token != G_TOKEN_FLOAT && scanner->token != G_TOKEN_STRING)

but i doubt that this is your problem, see below.

> read floats *AND* strings but failed and I get
> 
> test text:1: error: unexpected string constant "J02161999", expected number (float)
> 
> If I set "Datum: J02161999\n" (the '-' int the former string caused trouble)
> or
> 
> test text:1: error: scanner: digit is beyond radix
> 
> if I leave "Datum: 02161999\n" ... a string of spaces only.  I expect
> that I have to configure the scanner to read strings (like a date or
> a name), but I can't find any way how to do that.

you need to understand that the scanner will parse it's input and
tokenize it, it is up to you to interpret these tokens, not define
their types before they get parsed, e.g. watch gscanner parse a string:

"hi i am 17"
 |  | |  | 
 |  | |  v
 |  | v  TOKEN_INT, value: 17
 |  v TOKEN_IDENTIFIER, value: "am"
 v  TOKEN_CHAR, value: 'i'
TOKEN_IDENTIFIER, value: "hi"

if you configure the scanner with
scanner->config->int_2_float = TRUE;
scanner->config->char_2_token = TRUE;
scanner->config->scan_symbols = TRUE;

and add "am" as a symbol with
g_scanner_add_symbol (scanner, "am", "symbol value");

gscanner will parse it as

"hi i am 17"
 |  | |  | 
 |  | |  v
 |  | v  TOKEN_FLOAT, value: 17.0  (automatic int->float conversion)
 |  | TOKEN_SYMBOL, value: "symbol value"  (a successfull hash table lookup
 |  |                                       turned a TOKEN_IDENTIFIER into a
 |  |                                       TOKEN_SYMBOL and took over the
 |  v                                       symbol's value)
 v  'i'  ('i' can be a valid token as well, as all chars >0 and <256)
TOKEN_IDENTIFIER, value: "hi"

you need to match the token sequence with your code, and if you encounter
something that you don't want, you error out:

/* expect an identifier ("hi") */
g_scanner_get_next_token (scanner);
if (scanner->token != G_TOKEN_IDENTIFIER)
  return G_TOKEN_IDENTIFIER;
/* expect a token 'i' */
g_scanner_get_next_token (scanner);
if (scanner->token != 'i')
  return 'i';
/* expect a symbol ("am") */
g_scanner_get_next_token (scanner);
if (scanner->token != G_TOKEN_SYMBOL)
  return G_TOKEN_SYMBOL;
/* expect a float (17.0) */
g_scanner_get_next_token (scanner);
if (scanner->token != G_TOKEN_FLOAT)
  return G_TOKEN_FLOAT;

if you got past here, you have parsed "hi i am 17" and would have
accepted "dooh i am 42" and  "bah i am 0.75" as well, but you would
have not accepted "hi 7 am 17" or "hi i hi 17".

> 
> Kind regards

hope, this helps.

> 
>    Andreas.
> 

---
ciaoTJ