Language agnostic test suite and grammar

[nazavode]

I'm currently developing a Python FBP runtime focused on brevity meant to be a building block for more complex architectures. I wrote a PEG based parser for FBP DSL and I'm looking for as many examples as I could find to incorporate them into my test suite. What do you think about a language agnostic repo for test data just like some folks did for TOML:

https://github.com/BurntSushi/toml-test

In that repo you can find lots of valid and invalid test cases, extremely time saving for someone who's writing an encoder/decoder.

The same thing goes for the language specification: snooping around in your projects here on GitHub, I guessed that some language features are obscure to say at least, and a proper language specification is missing. This FBP language repo could become a place where to find a proper grammar (EBNF?), links to known parsers, etc.

[jonnor]

I think we should pull the test data and assertions out of the existing tests into example files, and then use them in data-driven testing. I don't think another repo is needed.

There is a PEG-based parser in here, which kinda-mostly defines the grammar. However its pretty cluttered, and the grammar is probably not the cleanest either. https://github.com/flowbased/fbp/blob/master/grammar/fbp.peg

[jonnor]

I like putting the valid cases into separate files under examples/, as this is often where people look for such things. We can have pairs of .fbp (input) and .json (expected output) files.
For erronous cases I like to test that meaningful error message is emitted. To ensure testcase stimulates the right error case, and because good error messages is very helpful when programming.
An invalid .fbp file can be the input, and it can have a comment in it describing the error (and thus the expected error message).

Pull requests very welcomed!

[jpaulm]

There is also a parser for FBP DSL written in JavaScript - https://github.com/jpaulm/parsefbp . Personally I have never found PEG-based parsers that easy to work with. ParseFBP has a simple structure, and is designed to be easy to invoke and modify. ParseFBP uses a technology called "BabelParser", which I have been using for parsing jobs for decades - see https://www.npmjs.com/package/babelparser - and can in fact be used for other parsing jobs.

[nazavode]

I like putting the valid cases into separate files under examples/, as this is often where people look for such things. We can have pairs of .fbp (input) and .json (expected output) files.

That makes a lot of sense to me, it would be great to have a way to cross-check multiple representations of the same graph.

I think we should pull the test data and assertions out of the existing tests into example files, and then use them in data-driven testing. I don't think another repo is needed.

Could you point me to some test data I can scrape for examples please? I would like to try to submit a PR.

ParseFBP uses a technology called "BabelParser", which I have been using
for parsing jobs for decades

Thanks for the note, it looks good but I was thinking about something more standard, language agnostic and possibly widespread, maybe a pure grammar, something similar to a BNF. Anyway, I agree with you about PEG, mixing lexer and grammar in the same place doesn't sound neat at all (pretty convenient though).

[nazavode]

p.s: @jonnor I don't know whether this is a GitHub glitch or something, but I see that you replied in place of my comment modifying it actually :)

[jonnor]

I think we should pull the test data and assertions out of the existing tests into example files, and then use them in data-driven testing. I don't think another repo is needed.

Could you point me to some test data I can scrape for examples please? I would like to try to submit a PR.

I think we should start with the existing tests. Each fbpData here is generally a string of input data:
https://github.com/flowbased/fbp/blob/master/spec/fbp.coffee#L68

There is a decent amount of .FBP files out there, but we don't know beforehand if these are valid or not.

https://github.com/search?p=14&q=graphs+in%3Apath+extension%3Afbp&ref=searchresults&type=Code&utf8=%E2%9C%93

[jonnor]

@nazavode my apologies! The glitch was in my brain I fear... I wanted to have the quote so I clicked edit, then must have forgotten to copy-paste into a new comment.
I have restored your comment now to best of ability, based on the email notification I got when you originally made it.

[nazavode]

@jonnor cheers :)

Thanks for the pointers, I'm going to look into these trying to come up with something useful (beginning from your own coffee unit tests I think).