Upgrade test script

[arsalan0c]

With this PR, src/test/framework/main.js can be used to write tests, and run with the test script.

It checks for source-test at the first line of each test file. If present, it concatenates the source-test framework, the source file and the test file, and runs it with js-slang. Otherwise, it uses the existing test framework.

In the following example, the first three tests use the existing test framework while the last six use source-test:

Chapter and variants can also be specified in the first line of each test file, for example:

// chapter=3 variant=non-det

If they are not specified, their default values are used.

[martin-henz]

Martin Henz, [8 Apr 2020 at 9:24:43 AM]:
...Nearly there!

Note that there are several source files .js to be tested, in each folder. src//*.js
So we will need one source-test.js file for each of them. So there should be a file
src/steppers/tests/source_0._.js or something like that, instead of your
src/steppers/tests/source-test.js

Can you check in the intended source-test file in steppers/tests , also?

[arsalan0c]

Martin Henz, [8 Apr 2020 at 9:24:43 AM]:
...Nearly there!

Note that there are several source files .js to be tested, in each folder. src//*.js
So we will need one source-test.js file for each of them. So there should be a file
src/steppers/tests/source_0._.js or something like that, instead of your
src/steppers/tests/source-test.js

Can you check in the intended source-test file in steppers/tests , also?

Yes I missed that!

A suggestion: we can use the presence of a comment in the first line of the test file (eg. // source-test) in order to check whether to use source-test or the existing test framework. With this, users can stick to a single naming format for test files.

I've added this change and updated the PR description and screenshot.

[martin-henz]

We are getting very close to something beautiful, here. One more change would allow us to specify the source chapter and variant, with a command line option to the test script. We could do that with a header comment in the test program:
// chap=3 variant=non-det

This way you could use your test framework for testing your non-det examples!

[arsalan0c]

We are getting very close to something beautiful, here. One more change would allow us to specify the source chapter and variant, with a command line option to the test script. We could do that with a header comment in the test program:
// chap=3 variant=non-det

This way you could use your test framework for testing your non-det examples!

To clarify the command line option, do you mean a command line option for js-slang which takes a variant?

See source-academy/js-slang#549

[martin-henz]

Very nice!

[martin-henz]

We are getting very close to something beautiful, here. One more change would allow us to specify the source chapter and variant, with a command line option to the test script. We could do that with a header comment in the test program:
// chap=3 variant=non-det
This way you could use your test framework for testing your non-det examples!

To clarify the command line option, do you mean a command line option for js-slang which takes a variant?

See source-academy/js-slang#549

I think you did the right thing: an option in the header of the test files that specifies the chapter and variant.