staging/sm has a lot of tests that don't use the test262 harness

[syg]

There are calls to gc, assertEq, etc. One of the requirements to landing to staging/ is that the tests are runnable with the usual test262 harness.

@Ms2ger Can SM's harness-isms be linted against in the future to avoid landing tests that don't run?

[Ms2ger]

Looks like gc() slipped through the cracks indeed. assertEq was caught in the tests, but annoyingly not in the harness files. I've compared results with V8, so I'm confused why I didn't catch those. In any case I'll get them fixed.

[Ms2ger]

Actually, assertEq should be covered by harness/sm/non262.js. Can you give more details on what goes wrong?

[syg]

Actually, assertEq should be covered by harness/sm/non262.js. Can you give more details on what goes wrong?

Did some digging. The issue is that the order of files in the includes: array matters (at least for V8's test runner, but I imagine it does for everyone?), and sm/non262.js is being included too late in many (all?) of the tests.

To take a concrete example, sign.js#L5 has includes: [sm/non262-Math-shell.js, sm/non262-shell.js, sm/non262.js]. sm/non262-Math-shell.js#L54 calls assertEq on the toplevel, but at that point non262.js hasn't been run yet, so it fails.

If the includes: lines put sm/non262.js first, then it works.

[Ms2ger]

Oh, I finally figured out why I didn't catch it. test262-harness just concatenates the included files, and the function declaration is hoisted, so assertEq was available even though the includes were in the wrong order. I'll try to get a PR to reorder the includes up asap, and add documentation that they should be evaluated in order.