feat(test): test opcode programs in different scenarios

[winsvega]

🗒️ Description

Conversion of opcode diff places tests by ori.
A test defines series of test scenarios that are run on each parametrized opcode sequence.
Then we check if the sequence worked as expected in a given scenario.

I think this is a powerful method to template test any new given opcode.
we already have pre defined scenarios. then we just add one more parameter with what we want to test, and it will be covered on all the cases automatically.

Cases can be like:
callcode->staticcall-> [opcode]
create2-> [opcode]
[opcode] -> revert

check it out. so we can define opcode programs and scenarios. then the test will put each opcode program in each scenario and verify that it's result is the same (perhaps result will be complex depending on context and fork)

the idea is so far to have a template test and then we can easily just add opcode programms and it will be run in all crazy combinations.
likce call delegate call suicide revert and so on

This is still WIP.

🔗 Related Issues

#184

✅ Checklist

• All: Set appropriate labels for the changes.
• All: Considered squashing commits to improve commit history.
• All: Added an entry to CHANGELOG.md.
• All: Considered updating the online docs in the ./docs/ directory.
• Tests: All converted JSON/YML tests from ethereum/tests have been added to converted-ethereum-tests.txt.
• Tests: A PR with removal of converted JSON/YML tests from ethereum/tests have been opened.
• Tests: Included the type and version of evm t8n tool used to locally execute test cases: e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
• Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.

[chfast]

Can you give it any better name? The "opcode diff places" has no meaning.

[winsvega]

it translates as opcode in different (logical) places

[marioevz]

A few suggestions after a quick review. I haven't checked files in the ./scenarios/ scenarios/ folder, but will do on the re-review.

[marioevz]

More comments.

I feel like the programs should be dynamic based on the fork, and the approach could be improved in the sense that the program could be a pytest.fixture that takes the fork as input and then resolves to the actual bytecode.

See the comment in the invalid_opcodes.py file regarding the list of opcodes being invalid, since that is going to be changed frequently depending on the fork, it's going to be a pain to maintain, but we have the tools in pytest to do the maintenance more automatic and seamless.

[marioevz]

I think having the inputs as a separate structure that gets used in three different places is not a great pattern.

It seems to me like we should have a single class that can generate all types of combinations:

    combinations_generator: ScenarioGenerator = ScenarioGenerator(
        fork=fork,
        pre=pre,
        operation_code=operation,
        external_address=external_address,
        external_balance=external_balance,
    )

    for combination in combinations_generator.generate_call_combinations():
        if not debug.scenario_name or combination.name == debug.scenario_name:
            scenarios_list.append(combination)

    for combination in combinations_generator.generate_create_combinations():
        if not debug.scenario_name or combination.name == debug.scenario_name:
            scenarios_list.append(combination)

    for combination in combinations_generator.generate_revert_combinations():
        if not debug.scenario_name or combination.name == debug.scenario_name:
            scenarios_list.append(combination)

[winsvega]

here generate_X is dynamic.
so ideally we iterate over array of functions and call each function with environment conditions.
and when we want to add new scenario generator it adds a function into that list

[marioevz]

This specific property, program_selfbalance, might be better off as a property of the operation.

[winsvega]

here I calculate manually how much the selfbalance is supposed to be in this context. and pass it as expected result to verifier. if program return different value while checking selfbalance there will be an error
this is equvivalent to post state defenition

[marioevz]

Suggested change

	def test_scenarios(
	@pytest.mark.execute(pytest.mark.skip(reason="gas usage"))
	def test_scenarios(

This test has to be skipped in execute mode because it uses 500 million gas.

[winsvega]

should not be that much. I need to correct

[winsvega]

can make a test per scenario. right now all scenarios combined in one test for each program.
that would be a lot of test files though, but easier for debug

[marioevz]

I think this is what you meant:

Suggested change

	data=b"0x11223344",
	data=b"\x11\x22\x33\x44",

The current one creates a byte array with the ascii equivalent of the "0x11223344" string.

[winsvega]

is there a handy constructor like BYTES("0x11223344") ?

[danceratopz]

bytes.fromhex(), example here: #1067 (comment).

[winsvega]

I see here is confusion
bytes.fromhex("11223344"),
prefix 0x is not allowed, can mistake with dec

[marioevz]

I feel like this should not be a static list, and rather should be derived from the fork properties.

[winsvega]

yes this can be built depending on fork

[winsvega]

ah, but the programs does not know about fork. I would have to refactor all programs to take fork as an argument.

[winsvega]

hm I can make this a predeployed contract in scenarios if you concern so much. but making programs a fixtures that accept fork is rather difficult design decision. programs are intended as byte sequences that are to be deployed in different scenarios and tested

[winsvega]

pytest.fixture

can you explain this part a little.
currently the program is just a bytecode that is deployed and executed inside a dynamic address that is created by scenario. depending on scenario this address or rather execution context is prepared and executed. with this we achieve following model: define bytecodeX and execute it in different contextsY.