Review and rewrite more explicit assertion logic for f3_test.go

[masih]

The f3_test.go tests the upper level F3 functionality (i.e. including pubub sub-systems, manifest propagation, etc.). It is a valuable suit of tests that have caught a number of issues in the past.

These tests, however, loosely rely on a fake clock: The EC clock is fake but that does not offer control over the progress of GPBFT itself. This is OK but it means the test should be more explicit about asserting things. For example, it is better to assert that nodes have reached some instance and a specific phase/round. As a result, it has become time consuming to debug issues when these tests fail.

There are adhoc sleeps in a number of places that should ideally be removed once a more precise eventual assertion is made about the state of each node in the test environment.

[BigLep]

@masih : is this a cleanup/improvement you would try and squeeze in before nv25 or is it realistically a potential post f3 activation item?

[masih]

I would like to get this done before activation.