Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Oban telemetry start event is not needed + tests #76

Conversation

mrdotb
Copy link

@mrdotb mrdotb commented Aug 29, 2024

👋

The oban job metadata can be accessed from the exception event metadata so it's not needed to attach to the start event.

Testing the oban integration is not easy since Application.spec is used directly so I wrapped it inside a Utils module than I can mock in order to simulate a oban event.

@odarriba odarriba self-requested a review August 29, 2024 20:59
Copy link
Contributor

@odarriba odarriba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your time on this change! 🫶

I will mark this one as changes required because we cannot remove the start event as users may lose context that they have now on their manually reported errors.

Aside from that, I think we can avoid the mock and I'm not sure if the testing strategy is correct or not (however, taking the time to introduce tests is really appreciated 🤗 )

Comment on lines +55 to +71
defp execute_job_exception(additional_metadata \\ %{}) do
raise "Exception!"
catch
kind, reason ->
metadata =
Map.merge(sample_metadata(), %{
reason: reason,
kind: kind,
stacktrace: __STACKTRACE__
})

:telemetry.execute(
[:oban, :job, :exception],
%{duration: 123 * 1_000_000},
Map.merge(metadata, additional_metadata)
)
end
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if this is the proper way to test this. Integration tests are missing because we do not have yet a clear idea of how to test it.

ideally, we would need to spin up an Oban job using Oban itself, and ensure the library is the one that emits events.

If we do that instead, any change in Oban will break the integration and the tests will continue to pass.

Let me ping @crbelaus to see his thoughts on this.

Copy link
Author

@mrdotb mrdotb Aug 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mocking the :telemetry is the way chosen on https://github.com/appsignal/appsignal-elixir/blob/main/test/appsignal/oban_test.exs so it's a good reference

@crbelaus
Copy link
Contributor

crbelaus commented Aug 30, 2024

Thanks for dedicating your time to write this pull request, but I do not agree with this approach. I feel like this test too complicated because it is mixing multiple different concerns.

On the one hand it is testing the Telemetry package itself by checking first if an event handler is attached and then if sending an event effectively sends it to the attached handler.
Telemetry already tests this. If I call :telemetry.attach it is safe to assume that the event handler is attached, otherwise there would be a bug in the Telemetry package. Same happens with :telemetry.execute.

On the other hand we are not testing if the ErrorTracker captures the error, just that it emitted a particular Telemetry event. Something that is already tested elsewhere.


The way I see it, I would directly call ErrorTracker.Integrations.Oban.handle_event/4 with the desired information and then check if the expected behaviour happens (either the context is updated or an error is recorded).

@crbelaus crbelaus closed this Aug 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants