fix (-Inf)^-1 inconsistency

[oscardssmith]

fixes #55831
Turns out all we needed was to change the xnlo initialization. @nsajko mind if I steal your testset?

[nsajko]

Nice!

Here's the testset:

@testset "inconsistency, possible bug" begin
    @testset "T: $T" for T ∈ (Float16, Float32, Float64, BigFloat)
        @testset "literal `-1`" begin
            @test -0.0 === Float64(T(-Inf)^-1)
        end
        @testset "`Int(-1)`" begin
            @test -0.0 === Float64(T(-Inf)^Int(-1))
        end
    end
end

Trying it locally shows that the inconsistency is gone with the PR.

[fingolfin]

Looks sane to me. I've resolved a conflict and hope this will now pass CI