MSTest unexpectedly changes the runtime type of test data provided to parameterized tests

[scharnyw]

Describe the bug

It appears that MSTest will, under certain conditions, clandestinely change the runtime type of test data provided to parameterized tests. See repro code below.

Steps To Reproduce

Run the following tests:

[TestClass]
public class TestClass
{
    [DataTestMethod]
    [DataRow(new object[] { 1d, "test" }, 2d)]
    [DataRow(new object[] { 1.1, "test" }, 2d)]
    public void DataRowTestMethod(object[] arrayData, double doubleData)
    {
        var boxedDouble = arrayData[0];
        var type = boxedDouble.GetType();

        var unboxedDouble = (double)boxedDouble;
    }

    [DataTestMethod]
    [DynamicData(nameof(DynamicTestData))]
    public void DynamicDataTestMethod(object[] arrayData, double doubleData)
    {
        var boxedDouble = arrayData[0];
        var type = boxedDouble.GetType();

        var unboxedDouble = (double)boxedDouble;
    }

    public static IEnumerable<object[]> DynamicTestData =>
        new object[][]
        {
            new object[]
            {
                new object[] { 1d, "test" }, 2d
            },
            new object[]
            {
                new object[] { 1.1, "test" }, 2d
            },
        };
}

Expected behavior

type should be System.Double in all 4 cases, and all tests should pass with no error.

Actual behavior

type is System.Int32 in the two cases where 1d is specified as the double value, and System.Decimal in the two cases where 1.1 is specified as the value. Therefore the unboxing fails in all 4 cases.

Additional context

VS 2022
.NET 8
MSTest.TestAdapter 3.7.3
MSTest.TestFramework 3.7.3

[Youssef1313]

This is unfortunately a known behavior. The problem is that we need to serialize the test arguments in discovery, and deserialize back when executing. Currently, there is no perfect serialization mechanism that we can use.

The only workaround is to not expand the test during discovery, and do so only during execution. You can do that in two ways:

1. Apply that globally for all tests in your assembly by adding [assembly: TestDataSourceOptions(TestDataSourceUnfoldingStrategy.Fold)]

2. Apply that to individual DataRows/DynamicDatas:

   [TestMethod]
   [DataRow(new object[] { 1d, "test" }, 2d, UnfoldingStrategy = TestDataSourceUnfoldingStrategy.Fold)]
   [DataRow(new object[] { 1.1, "test" }, 2d, UnfoldingStrategy = TestDataSourceUnfoldingStrategy.Fold)]
   public void DataRowTestMethod(object[] arrayData, double doubleData)
   {
       var boxedDouble = arrayData[0];
       var type = boxedDouble.GetType();
   
       var unboxedDouble = (double)boxedDouble;
   }
   
   [TestMethod]
   [DynamicData(nameof(DynamicTestData), UnfoldingStrategy = TestDataSourceUnfoldingStrategy.Fold)]
   public void DynamicDataTestMethod(object[] arrayData, double doubleData)
   {
       var boxedDouble = arrayData[0];
       var type = boxedDouble.GetType();
   
       var unboxedDouble = (double)boxedDouble;
   }
   
   public static IEnumerable<object[]> DynamicTestData =>
       new object[][]
       {
           new object[]
           {
               new object[] { 1d, "test" }, 2d
           },
           new object[]
           {
               new object[] { 1.1, "test" }, 2d
           },
       };

   Side note: We recommend that you move from DataTestMethod to TestMethod. They are equivalent and are currently interchangeable. We may deprecate DataTestMethod in the future.

Note that "Folding" means that the data source isn't expanded during discovery. This affects how the test is shown in Test Explorer, and also affects the TRX format in that you will see multiple results associated to a single test.

I'll close this as a duplicate of #1462.