Add optional automatic conversion between .NET and JavaScript arrays

[CommonGuy]

It would be very helpful to have an optional automatic conversion between .NET and JavaScript arrays (similar to DateTime/Date and Task/Promise conversions).

Some conversation and a workaround are described in #228.

My main motivation is exposing an external .NET library (for example https://github.com/bchavez/Bogus with lots of methods) via ClearScript as a "clean" JavaScript interface. This works pretty well, with the exception of arrays. Since arrays are exposed to JavaScript as their underlying .NET type, certain JavaScript functions (such as Array.isArray or Array.prototype.find()) do not work.

One workaround would be to completely wrap the external .NET library in a custom .NET wrapper to convert all arrays to JavaScript arrays before returning them, but that is a lot of work (especially with nested objects, one would need to create new classes for each of them).

Another workaround is to return the .NET array as is, but then creating and calling a toJsArray() method from the JavaScript side. The main drawback here is that one needs to call this method everywhere and that the JavaScript "client" needs to know about it. When trying to expose a "clean" JavaScript API to users, this leads to weird cases where you need to tell users to call toJsArray() on all arrays before using them.

[ClearScriptLib]

Hi @CommonGuy,

My main motivation is exposing an external .NET library [...] via ClearScript as a "clean" JavaScript interface.

ClearScript's goal is to give script code full access to .NET's capabilities – and make it possible to use .NET to provide whatever script interface may be required. Depending on the .NET API you're starting with – and your definition of "clean" – the desired result is likely not only a matter of personal taste but also impossible to achieve without assistance from the host.

The JavaScript and .NET type systems are radically different. A .NET method, for example, can have multiple outputs, generic type parameters, overloads, default arguments, etc. There's really no objectively obvious way to expose such a method to JavaScript. And because there are also differences in things like naming conventions, fundamental types, built-in APIs, and common patterns, it's very unlikely that a default mapping that aims to maximize access would align with anyone's notion of an ideal script interface.

All that having been said, you're quite right: If you're starting with a large .NET API, reshaping it into a clean script interface may not be trivial. In fact, it's difficult to imagine that automatic array conversion is all that it would take.

Nevertheless, we've received numerous requests for that capability. We've been reluctant to add it for the following reasons:

1. The conversion would be lossy. Managed arrays are strongly typed, but they'd lose their type information when passed to the script engine. Conversely, JavaScript arrays are not strongly typed and would therefore always be converted to object[]. That would make it impossible for script code to call .NET methods that expect strongly typed arrays.
2. The conversion would be expensive, as it would incur a "round trip" to copy each element – even if the method or function being called only accesses a subset of them.

Opinions?

[handerss-spotfire]

(Just adding my opinions to the questions asked)

We have similar issues when exposing a large C# API containing endpoints which expect types like IList, IList{T}, object[], IEnumerable{T} etc. In an "ideal" scenario for us the user would populate a JS array with objects of the same type and then be able to call a method which takes a collection of that type. Having this call be O(n) is fine for us (our current solution is does essentially this by wrapping host.newArr and forcing the user to specify the type).

Having the conversion be to object[] only would not be very valuable, and losing the ability to call (some) strongly typed APIs would probably make it unusable.

[CommonGuy]

IMO the mentioned drawbacks are fine, since this would be an opt-in setting. Only users that explicitely want an automatic conversion from .NET arrays to JS arrays will set this flag. All other (and existing) users won't be affected.

As for the performance, currently I would need to copy/convert each array with a call to the ClearScript/V8 engine individually, which already isn't that good for speed. But at least in my scenario, having optimal performance is not the main goal.

Another option could be to allow "interceptors" in the conversion part of ClearScript, so users could write their own custom conversions. This may be a better option if you have lots of similar requests for other data types

[ClearScriptLib]

@handerss-spotfire said:

In an "ideal" scenario for us the user would populate a JS array with objects of the same type and then be able to call a method which takes a collection of that type.

The problem is that, in .NET – or, at least, in the method binding scheme used by C# and ClearScript – the call signature selects the method, not the other way around. An argument of type object[] can't select a method that expects a strongly typed collection.

@CommonGuy said:

Another option could be to allow "interceptors" in the conversion part of ClearScript, so users could write their own custom conversions. This may be a better option if you have lots of similar requests for other data types

We've experimented with that many times over the years but haven't found a way to do it safely and cleanly. The marshaling process between .NET and JavaScript is just too intricate.