Go Generation | list<tuple<string,...>> -> map[string]...

[jordan-rash]

Per the discussion here, it appears that a list<tuple<string,...>> is the wit equivalent of a python dictionary or more specifically for my use case, a go map.

If wit isn't going to bring in a new type for dictionaries/maps, I think it would be good to have the code generation output a map[string]... when list<tuple<string,...>> is parsed.

Thanks team!!

CC: @Mossaka

[sunfishcode]

I imagine this would encounter similar obstacles; if you pass a list<tuple<string, ..>>, the bindings don't know that they keys are unique, and they also wouldn't know if the order of the list is meant to be significant.

Could you say more about your use cases? A common pattern in wit is that something which one might use a dictionary for can often be a record, with option fields for fields which may be omitted.

[jordan-rash]

Thanks for the feedback @sunfishcode!!

I agree with you that the list<tuple<string,...>> syntax is not ideal and I agree on all your points WRT uniqueness and order. I would love to eventually see a dictionary<K,V> or map<K,V> type introduced to wit.

As far as uniqueness/order goes, I know that maps in go don't promise order anyway, and keys are always unique. But I don't know how that transfers across languages...meaning, if we were to do this change, would a list<tuple<string,...>> mean the same in every language? 🤔

As far as my use-case goes, its more around preserving (where possible) idiomatic go the in wasi ecosystem. For example, currently, when I want a simple map[string]string, and I use the list<tuple<string,...>> syntax, i get something along the lines of

type DerpTypesTuple2StringStringT struct {
	F0 string // key
	F1 string // value
}

type DerpTypesIface struct {
	Values     []DerpTypesTuple2StringStringT
}

which just feels really clunky, but ultimately can do the same thing.

[jordan-rash]

Zhou just pointed out to me that I missed your point that List<Tuple<String, ...>> has an order, and can have repeated keys, which are different from a map.
Now that I have been straighten out, I agree with all your points.

That said, my biggest concern is preserving idiomatic go as much as possible and I think maps are really important to the language so it would be great to figure out how to get them in there

[sunfishcode]

I'd like to explore one level deeper: could you describe what kinds of things you most often use map[string]string for, around wit interface boundaries?

For example, in some languages, dictionaries are a common way to implement a kind of named function parameters, like divide({ numerator: 21, denominator: 3 }). If that's the kind of thing you're doing, that gives us more avenues to explore for how we might produce ergonomic bindings.

[ricochet]

Config property bags where the fields and types are not known ahead of time are the prime example.

https://github.com/wasmCloud/interfaces/blob/47825448f9c355a35453047d2cd8d5bccad60e3f/core/wasmcloud-core.smithy#L92

[jordan-rash]

Bailey saves me.

That link she shared is my exact example. In that smithy file (of which I am in the middle of converting to wit), there are 3 types that are map[string]string and as she pointed out, they are all configuration type things, specifically key/value, that are not known until runtime. I guess I should point out here, its not just map[string]string its actually map[string]interface{}

I am also thinking about things like http-headers which we would traditionally represent as a map[string][]string

A third example that comes to mind is around JSON marshaling/unmarshaling. A map[string]any would allow for the Go JSON package to be utilized without having to write custom Marshallers. This example is more of a DX QoL thing than anything else

[sunfishcode]

@jordan-rash For HTTP headers, the approach taken in wasi-http is to define (what will be) a new resource, with a get(name: string) function for looking up keys. That is one of the tools in the toolbox, and it works well in cases like HTTP where HTTP has its own semantics for how headers work, and where implementations may want to do some of the work lazily instead of providing an eagerly populated dictionary.

As you say, list<tuple<string, ...>> is another tool in the toolbox.

For map[string]interace{}, if I understand correctly, that's a dictionary of callbacks, which is a bigger topic :-).

@ricochet Looking at just that file, if I were asked to implement that interface, I wouldn't know what kinds of values I might get passed. As a user, wouldn't know what kinds of values I could pass. Is there some additional documentation for that interface somewhere?

More broadly, in some systems, changing APIs is hard, because of backwards compatibility, and because there can be a lot of things to update, and in these kinds of systems, dictionaries are often considered essential, providing the flexibility to add fields or arguments to APIs without having to change the actual APIs. However, with interface subtyping (once it's implemented), if an implementor wants to add configuration knobs, they'll be able to add new option arguments/fields to the interface, and it won't need to break compatibility. And ideally all the boilerplate needed for the new arguments/fields is auto-generated. So it might not be so difficult to have actual APIs just add new fields or new arguments when they need to.

Even more broadly, none of these is or will be the answer all of the time. There are a lot of different use cases out there. I'm aiming to discuss the options and give some context.

[ricochet]

You're essentially looking at a piece of middleware. It doesn't define the upstream API and this is an adapter for things like serialization. The other smithy files in the repo are examples of concrete values that could be passed in.

I'm unsure if subtyping could work for this but maybe a resources implementation similar to http headers might be the best fit.

[autodidaddict]

I could be misunderstanding how this works, but if a resource requires an implementor to implement the get(name:string) function, then this becomes a pretty big developer burden when the underlying storage mechanism is "just a map". Everytime I expose an interface that accepts a map, I have to code up a concrete type that supports a getter and setter. I see the value in this abstraction... but it's such a common one that is implemented in pretty much every language that can target wasm.

I think the use case that I want support for that I don't see represented is the "property bag", which is typically a map/dictionary from string -> string. Despite our best efforts in application development, there are times when we need to be able to accept an arbitrary bundle of keyed data. Sometimes people use this to help with backwards compatibility, and other times it's necessary for information exchange between disparate systems.

An ordered list of tuples isn't what I need to satisfy the above use case. Even HTTP headers don't require being ordered by key, we just need a set of the header keys that point to the values. A code generator shouldn't make the assumption that the use of list<tuple<a,b>> is destined for a hash map, because, as pointed out above, the ordered nature of that list could be required by the application.

[Mossaka]

I've created an issue to discuss various places we can improve the developer experience of generated Go bindings at #612 . I am linking this issue to it.