Handler hell?

[kamituel]

Hi Matthias!

As our app grows, we're getting more and more tired of having commands being one-directional only. That's particularly annoying when commands are used to trigger WebSockets / AJAX requests that should always (in our architecture) get an answer - either confirmation, or some error response.

Currently, the way to handle it is to do something like:

(defn user-action [{:keys [put-fn]}]
  (put-fn [:websockets/send-something ...]))

(defn handle-response [{:keys [msg-payload]}]
   ...)

It's fine in such a simple example as above, however, once we have tens of such handlers, it's getting frustrating. It's even more cumbersome if one async action should be taken after first one completes, conditionally, etc...

In a perfect world, I'd love to just write:

(defn user-action  [{:keys [put-fn2]}]
  (let [res (put-fn2 [:websockets/send-something ...])]
     ... handle res, possibly use put-fn2 again ...))

That's obviously not so easy with current systems-toolbox architecture. First of all, it'd require response to be routed back correctly to the place it originated, which means put-fn2, or switchboard routing, would need to do some magic (that could of course be done manually too, but it'd be nice to have :cmd/route-bidirectional that'd work like a pair of :cmd/route's that have :from: and :to reversed). Also, put-fn2 would need to receive the response and return it.

However, it'd make it possible to write complex asynchronous logic in a synchronous fashion, and that's a big deal. What I described here is pretty much what generators and promises offer in ES6. That's why I used a title "handler-hell" ;)

What are your thoughts about it? Is that something you'd be interested in supporting in systems-toolbox?

Cheers!

[matthiasn]

Hey Kamil,

Yes, I'm certainly interested in supporting that pattern, I think that'd be very helpful. The :cmd/route-bidirectional shouldn't be too hard, do you want to submit a PR for that? Otherwise I can look at it next week.

Regarding the put-fn2, do I understand you correctly that basically the handler would be parked until the result is back? That would be useful, do you have an idea how to implement that? PRs welcome for that one, too.

This issue is nicely timed. I recently thought more in the direction of visualizing complex routing to make systems easier to understand, but it would indeed be better if that wasn't so necessary in the first place.

Cheers,
Matthias

[kamituel]

The first one - :cmd/route-bidirectional seems easy, and orthogonal to the other one.

The second one is more interesting, and more useful too. I was thinking we could wrap handler function's body into core.async's go wrapper, and then, in put-fn2, we'd tap into receiving end's channel waiting (using channels) for the response, with an optional timeout. This pattern is described here: http://bryangilbert.com/blog/2013/07/19/escaping-callback-hell-with-core-async/, (impl. might not be optimal, but the general idea seems to be what I have in mind).

I can try to find some time to look at it, but I'm not sure I'll have it soon. If you start working on that let me know so we don't overlap, ok?

Great, that'd make some of our code much simpler, and we'd still keep the benefits of exchanging separate messages both ways (like inspectability).

[BartAdv]

I'd like to ask a question: why do we need 'local' put-fn (by local I mean that we can only fetch it in message handler). I see put-fn as a way to send well formed messages (it adds meta and stuff) to a target that's specified by a switchboard. In this light, put-fn could be library helper, and it would take a channel as an argument. And yes, our handlers would receive channels on which they could put outcoming messages and get incoming messages. I see it as a big hindrance that the whole power of core.async (one of the clojure's strength) is hidden from us!

If there's any flaw in my understanding, I'd love a clarification.