Hi mond,
I've been using HTTPkit with Compojure and core.async (all fronted by Nginx
so the entire stack is async FWIW).
To glue core.async with HTTPkit when handling inbound requests, I have a
utility function
(defn handle-async! [handler req]
(http-server/with-channel req channel
(take! (handler req) #(http-server/send!
channel %))))
and I make use of that utility function when defining routes
(defroutes all-routes
(GET "/covers/v1/:cover" [] (partial http/handle-async!
fetch-cover!))
(context "/books/v1" []
(GET "/search" [] (partial http/handle-async!
search-handler!))
(GET "/:isbn" [] (partial http/handle-async!
get-book-by-isbn!))))
The handler functions fetch-cover!, search-handler!, get-book-by-isbn!, all
return channels (they either call go for stuff that needs to be async, or
to-chan on collections read directly from memory.
- Mike
On Saturday, September 27, 2014 5:01:36 AM UTC-4, mond wrote:
>
> Hi James,
>
> Er, nice tip on the routes - that was another thing that I didn't expect
> to work but was happy when it did ;-) I will of course adapt it to your
> recommendation.
>
> Speaking to the main point, no I don't want to put a channel on to the
> response so that's a mistake that I see and would like to avoid. I would
> like to understand where I have gone wrong.
>
> Maybe I need to use HTTPkit instead?
>
> Thanks
>
> Ray
>
>
>
> On Saturday, 27 September 2014 02:08:50 UTC+2, James Reeves wrote:
>>
>> Hi Ray,
>>
>> I don't entirely understand why you expected this to work. Channels
>> aren't a valid Ring response body. The error message is essentially telling
>> you that Compojure has no way of turning the channel object you've returned
>> into a valid response.
>>
>> The other problem you have is that the Ring Jetty adapter doesn't have
>> any support for asynchronous operations.
>>
>> Another small point. You're using "*" as an argument name, but this isn't
>> really recommended. This only works by coincidence, and it may be removed
>> in future versions. Instead use something like:
>>
>> (GET ["/:brand/:country/:resource" :resource #".*"] [brand country
>> resource] ...)
>>
>> - James
>>
>> On 27 September 2014 00:14, mond <[email protected]> wrote:
>>
>>> My first core.async program ... all works outside of the web app but
>>> barfs once I put the functions inside a web container. I hope somebody in
>>> the group can point to my obvious mistake...
>>>
>>> The idea is that the function 'respond-within-sla' will give back a
>>> result or a come back later message after N milliseconds. It is passed a
>>> number of ms, three functions and the arguments for the final function ...
>>> which is the one that should operate within the SLA.
>>>
>>> (defn respond-within-sla [expected-result-milliseconds respond-ok
>>> respond-later data-fetcher & args]
>>> (let [data-channel (timeout expected-result-milliseconds)]
>>> (go (if-let [data (<!! data-channel)]
>>> ((async/close! data-channel)
>>> (respond-ok data))
>>> (respond-later)))
>>> (go
>>> (>! data-channel (apply data-fetcher args)))))
>>>
>>> To keep the volume of code to parse to a minimum I have made a few toy
>>> functions that demonstrate the failure...
>>>
>>> ; test funcs
>>>
>>> (defn ok [data]
>>> (prn (str "send HTTP 200 ... got data " data)))
>>>
>>> (defn later []
>>> (prn (str "send HTTP 202 ... request received, come back later")))
>>>
>>> (defn fetcher [arg1 arg2 arg3]
>>> (prn (str "fetching data with args " arg1 " " arg2 " " arg3))
>>> "response-data")
>>>
>>> (defn failer [& args]
>>> (Thread/sleep 1000)
>>> (str "never gets here " args))
>>>
>>> ; test funcs
>>>
>>> (defn ok [data]
>>> (prn (str "send HTTP 200 ... got data " data)))
>>>
>>> (defn later []
>>> (prn (str "send HTTP 202 ... request received, come back later")))
>>>
>>> (defn fetcher [arg1 arg2 arg3]
>>> (prn (str "fetching data with args " arg1 " " arg2 " " arg3))
>>> "response-data")
>>>
>>> (defn failer [& args]
>>> (Thread/sleep 1000)
>>> (str "never gets here " args))
>>>
>>> (defn generate-response [brand country resource]
>>> (let [sla (or (env :sla-milliseconds) 100)]
>>> (respond-within-sla sla ok later fetcher brand country resource)))
>>>
>>> (defn generate-fail [brand country resource]
>>> (let [sla (or (env :sla-milliseconds) 100)]
>>> (respond-within-sla sla ok later failer brand country resource)))
>>>
>>> From within the REPL it all works fine...
>>>
>>> (generate-response "A" "B" "C")
>>> => #<ManyToManyChannel
>>> clojure.core.async.impl.channels.ManyToManyChannel@4b7ae3f7>
>>> "fetching data with args A B C"
>>> "send HTTP 200 ... got data response-data"
>>> (generate-fail "A" "B" "C")
>>> => #<ManyToManyChannel
>>> clojure.core.async.impl.channels.ManyToManyChannel@4eb8b5a9>
>>> "send HTTP 202 ... request received, come back later"
>>>
>>> Here is the compojure route..
>>>
>>> (defroutes app
>>> (GET "/:brand/:country/*" [brand country *]
>>> (generate-response brand country *))
>>>
>>> (ANY "*" []
>>> (route/not-found "You must use a REST style to specify
>>> brand and country keys in the URL")))
>>>
>>> If I now start it up 'lein run' and try to exercise the functions from
>>> the web server...
>>>
>>> $ curl -I http://localhost:5000/A/B/D.jpg
>>> HTTP/1.1 500 Server Error
>>> Date: Fri, 26 Sep 2014 23:02:03 GMT
>>> Content-Length: 0
>>> Connection: close
>>> Server: Jetty(7.6.8.v20121106)
>>>
>>> And on the server I see this:
>>>
>>> $ lein run
>>> Compiling redirector.web
>>> 2014-09-27 01:01:48.426:INFO:oejs.Server:jetty-7.6.8.v20121106
>>> 2014-09-27 01:01:48.458:INFO:oejs.AbstractConnector:Started
>>> [email protected]:5000
>>> 2014-09-27 01:02:03.535:WARN:oejs.AbstractHttpConnection:/A/B/D.jpg
>>> java.lang.IllegalArgumentException: No implementation of method: :render
>>> of protocol: #'compojure.response/Renderable found for class:
>>> clojure.core.async.impl.channels.ManyToManyChannel
>>> at clojure.core$_cache_protocol_fn.invoke(core_deftype.clj:544)
>>> at compojure.response$fn__213$G__208__220.invoke(response.clj:9)
>>> at compojure.core$make_route$fn__332.invoke(core.clj:100)
>>> at compojure.core$if_route$fn__320.invoke(core.clj:46)
>>> at compojure.core$if_method$fn__313.invoke(core.clj:33)
>>> at compojure.core$routing$fn__338.invoke(core.clj:113)
>>> at clojure.core$some.invoke(core.clj:2515)
>>> at compojure.core$routing.doInvoke(core.clj:113)
>>> at clojure.lang.RestFn.applyTo(RestFn.java:139)
>>> at clojure.core$apply.invoke(core.clj:626)
>>> at compojure.core$routes$fn__342.invoke(core.clj:118)
>>> at clojure.lang.Var.invoke(Var.java:379)
>>> at
>>> ring.middleware.keyword_params$wrap_keyword_params$fn__534.invoke(keyword_params.clj:35)
>>> at
>>> ring.middleware.nested_params$wrap_nested_params$fn__576.invoke(nested_params.clj:84)
>>> at ring.middleware.params$wrap_params$fn__507.invoke(params.clj:64)
>>> at
>>> ring.middleware.multipart_params$wrap_multipart_params$fn__612.invoke(multipart_params.clj:118)
>>> at ring.middleware.flash$wrap_flash$fn__1286.invoke(flash.clj:35)
>>> at ring.middleware.session$wrap_session$fn__1273.invoke(session.clj:98)
>>> at ring.adapter.jetty$proxy_handler$fn__1426.invoke(jetty.clj:18)
>>> at
>>> ring.adapter.jetty.proxy$org.eclipse.jetty.server.handler.AbstractHandler$ff19274a.handle(Unknown
>>>
>>> Source)
>>> at
>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
>>> at org.eclipse.jetty.server.Server.handle(Server.java:363)
>>> at
>>> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483)
>>> at
>>> org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920)
>>> at
>>> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982)
>>> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
>>> "fetching data with args A B D.jpg"
>>> at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
>>> at
>>> org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
>>> at
>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
>>> at
>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
>>> at
>>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
>>> at
>>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
>>> at java.lang.Thread.run(Thread.java:744)
>>> "send HTTP 200 ... got data response-data"
>>>
>>> FYI I have tested the functions prior to adding the core.async code and
>>> they all work fine from within Jetty and on Heroku.
>>>
>>> It seems like somehow I am getting the wrong thing put into my response.
>>>
>>> I am guessing that I have configured something wrong so here is my
>>> project.clj
>>>
>>> (defproject redirector "1.0.0-SNAPSHOT"
>>> :description "Clojure HTTP redirector"
>>> :url "http://redirector.herokuapp.com";
>>> :license {:name "Eclipse Public License v1.0"
>>> :url "http://www.eclipse.org/legal/epl-v10.html"}
>>> :dependencies [[org.clojure/clojure "1.6.0"]
>>> [compojure "1.1.9"]
>>> [ring/ring-jetty-adapter "1.2.2"]
>>> [com.novemberain/monger "2.0.0"]
>>> [com.taoensso/carmine "2.7.0"]
>>> [environ "0.5.0"]
>>> [org.clojure/core.async "0.1.346.0-17112a-alpha"]]
>>> :min-lein-version "2.0.0"
>>> :plugins [[environ/environ.lein "0.2.1"]]
>>> :hooks [environ.leiningen.hooks]
>>> :main "redirector.web"
>>> :aot :all
>>> :uberjar-name "redirector-standalone.jar"
>>> :profiles {:production {:env {:production true}}})
>>>
>>> Any ideas, help would be greatly appreciated.
>>>
>>> Thanks
>>>
>>> Ray
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to [email protected]
>>> Note that posts from new members are moderated - please be patient with
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> [email protected]
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
