Google SoC 2010
Hi! I would like to know more about the: "Create a dazzling web ui for Cassandra" issue, since, it fits my profile (IMHO). I'm a Computer Science undergraduate student in Rosario, Argentina; and I also run my own company (a "very small" business), which develops websites and mainly, web-based applications for small business (mini ERPs, etc). My main focus is user interaction, I have a strong graphic design and visual communication knowledge, which I studied solely, apart from my main career. I'm kind of what Adobe folks call a "Devigner" or "Desigloper" right now, I work doubly as a programmer (both server and client-side) and an interaction designer. I also implement lots of techniques on usability, interaction design, and I'm always following trends in that path. I'm now developing rich user interfaces, taking advantage of the nowaday browser's capabilities. So! If you may please contact me with further information on the task, I will really aprecciate it! I'm very interested in the Cassandra project, and would be great to be at least a tiny part of it!
Re: Google SoC 2010
I forgot to mention: I'm interested in the issue, even if it's "outside" the Google SoC! I would really like to push my skills to it. Anyway, in Argentina, SoC happens to be on winter. ;) On Wed, Mar 31, 2010 at 8:33 AM, Pablo Cuadrado wrote: > Hi! > > I would like to know more about the: "Create a dazzling web ui for > Cassandra" issue, since, it fits my profile (IMHO). > > I'm a Computer Science undergraduate student in Rosario, Argentina; > and I also run my own company (a "very small" business), which > develops websites and mainly, web-based applications for small > business (mini ERPs, etc). > > My main focus is user interaction, I have a strong graphic design and > visual communication knowledge, which I studied solely, apart from my > main career. I'm kind of what Adobe folks call a "Devigner" or > "Desigloper" right now, I work doubly as a programmer (both server and > client-side) and an interaction designer. I also implement lots of > techniques on usability, interaction design, and I'm always following > trends in that path. > > I'm now developing rich user interfaces, taking advantage of the > nowaday browser's capabilities. > > So! If you may please contact me with further information on the task, > I will really aprecciate it! I'm very interested in the Cassandra > project, and would be great to be at least a tiny part of it! >
Re: Google SoC 2010
Thanks! I was just taking a look at the polarrose for the JMX bridge (I used Cacti on one of my FreeBSD servers), and the diverse implementations of PHP clients I've found around the Thrift interface. I couldn't agree more, the first step is to figure out the client's communication layer. Which should be a truly independent communication layer, in case any changes / protocols emerge in the future. ( I was checking also some the "Web Services Connector for JMX (tm) Agents" from: http://jcp.org/en/jsr/detail?id=262 ...and just realized former Sun sites have the Oracle logo now! :P ) This is a great date to start thinking around, from April's 1st to 5th we have the easter holidays down here, so a lot of free time. Please tell me if some kind of "formal" application is needed, I will evaluate the needs, goals, possible solutions (servlet, REST, whatsoever) and try to deliver a rough (I couldn't stress ROUGH enough...) roadmap thinking on the final deliverable desired. - Also, just wondering: is the simulator reliable? for monitoring testing purposes... anyway, I'm just jumping wy ahead... On Wed, Mar 31, 2010 at 9:36 AM, Gary Dusbabek wrote: > Hi Pablo, > > You're free to take this anywhere you want. I've attached a few notes > to the jira ticket that should get you pointed in the right direction. > > You might wish to start by familiarizing yourself with some of the > management operations that are available (e.g. nodetool, use jconsole > to see what exposed in JMX, etc.). > > Gary. > > On Wed, Mar 31, 2010 at 06:33, Pablo Cuadrado wrote: >> Hi! >> >> I would like to know more about the: "Create a dazzling web ui for >> Cassandra" issue, since, it fits my profile (IMHO). >> >> I'm a Computer Science undergraduate student in Rosario, Argentina; >> and I also run my own company (a "very small" business), which >> develops websites and mainly, web-based applications for small >> business (mini ERPs, etc). >> >> My main focus is user interaction, I have a strong graphic design and >> visual communication knowledge, which I studied solely, apart from my >> main career. I'm kind of what Adobe folks call a "Devigner" or >> "Desigloper" right now, I work doubly as a programmer (both server and >> client-side) and an interaction designer. I also implement lots of >> techniques on usability, interaction design, and I'm always following >> trends in that path. >> >> I'm now developing rich user interfaces, taking advantage of the >> nowaday browser's capabilities. >> >> So! If you may please contact me with further information on the task, >> I will really aprecciate it! I'm very interested in the Cassandra >> project, and would be great to be at least a tiny part of it! >> >
Re: Google SoC 2010
Ok! That's new territory for me too. As for the simulator, it's also best to test monitoring on a *real* cluster, I have many machines around me all day... Thanks for the support! It's really encouraging. I'll be in touch. On Wed, Mar 31, 2010 at 10:16 AM, Gary Dusbabek wrote: > On Wed, Mar 31, 2010 at 08:03, Pablo Cuadrado wrote: >> >> This is a great date to start thinking around, from April's 1st to 5th >> we have the easter holidays down here, so a lot of free time. >> >> Please tell me if some kind of "formal" application is needed, I will >> evaluate the needs, goals, possible solutions (servlet, REST, >> whatsoever) and try to deliver a rough (I couldn't stress ROUGH >> enough...) roadmap thinking on the final deliverable desired. >> > > I believe the Apache Foundation is already listed as a mentoring > organization (I can't find a list anywhere though), so I think you > just need to submit an application to gsoc. > http://code.google.com/soc/ > > I've never participated as a student or mentor, so this is new > territory for me. I can't speak for other devs, but I would be happy > to look at your proposal before it goes to google and offer feedback. > >> >> Also, just wondering: is the simulator reliable? for monitoring >> testing purposes... anyway, I'm just jumping wy ahead... > > There was something submitted to a jira ticket a few months ago along > this line, but it never made it into contrib and hasn't progressed > afaik. I wouldn't go that route. It is pretty simple to bring up a > test cluster on a dev machine anyway. > > Gary. >
Re: Google SoC 2010
Many thanks for the guidance Gabriele! This kind of feedback is just what you're talking about! On Wed, Mar 31, 2010 at 10:24 AM, gabriele renzi wrote: > On Wed, Mar 31, 2010 at 3:03 PM, Pablo Cuadrado > wrote: > > > Please tell me if some kind of "formal" application is needed, I will >> evaluate the needs, goals, possible solutions (servlet, REST, >> whatsoever) and try to deliver a rough (I couldn't stress ROUGH >> enough...) roadmap thinking on the final deliverable desired. > > just my 2c as a former google SoC member as a student: you need to > apply through their online service, but your request will be evaluated > both from google people and the project-you-are-applying-to people, so > it _is_ a great idea to discuss it in advance with cassandra's devs > but you also have to take a bit of care in writing your application. > List intermediate measureable goals, explain why you are qualified to > do this, write down how you're gonna handle some specific issues, > describe properly what is your goal and why it will be good for the > project. > The feedback from the community is a good way to fill in these things :) > > Good luck :) >
JMX-REST interface working!
Hi there! In a first progressive step, I managed to adapt a simple (yet effective) REST interface I've found at: http://blogs.sun.com/jmxnetbeans/entry/restful_access_to_jmx_instrumentation The example was connecting to the first MBean server found on the virtual machine, so it was showing many JVM MBeans, but not the Cassandra ones. I added a JMXConnector articulation (connecting to the default 8080 port) to enable remote MBean browsing... with an awful implementation... HORRIBLE!... But it was just a simple test... It even supports reading of tabular, composite, collections, complex MBeans. Runs on it's own http server (in port 2021 in my example), this is a working URL which returns the requested data: http://localhost:2021/jmx/org.apache.cassandra.db:type=ColumnFamilyStores,name=Keyspace1,columnfamily=Standard2/MemtableDataSize So, that huge URI returns a single number!!! :P Limitations so far: - TO-DO items: Set, Invoke and Notifications handling. - The "request-per-mbean-attribute" approach is a bit rough, the web client will be doing a request for every single magnitude being measured... the interface should have more commands to retrieve related sets of information, possibly JSON formatted. - Regards!
python web framework suggestions (for Cassandra Web UI) needed
Hi! I made a proposal about building a Cassandra Web UI. One of it's main components, will be Python on the server side. However, as Gary D. pointed out, it will be interesting to get your opinions on which framework to use. I suggested Django for being well-known and largely documented, but any other would do. As far as my experience goes on web development, this is what I -IMHO- think of any web framework, despite the language: - Really small footprint is a plus: "do we really need to include that, and that, and that other thing?" - Flexibility and freedom of code, another plus: "do I really need to inherit that class to do that" - Unneeded features tend to get in our way: like the "auto admin" panels of Django. Or the "FormAlchemy" and "SQLAlchemy" features in Pylons. - Templating features should be truly flexible, and do fast template parsing. Well, suggestions are needed! I would like to know your opinions on Django, Pylons, web2py, TurboGears, etc. Regards!
Re: python web framework suggestions (for Cassandra Web UI) needed
I like Pylons also, for what I've read. Haven't worked with it so far, but I'll give it try today to see how it performs. Joseph: That's great! I'm also thinking on Lazyboy, and a restful interface. I'll take a look at it. On Fri, Apr 9, 2010 at 1:10 PM, Joseph Bowman wrote: > Way back when I wanted to try and use node.js and Cassandra, I started work > on a restful interface using Tornado and Lazyboy. I've since moved on from > that idea and the project is way out of date, but you can see what I had > done at this project on github - http://github.com/joerussbowman/jsondra > > On Fri, Apr 9, 2010 at 11:49 AM, Matthew Dennis wrote: > >> +1 for pylons, I've been quite happy with it so far - lightweight, very >> flexible, loosely coupled components... >> >> >> On Apr 9, 2010, at 10:23 AM, Gary Dusbabek wrote: >> >> I like pylons. Easy templating and relatively light weight. In my >>> experience, it was easier to get something working in pylons than >>> django, but I am impatient. >>> >>> Gary. >>> >>> >>> On Fri, Apr 9, 2010 at 09:55, Pablo Cuadrado >>> wrote: >>> >>>> Hi! >>>> >>>> I made a proposal about building a Cassandra Web UI. One of it's main >>>> components, will be Python on the server side. >>>> >>>> However, as Gary D. pointed out, it will be interesting to get your >>>> opinions on which framework to use. >>>> >>>> I suggested Django for being well-known and largely documented, but >>>> any other would do. >>>> >>>> As far as my experience goes on web development, this is what I -IMHO- >>>> think of any web framework, despite the language: >>>> >>>> - Really small footprint is a plus: "do we really need to include >>>> that, and that, and that other thing?" >>>> - Flexibility and freedom of code, another plus: "do I really need to >>>> inherit that class to do that" >>>> - Unneeded features tend to get in our way: like the "auto admin" >>>> panels of Django. Or the "FormAlchemy" and "SQLAlchemy" features in >>>> Pylons. >>>> - Templating features should be truly flexible, and do fast template >>>> parsing. >>>> >>>> Well, suggestions are needed! I would like to know your opinions on >>>> Django, Pylons, web2py, TurboGears, etc. >>>> >>>> Regards! >>>> >>>> >
Re: python web framework suggestions (for Cassandra Web UI) needed
Joseph: Of course, I understand it's out of date but I'm sure it worths a look! Dan: You're right, it looks like Pylons is more suitable. Some pro's I see: - Mako seems to be a faster template engine than Django's one. - Looks to be really WSGI oriented from scratch. - As for the ORM, it just won't be suitable for Cassandra, ORMs nowadays are built obviously to work on the usual RDBMS... Now, I was just thinking, in terms of "objects" and "relational mapping", I'm guessing that "object mapping" is even easier in Cassandra than in a complex relational database. I had some nightmares working with DoctrineORM and PHP with largely normalized complex databases... On Fri, Apr 9, 2010 at 1:29 PM, Dan Di Spaltro wrote: > I like Django. Its wide adoption, great docs and included batteries > make it an easy sell. > > But what your describing is more like a pylons, aka if you dont want > an orm in Pylons, don't include it. > > On Fri, Apr 9, 2010 at 8:49 AM, Matthew Dennis wrote: >> +1 for pylons, I've been quite happy with it so far - lightweight, very >> flexible, loosely coupled components... >> >> On Apr 9, 2010, at 10:23 AM, Gary Dusbabek wrote: >> >>> I like pylons. Easy templating and relatively light weight. In my >>> experience, it was easier to get something working in pylons than >>> django, but I am impatient. >>> >>> Gary. >>> >>> >>> On Fri, Apr 9, 2010 at 09:55, Pablo Cuadrado >>> wrote: >>>> >>>> Hi! >>>> >>>> I made a proposal about building a Cassandra Web UI. One of it's main >>>> components, will be Python on the server side. >>>> >>>> However, as Gary D. pointed out, it will be interesting to get your >>>> opinions on which framework to use. >>>> >>>> I suggested Django for being well-known and largely documented, but >>>> any other would do. >>>> >>>> As far as my experience goes on web development, this is what I -IMHO- >>>> think of any web framework, despite the language: >>>> >>>> - Really small footprint is a plus: "do we really need to include >>>> that, and that, and that other thing?" >>>> - Flexibility and freedom of code, another plus: "do I really need to >>>> inherit that class to do that" >>>> - Unneeded features tend to get in our way: like the "auto admin" >>>> panels of Django. Or the "FormAlchemy" and "SQLAlchemy" features in >>>> Pylons. >>>> - Templating features should be truly flexible, and do fast template >>>> parsing. >>>> >>>> Well, suggestions are needed! I would like to know your opinions on >>>> Django, Pylons, web2py, TurboGears, etc. >>>> >>>> Regards! >>>> >> > > > > -- > Dan Di Spaltro >
Re: python web framework suggestions (for Cassandra Web UI) needed
Gabriele: Yes, the idea is to make it light-weighted. However, I may add: it would be nice (for us all) to use a framework which the community feels comfortable with. I'm trying to find a balance between features and footprint, having a small footprint is very important, but also, we want something scalable for adding features on next versions of the UI. Sessions, IMHO, are useful in many ways on web interfaces, for example, in user authentication (which the UI should have), preferences, etc. On Fri, Apr 9, 2010 at 1:42 PM, gabriele renzi wrote: > On Fri, Apr 9, 2010 at 4:55 PM, Pablo Cuadrado > wrote: >> - Really small footprint is a plus: "do we really need to include >> that, and that, and that other thing?" > > as I can imagine your app won't have any state per se, so you don't > have any DB issues, you probably won't even need sessions, why not use > simpler environments? I loved CherryPy some years ago, and there are > plenty of new microframeworks such as Bottle which seem more fitting > to _this_ bullet point than django and pylons. >
Re: python web framework suggestions (for Cassandra Web UI) needed
Joseph: Is it somehow similar to Twisted? am I wrong? On Fri, Apr 9, 2010 at 1:55 PM, Joseph Bowman wrote: > Well Tornado is light weight, it is it's own web server as well, so no need > to run something like apache in front of it, and is a nice light framework. > It's an eventd style process, so supports lots of connections very well, > which would give you more flexibility is designing clients to work with it. > > http://www.tornadoweb.org/ > > On Fri, Apr 9, 2010 at 12:51 PM, Pablo Cuadrado > wrote: > >> Gabriele: >> >> Yes, the idea is to make it light-weighted. However, I may add: it >> would be nice (for us all) to use a framework which the community >> feels comfortable with. >> >> I'm trying to find a balance between features and footprint, having a >> small footprint is very important, but also, we want something >> scalable for adding features on next versions of the UI. >> >> Sessions, IMHO, are useful in many ways on web interfaces, for >> example, in user authentication (which the UI should have), >> preferences, etc. >> >> On Fri, Apr 9, 2010 at 1:42 PM, gabriele renzi wrote: >> > On Fri, Apr 9, 2010 at 4:55 PM, Pablo Cuadrado >> wrote: >> >> - Really small footprint is a plus: "do we really need to include >> >> that, and that, and that other thing?" >> > >> > as I can imagine your app won't have any state per se, so you don't >> > have any DB issues, you probably won't even need sessions, why not use >> > simpler environments? I loved CherryPy some years ago, and there are >> > plenty of new microframeworks such as Bottle which seem more fitting >> > to _this_ bullet point than django and pylons. >> > >> >
Re: python web framework suggestions (for Cassandra Web UI) needed
Yes, I'm planning on Lazyboy. The Performance part on the Tornado wiki is quite impressive. Do you think it's accurate? http://www.tornadoweb.org/documentation#performance On Fri, Apr 9, 2010 at 2:02 PM, Joseph Bowman wrote: > A little different approach than Twisted, a lot less there, and yea no > thrift generator, but if you plan on using Lazyboy you'd be fine. > > On Fri, Apr 9, 2010 at 12:59 PM, Brandon Williams wrote: > >> On Fri, Apr 9, 2010 at 11:57 AM, Pablo Cuadrado > >wrote: >> >> > Joseph: >> > >> > Is it somehow similar to Twisted? am I wrong? >> >> >> Yes, minus every protocol other than HTTP, daemonization utils, etc. Oh, >> and thrift doesn't have a generator for it last I checked. >> >> -Brandon >> >
Re: python web framework suggestions (for Cassandra Web UI) needed
It is indeed a web framework, and made for sys admins to interact with Cassandra, not for hosting millions of users concurrently. And you're right: those are helloworld benchmarks. I was concerned a few days ago about the sync/async issue, browsing over examples on Telephus, Twissandra, Lazyboy, Pycassa... then I thought that Lazyboy is largely being used in production AFAIK, so I've just kept it in my mind. However, the communication layer for the web UI, should (and hopefully it will) be independent, in case we want to make this changes in the future. On Fri, Apr 9, 2010 at 2:10 PM, Joseph Bowman wrote: > I don't really consider any hello world benchmarks valid, you'd want to > investigate what your implementation would entail in different frameworks > and do mini-benchmarks to validate which is faster. But, if it's just a web > framework, as Brandon said, I doubt performance will matter to any great > degree. You'd be more concerned about Cassandra's performance, which is > pretty darn good. > > On Fri, Apr 9, 2010 at 1:07 PM, Brandon Williams wrote: > >> On Fri, Apr 9, 2010 at 12:04 PM, Pablo Cuadrado > >wrote: >> >> > Yes, I'm planning on Lazyboy. >> > >> > The Performance part on the Tornado wiki is quite impressive. Do you >> > think it's accurate? >> > >> > http://www.tornadoweb.org/documentation#performance >> >> >> Using Lazyboy, you'd be mixing blocking sockets with a nonblocking event >> loop, so performance is likely less than optimal. That said, I doubt >> performance is a concern with a web UI. >> >> -Brandon >> >
Re: python web framework suggestions (for Cassandra Web UI) needed
So far, I think I'm going for Pylons, the discussion gave this "pseudo-votes": Pylons +3 Tornado +1 Django +1 Undetermined +2 Any further comments on this issue will be appreciated! Thanks! On Sun, Apr 11, 2010 at 4:58 AM, Eric Florenzano wrote: > Bottom line with this kind of a project is to go with what you're most > familiar with. If you're equally unfamiliar with all frameworks, then the > quality of documentation becomes more important. > > Personally, I'd take a hard look at Werkzeug--it's a library, not a > framework. Which means you get to pick and choose what bits you want in a > sort of a-la-carte way. In the end, similarly to Pylons or Django, you get > a WSGI app that can be served out of the many different WSGI-aware web > servers like Apache's mod_wsgi, gunicorn, cherrypy, or even the builtin > wsgiref from the standard library. > > Anyway, I'm not sure if that helps or makes things more confusing :) > > Thanks, > Eric Florenzano > > On Fri, Apr 9, 2010 at 10:23 AM, Pablo Cuadrado > wrote: > >> It is indeed a web framework, and made for sys admins to interact with >> Cassandra, not for hosting millions of users concurrently. >> >> And you're right: those are helloworld benchmarks. >> >> I was concerned a few days ago about the sync/async issue, browsing >> over examples on Telephus, Twissandra, Lazyboy, Pycassa... then I >> thought that Lazyboy is largely being used in production AFAIK, so >> I've just kept it in my mind. >> >> However, the communication layer for the web UI, should (and hopefully >> it will) be independent, in case we want to make this changes in the >> future. >> >> On Fri, Apr 9, 2010 at 2:10 PM, Joseph Bowman >> wrote: >> > I don't really consider any hello world benchmarks valid, you'd want to >> > investigate what your implementation would entail in different frameworks >> > and do mini-benchmarks to validate which is faster. But, if it's just a >> web >> > framework, as Brandon said, I doubt performance will matter to any great >> > degree. You'd be more concerned about Cassandra's performance, which is >> > pretty darn good. >> > >> > On Fri, Apr 9, 2010 at 1:07 PM, Brandon Williams >> wrote: >> > >> >> On Fri, Apr 9, 2010 at 12:04 PM, Pablo Cuadrado < >> pablocuadr...@gmail.com >> >> >wrote: >> >> >> >> > Yes, I'm planning on Lazyboy. >> >> > >> >> > The Performance part on the Tornado wiki is quite impressive. Do you >> >> > think it's accurate? >> >> > >> >> > http://www.tornadoweb.org/documentation#performance >> >> >> >> >> >> Using Lazyboy, you'd be mixing blocking sockets with a nonblocking event >> >> loop, so performance is likely less than optimal. That said, I doubt >> >> performance is a concern with a web UI. >> >> >> >> -Brandon >> >> >> > >> >
Re: python web framework suggestions (for Cassandra Web UI) needed
Some thoughts I've had last night while handling a beer: A web UI for Cassandra, will be sometimes querying remote APIs (JMX, Thrift), for every node in the cluster. So, about the non-blocking/blocking issue that Brandon pointed out: - In a totally synchronous/blocking approach, I expect this scenario: request_some_value_for_every_node: for every_node request_value_from_api_and_wait (this is a request itself) add_value_to_some_list return list_of_values -- In a more async/non-blocking approach, this other case: request_some_value_for_every_node(callback: when_its_done): for every_node request_value_from_api_and_wait add_value_to_some_list when_its_done: return list_of_values --- I wast just wondering if we should leave async servers/frameworks out of the question, or, in the opposite, think about how we can get a benefit out of them, having in mind that the UI will be querying API methods for nodes in an entire cluster sometimes. Here's some info about various servers including Tornado: http://nichol.as/asynchronous-servers-in-python On Mon, Apr 12, 2010 at 9:05 AM, Pablo Cuadrado wrote: > So far, I think I'm going for Pylons, the discussion gave this "pseudo-votes": > > Pylons +3 > Tornado +1 > Django +1 > Undetermined +2 > > Any further comments on this issue will be appreciated! > > Thanks! > > > On Sun, Apr 11, 2010 at 4:58 AM, Eric Florenzano wrote: >> Bottom line with this kind of a project is to go with what you're most >> familiar with. If you're equally unfamiliar with all frameworks, then the >> quality of documentation becomes more important. >> >> Personally, I'd take a hard look at Werkzeug--it's a library, not a >> framework. Which means you get to pick and choose what bits you want in a >> sort of a-la-carte way. In the end, similarly to Pylons or Django, you get >> a WSGI app that can be served out of the many different WSGI-aware web >> servers like Apache's mod_wsgi, gunicorn, cherrypy, or even the builtin >> wsgiref from the standard library. >> >> Anyway, I'm not sure if that helps or makes things more confusing :) >> >> Thanks, >> Eric Florenzano >> >> On Fri, Apr 9, 2010 at 10:23 AM, Pablo Cuadrado >> wrote: >> >>> It is indeed a web framework, and made for sys admins to interact with >>> Cassandra, not for hosting millions of users concurrently. >>> >>> And you're right: those are helloworld benchmarks. >>> >>> I was concerned a few days ago about the sync/async issue, browsing >>> over examples on Telephus, Twissandra, Lazyboy, Pycassa... then I >>> thought that Lazyboy is largely being used in production AFAIK, so >>> I've just kept it in my mind. >>> >>> However, the communication layer for the web UI, should (and hopefully >>> it will) be independent, in case we want to make this changes in the >>> future. >>> >>> On Fri, Apr 9, 2010 at 2:10 PM, Joseph Bowman >>> wrote: >>> > I don't really consider any hello world benchmarks valid, you'd want to >>> > investigate what your implementation would entail in different frameworks >>> > and do mini-benchmarks to validate which is faster. But, if it's just a >>> web >>> > framework, as Brandon said, I doubt performance will matter to any great >>> > degree. You'd be more concerned about Cassandra's performance, which is >>> > pretty darn good. >>> > >>> > On Fri, Apr 9, 2010 at 1:07 PM, Brandon Williams >>> wrote: >>> > >>> >> On Fri, Apr 9, 2010 at 12:04 PM, Pablo Cuadrado < >>> pablocuadr...@gmail.com >>> >> >wrote: >>> >> >>> >> > Yes, I'm planning on Lazyboy. >>> >> > >>> >> > The Performance part on the Tornado wiki is quite impressive. Do you >>> >> > think it's accurate? >>> >> > >>> >> > http://www.tornadoweb.org/documentation#performance >>> >> >>> >> >>> >> Using Lazyboy, you'd be mixing blocking sockets with a nonblocking event >>> >> loop, so performance is likely less than optimal. That said, I doubt >>> >> performance is a concern with a web UI. >>> >> >>> >> -Brandon >>> >> >>> > >>> >> >
Re: python web framework suggestions (for Cassandra Web UI) needed
David: Personally, I'm between Tornado and Pylons. I'm willing to run a few test cases on both scenarios. On Tue, Apr 13, 2010 at 2:27 PM, David Timothy Strauss wrote: > Please use something like Tornado. A Cassandra web GUI should not require a > heavyweight framework and hopefully nothing other than a Cassandra connection > to run. > > -Original Message----- > From: Pablo Cuadrado > Date: Fri, 9 Apr 2010 11:55:46 > To: > Subject: python web framework suggestions (for Cassandra Web UI) needed > > Hi! > > I made a proposal about building a Cassandra Web UI. One of it's main > components, will be Python on the server side. > > However, as Gary D. pointed out, it will be interesting to get your > opinions on which framework to use. > > I suggested Django for being well-known and largely documented, but > any other would do. > > As far as my experience goes on web development, this is what I -IMHO- > think of any web framework, despite the language: > > - Really small footprint is a plus: "do we really need to include > that, and that, and that other thing?" > - Flexibility and freedom of code, another plus: "do I really need to > inherit that class to do that" > - Unneeded features tend to get in our way: like the "auto admin" > panels of Django. Or the "FormAlchemy" and "SQLAlchemy" features in > Pylons. > - Templating features should be truly flexible, and do fast template parsing. > > Well, suggestions are needed! I would like to know your opinions on > Django, Pylons, web2py, TurboGears, etc. > > Regards! >
Re: python web framework suggestions (for Cassandra Web UI) needed
Well, so far we are: Pylons +3 Tornado +2 Django +1 Undetermined +2 I will do some tests and further research, and then I'll see if I change my vote (now in Pylons) to Tornado. On Tue, Apr 13, 2010 at 3:04 PM, David Timothy Strauss wrote: > I would be happy to help get this going on Tornado. I don't have experience > with Pylons. > > -Original Message- > From: Pablo Cuadrado > Date: Tue, 13 Apr 2010 14:33:11 > To: ; > Subject: Re: python web framework suggestions (for Cassandra Web UI) needed > > David: > > Personally, I'm between Tornado and Pylons. I'm willing to run a few > test cases on both scenarios. > > On Tue, Apr 13, 2010 at 2:27 PM, David Timothy Strauss > wrote: >> Please use something like Tornado. A Cassandra web GUI should not require a >> heavyweight framework and hopefully nothing other than a Cassandra >> connection to run. >> >> -Original Message- >> From: Pablo Cuadrado >> Date: Fri, 9 Apr 2010 11:55:46 >> To: >> Subject: python web framework suggestions (for Cassandra Web UI) needed >> >> Hi! >> >> I made a proposal about building a Cassandra Web UI. One of it's main >> components, will be Python on the server side. >> >> However, as Gary D. pointed out, it will be interesting to get your >> opinions on which framework to use. >> >> I suggested Django for being well-known and largely documented, but >> any other would do. >> >> As far as my experience goes on web development, this is what I -IMHO- >> think of any web framework, despite the language: >> >> - Really small footprint is a plus: "do we really need to include >> that, and that, and that other thing?" >> - Flexibility and freedom of code, another plus: "do I really need to >> inherit that class to do that" >> - Unneeded features tend to get in our way: like the "auto admin" >> panels of Django. Or the "FormAlchemy" and "SQLAlchemy" features in >> Pylons. >> - Templating features should be truly flexible, and do fast template parsing. >> >> Well, suggestions are needed! I would like to know your opinions on >> Django, Pylons, web2py, TurboGears, etc. >> >> Regards! >> >
Re: python web framework suggestions (for Cassandra Web UI) needed
Pylons +4 Tornado +2 Django +1 Undetermined +2 Do I smell Pylons? :) On Tue, Apr 13, 2010 at 3:13 PM, Brandon Williams wrote: > On Tue, Apr 13, 2010 at 1:09 PM, Pablo Cuadrado > wrote: > >> Well, so far we are: >> >> Pylons +3 >> Tornado +2 >> Django +1 >> Undetermined +2 >> > > +1 for pylons. I think a more popular framework is likely to have better > chances of being maintained and avoiding bitrot in the long haul. > > -Brandon >
Re: python web framework suggestions (for Cassandra Web UI) needed
Yes, in the big picture, I guess Pylons has a greater community and adoption. On Tue, Apr 13, 2010 at 4:47 PM, Miguel Verde wrote: > -1 vote for Tornado. As far as I know, it still doesn't run on Windows and > has had only 2 releases. I don't think it makes sense to push that as a > dependency to potential deployers of this UI. > > There are many alternatives with broader platform support and more > consistent mainline releases: Bottle, CherryPy, Django, Paste/Pylons, > Werkzeug > On Tue, Apr 13, 2010 at 2:37 PM, Paul Bohm wrote: > >> tornado +1 - it's lightweight and has few external dependencies. >> >> Pylons +4 >> Tornado +3 >> Django +1 >> Undetermined +2 >> >> On Tue, Apr 13, 2010 at 11:29 AM, Ray Slakinski >> wrote: >> > I'll put in a vote for django, you can use managers to handle a lot of >> the object code. >> > >> > Ray Slakinski >> > >> > On 2010-04-13, at 2:15 PM, Pablo Cuadrado wrote: >> > >> >> Pylons +4 >> >> Tornado +2 >> >> Django +1 >> >> Undetermined +2 >> >> >> >> Do I smell Pylons? :) >> >> >> >> On Tue, Apr 13, 2010 at 3:13 PM, Brandon Williams >> wrote: >> >>> On Tue, Apr 13, 2010 at 1:09 PM, Pablo Cuadrado < >> pablocuadr...@gmail.com>wrote: >> >>> >> >>>> Well, so far we are: >> >>>> >> >>>> Pylons +3 >> >>>> Tornado +2 >> >>>> Django +1 >> >>>> Undetermined +2 >> >>>> >> >>> >> >>> +1 for pylons. I think a more popular framework is likely to have >> better >> >>> chances of being maintained and avoiding bitrot in the long haul. >> >>> >> >>> -Brandon >> >>> >> > >> > >> >
Re: python web framework suggestions (for Cassandra Web UI) needed
Todd: Very valuable comments, that's just the kind of problem I believe should be addressed with an asynchronous approach. On Tue, Apr 13, 2010 at 5:20 PM, Todd Lipcon wrote: > Not going to vote either way, as I haven't contributed to Cassandra in over > a year :) But one quick opinion: > > The big downside I see with Django and other WSGI based frameworks is that > it becomes difficult to run "background threads" in a clean way. Someone > mentioned this earlier, but worth repeating: there will be operations that > the management console wants to do that involve talking to all Cassandra > nodes. You will not want to do these for every request - rather it makes > sense to poll nodes in the background and maintain some in-memory data > structure. > > I think doing this kind of background monitoring in an async framework like > Tornado or Twisted is likely to be a lot easier. In building Cloudera > Desktop we ran into this issue and separated our health monitoring component > into a twisted service which then exposes data to the Django frontend. Had > Tornado been around when we started the Desktop project we might have > considered it instead. > > Thanks > -Todd > > On Tue, Apr 13, 2010 at 12:51 PM, Pablo Cuadrado > wrote: > >> Yes, in the big picture, I guess Pylons has a greater community and >> adoption. >> >> On Tue, Apr 13, 2010 at 4:47 PM, Miguel Verde >> wrote: >> > -1 vote for Tornado. As far as I know, it still doesn't run on Windows >> and >> > has had only 2 releases. I don't think it makes sense to push that as a >> > dependency to potential deployers of this UI. >> > >> > There are many alternatives with broader platform support and more >> > consistent mainline releases: Bottle, CherryPy, Django, Paste/Pylons, >> > Werkzeug >> > On Tue, Apr 13, 2010 at 2:37 PM, Paul Bohm wrote: >> > >> >> tornado +1 - it's lightweight and has few external dependencies. >> >> >> >> Pylons +4 >> >> Tornado +3 >> >> Django +1 >> >> Undetermined +2 >> >> >> >> On Tue, Apr 13, 2010 at 11:29 AM, Ray Slakinski < >> ray.slakin...@gmail.com> >> >> wrote: >> >> > I'll put in a vote for django, you can use managers to handle a lot of >> >> the object code. >> >> > >> >> > Ray Slakinski >> >> > >> >> > On 2010-04-13, at 2:15 PM, Pablo Cuadrado wrote: >> >> > >> >> >> Pylons +4 >> >> >> Tornado +2 >> >> >> Django +1 >> >> >> Undetermined +2 >> >> >> >> >> >> Do I smell Pylons? :) >> >> >> >> >> >> On Tue, Apr 13, 2010 at 3:13 PM, Brandon Williams >> >> wrote: >> >> >>> On Tue, Apr 13, 2010 at 1:09 PM, Pablo Cuadrado < >> >> pablocuadr...@gmail.com>wrote: >> >> >>> >> >> >>>> Well, so far we are: >> >> >>>> >> >> >>>> Pylons +3 >> >> >>>> Tornado +2 >> >> >>>> Django +1 >> >> >>>> Undetermined +2 >> >> >>>> >> >> >>> >> >> >>> +1 for pylons. I think a more popular framework is likely to have >> >> better >> >> >>> chances of being maintained and avoiding bitrot in the long haul. >> >> >>> >> >> >>> -Brandon >> >> >>> >> >> > >> >> > >> >> >> > >> > > > > -- > Todd Lipcon > Software Engineer, Cloudera >
Re: python web framework suggestions (for Cassandra Web UI) needed
People: The "votation" is not relevant, the discussion is, and the reasons that support each vote. And those are itself part of the research I'm doing, and it surely will help in making the decision. But, they are an important part, because I didn't want to make a decision in "solo", since this is a project relevant to any Cassandra user. So, let's not discuss whether to vote or not, but expose why this or that framework should be used. On Tue, Apr 13, 2010 at 5:29 PM, Eric Florenzano wrote: > On Tue, Apr 13, 2010 at 1:20 PM, Todd Lipcon wrote: >> >> The big downside I see with Django and other WSGI based frameworks is that >> it becomes difficult to run "background threads" in a clean way. > > > This can be handled by using Eventlet or GEvent. See the gevent webchat > demo, for example, to see how it can be done cleanly: > http://bitbucket.org/denis/gevent/src/tip/examples/webchat/ > > Thanks, > Eric Florenzano >
Re: python web framework suggestions (for Cassandra Web UI) needed
Well, there's almost a week since the first message in this thread! Arriving some conclusion, I think I'm going with Pylons for the UI. Asynchronous behaviours (if needed) can be achieved with some integration with another server like Tornado. There's even some experiments of asynchronous modules in Pylons with Cogen or a WSGI asynchronous framework. Hopefully, the communication modules (between the UI and the APIs) would be encapsulated enough to be independent from the UI implementation. Thanks to all of you for your opinions! On Tue, Apr 13, 2010 at 6:28 PM, Pablo Cuadrado wrote: > People: > > The "votation" is not relevant, the discussion is, and the reasons > that support each vote. And those are itself part of the research I'm > doing, and it surely will help in making the decision. > > But, they are an important part, because I didn't want to make a > decision in "solo", since this is a project relevant to any Cassandra > user. > > So, let's not discuss whether to vote or not, but expose why this or > that framework should be used. > > On Tue, Apr 13, 2010 at 5:29 PM, Eric Florenzano wrote: >> On Tue, Apr 13, 2010 at 1:20 PM, Todd Lipcon wrote: >>> >>> The big downside I see with Django and other WSGI based frameworks is that >>> it becomes difficult to run "background threads" in a clean way. >> >> >> This can be handled by using Eventlet or GEvent. See the gevent webchat >> demo, for example, to see how it can be done cleanly: >> http://bitbucket.org/denis/gevent/src/tip/examples/webchat/ >> >> Thanks, >> Eric Florenzano >> >
Re: admin web UI
Hi! I was working on a separate monitoring client, and not a per-node approach. At the time, we talked on having a single server that queries the nodes on a Cassandra cluster. There's some discussions on the list, about technologies being used, etc. I was pretty busy these last weeks (and some days out of town), I'm actually working on a python based UI on the server side, and got some time for testing pylons and tornado for two different approaches, one with synchronous and one with asynchronous requests. Also, someone just told me of another JMX/Rest interface, different from the one I was going to use (and adapt). You can find an architecture draft here: http://www.estudioquadra.com/gsoc2010/ At first time Django was proposed (just for being the most well-known), but it's oversized, and later we decided to go with Pylons. On Tue, May 4, 2010 at 11:21 AM, Eric Evans wrote: > On Tue, 2010-05-04 at 08:41 +0300, Ran Tavory wrote: >> How about the following compromise: >> Add a simple web server to each node with only one simple servlet that >> simply spits out all JMX stats on one page. Not fancy, no graphs, >> simply the same values you can get from jconsole, but on a web page. >> To me it seems like a fair tradeoff b/w maintenance and easier out of >> the box management. Shooting up jconsole for each server is >> cumbersome, at least in the environment I work in (firewalls, high >> latency etc) so a web interface can be nice. > > It still seems superfluous to me, but I'd be open to something > fire-and-forget (i.e. wouldn't need updating each time something new was > added). > > -- > Eric Evans > eev...@rackspace.com > >
Re: admin web UI
Anthony, very nice to see this, this is exactly the kind of thing I've started on a web UI! I'll try to get more things done this week and post it somewhere for those who are interested. On Tue, May 4, 2010 at 2:49 PM, Anthony Molinaro wrote: > And just to show you what the dashboards look like here's a couple > of screen shots > > Jconsole like page of jvm stats > http://herbie.ddv.com/~anthonym/mondemand-2.png > > Cassandra specific memtable stats > http://herbie.ddv.com/~anthonym/mondemand-1.png > > -Anthony > > On Tue, May 04, 2010 at 10:03:52AM -0700, Michael Lum wrote: >> On 5/4/2010 7:21 AM, Eric Evans wrote: >> >On Tue, 2010-05-04 at 08:41 +0300, Ran Tavory wrote: >> >>How about the following compromise: >> >>Add a simple web server to each node with only one simple servlet that >> >>simply spits out all JMX stats on one page. Not fancy, no graphs, >> >>simply the same values you can get from jconsole, but on a web page. >> >>To me it seems like a fair tradeoff b/w maintenance and easier out of >> >>the box management. Shooting up jconsole for each server is >> >>cumbersome, at least in the environment I work in (firewalls, high >> >>latency etc) so a web interface can be nice. >> > >> >It still seems superfluous to me, but I'd be open to something >> >fire-and-forget (i.e. wouldn't need updating each time something new was >> >added). >> >> This is how we monitor our Cassandra clusters. Each Cassandra node runs >> a process that polls the JMX stats and then fires off events to a set of >> configured management nodes using either UDP or multicast, depending on >> the network. New Cassandra nodes in the same cluster and datacenter >> have the same config (and are configured centrally anyways), and the >> management nodes automatically add new nodes based on the events they >> receive, so all the graphs, dashboards, monitors, and downstream tools >> pick all of this up without needing a change. This way we don't need to >> fire up jconsole for hundreds of nodes and can do other interesting >> cluster-wide aggregations. Also, we don't have to remember to setup >> monitoring when the cluster grows. >> >> All the tools used are open source, and I'd be happy to share more >> detail if there is interest. > > -- > > Anthony Molinaro >
Re: admin web UI
Of course, it would very nice to have some tool out of the box. For the 2nd point, an external tool can be fast enough to know what's happening now. There could be a cluster monitoring part, and a per-node monitoring interface, from the same tool. That was my goal at least... On Tue, May 4, 2010 at 3:05 PM, Ran Tavory wrote: > IMO there's a good case for both external monitoring tools and per-host > minimalistic interface but I see Eric's point that every piece of code will > require its maintenance. A cluster monitoring tool is definitely required. > An embedded one has two nice properties: > 1. It works out of the box > 2. If I want to know NOW what's going on on the host, I don't have to poll > like crazy (or wait 5 minutes for the next sample), I can just open the > browser and see. > > I'll see if I can get something simple running and ping back, but I can't > commit on a timeline. > Is adding jetty acceptable? Any other preferences? > > > On Tue, May 4, 2010 at 8:49 PM, Anthony Molinaro < > antho...@alumni.caltech.edu> wrote: > >> And just to show you what the dashboards look like here's a couple >> of screen shots >> >> Jconsole like page of jvm stats >> http://herbie.ddv.com/~anthonym/mondemand-2.png >> >> Cassandra specific memtable stats >> http://herbie.ddv.com/~anthonym/mondemand-1.png >> >> -Anthony >> >> On Tue, May 04, 2010 at 10:03:52AM -0700, Michael Lum wrote: >> > On 5/4/2010 7:21 AM, Eric Evans wrote: >> > >On Tue, 2010-05-04 at 08:41 +0300, Ran Tavory wrote: >> > >>How about the following compromise: >> > >>Add a simple web server to each node with only one simple servlet that >> > >>simply spits out all JMX stats on one page. Not fancy, no graphs, >> > >>simply the same values you can get from jconsole, but on a web page. >> > >>To me it seems like a fair tradeoff b/w maintenance and easier out of >> > >>the box management. Shooting up jconsole for each server is >> > >>cumbersome, at least in the environment I work in (firewalls, high >> > >>latency etc) so a web interface can be nice. >> > > >> > >It still seems superfluous to me, but I'd be open to something >> > >fire-and-forget (i.e. wouldn't need updating each time something new was >> > >added). >> > >> > This is how we monitor our Cassandra clusters. Each Cassandra node runs >> > a process that polls the JMX stats and then fires off events to a set of >> > configured management nodes using either UDP or multicast, depending on >> > the network. New Cassandra nodes in the same cluster and datacenter >> > have the same config (and are configured centrally anyways), and the >> > management nodes automatically add new nodes based on the events they >> > receive, so all the graphs, dashboards, monitors, and downstream tools >> > pick all of this up without needing a change. This way we don't need to >> > fire up jconsole for hundreds of nodes and can do other interesting >> > cluster-wide aggregations. Also, we don't have to remember to setup >> > monitoring when the cluster grows. >> > >> > All the tools used are open source, and I'd be happy to share more >> > detail if there is interest. >> >> -- >> >> Anthony Molinaro >> >
