Hi Vinod, Thanks for the support and comments. See my more comments
below:
On Sep 18, 2011, at 6:55 AM, Vinod Kumar Vavilapalli wrote:
Great addition to Apache, +1.
Some comments on the proposal:
- In the "Initial Goals" section, you referred to "Add a cluster
management
system". Not sure of what it entirely means, but if it is what I
think it
is, the new resource-management layer in Hadoop (YARN/ JIRA issue
MAPREDUCE-279) can be leveraged for this. YARN is going to be
available in
Hadoop 0.23 release. Willing to help in this regard in any way
possible if
you think it makes sense for S4, can have a separate discussion too.
To give you an idea of what we have in mind, here are a few points:
- Monitor Nodes and PEs.
- Ability to query specific PE instance variables.
- Add remove nodes.
- Monitor logs.
- Monitor CPU/Memory usage per node.
- Load/unload apps.
In general, it sounds right to assess if it is viable to use YARN for
S4 cluster management and we would appreciate your help. If it is ok
with you, I'd rather not include it in the proposal, though, since it
is an issue we could consider independently of the project going into
incubator. Let me know if you agree.
- You may want to capture the discussion about Hdaoop/Flume/Kafka in
the
"Relationships with Other Apache Projects" section.
I have added a summary to the proposal, thanks for pointing it out.
- Seen elsewhere that proposals list down the list of companies of the
initial set of committers (section "Homogeneous Developers").
I have also added a list here.
-Flavio
On Thu, Sep 15, 2011 at 1:49 AM, Leo Neumeyer
<leoneume...@gmail.com> wrote:
Dear all,
I would like to propose S4 to be an Apache Incubator project. S4
is a
distributed streaming platform written in Java.
Here is a link to the proposal in the Incubator wiki:
http://wiki.apache.org/incubator/S4Proposal
Thanks,
Leo Neumeyer
http://s4.io
http://twitter.com/leoneu
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org
