Gianugo Rabellino wrote:
Daniel,
thank you for taking the time to write this. I understand your
frustration, since the behaviour you're describing in your email has
clearly been a recurring pattern lately on [EMAIL PROTECTED] However, I'd
like to ask you to try and see beyond what happened,
Sorry, while I'm not juging people, I base my trust on what they will
deliver in the future on what they have delivered in the past, rather
than on promisses that everything will be different this time.
asking yourself
why we have been able to build Cocoon 2 but we seem not to be able to
go any further: the answer to that, to me, clearly denotes the need
for some kind of (soft?) revolution, e.g. Cocoon 3.
We are having a soft revolution right now, I ask you and the rest of the
community for some persistance instead of start running in still another
direction.
My reading of what's happening over here is that we've got to a point
where Cocoon internals have become way too complicated, way to
difficult to understand and way too difficult to interact with: I
think we'd need no more than one hand to count the number of (active)
people over here who could comfortably talk about the pipeline engine,
the sitemap interpreter, the caching sytem and all that.
How can you be so certain that we would getting less complicated if we
started from scratch? It might be complicated because it does something
complicated. As long as you don't understand the internals you cannot
really know, can you?
Managing complexity is much about finding the right concepts and work
really hard to get the right interfaces. The people who built Cocoon
from the begining spent a lot of time and effort in getting the apis
right. Since then we have made the apis bulky whithout doing that much
refactoring to keep things simple.
What we need to do is to refactor and rearchitecture Cocoon so that we
get rid of bulk that isn't used anymore. And to adapt for things that we
would like to add.
Every time I try to discuss apis people get bored. So I have not much
faith in that the current community would end up with something better
and more elegant than what we have if we started from scratch. Neither
do I see that anyone would be prepared to spend the time and effort that
it would take.
But please, suprise me.
We could go
on for ages and try to find a temporary solution (no matter how
architecturally sound) hoping that people like you get interested into
the framework in order to better separate stuff, but to me it's pretty
clear that we wouldn't be going anywhere far. So:
The main problem with our community, IMO, is the lack of focus and
persistance. We have so many great ideas, but as large parts of the
community always are running towards the next shiny thing and have very
little interest in polishing the design, contracts and implementations.
I don't buy it. It might look like that, and I understand why you read
it as that, but to me the truth is that there is little to no interest
in providing a small relief to a huge headache (I still have to
understand why real blocks would rock, apart from being ubercool
technically wise)
I'm sorry that Stefano and I and others have failed to communicate to
you why blocks rocks. I guess I just can say that they do rock and that
they solve a lot of our current problems.
when the amount of code to understand before you can
actually "fix" something is so massive that it makes your head spin.
And this is just talking about fixing existing stuff, when it's pretty
clear that we need to move way forward if we want to stay ahead of the
flock (something we *always* did in the past and we're failing to
achieve at this point).
After having been around for a couple of years, I start to recognize our
development cycle: First a new sexy idea is suggested, and we all jumps
up and down shouting out our excitement. Then we might start to discuss
design and implementation, and most people lose interest or maybe drop
by and complains about that the discussion have become technical and
that it bire them stiff. After that some people might even implement it.
And as it often takes a lot of work to design and implement cool stuff
it takes a while. And when it finally start to get somewhere, people are
not even remotely interested anymore because there is something new and
shiny to jump up and down and shout about.
So true. But you have to live with it and ask yourself why that new
and shiny stuff that got people so excited wasn't that sexy after all
to overcome the inertia of actually making it work. Either we're a
bunch of creative people with no technical skill (and no, I don't
believe that), or there is just *way* too much inertia to make
anything happen with the current codebase. I clearly buy the latter.
I think we are a bunch of CTO types that where really good programmers
in the past but have been used to that someone else does the programming
and that it is enough for us to talk ;)
Community based Open Source shines when there is a (small) number of
technically strong people that provide a codebase where even medium to
low range guys can interact with: this happened in the very first days
of Cocoon 2, when everyone and his dog provided additional components
so fast that we soon filled up the available space, so what's next?
Real blocks.
Are we looking for more de-facto one man shows with code spikes that
inevitably lead to frustration and make people disaffect or leave? I'd
much rather jump on what are the challenges we foresee and try to
catch them, even if that means ditching most of what we've got so far.
It is so easy to ditch other peoples work, isn't it?
Instead of focus on our core ideas and functionalities, and ensure that
we have real high quality implementations of them, we just add still
another prototype implementation of something new whithout bothering
about how it integrates with the rest of what we have.
This is not so true. Sylvain's discussion is catalyzing an important
message there: there is really no reason to bother about integrating
with existing stuff, because this would just make things worse.
Rewrites are risky, that's true, but anything that takes more than
three years to become real is doomed to fail (this -
http://tinyurl.com/86ke5 - isn't the earliest example I could find,
but it shows you we were talking blocks back in 2002). When are we
going to realize it?
I started to work on blocks in April this year,
http://marc.theaimsgroup.com/?l=xml-cocoon-cvs&m=111411975927551&w=2,
since then I and others have implemented most of what was discussed in
2002 and have a good plan for implementing the rest.
What happened was that I decided that I wanted blocks and started to
incrementally and persistently work towards getting them.
So I'm sorry, I will not jump up and down and shout together with the
rest of you this time. I will persist and continue with the work in
trunk on what I think is important:
* finishing blocks
* make the m2 build system work and become the only one
* binary releases
* strengthen the contracts
* improve the quality
* split up Cocoon in smaller parts
* deprecate things that not are or not should be used anymore
* strive for coherence
* refactoring and rearchitecturing mercelesly
* 2.2 during Q1
This work will continue to be evolutionary and incremental. And when we
have the block system and have cleaned the contracts and the messy
implementation of the core, it will be easy and safe to experiment with
new great ideas. And we are getting rather close to this point, we just
need to finish what we have started. And for the messines we solve that
with refactoring.
Gosh, Daniel, an outstanding agenda. I'll be watching closely what
seems to be a notable list of accomplishments, hoping to help out as
well (whatever happens to 3.0, I intend to provide strong and
long-standing support for 2.x as well, so every step further is most
welcome). But I'm also most interested in seeing things moving forward
from a community point of view, stopping the drift of people looking
elsewhere (there must be a reason for that, right?).
I guess there are many reasons for people looking elsewhere. IMO, some
important reasons is that Cocoon is incoherent and that it is hard to
develop it further. That we never release so that you never get the
pleasure of being able to use your new code and innovations in new
projects or seeing other people use it.
My focus the last year or so have been on attacking these problems.
quite risky to try to rewrite everything from scratch, although I have
felt the urge myself countless time while struggling with the messy
internals. But before you go ahead and rewrite everything from scratch,
take some minutes and read why Joel Spolesky thinks that rewriting
software from scratch is the:
*single worst strategic mistake* that any software company can make
http://www.joelonsoftware.com/articles/fog0000000069.html.
Two notes here:
1. Joel is talking about companies. Open Source is *much* different.
It is, but it is so different wrt rewriting from scratch?
2. I don't read Sylvain's proposal as a "let's dump everything". What
I see is a different and fresh approach to real issues, and a solution
that goes towards maximum reuse of what we've got so far, ditching as
much as we can of the infrastructure burden we've been carrying on for
ages.
I agree with Sylvain about much of the suggested improvements, but not
about starting from scratch.
For the actual functionality that Sylvain proposed I think much of it is
great and I have argued for things similar to some of them earlier. And
they can be added in an incremental way.
Here is where we basically disagree. I have this feeling that, unless
a major revamp, there is a very slim chance for Cocoon to progress
incrementally. I thought OSGi would have helped, with modularization
and the like, but as you can see, this is clearly has not been the
case:
Please show some patience or better take part in it yourself, we are
getting there.
maybe the problem is elsewhere, e.g. the current core being too
heavyweight and the user visible part being way to hard to understand?
Yes, it needs refactoring.
These are my major gripes at the moment, and while I still do believe
that the steep learning curve of Cocoon pays off big time, I've come
to think lately that we could actually do something to make it better.
Most definitively.
Leaving our core offering and rewriting everything from scratch means
community and product suicide, don't do it.
Again, I don't feel this is rewriting from scratch. And while I see
your point, I can clearly a stagnation and starvation risk ahead if we
don't shift gears and move on. How would you risk manage that?
I risk manage it by committing my own work in what I believe in and by
trying to convince other people that it is worthwhile to work on the
same stuff. I also risk manage by avoid forking and start from scratch,
which have killed a lot of NG work previously.
The most important part of my risk management is by prioritizing and
focus on the single most important thing until it is good enough.
/Daniel
