context problems: incorrect auto-deploy of webapps when context specifiers exist
Hi people, Webapps with explicit Context specifiers in server.xml are not correctly recognized according to their docBase specifiers. Tomcat consequently incorrectly deploys duplicates of these apps under default context specifiers. For apps involving resources or non-trivial behaviour, eg sockets, this will result in resource assignment being taken by the wrong instance. Reloading may also then become ineffective/ non-correctly functional as the intended instance requested for reload is not the actual instance handling the resource or serving socket connections.
context problems: edit /conf/[engine]/[host]/someApp.xml scrubs app in /webapps
Hi people, Editing user-created Context specifiers in /conf/[engine]/[hostname], for webapps deployed as per traditional in /webapps, appears to incorrectly cause the webapp to be scrubbed with web.xml being deleted and the webapp becoming non-functional. No recovery is possible with manual re-deployment from the Development machine. This is a departure from the traditional deployment concept where the /webapps directory was under development/ IT ownership.
context problems: docBase inside the host appBase warnings
Hi people, Specifying a Context descriptor for a traditionally-deployed webapp, in /webapps, produces warnings when using the supposed 'new and recommended' /conf/[engine]/[hostname]/someApp.xml context specifiers. WARNING: A docBase XXX inside the host appBase has been specified, and will be ignored This breaks compatability with normal/ traditional deployment methods. Neither am I aware of a better alternative? The message also appears incorrect in that the webapp will actually be deployed from this location. See later though for incorrect recognition, incorrect scrubbing/ deletion etc separate webapp/ context problems.
context problems: META-INF/context.xml context paths non-functional
Hi people, Posting a series of issues in Context Specifications and related areas. This area seems rather more troublesome than it should, so I think I need to go through all of these and make people aware of how much is going on in this area... Context specifiers placed within the webapp at /META-INF/context.xml are parsed, but Context Paths specified here are non-functional and do not work. Other users also report this problem breaking compatability with previous TC versions.
context problems: Context Path specifiers broken in TC 5.5
Hi people, Context Path specifiers seem substantially non-functional in the new deployer technology. Collection of bugs related to this area: Bug describing context paths in webapp META-INF/context.xml being ignored: http://issues.apache.org/bugzilla/show_bug.cgi?id=39120 Bugs describing non-functional ContextSpecifier.ContextPath and complicated tree-structure nonsense completely unnecessary to implement such feature: http://marc.theaimsgroup.com/?l=tomcat-user&m=114436878122925&w=2 http://issues.apache.org/bugzilla/show_bug.cgi?id=34057 http://issues.apache.org/bugzilla/show_bug.cgi?id=35035 Message describing problems with possibly-related problems with Context Path specifiers. http://marc.theaimsgroup.com/?l=tomcat-user&m=113877278215302&w=2 Why is there any tree logic necessary? What is conceptually complicated with a Context Path specification, other than it being a simple String property of a Context object? -- granted that, in-memory, you might want a PATRICIA trie or somesuch to look them up; -- but to load/ store & specify them, tree structure complexity seems questionable. Why is there almost no simple, functioning, means to specify basic Context details? ???
context problems: other threads on non-functional Context Specifiers
Hi people, Threads from other people describing non-functionality with Context Specifiers in 5.5 Reloading and JDBC specification problems with Context: http://groups.google.co.nz/group/comp.lang.java.programmer/browse_frm/thread/7016429627d389c9/2b95e1e1aa8d59ce?lnk=st&q=tomcat+5.5+contexts&rnum=1&hl=en#2b95e1e1aa8d59ce Reloading spec problems with Context: http://groups.google.co.nz/group/comp.lang.java/browse_frm/thread/59710d5248233182/c26cfd98d05b270a?lnk=st&q=tomcat+5.5+contexts&rnum=2&hl=en#c26cfd98d05b270a Compatability lost & user difficulties migrating from 4.1 to 5.5, not sure whether this guy got them resolved or not: http://marc.theaimsgroup.com/?l=tomcat-user&m=113649241612245&w=2
context problems: not addressed; summary
Hi people, As posted thru this series, Context Specifiers (which should be conceptually trivial) seem to have become a problem area. To some extent Deployment also seems to be touched by this. I identify the problem as Design and lack of conceptual clarity, rather than code. We can see a lack of clarity of Ownership in two important areas of data, introduced with 5.5 Neither the /conf/[engine]/[host] nor the /webapps directory is clearly owned by either the (IT) User or Tomcat. Consequently it is fundamentally unclear whether Configuration Changes should require tomcat to apply secondary effects/ modify these data automatically, or whether the IT User has already done so/ or does not require or intend any such modification or changes to be made. We're also seeing noted undesirable automatic effects such as Tomcat stupidly scrubbing web.xml from a deployed webapp, breaking it completely, after minor tidy-up edits on a user-created Context descriptor. Having worked with data replication, persistence etc architectures for decade plus I can say this is just not deterministic enough, and that I don't like any lack of clarity over who owns data and which direction flow-on effects apply. I do not believe it is possible to implement a sound/ reliable system without a greater degree of clarity in terms of Ownership, Responsibility and Effects. Nor do I believe the lack of these has any benefits whatsoever. I understand that some changes may be being driven to improve support for farm deployment, or to fix broken behaviours from 5.0. But I do not believe this design or these mixed-ownership concepts are fundamentally sound, or logically can be. I am also quite suprised how a trivial Context Specifier object, can now be specified in 3 or 4 ways and places, yet is moderately non-functional in most of these; with Context Path specification being almost completely broken. At this point I will point out that I'm not the first person to run into most of these context problems. In the majority they're pretty much all reported bugs, which one person has blown off as: - RESOLVED WON'TFIX without them being resolved in the slightest. Also we get comments such as: > If you don't like it, you'll have to use a different container. And regarding my desire to specify Context Paths -- hello, these are a basic spec, not missile guidance technology. > Ok, so you have specific needs. This is great, but at this point, there does > not > seem to be anyone willing to redo the deployer just so that we can support > your > use case. My use case is being able to specify a Context Path, and having that work ? Perhaps my bug report came across as a rant, because it had covered several attempts all of which featured Contexts being non-functional or deployment auto-deleting what it shouldn't have, because I'd already read the docs, searched the web, read the NGs, read the bug reports, and seen these issues already being raised, with users being waved away/ blatantly disregarded... --- I'd be interested to know what other Tomcat-Dev members think. Bear in mind that I'm not a deep TC developer but have been building multi-task kernels/ fault-tolerant reliability/ database replication/ flow-on effect systems/ information robustness/ database technology/ business apps, for well over a decade now. Here's hoping we can bring a bit more light to this area. You can also email me personally at: thomasw 'at' connectorsystems 'dot' co 'dot' nz. Cheers, Thomas
prediction; context problems
Hi people, Sriram, Remy, Costin, Thanks Sriram for the +1 on this! As you guys have read, I think something is up in the the area of Context information; specifiers, parsing, and storage of current deployment state. I've previously been talking about determinism as one of the problems, but I don't find much indication that people get what I'm pointing out or are thinking about this at the 'concept' level. I'm going to some effort here to clarify this, and communicate these areas. I would therefore expect competent people to discuss Issues, not "issues". And please Costin, before you write too much about 'can't parse rant', perhaps try finding the Enter key on your own keyboard. :-) --- My understanding of the new Context/ Deployment design, is that deployed Context Specifiers are located in /conf/[engine]/[host]. My further understanding is, that either Tomcat or the User can create these; tomcat from a variety of original sources. Based on this I'd like to make a couple of predictions; re: erroneous behaviours. 1) Tomcat will be unable to know reliably when a Source context-specifier has been deleted, in order to delete the resultant Deployed context record. 2) Tomcat does not have enough information to know whether a given Deployed context-specifier is owned by Tomcat or the User. Apparently there may already be reports of #1. Number #2 is likely to lead to to Tomcat auto-scrubbing deployed apps in the /webapps folder when a context specifier is manually edited, whether these are owned by tomcat or not. -- Now, for a recommended solution. 1) never mix Ownership of anything. 2) store the list of Deployed Context specifiers, away from the user /conf directory; - storage in /working would be suitable. - keep ownership of this entirely to TC; then we will always know what is going on. 3) sub-folder and named files structure for deployed Master Contexts is unnecessary. - it's complex & unnecessary, toss it entirely. - store as XML file instead. - all Context elements flat within the head; no need to structure/ nest or context path. 4) bring back Context.ContextPath attribute, currently non-functional - current implementation disregards in most cases (!) - determine simple identification of Context Path from deployed/ context sources - aka. use the Context Path if it's spec'd - otherwise, use the default 5) fix recognition of Contexts by DocBase, currently broken - this will fix Contexts explicitly specified in 'server.xml', from also being deployed as duplicates under default settings. --- Anyway, hopefully this a good explanation of the issues, and helpful to the Tomcat developers and contributors in identifying what's wrong here. I'm aware there are some Clustering issues, driving the current design, but don't know exactly what these are. My proposed solution delivers Determinism, and from a replication perspective, providing solid reliability for Context info would certainly simplify clustering this information. Obviously, I've already put 8 - 10 hours of thought into this. But, more importantly, I've seen such 'unreliable' designs before, know what a reliable one is, and can tell the symptoms & difference pretty much instantly. As always, the proof is in the pudding; everything I'm raising & reporting is there, the Context stuff can exhibit ranges of undesirable behaviour (due to lack of determinism), the Context Path stuff is almost completely broken, etc. If this *wasn't* the case I'd be full of shit, but it is, so let's look at moving forward; will those Clustering requirements interact well with the proposed solution, what other future goals are desirable, etc. Let me know what you think, Cheers, Thomas Connector Systems Ltd thomasw 'at' connectorsystems 'dot'co'dot' nz
re: Deployement of war and context.xml files
Hi Henri, Apparently /conf/Catalina/localhost is meant to represent the 'current deployed state' of contexts/ webapps deployed. So it's reasonable from this understanding, that the Context specifier file is deleted when you undeploy the app; the context is being removed from the 'current state'. Since the complete deployer rewrite in TC 5.5, Tomcat now takes significant ownership of both /conf/[engine]/[host] and /webapps. My guess is that with the Customer-specific app customization, that you wish to have the same .war deployed with different varying Context specifiers. (Probably different context paths for these, too? I found huge problems with context paths since 5.5). And that you want each Context specifier to be a separate file, for modularity. The sensible way for Deployment to work, would be to supply a Context specifier file and a WAR or DocBase. These would be placed into a location owned by the User (you). You would then tell Tomcat to deploy (mount) the Context, or allow it to auto-deploy... Unfortunately 5.5 seems to be a balls-up, with Tomcat auto-creating 'current mounted state' contexts for WARs and webapp folders it finds, auto-deleting webapps/ folders on undeploy or after editing 'current state' context files, failing to match explicit Context specifiers and thus producing spurious default deployments of the webapps, and with the 'Context Path' attribute almost completely broken. As far as I am aware the locations to specify Contexts are, in order; 'server.xml' as an XML element; META-INF/context.xml inside the webapp; or /conf/[engine]/[host] directory. For deployment-specification customization obviously you need a Context file, independent of the webapp; the only place to put this is in /conf/[engine]/[host]; this location is most subject to the auto-magic working against you. Best answer:- - use an Ant 'deploy' script from your Dev system - call Manager.Deploy and also specify a Context xml file - may need to create a staging directory on the server under your control; so Tomcat has somewhere it can see to pick up the files; but safe from TC's auto-scrubbing within /webapps - lastly; Context Path specifiers, or possibly other important bits, may just not work :-( I would like to see very great clarification in this area. (Hi guys). I think there is a semantic misunderstanding in failing to distinguish MOUNTING from DEPLOYMENT of webapps. Mounting should be Tomcat's responsibility, deploying - copying it to where TC can mount it or replicate it around the cluster - should be the User's responsibility. This semantic error appears to have led to a (broad) range of TC behaviours in 5.5 which are different from previous deployment concepts; and just plain unlikable to developers. Now if MOUNTING and mounted state were kept separate from DEPLOYMENT, /webapps and deployment area would not be subject to being scrubbed & trodden-upon like they currently are. Information flows & effects would be well-defined, as opposed to current flows which are poorly-defined and effects which are undesirable. Keeping terminology clear, information well structured, and avoiding conflation of ideas, are key to design of simple, robust & correct software. Basically I'd say deployment in 5.5 is an example of failure in these areas. Lastly, stop tramping on my /webapps directory already! Hope you can sort out your deployment to work effectively in this version - I ended up using a workaround :-( Anyway, let me know if this helps. Regards, Thomas
Re: Creating a context and deploying a new application on the fly.
Hi David,
Looks like there may be a problem with your Context Resource code.
>From your programmatic configuration:
>ContextResource resource = new ContextResource();
>resource.setName("jdbc/reference");
>From your commented 'origin' example:
> type="javax.sql.DataSource"
Try setting the ContextResource.Name as per your original; assuming the
'origin' example is what your app was coded to work with and that you are
re-implementing this from XML to a programmatic config, the mis-matched name
would be erroneous.
Hi Yoav.
Cheers,
Thomas
re: context folders and Delete on tomcat root; funny [Bug 39594]
Hi people, Pedro, This is both a sad & funny true story. http://issues.apache.org/bugzilla/show_bug.cgi?id=39594 > We were trying to make our servlets use relative path so that our deplayment > would > be faster. In the process we try putting $Catalina_home in the path for the > base doc > [etc] > After this we noticed a folder called $CATALINA_HOME in the root dir for > tomacat, we > never created such a dir so we whent ahead and deleted it. For our surprise, > that > folder was linked to the root folder for tomcat and deleted everything. --- As I said, this was both humorous and tragic. Humorous because Tomcat and it's folders thoughtfully nuked Pedro's root - I can just see the expression on his face and mouth hanging slightly open. Gotcha! Tragic, because Remy is going to mark this issue as 'RESOLVED WILL-NOT-FIX'. No, Pedro, this is intended behaviour. It's better than the Deployer in 5.0 because I wrote it. Anyone can write a different deployer implementation. If you don't like it, you can use a different platform. Anyone can use a different platform. Have a nice day. ??? what ??! --- Sorry about the humour Pedro -- I've had similar happen to me and can only laugh because I know what a ghastly suprise this is. What really makes this funny, is that all this amusement comes from Remy rewriting the Context/ Deployer implementation in 5.5 and coming over all defensive about its defects... marking all reported issues in this area as RESOLVED (they're not) WILL-NOT-FIX (he won't). Well, at least the second part is true. Go on, prove me right. Cheers, Thomas PS: Full and absolute good faith - any pointed comments have come *after* seeing defects & bug reports from others systematically being ignored & blown off. If people come to me with a problem in my code, I fix it first, ask about future/ related requirements, check against design cases and stub these for future coverage... sometimes 2 - 5 years ahead. Sure beats telling people to f** off.
