Re: [PHP] ORM doctrine
> I understand cache well, both the benefits (save DB trip) and shortfalls > (outdated by DB, management, etc.). Most of the apps that I've seen so far > used cache to solve a problem that shouldn't happen in the 1st place. For > example, during recent my quest looking PHP MVC framework and sample apps, I > saw OpenCart, an e-commerce app based on custom MVC framework. Installed it > for a test run. It looks good and seems to perform well with the included > sample data. Then I filled up some more sample data: over 3000 categories, > over 2000 manufacturers, over 300,000 products. No other changes made such > specials, features, etc. Although the app supports i18n, the sample data is > just one language for a decent DB size of about 100MB. The app took over 30 > seconds to respond for any link. Then I disable the cache and began > debugging. I made 1 minor addition to the DB and 1 minor change in the code > base - parts on 1 line - I shorten the response time by about 10 seconds. > What I just did proved my 2nd and 3rd point :) > > Regards, > Tommy > An app that takes 30 secs to respond using cache is doing it wrong. You shaving 10 secs off the request without cache is fine but you should have brought it much further down with a cache. Waiting 20 secs for the response is still useless. Regards Peter -- WWW: plphp.dk / plind.dk LinkedIn: plind BeWelcome/Couchsurfing: Fake51 Twitter: kafe15 -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] ORM doctrine
> -Original Message- > From: Peter Lind [mailto:peter.e.l...@gmail.com] > Sent: Sunday, December 12, 2010 1:18 AM > To: Tommy Pham > Cc: php-general List > Subject: Re: [PHP] ORM doctrine > > > I understand cache well, both the benefits (save DB trip) and > > shortfalls (outdated by DB, management, etc.). Most of the apps that > > I've seen so far used cache to solve a problem that shouldn't happen > > in the 1st place. For example, during recent my quest looking PHP MVC > > framework and sample apps, I saw OpenCart, an e-commerce app based > on > > custom MVC framework. Installed it for a test run. It looks good and > > seems to perform well with the included sample data. Then I filled up > > some more sample data: over 3000 categories, over 2000 manufacturers, > > over 300,000 products. No other changes made such specials, features, > > etc. Although the app supports i18n, the sample data is just one > > language for a decent DB size of about 100MB. The app took over 30 > > seconds to respond for any link. Then I disable the cache and began > > debugging. I made 1 minor addition to the DB and 1 minor change in > > the code base - parts on 1 line - I shorten the response time by about > > 10 seconds. What I just did proved my 2nd and 3rd point :) > > > > Regards, > > Tommy > > > > An app that takes 30 secs to respond using cache is doing it wrong. > You shaving 10 secs off the request without cache is fine but you should > have brought it much further down with a cache. Waiting 20 secs for the > response is still useless. > > Regards > Peter > > -- > > WWW: plphp.dk / plind.dk > LinkedIn: plind > BeWelcome/Couchsurfing: Fake51 > Twitter: kafe15 > My goal wasn't to optimize any app. I just want to learn 'implement abstract MVC concept into concrete, ...' as I mentioned in Adam's thread about choosing framework or not. If I was to optimize it, I wouldn't have any problems lowering the response time down to under 3 seconds w/o cache. Regards, Tommy -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] ORM doctrine
On Sunday, 12 December 2010, Tommy Pham wrote: >> -Original Message- >> From: Peter Lind [mailto:peter.e.l...@gmail.com] >> Sent: Sunday, December 12, 2010 1:18 AM >> To: Tommy Pham >> Cc: php-general List >> Subject: Re: [PHP] ORM doctrine >> >> > I understand cache well, both the benefits (save DB trip) and >> > shortfalls (outdated by DB, management, etc.). Most of the apps that >> > I've seen so far used cache to solve a problem that shouldn't happen >> > in the 1st place. For example, during recent my quest looking PHP MVC >> > framework and sample apps, I saw OpenCart, an e-commerce app based >> on >> > custom MVC framework. Installed it for a test run. It looks good and >> > seems to perform well with the included sample data. Then I filled up >> > some more sample data: over 3000 categories, over 2000 manufacturers, >> > over 300,000 products. No other changes made such specials, features, >> > etc. Although the app supports i18n, the sample data is just one >> > language for a decent DB size of about 100MB. The app took over 30 >> > seconds to respond for any link. Then I disable the cache and began >> > debugging. I made 1 minor addition to the DB and 1 minor change in >> > the code base - parts on 1 line - I shorten the response time by about >> > 10 seconds. What I just did proved my 2nd and 3rd point :) >> > >> > Regards, >> > Tommy >> > >> >> An app that takes 30 secs to respond using cache is doing it wrong. >> You shaving 10 secs off the request without cache is fine but you should >> have brought it much further down with a cache. Waiting 20 secs for the >> response is still useless. >> >> Regards >> Peter >> >> -- >> >> WWW: plphp.dk / plind.dk >> LinkedIn: plind >> BeWelcome/Couchsurfing: Fake51 >> Twitter: kafe15 >> > > My goal wasn't to optimize any app. I just want to learn 'implement abstract > MVC concept into concrete, ...' as I mentioned in Adam's thread about > choosing framework or not. If I was to optimize it, I wouldn't have any > problems lowering the response time down to under 3 seconds w/o cache. > > Regards, > Tommy > Your posts seem to indicate that caches are only useful when other parts of the app have been done wrong. My point was that this is a fairly fundamental misunderstanding of caches - regardless of what you are or aren't capable of optimizing. Regards Peter -- WWW: plphp.dk / plind.dk LinkedIn: plind BeWelcome/Couchsurfing: Fake51 Twitter: kafe15 -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] ORM doctrine
Peter Lind wrote: Your posts seem to indicate that caches are only useful when other parts of the app have been done wrong. My point was that this is a fairly fundamental misunderstanding of caches - regardless of what you are or aren't capable of optimizing. CHACHES are only useful when there are static views of the information available. Only static elements can be cached with any real chance of performance improvement, so part of what Tommy is saying is correct. Although the way he has worded that is perhaps a little misleading? A framework should provide a base that applies caching at the correct level, such as pre-processing smarty templates so that they can be loaded quicker, or caching static data such as selection lists ... which DOES require a more in depth understanding of the target since you need to know what lists are slow changing and what are dynamic. I suppose the discussion here is perhaps where a library becomes a fully fledged framework, and 'ORM' is another area where there is a diversity of database abstraction libraries in addition to the more common ones. Data caching SHOULD always be the domain of the database, so duplicating that in PHP is pintless. It would be nice to have a more prominent leader which could be supported by integral caching and other performance enhancing tools, but Tommy is right when he says that caching 'used' to improve performance in some of these frameworks where all that is required is a much better understanding of the base data. bitweaver port eliminated some 90% of the queries on the database simply by restructuring things away from the original tikiwiki format. So much so that the caching process actually produces a substantial additional improvement, while the original structure was spending most of it's time simply trying to work out what cahe elements to read ... and of cause eaccelerator adds another layer of 'caching' over that :) -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk// Firebird - http://www.firebirdsql.org/index.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] ORM doctrine
> -Original Message- > From: Peter Lind [mailto:peter.e.l...@gmail.com] > Sent: Sunday, December 12, 2010 1:44 AM > To: Tommy Pham > Cc: php-general List > Subject: Re: [PHP] ORM doctrine > > On Sunday, 12 December 2010, Tommy Pham > wrote: > >> -Original Message- > >> From: Peter Lind [mailto:peter.e.l...@gmail.com] > >> Sent: Sunday, December 12, 2010 1:18 AM > >> To: Tommy Pham > >> Cc: php-general List > >> Subject: Re: [PHP] ORM doctrine > >> > >> > I understand cache well, both the benefits (save DB trip) and > >> > shortfalls (outdated by DB, management, etc.). Most of the apps > >> > that I've seen so far used cache to solve a problem that shouldn't > >> > happen in the 1st place. For example, during recent my quest > >> > looking PHP MVC framework and sample apps, I saw OpenCart, an > >> > e-commerce app based > >> on > >> > custom MVC framework. Installed it for a test run. It looks good > >> > and seems to perform well with the included sample data. Then I > >> > filled up some more sample data: over 3000 categories, over 2000 > >> > manufacturers, over 300,000 products. No other changes made such > >> > specials, features, etc. Although the app supports i18n, the > >> > sample data is just one language for a decent DB size of about > >> > 100MB. The app took over 30 seconds to respond for any link. Then > >> > I disable the cache and began debugging. I made 1 minor addition > >> > to the DB and 1 minor change in the code base - parts on 1 line - I > >> > shorten the response time by about > >> > 10 seconds. What I just did proved my 2nd and 3rd point :) > >> > > >> > Regards, > >> > Tommy > >> > > >> > >> An app that takes 30 secs to respond using cache is doing it wrong. > >> You shaving 10 secs off the request without cache is fine but you > >> should have brought it much further down with a cache. Waiting 20 > >> secs for the response is still useless. > >> > >> Regards > >> Peter > >> > >> -- > >> > >> WWW: plphp.dk / plind.dk > >> LinkedIn: plind > >> BeWelcome/Couchsurfing: Fake51 > >> Twitter: kafe15 > >> > > > > My goal wasn't to optimize any app. I just want to learn 'implement > abstract MVC concept into concrete, ...' as I mentioned in Adam's thread > about choosing framework or not. If I was to optimize it, I wouldn't have > any problems lowering the response time down to under 3 seconds w/o > cache. > > > > Regards, > > Tommy > > > > Your posts seem to indicate that caches are only useful when other parts of > the app have been done wrong. My point was that this is a fairly > fundamental misunderstanding of caches - regardless of what you are or > aren't capable of optimizing. > > Regards > Peter > > -- > > WWW: plphp.dk / plind.dk > LinkedIn: plind > BeWelcome/Couchsurfing: Fake51 > Twitter: kafe15 > I know your intention of using cache. I know it has its uses. But for most folks, they see it as one of the main parts of the solution to performance problem. I see the application design (flow, coding best practices, etc...) is the solution while cache is merely icing on the cake, as the cliché goes. I'd would optimize the app 1st before I rely on any cache. The current project I'm working on would be able to support 10-20 languages. Each language would have 500k-750k rows in some tables. This to me is a decent size DB of around a few GB. I'm expecting the performance to diminish once the row counts approaches and go beyond 1M that's because I'm not a DBA :). IMO, for the better yield in performance: time spent optimizing the app > time spent implementing cache. Regards, Tommy -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] ORM doctrine
> -Original Message- > From: Lester Caine [mailto:les...@lsces.co.uk] > Sent: Sunday, December 12, 2010 2:10 AM > To: php-general List > Subject: Re: [PHP] ORM doctrine > > Peter Lind wrote: > > Your posts seem to indicate that caches are only useful when other > > parts of the app have been done wrong. My point was that this is a > > fairly fundamental misunderstanding of caches - regardless of what you > > are or aren't capable of optimizing. > > CHACHES are only useful when there are static views of the information > available. Only static elements can be cached with any real chance of > performance improvement, so part of what Tommy is saying is correct. > Although the way he has worded that is perhaps a little misleading? Yes, as you noticed, I'm not very good at putting my understanding into words. Thanks for putting it correctly as the way I understood it. Regards, Tommy > > A framework should provide a base that applies caching at the correct level, > such as pre-processing smarty templates so that they can be loaded quicker, > or caching static data such as selection lists ... which DOES require a more > in > depth understanding of the target since you need to know what lists are > slow changing and what are dynamic. I suppose the discussion here is > perhaps where a library becomes a fully fledged framework, and 'ORM' is > another area where there is a diversity of database abstraction libraries in > addition to the more common ones. Data caching SHOULD always be the > domain of the database, so duplicating that in PHP is pintless. > > It would be nice to have a more prominent leader which could be supported > by integral caching and other performance enhancing tools, but Tommy is > right when he says that caching 'used' to improve performance in some of > these frameworks where all that is required is a much better understanding > of the base data. > bitweaver port eliminated some 90% of the queries on the database simply > by restructuring things away from the original tikiwiki format. So much so > that the caching process actually produces a substantial additional > improvement, while the original structure was spending most of it's time > simply trying to work out what cahe elements to read ... and of cause > eaccelerator adds another layer of 'caching' over that :) > > -- > Lester Caine - G8HFL > - > Contact - http://lsces.co.uk/wiki/?page=contact > L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - > http://enquirysolve.com/ Model Engineers Digital Workshop - > http://medw.co.uk// Firebird - http://www.firebirdsql.org/index.php > -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] ORM doctrine
On Sunday, 12 December 2010, Tommy Pham wrote: >> -Original Message- >> From: Lester Caine [mailto:les...@lsces.co.uk] >> Sent: Sunday, December 12, 2010 2:10 AM >> To: php-general List >> Subject: Re: [PHP] ORM doctrine >> >> Peter Lind wrote: >> > Your posts seem to indicate that caches are only useful when other >> > parts of the app have been done wrong. My point was that this is a >> > fairly fundamental misunderstanding of caches - regardless of what you >> > are or aren't capable of optimizing. >> >> CHACHES are only useful when there are static views of the information >> available. Only static elements can be cached with any real chance of >> performance improvement, so part of what Tommy is saying is correct. >> Although the way he has worded that is perhaps a little misleading? Possibly. However, thinking that a cache is something you apply at the end of development means you may well bar yourself from better uses of one. Ie. thinking that a cache is "icing on the cake" prohibits you from "mixing it into the cake" (to further abuse a metaphor) which could give you "a better recipe". I may have misunderstood the topic, but a cache to me is more than just storing views. It's also the db cache, memcache, apc, etc. You have to think about how you use these - some of them can't just be slapped on to your app after development. >> Data caching SHOULD always be the >> domain of the database, so duplicating that in PHP is pintless. So you're saying one should never use memcache for storing data from the db? Regards Peter -- WWW: plphp.dk / plind.dk LinkedIn: plind BeWelcome/Couchsurfing: Fake51 Twitter: kafe15 -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] code quest - ECHO?!?
Kirk Bailey wrote: > Ok, so what is echo, and how is it different from print. > > The code in code quest used echo. I have a copy of learning php 5.0 from > O'Reilly, and noplace does it mention echo. Why? What's the difference? > IS there a difference? Is there an advantage to either? Please clarify > for this newbie. > The documentation says it all better than I can: http://php.net/manual/en/function.echo.php http://php.net/manual/en/function.print.php Cheers -- David Robley OPERATOR! Trace this call and tell me where I am. Today is Sweetmorn, the 54th day of The Aftermath in the YOLD 3176. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] ORM doctrine
> -Original Message- > From: Peter Lind [mailto:peter.e.l...@gmail.com] > Sent: Sunday, December 12, 2010 3:05 AM > To: Tommy Pham > Cc: Lester Caine; php-general List > Subject: Re: [PHP] ORM doctrine > > On Sunday, 12 December 2010, Tommy Pham > wrote: > >> -Original Message- > >> From: Lester Caine [mailto:les...@lsces.co.uk] > >> Sent: Sunday, December 12, 2010 2:10 AM > >> To: php-general List > >> Subject: Re: [PHP] ORM doctrine > >> > >> Peter Lind wrote: > >> > Your posts seem to indicate that caches are only useful when other > >> > parts of the app have been done wrong. My point was that this is a > >> > fairly fundamental misunderstanding of caches - regardless of what > >> > you are or aren't capable of optimizing. > >> > >> CHACHES are only useful when there are static views of the > >> information available. Only static elements can be cached with any > >> real chance of performance improvement, so part of what Tommy is > saying is correct. > >> Although the way he has worded that is perhaps a little misleading? > > Possibly. However, thinking that a cache is something you apply at the end > of development means you may well bar yourself from better uses of one. > Ie. thinking that a cache is "icing on the cake" prohibits you from "mixing it > into the cake" (to further abuse a metaphor) which could give you "a better > recipe". I'm not sure about being 'a better cake' with all that sugar, you'll have a higher chance of diabetes. Therefore in trying to achieve a healthy (lean & mean) app, you'll more chances of having problems than it intend to solve. Regards, Tommy > I may have misunderstood the topic, but a cache to me is more than just > storing views. It's also the db cache, memcache, apc, etc. You have to think > about how you use these - some of them can't just be slapped on to your > app after development. > > > >> Data caching SHOULD always be the > >> domain of the database, so duplicating that in PHP is pintless. > > So you're saying one should never use memcache for storing data from the > db? > > Regards > Peter > > -- > > WWW: plphp.dk / plind.dk > LinkedIn: plind > BeWelcome/Couchsurfing: Fake51 > Twitter: kafe15 > -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] ORM doctrine
Peter Lind wrote: I may have misunderstood the topic, but a cache to me is more than just storing views. It's also the db cache, memcache, apc, etc. You have to think about how you use these - some of them can't just be slapped on to your app after development. >> Data caching SHOULD always be the >> domain of the database, so duplicating that in PHP is pintless. So you're saying one should never use memcache for storing data from the db? That is probably one of the areas where care should be taken in WHAT you are caching. CURRENTLY too much caching goes on, and I'm sure you have seen pages that do not make sense simply because some element used to create it has been cached and is now out of date. memcache may well be a candidate to use for a cached list of choices for a form, but on a distributed database ( such as Firebird ) then how do you KNOW if the list has been updated? Only the database actually knows that, and a good database will provide ALL of the data caching that you need ... or at least it should if the DATABASE is designed correctly. The reason for 'caching' needs to be understood before it is applied in order to avoid the black holes that random caching is causing nowadays already. How often do you hear "wipe the xxx browser cache"? And I know if I am changing theme elements in bitweaver or phpgedview then I HAVE to wipe the cache to ensure that smarty rebuilds the relevant pages. -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk// Firebird - http://www.firebirdsql.org/index.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Re: Announcing New PHP Extension: FileConv
Nice one Dan, and thanks! :) Daniel Brown wrote: Happy Saturday, folks; I've finally gotten around to releasing my latest PHP extension (which was actually written about two years ago). Named FileConv, it adds native functions for converting back and forth between DOS, *NIX, and legacy MacOS file formats. It's compact, comes with a basic installer, and can convert a 1MB text file to/from any of the included formats in approximately one-tenth of one second. Unlike many versions that you could otherwise use from the command line, this library allows the file to retain its original timestamps, as well. I ran through some recursive directories of files with a test script, determining which files were text versus which were binary, detected the formatting of the text files, and converted to a different format at random (if Mac, go to DOS/*NIX, if *NIX, go to Mac/DOS, etc.). Approximately 1.5GB of files were scanned, detected, and translated, with a cost of 1 minute 24 seconds. You can read more about the library and download it yourself at http://links.parasane.net/n4c4 . After doing some code cleanup when I have time, the next step will be continuing with the process of getting it into PECL, as was initially planned two years ago. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] ORM doctrine
On Sunday, 12 December 2010, Tommy Pham wrote: >> -Original Message- >> From: Lester Caine [mailto:les...@lsces.co.uk] >> Sent: Sunday, December 12, 2010 2:10 AM >> To: php-general List >> Subject: Re: [PHP] ORM doctrine >> >> Peter Lind wrote: >> > Your posts seem to indicate that caches are only useful when other >> > parts of the app have been done wrong. My point was that this is a >> > fairly fundamental misunderstanding of caches - regardless of what you >> > are or aren't capable of optimizing. >> >> CHACHES are only useful when there are static views of the information >> available. Only static elements can be cached with any real chance of >> performance improvement, so part of what Tommy is saying is correct. >> Although the way he has worded that is perhaps a little misleading? Possibly. However, thinking that a cache is something you apply at the end of development means you may well bar yourself from better uses of one. Ie. thinking that a cache is "icing on the cake" prohibits you from "mixing it into the cake" (to further abuse a metaphor) which could give you "a better recipe". I may have misunderstood the topic, but a cache to me is more than just storing views. It's also the db cache, memcache, apc, etc. You have to think about how you use these - some of them can't just be slapped on to your app after development. >> Data caching SHOULD always be the >> domain of the database, so duplicating that in PHP is pintless. So you're saying one should never use memcache for storing data from the db? Regards Peter -- WWW: plphp.dk / plind.dk LinkedIn: plind BeWelcome/Couchsurfing: Fake51 Twitter: kafe15 -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] ORM doctrine
On Sunday, 12 December 2010, Lester Caine wrote: > Peter Lind wrote: > > I may have misunderstood the topic, but a cache to me is more than > just storing views. It's also the db cache, memcache, apc, etc. You > have to think about how you use these - some of them can't just be > slapped on to your app after development. > > >>> Data caching SHOULD always be the >>> domain of the database, so duplicating that in PHP is pintless. > > > > So you're saying one should never use memcache for storing data from the db? > > > That is probably one of the areas where care should be taken in WHAT you are > caching. CURRENTLY too much caching goes on I have not seen this problem but I have often seen sites not caching things they should, so I'd say you're wrong. I don't think its relevant one way or the other though. > memcache may well be a candidate to use for a cached list of choices for a > form, but on a distributed database ( such as Firebird ) then how do you KNOW > if the list has been updated? This is relevant to very few developers. Much more relevant is the overhead used in contacting the single mysql/postgresql/etc database that your provider put on another machine. > Only the database actually knows that, and a good database will provide ALL > of the data caching that you need ... or at least it should if the DATABASE > is designed correctly. See above. > > The reason for 'caching' needs to be understood before it is applied in order > to avoid the black holes that random caching is causing nowadays already. How > often do you hear "wipe the xxx browser cache"? And I know if I am changing > theme elements in bitweaver or phpgedview then I HAVE to wipe the cache to > ensure that smarty rebuilds the relevant pages. Which underlines my point: caching is not icing on the cake but should be thought of sooner in the process, contrary to Tommys point. Regards Peter -- WWW: plphp.dk / plind.dk LinkedIn: plind BeWelcome/Couchsurfing: Fake51 Twitter: kafe15 -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] ORM doctrine
Hi All, Apologies for wading in to this discussion, however I must point out that caching at every level is very very important. As Peter says, caching is not an optimization to be thought of later, it is not the icing on the cake, rather, caching is one of the most critical design factors, to be considered on every component, layer and tier in an application, at every edge. Our end goal as developers, is not to create big ol fancy systems, it's to create systems which answer questions as quickly as possible, every time a request comes in to one of our applications, our job is to have created code which can respond as quickly as possible. The ultimate level of optimization in any system, is to have the answer ready before the question has been asked, to have the response ready before the request has been made. Every time any part of your application runs more than once and produces the same result, you've failed a little bit. Every time your entire application runs and produces the same response as the last time it ran, you've failed a lot. These failures soon mount up to a fail of epic proportions. We're talking about applications which are not just 10% more performant across the network, but several thousand times more. The difference between a server handling 10 concurrent requests per second and 1 concurrent requests per second. Caching is not a minor optimization to be thought of later, it's not something to take lightly, it's pretty much the most important design principal there is, period. As a web developer, the most important interface you need to introduce caching on, is HTTP - a "static" or "published" HTML document which leverages HTTP caching using Last-Modified / ETag headers will give circa 100 times better performance (or more) than a dynamically generated each time doc. Not only that, but transparent proxies on the network can come in to effect and reduce weight on the server to zero, and further, web browsers will serve repeat requests to the document straight from RAM, again leading to zero weight. When you include those two factors it doesn't take long to see that the performance difference over the network is so good that it's nigh on unmeasurable. In your application itself, caching can be introduced at every level, you've already got filesystem io caches provided by the operating system, a well tuned db server cache can make a big difference as well, then on to opcode caches in languages like PHP since it's interpreted rather than compiled, and then on to code optimizations such as using static class cache's behind getByID methods and similar, and finally down to micro optimizations, static code analysis and dead code elimination, replacing (4*10) with (40), inlining static class members / constants and such like. Finally, language specific nuances and micro optimizations. Again, I stress, caching is not an optimization, an application which is designed to not repeat itself (via caching), is a good, scalable, ultra performant application, and that's what we, as developers, are paid to create. Best, and thanks for taking the effort to point this out to the list thus far Peter, Nathan Peter Lind wrote: On Sunday, 12 December 2010, Tommy Pham wrote: Peter Lind wrote: Your posts seem to indicate that caches are only useful when other parts of the app have been done wrong. My point was that this is a fairly fundamental misunderstanding of caches - regardless of what you are or aren't capable of optimizing. CHACHES are only useful when there are static views of the information available. Only static elements can be cached with any real chance of performance improvement, so part of what Tommy is saying is correct. Although the way he has worded that is perhaps a little misleading? Possibly. However, thinking that a cache is something you apply at the end of development means you may well bar yourself from better uses of one. Ie. thinking that a cache is "icing on the cake" prohibits you from "mixing it into the cake" (to further abuse a metaphor) which could give you "a better recipe". I may have misunderstood the topic, but a cache to me is more than just storing views. It's also the db cache, memcache, apc, etc. You have to think about how you use these - some of them can't just be slapped on to your app after development. Data caching SHOULD always be the domain of the database, so duplicating that in PHP is pintless. So you're saying one should never use memcache for storing data from the db? Regards Peter -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] ORM doctrine
> -Original Message- > From: Peter Lind [mailto:peter.e.l...@gmail.com] > Sent: Sunday, December 12, 2010 5:27 AM > To: Lester Caine > Cc: php-general@lists.php.net > Subject: Re: [PHP] ORM doctrine > > > > > The reason for 'caching' needs to be understood before it is applied in > order to avoid the black holes that random caching is causing nowadays > already. How often do you hear "wipe the xxx browser cache"? And I know > if I am changing theme elements in bitweaver or phpgedview then I HAVE > to wipe the cache to ensure that smarty rebuilds the relevant pages. > > Which underlines my point: caching is not icing on the cake but should be > thought of sooner in the process, contrary to Tommys point. > > Regards > Peter > > -- > > WWW: plphp.dk / plind.dk > LinkedIn: plind > BeWelcome/Couchsurfing: Fake51 > Twitter: kafe15 > > If the app is well designed, then you still could implement it later w/o major rewrite. Example, suppose your new client currently doesn't see the need for auth/acl nor do they have the budget for it ATM. However, being experienced, you foresee their needs based their business, model & industry, how would you design the app so as to include that foreseeable future? Isn't that where knowledge and experience comes in? Request > acl/auth > process request > cache/db > return results. Isn't that how the core basic flow should be? Thus, if you removed acl/auth & cache, the app would still work. That's one of the things I've mentioned in Adam's thread of choosing framework or not, is the flexibility to pick and choose. Another example, would you use an array or a class object to represent the value of a row in a particular DB table? Some folks may make those decisions lightly. IMO, such important decisions would make all the difference in having "workable (learning, understanding, maintaining, etc.), reliable, and flexible (modular, 3rd party add-ins, etc.) code while retaining good performance." Regards, Tommy -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] ORM doctrine
Tommy Pham wrote: -Original Message- From: Peter Lind [mailto:peter.e.l...@gmail.com] Sent: Sunday, December 12, 2010 5:27 AM To: Lester Caine Cc: php-general@lists.php.net Subject: Re: [PHP] ORM doctrine The reason for 'caching' needs to be understood before it is applied in order to avoid the black holes that random caching is causing nowadays already. How often do you hear "wipe the xxx browser cache"? And I know if I am changing theme elements in bitweaver or phpgedview then I HAVE to wipe the cache to ensure that smarty rebuilds the relevant pages. Which underlines my point: caching is not icing on the cake but should be thought of sooner in the process, contrary to Tommys point. If the app is well designed, then you still could implement it later w/o major rewrite. It's only well designed if caching is considered from the start - thus, as Peter says, caching is not the icing on the cake, but should, must, be thought of in the initial design process - if it's not, then the app isn't well designed. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] code quest - ECHO?!?
Groovy; they appear to be identical in all but name. IDENTICAL. Or am I missing a subtle definition difference? David Robley wrote: Kirk Bailey wrote: Ok, so what is echo, and how is it different from print. The code in code quest used echo. I have a copy of learning php 5.0 from O'Reilly, and noplace does it mention echo. Why? What's the difference? IS there a difference? Is there an advantage to either? Please clarify for this newbie. The documentation says it all better than I can: http://php.net/manual/en/function.echo.php http://php.net/manual/en/function.print.php Cheers -- end Very Truly yours, - Kirk Bailey, Largo Florida kniht +-+ | BOX | +-+ think
Re: [PHP] ORM doctrine
Nathan Rixham wrote: In your application itself, caching can be introduced at every level, you've already got filesystem io caches provided by the operating system, a well tuned db server cache can make a big difference as well, then on to opcode caches in languages like PHP since it's interpreted rather than compiled, and then on to code optimizations such as using static class cache's behind getByID methods and similar, and finally down to micro optimizations, static code analysis and dead code elimination, replacing (4*10) with (40), inlining static class members / constants and such like. Finally, language specific nuances and micro optimizations. Actually THAT probably sums things up nicely. An approach suitable for MySQL WILL probably be wrong for Postgres or Firebird. Certainly the optimised SQL I use for my own applications is much improved if I simply ignore compatibility with the databases other developers are using. Libraries like doctrine and even PDO are probably not using the best approach that a database specific approch may take. Firebird will maintain the 'results' of previous searches in cache and give results sets a lot faster, such as being ready to return the next page of results BEFORE the web page asks for it ;) But a database agnostic approach is not as efficient. -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk// Firebird - http://www.firebirdsql.org/index.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] ORM doctrine
Lester Caine wrote: Nathan Rixham wrote: In your application itself, caching can be introduced at every level, you've already got filesystem io caches provided by the operating system, a well tuned db server cache can make a big difference as well, then on to opcode caches in languages like PHP since it's interpreted rather than compiled, and then on to code optimizations such as using static class cache's behind getByID methods and similar, and finally down to micro optimizations, static code analysis and dead code elimination, replacing (4*10) with (40), inlining static class members / constants and such like. Finally, language specific nuances and micro optimizations. Actually THAT probably sums things up nicely. An approach suitable for MySQL WILL probably be wrong for Postgres or Firebird. Certainly the optimised SQL I use for my own applications is much improved if I simply ignore compatibility with the databases other developers are using. Libraries like doctrine and even PDO are probably not using the best approach that a database specific approch may take. Firebird will maintain the 'results' of previous searches in cache and give results sets a lot faster, such as being ready to return the next page of results BEFORE the web page asks for it ;) But a database agnostic approach is not as efficient. Yes, but you snipped a key paragraph there, because all the forms of caching and optimization listed above, including db cache optimization will only increase performance of the app by small percentages, whereas moving to a publishing model and producing static output will increase performance by several factors: [[[ As a web developer, the most important interface you need to introduce caching on, is HTTP - a "static" or "published" HTML document which leverages HTTP caching using Last-Modified / ETag headers will give circa 100 times better performance (or more) than a dynamically generated each time doc. Not only that, but transparent proxies on the network can come in to effect and reduce weight on the server to zero, and further, web browsers will serve repeat requests to the document straight from RAM, again leading to zero weight. When you include those two factors it doesn't take long to see that the performance difference over the network is so good that it's nigh on unmeasurable. ]]] Feel free to ignore this yourself, but please don't promote a bit of SQL and db server optimization as being the most important factor in optimizing PHP applications, it is important, but the net result is minimal in comparison to leveraging HTTP caching and static publishing of components or entire documents. Best, Nathan -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] ORM doctrine
On 2010-12-12, at 11:45 AM, Nathan Rixham wrote: > Lester Caine wrote: >> Nathan Rixham wrote: >>> In your application itself, caching can be introduced at every level, >>> you've already got filesystem io caches provided by the operating >>> system, a well tuned db server cache can make a big difference as well, >>> then on to opcode caches in languages like PHP since it's interpreted >>> rather than compiled, and then on to code optimizations such as using >>> static class cache's behind getByID methods and similar, and finally >>> down to micro optimizations, static code analysis and dead code >>> elimination, replacing (4*10) with (40), inlining static class members / >>> constants and such like. Finally, language specific nuances and micro >>> optimizations. >> Actually THAT probably sums things up nicely. An approach suitable for MySQL >> WILL probably be wrong for Postgres or Firebird. Certainly the optimised SQL >> I use for my own applications is much improved if I simply ignore >> compatibility with the databases other developers are using. Libraries like >> doctrine and even PDO are probably not using the best approach that a >> database specific approch may take. Firebird will maintain the 'results' of >> previous searches in cache and give results sets a lot faster, such as being >> ready to return the next page of results BEFORE the web page asks for it ;) >> But a database agnostic approach is not as efficient. > > Yes, but you snipped a key paragraph there, because all the forms of caching > and optimization listed above, including db cache optimization will only > increase performance of the app by small percentages, whereas moving to a > publishing model and producing static output will increase performance by > several factors: > > [[[ > As a web developer, the most important interface you need to introduce > caching on, is HTTP - a "static" or "published" HTML document which leverages > HTTP caching using Last-Modified / ETag headers will give circa 100 times > better performance (or more) than a dynamically generated each time doc. Not > only that, but transparent proxies on the network can come in to effect and > reduce weight on the server to zero, and further, web browsers will serve > repeat requests to the document straight from RAM, again leading to zero > weight. When you include those two factors it doesn't take long to see that > the performance difference over the network is so good that it's nigh on > unmeasurable. > ]]] > > Feel free to ignore this yourself, but please don't promote a bit of SQL and > db server optimization as being the most important factor in optimizing PHP > applications, it is important, but the net result is minimal in comparison to > leveraging HTTP caching and static publishing of components or entire > documents. > > Best, > > Nathan > > -- > PHP General Mailing List (http://www.php.net/) > To unsubscribe, visit: http://www.php.net/unsub.php > I would definitely agree with Nathan here. In our classic asp (don't ask) the vp of our company designed a survey app ( poorly I might add ) that dynamically generated the form and all of it's associated drop downs including the locations ( some 2000 of them ). The client was requiring 1000 user concurrency and the app as it was only handled 10 users. I re-architected the app to cache out the form since it was identical for each user, and we easily hit the required benchmark. Form caching is one of the most important omptimizations that can be made ( especially in our app where a major ( and I am not allowed to fix it ) design flaw loops through the form and associated data set 4 ( yes, 4! ) times and nothing is cached ). Db calls are expensive but not as expensive as extremely poor design! Bastien Koert Sent from my iPhone -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] ORM doctrine
Nathan Rixham wrote: Lester Caine wrote: Nathan Rixham wrote: In your application itself, caching can be introduced at every level, you've already got filesystem io caches provided by the operating system, a well tuned db server cache can make a big difference as well, then on to opcode caches in languages like PHP since it's interpreted rather than compiled, and then on to code optimizations such as using static class cache's behind getByID methods and similar, and finally down to micro optimizations, static code analysis and dead code elimination, replacing (4*10) with (40), inlining static class members / constants and such like. Finally, language specific nuances and micro optimizations. Actually THAT probably sums things up nicely. An approach suitable for MySQL WILL probably be wrong for Postgres or Firebird. Certainly the optimised SQL I use for my own applications is much improved if I simply ignore compatibility with the databases other developers are using. Libraries like doctrine and even PDO are probably not using the best approach that a database specific approch may take. Firebird will maintain the 'results' of previous searches in cache and give results sets a lot faster, such as being ready to return the next page of results BEFORE the web page asks for it ;) But a database agnostic approach is not as efficient. Yes, but you snipped a key paragraph there, because all the forms of caching and optimization listed above, including db cache optimization will only increase performance of the app by small percentages, whereas moving to a publishing model and producing static output will increase performance by several factors: [[[ As a web developer, the most important interface you need to introduce caching on, is HTTP - a "static" or "published" HTML document which leverages HTTP caching using Last-Modified / ETag headers will give circa 100 times better performance (or more) than a dynamically generated each time doc. Not only that, but transparent proxies on the network can come in to effect and reduce weight on the server to zero, and further, web browsers will serve repeat requests to the document straight from RAM, again leading to zero weight. When you include those two factors it doesn't take long to see that the performance difference over the network is so good that it's nigh on unmeasurable. ]]] Feel free to ignore this yourself, but please don't promote a bit of SQL and db server optimization as being the most important factor in optimizing PHP applications, it is important, but the net result is minimal in comparison to leveraging HTTP caching and static publishing of components or entire documents. For fixed pages this is the best way of handling the information. And handling those fixed pages is ... from my point of view ... not a problem since they can be cached at that level, or even stored locally in the browser cache. I've just been hitting re-load every time for a few updates I've just been processing! In order to actually see the result. But for the majority of my work, the data to be displayed is being rebuilt with every browser hit. In that case generating dynamic pages fast becomes the bottleneck. -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk// Firebird - http://www.firebirdsql.org/index.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] ORM doctrine
On Wed, Dec 8, 2010 at 10:02 PM, Tommy Pham wrote: > Hi, > > Has anyone used doctrine before? I know Nathan mentioned it in the other > thread but I was wondering how does it handle multi table joins query, > about > its performance and whether it uses any type of caching. > > Thanks, > Tommy Thinking of those who might later be searching the archives, perhaps further discussion of caching should be broken up into its own thread, as it seems to be moving off-topic. Let it snow, let it snow, let it snow. Adam -- Nephtali: A simple, flexible, fast, and security-focused PHP framework http://nephtaliproject.com
Re: [PHP] ORM doctrine
Lester Caine wrote: For fixed pages this is the best way of handling the information. And handling those fixed pages is ... from my point of view ... not a problem since they can be cached at that level, or even stored locally in the browser cache. I've just been hitting re-load every time for a few updates I've just been processing! In order to actually see the result. But for the majority of my work, the data to be displayed is being rebuilt with every browser hit. In that case generating dynamic pages fast becomes the bottleneck. If you've got an example, and you'd like to know how to approach these problems, I'd be happy to go through the process of making these always dynamic pages HTTP friendly with you :) (and on the list or in private) Best, Nathan -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] code quest - ECHO?!?
They are almost identical. Echo supports multiple parameters like "echo $a, $b;" print is 20% slower than echo (by some tests). "echo" is shorter than "print" so it's easy to write. In fact it's all a matter of taste. The same reason we user die instead of exit. Alex On Sun, Dec 12, 2010 at 6:23 PM, Kirk Bailey wrote: > Groovy; they appear to be identical in all but name. IDENTICAL. Or am I > missing a subtle definition difference? > > > > David Robley wrote: > >> Kirk Bailey wrote: >> >> >> >>> Ok, so what is echo, and how is it different from print. >>> >>> The code in code quest used echo. I have a copy of learning php 5.0 from >>> O'Reilly, and noplace does it mention echo. Why? What's the difference? >>> IS there a difference? Is there an advantage to either? Please clarify >>> for this newbie. >>> >>> >>> >> >> The documentation says it all better than I can: >> >> http://php.net/manual/en/function.echo.php >> http://php.net/manual/en/function.print.php >> >> Cheers >> >> > > -- > end > > Very Truly yours, >- Kirk Bailey, > Largo Florida > > kniht+-+ > | BOX | +-+think >
Re: [PHP] code quest - ECHO?!?
And the obvious difference, print returns true on success. I'm not sure what would cause an echo it print to ever fail, so it really doesn't make a huge difference. Thanks, Ash http://www.ashleysheridan.co.uk - Reply message - From: "Alexandru Patranescu" Date: Sun, Dec 12, 2010 18:56 Subject: [PHP] code quest - ECHO?!? To: "Kirk Bailey" Cc: They are almost identical. Echo supports multiple parameters like "echo $a, $b;" print is 20% slower than echo (by some tests). "echo" is shorter than "print" so it's easy to write. In fact it's all a matter of taste. The same reason we user die instead of exit. Alex On Sun, Dec 12, 2010 at 6:23 PM, Kirk Bailey wrote: > Groovy; they appear to be identical in all but name. IDENTICAL. Or am I > missing a subtle definition difference? > > > > David Robley wrote: > >> Kirk Bailey wrote: >> >> >> >>> Ok, so what is echo, and how is it different from print. >>> >>> The code in code quest used echo. I have a copy of learning php 5.0 from >>> O'Reilly, and noplace does it mention echo. Why? What's the difference? >>> IS there a difference? Is there an advantage to either? Please clarify >>> for this newbie. >>> >>> >>> >> >> The documentation says it all better than I can: >> >> http://php.net/manual/en/function.echo.php >> http://php.net/manual/en/function.print.php >> >> Cheers >> >> > > -- > end > > Very Truly yours, >- Kirk Bailey, > Largo Florida > > kniht+-+ > | BOX | +-+think >
RE: [PHP] ORM doctrine
> -Original Message-
> From: Nathan Rixham [mailto:nrix...@gmail.com]
> Sent: Sunday, December 12, 2010 8:23 AM
> To: Tommy Pham
> Cc: 'Peter Lind'; php-general@lists.php.net; 'Lester Caine'
> Subject: Re: [PHP] ORM doctrine
>
> Tommy Pham wrote:
> >> -Original Message-
> >> From: Peter Lind [mailto:peter.e.l...@gmail.com]
> >> Sent: Sunday, December 12, 2010 5:27 AM
> >> To: Lester Caine
> >> Cc: php-general@lists.php.net
> >> Subject: Re: [PHP] ORM doctrine
> >>
> >
> >
> >
> >>> The reason for 'caching' needs to be understood before it is applied
> >>> in
> >> order to avoid the black holes that random caching is causing
> >> nowadays already. How often do you hear "wipe the xxx browser cache"?
> >> And I know if I am changing theme elements in bitweaver or phpgedview
> >> then I HAVE to wipe the cache to ensure that smarty rebuilds the relevant
> pages.
> >>
> >> Which underlines my point: caching is not icing on the cake but
> >> should be thought of sooner in the process, contrary to Tommys point.
> >>
> >
> > If the app is well designed, then you still could implement it later w/o
> major rewrite.
>
> It's only well designed if caching is considered from the start - thus, as
> Peter
> says, caching is not the icing on the cake, but should, must, be thought of in
> the initial design process - if it's not, then the app isn't well designed.
I'll take a crack at it ;)
Bare minimum:
- parseRequest();
- fetchData();
- output();
With auth / acl:
parseRequest()
{
// parse
// add auth/acl and redirect accordingly
}
fetchData();
output();
With auth/acl + cache:
parseRequest()
{
// parse
// add auth/acl and redirect accordingly
}
fetchData()
{
If ($useCache) getCache();
else getFromDB();
}
output();
That seems to me as a minor rewrite with lots of additional modules from bare
minimum to auth/acl+cache, as I've stated before with the points: Understand
the problem, understand the objective + possible growth/expansion, app design
(framework and what not). So whether I choose to implement cache is relevant,
IMO, because data to me is either cache or from DB depending in specific cases
(which is handle by the caching module). If from cache, that's the cache
management problem (a shortcomings).
Regards,
Tommy
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] ORM doctrine
> -Original Message- > From: Bastien [mailto:phps...@gmail.com] > Sent: Sunday, December 12, 2010 9:20 AM > To: Nathan Rixham > Cc: Lester Caine; php-general List > Subject: Re: [PHP] ORM doctrine > > > > On 2010-12-12, at 11:45 AM, Nathan Rixham wrote: > > > Lester Caine wrote: > >> Nathan Rixham wrote: > >>> In your application itself, caching can be introduced at every > >>> level, you've already got filesystem io caches provided by the > >>> operating system, a well tuned db server cache can make a big > >>> difference as well, then on to opcode caches in languages like PHP > >>> since it's interpreted rather than compiled, and then on to code > >>> optimizations such as using static class cache's behind getByID > >>> methods and similar, and finally down to micro optimizations, static > >>> code analysis and dead code elimination, replacing (4*10) with (40), > >>> inlining static class members / constants and such like. Finally, > >>> language specific nuances and micro optimizations. > >> Actually THAT probably sums things up nicely. An approach suitable for > MySQL WILL probably be wrong for Postgres or Firebird. Certainly the > optimised SQL I use for my own applications is much improved if I simply > ignore compatibility with the databases other developers are using. > Libraries like doctrine and even PDO are probably not using the best > approach that a database specific approch may take. Firebird will maintain > the 'results' of previous searches in cache and give results sets a lot faster, > such as being ready to return the next page of results BEFORE the web page > asks for it ;) But a database agnostic approach is not as efficient. > > > > Yes, but you snipped a key paragraph there, because all the forms of > caching and optimization listed above, including db cache optimization will > only increase performance of the app by small percentages, whereas > moving to a publishing model and producing static output will increase > performance by several factors: > > > > [[[ > > As a web developer, the most important interface you need to introduce > caching on, is HTTP - a "static" or "published" HTML document which > leverages HTTP caching using Last-Modified / ETag headers will give circa > 100 times better performance (or more) than a dynamically generated each > time doc. Not only that, but transparent proxies on the network can come in > to effect and reduce weight on the server to zero, and further, web > browsers will serve repeat requests to the document straight from RAM, > again leading to zero weight. When you include those two factors it doesn't > take long to see that the performance difference over the network is so > good that it's nigh on unmeasurable. > > ]]] > > > > Feel free to ignore this yourself, but please don't promote a bit of SQL and > db server optimization as being the most important factor in optimizing PHP > applications, it is important, but the net result is minimal in comparison to > leveraging HTTP caching and static publishing of components or entire > documents. > > > > Best, > > > > Nathan > > > > -- > > PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: > > http://www.php.net/unsub.php > > > > I would definitely agree with Nathan here. In our classic asp (don't ask) the > vp of our company designed a survey app ( poorly I might add ) that > dynamically generated the form and all of it's associated drop downs > including the locations ( some 2000 of them ). The client was requiring 1000 > user concurrency and the app as it was only handled 10 users. I re- > architected the app to cache out the form since it was identical for each > user, and we easily hit the required benchmark. You're just proving the points I made. The app was poorly designed because he/she didn't truly understand my point #2: understand the objective + possible growth/expansion. Also, mostly like that he/she didn't fully understand point #1: understand the problem (needs of the business, programming language in use, etc). Failing both points 1 & 2 will make point 3 even far worse. Hence you had to 're-architect'. Regards, Tommy > > Form caching is one of the most important omptimizations that can be made > ( especially in our app where a major ( and I am not allowed to fix it ) design > flaw loops through the form and associated data set 4 ( yes, 4! ) times and > nothing is cached ). > > Db calls are expensive but not as expensive as extremely poor design! > > Bastien Koert > > Sent from my iPhone > -- -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Parsing a phrase
Hello all. I have a page where the user can enter a search phrase and upon submitting, the search phrase is queried in MySQL. However, I need to modify is so each word in the phrase is searched for... not just the exact phrase. So, "big blue hat" will return results like: "A big hat - blue in color" "Hat - blue, big" SQL would look like WHERE (item_description like "%big%" and item_description like "%blue %" and item_description like "%hat%" ) So, via PHP, what is the best way to extract each word from the search phrase to it's own variable so I can place them dynamically into the SQL statement. Thanks, --Rick -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] ORM doctrine
Tommy Pham wrote:
-Original Message-
From: Nathan Rixham [mailto:nrix...@gmail.com]
Sent: Sunday, December 12, 2010 8:23 AM
To: Tommy Pham
Cc: 'Peter Lind'; php-general@lists.php.net; 'Lester Caine'
Subject: Re: [PHP] ORM doctrine
Tommy Pham wrote:
-Original Message-
From: Peter Lind [mailto:peter.e.l...@gmail.com]
Sent: Sunday, December 12, 2010 5:27 AM
To: Lester Caine
Cc: php-general@lists.php.net
Subject: Re: [PHP] ORM doctrine
The reason for 'caching' needs to be understood before it is applied
in
order to avoid the black holes that random caching is causing
nowadays already. How often do you hear "wipe the xxx browser cache"?
And I know if I am changing theme elements in bitweaver or phpgedview
then I HAVE to wipe the cache to ensure that smarty rebuilds the relevant
pages.
Which underlines my point: caching is not icing on the cake but
should be thought of sooner in the process, contrary to Tommys point.
If the app is well designed, then you still could implement it later w/o
major rewrite.
It's only well designed if caching is considered from the start - thus, as Peter
says, caching is not the icing on the cake, but should, must, be thought of in
the initial design process - if it's not, then the app isn't well designed.
I'll take a crack at it ;)
Bare minimum:
- parseRequest();
- fetchData();
- output();
With auth / acl:
parseRequest()
{
// parse
// add auth/acl and redirect accordingly
}
fetchData();
output();
With auth/acl + cache:
parseRequest()
{
// parse
// add auth/acl and redirect accordingly
}
fetchData()
{
If ($useCache) getCache();
else getFromDB();
}
output();
That seems to me as a minor rewrite with lots of additional modules from bare
minimum to auth/acl+cache, as I've stated before with the points: Understand
the problem, understand the objective + possible growth/expansion, app design
(framework and what not). So whether I choose to implement cache is relevant,
IMO, because data to me is either cache or from DB depending in specific cases
(which is handle by the caching module). If from cache, that's the cache
management problem (a shortcomings).
That is only one form of relatively minor caching, and you've already
missed most of the opportunities because you're already in a dynamic
application / script environment there... try scoping out to the bigger
picture here:
3 UI instances with 2 different presentation tiers
2 Application instances
1 db instance
And here are pretty much the full entry points
GET /something HTTP/1.1
POST /processor HTTP/1.1
You're job is to respond to those calls as quickly as possible,
Here are six simple interface edges you can cache on:
1
--http---
2 | | |
UI UI UI
3 | | |
--
4| |
AppApp
5| |
--
6|
DB
You're talking about caching 5 or 6 levels down, thus already missing 4
other opportunities, which are exponentially more important.
Best,
Nathan
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Re: Parsing a phrase
Rick Dwyer wrote: Hello all. I have a page where the user can enter a search phrase and upon submitting, the search phrase is queried in MySQL. However, I need to modify is so each word in the phrase is searched for... not just the exact phrase. So, "big blue hat" will return results like: "A big hat - blue in color" "Hat - blue, big" SQL would look like WHERE (item_description like "%big%" and item_description like "%blue%" and item_description like "%hat%" ) You may be better to use full text and MATCH for this, see: http://dev.mysql.com/doc/refman/5.1/en/fulltext-boolean.html However.. So, via PHP, what is the best way to extract each word from the search phrase to it's own variable so I can place them dynamically into the SQL statement. There are many ways you can do this: http://php.net/explode http://php.net/str_split http://php.net/preg_split Many examples can be found on the above pages, and you're real solution depends on how many edge-cases you want to cover, but the above will cover most approaches :) Best, Nathan -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: Parsing a phrase
Thanks Nathan. The MySQL Match/Against will probably work well... but I would need to somehow add a "+" to the beginning of each word in the phrase so PHP will still be involved. --Rick On Dec 12, 2010, at 2:51 PM, Nathan Rixham wrote: Rick Dwyer wrote: Hello all. I have a page where the user can enter a search phrase and upon submitting, the search phrase is queried in MySQL. However, I need to modify is so each word in the phrase is searched for... not just the exact phrase. So, "big blue hat" will return results like: "A big hat - blue in color" "Hat - blue, big" SQL would look like WHERE (item_description like "%big%" and item_description like "%blue%" and item_description like "%hat%" ) You may be better to use full text and MATCH for this, see: http://dev.mysql.com/doc/refman/5.1/en/fulltext-boolean.html However.. So, via PHP, what is the best way to extract each word from the search phrase to it's own variable so I can place them dynamically into the SQL statement. There are many ways you can do this: http://php.net/explode http://php.net/str_split http://php.net/preg_split Many examples can be found on the above pages, and you're real solution depends on how many edge-cases you want to cover, but the above will cover most approaches :) Best, Nathan -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] ORM doctrine
> -Original Message-
> From: Nathan Rixham [mailto:nrix...@gmail.com]
> Sent: Sunday, December 12, 2010 11:41 AM
> To: Tommy Pham
> Cc: 'Peter Lind'; php-general@lists.php.net; 'Lester Caine'
> Subject: Re: [PHP] ORM doctrine
>
> Tommy Pham wrote:
> >> -Original Message-
> >> From: Nathan Rixham [mailto:nrix...@gmail.com]
> >> Sent: Sunday, December 12, 2010 8:23 AM
> >> To: Tommy Pham
> >> Cc: 'Peter Lind'; php-general@lists.php.net; 'Lester Caine'
> >> Subject: Re: [PHP] ORM doctrine
> >>
> >> Tommy Pham wrote:
> -Original Message-
> From: Peter Lind [mailto:peter.e.l...@gmail.com]
> Sent: Sunday, December 12, 2010 5:27 AM
> To: Lester Caine
> Cc: php-general@lists.php.net
> Subject: Re: [PHP] ORM doctrine
>
> >>>
> >>>
> > The reason for 'caching' needs to be understood before it is
> > applied in
> order to avoid the black holes that random caching is causing
> nowadays already. How often do you hear "wipe the xxx browser
> cache"?
> And I know if I am changing theme elements in bitweaver or
> phpgedview then I HAVE to wipe the cache to ensure that smarty
> rebuilds the relevant
> >> pages.
> Which underlines my point: caching is not icing on the cake but
> should be thought of sooner in the process, contrary to Tommys point.
>
> >>> If the app is well designed, then you still could implement it later
> >>> w/o
> >> major rewrite.
> >>
> >> It's only well designed if caching is considered from the start -
> >> thus, as Peter says, caching is not the icing on the cake, but
> >> should, must, be thought of in the initial design process - if it's not,
> >> then
> the app isn't well designed.
> >
> > I'll take a crack at it ;)
> >
> > Bare minimum:
> > - parseRequest();
> > - fetchData();
> > - output();
> >
> > With auth / acl:
> > parseRequest()
> > {
> > // parse
> > // add auth/acl and redirect accordingly } fetchData(); output();
> >
> > With auth/acl + cache:
> > parseRequest()
> > {
> > // parse
> > // add auth/acl and redirect accordingly }
> > fetchData()
> > {
> >If ($useCache) getCache();
> > else getFromDB();
> > }
> > output();
> >
> > That seems to me as a minor rewrite with lots of additional modules from
> bare minimum to auth/acl+cache, as I've stated before with the points:
> Understand the problem, understand the objective + possible
> growth/expansion, app design (framework and what not). So whether I
> choose to implement cache is relevant, IMO, because data to me is either
> cache or from DB depending in specific cases (which is handle by the caching
> module). If from cache, that's the cache management problem (a
> shortcomings).
>
> That is only one form of relatively minor caching, and you've already missed
> most of the opportunities because you're already in a dynamic application /
> script environment there... try scoping out to the bigger picture here:
>
> 3 UI instances with 2 different presentation tiers
> 2 Application instances
> 1 db instance
>
> And here are pretty much the full entry points
>
>GET /something HTTP/1.1
>
>POST /processor HTTP/1.1
>
> You're job is to respond to those calls as quickly as possible,
>
> Here are six simple interface edges you can cache on:
>
> 1
>--http---
> 2 | | |
> UI UI UI
> 3 | | |
> --
> 4| |
> AppApp
> 5| |
>--
> 6|
> DB
>
> You're talking about caching 5 or 6 levels down, thus already missing 4 other
> opportunities, which are exponentially more important.
>
> Best,
>
> Nathan
Example, user never been to site before and found link on google. 1st thing
that has to be done is to insure that url is valid format and not something
like /goto/here/../try/to/hack/your/server or induce some other URL hacking to
create a buffer overflow. Does this involve cache? Hence parseRequest().
Second, if the URL is valid, check if the user is logged in and authorized? If
the user is authorized, continue processing the request and get the data.
Thus: if ($useCache) getCache(); /* handles by cache module for all that’s
needed static/dynamic */ else getFromDb();. Finally, output() is responsible
for transmitting the info whether it was from cache or DB. Such as if
$countries represents the list of countries, then both cache & db should return
the exact values (provided no change since last cache fetch). Now that the
user has been to the site and clicks on a link on the page which sends request
back. Again, parseRequest(); fetchData(); and the cache module within the
fetchData() has to determine where to get the required data... again - cache
management problem. And so on so forth... Whether is caching is done via local
Re: [PHP] Re: Parsing a phrase
I have it working now using preg_replace. --Rick On Dec 12, 2010, at 3:50 PM, Rick Dwyer wrote: Thanks Nathan. The MySQL Match/Against will probably work well... but I would need to somehow add a "+" to the beginning of each word in the phrase so PHP will still be involved. --Rick On Dec 12, 2010, at 2:51 PM, Nathan Rixham wrote: Rick Dwyer wrote: Hello all. I have a page where the user can enter a search phrase and upon submitting, the search phrase is queried in MySQL. However, I need to modify is so each word in the phrase is searched for... not just the exact phrase. So, "big blue hat" will return results like: "A big hat - blue in color" "Hat - blue, big" SQL would look like WHERE (item_description like "%big%" and item_description like "%blue%" and item_description like "%hat%" ) You may be better to use full text and MATCH for this, see: http://dev.mysql.com/doc/refman/5.1/en/fulltext-boolean.html However.. So, via PHP, what is the best way to extract each word from the search phrase to it's own variable so I can place them dynamically into the SQL statement. There are many ways you can do this: http://php.net/explode http://php.net/str_split http://php.net/preg_split Many examples can be found on the above pages, and you're real solution depends on how many edge-cases you want to cover, but the above will cover most approaches :) Best, Nathan -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] ORM doctrine
> -Original Message-
> From: Tommy Pham [mailto:tommy...@gmail.com]
> Sent: Sunday, December 12, 2010 12:53 PM
> To: 'nrix...@gmail.com'
> Cc: 'Peter Lind'; 'php-general@lists.php.net'; 'Lester Caine'
> Subject: RE: [PHP] ORM doctrine
>
> > -Original Message-
> > From: Nathan Rixham [mailto:nrix...@gmail.com]
> > Sent: Sunday, December 12, 2010 11:41 AM
> > To: Tommy Pham
> > Cc: 'Peter Lind'; php-general@lists.php.net; 'Lester Caine'
> > Subject: Re: [PHP] ORM doctrine
> >
> > Tommy Pham wrote:
> > >> -Original Message-
> > >> From: Nathan Rixham [mailto:nrix...@gmail.com]
> > >> Sent: Sunday, December 12, 2010 8:23 AM
> > >> To: Tommy Pham
> > >> Cc: 'Peter Lind'; php-general@lists.php.net; 'Lester Caine'
> > >> Subject: Re: [PHP] ORM doctrine
> > >>
> > >> Tommy Pham wrote:
> > -Original Message-
> > From: Peter Lind [mailto:peter.e.l...@gmail.com]
> > Sent: Sunday, December 12, 2010 5:27 AM
> > To: Lester Caine
> > Cc: php-general@lists.php.net
> > Subject: Re: [PHP] ORM doctrine
> >
> > >>>
> > >>>
> > > The reason for 'caching' needs to be understood before it is
> > > applied in
> > order to avoid the black holes that random caching is causing
> > nowadays already. How often do you hear "wipe the xxx browser
> > cache"?
> > And I know if I am changing theme elements in bitweaver or
> > phpgedview then I HAVE to wipe the cache to ensure that smarty
> > rebuilds the relevant
> > >> pages.
> > Which underlines my point: caching is not icing on the cake but
> > should be thought of sooner in the process, contrary to Tommys
> point.
> >
> > >>> If the app is well designed, then you still could implement it
> > >>> later w/o
> > >> major rewrite.
> > >>
> > >> It's only well designed if caching is considered from the start -
> > >> thus, as Peter says, caching is not the icing on the cake, but
> > >> should, must, be thought of in the initial design process - if it's
> > >> not, then
> > the app isn't well designed.
> > >
> > > I'll take a crack at it ;)
> > >
> > > Bare minimum:
> > > - parseRequest();
> > > - fetchData();
> > > - output();
> > >
> > > With auth / acl:
> > > parseRequest()
> > > {
> > > // parse
> > > // add auth/acl and redirect accordingly } fetchData(); output();
> > >
> > > With auth/acl + cache:
> > > parseRequest()
> > > {
> > > // parse
> > > // add auth/acl and redirect accordingly }
> > > fetchData()
> > > {
> > >If ($useCache) getCache();
> > > else getFromDB();
> > > }
> > > output();
> > >
> > > That seems to me as a minor rewrite with lots of additional modules
> > > from
> > bare minimum to auth/acl+cache, as I've stated before with the points:
> > Understand the problem, understand the objective + possible
> > growth/expansion, app design (framework and what not). So whether I
> > choose to implement cache is relevant, IMO, because data to me is
That's supposed to say 'cache is irrelevant'. Time for me to hit the sack
I've been up about 23 hours w/o caffeine :D
> > either cache or from DB depending in specific cases (which is handle
> > by the caching module). If from cache, that's the cache management
> > problem (a shortcomings).
> >
> > That is only one form of relatively minor caching, and you've already
> > missed most of the opportunities because you're already in a dynamic
> > application / script environment there... try scoping out to the bigger
> picture here:
> >
> > 3 UI instances with 2 different presentation tiers
> > 2 Application instances
> > 1 db instance
> >
> > And here are pretty much the full entry points
> >
> >GET /something HTTP/1.1
> >
> >POST /processor HTTP/1.1
> >
> > You're job is to respond to those calls as quickly as possible,
> >
> > Here are six simple interface edges you can cache on:
> >
> > 1
> >--http---
> > 2 | | |
> > UI UI UI
> > 3 | | |
> > --
> > 4| |
> > AppApp
> > 5| |
> >--
> > 6|
> > DB
> >
> > You're talking about caching 5 or 6 levels down, thus already missing
> > 4 other opportunities, which are exponentially more important.
> >
> > Best,
> >
> > Nathan
>
> Example, user never been to site before and found link on google. 1st thing
> that has to be done is to insure that url is valid format and not something
> like /goto/here/../try/to/hack/your/server or induce some other URL
> hacking to create a buffer overflow. Does this involve cache? Hence
> parseRequest(). Second, if the URL is valid, check if the user is logged in
> and authorized? If the user is authorized, continue processing the request
> and get the data. Thus: if ($useCache) getCache(); /* handles by cache
> module for all that’s needed s
Re: [PHP] Scalable Vector Graphics with PHP
Nathan, Thank you very much for your response, we are going to visualize network management system(topolgy view) with SVG and AJAX. We found a jQuery plugin, following are urls. http://plugins.jquery.com/project/svg http://keith-wood.name/svg.html On Thu, Dec 9, 2010 at 10:50 PM, Nathan Nobbe wrote: > On Thu, Dec 9, 2010 at 4:55 AM, sudarshana sampath < > sudarshanasamp...@gmail.com> wrote: > >> Hi, >> >> We are going add a topology view to our Network Management System. >> Our Network Management System is based on PHP, CakePHP, jQuery and other >> web >> related tools(middle tier written using C++). >> >> So we are going to work with Scalable Vector Graphics. >> >> We are looking for the best solution for doing that. >> >> Are there any extensions, plugins, frameworks available for SVG related >> things ? >> > > not sure exactly what youre trying to accomplish, but obviously you could > use any number of vector programs over the cli from php. > > you might also have a peak at the cairo library which php allegedly > supports (ive never tried it myself). > > http://us.php.net/manual/en/intro.cairo.php > > -nathan > -- Cheers, Sudarshana Sampath.
Re: [PHP] ORM doctrine
Nathan Rixham wrote: For fixed pages this is the best way of handling the information. And handling those fixed pages is ... from my point of view ... not a problem since they can be cached at that level, or even stored locally in the browser cache. I've just been hitting re-load every time for a few updates I've just been processing! In order to actually see the result. But for the majority of my work, the data to be displayed is being rebuilt with every browser hit. In that case generating dynamic pages fast becomes the bottleneck. If you've got an example, and you'd like to know how to approach these problems, I'd be happy to go through the process of making these always dynamic pages HTTP friendly with you :) (and on the list or in private) I'm quite happy with my current page generating process, but we are always looking for ways of improving the base bitweaver framework. When the original port was done from tikiwiki, we changed from a large percentage of processing time accessing the database to establish even if a user was allowed to access a page to something a lot more responsive and a lot less sensitive to DOS attack. Certainly improving the 'static' base on which the dynamic information is laid is an area where page generation times could be reduced. doctrine is an abstraction layer that I had a play with some time back, but in many cases we are building a multi-table query that returns all of the data in a single result set ... including the selected security model ... and that is something that ORM does not seem to provide an easy access to? But I stand to be corrected on that. -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk// Firebird - http://www.firebirdsql.org/index.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
