On 18/05/11 2:46 AM, Chris Corbyn wrote:
This looks really interesting, thanks for sharing. I'll have a play
around tomorrow. I wonder if it would be able to do this at the adapter
level in some way:
records = SqlQuery.execute(User.all(:something => "other") |
User.all(:whatever => "else"))
or a method on the DataObjects adapter
records =
DataMapper.repository(:default).adapter.execute_query(User.all(:something =>
"other") | User.all(:whatever => "else"))
yep, from emmanual's reply:
collection = Admin.all(:role => :super_admin)
repository.adapter.select(
repository.adapter.send(:select_statement, collection.query)
)
Then you get the benefit of all the associations you've defined, but you
by-pass the creation of all the model instances. Since the SQL isn't
executed until the recordset is iterated, there's theoretically time to
capture the query and do it without DM.
Or maybe a whole bunch of stuff that happens before any resources are
loaded is slow, so creating the query for passing to the adapter would
be an issue? Just thinking out loud.
no, query creation is fast
Did anyone ever work on other identity map backends?
https://github.com/datamapper/dm-core/wiki/Roadmap---Identity-Map
Although it sounds like the IM only kicks in for the most basic #get, so
it probably wouldn't help much as a caching strategy. If model
instantiation really is that expensive, and it's possible to use an IM
backed by something like Redis/memcached, you could in theory by-pass
the instantiation for anything already loaded, even if you've already
executed the SQL to get the keys you're interested in.
El 17/05/2011, a las 17:46, Xavier (DBIYF) escribió:
On 17/05/11 12:18 PM, Chris Corbyn wrote:
I'm looking at the page, then looking at your "2 seconds" and trying
to piece that together ;) How many models are needed (ballpark
figure) to render this page? Are you fetching meta data from from
other models (such as the name of the author, institution name,
comment counts, views etc)? We tend to denormalize commonly accessed
meta data that is "academically" normalized into different tables,
and we use observers to keep this denormalized data up-to-date. This
is often one of our biggest performance gains, since we cut down on
the number of models (and queries) we need to instantiate for things
like lists. Obviously indexes come before other stuff.
class heirachy of maybe seven models, 100s of objects. You can see
there is a lot of content there.
We have some refactoring underway to get this down significantly.
On the subject of caching. I understand DM uses an identity map. In
the context of a rails app, does the identity map live beyond the
duration of the request? Does this help to avoid the instantiation
overhead of the models on such a page? Also, is it possible to use an
identity map based off of something like memcached or Redis so that
multiple servers can enjoy the benefit of it? Effectively (assuming
the SQL you're running manually is the same SQL DM was running) you
can always pull those objects from a cache/identity map and only
update the cached version if it's changed. You can keep the cache
warmed via a background task.
No, identity map is per request (otherwise you'd run into memory issues).
Within the context of a request, populating the identity map (eager
loading records yourself so DM picks them up) didn't provide us with a
significant performance boost.
Can you share more about whether or not you involve DM in your pure
SQL approach, or is it entirely ad-hoc and losing the benefit of the
underlying data-mapper principles? I fear some of our pages may have
about the same, if not more, model usage as your front page does,
though we do tend to place a lot of focus on caching strategies
(template, denormalization, model (in some cases) and caching proxy
where we can). We are a very highly trafficked site (also based in
Australia incidentally ;)) and occasionally fall subject to DDOS
attacks (fortunately not very elaborate ones so far and we
automatically scale up). TBH, I doubt DM could be any more expensive
that our in-house PHP solution that was developed about 6 years ago
just as PHP frameworks were beginning to appear and has just had
layer upon layer built on top of it, with very few core changes...
I've been pushing for a long time to move to something more modern
and actively developed, so it would be a shame if we f
ind that this means we're running more servers as a result (unlikely).
See my reply to Emmanuel.
Cheers,
Xavier
Just thinking out loud.
El 17/05/2011, a las 08:39, Xavier (DBIYF) escribió:
On 16/05/11 10:13 PM, mileszs wrote:
I recently gave a presentation at the Indianapolis Ruby Brigade
monthly meeting, covering aspects of DM I like, and recommending the
crowd give it a shot in their next project. I was asked about
performance, as I knew I would be, but I hadn't really taken a good
run at a performance comparison at the time (which seems like a huge
failure on my part, considering I've put a Rails 3 DM app in to
production). Thus, I sat down yesterday intending to get some numbers
I could show people. Unfortunately, the results were surprisingly
negative for DataMapper. I am hoping that I'm unwittingly performing a
poor benchmark.
As a real world data point, our home page takes upwards of two
seconds to render with datamapper, and ~300ms using SQL wrapped with
structs. I haven't been able to replicate this in a stand alone
rails app yet, and haven't compared to AR. DM object instantiation
is pretty expensive.
So YMMV, but in our experience we have found DM to be excellent for
our backend stuff but not ideal for high traffic pages.
(page caching is an option, but I believe it covers up the
underlying issue)
[1] theconversation.edu.au <http://theconversation.edu.au>, there's
a lot of stuff on it
Xavier
--
You received this message because you are subscribed to the Google
Groups "DataMapper" group.
To post to this group, send email to [email protected]
<mailto:[email protected]>.
To unsubscribe from this group, send email to
[email protected]
<mailto:[email protected]>.
For more options, visit this group at
http://groups.google.com/group/datamapper?hl=en.
--
You received this message because you are subscribed to the Google
Groups "DataMapper" group.
To post to this group, send email to [email protected]
<mailto:[email protected]>.
To unsubscribe from this group, send email to
[email protected]
<mailto:[email protected]>.
For more options, visit this group at
http://groups.google.com/group/datamapper?hl=en.
--
You received this message because you are subscribed to the Google
Groups "DataMapper" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/datamapper?hl=en.
--
You received this message because you are subscribed to the Google Groups
"DataMapper" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/datamapper?hl=en.
