Guido Günther wrote:
Searches seems to not use the index when there's a time-range element in
the filter, but the exception detecting this is only thrown from the
filter expression generator for prop-filters, ... which again means that
if you don't have a prop-filter in your query, you'll get a search using
the index, which will ignore these 356+ days events.
Could you work on a fix? It seems you've got most of the details
together already. I attached your analysis to Trac ticket #287.
I don't have any "perfect" fix. That would be a more complex patch and
would have to take into account upcoming changes for 1.3.
Here's some more analysis and suggestions for work-arounds, though:
According to RFC4791 a simple calendar-query REPORT would look like this:
================
<calendar-query xmlns:D="DAV:" xmlns="urn:ietf:params:xml:ns:caldav">
<D:prop>
<calendar-data/>
</D:prop>
<filter>
<comp-filter name="VCALENDAR">
<comp-filter name="VEVENT">
<time-range start="20070420T120039Z" end="20070620T120039Z"/>
</comp-filter>
</comp-filter>
</filter>
</calendar-query>
=================
The "time-range" element is almost in all use cases part of a
comp-filter. (It could also be used in a prop-filter.)
When an event is inserted into the database its instances are calculated
up to 356 days into the future. Any such instances are put into the
TIMESPAN table of the index. If this is not a complete expansion
RECURRANCE_MAX is set in the RESOURCE table to indicate how far we have
indexed this event.
As wsanchez says in Apples Trac ticket 207:
"The index is supposed to indicate how far out expansion has happened
for each event, and if a query against the index goes beyond that time,
the expansion should be brought out farther, up to a defined limit.
If we have expanded to the limit, then any queries that goes beyond the
limit would have to read the icalendar data to see if that event matches
(far more expensive, but the idea is that such queries should be rare)."
However... RECURRANCE_MAX is never used in the DCS 1.2 code as a quick
grep of the source will show!!
So no where is the expansion ever brought out farther for events beyond
356 days from creation!
This means that searches using the index will never see such events -
even though they are stored correctly in the database.
Searches are using the index if index_query_ok becomes true in
twistedcaldav/method/report_calquery.py which it does when
index.searchValid is true which in turn is the case when
query.calendarquery.sqlcalendarquery does not encounter a ValueError.
Now, what to do about it?
In the DCS v/1.2 source in twistedcaldav/query/calendarquery.py line 159
a ValueError exception is raised since we don't handle time-range within
prop-filter. Obviously (as described above) we don't correctly handle
time-range within comp-filter, but the equivalent exception is not
raised in line 102-105.
On fix would be to completely disable use of the index by just raising a
ValueError in calendarquery.py by replacing line 104, 105 with
"raise ValueError".
This disables the whole TIMESPAN index.
Another not as brutal, but equally simple fix would be to take
RECURRANCE_MAX into account, but modifying the SQL qualifier which
decides which events to include in the search result which is defined in
twistedcaldav/query/sqlgenerator.py line 49:
Changing:
TIMESPANTEST = "((TIMESPAN.FLOAT == 'N' AND TIMESPAN.START < %s AND
TIMESPAN.END > %s) OR (TIMESPAN.FLOAT == 'Y' AND TIMESPAN.START < %s AND
TIMESPAN.END > %s)) AND TIMESPAN.NAME == RESOURCE.NAME"
To:
TIMESPANTEST = "((((TIMESPAN.FLOAT == 'N' AND TIMESPAN.START < %s AND
TIMESPAN.END > %s) OR (TIMESPAN.FLOAT == 'Y' AND TIMESPAN.START < %s AND
TIMESPAN.END > %s)) AND TIMESPAN.NAME == RESOURCE.NAME) OR
(RESOURCE.RECURRANCE_MAX < %s))"
This will use the index for actually indexed events but also include
every event beyond the index expansion date.
This will return "wrong" results and thus violate the CalDAV RFC section
7.8 which says:
"The response body for a successful CALDAV:calendar-query REPORT
request MUST contain a DAV:response element for each iCalendar
object that matched the search filter."
... but that would be ok IFF the only clients accessing the server do
their own recurrence expansion and filtering.
The solution seems to be in the roadmap for DCS 1.3, however I would
regard this as a major flaw for 1.2 users.
It certainly is.
Unfortunately, Apple have bumped the milestone for this bug to 2.x.
I don't understand that. This is a serious blocker. The calendar can not
handle events like "same time next year" at all - which I would assume
is often used.
/Peter
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
