Test failures under Oracle

[Aymeric Augustin]

Hi folks,

At the moment, a few tests are failing under Oracle. I've created tickets for 
each of them:

https://code.djangoproject.com/ticket/20010
https://code.djangoproject.com/ticket/20011
https://code.djangoproject.com/ticket/20012
https://code.djangoproject.com/ticket/20013
https://code.djangoproject.com/ticket/20014
https://code.djangoproject.com/ticket/20015

The core team is notoriously under-qualified when it comes to maintaining 
support for Oracle — we don't have 6-figure budgets for consulting and Oracle's 
online docs are a sad joke.

If you'd like future versions of Django to continue supporting Oracle, please 
have a look at these tickets and help us fix them!

Thanks,

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Petite Abeille]

On Mar 10, 2013, at 9:52 AM, Aymeric Augustin 
 wrote:

>  Oracle's online docs are a sad joke

Specifically? The Oracle document is rather extensive and detailed. What's 
confusing you?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Aymeric Augustin]

On 10 mars 2013, at 10:34, Petite Abeille  wrote:

> On Mar 10, 2013, at 9:52 AM, Aymeric Augustin 
>  wrote:
> 
>> Oracle's online docs are a sad joke
> 
> Specifically? The Oracle document is rather extensive and detailed. What's 
> confusing you?


They're purely reference docs. They make lots of assumptions about things 
you're already supposed to know. They aren't educational at all.

If you're familiar with Oracle — ie. you paid for training & certification, or 
you're using it regularly — that's probably OK.

I'm only dealing with Oracle to fix failures in Django's test suite or ensure 
that new features are supported under Oracle. Clearly I'm not smart or 
knowledgeable enough to take advantage of its docs.

-- 
Aymeric.



-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Petite Abeille]

On Mar 10, 2013, at 12:58 PM, Aymeric Augustin 
 wrote:

> On 10 mars 2013, at 10:34, Petite Abeille  wrote:
> 
>> On Mar 10, 2013, at 9:52 AM, Aymeric Augustin 
>>  wrote:
>> 
>>> Oracle's online docs are a sad joke
>> 
>> Specifically? The Oracle document is rather extensive and detailed. What's 
>> confusing you?
> 
> 
> They're purely reference docs.

Yes, the reference documentation is very extensive.

> They make lots of assumptions about things you're already supposed to know.

Yes, references are not tutorials.

> They aren't educational at all.

Yes. On the other hand, there are large swath of more conceptual materials as 
well, such as the "Developer Essentials" series:

http://www.oracle.com/pls/db112/homepage

Granted, this can all be a bit of a mouthful due to the sheer amount of 
documentation and tutorial material available. But it's there.

> If you're familiar with Oracle — ie. you paid for training & certification, 
> or you're using it regularly — that's probably OK.

Sure enough.

> I'm only dealing with Oracle to fix failures in Django's test suite or ensure 
> that new features are supported under Oracle. Clearly I'm not smart or 
> knowledgeable enough to take advantage of its docs.

If you can formulate the issues in terms of SQL, then perhaps someone could try 
to help sort it out.


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Florian Apolloner]

Hi,

On Sunday, March 10, 2013 6:33:08 PM UTC+1, Petite Abeille wrote:
>
> > I'm only dealing with Oracle to fix failures in Django's test suite or 
> ensure that new features are supported under Oracle. Clearly I'm not smart 
> or knowledgeable enough to take advantage of its docs. 
> If you can formulate the issues in terms of SQL, then perhaps someone 
> could try to help sort it out. 
>

It's not always just SQL and even then, before formulating them in SQL it's 
easier to just ask the Oracle users to take a look at the failing issues 
and provide help there… Eg: https://code.djangoproject.com/ticket/20014 is 
a perfect example where someone with Oracle knowledge can chime in, but 
everyone else has probably hours in front of him to figure out how the 
query should look like (and once he has the query the issue is solved ;)).

Regards,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Ian Kelly]

On Sun, Mar 10, 2013 at 1:52 AM, Aymeric Augustin
 wrote:
> Hi folks,
>
> At the moment, a few tests are failing under Oracle. I've created tickets for 
> each of them:
>
> https://code.djangoproject.com/ticket/20010
> https://code.djangoproject.com/ticket/20011
> https://code.djangoproject.com/ticket/20012
> https://code.djangoproject.com/ticket/20013
> https://code.djangoproject.com/ticket/20014
> https://code.djangoproject.com/ticket/20015
>
> The core team is notoriously under-qualified when it comes to maintaining 
> support for Oracle — we don't have 6-figure budgets for consulting and 
> Oracle's online docs are a sad joke.
>
> If you'd like future versions of Django to continue supporting Oracle, please 
> have a look at these tickets and help us fix them!

#20015 looks like an expected failure.  Oracle supports lookups of
date fields using strings by implicitly converting the string to a
date.  The test is doing a startswith lookup that is going to produce
sql that looks something like "WHERE date_column LIKE '2008%'".
That's not going to work because Oracle can't convert '2008%' into a
date.  So the test here is testing that the
'supports_date_lookup_using_string' feature is more comprehensive than
Oracle actually supports.

I'll try to make time to look at the other tickets in more depth tomorrow.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Petite Abeille]

On Mar 10, 2013, at 7:04 PM, Florian Apolloner  wrote:

> It's not always just SQL and even then, before formulating them in SQL it's 
> easier to just ask the Oracle users to take a look at the failing issues 
> and provide help there… Eg: https://code.djangoproject.com/ticket/20014 is 
> a perfect example where someone with Oracle knowledge can chime in, but 
> everyone else has probably hours in front of him to figure out how the 
> query should look like (and once he has the query the issue is solved ;)).

Assuming this are the tests:

https://github.com/django/django/blob/master/tests/introspection/tests.py

Picking a random example:

# The following test fails on Oracle due to #17202 (can't correctly
# inspect the length of character columns).
@expectedFailureOnOracle
def test_get_table_description_col_lengths(self):
cursor = connection.cursor()
desc = connection.introspection.get_table_description(cursor, 
Reporter._meta.db_table)
self.assertEqual(
[r[3] for r in desc if datatype(r[1], r) == 'CharField'],
[30, 30, 75]
)

get_table_description is define as:

def get_table_description(self, cursor, table_name):
"Returns a description of the table, with the DB-API cursor.description 
interface."
cursor.execute("SELECT * FROM %s WHERE ROWNUM < 2" % 
self.connection.ops.quote_name(table_name))
description = []
for desc in cursor.description:
description.append(FieldInfo(*((desc[0].lower(),) + desc[1:])))
return description


https://github.com/django/django/blob/master/django/db/backends/oracle/introspection.py#L46

In this case, two factors are playing against you:

(1) Whereabout way to get table metadata (i.e. query the table to figure out 
its data to figure out its meta data). Instead, using the data dictionary 
directly would be more reliable and to the point, e.g. select owner, 
table_name, column_name, ... from [user|all]_tab_columns. 

(2) Distinction between char length vs. byte length. See DATA_LENGTH vs. 
CHAR_LENGTH vs. CHAR_USED. Related to NLS_LENGTH_SEMANTICS. The short of it, 
bytes != chars.


Some other random comments:

(A)

def table_name_converter(self, name):
"Table name comparison is case insensitive under Oracle"
return name.lower()

https://github.com/django/django/blob/master/django/db/backends/oracle/introspection.py#L54

That's not quite the case, even if it would appear so. Contrast "Fubar" vs. 
"FUBAR" vs FUBAR. (note the double quotes). See quoted identifier vs. nonquoted 
identifier. The short of it, identifier can be case sensitive, even though it's 
best to stay clear from such a deep rabbit hole.


(B) oracle / introspection.py uses the USER_ flavor of the data dictionary 
(e.g. USER_TABLES).  The USER_ flavor only shows objects which are directly 
owned by the schema. Which may be quite restrictive. You may be better off 
using the ALL_ flavor, which shows all the objects visible to the schema, 
irrespectively of ownership.


( C )  Try to formulate queries using the ANSI join syntax instead of the 
legacy Oracle one (i.e. left join vs. (+) ). The ANSI syntax is clearer, less 
error prone, and, well, more portable.


As far as that test_get_key_columns failure goes, I couldn't track down the 
code for connection.introspection.get_key_columns… but I suspect it has 
something to do with point (1)...






-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Petite Abeille]

On Mar 10, 2013, at 7:11 PM, Ian Kelly  wrote:

> #20015 looks like an expected failure.  Oracle supports lookups of
> date fields using strings by implicitly converting the string to a
> date.  The test is doing a startswith lookup that is going to produce
> sql that looks something like "WHERE date_column LIKE '2008%'".

If you have a date type, stick to a date type.  Converting to a varchar to run 
a like query is counterproductive. If one is looking for a date, just look for 
a date, e.g. ' extract( year from date_column ) = 2008 ' or such.

Otherwise, you must explicitly convert the date to a specific varchar format, 
e.g. . to_char( date_column, 'MMDD' ) like '2008%'

If you are not explicit, the session NLS format is applied, with unpredictable 
results.

> That's not going to work because Oracle can't convert '2008%' into a
> date.  So the test here is testing that the
> 'supports_date_lookup_using_string' feature is more comprehensive than
> Oracle actually supports.

Such string to date conversion depends on the session NSL settings if not 
explicit.


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Petite Abeille]

On Mar 10, 2013, at 7:56 PM, Petite Abeille  wrote:

> If you are not explicit, the session NLS format is applied, with 
> unpredictable results.

For the record, one can always set this explicitly at the session level as 
well, e.g.:

alter session set nls_date_format = '-MM-DD"T"HH24:MI:SS' or such. Ditto 
for timestamps.


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Aymeric Augustin]

On 10 mars 2013, at 20:08, Petite Abeille  wrote:

> 
> On Mar 10, 2013, at 7:56 PM, Petite Abeille  wrote:
> 
>> If you are not explicit, the session NLS format is applied, with 
>> unpredictable results.
> 
> For the record, one can always set this explicitly at the session level as 
> well, e.g.:
> 
> alter session set nls_date_format = '-MM-DD"T"HH24:MI:SS' or such. Ditto 
> for timestamps.


Django does this already:
https://github.com/django/django/blob/master/django/db/backends/oracle/base.py#L548-L555

-- 
Aymeric.



-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Petite Abeille]

On Mar 10, 2013, at 8:28 PM, Aymeric Augustin 
 wrote:

> Django does this already:
> https://github.com/django/django/blob/master/django/db/backends/oracle/base.py#L548-L555

Perfect. 

In the case of #20015, the issue is the other way round… the literal '2008%' 
cannot be implicitly converted to a date. Implicit conversion always go to the 
narrowest type. You would be better off converting it explicitly, e.g. to_date( 
'2008', ''  ) or such. Then the query become date >= to_date( '2008', 
''  ). Alternatively, you could convert the date column to a varchar: 
to_char( date ) like '2008%'… 



-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Florian Apolloner]

Hi,

On Sunday, March 10, 2013 7:48:02 PM UTC+1, Petite Abeille wrote:
>
> (1) Whereabout way to get table metadata (i.e. query the table to figure 
> out its data to figure out its meta data). Instead, using the data 
> dictionary directly would be more reliable and to the point, e.g. select 
> owner, table_name, column_name, ... from [user|all]_tab_columns. 
>

Patches welcome…

(B) oracle / introspection.py uses the USER_ flavor of the data dictionary 
> (e.g. USER_TABLES).  The USER_ flavor only shows objects which are directly 
> owned by the schema. Which may be quite restrictive. You may be better off 
> using the ALL_ flavor, which shows all the objects visible to the schema, 
> irrespectively of ownership. 
>

Probably, though no one complained so far ;)

As far as that test_get_key_columns failure goes, I couldn't track down the 
> code for connection.introspection.get_key_columns… but I suspect it has 
> something to do with point (1)... 
>

Well the issue is that nobody wrote get_key_columns yet, so we'd need a 
patch which adds this method to the oracle backend, examples can be taken 
from postgres 
https://github.com/django/django/blob/master/django/db/backends/postgresql_psycopg2/introspection.py#L70-85
 
Postgres supports information_schema which is part of the SQL standard, I 
don't think Oracle supports it. If you know the oracle tables, please 
provide us with the query which brings out the needed data…

Thx & Regards,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Petite Abeille]

On Mar 10, 2013, at 8:45 PM, Florian Apolloner  wrote:

> Patches welcome…

Yes, I wish I knew Python. Sadly I don't.  :)

> Well the issue is that nobody wrote get_key_columns yet, so we'd need a 
> patch which adds this method to the oracle backend, examples can be taken 
> from postgres 
> https://github.com/django/django/blob/master/django/db/backends/postgresql_psycopg2/introspection.py#L70-85
>  


If this is about getting the various constraints, this looks like 
'get_relations' method:

https://github.com/django/django/blob/master/django/db/backends/oracle/introspection.py#L65

> Postgres supports information_schema which is part of the SQL standard, I 
> don't think Oracle supports it.

No, but the Oracle data dictionary is much more extensive than the information 
schema, so it's rather straightforward to get what one wants:

http://docs.oracle.com/cd/B28359_01/server.111/b28318/datadict.htm

> If you know the oracle tables, please 
> provide us with the query which brings out the needed data…

I'm not clear what 'get_key_columns' does, but based on the Postgres code, I 
suspect it tries to get all the referential constraints keys, right? Looks like 
this is what 'get_relations' does as well already. Perhaps just a name mismatch.

Anyhow, for the record, here are various introspection queries examples which 
may or may not be of interest (these are aggregates, so not necessarily at the 
granularity you might want):


PrimaryKey
as
(
  select/*+ materialize */
all_cons_columns.owner,
all_cons_columns.table_name,
all_cons_columns.column_name,
all_cons_columns.constraint_name
  from  TableSet

  join  all_constraints
  onall_constraints.owner = TableSet.owner
  and   all_constraints.table_name = TableSet.table_name
  
  join  all_cons_columns
  onall_cons_columns.owner = all_constraints.owner
  and   all_cons_columns.constraint_name = all_constraints.constraint_name
  
  where all_constraints.constraint_type = 'P'
),
UniqueKey
as
(
  select/*+ materialize */
all_cons_columns.owner,
all_cons_columns.table_name,
all_cons_columns.column_name,
listagg( all_cons_columns.constraint_name, ', ' ) within group( 
order by all_cons_columns.constraint_name ) as constraint_name,
count( distinct all_cons_columns.constraint_name ) as 
constraint_count
  from  TableSet

  join  all_constraints
  onall_constraints.owner = TableSet.owner
  and   all_constraints.table_name = TableSet.table_name
  
  join  all_cons_columns
  onall_cons_columns.owner = all_constraints.owner
  and   all_cons_columns.constraint_name = all_constraints.constraint_name
  
  where all_constraints.constraint_type = 'U'

  group by  all_cons_columns.owner,
all_cons_columns.table_name,
all_cons_columns.column_name
),
ForeignKey
as
(
  select/*+ materialize */
all_cons_columns.owner,
all_cons_columns.table_name,
all_cons_columns.column_name,
listagg( all_cons_columns.constraint_name, ', ' ) within group( 
order by all_cons_columns.constraint_name ) as constraint_name,
count( distinct all_cons_columns.constraint_name ) as 
constraint_count,
listagg( r_cons_columns.owner || '.' || r_cons_columns.table_name 
|| '.' || r_cons_columns.column_name ) within group( order by 
r_cons_columns.owner, r_cons_columns.table_name, r_cons_columns.column_name ) 
as r_constraint_column,
listagg( r_cons_columns.constraint_name, ', ' ) within group( order 
by r_cons_columns.constraint_name ) as r_constraint_name
  from  TableSet

  join  all_constraints
  onall_constraints.owner = TableSet.owner
  and   all_constraints.table_name = TableSet.table_name
  
  join  all_cons_columns
  onall_cons_columns.owner = all_constraints.owner
  and   all_cons_columns.constraint_name = all_constraints.constraint_name

  join  all_constraints r_constraints
  onr_constraints.owner = all_constraints.r_owner
  and   r_constraints.constraint_name = all_constraints.r_constraint_name

  join  all_cons_columns r_cons_columns
  onr_cons_columns.owner = r_constraints.owner
  and   r_cons_columns.constraint_name = r_constraints.constraint_name
  and   r_cons_columns.position = all_cons_columns.position
  
  where all_constraints.constraint_type = 'R'

  group by  all_cons_columns.owner,
all_cons_columns.table_name,
all_cons_columns.column_name
),
IndexName
as
(
  select/*+ materialize */
all_ind_columns.table_owner as owner,
all_ind_columns.table_name,
all_ind_columns.column_name,
listagg( all_ind_columns.index_name, ', ' ) within group( order by 
all_ind_columns.index_name ) as index_name
  from  TableSet

  join  all_i

Re: Test failures under Oracle

[Florian Apolloner]

Hi,

On Sunday, March 10, 2013 9:00:57 PM UTC+1, Petite Abeille wrote:
>
> > Patches welcome… 
> Yes, I wish I knew Python. Sadly I don't.  :) 
>

Interesting. Out of curiosity may I ask what brought you to this ML then? 
(Don't get me wrong, it's just not that often that people write to this 
mailing list without using Python) 

If this is about getting the various constraints, this looks like 
> 'get_relations' method: 
>

Yeah, in general get_key_columns returns (for the table passed in): The 
names of columns which are foreign keys + the table name and column name of 
the target table.

I'm not clear what 'get_key_columns' does, but based on the Postgres code, 
> I suspect it tries to get all the referential constraints keys, right? 
> Looks like this is what 'get_relations' does as well already. Perhaps just 
> a name mismatch. 
>

It's certainly not a name mismatch but I agree that there might be some 
overlap.

Regards,
Florian 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Petite Abeille]

On Mar 10, 2013, at 8:28 PM, Aymeric Augustin 
 wrote:

> https://github.com/django/django/blob/master/django/db/backends/oracle/base.py#L548-L555

Last but not least… I couldn't help notice these suspicious looking operators:

https://github.com/django/django/blob/master/django/db/backends/oracle/base.py#L479
https://github.com/django/django/blob/master/django/db/backends/oracle/base.py#L495

E.g. TRANSLATE … USING NCHAR_CS… and LIKEC…

I suspect these are some sort of workarounds some fundamental charset encoding 
misunderstandings :)

Perhaps best to revisit that in light of "Supporting Multilingual Databases 
with Unicode" and "Programming with Unicode":

http://docs.oracle.com/cd/E14072_01/server.112/e10729/ch6unicode.htm
http://docs.oracle.com/cd/E14072_01/server.112/e10729/ch7progrunicode.htm

Just my 2¢ though.


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Petite Abeille]

On Mar 10, 2013, at 9:12 PM, Florian Apolloner  wrote:

> Interesting. Out of curiosity may I ask what brought you to this ML then? 

Ah, oh, yes, well… subscribed to the django mailing lists a while back to see 
what was all the fuss about :)

That specific thread caught my eye, as I'm familiar with Oracle. And sympathize 
with the OP apparent frustration with the beast. 

> (Don't get me wrong, it's just not that often that people write to this 
> mailing list without using Python) 

Yeah, well, slow Sunday morning :D

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Ian Kelly]

On Sun, Mar 10, 2013 at 1:44 PM, Petite Abeille
 wrote:
>
> On Mar 10, 2013, at 8:28 PM, Aymeric Augustin 
>  wrote:
>
>> Django does this already:
>> https://github.com/django/django/blob/master/django/db/backends/oracle/base.py#L548-L555
>
> Perfect.
>
> In the case of #20015, the issue is the other way round… the literal '2008%' 
> cannot be implicitly converted to a date. Implicit conversion always go to 
> the narrowest type. You would be better off converting it explicitly, e.g. 
> to_date( '2008', ''  ) or such. Then the query become date >= to_date( 
> '2008', ''  ). Alternatively, you could convert the date column to a 
> varchar: to_char( date ) like '2008%'…

The lookup requested by the user in this somewhat obscure scenario is
fundamentally a string lookup, though.  For example, this alternate
lookup is entirely legitimate as far as Django is concerned:

MyModel.objects.filter(somedate__startswith='200')

Which could match anything with the year 200 but more likely is
intended as a decade match, matching any date in the 200X decade.  To
support that, the query would have to become something like "somedate
between to_date('2000', '') and to_date('2009-12-31
23:59:59.99', '-MM-DD HH:MM:SS.FF')"

And there would have to be different conversions for other possible
substring lookups.  Generating those kinds of queries from Django
string lookups would be a hairy mess.

Digging into the code a bit further, I see that the postgresql backend
accomplishes this by tacking '::text' onto the end of the field name
when the lookup is a string-based lookup like startswith.  Oracle
could probably do something similar with the TO_CHAR function.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Petite Abeille]

On Mar 10, 2013, at 9:28 PM, Ian Kelly  wrote:

> Digging into the code a bit further, I see that the postgresql backend
> accomplishes this by tacking '::text' onto the end of the field name
> when the lookup is a string-based lookup like startswith.  Oracle
> could probably do something similar with the TO_CHAR function.

Yep… explicitly convert the date column to a varchar and apply the like clause 
to it. And you are done.

That said… from a db perspective… if you have more than an handful of rows… 
this is not going to fly much…  just saying…


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Ian Kelly]

On Sun, Mar 10, 2013 at 2:15 PM, Petite Abeille
 wrote:
>
> On Mar 10, 2013, at 8:28 PM, Aymeric Augustin 
>  wrote:
>
>> https://github.com/django/django/blob/master/django/db/backends/oracle/base.py#L548-L555
>
> Last but not least… I couldn't help notice these suspicious looking operators:
>
> https://github.com/django/django/blob/master/django/db/backends/oracle/base.py#L479
> https://github.com/django/django/blob/master/django/db/backends/oracle/base.py#L495
>
> E.g. TRANSLATE … USING NCHAR_CS… and LIKEC…
>
> I suspect these are some sort of workarounds some fundamental charset 
> encoding misunderstandings :)
>
> Perhaps best to revisit that in light of "Supporting Multilingual Databases 
> with Unicode" and "Programming with Unicode":
>
> http://docs.oracle.com/cd/E14072_01/server.112/e10729/ch6unicode.htm
> http://docs.oracle.com/cd/E14072_01/server.112/e10729/ch7progrunicode.htm

These particular lookups have a long history of being tweaked due to
users coming up with installations where the existing queries did not
work.  See tickets #5985, #11017 and #14149.  I'd rather not reopen
this issue unless the current implementation can documentably be shown
to be broken.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Petite Abeille]

On Mar 10, 2013, at 9:36 PM, Ian Kelly  wrote:

> These particular lookups have a long history of being tweaked due to
> users coming up with installations where the existing queries did not
> work.  See tickets #5985, #11017 and #14149.  I'd rather not reopen
> this issue unless the current implementation can documentably be shown
> to be broken.

Fair enough…

Another point perhaps worthwhile mentioning:

# There's no way for the DatabaseOperations class to know the
# currently active Oracle version, so we do some setups here.

https://github.com/django/django/blob/master/django/db/backends/oracle/base.py#L574

That information can be extracted from v$version, v$instance, 
dbms_db_version.version, product_component_version, etc, etc. The choice is 
yours.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Shai Berger]

On Sunday 10 March 2013, Florian Apolloner wrote:
> and provide help there… Eg: https://code.djangoproject.com/ticket/20014 is
> a perfect example where someone with Oracle knowledge can chime in,

Here's a starting point, a query for the constraints affecting a table, from 
the South backend for Oracle: 
https://bitbucket.org/andrewgodwin/south/src/8ecfffbaf5eba680bfe97e61fab6eeaaad53ea2a/south/db/oracle.py?at=default#cl-304

(If nobody else fixes these bugs, I'll definitely get involved, but I can't 
promise when. Thought I could at least share a pointer until then).

Shai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Memcache not caching big values (> 1Mb)

[Luciano Pacheco]

Hi all,

Memcache backend, using python-memcache silently ignores data bigger than
1Mb.

This is more serious on using cache for pages (full body) where it's more
likely to have larger data being cached.

Also, larger pages usually are the ones that need more cache, because their
more costly to generate.

I've propose a fix that address this issue and 2 more:

1 - Compress cache data
https://code.djangoproject.com/ticket/16116

Because body page has a good compression rate this feature helps a lot on
this issue.

2 - Pickle protocol
https://code.djangoproject.com/ticket/19810#no2
Most of django backends uses higher pickle protocol, but it isn't true for
Memcache

3 - cache_db session "forgetting" big values
https://code.djangoproject.com/ticket/16358

Here is the pull request to address these issues:
https://github.com/django/django/pull/736/commits

It has changes in the docs and memcache backends, but there isn't any test,
because I couldn't figure out how to test them. :( Any advice on how to
test them?

Any feedback is welcome, I'm willing to make any necessary changes to get
it on the 1.6 train :)

[],
-- 
Luciano Pacheco
blog.lucmult.com.br

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

MySQL, Django 1.6, Python3

[Norberto Bensa]

Hello,

I wanted to run Django with Python 3 but mysql was a showstopper until 
today. Thanks to G+ I found a few links of interest:

http://stackoverflow.com/questions/13320343/can-i-use-mysql-on-djangodev-1-6-x-with-python3-x

https://groups.google.com/forum/?fromgroups=#!topic/django-developers/CtTpgMjN5J8

https://github.com/clelland/MySQL-for-Python-3 (m4p3)


So I git-cloned django and m4p3 but I got a problem.


'use_unicode' is an invalid keyword argument for this functionRequest 
> Method:GETRequest URL:http://localhost:8080/propietario/Django Version:
> 1.6.dev20130310222819Exception Type:TypeErrorException Value:
>
> 'use_unicode' is an invalid keyword argument for this function
>
> Exception 
> Location:/home/ubuntu/.virtualenvs/test3/git/MySQL-for-Python-3/MySQLdb/connections.py
>  
> in __init__, line 175Python Executable:
> /home/ubuntu/.virtualenvs/test3/bin/pythonPython Version:3.2.3
>
>
I thought, well, let's disable use_unicode in my settings.py. I added 
OPTIONS: { 'use_unicode': False } to DATABASES.default, but no way. Same 
error. So I patched 
django/db/backends/mysql/base.py:DatabaseWrapper:get_connection_params() 
and commented 'use_unicode':True. And now I can use Django, Mysql, and 
Python 3


The question is: Is there any option I'm overlooking and this mix can be 
used with patching Django or is Mysql+python3 just not supported (for now)?


Thanks!
Norberto

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Testing a django_localflavor_XX package

[Alon Nisser]

how am I supposed to test this? tried following some examples in other 
repo, but all failed. I wrote the test itself but it Doesn't run due to 
"You must either define the environment variable DJANGO_SETTINGS_MODULE or 
call settings.configure() before accessing settings"
my *fork here *

since I'm a django testing newbie I'll be glad for detailed instructions

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Django Admin Revamp - Any updates?

[Pantelis Petridis]

I'd like to add yawd-admin  to the 
list. It's bootstrap, up-to date with the latest core admin changes, 
stable, the github master fully supports django1.5 and has many goodies 
like modal inlines, fancybox popups for add_related , custom widgets, db 
settings, google analytics support etc. 

On Friday, December 14, 2012 11:36:19 AM UTC+2, is_null wrote:
>
> There are *many* apps providing bootstrap templates for 
> django.contrib.admin, here a few:
>
> - https://github.com/michaelhelmick/django-bootstrap-admin
> - https://github.com/gkuhn1/django-admin-templates-twitter-bootstrap
> - https://github.com/riccardo-forina/django-admin-bootstrapped
> - https://github.com/aobo711/bootstrap-django-admin
> - I myself did such templates.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Ian Kelly]

On Sun, Mar 10, 2013 at 3:03 PM, Petite Abeille
 wrote:
> Another point perhaps worthwhile mentioning:
>
> # There's no way for the DatabaseOperations class to know the
> # currently active Oracle version, so we do some setups here.
>
> https://github.com/django/django/blob/master/django/db/backends/oracle/base.py#L574
>
> That information can be extracted from v$version, v$instance, 
> dbms_db_version.version, product_component_version, etc, etc. The choice is 
> yours.

This is actually a timing concern.  The DatabaseOperations class can't
determine the Oracle version on its own because it doesn't have a
connection instance.  So instead the DatabaseWrapper selects the
appropriate regex method for the Oracle version when the connection is
created.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Update on localflavor move

[Tai Lee]

Hi Aymeric & Adrian,

I didn't see any further discussion or consensus on the issue of the 
generic localflavor. The 1.5 docs and (accelerated) deprecation of 
localflavor are a little hazy regarding the generic localflavor.

The 1.5 docs say that it hasn't been removed (yet), but doesn't say it will 
remain where it is, or move into core, or simply disappear with no 
equivalent external package.

The docs also said that (all of) `django.contrib.localflavor` would be 
removed, and any import below `django.contrib.localflavor` currently raises 
a warning with a message to use `django-localflavor-*` packages instead. 
But for generic, there is none.

So as it stands, anyone who uses `django.contrib.localflavor.generic` (all 
my projects do) will find that it has been removed in 1.6 with no 
equivalent in core and no equivalent external package.

I would prefer to see these reinstated in django core for 1.6, and the 
1.5.x docs updated with an explanation on where they can be imported from 
when upgrading to 1.6.

It's probably a bit much to ask for the non-biased international format 
fields to become the defaults, but I would definitely like to see these 
move into core before `django.contrib.localflavor` goes away.

It would seem a bit silly to me to create an external 
`django-localflavor-generic` package with 3 international (not locale 
specific) format fields and no other localisation. Django has always had 
great international support in core, and I doubt these three fields are 
much of a burden to maintain.

Also, as a separate issue I noticed that only a handful of the 
`django-localflavor-*` packages are on PyPI. This makes the method 
inconsistent and potentially more difficult (depending on your locale) to 
add them as requirements to a project.

Cheers.
Tai.


On Tuesday, 16 October 2012 07:17:43 UTC+11, Aymeric Augustin wrote:
>
>
> 1) Can we move the fields defined in 
> django.contrib.localflavor.generic.forms to django.forms? 
>
> Currently the US-biased fields are in django.forms and the non-biased ones 
> in django.contrib.localflavor.generic.forms — an interesting perspective :) 
>
> The alternative is to deprecate them entirely. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.