Re: Last call: Meta API (GSoC 2014/PR 3114) is ready for commit

2015-01-07 Thread Daniel Pyrathon
Hi all,

My project has just landed in master, since last night (EU time).

I just wanted to say THANK YOU for everyone that helped me review and fix 
my project. I would have never managed to reach such maturity by myself, 
and one of the best things is that I really learned a lot by contributing 
to Django (Yes! this is also a pitch to any potential SoC applicants for 
next year).

First of all, I want to thank my mentor (Russell) for his *enormous* 
patience and for all the time he dedicated to the project.
Secondly, I want to thank all the community and everyone who got involved 
in my pull request by submitting comments and/or contacting me directly for 
suggestions and advice, your suggestions have greatly shaped the final 
release and have also helped me improve as a developer.
Last but not least, I want to (re)thank the people from Django Under the 
Hood because the brainstorming that happened over the sprints as well as 
the very technical talks have greatly helped this project grow.

This is not a farewell, I plan on sticking around (as I am sure tickets 
will be assigned to me very soon!) so feel free to blame me on IRC or 
Twitter (PirosB3).

Daniel Pyrathon

On Monday, January 5, 2015 11:11:13 AM UTC+1, Tom Christie wrote:
>
> Wonderful - great work Daniel and thanks to everyone else for their hard 
> work on review & guidance.
> Lovely stuff.
>
>   Tom
>
> On Monday, 5 January 2015 05:12:41 UTC, Russell Keith-Magee wrote:
>>
>> Hi all,
>>
>> Following up on my pre-Christmas email - I believe that PR 3114 - the 
>> formalisation of _meta - is now ready to land.
>>
>> https://github.com/django/django/pull/3114
>>
>> The code has received a detailed review by several members of the core 
>> team, including Tim, Carl, Collin, and myself; we've had review notes from 
>> many others. The documentation has also been reviewed. The issues raised by 
>> those code and documentation reviews have all (to the best of my knowledge) 
>> been addressed.
>>
>> Other than last-minute cleanups (e.g., PEP8 compliance, -Wall checks 
>> etc), and the conclusion of one discussion about how to express one 
>> particular set of relationships in code, I believe the branch is ready to 
>> land.
>>
>> Unless there are objections, my intention is to land this patch later 
>> this week (around Wednesday/Thursday UTC). If anyone has any objections, 
>> speak now, etc etc.
>>
>> Once again, a huge thanks to Daniel for his great work over the summer 
>> (and continued work into the winter as we refined the patch for trunk), and 
>> to everyone else who has contributed to reviewing his work.
>>
>> Yours,
>> Russ Magee %-)
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/aad6a087-d142-4eae-a054-c96133256ae4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Gsoc 2015: SQLAlchemy / NoSQL integration

2015-03-08 Thread Daniel Pyrathon
Hi Abhishek,

I am the GSOC 2014 student who refactored the Meta API, please contact me
if you need any details on my work, or if you want to know more about
implementation and/or design of the new Meta API. You can also find me on
IRC as pirosb3.

Good luck!
Daniel

2015-03-02 1:32 GMT+01:00 Russell Keith-Magee :

>
> Hi Abhishek,
>
> On Sun, Mar 1, 2015 at 1:16 PM, Abhishek Kumar 
> wrote:
>
>> Hi,
>>
>> I am Abhishek and am interested in the project titled "SQLAlchemy / NoSQL
>> integration". I am good at data-structures and algorithms and have
>> worked with C,C++, PHP, Python,C#, Javascript and Mysql. I have an
>> internship experience at Microsoft, India.
>>
>> I have some questions which will help me in understanding the tasks
>> involved in this project better.
>>
>> 1)What was the formalised meta that came out of the 2014 soc project..?
>> Where to find the details of it..?
>>
>
> The new Meta interface is documented as part of Django 1.8:
>
> https://docs.djangoproject.com/en/1.8/ref/models/meta/
>
>
>> 2) I am new to Django. So i would be learning about it. Please let me
>> know what things i should focus more on w.r.t this project.
>>
>
> The most important part is probably Django's query language - you need to
> get comfortable with how you construct queries, and what those queries
> "mean" - both in the sense of the SQL they generate, and the underlying
> "idea" that the query is trying to capture.
>
> It would also be worth becoming familiar with Django's Admin interface -
> especially with regard to what queries it generates, and when.
>
>
>> 3) Before beginning to work for the project, what things i need to do to
>> make me eligible for it..? Bug fixes..?
>>
>
> The only hard requirement is to write a project proposal. However, if you
> get involved with the community, that may improve your chances, as we will
> get a chance to know you; if there are some details lacking in your
> proposal, we might be willing to overlook them if we have a better
> understanding of who you are, and how you interact with the community.
>
> Yours,
> Russ Magee %-)
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAJxq848ra-rbFzLZgktfMaU%3D%3DuYQstDSu1NFjD-XwG374401%2BQ%40mail.gmail.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
*

PirosB3

https://github.com/PirosB3

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAOmo0o93%2B3KwAxBNW7zOnE0AeQ0t088Q%2BP_BUip84YD2ZLJf_A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Gsoc 2015: SQLAlchemy / NoSQL integration

2015-03-21 Thread Daniel Pyrathon
Hi Abhishek,

I am in the process of updating the mailer in the next month. I will let
you know when this has been done.

Regards,
Daniel

2015-03-16 11:53 GMT+01:00 Abhishek Kumar :

> Hi,
>
> I was trying to run the django-mailer
>  written by Daniel. On doing
> migrate or runserver i am getting the following error:
>
> django.core.management.base.SystemCheckError: SystemCheckError: System
> check identified some issues:
>
> ERRORS:
> : (admin.E106) The value of
> 'mailchecker.admin.MessageInline.model' must be a Model.
>
> Any suggestions on resolving it..?
>
>
>
>
> On Monday, 2 March 2015 05:03:59 UTC+5:30, Abhishek Kumar wrote:
>>
>> Hi,
>>
>> I am Abhishek and am interested in the project titled "SQLAlchemy / NoSQL
>> integration". I am good at data-structures and algorithms and have
>> worked with C,C++, PHP, Python,C#, Javascript and Mysql. I have an
>> internship experience at Microsoft, India.
>>
>> I have some questions which will help me in understanding the tasks
>> involved in this project better.
>>
>> 1)What was the formalised meta that came out of the 2014 soc project..?
>> Where to find the details of it..?
>> 2) I am new to Django. So i would be learning about it. Please let me
>> know what things i should focus more on w.r.t this project.
>> 3) Before beginning to work for the project, what things i need to do to
>> make me eligible for it..? Bug fixes..?
>>
>> I will choose the particular  Nosql database later.
>>
>> Thanks,
>> Abhishek Kumar
>> CSE, IIIT Hyderabad
>> India
>>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/c0c29b70-c849-464b-b34f-a1c02cb3e4bf%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
*

PirosB3

https://github.com/PirosB3

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAOmo0o9iG4LVsru0kYSR9-W7C_deqS7%3D8UojV8gsfyyDUVvtNg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


[GSOC] Introduction and task proposal

2014-03-05 Thread Daniel Pyrathon


Hi,

My name is Daniel Pyrathon. I am currently a third year BSc student in 
Computer Science at the University of Plymouth. 

I love programming and I have always been active in the Open Source 
community (especially Python). In the past years I have written lots of 
Python, Javascript, Ruby, Java, and I am currently using C++ for many 
university projects. I have attended the last 3 EuroPython conferences and 
I have been a staff member of the conference for the last 2 years.

I am currently looking for a way to contribute to Django. Working on Django 
would increase my knowledge of the framework as well as let me share my own 
experience.

Reading the ideas list I found 2 of them that are very interesting for me, 
and so the reason behind this post is not only to present myself but also 
to discuss their feasibility.

Formalizing the Meta object

This task is very challenging because it digs in the internals of Django. I 
feel that I could learn a lot from it because I am very committed to 
refactoring and write most of my code in TDD. I have also experience with 
backwards compatibility. 

Do you have any resources (code) I should read to get up to date and to 
understand better how it is currently implemented?

Improved error reporting

The idea of making people’s lives better by improving error messages is 
fundamental. There would be a lot to discuss: what type of imports would we 
want to mask? I have read BetterErrorMessages and would be happy to get 
started soon. My idea behind this task would be to expand on this ticket: 
what would be great is to add a web console with live REPL support, similar 
to what Werkzeug debugger does. This could be a great starting point and 
would lead to a better use of Django.

Said this, I have to be very honest. I have never contributed to Django up 
till now and I want to hear your feedback on which proposal would suit me 
best. However I learn a lot through experience and I am attracted by new 
and challenging tasks.
Also, it would be nice if I could have some suggestions on what to read and 
if there are some specific parts of the code I should be directed to.

Please let me know,
Daniel Pyrathon

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0b1cc1e7-063a-48ea-9c92-eaa0344d396d%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: [GSOC] Introduction and task proposal

2014-03-11 Thread Daniel Pyrathon
Hi Russel,

Sorry for getting back now (I did not have notifications set up!). Thank 
you very much for the suggestions, I will have a look at models/options.py 
right away.

Regards,
Daniel Pyrathon

On Thursday, March 6, 2014 12:03:04 AM UTC, Russell Keith-Magee wrote:
>
> Hi Daniel,
>
> On Wed, Mar 5, 2014 at 11:48 PM, Daniel Pyrathon 
> 
> > wrote:
>
>> Hi,
>>
>> My name is Daniel Pyrathon. I am currently a third year BSc student in 
>> Computer Science at the University of Plymouth. 
>>
>> I love programming and I have always been active in the Open Source 
>> community (especially Python). In the past years I have written lots of 
>> Python, Javascript, Ruby, Java, and I am currently using C++ for many 
>> university projects. I have attended the last 3 EuroPython conferences and 
>> I have been a staff member of the conference for the last 2 years.
>>
>> I am currently looking for a way to contribute to Django. Working on 
>> Django would increase my knowledge of the framework as well as let me share 
>> my own experience.
>>
>> Reading the ideas list I found 2 of them that are very interesting for 
>> me, and so the reason behind this post is not only to present myself but 
>> also to discuss their feasibility.
>>
>> Formalizing the Meta object
>>
>> This task is very challenging because it digs in the internals of Django. 
>> I feel that I could learn a lot from it because I am very committed to 
>> refactoring and write most of my code in TDD. I have also experience with 
>> backwards compatibility. 
>>
>> Do you have any resources (code) I should read to get up to date and to 
>> understand better how it is currently implemented?
>>
>
> Unfortunately not; at least, not other than just trying to untangle the 
> mess that is django/db/models/options.py and the things that depend on it 
> (ModelForms and Admin in particular). This project really is the very model 
> of untangling a ball of string. The reason it's listed as a potential 
> project is specifically *because* there are no resources we can point you 
> at.
>
> If, after looking at the code, you feel it's a little "thin" for 12 weeks, 
> one way to bulk it out is to build a proof of concept alternate 
> implementation of Meta. Aside from the "it would be good to document this" 
> aspect, the practical reason for wanting to formalise Meta is that it would 
> allow people to build duck-typed implementations of Meta. If you can build 
> an alternate implementation of the Meta interface, then you should be able 
> to deploy your "duck" into a Django project and build a ModelForm, or 
> browse it in Admin.
>
> So, for example, you could build a wrapper around a NoSQL store, or around 
> an LDAP or email store, that *looked* like a Django model from the outside. 
> This means you could view your NoSQL data, or LDAP records, or emails in 
> Admin without needing to go through a SQL database. 
>
> The aim of proof of concept wouldn't be to commit something to Django's 
> core - it would be to build an external, standalone proof-of-concept, 
> demonstrating that your documentation was complete and correct. Depending 
> on what backend you choose, it might turn into a fully viable project on 
> it's own.
>  
>
>> Improved error reporting
>>
>> The idea of making people’s lives better by improving error messages is 
>> fundamental. There would be a lot to discuss: what type of imports would we 
>> want to mask? I have read BetterErrorMessages and would be happy to get 
>> started soon. My idea behind this task would be to expand on this ticket: 
>> what would be great is to add a web console with live REPL support, similar 
>> to what Werkzeug debugger does. This could be a great starting point and 
>> would lead to a better use of Django.
>>
>
> This is an interesting idea; however, I see two problems:
>
> 1) It would involve reinventing the wheel. Werkzeug exists, and does its 
> job well; a GSoC project to "duplicate Werkzeug" doesn't strike me as a 
> good use of GSoC resources. However, a project to integrate Werkzeug's live 
> debugging capabilities into Django might be more viable.
>
> 2) Security implications. Unfortunately, more than one site has been 
> launched with debug=True accidentally left on; all you need to do then is 
> stimulate a server error, and you have REPL shell access to the server. 
> This strikes me as a remarkably effective foot-gun :-) Before you get too 
> involved in the implementation, I'd want to know the security issues have 
> been locked down.
>  
&

Re: [GSOC] Introduction and task proposal

2014-03-18 Thread Daniel Pyrathon
Hi all,

As promised, I have been working on the formalizing Meta task.
I apologize to not have posted in the last days, but I have come back with 
my proposal. I want to post this now to the community, in order to gain 
feedback and re-iterate.

Formalizing Meta

Enabling users to build custom stores that work well with Django


https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit#

Kind regards,
Daniel

On Tuesday, March 11, 2014 10:48:43 PM UTC, Daniel Pyrathon wrote:
>
> Hi Russel,
>
> Sorry for getting back now (I did not have notifications set up!). Thank 
> you very much for the suggestions, I will have a look at models/options.py 
> right away.
>
> Regards,
> Daniel Pyrathon
>
> On Thursday, March 6, 2014 12:03:04 AM UTC, Russell Keith-Magee wrote:
>>
>> Hi Daniel,
>>
>> On Wed, Mar 5, 2014 at 11:48 PM, Daniel Pyrathon wrote:
>>
>>> Hi,
>>>
>>> My name is Daniel Pyrathon. I am currently a third year BSc student in 
>>> Computer Science at the University of Plymouth. 
>>>
>>> I love programming and I have always been active in the Open Source 
>>> community (especially Python). In the past years I have written lots of 
>>> Python, Javascript, Ruby, Java, and I am currently using C++ for many 
>>> university projects. I have attended the last 3 EuroPython conferences and 
>>> I have been a staff member of the conference for the last 2 years.
>>>
>>> I am currently looking for a way to contribute to Django. Working on 
>>> Django would increase my knowledge of the framework as well as let me share 
>>> my own experience.
>>>
>>> Reading the ideas list I found 2 of them that are very interesting for 
>>> me, and so the reason behind this post is not only to present myself but 
>>> also to discuss their feasibility.
>>>
>>> Formalizing the Meta object
>>>
>>> This task is very challenging because it digs in the internals of 
>>> Django. I feel that I could learn a lot from it because I am very committed 
>>> to refactoring and write most of my code in TDD. I have also experience 
>>> with backwards compatibility. 
>>>
>>> Do you have any resources (code) I should read to get up to date and to 
>>> understand better how it is currently implemented?
>>>
>>
>> Unfortunately not; at least, not other than just trying to untangle the 
>> mess that is django/db/models/options.py and the things that depend on it 
>> (ModelForms and Admin in particular). This project really is the very model 
>> of untangling a ball of string. The reason it's listed as a potential 
>> project is specifically *because* there are no resources we can point you 
>> at.
>>
>> If, after looking at the code, you feel it's a little "thin" for 12 
>> weeks, one way to bulk it out is to build a proof of concept alternate 
>> implementation of Meta. Aside from the "it would be good to document this" 
>> aspect, the practical reason for wanting to formalise Meta is that it would 
>> allow people to build duck-typed implementations of Meta. If you can build 
>> an alternate implementation of the Meta interface, then you should be able 
>> to deploy your "duck" into a Django project and build a ModelForm, or 
>> browse it in Admin.
>>
>> So, for example, you could build a wrapper around a NoSQL store, or 
>> around an LDAP or email store, that *looked* like a Django model from the 
>> outside. This means you could view your NoSQL data, or LDAP records, or 
>> emails in Admin without needing to go through a SQL database. 
>>
>> The aim of proof of concept wouldn't be to commit something to Django's 
>> core - it would be to build an external, standalone proof-of-concept, 
>> demonstrating that your documentation was complete and correct. Depending 
>> on what backend you choose, it might turn into a fully viable project on 
>> it's own.
>>  
>>
>>> Improved error reporting
>>>
>>> The idea of making people’s lives better by improving error messages is 
>>> fundamental. There would be a lot to discuss: what type of imports would we 
>>> want to mask? I have read BetterErrorMessages and would be happy to get 
>>> started soon. My idea behind this task would be to expand on this ticket: 
>>> what would be great is to add a web console with live REPL support, similar 
>>> to what Werkzeug debugger does. This could be a great starting point and 
>>> would lead to a bette

Re: [GSOC] Introduction and task proposal

2014-03-19 Thread Daniel Pyrathon
Hi!

Thanks for all the comments yesterday. They really helped me make the 
proposal stronger.
I have changed to proposal to reflect the changes. Would anyone like to 
have a look and, possibly, comment more?

https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit#heading=h.wyxx4xxijubt

Thanks,
Dan

On Wednesday, March 19, 2014 7:32:10 AM UTC, Aymeric Augustin wrote:
>
> On 19 mars 2014, at 06:27, Zach Borboa > 
> wrote: 
>
> > Curious, how do you get REPL shell access to the server with DEBUG=True 
> with a vanilla Django deployment? 
>
> That part of the discussion was about adding the werkzeug interactive 
> debugger to Django’s default error page. 
>
> -- 
> Aymeric. 
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/16264cb7-3f44-434d-865a-8c33baa4d921%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [GSOC] Introduction and task proposal

2014-03-26 Thread Daniel Pyrathon
Hi all,

It's been a while since I submitted my GSOC proposal. Although I am 
currently under exams, is there anything you would recommend me to do at 
this point (other than hope that my proposal is successful)?

Thanks,
Daniel Pyrathon

On Thursday, March 20, 2014 6:05:40 AM UTC, Russell Keith-Magee wrote:
>
>
> For the benefit of those reading along at home; I gave some revised notes 
> and had a conversation with Daniel on IRC a couple of hours ago.
>
> Yours,
> Russ Magee %-)
>
>
> On Thu, Mar 20, 2014 at 4:10 AM, Daniel Pyrathon 
> 
> > wrote:
>
>> Hi!
>>
>> Thanks for all the comments yesterday. They really helped me make the 
>> proposal stronger.
>> I have changed to proposal to reflect the changes. Would anyone like to 
>> have a look and, possibly, comment more?
>>
>>
>> https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit#heading=h.wyxx4xxijubt
>>
>> Thanks,
>> Dan
>>
>> On Wednesday, March 19, 2014 7:32:10 AM UTC, Aymeric Augustin wrote:
>>
>>> On 19 mars 2014, at 06:27, Zach Borboa  wrote: 
>>>
>>> > Curious, how do you get REPL shell access to the server with 
>>> DEBUG=True with a vanilla Django deployment? 
>>>
>>> That part of the discussion was about adding the werkzeug interactive 
>>> debugger to Django’s default error page. 
>>>
>>> -- 
>>> Aymeric. 
>>>
>>>
>>>
>>>
>>>  -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to 
>> django-d...@googlegroups.com
>> .
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/16264cb7-3f44-434d-865a-8c33baa4d921%40googlegroups.com<https://groups.google.com/d/msgid/django-developers/16264cb7-3f44-434d-865a-8c33baa4d921%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ad5d6016-2c6a-45b4-91c8-0e4500befc74%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [GSOC] Introduction and task proposal

2014-04-01 Thread Daniel Pyrathon
Hi Josh,

Sorry for getting back now. I have just finished my exam session, so I will 
be trying out the djangocore-box VM.

Regards,
Daniel Pyrathon

On Thursday, March 27, 2014 5:45:36 AM UTC+1, Josh Smeaton wrote:
>
> If you haven't already got all your databases installed on your 
> development machine, I *highly* recommend checking out 
> https://github.com/jphalip/djangocore-box. It's a vagrant VM environment 
> that already has MySQL, PostgreSQL, and SQLite installed (plus most GIS 
> installations), along with various versions of python. Unfortunately it 
> doesn't have Oracle - but being able to run tests on 3/4 databases is 
> especially handy.
>
> Josh
>
> On Thursday, 27 March 2014 11:14:35 UTC+11, Russell Keith-Magee wrote:
>>
>>
>> Hi Daniel,
>>
>> Nope - other than "cross your fingers" that your proposal is accepted :-)
>>
>> But seriously…
>>
>> If you haven't already, I'd suggest reading Django's contribution docs, 
>> and getting your development environment set up. Make sure you can run 
>> Django's test suite. If you're particularly enthused, try your hand at 
>> working on a patch for an open ticket so you can get used to the toolchain 
>> and the review process. That way you'll be up to speed with Django's 
>> development process, so when the GSoC period starts, you'll be comfortable 
>> with our basic process, so you'll just have to deal with the actual GSoC 
>> work.
>>
>> Yours,
>> Russ Magee %-)
>>
>> On Wed, Mar 26, 2014 at 7:22 PM, Daniel Pyrathon wrote:
>>
>>> Hi all,
>>>
>>> It's been a while since I submitted my GSOC proposal. Although I am 
>>> currently under exams, is there anything you would recommend me to do at 
>>> this point (other than hope that my proposal is successful)?
>>>
>>> Thanks,
>>> Daniel Pyrathon
>>>
>>>
>>> On Thursday, March 20, 2014 6:05:40 AM UTC, Russell Keith-Magee wrote:
>>>
>>>>
>>>> For the benefit of those reading along at home; I gave some revised 
>>>> notes and had a conversation with Daniel on IRC a couple of hours ago.
>>>>
>>>> Yours,
>>>> Russ Magee %-)
>>>>
>>>>
>>>> On Thu, Mar 20, 2014 at 4:10 AM, Daniel Pyrathon wrote:
>>>>
>>>>> Hi!
>>>>>
>>>>> Thanks for all the comments yesterday. They really helped me make the 
>>>>> proposal stronger.
>>>>> I have changed to proposal to reflect the changes. Would anyone like 
>>>>> to have a look and, possibly, comment more?
>>>>>
>>>>> https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFB
>>>>> kCXgy0Jwo/edit#heading=h.wyxx4xxijubt
>>>>>
>>>>> Thanks,
>>>>> Dan
>>>>>
>>>>> On Wednesday, March 19, 2014 7:32:10 AM UTC, Aymeric Augustin wrote:
>>>>>
>>>>>> On 19 mars 2014, at 06:27, Zach Borboa  wrote: 
>>>>>>
>>>>>> > Curious, how do you get REPL shell access to the server with 
>>>>>> DEBUG=True with a vanilla Django deployment? 
>>>>>>
>>>>>> That part of the discussion was about adding the werkzeug interactive 
>>>>>> debugger to Django’s default error page. 
>>>>>>
>>>>>> -- 
>>>>>> Aymeric. 
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>  -- 
>>>>> You received this message because you are subscribed to the Google 
>>>>> Groups "Django developers" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send 
>>>>> an email to django-develop...@googlegroups.com.
>>>>> To post to this group, send email to django-d...@googlegroups.com.
>>>>>
>>>>> Visit this group at http://groups.google.com/group/django-developers.
>>>>> To view this discussion on the web visit https://groups.google.com/d/
>>>>> msgid/django-developers/16264cb7-3f44-434d-865a-
>>>>> 8c33baa4d921%40googlegroups.com<https://groups.google.com/d/msgid/django-developers/16264cb7-3f44-434d-865a-8c33baa4d921%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>> .
>>>>>
>>>>> For more options, visit https://groups.google.com/d/optout.

[GSOC] Hi from Daniel!

2014-04-23 Thread Daniel Pyrathon
Hi,

My name is Daniel, and this summer I will be working on formalising Meta.
This will allow developers to build custom implementations of Meta and 
deploy them into their Django project. In case you missed my proposal, you 
can have a look at it 
here: 
https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit#heading=h.wyxx4xxijubt

I'm really looking forward to getting started: I am already in contact with 
my mentor and I was hoping to get to know the community a bit more too :)
Also, if you have any suggestions please let me know. I am quite new and I 
have only contributed a few patches.

Regards,
Daniel Pyrathon

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/44198942-c5cf-4205-9eb1-395ea8c16fa4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [GSOC] Hi from Daniel!

2014-05-04 Thread Daniel Pyrathon
Hi Ramiro,

Sorry for the late reply! I am currently stormed in university exams and 
this made me forget to get back to you. Luckily this will finish soon.

I am really happy you are looking forward to the project. It's great to 
know that you will be participating in the discussions. Please give me as 
much feedback as possible, especially in early stages. I want to make sure 
I am going in the right direction.
I will be posting an update every Friday on my work, this will be useful 
for you and anyone else to understand my progress and give me suggestions.

For anything else, just give me a shout

Regards,
Daniel Pyrathon

On Thursday, April 24, 2014 1:29:03 AM UTC+1, Ramiro Morales wrote:
>
> Hi Daniel, 
>
> On Wed, Apr 23, 2014 at 4:23 PM, Daniel Pyrathon 
> > 
> wrote: 
> > Hi, 
> > 
> > My name is Daniel, and this summer I will be working on formalising 
> Meta. 
> > This will allow developers to build custom implementations of Meta and 
> > deploy them into their Django project. In case you missed my proposal, 
> you 
> > can have a look at it here: 
> > 
> https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit#heading=h.wyxx4xxijubt
>  
> > 
> > I'm really looking forward to getting started: I am already in contact 
> with 
> > my mentor and I was hoping to get to know the community a bit more too 
> :) 
>
> I hope you enjoy your summer working on this project. 
>
> I have some ideas regarding i18n-related (de)coupling of Meta 
> atributes like verbose_name{,_plural} to/from such a core part of 
> Django. I'll be participating in the discussion and following your 
> work. 
>
> Welcome and good luck! 
>
> Regards, 
>
> -- 
> Ramiro Morales 
> @ramiromorales 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/62d88ea0-dea5-4602-b123-e2aadc40c48b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[GSOC] Weekly update

2014-05-19 Thread Daniel Pyrathon
Hi All,

Today I will be starting my weekly updates on my SoC project: refactoring 
Meta to a stable API. For anyone who missed out, you will be able to view 
it 
here: 
https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit

This week is the first official week of SoC. Me and my mentor (Russell) are 
initially approaching the work in the following way:

   - *Document the existing Meta API*
   For each endpoint, document the following:
 - Input parameters and return type
 - Caching pattern used
 - Where it's called from (internally and externally to Meta)
 - Why is it being called
 - When is it being called
   
   - *Propose an initial refactor plan*
   Once the documentation has been done, I should have a better idea of the 
   current implementation. This will allow me to mock a proposed 
   implementation that will be reviewed at my next update call, on Monday.

My next update will be posted on Friday, just to make sure the community is 
informed of my progress. For any major updates that require community 
approval, I will be creating separate threads.
My name on the internet is pirosb3, so if you want to have a chat about my 
progress feel free to contact me! The branch I am currently working on 
is https://github.com/PirosB3/django/tree/meta_documentation

Regards,
Daniel Pyrathon

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c0d9a1f7-4fae-4c47-aae1-59b472c46939%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [GSOC] Weekly update

2014-05-23 Thread Daniel Pyrathon
Hi all,

In the last days I have built a documentation of the current state of 
Options. Based on feedback and prototyping I have thought of a potential 
interface for _meta that can solve the issues currently present, such as 
redundancy (in code and in caching systems). The interface has also been 
thought to be maintainable and is a base that can be used to create custom 
meta stores.
Obviously this is far from perfect, It will need many iterations and maybe 
it is too complex. I would really love to gain as much feedback as possible 
so it can be discussed with Russell and the community on Monday.

The documentation of _meta can be found 
here: 
https://github.com/PirosB3/django/blob/meta_documentation/docs/ref/models/meta.txt
I will be refining the document in the next days, I will also be publishing 
the docs on a webserver and will be linking a URL soon.

My proposal has been published here:
https://gist.github.com/PirosB3/371704ed40ed093d5a82
In the next days I will be iterating over the feedback gained, and based on 
one very interesting suggestion on IRC, I will try to see how my API syntax 
looks in modelforms.py.

As said previously, and feedback is greatly appreciated.

Hi from Pycon IT!

Daniel Pyrathon

On Tuesday, May 20, 2014 3:25:45 PM UTC+2, Josh Smeaton wrote:
>
> Best of luck!
>
> On Tuesday, 20 May 2014 03:56:06 UTC+10, Daniel Pyrathon wrote:
>>
>> Hi All,
>>
>> Today I will be starting my weekly updates on my SoC project: refactoring 
>> Meta to a stable API. For anyone who missed out, you will be able to view 
>> it here: 
>> https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit
>>
>> This week is the first official week of SoC. Me and my mentor (Russell) 
>> are initially approaching the work in the following way:
>>
>>- *Document the existing Meta API*
>>For each endpoint, document the following:
>>  - Input parameters and return type
>>  - Caching pattern used
>>  - Where it's called from (internally and externally to Meta)
>>  - Why is it being called
>>  - When is it being called
>>
>>- *Propose an initial refactor plan*
>>Once the documentation has been done, I should have a better idea of 
>>the current implementation. This will allow me to mock a proposed 
>>implementation that will be reviewed at my next update call, on Monday.
>>
>> My next update will be posted on Friday, just to make sure the community 
>> is informed of my progress. For any major updates that require community 
>> approval, I will be creating separate threads.
>> My name on the internet is pirosb3, so if you want to have a chat about 
>> my progress feel free to contact me! The branch I am currently working on 
>> is https://github.com/PirosB3/django/tree/meta_documentation
>>
>> Regards,
>> Daniel Pyrathon
>>
>
On Tuesday, May 20, 2014 3:25:45 PM UTC+2, Josh Smeaton wrote:
>
> Best of luck!
>
> On Tuesday, 20 May 2014 03:56:06 UTC+10, Daniel Pyrathon wrote:
>>
>> Hi All,
>>
>> Today I will be starting my weekly updates on my SoC project: refactoring 
>> Meta to a stable API. For anyone who missed out, you will be able to view 
>> it here: 
>> https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit
>>
>> This week is the first official week of SoC. Me and my mentor (Russell) 
>> are initially approaching the work in the following way:
>>
>>- *Document the existing Meta API*
>>For each endpoint, document the following:
>>  - Input parameters and return type
>>  - Caching pattern used
>>  - Where it's called from (internally and externally to Meta)
>>  - Why is it being called
>>  - When is it being called
>>
>>- *Propose an initial refactor plan*
>>Once the documentation has been done, I should have a better idea of 
>>the current implementation. This will allow me to mock a proposed 
>>implementation that will be reviewed at my next update call, on Monday.
>>
>> My next update will be posted on Friday, just to make sure the community 
>> is informed of my progress. For any major updates that require community 
>> approval, I will be creating separate threads.
>> My name on the internet is pirosb3, so if you want to have a chat about 
>> my progress feel free to contact me! The branch I am currently working on 
>> is https://github.com/PirosB3/django/tree/meta_documentation
>>
>> Regards,
>> Daniel Pyrathon
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django 

Re: [GSOC] Weekly update

2014-05-25 Thread Daniel Pyrathon
Hi Chris,

Oh sorry about that! big typo over there. Modifying the gist.

Thanks,
Daniel Pyrathon

On Saturday, May 24, 2014 7:44:15 AM UTC+2, Chris Beaven wrote:
>
> Hi Daniel,
>
> The proposal looks interesting - I've only skimmed it so far but one 
> question: you mention User.get_model() several times -- do you mean 
> User.get_meta()?
>
> On Saturday, May 24, 2014 7:05:02 AM UTC+12, Daniel Pyrathon wrote:
>>
>> Hi all,
>>
>> In the last days I have built a documentation of the current state of 
>> Options. Based on feedback and prototyping I have thought of a potential 
>> interface for _meta that can solve the issues currently present, such as 
>> redundancy (in code and in caching systems). The interface has also been 
>> thought to be maintainable and is a base that can be used to create custom 
>> meta stores.
>> Obviously this is far from perfect, It will need many iterations and 
>> maybe it is too complex. I would really love to gain as much feedback as 
>> possible so it can be discussed with Russell and the community on Monday.
>>
>> The documentation of _meta can be found here: 
>> https://github.com/PirosB3/django/blob/meta_documentation/docs/ref/models/meta.txt
>> I will be refining the document in the next days, I will also be 
>> publishing the docs on a webserver and will be linking a URL soon.
>>
>> My proposal has been published here:
>> https://gist.github.com/PirosB3/371704ed40ed093d5a82
>> In the next days I will be iterating over the feedback gained, and based 
>> on one very interesting suggestion on IRC, I will try to see how my API 
>> syntax looks in modelforms.py.
>>
>> As said previously, and feedback is greatly appreciated.
>>
>> Hi from Pycon IT!
>>
>> Daniel Pyrathon
>>
>> On Tuesday, May 20, 2014 3:25:45 PM UTC+2, Josh Smeaton wrote:
>>>
>>> Best of luck!
>>>
>>> On Tuesday, 20 May 2014 03:56:06 UTC+10, Daniel Pyrathon wrote:
>>>>
>>>> Hi All,
>>>>
>>>> Today I will be starting my weekly updates on my SoC project: 
>>>> refactoring Meta to a stable API. For anyone who missed out, you will be 
>>>> able to view it here: 
>>>> https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit
>>>>
>>>> This week is the first official week of SoC. Me and my mentor (Russell) 
>>>> are initially approaching the work in the following way:
>>>>
>>>>- *Document the existing Meta API*
>>>>For each endpoint, document the following:
>>>>  - Input parameters and return type
>>>>  - Caching pattern used
>>>>  - Where it's called from (internally and externally to Meta)
>>>>  - Why is it being called
>>>>  - When is it being called
>>>>
>>>>- *Propose an initial refactor plan*
>>>>Once the documentation has been done, I should have a better idea 
>>>>of the current implementation. This will allow me to mock a proposed 
>>>>implementation that will be reviewed at my next update call, on Monday.
>>>>
>>>> My next update will be posted on Friday, just to make sure the 
>>>> community is informed of my progress. For any major updates that require 
>>>> community approval, I will be creating separate threads.
>>>> My name on the internet is pirosb3, so if you want to have a chat about 
>>>> my progress feel free to contact me! The branch I am currently working on 
>>>> is https://github.com/PirosB3/django/tree/meta_documentation
>>>>
>>>> Regards,
>>>> Daniel Pyrathon
>>>>
>>>
>> On Tuesday, May 20, 2014 3:25:45 PM UTC+2, Josh Smeaton wrote:
>>>
>>> Best of luck!
>>>
>>> On Tuesday, 20 May 2014 03:56:06 UTC+10, Daniel Pyrathon wrote:
>>>>
>>>> Hi All,
>>>>
>>>> Today I will be starting my weekly updates on my SoC project: 
>>>> refactoring Meta to a stable API. For anyone who missed out, you will be 
>>>> able to view it here: 
>>>> https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit
>>>>
>>>> This week is the first official week of SoC. Me and my mentor (Russell) 
>>>> are initially approaching the work in the following way:
>>>>
>>>>- *Document the existing Meta API*
>>>>For each endpoint, document the following:
>>

Re: [GSOC] Weekly update

2014-05-25 Thread Daniel Pyrathon
Hi Josh,

The meta API specified in the docs 
(https://github.com/PirosB3/django/blob/meta_documentation/docs/ref/models/meta.txt)
 
is the current API. I have documented this in order to understand more of 
the current implementation and it will be good to show a comparison when a 
new meta API will be approved.

My current proposal (https://gist.github.com/PirosB3/371704ed40ed093d5a82) 
and it will be discussed tomorrow with Russell. I will post as soon as I 
have updates.

Daniel Pyrathon 

On Saturday, May 24, 2014 10:37:49 AM UTC+2, Josh Smeaton wrote:
>
> Hi Daniel,
>
> Nice work putting that document together. Is the meta document you put 
> together the current API or is it the API you are proposing? If the latter, 
> a few suggestions (and if others disagree, please point that out):
>
> - Remove all mention of caching. That should be an implementation detail 
> only, and not a requirement for other implementations.
> - the *_with_model methods really rub me up the wrong way. I would prefer 
> always returning the _with_model variant, and letting the caller discard 
> the model if they don't need it.
> - I'm not a fan of virtual and concrete fields, though I have to admit I'm 
> not sure how they're different, especially in the context of different 
> implementations.
> - Not sure that m2m should be differentiated from related.
> - init_name_map should be an implementation detail.
> - normalize_together should be an implementation detail.
>
> Regards,
>
> Josh
>
> On Saturday, 24 May 2014 05:05:02 UTC+10, Daniel Pyrathon wrote:
>>
>> Hi all,
>>
>> In the last days I have built a documentation of the current state of 
>> Options. Based on feedback and prototyping I have thought of a potential 
>> interface for _meta that can solve the issues currently present, such as 
>> redundancy (in code and in caching systems). The interface has also been 
>> thought to be maintainable and is a base that can be used to create custom 
>> meta stores.
>> Obviously this is far from perfect, It will need many iterations and 
>> maybe it is too complex. I would really love to gain as much feedback as 
>> possible so it can be discussed with Russell and the community on Monday.
>>
>> The documentation of _meta can be found here: 
>> https://github.com/PirosB3/django/blob/meta_documentation/docs/ref/models/meta.txt
>> I will be refining the document in the next days, I will also be 
>> publishing the docs on a webserver and will be linking a URL soon.
>>
>> My proposal has been published here:
>> https://gist.github.com/PirosB3/371704ed40ed093d5a82
>> In the next days I will be iterating over the feedback gained, and based 
>> on one very interesting suggestion on IRC, I will try to see how my API 
>> syntax looks in modelforms.py.
>>
>> As said previously, and feedback is greatly appreciated.
>>
>> Hi from Pycon IT!
>>
>> Daniel Pyrathon
>>
>> On Tuesday, May 20, 2014 3:25:45 PM UTC+2, Josh Smeaton wrote:
>>>
>>> Best of luck!
>>>
>>> On Tuesday, 20 May 2014 03:56:06 UTC+10, Daniel Pyrathon wrote:
>>>>
>>>> Hi All,
>>>>
>>>> Today I will be starting my weekly updates on my SoC project: 
>>>> refactoring Meta to a stable API. For anyone who missed out, you will be 
>>>> able to view it here: 
>>>> https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit
>>>>
>>>> This week is the first official week of SoC. Me and my mentor (Russell) 
>>>> are initially approaching the work in the following way:
>>>>
>>>>- *Document the existing Meta API*
>>>>For each endpoint, document the following:
>>>>  - Input parameters and return type
>>>>  - Caching pattern used
>>>>  - Where it's called from (internally and externally to Meta)
>>>>  - Why is it being called
>>>>  - When is it being called
>>>>
>>>>- *Propose an initial refactor plan*
>>>>Once the documentation has been done, I should have a better idea 
>>>>of the current implementation. This will allow me to mock a proposed 
>>>>implementation that will be reviewed at my next update call, on Monday.
>>>>
>>>> My next update will be posted on Friday, just to make sure the 
>>>> community is informed of my progress. For any major updates that require 
>>>> community approval, I will be creating separate threads.
>>>> My name on

Re: [GSOC] Weekly update

2014-05-25 Thread Daniel Pyrathon
Hi All,

Just to make you know, I have put up the current _meta API documentation 
here:
http://162.219.6.191:8000/ref/models/meta.html?highlight=_meta
As always, feel free to ask questions.

Daniel

On Monday, May 26, 2014 1:26:27 AM UTC+2, Daniel Pyrathon wrote:
>
> Hi Josh,
>
> The meta API specified in the docs (
> https://github.com/PirosB3/django/blob/meta_documentation/docs/ref/models/meta.txt)
>  
> is the current API. I have documented this in order to understand more of 
> the current implementation and it will be good to show a comparison when a 
> new meta API will be approved.
>
> My current proposal (https://gist.github.com/PirosB3/371704ed40ed093d5a82) 
> and it will be discussed tomorrow with Russell. I will post as soon as I 
> have updates.
>
> Daniel Pyrathon 
>
> On Saturday, May 24, 2014 10:37:49 AM UTC+2, Josh Smeaton wrote:
>>
>> Hi Daniel,
>>
>> Nice work putting that document together. Is the meta document you put 
>> together the current API or is it the API you are proposing? If the latter, 
>> a few suggestions (and if others disagree, please point that out):
>>
>> - Remove all mention of caching. That should be an implementation detail 
>> only, and not a requirement for other implementations.
>> - the *_with_model methods really rub me up the wrong way. I would prefer 
>> always returning the _with_model variant, and letting the caller discard 
>> the model if they don't need it.
>> - I'm not a fan of virtual and concrete fields, though I have to admit 
>> I'm not sure how they're different, especially in the context of different 
>> implementations.
>> - Not sure that m2m should be differentiated from related.
>> - init_name_map should be an implementation detail.
>> - normalize_together should be an implementation detail.
>>
>> Regards,
>>
>> Josh
>>
>> On Saturday, 24 May 2014 05:05:02 UTC+10, Daniel Pyrathon wrote:
>>>
>>> Hi all,
>>>
>>> In the last days I have built a documentation of the current state of 
>>> Options. Based on feedback and prototyping I have thought of a potential 
>>> interface for _meta that can solve the issues currently present, such as 
>>> redundancy (in code and in caching systems). The interface has also been 
>>> thought to be maintainable and is a base that can be used to create custom 
>>> meta stores.
>>> Obviously this is far from perfect, It will need many iterations and 
>>> maybe it is too complex. I would really love to gain as much feedback as 
>>> possible so it can be discussed with Russell and the community on Monday.
>>>
>>> The documentation of _meta can be found here: 
>>> https://github.com/PirosB3/django/blob/meta_documentation/docs/ref/models/meta.txt
>>> I will be refining the document in the next days, I will also be 
>>> publishing the docs on a webserver and will be linking a URL soon.
>>>
>>> My proposal has been published here:
>>> https://gist.github.com/PirosB3/371704ed40ed093d5a82
>>> In the next days I will be iterating over the feedback gained, and based 
>>> on one very interesting suggestion on IRC, I will try to see how my API 
>>> syntax looks in modelforms.py.
>>>
>>> As said previously, and feedback is greatly appreciated.
>>>
>>> Hi from Pycon IT!
>>>
>>> Daniel Pyrathon
>>>
>>> On Tuesday, May 20, 2014 3:25:45 PM UTC+2, Josh Smeaton wrote:
>>>>
>>>> Best of luck!
>>>>
>>>> On Tuesday, 20 May 2014 03:56:06 UTC+10, Daniel Pyrathon wrote:
>>>>>
>>>>> Hi All,
>>>>>
>>>>> Today I will be starting my weekly updates on my SoC project: 
>>>>> refactoring Meta to a stable API. For anyone who missed out, you will be 
>>>>> able to view it here: 
>>>>> https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit
>>>>>
>>>>> This week is the first official week of SoC. Me and my mentor 
>>>>> (Russell) are initially approaching the work in the following way:
>>>>>
>>>>>- *Document the existing Meta API*
>>>>>For each endpoint, document the following:
>>>>>  - Input parameters and return type
>>>>>  - Caching pattern used
>>>>>  - Where it's called from (internally and externally to Meta)
>>>>>  - Why is it being called
>>>>>  - When is it being called
>>>>

Re: [GSOC] Weekly update

2014-06-01 Thread Daniel Pyrathon
Hi All,

An update on my side, some interesting work has happened this week: me and 
Russell have decided to start on the implementation early in order to 
understand better the internals of Options. Currently I am working on the 
following:

*Providing one single function: get_fields(types, opts, **kwargs)*
Previously we had identified a number of functions that contained similar 
data but had a different return type. We are going to provide one function 
that takes a set of field types + options and returns the same output 
everywhere: ((field_name, field), ...). This has the benefit of simplicity 
and is more maintainable than the previous approach.

TYPES = DATA, M2M, FK, RELATED_OBJECT, RELATED_M2M
OPTS = NONE, LOCAL_ONLY, CONCRETE, INCLUDE_HIDDEN, INCLUDE_PROXY, VIRTUAL

*Providing two functions for retrieving details of a specific field*
As specified in my previous document, in many parts of the code we just 
want to retrieve a field object by name, other times we have a field but 
need other metadata such as: owner (model_class), direct (bool), m2m 
(bool). We will provide two functions:

get_field(field_name) -> field_instance

get_field_details(field_instance) -> direct, m2m, (still to be defined)

While we still have not entirely defined what *get_field_details *will 
return, this will be done soon.


*Building a test suite for the existing API*
The new API development will be driven by a test suite that will compare 
the current (legacy) API with the new implementation. While return types 
will be different, we are asserting that all the correct fields and 
metadata are returned. Building a test suite means we can start 
implementing before a final API spec is finalised. It also means we can 
iterate faster and, from my perspective, I also understand a lot more of 
the current implementation. We are testing each combination of fields and 
options together.

My current implementation is visible 
here: https://github.com/PirosB3/django/compare/soc2014_meta_refactor

For any questions or suggestions, let me know.

Daniel Pyrathon


On Monday, May 26, 2014 1:28:31 AM UTC+2, Daniel Pyrathon wrote:
>
> Hi All,
>
> Just to make you know, I have put up the current _meta API documentation 
> here:
> http://162.219.6.191:8000/ref/models/meta.html?highlight=_meta
> As always, feel free to ask questions.
>
> Daniel
>
> On Monday, May 26, 2014 1:26:27 AM UTC+2, Daniel Pyrathon wrote:
>>
>> Hi Josh,
>>
>> The meta API specified in the docs (
>> https://github.com/PirosB3/django/blob/meta_documentation/docs/ref/models/meta.txt)
>>  
>> is the current API. I have documented this in order to understand more of 
>> the current implementation and it will be good to show a comparison when a 
>> new meta API will be approved.
>>
>> My current proposal (https://gist.github.com/PirosB3/371704ed40ed093d5a82) 
>> and it will be discussed tomorrow with Russell. I will post as soon as I 
>> have updates.
>>
>> Daniel Pyrathon 
>>
>> On Saturday, May 24, 2014 10:37:49 AM UTC+2, Josh Smeaton wrote:
>>>
>>> Hi Daniel,
>>>
>>> Nice work putting that document together. Is the meta document you put 
>>> together the current API or is it the API you are proposing? If the latter, 
>>> a few suggestions (and if others disagree, please point that out):
>>>
>>> - Remove all mention of caching. That should be an implementation detail 
>>> only, and not a requirement for other implementations.
>>> - the *_with_model methods really rub me up the wrong way. I would 
>>> prefer always returning the _with_model variant, and letting the caller 
>>> discard the model if they don't need it.
>>> - I'm not a fan of virtual and concrete fields, though I have to admit 
>>> I'm not sure how they're different, especially in the context of different 
>>> implementations.
>>> - Not sure that m2m should be differentiated from related.
>>> - init_name_map should be an implementation detail.
>>> - normalize_together should be an implementation detail.
>>>
>>> Regards,
>>>
>>> Josh
>>>
>>> On Saturday, 24 May 2014 05:05:02 UTC+10, Daniel Pyrathon wrote:
>>>>
>>>> Hi all,
>>>>
>>>> In the last days I have built a documentation of the current state of 
>>>> Options. Based on feedback and prototyping I have thought of a potential 
>>>> interface for _meta that can solve the issues currently present, such as 
>>>> redundancy (in code and in caching systems). The interface has also been 
>>>> thought to be maintainable and is a base that can be used to create custom 
>>>> meta s

Re: [GSOC] Weekly update

2014-06-06 Thread Daniel Pyrathon
Hi All,

Based on the work done last week, this week I have worked on the following:

*1) Covered the current _meta implementation of unittests*
The current Options is not covered by unit tests at all, I have created the 
model_options test module containing one or more unit tests for each 
endpoint and usage. Each endpoint is tested with many different models and 
fields (data, m2m, related objects, related m2m, virtual, ..). Each unit 
test asserts equality in field naming and field type. Endpoints that return 
the model are also tested for equality.

This branch is found here: 
https://github.com/PirosB3/django/tree/soc2014_meta_unittests

*2) Pulled in tests from soc2014_meta_unittests and tested the new 
implementation*
The previous branch that I developed on contains the new API 
implementation, I have pulled in the tests from soc2014_meta_unittests and 
I have switched the old API calls to the new API calls (with a few 
adjustments). I have successfully made all tests pass even though I have 
found some design issues that need to be addressed in my next update call.

This branch is found here: 
https://github.com/PirosB3/django/tree/soc2014_meta_refactor

*3) Created a new branch that maps the old implementation with the new*
Today I started putting my new API in "production". This is obviously 
nowhere near a finalised version but it helps me spot some edge cases that 
are not in the unit-tests. Each issue found has been converted into a 
standalone unit-test and has been proved to fail.
Unfortunately, this made me realise of other design issues related to the 
new return type of get_fields -> (field_name, field), A decision will be 
taken on monday.

This branch is found here: 
https://github.com/PirosB3/django/tree/soc2014_meta_refactor_implementation 
as is for personal use only

For any questions, let me know!
Dan
 

On Sunday, June 1, 2014 6:10:53 PM UTC+2, Daniel Pyrathon wrote:
>
> Hi All,
>
> An update on my side, some interesting work has happened this week: me and 
> Russell have decided to start on the implementation early in order to 
> understand better the internals of Options. Currently I am working on the 
> following:
>
> *Providing one single function: get_fields(types, opts, **kwargs)*
> Previously we had identified a number of functions that contained similar 
> data but had a different return type. We are going to provide one function 
> that takes a set of field types + options and returns the same output 
> everywhere: ((field_name, field), ...). This has the benefit of simplicity 
> and is more maintainable than the previous approach.
>
> TYPES = DATA, M2M, FK, RELATED_OBJECT, RELATED_M2M
> OPTS = NONE, LOCAL_ONLY, CONCRETE, INCLUDE_HIDDEN, INCLUDE_PROXY, VIRTUAL
>
> *Providing two functions for retrieving details of a specific field*
> As specified in my previous document, in many parts of the code we just 
> want to retrieve a field object by name, other times we have a field but 
> need other metadata such as: owner (model_class), direct (bool), m2m 
> (bool). We will provide two functions:
>
> get_field(field_name) -> field_instance
>
> get_field_details(field_instance) -> direct, m2m, (still to be defined)
>
> While we still have not entirely defined what *get_field_details *will 
> return, this will be done soon.
>
>
> *Building a test suite for the existing API*
> The new API development will be driven by a test suite that will compare 
> the current (legacy) API with the new implementation. While return types 
> will be different, we are asserting that all the correct fields and 
> metadata are returned. Building a test suite means we can start 
> implementing before a final API spec is finalised. It also means we can 
> iterate faster and, from my perspective, I also understand a lot more of 
> the current implementation. We are testing each combination of fields and 
> options together.
>
> My current implementation is visible here: 
> https://github.com/PirosB3/django/compare/soc2014_meta_refactor
>
> For any questions or suggestions, let me know.
>
> Daniel Pyrathon
>
>
> On Monday, May 26, 2014 1:28:31 AM UTC+2, Daniel Pyrathon wrote:
>>
>> Hi All,
>>
>> Just to make you know, I have put up the current _meta API documentation 
>> here:
>> http://162.219.6.191:8000/ref/models/meta.html?highlight=_meta
>> As always, feel free to ask questions.
>>
>> Daniel
>>
>> On Monday, May 26, 2014 1:26:27 AM UTC+2, Daniel Pyrathon wrote:
>>>
>>> Hi Josh,
>>>
>>> The meta API specified in the docs (
>>> https://github.com/PirosB3/django/blob/meta_documentation/docs/ref/models/meta.txt)
>>>  
>>> is the current API. I have documented this in order to understand more of 
>>

Re: [GSOC] Weekly update

2014-06-13 Thread Daniel Pyrathon
Hi All,

This week's work was a follow-up of last week's work, with some new 
discoveries with regards to the new API:

*1) Improved the current _meta implementation of unittests*
I refactored the current meta unittest branch into a more compact version. 
I also added test cases for Generic Foreign Keys, RelatedField and more 
improvements on virtual fields. This section will actually be our first 
merge to master: a unit test suite for the current implementation.

This branch is found here: 
https://github.com/PirosB3/django/tree/soc2014_meta_unittests

*2) Refactored the new _meta API spec*
By implementing the new API, I found new redundancies in the current 
implementation that we can avoid in the new API spec:

*1) Only return field instances*
the current implementation has a common return pattern: (field_object, 
model, direct, m2m). After a few tests I realized that model, direct, m2m 
and field_name are all redundant and can be derived from only field_object. 
Since there are only 3-4 places that actually use direct and m2m it makes 
sense to remove this function from the new API spec.
Here I show how all the information can be derived form field_instance 
(https://gist.github.com/PirosB3/6cb4badbb1b8c2e41a96/revisions), I ran the 
entire test suite and things look good. The only issue I can see now is 
from a performance point of view.

*2) Provide only 2 API endpoints*

 - get_fields(types, opts) -> (field_instance, field_instance, ...)
 - get_field(field_name) -> field_instance

The rest is all (I might be very wrong! don't trust me) redundant. By 
continuing with this iterative approach, I will soon find out if I am 
correct or not ;)


This branch is found here: 
https://github.com/PirosB3/django/tree/soc2014_meta_refactor

For any questions, as usual, let me know
Dan


On Friday, June 6, 2014 7:03:36 PM UTC+2, Daniel Pyrathon wrote:
>
> Hi All,
>
> Based on the work done last week, this week I have worked on the following:
>
> *1) Covered the current _meta implementation of unittests*
> The current Options is not covered by unit tests at all, I have created 
> the model_options test module containing one or more unit tests for each 
> endpoint and usage. Each endpoint is tested with many different models and 
> fields (data, m2m, related objects, related m2m, virtual, ..). Each unit 
> test asserts equality in field naming and field type. Endpoints that return 
> the model are also tested for equality.
>
> This branch is found here: 
> https://github.com/PirosB3/django/tree/soc2014_meta_unittests
>
> *2) Pulled in tests from soc2014_meta_unittests and tested the new 
> implementation*
> The previous branch that I developed on contains the new API 
> implementation, I have pulled in the tests from soc2014_meta_unittests and 
> I have switched the old API calls to the new API calls (with a few 
> adjustments). I have successfully made all tests pass even though I have 
> found some design issues that need to be addressed in my next update call.
>
> This branch is found here: 
> https://github.com/PirosB3/django/tree/soc2014_meta_refactor
>
> *3) Created a new branch that maps the old implementation with the new*
> Today I started putting my new API in "production". This is obviously 
> nowhere near a finalised version but it helps me spot some edge cases that 
> are not in the unit-tests. Each issue found has been converted into a 
> standalone unit-test and has been proved to fail.
> Unfortunately, this made me realise of other design issues related to the 
> new return type of get_fields -> (field_name, field), A decision will be 
> taken on monday.
>
> This branch is found here: 
> https://github.com/PirosB3/django/tree/soc2014_meta_refactor_implementation 
> as is for personal use only
>
> For any questions, let me know!
> Dan
>  
>
> On Sunday, June 1, 2014 6:10:53 PM UTC+2, Daniel Pyrathon wrote:
>>
>> Hi All,
>>
>> An update on my side, some interesting work has happened this week: me 
>> and Russell have decided to start on the implementation early in order to 
>> understand better the internals of Options. Currently I am working on the 
>> following:
>>
>> *Providing one single function: get_fields(types, opts, **kwargs)*
>> Previously we had identified a number of functions that contained similar 
>> data but had a different return type. We are going to provide one function 
>> that takes a set of field types + options and returns the same output 
>> everywhere: ((field_name, field), ...). This has the benefit of simplicity 
>> and is more maintainable than the previous approach.
>>
>> TYPES = DATA, M2M, FK, RELATED_OBJECT, RELATED_M2M
>> OPTS = NONE, LOCAL_ONLY, CONCRETE, INCLUDE_HIDDEN, INCLUDE_PROXY, VIRTUAL
>>

[GSOC] model/options.py test suite released

2014-06-17 Thread Daniel Pyrathon
Hi All,

I have opened a Pull Request for a unit test suite for the current 
implementation of model/options.
The PR can be found 
here: https://github.com/PirosB3/django/tree/soc2014_meta_unittests and is 
part of a 1-3 delivery for my SoC project. This will be a good moment to 
collect feedback and advice, so if you would like to say something, just 
write it here or on the PR.

Regards,
Daniel Pyrathon


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0fb936e5-d9a5-4150-a58e-09bbebf30835%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [GSOC] Weekly update

2014-06-21 Thread Daniel Pyrathon
Hi All,

This week I have done the following

*- Openend PR and merged Options (_meta) unittests*
https://github.com/django/django/pull/2823
This is a working test suite of model/options.py file. Is is currently 
testing the current Options implementation.

*- Re-implemented test-suite in the new API*
My new proposed API has been implemented on top of the Options unittests. 
This means new and old API give the same results! and therefore the new API 
is a working re-implementation.

 *- Performance optimizations*
Currently, the biggest issue I have had (in the new API) regards 
performance. I am doing small optimisations and benchmarking with cProfile.
If anyone has some good hints on how to benchmark single functions in 
Django please let me know!

Regards,
Dan


On Friday, June 13, 2014 10:11:41 PM UTC+2, Aymeric Augustin wrote:
>
> I like this simplification a lot. I hope you can make it work.
>
> Just check that you don’t "overload" fields, by storing in field objects 
> information that doesn’t belong there.
>
> -- 
> Aymeric.
>
>
>  
> On 13 juin 2014, at 19:53, Daniel Pyrathon > 
> wrote:
>
> Hi All,
>
> This week's work was a follow-up of last week's work, with some new 
> discoveries with regards to the new API:
>
> *1) Improved the current _meta implementation of unittests*
> I refactored the current meta unittest branch into a more compact version. 
> I also added test cases for Generic Foreign Keys, RelatedField and more 
> improvements on virtual fields. This section will actually be our first 
> merge to master: a unit test suite for the current implementation.
>
> This branch is found here: 
> https://github.com/PirosB3/django/tree/soc2014_meta_unittests
>
> *2) Refactored the new _meta API spec*
> By implementing the new API, I found new redundancies in the current 
> implementation that we can avoid in the new API spec:
>
> *1) Only return field instances*
> the current implementation has a common return pattern: (field_object, 
> model, direct, m2m). After a few tests I realized that model, direct, m2m 
> and field_name are all redundant and can be derived from only field_object. 
> Since there are only 3-4 places that actually use direct and m2m it makes 
> sense to remove this function from the new API spec.
> Here I show how all the information can be derived form field_instance (
> https://gist.github.com/PirosB3/6cb4badbb1b8c2e41a96/revisions), I ran 
> the entire test suite and things look good. The only issue I can see now is 
> from a performance point of view.
>
> *2) Provide only 2 API endpoints*
>
>  - get_fields(types, opts) -> (field_instance, field_instance, ...)
>  - get_field(field_name) -> field_instance
>
> The rest is all (I might be very wrong! don't trust me) redundant. By 
> continuing with this iterative approach, I will soon find out if I am 
> correct or not ;)
>
>
> This branch is found here: 
> https://github.com/PirosB3/django/tree/soc2014_meta_refactor
>
> For any questions, as usual, let me know
> Dan
>
>
> On Friday, June 6, 2014 7:03:36 PM UTC+2, Daniel Pyrathon wrote:
>>
>> Hi All,
>>
>> Based on the work done last week, this week I have worked on the 
>> following:
>>
>> *1) Covered the current _meta implementation of unittests*
>> The current Options is not covered by unit tests at all, I have created 
>> the model_options test module containing one or more unit tests for each 
>> endpoint and usage. Each endpoint is tested with many different models and 
>> fields (data, m2m, related objects, related m2m, virtual, ..). Each unit 
>> test asserts equality in field naming and field type. Endpoints that return 
>> the model are also tested for equality.
>>
>> This branch is found here: 
>> https://github.com/PirosB3/django/tree/soc2014_meta_unittests
>>
>> *2) Pulled in tests from soc2014_meta_unittests and tested the new 
>> implementation*
>> The previous branch that I developed on contains the new API 
>> implementation, I have pulled in the tests from soc2014_meta_unittests and 
>> I have switched the old API calls to the new API calls (with a few 
>> adjustments). I have successfully made all tests pass even though I have 
>> found some design issues that need to be addressed in my next update call.
>>
>> This branch is found here: 
>> https://github.com/PirosB3/django/tree/soc2014_meta_refactor
>>
>> *3) Created a new branch that maps the old implementation with the new*
>> Today I started putting my new API in "production". This is obviously 
>> nowhere near a finalised version but it helps me spot some edge cases that 
>> are no

Re: [GSOC] Weekly update

2014-06-22 Thread Daniel Pyrathon
Sure!

https://github.com/PirosB3/django/compare/soc2014_meta_refactor_performance?expand=1

This contains the following:
 - Recently merged unit test suite for model/options
 - New API implementation: get_new_fields, get_new_field.
 - Implementation of recently merged unit test suite with new API
 - Small optimizations (BIG work in progress)

If you have any suggestions please let me know!

Dan

On Sunday, June 22, 2014 2:14:56 AM UTC+2, Curtis Maloney wrote:
>
> Is there somewhere I can look at your work in progress code?
>
>
> On 21 June 2014 19:57, Daniel Pyrathon > 
> wrote:
>
>> Hi All,
>>
>> This week I have done the following
>>
>> *- Openend PR and merged Options (_meta) unittests*
>> https://github.com/django/django/pull/2823
>> This is a working test suite of model/options.py file. Is is currently 
>> testing the current Options implementation.
>>
>> *- Re-implemented test-suite in the new API*
>> My new proposed API has been implemented on top of the Options unittests. 
>> This means new and old API give the same results! and therefore the new API 
>> is a working re-implementation.
>>
>>  *- Performance optimizations*
>> Currently, the biggest issue I have had (in the new API) regards 
>> performance. I am doing small optimisations and benchmarking with cProfile.
>> If anyone has some good hints on how to benchmark single functions in 
>> Django please let me know!
>>
>> Regards,
>> Dan
>>
>>
>> On Friday, June 13, 2014 10:11:41 PM UTC+2, Aymeric Augustin wrote:
>>
>>> I like this simplification a lot. I hope you can make it work.
>>>
>>> Just check that you don’t "overload" fields, by storing in field objects 
>>> information that doesn’t belong there.
>>>
>>> -- 
>>> Aymeric.
>>>
>>>
>>>  
>>> On 13 juin 2014, at 19:53, Daniel Pyrathon  wrote:
>>>
>>> Hi All,
>>>
>>> This week's work was a follow-up of last week's work, with some new 
>>> discoveries with regards to the new API:
>>>
>>> *1) Improved the current _meta implementation of unittests*
>>> I refactored the current meta unittest branch into a more compact 
>>> version. I also added test cases for Generic Foreign Keys, RelatedField and 
>>> more improvements on virtual fields. This section will actually be our 
>>> first merge to master: a unit test suite for the current implementation.
>>>
>>> This branch is found here: https://github.com/
>>> PirosB3/django/tree/soc2014_meta_unittests
>>>
>>> *2) Refactored the new _meta API spec*
>>> By implementing the new API, I found new redundancies in the current 
>>> implementation that we can avoid in the new API spec:
>>>
>>> *1) Only return field instances*
>>> the current implementation has a common return pattern: (field_object, 
>>> model, direct, m2m). After a few tests I realized that model, direct, m2m 
>>> and field_name are all redundant and can be derived from only field_object. 
>>> Since there are only 3-4 places that actually use direct and m2m it makes 
>>> sense to remove this function from the new API spec.
>>> Here I show how all the information can be derived form field_instance (
>>> https://gist.github.com/PirosB3/6cb4badbb1b8c2e41a96/revisions), I ran 
>>> the entire test suite and things look good. The only issue I can see now is 
>>> from a performance point of view.
>>>
>>> *2) Provide only 2 API endpoints*
>>>
>>>  - get_fields(types, opts) -> (field_instance, field_instance, ...)
>>>  - get_field(field_name) -> field_instance
>>>
>>> The rest is all (I might be very wrong! don't trust me) redundant. By 
>>> continuing with this iterative approach, I will soon find out if I am 
>>> correct or not ;)
>>>
>>>
>>> This branch is found here: https://github.com/
>>> PirosB3/django/tree/soc2014_meta_refactor
>>>
>>> For any questions, as usual, let me know
>>> Dan
>>>
>>>
>>> On Friday, June 6, 2014 7:03:36 PM UTC+2, Daniel Pyrathon wrote:
>>>>
>>>> Hi All,
>>>>
>>>> Based on the work done last week, this week I have worked on the 
>>>> following:
>>>>
>>>> *1) Covered the current _meta implementation of unittests*
>>>> The current Options is not covered by unit tests at all, I have created 
>>>> the model_options test module contai

Re: [GSOC] Weekly update

2014-06-22 Thread Daniel Pyrathon
RE work in progress code

The main functions to optimize are get_new_fields, get_new_field, they are 
found 
here: 
https://github.com/PirosB3/django/blob/a92c37f0cad6bdd7a3b24ef235c81a7dab6bee72/django/db/models/options.py

On Sunday, June 22, 2014 11:49:05 AM UTC+2, Daniel Pyrathon wrote:
>
> Sure!
>
>
> https://github.com/PirosB3/django/compare/soc2014_meta_refactor_performance?expand=1
>
> This contains the following:
>  - Recently merged unit test suite for model/options
>  - New API implementation: get_new_fields, get_new_field.
>  - Implementation of recently merged unit test suite with new API
>  - Small optimizations (BIG work in progress)
>
> If you have any suggestions please let me know!
>
> Dan
>
> On Sunday, June 22, 2014 2:14:56 AM UTC+2, Curtis Maloney wrote:
>>
>> Is there somewhere I can look at your work in progress code?
>>
>>
>> On 21 June 2014 19:57, Daniel Pyrathon  wrote:
>>
>>> Hi All,
>>>
>>> This week I have done the following
>>>
>>> *- Openend PR and merged Options (_meta) unittests*
>>> https://github.com/django/django/pull/2823
>>> This is a working test suite of model/options.py file. Is is currently 
>>> testing the current Options implementation.
>>>
>>> *- Re-implemented test-suite in the new API*
>>> My new proposed API has been implemented on top of the Options 
>>> unittests. This means new and old API give the same results! and therefore 
>>> the new API is a working re-implementation.
>>>
>>>  *- Performance optimizations*
>>> Currently, the biggest issue I have had (in the new API) regards 
>>> performance. I am doing small optimisations and benchmarking with cProfile.
>>> If anyone has some good hints on how to benchmark single functions in 
>>> Django please let me know!
>>>
>>> Regards,
>>> Dan
>>>
>>>
>>> On Friday, June 13, 2014 10:11:41 PM UTC+2, Aymeric Augustin wrote:
>>>
>>>> I like this simplification a lot. I hope you can make it work.
>>>>
>>>> Just check that you don’t "overload" fields, by storing in field 
>>>> objects information that doesn’t belong there.
>>>>
>>>> -- 
>>>> Aymeric.
>>>>
>>>>
>>>>  
>>>> On 13 juin 2014, at 19:53, Daniel Pyrathon  wrote:
>>>>
>>>> Hi All,
>>>>
>>>> This week's work was a follow-up of last week's work, with some new 
>>>> discoveries with regards to the new API:
>>>>
>>>> *1) Improved the current _meta implementation of unittests*
>>>> I refactored the current meta unittest branch into a more compact 
>>>> version. I also added test cases for Generic Foreign Keys, RelatedField 
>>>> and 
>>>> more improvements on virtual fields. This section will actually be our 
>>>> first merge to master: a unit test suite for the current implementation.
>>>>
>>>> This branch is found here: https://github.com/
>>>> PirosB3/django/tree/soc2014_meta_unittests
>>>>
>>>> *2) Refactored the new _meta API spec*
>>>> By implementing the new API, I found new redundancies in the current 
>>>> implementation that we can avoid in the new API spec:
>>>>
>>>> *1) Only return field instances*
>>>> the current implementation has a common return pattern: (field_object, 
>>>> model, direct, m2m). After a few tests I realized that model, direct, m2m 
>>>> and field_name are all redundant and can be derived from only 
>>>> field_object. 
>>>> Since there are only 3-4 places that actually use direct and m2m it makes 
>>>> sense to remove this function from the new API spec.
>>>> Here I show how all the information can be derived form field_instance (
>>>> https://gist.github.com/PirosB3/6cb4badbb1b8c2e41a96/revisions), I ran 
>>>> the entire test suite and things look good. The only issue I can see now 
>>>> is 
>>>> from a performance point of view.
>>>>
>>>> *2) Provide only 2 API endpoints*
>>>>
>>>>  - get_fields(types, opts) -> (field_instance, field_instance, ...)
>>>>  - get_field(field_name) -> field_instance
>>>>
>>>> The rest is all (I might be very wrong! don't trust me) redundant. By 
>>>> continuing with this iterative approach, I will soon find out if I am 
>>>> correct or no

Re: [GSOC] Weekly update

2014-07-11 Thread Daniel Pyrathon
Hi Josh,

Thanks to your advice.

On Saturday, July 5, 2014 12:11:07 PM UTC+2, Josh Smeaton wrote:
>
> Excellent work, well done. I have a couple of questions/ideas though.
>
> 1. The use of bit flags irks me. Would it be possible just to use numerous 
> keyword arguments to the new get_fields ?
>

The new API dosen't use bit flags anymore, it has all been refactored out 
this week. It is much better now.
 

> 2. Since you've reduced the API to effectively two functions (get_fields, 
> get_field), is it still necessary to contain those methods on an inner 
> object? Can't the new methods live directly on the model? If they can't be 
> put directly on the model for some reason, I would consider a better name 
> than "meta". I can't think of a reason that the name meta should be used 
> other than history. If you were to think about the purpose of the API 
> without the associated history, what would you call it?
>

That's an interesting point! Do you think this is feasible? how many people 
will want to look fields up? I expect very few people using this API (only 
people that know what they are doing), but I might be very wrong.
Do we want to discuss this on a separate thread? I know Russell would like 
to write his own opinions too.
 

>
> log = Log.objects.get(pk=1)
> log.get_fields(concrete=True, m2m=True, related=True)
>
> log.get_field('log_level') # we're being explicit about the field name, no 
> need for options
>
>
> Thoughts?
>
> I'll take a better look at the PR and see if I can offer anything with 
> regards to your performance optimisations.
>
> Josh 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/dcece755-065e-4bbe-a3b1-c134a8a0162e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [GSOC] Weekly update

2014-07-11 Thread Daniel Pyrathon
Hi Curtis,

Of course! I will be happy to open a document with my performance tuning 
experience. Said this, I am very far away from being a "performance master" 
and I am still looking at places where my code can improve. It would be 
nice if it was a wiki page, where everyone can share and correct. 
Collective knowledge!
Where would you like me to write the document?

On Saturday, July 5, 2014 12:38:00 PM UTC+2, Curtis Maloney wrote:
>
> Can I ask as a favour to future works that you record all the performance 
> tuning approaches you use?
>
> Many of these will be obvious to seasoned Python programmers, but some of 
> them won't be, and it would be nice to have a catalogue of "things to look 
> out for" to apply to critical paths in Django.
>
> --
> Curtis
>
>
>
> On 5 July 2014 20:11, Josh Smeaton > 
> wrote:
>
>> Excellent work, well done. I have a couple of questions/ideas though.
>>
>> 1. The use of bit flags irks me. Would it be possible just to use 
>> numerous keyword arguments to the new get_fields ?
>> 2. Since you've reduced the API to effectively two functions (get_fields, 
>> get_field), is it still necessary to contain those methods on an inner 
>> object? Can't the new methods live directly on the model? If they can't be 
>> put directly on the model for some reason, I would consider a better name 
>> than "meta". I can't think of a reason that the name meta should be used 
>> other than history. If you were to think about the purpose of the API 
>> without the associated history, what would you call it?
>>
>> log = Log.objects.get(pk=1)
>> log.get_fields(concrete=True, m2m=True, related=True)
>>
>> log.get_field('log_level') # we're being explicit about the field name, 
>> no need for options
>>
>>
>> Thoughts?
>>
>> I'll take a better look at the PR and see if I can offer anything with 
>> regards to your performance optimisations.
>>
>> Josh 
>>  
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/7496ef6d-4e88-41f4-b2b8-1bd2efa96aea%40googlegroups.com
>>  
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ccf5ffb4-dd3c-492a-80e6-eaae77934373%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [GSOC] Weekly update

2014-07-11 Thread Daniel Pyrathon
Hi All,

Following this week's work. I have made progress in optimisation and design 
of the API.

I have opened a wiki page that contains all:
- Main concepts
- Decision process
- Benchmarks
- API designs
- How you can help!

Please see attached: https://code.djangoproject.com/wiki/new_meta_api

For anything, as usual, just let me know :)
Daniel

On Saturday, July 5, 2014 12:11:07 PM UTC+2, Josh Smeaton wrote:
>
> Excellent work, well done. I have a couple of questions/ideas though.
>
> 1. The use of bit flags irks me. Would it be possible just to use numerous 
> keyword arguments to the new get_fields ?
> 2. Since you've reduced the API to effectively two functions (get_fields, 
> get_field), is it still necessary to contain those methods on an inner 
> object? Can't the new methods live directly on the model? If they can't be 
> put directly on the model for some reason, I would consider a better name 
> than "meta". I can't think of a reason that the name meta should be used 
> other than history. If you were to think about the purpose of the API 
> without the associated history, what would you call it?
>
> log = Log.objects.get(pk=1)
> log.get_fields(concrete=True, m2m=True, related=True)
>
> log.get_field('log_level') # we're being explicit about the field name, no 
> need for options
>
>
> Thoughts?
>
> I'll take a better look at the PR and see if I can offer anything with 
> regards to your performance optimisations.
>
> Josh 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/02f84164-7891-44b3-a11a-068c3fc01d46%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [GSOC] Weekly update

2014-07-18 Thread Daniel Pyrathon
Hi All,

As usual, here are my updates:

*- Formalised pull request, looking for a formal review*
Last week I published the wiki page for my meta implementation. This week I 
have been working on improving internal code documentation, optimising a 
few bits and bobs, and added the formal documentation.
As usual, all code is available 
here: https://github.com/django/django/pull/2894.
Now it's time for a real *full* *review *of the code, as maybe a few 
concepts and terms will change but the code is formal and most things might 
not change.

*- Published new benchmarks, and explanations for some slowdowns*
The benchmarks for my new implementation are available on the PR 
(https://github.com/django/django/pull/2894). These benchmarks are taken 
from DjangoBench (all numbers are the median of 2000 runs of each test). 
You might see that, while overall performance has increased compared to the 
current implementation, there are some very odd results. This week I have 
spent some time investigating the reason for some of the major slowdowns 
and I have come to the following conclusion:
DjangoBench profiling is implemented by spinning up a separate process for 
each benchmark. 
(https://github.com/django/djangobench/blob/master/djangobench/main.py#L135). 
If you look at the chapter "Compute inverse relation map on first access" 
(https://code.djangoproject.com/wiki/new_meta_api#Computeinverserelationmaponfirstaccess)
 
you will see that the new caching model is more expensive on startup, but 
is only done once per process. This means we pay a little more price on 
startup, but then the Options internals do much less work. Although I 
cannot say I am 100% correct (and I will be investigating further) I 
believe the slowdown on some benchmarks is related to this, and therefore 
it may be related more on the way it is benchmarked, rather than the 
benchmark itself.

*- Started on my GMail Store*
While I was waiting for review, I have started an implementation of my 
GMail store. The implementation is documented here 
(https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit#heading=h.wyxx4xxijubt)
 
and will be my final deliverable.
So far, I have had some interesting results. With very little code, and the 
new Options API, I didn't have to modify a single file inside Django for my 
implementation to work. All code just overrides some of the main Django 
classes (Manager, Query, QuerySet).

Currently, I have managed to:

   - Browse my GMail threads through Django admin
   - View a single GMail thread
   - Read an entire email!
   - Use the Console to do everything said above!

I have attached a picture.

<https://lh3.googleusercontent.com/-br9TSK3bRXA/U8lX7kp3qeI/Lbw/nr0oHgRLkRk/s1600/png.png>

The next step is to improve my implementation. I will be (hopefully) soon 
releasing a GitHub project that you will be able to use to browse, read and 
send your mail through Django admin and forms!

*- I WILL BE ATTENDING EUROPYTHON*
I'd love to meet up with anyone attending EuroPython this week. I will 
obviously participate at the sprints but if you want to catch up before it 
would be great. It will be a great idea to discuss the new API together and 
get to know the community more.
I might also be doing a lightning talk on the new Options API, or the GMail 
store depending on progress.

For anything, as usual, let me know
Daniel Python

On Friday, July 11, 2014 4:59:09 PM UTC+2, Daniel Pyrathon wrote:
>
> Hi All,
>
> Following this week's work. I have made progress in optimisation and 
> design of the API.
>
> I have opened a wiki page that contains all:
> - Main concepts
> - Decision process
> - Benchmarks
> - API designs
> - How you can help!
>
> Please see attached: https://code.djangoproject.com/wiki/new_meta_api
>
> For anything, as usual, just let me know :)
> Daniel
>
> On Saturday, July 5, 2014 12:11:07 PM UTC+2, Josh Smeaton wrote:
>>
>> Excellent work, well done. I have a couple of questions/ideas though.
>>
>> 1. The use of bit flags irks me. Would it be possible just to use 
>> numerous keyword arguments to the new get_fields ?
>> 2. Since you've reduced the API to effectively two functions (get_fields, 
>> get_field), is it still necessary to contain those methods on an inner 
>> object? Can't the new methods live directly on the model? If they can't be 
>> put directly on the model for some reason, I would consider a better name 
>> than "meta". I can't think of a reason that the name meta should be used 
>> other than history. If you were to think about the purpose of the API 
>> without the associated history, what would you call it?
>>
>> log = Log.objects.get(pk=1)
>> log.get_fields(concrete=True, m2m=True, related=True)
>>
>> log.get_fiel

Fields terminology for official Options API

2014-07-27 Thread Daniel Pyrathon
Hi All,

I am currently working on the new Options API 
(https://github.com/django/django/pull/2894).

The Options API is at the core of Django, it enables introspection of 
Django Models with the rest of the system. This enables lookups, queries, 
forms, admin to understand the capabilities of every model. The Options API 
is hidden under the _meta attribute of each model class. Options has always 
been a private API, but Django developers have always been using it in 
their own projects, in a non-official way.  As part of my SoC project, I am 
exposing the new API for public use. As we are also formalizing a new API 
that has always existed, It is important to revisit the terminology and 
document it.

The current terminology is described here 
https://code.djangoproject.com/wiki/new_meta_api 
in the section concepts. I would encourage all contributors to read the 
document and express their opinion, if they think something should be 
changed.

A few interesting comments up till now:

Loic on the PR:

   - I prefer "concrete" to "data" as it makes the parallel with "virtual" 
   more obvious.
   - "related_objects" > "reverse_rel"
   - "related_m2m" > "reverse_m2m"
   - What about reverse o2o, do they currently fall under "related_objects"?
   - The main issue I have with "related" is that it doesn't provide a 
   sense of direction, after all both sides of a FK have related objects 
   waiting on the other end. I also like the symmetry between "m2m" and 
   "reverse_m2m".
   - Django isn't always consistent (and sometime actually wrong) in naming 
   relations, especially indjango/db/models/fields/related.py. Now that we 
   start documenting all those things, I see value in getting it right.


Thanks,
Daniel

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/e2836a1b-ad2e-4f32-b052-3177796d97f7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [GSoC] Update of my work

2014-07-29 Thread Daniel Pyrathon
Nice work! amazing as usual Anubhav ;)


2014-07-27 9:24 GMT+02:00 anubhav joshi :

> Hi all
>
> My project "*Improving error reporting & Fixing up the related issues" 
> *proposal
> can be seen here: GSoC Proposal
> 
>
> I have been keeping a log of all the work I have been doing for past weeks
> on gist. I thought it'd be good to share it here.
> GSoC Work 
>
> The work done in 10 weeks is listed out in the gist.
> I still have few more issues left to be worked upon in the remainder of
> the time.
>
>
> Regards,
> Anubhav Joshi
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/62649896-e493-497f-9c23-c4f3c8418191%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
*

PirosB3

https://github.com/PirosB3 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAOmo0o9wQ68tafpLoSTGWLne%2BND%2BddkRtFGdSrspRmGTa9-znw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: [GSOC] Weekly update

2014-08-03 Thread Daniel Pyrathon
ave a direct 
entry on the database, but uses the data of 1 or more other fields to 
create special, abstract, structures.
An example of this could be a "composite field" such as a Point2D:

class City(models.Model):
  x = models.FloatField()
  y = models.FloatField()
  position = Point2D('x', 'y')

position is a virtual field because it does not have any presence on the 
database and relies on the information of 'x' and 'y'.

*Finally*
I apologise for posting late, EuroPython was a really good conference and 
it was great to meet some of the core developers, such as Tom, Honza, 
Baptiste. I feel I worked 50% of my time, and I will recover in the future.
In the mean-time, I hope to make up for it with:

 - Latest benchmarks! (coming soon)
 - A picture of me with Baptiste, at the sprints!

<https://lh3.googleusercontent.com/-85QCib0NdRA/U9401yTNqtI/LeY/vKzvQ-FtQHY/s1600/Photo+27-07-14+15+00+42.jpg>

 

On Saturday, July 19, 2014 10:42:55 PM UTC+2, Andre Terra wrote:
>
> On Sat, Jul 19, 2014 at 2:38 AM, Raffaele Salmaso  > wrote:
>
>> On Fri, Jul 18, 2014 at 7:32 PM, Daniel Pyrathon > > wrote:
>>
>>> *- Started on my GMail Store*
>>> (...)
>>>
>> Pirosb3, just three words: really nice work!
>>  
>>
>
> I absolutely agree! The possibilities are endless.
>
> Congratulations on delivering such a major contribution to the framework! 
> I am sure your work will be a key benchmark to future GSOC applicants.
>
>
> Cheers,
> AT
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/42037bd1-3ac0-4fdc-a359-4ae967283489%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [GSOC] Weekly update

2014-08-03 Thread Daniel Pyrathon
Hi Aymeric,

Thanks for writing back

On Sunday, August 3, 2014 4:24:27 PM UTC+2, Aymeric Augustin wrote:
>
> On 3 août 2014, at 15:11, Daniel Pyrathon > 
> wrote:
>
> *1) get_fields should return a list instead of a tuple*
> Previously, get_fields would return a tuple. The main reason was related 
> to memory and immutability. After discussing with Russell, we decided this 
> endpoint should actually return a list as it is a more suited data 
> structure.
>
>
> Can you clarify the reasons for this choice? If it’s just the purity of 
> using the “proper” data type, I’m not sure it beats the practicality of 
> returning an immutable iterable and avoiding the cost of a copy or the risk 
> of incorrect manipulation.
>

This is a design decision, tuple performs better. 
 

>
> The bugs that would occur if the cached value is altered inadvertently 
> could be very hard to track down.
>

Only a couple of days...
https://github.com/PirosB3/django/commit/2411ff58d032c38a3151c1e54198723a86e6a8af
 

>
> *2) Move tree cache out of the apps registry*
> The main optimisation for the new API (described here 
> https://code.djangoproject.com/wiki/new_meta_api#Mainoptimizationpoints) 
> is currently storing information on the Apps registry. After discussing 
> with Russell we agree this shouldn't be the correct place to keep this.
>
>
> +1
>
> A solution is to store related fields information separately on each model 
> (
> https://github.com/PirosB3/django/pull/5/files#diff-98e98c694c90e830f918eb5279134ab9R275).
>  
> This has been done, and all tests pass.
> Unfortunately, there are performance implications with this approach: we 
> are storing a new list on each model regardless of if it has related fields 
> or not. I will be discussing with Russell about performance tomorrow.
>
>
> Is the extra cost incurred only at start up ? How large is it? If it isn’t 
> too bad and and run time performance is unchanged, it could be acceptable.
>

Yes! only at start-up. Once cache warming has finished, everything should 
be as fast (maybe faster, given that we are accessing properties on the 
single Options instance).
 

>
> *3) Revisiting field types and options*
> Last week I opened this: 
> https://groups.google.com/forum/#!topic/django-developers/s2Lp2lTEjAE in 
> order to discuss with the community possible new field and option names. 
> This week I have actually revisited the fields and options for get_field/s 
> and I have come up with the following: (…)
>
>
> The latest iteration looks clear, consistent and straightforward. I like 
> it.
>

Thanks! Please feel free to comment with doubts or suggestions.
 

>
> -- 
> Aymeric.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/43fd8da7-b233-4914-8df8-14314276b1f0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [GSOC] Weekly update

2014-08-04 Thread Daniel Pyrathon
Hi All,

This has been resolved by using the ImmutableList datastructure

https://github.com/PirosB3/django/blob/soc2014_meta_refactor_upgrade_flags_get_field_tree/django/db/models/options.py#L629

On Monday, August 4, 2014 2:44:38 PM UTC+2, Collin Anderson wrote:
>
> if we do really need a list, could we gain some performance by caching 
> tuples and converting them to lists instead of caching lists?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/f4795467-24c7-4a61-af78-1a5b1a16299d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


State of concrete fields

2014-08-04 Thread Daniel Pyrathon
Hi All,

The current Options implementation has properties for "concrete fields".
Technically speaking, concrete fields are data fields without a column. The 
only concrete field in the codebase is ForeignObject. There are 2 classes 
that inherit from ForeignObject: GenericRelation and ForeignKey, but each 
of them fall into different categories (virtual and data). The only 
instances of a ForeignObject in the codebase are in the unit tests, and 
those occurrences can be replaced with a ForeignKey (not tested).
So, my question is:

- Do we really need concrete fields, if ForeignObject is internal and not 
used in any place other than unit tests?
- Are there third-party packages that inherit from ForeignObject? if yes, 
can anyone point which ones?

Perhaps akaariai and loic84 can help, as I know they have done work in this 
area.

As we are formalising the Options API 
(https://groups.google.com/forum/#!topic/django-developers/XfMUjK59Sls) it 
would be great to reduce complexity even more by removing the entire term 
"concrete fields".

Thanks,
Daniel Pyrathon

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c797b8ea-c7dd-44ea-9b3c-3d6aa5df5ddf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [GSOC] Weekly update

2014-08-06 Thread Daniel Pyrathon
Hi Łukasz,

Sorry for getting back to you now. Thanks Russell for the comments.
As Russell says, it's more of a conceptual choice. My first implementation 
of this API was using tuples for the same
reason you said above. I think ImmutableList is a good compromise between 
list and tuple. It overrides a tuple (so it should be performant) and is 
still conceptually a list.
Is this clear? please let me know if you want me to explain further our 
decision on this.

Daniel

On Monday, August 4, 2014 4:54:45 PM UTC+2, Łukasz Rekucki wrote:
>
> On 4 August 2014 16:14, Daniel Pyrathon > 
> wrote: 
> > Hi All, 
> > 
> > This has been resolved by using the ImmutableList datastructure 
> > 
> > 
> https://github.com/PirosB3/django/blob/soc2014_meta_refactor_upgrade_flags_get_field_tree/django/db/models/options.py#L629
>  
> > 
>
> But why? What's the benefit over using a tuple? ImmutableList is not 
> even a list, because it inherits from tuple. 
>
> The only other use of this data structure I could find is in upload 
> handlers and the rationale is that the field suddenly changes from 
> mutable to immutable. I don't think your case is the same, as fields 
> are always immutable. 
>
> Also, if fields() is immutable, then so should concrete_fields(), etc. 
>
>
> > 
> > On Monday, August 4, 2014 2:44:38 PM UTC+2, Collin Anderson wrote: 
> >> 
> >> if we do really need a list, could we gain some performance by caching 
> >> tuples and converting them to lists instead of caching lists? 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "Django developers" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > email to django-develop...@googlegroups.com . 
> > To post to this group, send email to django-d...@googlegroups.com 
> . 
> > Visit this group at http://groups.google.com/group/django-developers. 
> > To view this discussion on the web visit 
> > 
> https://groups.google.com/d/msgid/django-developers/f4795467-24c7-4a61-af78-1a5b1a16299d%40googlegroups.com.
>  
>
> > 
> > For more options, visit https://groups.google.com/d/optout. 
>
>
> -- 
> Łukasz Rekucki 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4a1a4b6e-87c5-4554-b47a-2d7dbdae6a64%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [GSOC] Weekly update

2014-08-08 Thread Daniel Pyrathon


On Wednesday, August 6, 2014 3:46:42 PM UTC+2, Tom Christie wrote:
>
> > This has been resolved by using the ImmutableList datastructure
>
> Looks fine. Behaviourally it's the same as a tuple, but with more explicit 
> errors if you attempt to modify it, which seems reasonable.
>
> Given that `.get_fields()` and `.fields` return `ImmutableList`, 
> presumably `.field_names`, `.concrete_fields` and `.local_concrete_fields` 
> should do as well right?
>

Of course, I am pushing a new version of this now
 

>
> > A tuple is *not* "just an immutable list".
>
> Interestingly, that's a point where we'd differ. In my mental model a 
> programming construct is defined *purely* by it's behaviour, with no room 
> for any 'conceptual' or 'theroretical' difference - as far as I'm concerned 
> an (unnamed) python tuple really *is* just an immutable python list.
>
> I think any further conversation on that is out of the scope of this list, 
> but I've heard both positions advocated, and figured it's an interesting 
> point to note. :)
>
> (And it does have the occasional impact on how we'd choose to write 
> documentation examples etc...)
>
> All the best & keep up the great work,
>
>   Tom
>
>
> On Wednesday, 6 August 2014 01:33:53 UTC+1, Russell Keith-Magee wrote:
>>
>>
>> On Tue, Aug 5, 2014 at 12:54 AM, Łukasz Rekucki  
>> wrote:
>>
>>> On 4 August 2014 16:14, Daniel Pyrathon  wrote:
>>> > Hi All,
>>> >
>>> > This has been resolved by using the ImmutableList datastructure
>>> >
>>> > 
>>> https://github.com/PirosB3/django/blob/soc2014_meta_refactor_upgrade_flags_get_field_tree/django/db/models/options.py#L629
>>> >
>>>
>>> But why? What's the benefit over using a tuple? ImmutableList is not
>>> even a list, because it inherits from tuple.
>>>
>>
>> Communication. 
>>
>> From a purist theoretical perspective, there shouldn't be any argument - 
>> the data we're talking about is a list. Lists have homogeneous elements; 
>> Tuples have heterogeneous elements, but have *positional* homogeneity. A 
>> "Point" is a tuple, because element 0 of the tuple "means" the x 
>> coordinate. A Database row is a tuple - The first element is the primary 
>> key (an integer), second is the "name" column (a string), and so on.
>>
>> A tuple is *not* "just an immutable list".
>>
>> From a pragmatic perspective, tuples are faster to work with, because 
>> they aren't carrying around all the baggage needed to support mutability. 
>> And, in this case, there is a risk of an end-user accidentally mutating the 
>> result of get_fields(), which, due to cache-related optimizations, means 
>> there's a risk of side effects.
>>
>> So - in this case, at a theoretical level, we are dealing with an list of 
>> homogeneous objects (fields) that must be either immutable, or copied every 
>> time it is used. Copying is a bit expensive (not *completely* prohibitive, 
>> but noticeable), so that means we need to look to immutability. At a 
>> theoretical I'm not wild about the fact that ImmutableList subclassing 
>> tuple - it should be subclassing *list* - but I'm willing to defer to 
>> pragmatism. Given that we're dealing with a relative internal detail that 
>> most users won't be exposed to, I'm willing to hold my nose and accept 
>> optimised solutions over theoretically pure solutions in the interests of 
>> *all* users having better performance.
>>
>> Yours,
>> Russ Magee %-)
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/47502873-a189-4aa0-8278-c7ab047dfd7f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [GSOC] Weekly update

2014-08-09 Thread Daniel Pyrathon
Hi All,

This week I have been working on mainly 3 tasks:

*- Formalizing the PR*
2 days were dedicated on fixing all the small issues on the PR. I am now 
ready for another review.

*- Simplified the API for get_field*
There are 3 used combinations of get_field() in the entire Django codebase


   - get_field(data=True, m2m=True) > 60% of the occurrences
   - get_field(data=True, m2m=True, related_objects=True, related_m2m=True, 
   virtual=True) > 39.9% of the occurrences
   - get_field(data=False, m2m=True) once only! (0.01% of occurrences).

The new API replaces the first 2 cases by generalizing: 
get_field(field_name, include_related=False, **kwargs).
The API is still 100% backwards compatible with the old API, until Django 
2.0.

Benefits:

   - better and simpler API
   - better memory management (caching)

*The mailer*
As my final deadline is near, I continued working on the mailer. The mailer 
is a custom store that allows Django admin and forms to interact with the 
GMail API.
I have published the source code for this 
here: https://github.com/PirosB3/django-mailer/
While the code is still "a big hack", I aim on getting a PoC fully working 
and then do some refactoring. This week I managed to get relations between 
custom models working, *all using the new Options API! *The result is I can 
now browse my mail through the admin panel!

<https://lh6.googleusercontent.com/-5lUx63hvlZw/U-Yprxl8AzI/Lg0/8qNh6qw3YtA/s1600/5512779.png>
*- Help me get people excited about this!*
The new Options API works, and allows developers to create custom stores. 
This will allow us to create official NoSQL stores, SqlAlchemy stores, and 
even a IMAP store. I think it's a great feature that has the potential to 
make Django more decoupled and more powerful.
Said this, it still needs loads of work. And I am sure the API can still 
improve. Let's get more people aware of the new possibilities and help me 
make a better API:

- You can retweet https://twitter.com/pirosb3/status/497701148665843715
- You can review https://github.com/django/django/pull/2894

Thanks!
Daniel
  


On Friday, August 8, 2014 6:37:06 PM UTC+2, Daniel Pyrathon wrote:
>
>
>
> On Wednesday, August 6, 2014 3:46:42 PM UTC+2, Tom Christie wrote:
>>
>> > This has been resolved by using the ImmutableList datastructure
>>
>> Looks fine. Behaviourally it's the same as a tuple, but with more 
>> explicit errors if you attempt to modify it, which seems reasonable.
>>
>> Given that `.get_fields()` and `.fields` return `ImmutableList`, 
>> presumably `.field_names`, `.concrete_fields` and `.local_concrete_fields` 
>> should do as well right?
>>
>
> Of course, I am pushing a new version of this now
>  
>
>>
>> > A tuple is *not* "just an immutable list".
>>
>> Interestingly, that's a point where we'd differ. In my mental model a 
>> programming construct is defined *purely* by it's behaviour, with no room 
>> for any 'conceptual' or 'theroretical' difference - as far as I'm concerned 
>> an (unnamed) python tuple really *is* just an immutable python list.
>>
>> I think any further conversation on that is out of the scope of this 
>> list, but I've heard both positions advocated, and figured it's an 
>> interesting point to note. :)
>>
>> (And it does have the occasional impact on how we'd choose to write 
>> documentation examples etc...)
>>
>> All the best & keep up the great work,
>>
>>   Tom
>>
>>
>> On Wednesday, 6 August 2014 01:33:53 UTC+1, Russell Keith-Magee wrote:
>>>
>>>
>>> On Tue, Aug 5, 2014 at 12:54 AM, Łukasz Rekucki  
>>> wrote:
>>>
>>>> On 4 August 2014 16:14, Daniel Pyrathon  wrote:
>>>> > Hi All,
>>>> >
>>>> > This has been resolved by using the ImmutableList datastructure
>>>> >
>>>> > 
>>>> https://github.com/PirosB3/django/blob/soc2014_meta_refactor_upgrade_flags_get_field_tree/django/db/models/options.py#L629
>>>> >
>>>>
>>>> But why? What's the benefit over using a tuple? ImmutableList is not
>>>> even a list, because it inherits from tuple.
>>>>
>>>
>>> Communication. 
>>>
>>> From a purist theoretical perspective, there shouldn't be any argument - 
>>> the data we're talking about is a list. Lists have homogeneous elements; 
>>> Tuples have heterogeneous elements, but have *positional* homogeneity. A 
>>> "Point" is a tuple, because element 0 of the tuple "means" the x 
>>> coordinate. A Databas

Re: [GSOC] Weekly update

2014-08-14 Thread Daniel Pyrathon
Hi All,

Since last week, the new API has changed: we went from 5 field types to 9 
field types. Below is a list of all field types, the bold ones are the ones 
that have just been added.

 * pure data (CharField etc)
* * relation data (FK)*
 * pure virtual (Point)
* * relation virtual (GFK)*
 * m2m (badly named; nothing at present, but possibly in a document store)
* * relation m2m (badly named; M2M fields)*
 * related object (Reverse FK) 
 * related m2m (Reverse M2M)
 * related virtual (GenericRelation)

More information on the new API can be seen on my documentation server 
at: 
http://5.101.98.142:49156/ref/models/meta.html#module-django.db.models.options

The old API can be seen at: 
http://5.101.98.142:49155/ref/models/meta.html#module-django.db.models.options

I will be posting again as soon as benchmarks finish. In the meantime it 
would be great to have some feedback.

Thanks,
Daniel 



On Saturday, August 9, 2014 4:10:31 PM UTC+2, Daniel Pyrathon wrote:
>
> Hi All,
>
> This week I have been working on mainly 3 tasks:
>
> *- Formalizing the PR*
> 2 days were dedicated on fixing all the small issues on the PR. I am now 
> ready for another review.
>
> *- Simplified the API for get_field*
> There are 3 used combinations of get_field() in the entire Django codebase
>
>
>- get_field(data=True, m2m=True) > 60% of the occurrences
>- get_field(data=True, m2m=True, related_objects=True, 
>related_m2m=True, virtual=True) > 39.9% of the occurrences
>- get_field(data=False, m2m=True) once only! (0.01% of occurrences).
>
> The new API replaces the first 2 cases by generalizing: 
> get_field(field_name, include_related=False, **kwargs).
> The API is still 100% backwards compatible with the old API, until Django 
> 2.0.
>
> Benefits:
>
>- better and simpler API
>- better memory management (caching)
>
> *The mailer*
> As my final deadline is near, I continued working on the mailer. The 
> mailer is a custom store that allows Django admin and forms to interact 
> with the GMail API.
> I have published the source code for this here: 
> https://github.com/PirosB3/django-mailer/
> While the code is still "a big hack", I aim on getting a PoC fully working 
> and then do some refactoring. This week I managed to get relations between 
> custom models working, *all using the new Options API! *The result is I 
> can now browse my mail through the admin panel!
>
>
> <https://lh6.googleusercontent.com/-5lUx63hvlZw/U-Yprxl8AzI/Lg0/8qNh6qw3YtA/s1600/5512779.png>
> *- Help me get people excited about this!*
> The new Options API works, and allows developers to create custom stores. 
> This will allow us to create official NoSQL stores, SqlAlchemy stores, and 
> even a IMAP store. I think it's a great feature that has the potential to 
> make Django more decoupled and more powerful.
> Said this, it still needs loads of work. And I am sure the API can still 
> improve. Let's get more people aware of the new possibilities and help me 
> make a better API:
>
> - You can retweet https://twitter.com/pirosb3/status/497701148665843715
> - You can review https://github.com/django/django/pull/2894
>
> Thanks!
> Daniel
>   
>
>
> On Friday, August 8, 2014 6:37:06 PM UTC+2, Daniel Pyrathon wrote:
>>
>>
>>
>> On Wednesday, August 6, 2014 3:46:42 PM UTC+2, Tom Christie wrote:
>>>
>>> > This has been resolved by using the ImmutableList datastructure
>>>
>>> Looks fine. Behaviourally it's the same as a tuple, but with more 
>>> explicit errors if you attempt to modify it, which seems reasonable.
>>>
>>> Given that `.get_fields()` and `.fields` return `ImmutableList`, 
>>> presumably `.field_names`, `.concrete_fields` and `.local_concrete_fields` 
>>> should do as well right?
>>>
>>
>> Of course, I am pushing a new version of this now
>>  
>>
>>>
>>> > A tuple is *not* "just an immutable list".
>>>
>>> Interestingly, that's a point where we'd differ. In my mental model a 
>>> programming construct is defined *purely* by it's behaviour, with no room 
>>> for any 'conceptual' or 'theroretical' difference - as far as I'm concerned 
>>> an (unnamed) python tuple really *is* just an immutable python list.
>>>
>>> I think any further conversation on that is out of the scope of this 
>>> list, but I've heard both positions advocated, and figured it's an 
>>> interesting point to note. :)
>>>
>>> (And it does have the occasional impact on how we'd choose to write 
>>> doc

Re: GSoC Meta refactor: Bikeshedding time!!

2014-08-18 Thread Daniel Pyrathon
Hi All,

First of all, thanks Russell for bringing this discussion up.

*Regarding get_fields complication*
Throughout the development of this project, I have realised that 90% of the 
API usage inside and outside of Django can rely entirely on 4 or 5 cached 
properties.
The most used API calls are:

- get all data fields
- get all m2m fields
- get all related data
- get all related m2m
- get field by name

These 5 are by far the most used endpoints of the API. Said this, there is 
a small set of very necessary endpoints that are called in only a few 
places, such as:

- get all related data with hidden fields 
- get all related data including proxy relations
- get all data fields that have a column

Some of these, have been refactored in-place and are not part of the API 
any more. Others unfortunately are still subsets of the API but I 
personally see very few people (none actually) wanting this information for 
other use.
For this reason, as an end-user, you should think of this API only as a set 
of (cached) properties as most likely you will never need to use the 
get_fields API directly. To make this a little clearer, I have attached an 
image of the API.




*Regarding FileField*
I think this is still a data field. The fact that it stores an image path 
and it's getter/setter does some magic does not change it's "identity". 
ImageField is, based on the definition of virtual so far, a virtual field.


On Monday, August 18, 2014 6:50:04 AM UTC+2, Russell Keith-Magee wrote:
>
> Hi Shai,
>
> On Sun, Aug 17, 2014 at 3:44 AM, Shai Berger  > wrote:
>
>> Hi,
>>
>> It seems to me that the taxonomy doesn't handle well FileField and 
>> ImageField.
>> It could be bundled in with ForeignKey (as the data it really represents 
>> is
>> only pointed at by the related column data), but not with the current 
>> wording.
>>
>> For ImageField, there is -- in addition to the above -- the relation to
>> height_field and width_field. It would appear to be a mix between a pure 
>> field
>> and a virtual field.
>>
>
> I'm not sure this is a problem. 
>
> I agree that conceptually, part of the data for a FileField/ImageField is 
> held "externally"; but it's a different kind of external. From the 
> database's perspective, the record is complete and correct when you're 
> storing a string. The fact that the string represents a file system path is 
> a very significant implementation detail - after all, you need to know to 
> show a file browsing dialog (or whatever UI you want) - but then, the same 
> is true of a date field needing a date picker, and a boolean field needing 
> a checkbox. It doesn't affect the way the database needs to interact with 
> the data it is storing.
>
> However, this might be a manifestation of the sort of problem Anssi raised 
> - that the taxonomy I've suggested is too rich, and that we need to 
> simplify to the practical use cases, rather than try and build a complex 
> and descriptive API.
>
> Russ %-)
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/16a14d37-383f-4e61-b026-20e16f38d4b7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: GSoC Meta refactor: Bikeshedding time!!

2014-08-18 Thread Daniel Pyrathon
Hi Anssi

With regards to Michal Petrucha's changes, I have the feeling you are 
correct. A solution to this could be to provide a cached property called 
**foreign_keys** that will disguise the actual identity of a ForeignKey. 
Once the Michael's changes are introduced, we could start deprecating the 
cached property and raise a soft warning announcing the change, just like 
we are doing with all the other API calls in Options.
Said this, if developers decide to go with the get_fields() API, then 
unfortunately this will not help.

On Monday, August 18, 2014 12:03:32 PM UTC+2, Anssi Kääriäinen wrote:
>
> On Monday, August 18, 2014 7:45:17 AM UTC+3, Russell Keith-Magee wrote:
>>
>> I understand what you're driving at here, and I've had similar thoughts 
>> over the course of the SoC. The catch is that this makes the API for 
>> get_fields() fairly complicated.
>>
>> If every field fits into one specific type, then get_fields() just 
>> requires a single boolean flag (do I include fields of type X) for each 
>> field type. We can also easily add new field types by adding new booleans 
>> to the API.
>>
>> However, if a field fits into multiple categories, then it's impossible 
>> (or, at least, exceedingly complicated) to make a single call to 
>> get_fields() that will specify all your field requirements. "Get me all 
>> non-virtual data fields" requires "virtual=False, data=True, m2m=False", 
>> but "Get all virtual data fields that represent m2ms" requires 
>> "virtual=True, data=False, m2m=True". You can't pass in both sets of 
>> arguments at the same time, so you either have to make multiple calls to 
>> get_fields(), or you have to invent some sort of query syntax for 
>> get_fields() that allows union queries. 
>>
>> Plus, at the end of the day, get_fields() is abstracted behind highly 
>> cached and optimised properties for key lookups. These properties are 
>> effectively a cached call to get_fields() with a specific set of arguments 
>> - so even if get_fields() doesn't expose a "one category per field" 
>> requirement, the API will require, at some level, names that have clear 
>> (and preferably non-overlapping) membership.
>>
>
> If fields are in multiple categories then users will want to do the full 
> range of set operation on the categories. Encoding that in to the API 
> doesn't sound promising.
>
> I don't think users actually want to get fields based on the suggested 
>>> categorization. I feel we get an easier to use and more flexible API if we 
>>> have higher level categories and allow fields to match multiple categories. 
>>> As a practical example if I want all relation fields, that is going to be 
>>> hard using the suggested API. Getting all relation fields is a more 
>>> realistic use case than getting related virtual objects.
>>>
>>
>> Quite probably true. As a point of interest, the current (as in, 1.6) API 
>> actually doesn't differentiate between category (a) "pure data" and 
>> category (b) "relating data (i.e., FK)" fields - if you ask for "data 
>> fields" you get pure data *and* foreign keys. So, at least as far as 
>> Django's own usage is concerned, you're correct in saying that taxonomy 
>> I've described isn't fully required. 
>>
>> Daniel's survey of internal usage reveals that there are three use cases 
>> for getting a list of fields in Django's internal API:
>>
>>  * Get all data and m2m fields (i.e., categories  a, b, and d). This is 
>> effectively "all fields on *this* model"
>>
>>  * Get all data, m2m, related objects, related m2m, and virtual fields 
>> (i.e., categories a, b, d, f, g, h, i - excluding c and e because Django 
>> doesn't currently have any fields of this type). This is "all fields on 
>> this model, or related to this model"
>>
>>  * Get all m2m fields (i.e., category d)
>>  
>> So - at the very least, we need names to describe those three groups. My 
>> intention with describing a richer taxonomy is to try and give names to 
>> other groupings of interest. 
>>
>> If we want to have all fields to match single and only single category, 
>>> then we need to redefine the categories to make sure ForeignKeys as virtual 
>>> fields are possible, and that more esoteric custom join based fields fit in 
>>> to the categorization.
>>>
>>
>> Agreed - that's why I threw this out there for discussion :-)
>>
>> Properties like "data", "virtual", "external", "related", "relating" - 
>> these are high level concepts describing the way a field manifests. 
>> However, that doesn't mean we need to expose these properties as part of 
>> the formal API.
>>
>> Part of the underlying problem here -- lets say we roll out Django 1.7 
>> with some version of this API, and in 1.8, foreign key fields change to 
>> become virtual. That effectively becomes backwards incompatible for queries 
>> that are sensitive to a "virtual" flag; but it doesn't change the 
>> underlying need to identify that a field is a foreign key. We need to 
>> capture the latter use case, but no

Re: GSoC Meta refactor: Bikeshedding time!!

2014-08-18 Thread Daniel Pyrathon
Hi Shai,

Thanks for getting back, so..

On Monday, August 18, 2014 1:44:49 PM UTC+2, Shai Berger wrote:
>
> Hi again, 
>
> Below, ">D" are quotations from Daniel's message I'm replying to, and 
> ">R" are from Russell's message that opened this thread. 
>
> >D  *Regarding FileField* 
>
> It took me some time to clear for myself why FileField is a data field, 
> and not 
> like FK: The point is not where the data is stored and whether the DB 
> field 
> contents are "it" or just a pointer -- the point is whether another model 
> is 
> involved. I think this is a key distinction; wording the taxonomy in these 
> terms can clarify certain other issues. As an example, 
>
> >R  a) "Pure" data fields - things like CharField, IntegerField, etc. 
> Fields 
> >R  that manifest as a single column on the model's underlying table. 
>
> this wording makes it a little unclear (at least to me) if parent fields 
> fall 
> into this category (as they do not "manifest as a single column on the 
> model's 
> underlying table"). But the higher-level language, "fields that store 
> primitive 
> data on the model instance itself" -- with a better word for "primitive", 
> or a 
> full explanation on what it means -- is something I find much clearer and 
> more 
> accurate.
>

Thanks, will change the docs.
 

>
> >D  ImageField is, based on the definition of virtual so far, a virtual 
> field. 
>
> Well, as it is not related nor relating, the only virtual field it could 
> be is: 
>
> >R  e) "Pure" virtual fields - Fields that are conceptual wrappers around 
> other 
> >R  fields on the same model. Virtual fields don't have column 
>  representations 
> >R  themselves; they are a wrapper around columns provided by other 
> fields. 
>
> But ImageField does have its own column representation. In fact, it is not 
> a 
> wrapper around the width and height fields in the way that, say, 
> GenericForeignKey is around the content_type and object_id fields -- if 
> you 
> change these fields, nothing about the image changes. So, yes, I maintain 
> that 
> it is a different beast.
>
 
That's correct! sorry about that, I am looking at the implementation of 
ImageField and it looks like the width_field and height_field are optional 
(is that correct?).

>R Pure" virtual fields - Fields that are conceptual wrappers around other 
fields on the same model. Virtual fields don't have column representations 
themselves; they are a wrapper around columns provided by other fields.

ImageField does have a column representation of it's own and It requires 
migrations, so this means that it looks nearer to a data field, but where 
to put it? As you say correctly, it's a beast of it's own.
Another solution could be to refactor ImageField to make it 100% virtual 
compatible, but if we do this it also makes sense to refactor a lot of 
other ambiguities in the codebase.
 

>
> > 
> > On Monday, August 18, 2014 6:50:04 AM UTC+2, Russell Keith-Magee wrote: 
> > > 
> > > I agree that conceptually, part of the data for a FileField/ImageField 
> is 
> > > held "externally"; but it's a different kind of external. From the 
> > > database's perspective, the record is complete and correct when you're 
> > > storing a string. The fact that the string represents a file system 
> path 
> > > is a very significant implementation detail - after all, you need to 
> > > know to show a file browsing dialog (or whatever UI you want) - but 
> > > then, the same is true of a date field needing a date picker, and a 
> > > boolean field needing a checkbox. It doesn't affect the way the 
> database 
> > > needs to interact with the data it is storing. 
> > > 
>
> As I said above, I think focusing on the database here is a red herring. 
> This 
> is a model-level API, and we should be focusing on the interactions 
> between 
> models and fields, and between them and other code. 
>
> In that spirit, I think some more relevant categories might be 
>
> 1) "hidden fields" (the parent_ptr?), 
> 2) "read-only fields" (currently I suspect this only applies to DateTime 
> fields 
> with auto_now=True, but I can imagine calculated aggregation fields), 
> 3) "fields you shouldn't mess with" (id, parent_ptr, image's width_field 
> and 
> height_field -- you can edit these, but in all probability you don't want 
> to), 
> 4)"fields you should only edit via API" (passwords) 
>
> HTH, 
> Shai. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c13c20f1-55b6-47c4-841b-f550abeea1a5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: GSoC Meta refactor: Bikeshedding time!!

2014-08-18 Thread Daniel Pyrathon
Hi Colin,

Thanks for getting back

On Monday, August 18, 2014 4:58:20 PM UTC+2, Collin Anderson wrote:
>
> The goal is to have "API methods that let you introspect the fields and 
> relations that exist on a model", right? Why go though the trouble of 
> finding the one specific type for each field (that we'll never be able to 
> change later)? Why have a get_fields() method with an ever-growing number 
> of kwargs?
>
> I want all "related" fields. Easy:
> (f for f in _meta.fields if hasattr(f, 'rel'))
>

> I want all read-only fields. Easy:
> (f for f in _meta.fields if not f.editable)
>
> I want all fields that can be edited through a form. Something like:
> (f for f in _meta.fields if hasattr(f, 'formfield'))
>
> I want all "local" fields (not that you should care). Easy:
> (f for f in _meta.fields if f.model == _meta.model)
>
> I want all fields that have an actual column in the database. Something 
> like
> (f for f in _meta.fields if f.db_type())
>
> I want all fields that function like a ManyToMany. Easy:
> (f for f in _meta.fields if f.get_internal_type() == 'ManyToManyField')
>
> I want all fields that have ForeignKey.to_field pointing to them. 
> Something like:
> set(_meta.get_field(fname) for rel in _meta.related for fname in 
> rel.to_fields)
>

Ha! I wish it was so easy :( unfortunately there are quite a bit of edge 
cases that need to be take into consideration. For example:
field.rel: many fields have a rel, but point to None. Also there are cases 
when rel points to a swapped model. In this case we need to do some more 
work.
ManyToMany: do we really want to hard-code this? or do we want to have an 
internal component handle it for us. What happens when NoSQL comes to 
Django (I really believe in this..).
get_fields() also gives us the possibility of changing internals without 
necessarily affecting behaviour. It's that layer of abstraction that I feel 
is needed.
Said this, if we can simplify it and still avoid duplication and provide 
the level of functionality it's giving us now, then that would be great.
 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/12ee8db0-dd85-4410-b402-ce9e343be744%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: GSoC Meta refactor: Bikeshedding time!!

2014-08-21 Thread Daniel Pyrathon
Hi Shai,

Thanks for the comments!

On Wednesday, August 20, 2014 8:21:36 PM UTC+2, Shai Berger wrote:
>
> On Wednesday 20 August 2014 10:29:49 Anssi Kääriäinen wrote: 
> > On Wednesday, August 20, 2014 11:19:33 AM UTC+3, Russell Keith-Magee 
> wrote: 
> > > 
> > > This yields the following formal API for _meta: 
> > >  * get_fields(data, many_to_many, related, include_hidden, 
> > > include_parents) 
> > >   
> > >  * @property data_fields 
> > > 
> > >  * @property many_to_many_fields 
> > > 
> > >  * @property related_objects 
> > >  (=> get_fields(data=False, many_to_many=False, related=True, 
> > >  include_hidden=False, include_parents=True) 
> > > 
> > +1 if we also consider defining and documenting an useful set of field 
> flags. 
>
> +1 Anssi. 
>
> > 
> > I wonder if a better name for the related category exists. My first 
> > instinct is that foreign key fields should match the related flag. Could 
> it 
> > be made cleaner that these are relations defined on remote model? Maybe 
> > just remote_relations could work? 
> > 
>
> Since this is a bikeshedding thread, I'll say that the name 
> "related_objects" 
> makes me itch a little as well -- mostly, because each of the objects 
> returned 
> by the property is not a related-object, but rather a manager of related- 
> objects.
>

the related_objects property should return objects of instance 
RelatedObject. What do you intend by manager
of related objects?
 

>
> I was considering "related_managers", but that is sort-of mixing the 
> "what" 
> with the "how", and also thinking about the possible Array(FK) fields, I 
> prefer 
> "related_collections". 
>

what about straight up "*relations"*?

get_fields(data, m2m, relations)
 

>
> Shai. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/e104c3a1-3ee4-41e7-9172-79a2073c74e7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: The greatest proposal yet: rename this damn group

2014-09-09 Thread Daniel Pyrathon
Hi,

I think that changing the name the community on google will create many
broken links, this will be a huge loss.
What we can do instead, is create 2 domains (or subdomains) such as
core-development.djangoproject.com and community.djangoproject.com that
will serve as the official URLs to publicise on blogs, official sites and
IRC. These ULRs will redirect to the related google forums.

Dan

On Tuesday, September 9, 2014, Erik Romijn  wrote:

> I think it would also be a great improvement if we all adopted a standard
> response for these kind of mails - because no matter what we do, some will
> still end up here.
>
> Almost entirely based on Daniele's previous responses, how about we use:
>
> > The best place to get answers to your questions is the django-users
> email list, > - the web
> interface is .
> >
> > The list you've posted to is django-developers, an email list is for the
> discussion of the development of Django itself.
> >
> > You might also find helpful the #django IRC channel on irc.freenode.net.
> >
> > I hope that helps,
>
> This focuses first on helping them get to the right place, with easily
> readable language, and then explains their error in a friendly way. In the
> past, we've occasionally sent somewhat more harsh replies, focusing more on
> how they did something wrong. Although I'm sure such replies were
> absolutely sent with the best intentions, it's not a pleasant first
> experience.
>
> Not sure what the best place is to keep this template easily accessible
> for anyone though. The wiki might be the most suitable.
>
> Erik
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com .
> To post to this group, send email to django-developers@googlegroups.com
> .
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/07EC2CEB-A0AC-4624-874C-6D8FBD9D594C%40solidlinks.nl
> .
> For more options, visit https://groups.google.com/d/optout.
>


-- 
*

PirosB3

https://github.com/PirosB3

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAOmo0o-g-aWLFpDu8T1Wmc5AaUbwoGzhuumbyO-boxdVV0_QTA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.