Re: Debugging technique

2020-10-05 Thread J. Pic 
Another nice debugger feature is to step up with "u", this will take
you to the parent frame where you can again inspect the variables.

I use this when I want to reverse engineer how the interpreter got to
a specific line.

Maybe worth mentioning that Werkzeug provides in-browser interactive
debuggers at each frame of a traceback, that's very nice.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: ValueError: arrays must all be same length

2020-10-05 Thread Shaozhong SHI 
Hi, I managed to flatten it with json_normalize first.

from pandas.io.json import json_normalize
atable = json_normalize(d)
atable

Then, I got this table.

brandId brandName careHome constituency
currentRatings.overall.keyQuestionRatings currentRatings.overall.rating
currentRatings.overall.reportDate currentRatings.overall.reportLinkId
currentRatings.reportDate dormancy ... providerId region registrationDate
registrationStatus regulatedActivities relationships reports specialisms
type uprn
0 BD510 BRAND MACC Care Y Birmingham, Northfield [{u'reportDate':
u'2020-10-01', u'rating': u'R... Requires improvement 2020-10-01
1157c975-c2f1-423e-a2b4-66901779e014 2020-10-01 N ... 1-101641521 West
Midlands 2013-12-16 Registered [{u'code': u'RA2', u'name': u'Accommodation

Then, I tried to expand the column
of currentRatings.overall.keyQuestionRatings, with

mydf =
pd.DataFrame.from_dict(atable['currentRatings.overall.keyQuestionRatings'][0])
mydf

Then, I got another table.

name rating reportDate reportLinkId
0 Safe Requires improvement 2020-10-01 1157c975-c2f1-423e-a2b4-66901779e014
1 Well-led Requires improvement 2020-10-01
1157c975-c2f1-423e-a2b4-66901779e014
2 Caring Good 2019-10-04 63ff05ec-4d31-406e-83de-49a271cfdc43
3 Responsive Good 2019-10-04 63ff05ec-4d31-406e-83de-49a271cfdc43
4 Effective Requires improvement 2019-10-04
63ff05ec-4d31-406e-83de-49a271cfdc43


How can I re-arrange to get a flatten table?

Apparently, the nested data is another table.

Regards,

Shao



On Sun, 4 Oct 2020 at 13:55, Tim Williams  wrote:

> On Sun, Oct 4, 2020 at 8:39 AM Tim Williams  wrote:
>
> >
> >
> > On Fri, Oct 2, 2020 at 11:00 AM Shaozhong SHI 
> > wrote:
> >
> >> Hello,
> >>
> >> I got a json response from an API and tried to use pandas to put data
> into
> >> a dataframe.
> >>
> >> However, I kept getting this ValueError: arrays must all be same length.
> >>
> >> Can anyone help?
> >>
> >> The following is the json text.  Regards, Shao
> >>
> >> (snip json_text)
> >
> >
> >> import pandas as pd
> >>
> >> import json
> >>
> >> j = json.JSONDecoder().decode(req.text)  ###req.json
> >>
> >> df = pd.DataFrame.from_dict(j)
> >>
> >
> > I copied json_text into a Jupyter notebook and got the same error trying
> > to convert this into a pandas DataFrame:When I tried to copy this into a
> > string, I got an error,, but without enclosing the paste in quotes, I got
> > the dictionary.
> >
> >
> (delete long response output)
>
>
> > for k in json_text.keys():
> > if isinstance(json_text[k], list):
> > print(k, len(json_text[k]))
> >
> > relationships 0
> > locationTypes 0
> > regulatedActivities 2
> > gacServiceTypes 1
> > inspectionCategories 1
> > specialisms 4
> > inspectionAreas 0
> > historicRatings 4
> > reports 5
> >
> > HTH,.
> >
> >
> This may also be more of a pandas issue.
>
> json.loads(json.dumps(json_text))
>
> has a successful round-trip
>
>
> > --
> >> https://mail.python.org/mailman/listinfo/python-list
> >>
> >
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: ValueError: arrays must all be same length

2020-10-05 Thread Tim Williams 
On Mon, Oct 5, 2020 at 6:47 AM Shaozhong SHI  wrote:

>
> Hi, I managed to flatten it with json_normalize first.
>
> from pandas.io.json import json_normalize
> atable = json_normalize(d)
> atable
>
> Then, I got this table.
>
> brandId brandName careHome constituency
> currentRatings.overall.keyQuestionRatings currentRatings.overall.rating
> currentRatings.overall.reportDate currentRatings.overall.reportLinkId
> currentRatings.reportDate dormancy ... providerId region registrationDate
> registrationStatus regulatedActivities relationships reports specialisms
> type uprn
> 0 BD510 BRAND MACC Care Y Birmingham, Northfield [{u'reportDate':
> u'2020-10-01', u'rating': u'R... Requires improvement 2020-10-01
> 1157c975-c2f1-423e-a2b4-66901779e014 2020-10-01 N ... 1-101641521 West
> Midlands 2013-12-16 Registered [{u'code': u'RA2', u'name':
> u'Accommodation
>
> Then, I tried to expand the column
> of currentRatings.overall.keyQuestionRatings, with
>
> mydf =
> pd.DataFrame.from_dict(atable['currentRatings.overall.keyQuestionRatings'][0])
> mydf
>
> Then, I got another table.
>
> name rating reportDate reportLinkId
> 0 Safe Requires improvement 2020-10-01
> 1157c975-c2f1-423e-a2b4-66901779e014
> 1 Well-led Requires improvement 2020-10-01
> 1157c975-c2f1-423e-a2b4-66901779e014
> 2 Caring Good 2019-10-04 63ff05ec-4d31-406e-83de-49a271cfdc43
> 3 Responsive Good 2019-10-04 63ff05ec-4d31-406e-83de-49a271cfdc43
> 4 Effective Requires improvement 2019-10-04
> 63ff05ec-4d31-406e-83de-49a271cfdc43
>
>
> How can I re-arrange to get a flatten table?
>
> Apparently, the nested data is another table.
>
> Regards,
>
> Shao
>
>
> I'm fairly new to pandas myself. Can't help there. You may want to post
this on Stackoverflow, or look for a similar issue on github.

https://stackoverflow.com/questions/tagged/pandas+json
https://github.com/pandas-dev/pandas/issues




>
> On Sun, 4 Oct 2020 at 13:55, Tim Williams  wrote:
>
>> On Sun, Oct 4, 2020 at 8:39 AM Tim Williams  wrote:
>>
>> >
>> >
>> > On Fri, Oct 2, 2020 at 11:00 AM Shaozhong SHI 
>> > wrote:
>> >
>> >> Hello,
>> >>
>> >> I got a json response from an API and tried to use pandas to put data
>> into
>> >> a dataframe.
>> >>
>> >> However, I kept getting this ValueError: arrays must all be same
>> length.
>> >>
>> >> Can anyone help?
>> >>
>> >> The following is the json text.  Regards, Shao
>> >>
>> >> (snip json_text)
>> >
>> >
>> >> import pandas as pd
>> >>
>> >> import json
>> >>
>> >> j = json.JSONDecoder().decode(req.text)  ###req.json
>> >>
>> >> df = pd.DataFrame.from_dict(j)
>> >>
>> >
>> > I copied json_text into a Jupyter notebook and got the same error trying
>> > to convert this into a pandas DataFrame:When I tried to copy this into a
>> > string, I got an error,, but without enclosing the paste in quotes, I
>> got
>> > the dictionary.
>> >
>> >
>> (delete long response output)
>>
>>
>> > for k in json_text.keys():
>> > if isinstance(json_text[k], list):
>> > print(k, len(json_text[k]))
>> >
>> > relationships 0
>> > locationTypes 0
>> > regulatedActivities 2
>> > gacServiceTypes 1
>> > inspectionCategories 1
>> > specialisms 4
>> > inspectionAreas 0
>> > historicRatings 4
>> > reports 5
>> >
>> > HTH,.
>> >
>> >
>> This may also be more of a pandas issue.
>>
>> json.loads(json.dumps(json_text))
>>
>> has a successful round-trip
>>
>>
>> > --
>> >> https://mail.python.org/mailman/listinfo/python-list
>> >>
>> >
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Thank you Larry Hastings!

2020-10-05 Thread Barry Warsaw 
They say being a Python Release Manager is a thankless job, so the Python 
Secret Underground (PSU), which emphatically does not exist, hereby officially 
doesn’t thank Larry for his years of diligent service as the Python 3.4 and 3.5 
release manager.

On the other hand, the Python Steering Council, Python Software Foundation, and 
worldwide Python community, all of which emphatically *do* exist, all extend 
our heartfelt thanks to Larry for his excellent stewardship of Python 3.4 and 
3.5!

Python 3.4 and 3.5 were both pivotal releases.  While the features of these two 
releases are too numerous to mention here, they introduced such staples as:

* asyncio
* enum
* pathlib
* async and await keywords
* matrix multiplication operators
* typing and zipapp modules

and so much more.  For details, see:

* https://docs.python.org/3/whatsnew/3.4.html
* https://docs.python.org/3/whatsnew/3.5.html

Larry’s first official release of 3.4.0a1 was on 2013-08-03 and his last Python 
3.5.10 release was 2020-09-05.  That’s 7 years of exemplary release managing!

Larry, from all of us, and from me personally, thank you so much for your 
invaluable contributions to Python.  Enjoy your retirement!

Cheers,
-Barry (on behalf of the PSC and PSF)



signature.asc
Description: Message signed with OpenPGP
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [python-committers] Thank you Larry Hastings!

2020-10-05 Thread Guido van Rossum 
Thank you Larry!

On Mon, Oct 5, 2020 at 11:39 AM Barry Warsaw  wrote:

> They say being a Python Release Manager is a thankless job, so the Python
> Secret Underground (PSU), which emphatically does not exist, hereby
> officially doesn’t thank Larry for his years of diligent service as the
> Python 3.4 and 3.5 release manager.
>
> On the other hand, the Python Steering Council, Python Software
> Foundation, and worldwide Python community, all of which emphatically *do*
> exist, all extend our heartfelt thanks to Larry for his excellent
> stewardship of Python 3.4 and 3.5!
>
> Python 3.4 and 3.5 were both pivotal releases.  While the features of
> these two releases are too numerous to mention here, they introduced such
> staples as:
>
> * asyncio
> * enum
> * pathlib
> * async and await keywords
> * matrix multiplication operators
> * typing and zipapp modules
>
> and so much more.  For details, see:
>
> * https://docs.python.org/3/whatsnew/3.4.html
> * https://docs.python.org/3/whatsnew/3.5.html
>
> Larry’s first official release of 3.4.0a1 was on 2013-08-03 and his last
> Python 3.5.10 release was 2020-09-05.  That’s 7 years of exemplary release
> managing!
>
> Larry, from all of us, and from me personally, thank you so much for your
> invaluable contributions to Python.  Enjoy your retirement!
>
> Cheers,
> -Barry (on behalf of the PSC and PSF)
>
> ___
> python-committers mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-committers.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/QGIHFU64TBYT56K6M5A5LYTYTSVFKHWQ/
> Code of Conduct: https://www.python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [python-committers] Thank you Larry Hastings!

2020-10-05 Thread Tal Einat 
On Mon, Oct 5, 2020 at 9:39 PM Barry Warsaw  wrote:
>
> They say being a Python Release Manager is a thankless job, so the Python 
> Secret Underground (PSU), which emphatically does not exist, hereby 
> officially doesn’t thank Larry for his years of diligent service as the 
> Python 3.4 and 3.5 release manager.
>
> On the other hand, the Python Steering Council, Python Software Foundation, 
> and worldwide Python community, all of which emphatically *do* exist, all 
> extend our heartfelt thanks to Larry for his excellent stewardship of Python 
> 3.4 and 3.5!
>
> Python 3.4 and 3.5 were both pivotal releases.  While the features of these 
> two releases are too numerous to mention here, they introduced such staples 
> as:
>
> * asyncio
> * enum
> * pathlib
> * async and await keywords
> * matrix multiplication operators
> * typing and zipapp modules
>
> and so much more.  For details, see:
>
> * https://docs.python.org/3/whatsnew/3.4.html
> * https://docs.python.org/3/whatsnew/3.5.html
>
> Larry’s first official release of 3.4.0a1 was on 2013-08-03 and his last 
> Python 3.5.10 release was 2020-09-05.  That’s 7 years of exemplary release 
> managing!
>
> Larry, from all of us, and from me personally, thank you so much for your 
> invaluable contributions to Python.  Enjoy your retirement!
>
> Cheers,
> -Barry (on behalf of the PSC and PSF)

These praises are certainly very well deserved!

You have my thanks as well, Larry.
- Tal Einat
-- 
https://mail.python.org/mailman/listinfo/python-list

[RELEASE] Python 3.9.0 is now available, and you can already test 3.10.0a1!

2020-10-05 Thread Łukasz Langa 
On behalf of the Python development community and the Python 3.9 release team, 
I’m pleased to announce the availability of Python 3.9.0.

Python 3.9.0 is the newest feature release of the Python language, and it 
contains many new features and optimizations. You can find Python 3.9.0 here:

https://www.python.org/downloads/release/python-390/ 


Most third-party distributors of Python should be making 3.9.0 packages 
available soon.

See the “What’s New in Python 3.9 
” document for more 
information about features included in the 3.9 series. Detailed information 
about all changes made in 3.9.0 can be found in its change log 
.

Maintenance releases for the 3.9 series will follow at regular bi-monthly 
intervals starting in late November of 2020.



OK, boring! Where is Python 4?

Not so fast! The next release after 3.9 will be 3.10. It will be an incremental 
improvement over 3.9, just as 3.9 was over 3.8, and so on.

In fact, our newest Release Manager, Pablo Galindo Salgado, prepared the first 
alpha release of what will become 3.10.0 a year from now. You can check it out 
here:

https://www.python.org/downloads/release/python-3100a1/ 


We hope you enjoy the new releases!

Thanks to all of the many volunteers who help make Python Development and these 
releases possible! Please consider supporting our efforts by volunteering 
yourself or through organization contributions to the Python Software 
Foundation.

https://www.python.org/psf/ 
More resources

Online Documentation 
PEP 596 , 3.9 Release Schedule
PEP 619 , 3.10 Release Schedule
Report bugs at https://bugs.python.org .
Help fund Python and its community .
Your friendly release team,
Ned Deily @nad 
Steve Dower @steve.dower 
Pablo Galindo Salgado @pablogsal 
Łukasz Langa @ambv 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [python-committers] Thank you Larry Hastings!

2020-10-05 Thread Łukasz Langa 

> On 5 Oct 2020, at 20:38, Barry Warsaw  wrote:
> 
> Larry, from all of us, and from me personally, thank you so much for your 
> invaluable contributions to Python.

Yes, definitely! Thank you.


> Enjoy your retirement!

Not so fast! Now you have all that extra free time to return to the Gilectomy! 🤓

- Ł

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [RELEASE] Python 3.9.0 is now available, and you can already test 3.10.0a1!

2020-10-05 Thread Terry Reedy 




OK, boring! Where is Python 4?

Not so fast! The next release after 3.9 will be 3.10. It will be an incremental 
improvement over 3.9, just as 3.9 was over 3.8, and so on.

In fact, our newest Release Manager, Pablo Galindo Salgado, prepared the first 
alpha release of what will become 3.10.0 a year from now. You can check it out 
here:

https://www.python.org/downloads/release/python-3100a1/ 



I am really happy to see this.
Partly because it is the first fully post-2.x release.
Partly because it is the first 3.xx release, breaking the '2 digit' 
minor version 'barrier' and mostly killing the spectre of an arbitrarily 
scheduled 4.0.


Even more, because it makes new features added since the 3.9 feature 
freeze late last May available to anyone.
It feels right to me that a bleeding-edge next-version preview release 
come out alone with the first production release of the new 'current' 
version.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list