Re: [Tutor] Most efficient way to replace ", " with "." in a array and/or dataframe
On 22 Sep 2019 04:27, Cameron Simpson wrote:
On 21Sep2019 20:42, Markos wrote:
>I have a table.csv file with the following structure:
>
>, Polyarene conc ,, mg L-1 ,,,
>Spectrum, Py, Ace, Anth,
>1, "0,456", "0,120", "0,168"
>2, "0,456", "0,040", "0,280"
>3, "0,152", "0,200", "0,280"
>
>I open as dataframe with the command:
>data = pd.read_csv ('table.csv', sep = ',', skiprows = 1)
[...]
>And the data_array variable gets the fields in string format:
>[['0,456' '0,120' '0,168']
[...]
>Please see the documentation for the >read_csv function here:
> https://pandas.pydata.org/pandas
>docs/stable/reference/api/pandas.read_cs>
>v.html?highlight=read_csv#pandas.read_csv
Do you think it's a deliberate design choice that decimal and thousands where
used here as params, and not a 'locale' param? It seems nice to be able to
specify e.g. locale='dutch' and then all the right lc_numeric, lc_monetary,
lc_time where used. Or even locale='nl_NL.1252' and you also wouldn't need
'encoding' as a separate param. Or might that be bad on windows where there's
no locale-gen? Just wondering...
Albert-Jan
--
https://mail.python.org/mailman/listinfo/python-list
Re: [Tutor] Most efficient way to replace ", " with "." in a array and/or dataframe
On 22Sep2019 07:39, Albert-Jan Roskam wrote:
On 22 Sep 2019 04:27, Cameron Simpson wrote:
On 21Sep2019 20:42, Markos wrote:
I have a table.csv file with the following structure:
, Polyarene conc ,, mg L-1 ,,,
Spectrum, Py, Ace, Anth,
1, "0,456", "0,120", "0,168"
2, "0,456", "0,040", "0,280"
3, "0,152", "0,200", "0,280"
I open as dataframe with the command:
data = pd.read_csv ('table.csv', sep = ',', skiprows = 1)
[...]
And the data_array variable gets the fields in string format:
[['0,456' '0,120' '0,168']
[...]
Please see the documentation for the >read_csv function here:
https://pandas.pydata.org/pandas
docs/stable/reference/api/pandas.read_cs>
v.html?highlight=read_csv#pandas.read_csv
Do you think it's a deliberate design choice that decimal and thousands
where used here as params, and not a 'locale' param? It seems nice to
be able to specify e.g. locale='dutch' and then all the right
lc_numeric, lc_monetary, lc_time where used. Or even
locale='nl_NL.1252' and you also wouldn't need 'encoding' as a separate
param. Or might that be bad on windows where there's no locale-gen?
Just wondering...
Locales are tricky; I don't know enough.
A locale parameter might be convenient for some things, but such things
are table driven. From an arbitrary Linux box nearby:
% locale -a
C
C.UTF-8
POSIX
en_AU.utf8
No "dutch" or similar there.
I doubt pandas would ship with such a thing. And the OP probably doesn't
know the originating locale anyway. Nor do _we_ know that those values
themselves were driven from some well known locale table.
The advantage of specifical decimal= and thousands= parameters is that
they do exactly what they say, rather than looking up a locale and
hoping for a specific side effect. So the specific parameters offer
better control.
The thousands= itself is a little parachial (for example, in India a
factor of 100 is a common division point[1]), but it may merely be used
to strip this character from the left portion of the number.
[1] https://en.wikipedia.org/wiki/Indian_numbering_system
So while I am not a pandas person, I would expect that decimal= and
thousands= are useful parameters for specific lexical situations (like
the OP's CSV data) and work regardless of any locale knowledge.
Cheers,
Cameron Simpson
--
https://mail.python.org/mailman/listinfo/python-list
Re: [Tutor] Most efficient way to replace ", " with "." in a array and/or dataframe
On 9/22/19, Albert-Jan Roskam wrote: > > Do you think it's a deliberate design choice that decimal and thousands > where used here as params, and not a 'locale' param? It seems nice to be > able to specify e.g. locale='dutch' and then all the right lc_numeric, > lc_monetary, lc_time where used. Or even locale='nl_NL.1252' and you also > wouldn't need 'encoding' as a separate param. Or might that be bad on > windows where there's no locale-gen? Just wondering... FYI, while Windows is distributed with many locales and also supports custom locales (not something I've had to work with), at least based on standard locale data, "nl_NL.1252" is not a valid locale for use with C setlocale in Windows. Classically, the C runtime in Windows supports "C" (but not "POSIX") and locales based on a language name or its non-standard three-letter abbreviation. A locale can also include a country/region name (full or three-letter abbreviation), plus an optional codepage. If the latter is omitted, it defaults to the ANSI codepage of the language, or of the system locale if the language has no ANSI codepage . Examples: >>> locale.setlocale(0, 'dutch') 'Dutch_Netherlands.1252' >>> locale.setlocale(0, 'nld') 'Dutch_Netherlands.1252' >>> locale.setlocale(0, 'nld_NLD.850') 'Dutch_Netherlands.850' There are also a few compatibility locales such as "american" and "canadian": >>> locale.setlocale(0, 'american') 'English_United States.1252' >>> locale.setlocale(0, 'canadian') 'English_Canada.1252' Classically, the Windows API represents locales not as language/region strings but as numeric locale identifiers (LCIDs). However, back in 2006, Windows Vista introduced locale names, plus a new set of functions that use locale names instead of LCIDs (e.g. GetLocaleInfoEx). Locale names are based on BCP-47 language tags, which include at least an ISO 639 language code. They can also include an optional ISO 15924 script code (e.g. "Latn" or "Cyrl") and an optional ISO 3166-1 region code. Strictly speaking, the codes in a BCP-47 language tag are delimited only by hyphens, but newer versions of Windows in most cases also allow underscore. The Universal CRT (used by Python 3.5+) supports Vista locale names. Recently it also supports using underscore instead of hyphen, plus an optional ".utf8" or ".utf-8" encoding. Only UTF-8 can be specified for a BCP-47 locale. If the encoding is not specified, BCP-47 locales use the language's ANSI codepage, or UTF-8 if the language has no ANSI codepage (e.g. "hi_IN"). Examples: >>> locale.setlocale(0, 'nl_NL') 'nl_NL' >>> locale.setlocale(0, 'nl_NL.utf8') 'nl_NL.utf8' Older versions of the Universal CRT do not support UTF-8 or underscore in BCP-47 locale names, so make sure a system is updated if you need these features. -- https://mail.python.org/mailman/listinfo/python-list
SQLObject 3.7.3
Hello!
I'm pleased to announce version 3.7.3, a bugfix release of branch
3.7 of SQLObject.
What's new in SQLObject
===
Bug fixes
-
* Avoid excessive parentheses around ``ALL/ANY/SOME()``.
Tests
-
* Add tests for cascade deletion.
* Add tests for ``sqlbuilder.ALL/ANY/SOME()``.
* Fix calls to ``pytest.mark.skipif`` - make conditions bool instead of str.
* Fix module-level calls to ``pytest.mark.skip`` - add reasons.
* Fix escape sequences ``'\%'`` -> ``'\\%'``.
CI
--
* Reduce the number of virtual machines/containers:
one OS, one DB, one python version, many drivers per VM.
* Fix sqlite test under Python 3.7+ at AppVeyor.
Contributors for this release are
For a more complete list, please see the news:
http://sqlobject.org/News.html
What is SQLObject
=
SQLObject is an object-relational mapper. Your database tables are described
as classes, and rows are instances of those classes. SQLObject is meant to be
easy to use and quick to get started with.
SQLObject supports a number of backends: MySQL, PostgreSQL, SQLite,
Firebird, Sybase, MSSQL and MaxDB (also known as SAPDB).
Python 2.7 or 3.4+ is required.
Where is SQLObject
==
Site:
http://sqlobject.org
Development:
http://sqlobject.org/devel/
Mailing list:
https://lists.sourceforge.net/mailman/listinfo/sqlobject-discuss
Download:
https://pypi.org/project/SQLObject/3.7.3
News and changes:
http://sqlobject.org/News.html
StackOverflow:
https://stackoverflow.com/questions/tagged/sqlobject
Example
===
Create a simple class that wraps a table::
>>> from sqlobject import *
>>>
>>> sqlhub.processConnection = connectionForURI('sqlite:/:memory:')
>>>
>>> class Person(SQLObject):
... fname = StringCol()
... mi = StringCol(length=1, default=None)
... lname = StringCol()
...
>>> Person.createTable()
Use the object::
>>> p = Person(fname="John", lname="Doe")
>>> p
>>> p.fname
'John'
>>> p.mi = 'Q'
>>> p2 = Person.get(1)
>>> p2
>>> p is p2
True
Queries::
>>> p3 = Person.selectBy(lname="Doe")[0]
>>> p3
>>> pc = Person.select(Person.q.lname=="Doe").count()
>>> pc
1
Oleg.
--
Oleg Broytmanhttps://phdru.name/[email protected]
Programmers don't die, they just GOSUB without RETURN.
--
https://mail.python.org/mailman/listinfo/python-list
Obtain all the content from https://free-ss.site/ by pycurl/requests.
Hi, I try to obtain the content from https://free-ss.site/ by pycurl/requests. But it seems that the above tools cannot get all of the content as the stuff obtained by the firefox. Is is possible to get all of the content just as the results via firefox for this website by using pycurl/requests? Regards -- https://mail.python.org/mailman/listinfo/python-list
Cheetah 3.2.4
Hello!
I'm pleased to announce version 3.2.4, a bugfix release of branch
3.2 of CheetahTemplate3.
What's new in CheetahTemplate3
==
Minor features:
- Import from ``collections`` for Python 2,
from ``collections.abc`` for Python 3.
Bug fixes:
- Fixed infinite recursion in ``ImportManager`` on importing
module ``_bootlocale`` inside ``open()``.
What is CheetahTemplate3
Cheetah3 is a free and open source template engine.
It's a fork of the original CheetahTemplate library.
Python 2.7 or 3.4+ is required.
Where is CheetahTemplate3
=
Site:
https://cheetahtemplate.org/
Development:
https://github.com/CheetahTemplate3
Download:
https://pypi.org/project/Cheetah3/3.2.4
News and changes:
https://cheetahtemplate.org/news.html
StackOverflow:
https://stackoverflow.com/questions/tagged/cheetah
Example
===
Below is a simple example of some Cheetah code, as you can see it's practically
Python. You can import, inherit and define methods just like in a regular Python
module, since that's what your Cheetah templates are compiled to :) ::
#from Cheetah.Template import Template
#extends Template
#set $people = [{'name' : 'Tom', 'mood' : 'Happy'}, {'name' : 'Dick',
'mood' : 'Sad'}, {'name' : 'Harry', 'mood' :
'Hairy'}]
How are you feeling?
#for $person in $people
$person['name'] is $person['mood']
#end for
Oleg.
--
Oleg Broytmanhttps://phdru.name/[email protected]
Programmers don't die, they just GOSUB without RETURN.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Most efficient way to replace ", " with "." in a array and/or dataframe
Markos writes:
> Hi,
>
> I have a table.csv file with the following structure:
>
> , Polyarene conc ,, mg L-1 ,,,
> Spectrum, Py, Ace, Anth,
> 1, "0,456", "0,120", "0,168"
> 2, "0,456", "0,040", "0,280"
> 3, "0,152", "0,200", "0,280"
>
> I open as dataframe with the command:
>
> data = pd.read_csv ('table.csv', sep = ',', skiprows = 1)
>
[snip]
> Also I'm also wondering if there would be any benefit of making this
> modification in dataframe before extracting the numeric fields to the
> array.
>
> Please, any comments or tip?
data = pd.read_csv ('table.csv', sep = ',', skiprows = 1, decimal=b',',
skipinitialspace=True)
--
Piet van Oostrum
WWW: http://piet.vanoostrum.org/
PGP key: [8DAE142BE17999C4]
--
https://mail.python.org/mailman/listinfo/python-list
Re: Most efficient way to replace ", " with "." in a array and/or dataframe
Em 22-09-2019 13:10, Piet van Oostrum escreveu:
Markos writes:
Hi,
I have a table.csv file with the following structure:
, Polyarene conc ,, mg L-1 ,,,
Spectrum, Py, Ace, Anth,
1, "0,456", "0,120", "0,168"
2, "0,456", "0,040", "0,280"
3, "0,152", "0,200", "0,280"
I open as dataframe with the command:
data = pd.read_csv ('table.csv', sep = ',', skiprows = 1)
[snip]
Also I'm also wondering if there would be any benefit of making this
modification in dataframe before extracting the numeric fields to the
array.
Please, any comments or tip?
data = pd.read_csv ('table.csv', sep = ',', skiprows = 1, decimal=b',',
skipinitialspace=True)
Thank you for the tip.
I didn't realize that I could avoid formatting problems in the dataframe
or array simply by using the read_csv command with the correct
parameters (sep and decimal).
I searched for information about the meaning of the letter "b" in the
parameter decimal=b',' but didn't find.
I found that it also works without the letter b.
Best Regards,
Markos
--
https://mail.python.org/mailman/listinfo/python-list
Re: Most efficient way to replace ", " with "." in a array and/or dataframe
On 9/22/19 7:05 PM, Markos wrote:
>
> Em 22-09-2019 13:10, Piet van Oostrum escreveu:
>> Markos writes:
>>
>>> Hi,
>>>
>>> I have a table.csv file with the following structure:
>>>
>>> , Polyarene conc ,, mg L-1 ,,,
>>> Spectrum, Py, Ace, Anth,
>>> 1, "0,456", "0,120", "0,168"
>>> 2, "0,456", "0,040", "0,280"
>>> 3, "0,152", "0,200", "0,280"
>>>
>>> I open as dataframe with the command:
>>>
>>> data = pd.read_csv ('table.csv', sep = ',', skiprows = 1)
>>>
>> [snip]
>>
>>> Also I'm also wondering if there would be any benefit of making this
>>> modification in dataframe before extracting the numeric fields to the
>>> array.
>>>
>>> Please, any comments or tip?
>> data = pd.read_csv ('table.csv', sep = ',', skiprows = 1,
>> decimal=b',', skipinitialspace=True)
>>
> Thank you for the tip.
>
> I didn't realize that I could avoid formatting problems in the
> dataframe or array simply by using the read_csv command with the
> correct parameters (sep and decimal).
>
> I searched for information about the meaning of the letter "b" in the
> parameter decimal=b',' but didn't find.
>
> I found that it also works without the letter b.
>
> Best Regards,
> Markos
The b indicates that the string is a 'bytes' string vs a text string.
This isn't so important to differentiate for ASCII characters, but can
make a difference if you have extended characters and the file might not
be in Unicode.
--
Richard Damon
--
https://mail.python.org/mailman/listinfo/python-list
Re: Most efficient way to replace ", " with "." in a array and/or dataframe
Em 22-09-2019 13:10, Piet van Oostrum escreveu:
Markos writes:
Hi,
I have a table.csv file with the following structure:
, Polyarene conc ,, mg L-1 ,,,
Spectrum, Py, Ace, Anth,
1, "0,456", "0,120", "0,168"
2, "0,456", "0,040", "0,280"
3, "0,152", "0,200", "0,280"
I open as dataframe with the command:
data = pd.read_csv ('table.csv', sep = ',', skiprows = 1)
[snip]
Also I'm also wondering if there would be any benefit of making this
modification in dataframe before extracting the numeric fields to the
array.
Please, any comments or tip?
data = pd.read_csv ('table.csv', sep = ',', skiprows = 1, decimal=b',',
skipinitialspace=True)
Thank you very much for all the contributions.
I didn't realize that I could avoid formatting problems in the dataframe
or array simply by using the read_csv command with the correct
parameters (sep and decimal).
I searched for information about the meaning of the letter b in the
parameter but did not find it.
I found that it also works without the letter b.
Best Regards,
Markos
--
https://mail.python.org/mailman/listinfo/python-list
Remote Work Wanted - Junior Python Developer
Hi, I would like to start working remotely programming in Python. I have an IT Degree and English is my first Language. I currently live in New Zealand, thus the remote work is the only option. I have 15+ years experience as a Delphi Developer, mainly working with business database applications. I have worked with Oracle, MySQL, SQL Server, Interbase, Firebird and many other database backends as well. I would like to join a team of developers that could help my crossover to Python. I think I can hit the ground running if pointed in the right direction and helped over any time consuming hurdles. As far as money goes, I am open to reasonable offers. cheers Shane -- https://mail.python.org/mailman/listinfo/python-list
Re: Remote Work Wanted - Junior Python Developer
On Mon, Sep 23, 2019 at 10:56 AM Shane Thomas wrote: > > Hi, > > I would like to start working remotely programming in Python. > > I have an IT Degree and English is my first Language. I currently live in New > Zealand, thus the remote work is the only option. > > I have 15+ years experience as a Delphi Developer, mainly working with > business database applications. I have worked with Oracle, MySQL, SQL Server, > Interbase, Firebird and many other database backends as well. > > I would like to join a team of developers that could help my crossover to > Python. > > I think I can hit the ground running if pointed in the right direction and > helped over any time consuming hurdles. > > As far as money goes, I am open to reasonable offers. > Cool! This list/newsgroup isn't a job board, but there are a number of good places to go looking. I'd start here, although IME remote jobs are a fairly small proportion of the listings: https://www.python.org/jobs/ Another solid place to look is Stack Overflow Jobs: https://stackoverflow.com/jobs I've no idea how many of the linked sites are of value, but there's a page on the Python Wiki specifically for sites with job postings about Python: https://wiki.python.org/moin/PythonJobs Hopefully that'll help! ChrisA -- https://mail.python.org/mailman/listinfo/python-list
