Dataframe to postgresql - Saving the dataframe to memory using StringIO
I found this last option is very interesting. Saving the dataframe to memory using StringIO https://naysan.ca/2020/06/21/pandas-to-postgresql-using-psycopg2-copy_from/ But, testing shows unicode argument expected, got 'str' Any working example for getting DataFrame into a PostgreSQL table directly? Regards, David -- https://mail.python.org/mailman/listinfo/python-list
Re: Dataframe to postgresql - Saving the dataframe to memory using StringIO
Try to save it in a binary field on PG using hdf5: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_hdf.html On Thu, 22 Oct 2020 at 11:29, Shaozhong SHI wrote: > I found this last option is very interesting. > > Saving the dataframe to memory using StringIO > > https://naysan.ca/2020/06/21/pandas-to-postgresql-using-psycopg2-copy_from/ > > But, testing shows > unicode argument expected, got 'str' > > Any working example for getting DataFrame into a PostgreSQL table directly? > > Regards, > > David > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Static typing—with annotations—function name & arguments, or result of call, or string
From my understanding, `ast.arguments` and `inspect.Signature` are the
two builtin ways of defining the function name and arguments in a
structured way.
What I am creating is a way of configuring… well let's be specific to
my use-case. I am building a way to configure TensorFlow.
One which is very type-driven, and will error as early-as-possible
when incorrect types or lack of required parameters are provided.
I can dynamically infer things like so:
```
import inspect
import tensorflow as tf
sig = inspect.signature(tf.keras.optimizers.Adam)
tuple({
"default": sig.parameters[param].default,
"name": param,
"typ": type(sig.parameters[param].default).__name__
if sig.parameters[param].default is not inspect._empty
and sig.parameters[param].annotation is inspect._empty
else sig.parameters[param].annotation,
}
for param in sig.parameters if param != 'name'
)
```
I can also parse the docstring, as I do in my doctrans library and tool.
Which will give me the options I can provide the class. So there's an
obvious solution, to generate these classes:
```
class TensorFlowConfig(object):
optimizer: Optional[Optimizer] = None
# Or maybe a `Protocol`?
class Optimizer(object): pass
class AdamConfig(Optimizer):
learning_rate: float = 0.001
beta_1: float = 0.9
beta_2: float = 0.999
epsilon: float = 1e-07
amsgrad: bool = False
kwargs: dict = {}
TensorFlowConfig().optimizer = AdamConfig()
```
But, keeping in mind the goal to expose everything in all interfaces,
the question then becomes how to generate an argparse parser from
this. And how to generate a function from this. And how to ensure that
whatever code-completion interface one uses in Python, that it can
figure out what the correct parameters are.
So I should get an error about incorrect type when I:
```
# CLI
--optimizer 'Adam' 'learning_rate = True'
# class*
TensorFlowConfig().optimizer = tf.keras.optimizers.Adam(learning_rate=True)
TensorFlowConfig().optimizer = AdamConfig(learning_rate=True)
# Function*
MyTensorFlowClass.train(optimizer=tf.keras.optimizers.Adam(learning_rate=True))
MyTensorFlowClass.train(optimizer=AdamConfig(learning_rate=True))
* both of these next two lines—after the heading—should be equivalent
```
Syntax like this should also be valid
```
TensorFlowConfig().optimizer = 'tf.keras.optimizers.Adam'
TensorFlowConfig().optimizer = 'Adam'
MyTensorFlowClass.train(optimizer='tf.keras.optimizers.Adam')
MyTensorFlowClass.train(optimizer='Adam')
```
Do I have huge annotations like this*, with an `Any` lopped on for
further analysis down the line? - So it can easily generate into
`choices` for argparse? [which will need to be subclassed in order to
enable that "--optimizer 'Adam' 'learning_rate = True'" syntax]
*
https://github.com/SamuelMarks/ml-params-tensorflow/blob/1d48502/ml_params_tensorflow/ml_params/trainer.py#L213-L215
What makes sense?
Thanks for your insights,
Samuel Marks
Charity | consultancy
| open-source |
LinkedIn
PS: This is the `doctrans` project I referenced
https://github.com/SamuelMarks/doctrans
--
https://mail.python.org/mailman/listinfo/python-list
Re: Dataframe to postgresql - Saving the dataframe to memory using StringIO
I would add that usually I do not recommend saving files on databases. I usually save the file on the disk and the path and mime on a dedicated table. -- https://mail.python.org/mailman/listinfo/python-list
Re: Dataframe to postgresql - Saving the dataframe to memory using StringIO
On Thu, Oct 22, 2020 at 8:28 PM Shaozhong SHI wrote: > > I found this last option is very interesting. > > Saving the dataframe to memory using StringIO > > https://naysan.ca/2020/06/21/pandas-to-postgresql-using-psycopg2-copy_from/ > > But, testing shows > unicode argument expected, got 'str' > > Any working example for getting DataFrame into a PostgreSQL table directly? > That error suggests that you're using a legacy version of Python. The page you're linking to has been written on the assumption that you're using Python 3. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Dataframe to postgresql - Saving the dataframe to memory using StringIO
On 10/22/20 7:23 AM, Marco Sulla wrote: I would add that usually I do not recommend saving files on databases. I usually save the file on the disk and the path and mime on a dedicated table. I used to do that because backing up the database became huge. Now I use ZFS snapshots with send/receive so the backup only copies the changed blocks so I keep it in the database. That way the data is available from any client with database access without the need to give access to the server. PostgreSQL can handle it. -- D'Arcy J.M. Cain Vybe Networks Inc. A unit of Excelsior Solutions Corporation - Propelling Business Forward http://www.VybeNetworks.com/ IM:[email protected] VoIP: sip:[email protected] OpenPGP_signature Description: OpenPGP digital signature -- https://mail.python.org/mailman/listinfo/python-list
Re: Dataframe to postgresql - Saving the dataframe to memory using StringIO
Never worked with ZFS, it sounds interesting. Anyway, IMHO it's much more simple to save to disk also for debugging: you have not to extract the data from the db if you need to inspect them. On Thu, 22 Oct 2020 at 14:39, D'Arcy Cain wrote: > On 10/22/20 7:23 AM, Marco Sulla wrote: > > I would add that usually I do not recommend saving files on databases. I > > usually save the file on the disk and the path and mime on a dedicated > > table. > > I used to do that because backing up the database became huge. Now I use > ZFS snapshots with send/receive so the backup only copies the changed > blocks > so I keep it in the database. That way the data is available from any > client with database access without the need to give access to the server. > > PostgreSQL can handle it. > > -- > D'Arcy J.M. Cain > Vybe Networks Inc. > A unit of Excelsior Solutions Corporation - Propelling Business Forward > http://www.VybeNetworks.com/ > IM:[email protected] VoIP: sip:[email protected] > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Dataframe to postgresql - Saving the dataframe to memory using StringIO
On Fri, Oct 23, 2020 at 12:15 AM Shaozhong SHI wrote: > > Thanks, Chris. > > What should I know or watch out if I decide to move from Python 2.7 to Python > 3? > > What are the key issues? Syntax? > Keep it on-list please :) Key issues? Well, for starters, you don't have to worry about whether your strings are Unicode or not. They just are, unless you explicitly need them to be bytes. Syntax is probably going to be fine, since you're taking examples that were written for Python 3 anyway. There's a LOT more features in Python 3, since it's had another decade of development than Python 2 has had. Give it a try. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Question on ABC classes
Hello guys, I am professional programmer but quite new to Python, and I am trying to get the grips of some peculiarities of the language. Here is a basic question: if I define an ABC class, I can still instantiate the class unless there are abstract methods defined in the class. (In the typical OO language the class would be not instantiable, period, since it's "abstract". But this is not so in Python, to the point that, also for uniformity, I am feeling compelled to define an @abstractmethod __init__ in my ABC classes, whether they need one or not, and whether there are other abstract methods in the class or not.) Now, I do read in the docs that that is as intended, but I am not understanding the rationale of it: why only if there are abstract methods defined in an ABC class is instantiation disallowed? IOW, why isn't subclassing from ABC enough? Thanks for any enlightenment, Julio -- https://mail.python.org/mailman/listinfo/python-list
Re: Dataframe to postgresql - Saving the dataframe to memory using StringIO
On 2020-10-22, Chris Angelico wrote:
> On Fri, Oct 23, 2020 at 12:15 AM Shaozhong SHI wrote:
>> What should I know or watch out if I decide to move from Python 2.7
>> to Python 3?
>
> Key issues? Well, for starters, you don't have to worry about whether
> your strings are Unicode or not. They just are, unless you explicitly
> need them to be bytes.
The 'bytes' thing is important. If you use serial ports, sockets, or
anything else that's raw bytes, that code will need to be examined
carefully.
The usage of the 'bytes' type in 3.x isn't at all like 2.x:
Python2:
>>> bytes('abcd')
'abcd'
>>> bytes(3)
'3'
>>> bytes([0,1,2])
'[0, 1, 2]'
Python3:
>>> bytes('abcd')
Traceback (most recent call last):
File "", line 1, in
TypeError: string argument without an encoding
>>> bytes(3)
b'\x00\x00\x00'
>>> bytes([0,1,2])
b'\x00\x01\x02
Python2:
>>> b'abcd'[2]
'c'
Python3:
>>> b'abcd'[2]
99
Moving from 2.x to 3.x isn't too bad, but trying to maintain
compatiblity with both is painful. At this point, I would probably
just abandon 2.x.
--
Grant
--
https://mail.python.org/mailman/listinfo/python-list
Re: Dataframe to postgresql - Saving the dataframe to memory using StringIO
On Fri, Oct 23, 2020 at 3:35 AM Grant Edwards wrote: > Moving from 2.x to 3.x isn't too bad, but trying to maintain > compatiblity with both is painful. At this point, I would probably > just abandon 2.x. > Definitely. No point trying to support both when you're starting with code from a Py3 example. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
GUI (tkinter) popularity and job prospects for
I have been a rails developer as well as JS/react. I had wanted to look at python a bit due to it's popularity. I looked at tkinter which seems to have quite a few examples out there, but when I searched indeed.com for tkinter and wxpython it appeared that there was hardly any job listings mentioning those. Why is that ? It's a bit of a demotivating factor to get very serious with tk etc. I was going to look at something like tensorflow perhaps, though I am not sure if machine learning is that easy to pickup or not -- https://mail.python.org/mailman/listinfo/python-list
Re: Question on ABC classes
On Thu, 22 Oct 2020 at 18:31, Julio Di Egidio wrote: > why > only if there are abstract methods defined in an ABC > class is instantiation disallowed? > Not sure because I never tried or needed, but if no @abstractsomething in A is defined and your B class is a subclass of A, B should be an abstract class, not a concrete class. -- https://mail.python.org/mailman/listinfo/python-list
Re: Question on ABC classes
On Thu, 22 Oct 2020 at 22:09, Marco Sulla
wrote:
> Not sure because I never tried or needed, but if no @abstractsomething in
> A is defined and your B class is a subclass of A, B should be an abstract
> class, not a concrete class.
>
Now I'm sure:
>>> from abc import ABC, abstractmethod
>>> class A(ABC): pass
...
>>> class B(A):
... @abstractmethod
... def hello(self):
... print("hello")
...
>>> B()
Traceback (most recent call last):
File "", line 1, in
TypeError: Can't instantiate abstract class B with abstract methods hello
>
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: Question on ABC classes
On 10/22/20 9:25 AM, Julio Di Egidio wrote: Now, I do read in the docs that that is as intended, but I am not understanding the rationale of it: why only if there are abstract methods defined in an ABC class is instantiation disallowed? IOW, why isn't subclassing from ABC enough? Let's say you subclass from ABC: class Abstract(ABC): pass Then you subclass from that: class Concrete(Abstract): pass Then subclass from that: class MoreConcrete(Concrete): pass If you do a issubclass(, ABC) you'll get True The idea behind abstract classes is the prevention of creating non-functional instances, which means if any abstract methods, properties, etc., are present in an abstract class, then it's instances will not be fully functional; contrariwise, if there are no abstract anythings in the class, then it is functional and there's no reason not to allow it to be created. Put another way: if ABC is anywhere in a class' parentage, then it is "abstract" -- the only way to tell if instantiating it is okay is by the presence/absence of abstract pieces in the class. -- ~Ethan~ -- https://mail.python.org/mailman/listinfo/python-list
Re: GUI (tkinter) popularity and job prospects for
On Thu, 22 Oct 2020, Lammie Jonson wrote: I looked at tkinter which seems to have quite a few examples out there, but when I searched indeed.com for tkinter and wxpython it appeared that there was hardly any job listings mentioning those. Why is that ? It's a bit of a demotivating factor to get very serious with tk etc. I was going to look at something like tensorflow perhaps, though I am not sure if machine learning is that easy to pickup or not Lammie, I'm not a computer professional but I've worked with computers for 40+ years, from main frames to personal ones. I also run a solo practitioner consultancy after working for others so I know both sides of the job market. My advice is free so there's no obligation for you to consider it. You're looking for a job from the wrong end. Let me offer an analogy I use to describe how you ought to go about your job search. I've often said that you can teach someone how to use a word processor but that does not make them a writer. Your rails/JS experiences and your interest in tkinter or wxpython is focused on tools, not the products produced by those tools. If you wanted to hire a professional chef how would you respond to someone who told you they had great knife skills? Your interest is in what they could do with those skills to prepare outstanding meals for you. After all, it's the meals you're buying, not how uniform are their diced onions. An employer will hire those who will benefit the employer's company and position. The tools used to provide those benefits are less important than the prospective employee's skill in using them to solve a problem or increase company profits and prove how great a manager is for hiring you. Make a list of what you can do and what you've accomplished, not a list of tools you know. I've no idea what you really want to do but define that in terms of both your personal satisfaction and what an employer would value. And keep in mind the three steps in getting the job/career you want: 1. The cover letter's only purpose is to get the recipient to read the enclosed resume. 2. The resume's only purpose is to get you an interview with an ultimate decision-maker (not a personnel gate keeper). 3. The interview's purpose is to sell yourself as a benefit to the company and proof of how clever the interviewer is by hiring you. Good luck ... and stay well, Rich -- https://mail.python.org/mailman/listinfo/python-list
Re: GUI (tkinter) popularity and job prospects for
Hi Lammie, On Thu, Oct 22, 2020 at 10:03 PM Lammie Jonson wrote: > > I looked at tkinter which seems to have quite a few examples out there, > but when I searched indeed.com for tkinter and wxpython it appeared that > there was hardly any job listings mentioning those. Why is that ? > My guess would be that that is because we are after quite a few years of a world-wide, cross-industry move away from making desktop apps towards making web apps and mobile apps. To my knowledge, there are very few job listings looking for mastery of any desktop UI technologies, certainly relative to web-based UI technologies. (There are exceptions to this, of course.) I was going to look at something like tensorflow perhaps, though I am not > sure if machine learning is that easy to pickup or not > "Data science", using machine learning or otherwise, has become very popular and there are many jobs in most tech hubs for those with relevant skills. It requires a somewhat different set of skills, such as a solid working knowledge of statistics. Generally speaking and without knowing your specific circumstances, I think this would be a much safer bet in terms of employment options than learning desktop UI techs. It's not for everyone though; how hard it will be for you is hard to say, it's an individual thing. It's also a different type of work, so your motivation and interest in it may not be the same, for better or worse. If you're interested, there are plenty of learning resources out there; perhaps try an introductory "data science" course or book and see how it feels to you. - Tal -- https://mail.python.org/mailman/listinfo/python-list
Re: GUI (tkinter) popularity and job prospects for
On 10/22/20 12:58 PM, Lammie Jonson wrote: > > I have been a rails developer as well as JS/react. I had wanted to > look at python a bit due to it's popularity. > > I looked at tkinter which seems to have quite a few examples out > there, but when I searched indeed.com for tkinter and wxpython it > appeared that there was hardly any job listings mentioning those. Why > is that ? It's a bit of a demotivating factor to get very serious > with tk etc. I doubt you'll find any jobs connected a particular Python GUI toolkit. Except maybe at Red Hat. Most likely you'll find Python used in web app development, back-end server system programming, scripting, etc. Of course Python can be used with a variety of GUI toolkits including Tk, wx, GTK+, and Qt, to make desktop applications of course, but it's not commercially common and I wouldn't ever expect to see that on a job posting. > I was going to look at something like tensorflow perhaps, though I am > not sure if machine learning is that easy to pickup or not Not sure anything difficult and worthwhile, even if it is popular and in demand, is something you can just "[easily] pick up." But good luck to you. If you can find some fun side projects to do with with it while gaining experience, I'm sure that will look good on a resume. -- https://mail.python.org/mailman/listinfo/python-list
Re: GUI (tkinter) popularity and job prospects for
On Fri, Oct 23, 2020 at 8:39 AM Michael Torrie wrote: > > I was going to look at something like tensorflow perhaps, though I am > > not sure if machine learning is that easy to pickup or not > > Not sure anything difficult and worthwhile, even if it is popular and in > demand, is something you can just "[easily] pick up." But good luck to > you. If you can find some fun side projects to do with with it while > gaining experience, I'm sure that will look good on a resume. True, but not everything worthwhile is difficult. If a competent programmer, knowing 20 distinct programming languages, needs to learn a 21st for a project, it should be straight-forward to master it in a short time (at least to the point of being able to muffle through - doesn't mean s/he would know all the idioms of course). Learning ML is certainly a big task, but learning some specific subpart of ML would be doable, and then you learn another subpart, and another and another, until you're employable. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: GUI (tkinter) popularity and job prospects for
Thanks, Yes, I have some sense about how to do job interviews and market myself which is always an ongoing process. I also just have an interest in different technologies that I may want to investigate as I can get bored with certain things a little at times. If some technology seems a little interesting to me but I don't see that there are a lot of job listings for it however, then I sometimes wonder why that is etc -- https://mail.python.org/mailman/listinfo/python-list
Re: Question on ABC classes
On Thursday, 22 October 2020 23:04:25 UTC+2, Ethan Furman wrote: > On 10/22/20 9:25 AM, Julio Di Egidio wrote: > > > Now, I do read in the docs that that is as intended, > > but I am not understanding the rationale of it: why > > only if there are abstract methods defined in an ABC > > class is instantiation disallowed? IOW, why isn't > > subclassing from ABC enough? > > Let's say you subclass from ABC: > >class Abstract(ABC): >pass > > Then you subclass from that: > >class Concrete(Abstract): >pass > > Then subclass from that: > >class MoreConcrete(Concrete): >pass > > If you do a > >issubclass(, ABC) > > you'll get > >True Ah, right, that's the point I was missing: how to tell the compiler when a more derived class is *not* abstract... I was indeed making the mistake of inheriting from ABC also in the derived classes, and omitting it in the classes that are eventually concrete, not realising that ABC isn't just a keywork or a decorator, so it gets inherited all the way. > The idea behind abstract classes is the prevention of creating > non-functional instances Well, in Python, not in any other OO language, where abstract is just synonym with must-override (hence cannot instantiate), no other constraints. I am now thinking whether I could achieve the "standard" behaviour via another approach, say with decorators, somehow intercepting calls to __new__... maybe. Anyway, abstract classes are the gist of most library code, and I remain a bit puzzled by the behaviour implemented in Python: but I am not complaining, I know it will take me some time before I actually understand the language... For now, thank you and Marco very much for your feedback, Julio -- https://mail.python.org/mailman/listinfo/python-list
Debugging a memory leak
Dear Python gurus,
I'm a maintainer of a python library "datatable" (can be installed from
PyPi), and i've been recently trying to debug a memory leak that occurs in
my library.
The program that exposes the leak is quite simple:
```
import datatable as dt
import gc # just in case
def leak(n=10**7):
for i in range(n):
z = dt.update()
leak()
gc.collect()
input("Press enter")
```
Note that despite the name, the `dt.update` is actually a class, though it
is defined via Python C API. Thus, this script is expected to create and
then immediately destroy 10 million simple python objects.
The observed behavior, however, is that the script consumes more and more
memory, eventually ending up at about 500M. The amount of memory the
program ends up consuming is directly proportional to the parameter `n`.
The `gc.get_objects()` does not show any extra objects however. The
`tracemalloc` module shows that there are indeed `n` objects leaked in the
`z=dt.update()` line, but doesn't give any extra details.
In order to dig deeper, I let the process wait on the "input()" line, and
wrote a script to dump the process' memory into a file. Then I scanned
through the file looking at any repeated patterns of 64-bit words. Inside
the memory dump, the following sequences were the most common:
```
0x - 28660404
0x0001024be6e8 - 4999762
0x000101cbdea0 - 119049
0x0054 - 59537
0x0fd00ff0 - 59526
0x0001 - 16895
0x - 12378
...
```
The most suspicious sequence here is 0x0001024be6e8, which if you look
at that address with lldb, is the address of the PyTypeObject "dt.update",
which looks approximately like this:
```
(lldb) p *(PyTypeObject*)(0x00010f4206e8)
(PyTypeObject) $0 = {
ob_base = {
ob_base = {
ob_refcnt = 8
ob_type = 0x00010ec216b0
}
ob_size = 0
}
tp_name = 0x00010f3a442c "datatable.update"
tp_basicsize = 48
tp_itemsize = 0
tp_dealloc = 0x00010f0a8040 (_datatable.cpython-36m-darwin.so`void
py::_safe_dealloc(_object*) at
xobject.h:270)
tp_print = 0x
tp_getattr = 0x
tp_setattr = 0x
tp_as_async = 0x
tp_repr = 0x00010eab3fa0 (Python`object_repr)
tp_as_number = 0x
tp_as_sequence = 0x
tp_as_mapping = 0x
tp_hash = 0x00010eb48640 (Python`_Py_HashPointer)
tp_call = 0x
tp_str = 0x00010eab40d0 (Python`object_str)
tp_getattro = 0x00010eaa1ae0 (Python`PyObject_GenericGetAttr)
tp_setattro = 0x00010eaa1ce0 (Python`PyObject_GenericSetAttr)
tp_as_buffer = 0x
tp_flags = 266240
...
```
Thus, I can be quite certain that 0x1024be6e8 is the address of the
`dt.update` type structure.
The way this address appears in the memory dump looks like this:
```
0x7f97875cbb10: 0x 0x 0x024be6e8 0x0001
0x7f97875cbb20: 0x 0x 0x 0x
0x7f97875cbb30: 0x 0x 0x 0x
0x7f97875cbb40: 0x 0x 0x024be6e8 0x0001
0x7f97875cbb50: 0x 0x 0x 0x
0x7f97875cbb60: 0x 0x 0x 0x
0x7f97875cbb70: 0x 0x 0x024be6e8 0x0001
0x7f97875cbb80: 0x 0x 0x 0x
0x7f97875cbb90: 0x 0x 0x 0x
0x7f97875cbba0: 0x 0x 0x024be6e8 0x0001
0x7f97875cbbb0: 0x 0x 0x 0x
0x7f97875cbbc0: 0x 0x 0x 0x
```
If i guess that all these represent the leaked objects, then inspecting any
of them shows the following:
```
(lldb) p *(PyObject*)(0x7f97875cbb10)
(PyObject) $2 = {
ob_refcnt = 0
ob_type = 0x00010f4206e8
}
```
Thus, the object didn't actually "leak" in the normal sense: its refcount
is 0 and it was reclaimed by the Python runtime (when i print a debug
message in tp_dealloc, i see that the destructor gets called every time).
Still, Python keeps requesting more and more memory from the system instead
of reusing the memory that was supposed to be freed.
I'm currently not sure where to go from here. Is there something wrong with
my python object that prevents it from being correctly processed by the
Python runtime? Because this doesn't seem to be the usual case of
incrementing the refcount too many times.
This behavior was observed in Python 3.6.6 and also Python 3.8.0b2.
--
https://mail.python.org/mailman/listinfo/python-list
Re: GUI (tkinter) popularity and job prospects for
On 2020-10-22 at 12:50:43 -0700, Rich Shepard wrote: > On Thu, 22 Oct 2020, Lammie Jonson wrote: > > I looked at tkinter which seems to have quite a few examples out > > there, but when I searched indeed.com for tkinter and wxpython it > > appeared that there was hardly any job listings mentioning > > those. Why is that ? It's a bit of a demotivating factor to get very > > serious with tk etc. I was going to look at something like > > tensorflow perhaps, though I am not sure if machine learning is that > > easy to pickup or not [...] > You're looking for a job from the wrong end. Let me offer an analogy I > use to describe how you ought to go about your job search. I've often > said that you can teach someone how to use a word processor but that > does not make them a writer. Your rails/JS experiences and your > interest in tkinter or wxpython is focused on tools, not the products > produced by those tools. > If you wanted to hire a professional chef how would you respond to > someone who told you they had great knife skills? Your interest is in > what they could do with those skills to prepare outstanding meals for > you. After all, it's the meals you're buying, not how uniform are > their diced onions. > An employer will hire those who will benefit the employer's company > and position. The tools used to provide those benefits are less > important than the prospective employee's skill in using them to solve > a problem or increase company profits and prove how great a manager is > for hiring you. > Make a list of what you can do and what you've accomplished, not a > list of tools you know. I've no idea what you really want to do but > define that in terms of both your personal satisfaction and what an > employer would value. I'll second that. I had a 40+ year career spanning multiple companies and multiple industries, and putting a tool on your resume may get you past a gatekeeper, but that's rarely why you actually get hired or paid. I picked up Python in 1997 to build some proofs-of-concept for some designs I was working on. I ended up with a tkinter based GUI, but only because the command line programs came together so quickly. IMO, the GUI didn't add anything to my design or to the proofs-of-concept. But I digress. Pick a few tools, learn them well, and use them to build something (often, those last two go hand in hand). Build something that interests you, or something related to an industry or a technology in which you'd like to work, or something that might be useful to you or to someone else. Pick a new tool here or there, but only as a means to improve some other area of knowledge, like building GUIs, or concurrency, or scientific software, or a different style of programming, or whatever. Happy hacking. -- “Whoever undertakes to set himself up as a judge of Truth and Knowledge is shipwrecked by the laughter of the gods.” – Albert Einstein Dan Sommers, http://www.tombstonezero.net/dan -- https://mail.python.org/mailman/listinfo/python-list
Re: GUI (tkinter) popularity and job prospects for
On 10/22/2020 2:58 PM, Lammie Jonson wrote: I looked at tkinter which seems to have quite a few examples out there, but when I searched indeed.com for tkinter and wxpython it appeared that there was hardly any job listings mentioning those. Why is that ? I think that commercial desktop applications with a python compatible GUI would likely use QT or a Python binding thereof. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Debugging a memory leak
On Fri, Oct 23, 2020 at 12:20 PM Pasha Stetsenko wrote: > I'm currently not sure where to go from here. Is there something wrong with > my python object that prevents it from being correctly processed by the > Python runtime? Because this doesn't seem to be the usual case of > incrementing the refcount too many times. Hard to say without seeing the source code. Is your code available anywhere? A few things to test: 1) Can you replicate this behaviour with only standard library classes? Try to find something implemented in C that uses tp_dealloc in a similar way to you. 2) Can you replicate this with an extremely simple cut-down class, and then publish the code for that class along with your question? 3) Does this happen on other operating systems or only on Mac OS? If you can't yourself test this, hopefully posting code from the other two questions will allow other people to try it. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: GUI (tkinter) popularity and job prospects for
Lammie Jonson於 2020年10月23日星期五 UTC+8上午5時20分45秒寫道： > Thanks, > > Yes, I have some sense about how to do job interviews and market myself > which is always an ongoing process. > > I also just have an interest in different technologies that I may want to > investigate as I can get bored with certain things a little at times. If some > technology seems a little interesting to me but I don't see that there are a > lot of job listings for it however, then I sometimes wonder why that is etc A job listing shows the interest of the society, how it connects with your interest? --Jach -- https://mail.python.org/mailman/listinfo/python-list
Re: Debugging a memory leak
> On Oct 22, 2020, at 5:51 PM, Pasha Stetsenko wrote:
>
> Dear Python gurus,
>
> I'm a maintainer of a python library "datatable" (can be installed from
> PyPi), and i've been recently trying to debug a memory leak that occurs in
> my library.
> The program that exposes the leak is quite simple:
> ```
> import datatable as dt
> import gc # just in case
>
> def leak(n=10**7):
>for i in range(n):
>z = dt.update()
>
> leak()
> gc.collect()
> input("Press enter")
> ```
Hi Pasha,
dt.update() is acting on some object(s) outside the leak function body. And so
even though, local objects z, i and n are eventually garbage collected, the
side-effects of dt.update() are not affected by the return from the leak
function. You need to look at your module and carefully trace what happens when
dt.update() is executed. It seems to me that any memory consumed when
dt.update() is executed will not be released when the leak function returns.
humbly,
Karen
--
https://mail.python.org/mailman/listinfo/python-list
Re: Question on ABC classes
On 23/10/20 2:13 pm, Julio Di Egidio wrote: I am now thinking whether I could achieve the "standard" behaviour via another approach, say with decorators, somehow intercepting calls to __new__... maybe. I'm inclined to step back and ask -- why do you care about this? Would it actually do any harm if someone instantiated your base class? If not, then it's probably not worth going out of your way to prevent it. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: Question on ABC classes
On Friday, 23 October 2020 07:36:39 UTC+2, Greg Ewing wrote: > On 23/10/20 2:13 pm, Julio Di Egidio wrote: > > I am now thinking whether I could achieve the "standard" > > behaviour via another approach, say with decorators, somehow > > intercepting calls to __new__... maybe. > > I'm inclined to step back and ask -- why do you care about this? > > Would it actually do any harm if someone instantiated your > base class? If not, then it's probably not worth going out > of your way to prevent it. This is the first little library I try to put together in Python, and it was natural for me to hit it with all the relevant decorations as well as type annotations in order to expose *strict* contracts, plus hopefully have intellisense work all over the place. After several days of struggling, I am indeed finding that impossible in Python, at least with the available tools, and I am indeed going out of my way to just get some half-decent results... But, if I give up on strict contracts, I can as well give up on type annotations and the whole lot, indeed why even subclass ABC? Which is maybe too drastic, maybe not: it's the next thing I am going to try, and see what I remain with. :) Of course, any more hints welcome... Julio -- https://mail.python.org/mailman/listinfo/python-list
