[Python-Dev] Re: Decreasing refcount for locals before popping frame

2022-04-29 Thread Thomas Grainger
Can you ping me on the airflow PR for this change? (@graingert) On Fri, Apr 29, 2022, 7:54 AM Malthe wrote: > On Fri, 29 Apr 2022 at 06:50, Thomas Grainger wrote: > > You can use a `__del__` method that warns on collection - like an > unawaited coroutine > > > > Also if you're in control of imp

[Python-Dev] Re: Decreasing refcount for locals before popping frame

2022-04-29 Thread Thomas Grainger
Does this only apply to DAGfiles? Eg https://airflow.apache.org/docs/apache-airflow/1.10.12/concepts.html#scope You can use a `__del__` method that warns on collection - like an unawaited coroutine Also if you're in control of importing the dagfile you can record all created dags and report any t

[Python-Dev] Re: Decreasing refcount for locals before popping frame

2022-04-29 Thread Malthe
On Fri, 29 Apr 2022 at 06:50, Thomas Grainger wrote: > You can use a `__del__` method that warns on collection - like an unawaited > coroutine > > Also if you're in control of importing the dagfile you can record all created > dags and report any that are missing from the globals of the module

[Python-Dev] Re: Decreasing refcount for locals before popping frame

2022-04-28 Thread Malthe
On Fri, 29 Apr 2022 at 06:38, Thomas Grainger wrote: > Can you show a run-able example of the successful and unsuccessful usage of > `with DAG(): ... `? from airflow import DAG # correct: dag = DAG("my_dag") # incorrect: DAG("my_dag") The with construct really has nothing to do with it, but i

[Python-Dev] Re: Decreasing refcount for locals before popping frame

2022-04-28 Thread Thomas Grainger
Can you show a run-able example of the successful and unsuccessful usage of `with DAG(): ... `? On Fri, Apr 29, 2022, 6:31 AM Malthe wrote: > Pablo Galindo Salgado wrote: > > As it has been mentioned there is no guarantee that your variable will > even > > be finalized (or even destroyed) after

[Python-Dev] Re: Decreasing refcount for locals before popping frame

2022-04-28 Thread Malthe
Pablo Galindo Salgado wrote: > As it has been mentioned there is no guarantee that your variable will even > be finalized (or even destroyed) after the frame finishes. For example, if > your variable goes into a reference cycle for whatever reason it may not be > cleared until a GC run happens (and

[Python-Dev] Re: Decreasing refcount for locals before popping frame

2022-04-28 Thread Malthe
Dennis Sweeney wrote: > I don't know if there's anything specifically stopping this, but from what I > understand, the precise moment that a finalizer gets called is unspecified, > so relying on any sort of behavior there is undefined and non-portable. > Implementations like PyPy don't always us

[Python-Dev] Re: Decreasing refcount for locals before popping frame

2022-04-28 Thread Pablo Galindo Salgado
As it has been mentioned there is no guarantee that your variable will even be finalized (or even destroyed) after the frame finishes. For example, if your variable goes into a reference cycle for whatever reason it may not be cleared until a GC run happens (and in some situations it may not even b

[Python-Dev] Re: Decreasing refcount for locals before popping frame

2022-04-28 Thread Dennis Sweeney
I don't know if there's anything specifically stopping this, but from what I understand, the precise moment that a finalizer gets called is unspecified, so relying on any sort of behavior there is undefined and non-portable. Implementations like PyPy don't always use reference counting, so their