celery multi celery.exceptions.TimeoutError: The operation timed out

[iMath]

I followed the official [Celery 
guide](https://docs.celeryproject.org/en/stable/getting-started/next-steps.html#in-the-background)
 to learn, everything works well when using `celery -A proj worker -l info` to  
start the worker, however , if I run the worker in the background using `celery 
multi restart w1 -A proj -l info --logfile=./celery.log`, then I got 


>>> from proj.tasks import add
>>> add.delay().get(timeout=19)
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/local/lib/python3.8/dist-packages/celery/result.py", line 230, 
in get
return self.backend.wait_for_pending(
  File "/usr/local/lib/python3.8/dist-packages/celery/backends/base.py", 
line 660, in wait_for_pending
meta = self.wait_for(
  File "/usr/local/lib/python3.8/dist-packages/celery/backends/base.py", 
line 696, in wait_for
raise TimeoutError('The operation timed out.')
celery.exceptions.TimeoutError: The operation timed out.
>>>
So what's wrong ?

Test environment :

 - Python 3.8.2 
 - celery multi v4.4.7
 - rabbitmqctl version --->3.8.2

and Project layout ([code 
files](https://github.com/celery/celery/files/5231502/proj.zip)):

> proj/__init__.py
> /celery.py
> /tasks.py

celery.py

from __future__ import absolute_import, unicode_literals

from celery import Celery

app = Celery('proj',
 broker='amqp://localhost',
 # backend='db+sqlite:///results.sqlite',  # The backend 
argument， If you don’t need results, it’s better to disable them.  Results can 
also be disabled for individual tasks by setting the @task(ignore_result=True) 
option.
 include=['proj.tasks'])  # The include argument is a list of 
modules to import when the worker starts. You need to add our tasks module here 
so that the worker is able to find our tasks.
app.conf.CELERY_RESULT_BACKEND = 'db+sqlite:///results.sqlite'

if __name__ == '__main__':
app.start()

tasks.py

from __future__ import absolute_import, unicode_literals
from .celery import app
import datetime
@app.task
def add():
return datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")


The log info as the following , I googled , but didn't find a working solutions 
```
[2020-09-15 22:10:03,229: INFO/MainProcess] Connected to 
amqp://guest:**@127.0.0.1:5672//
[2020-09-15 22:10:03,238: INFO/MainProcess] mingle: searching for neighbors
[2020-09-15 22:10:04,260: INFO/MainProcess] mingle: all alone
[2020-09-15 22:10:04,273: INFO/MainProcess] w1@iZwz962a07bhoio77q4ewxZ ready.
[2020-09-15 22:10:10,625: ERROR/MainProcess] Received unregistered task of type 
'proj.tasks.add'.
The message has been ignored and discarded.

Did you remember to import the module containing this task?
Or maybe you're using relative imports?

Please see
http://docs.celeryq.org/en/latest/internals/protocol.html
for more information.

The full contents of the message body was:
'[[], {}, {"callbacks": null, "errbacks": null, "chain": null, "chord": null}]' 
(77b)
Traceback (most recent call last):
  File 
"/usr/local/lib/python3.8/dist-packages/celery/worker/consumer/consumer.py", 
line 562, in on_task_received
strategy = strategies[type_]
KeyError: 'proj.tasks.add'

```
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Asyncio Queue implementation suggestion

[Barry Scott]

> On 17 Sep 2020, at 15:51, Dennis Lee Bieber  wrote:
> 
> On Wed, 16 Sep 2020 13:39:51 -0400, Alberto Sentieri <2...@tripolho.com>
> declaimed the following:
> 
> 
>> devices tested simultaneously, I soon run out of file descriptor. Well, 
>> I increased the number of file descriptor in the application and then I 
>> started running into problems like “ValueError: filedescriptor out of 
>> range in select()”. I guess this problem is related to a package called 
> 
> 
> https://man7.org/linux/man-pages/man2/select.2.html
> """
> An fd_set is a fixed size buffer. 
> """
> and
> """
> On success, select() and pselect() return the number of file
>   descriptors contained in the three returned descriptor sets (that
> is,
>   the total number of bits that are set in readfds, writefds,
>   exceptfds).
> """
> Emphasis "number of bits that are set" -- the two together implies that
> these are words/longwords/quadwords used a bitmaps, one fd per bit, and
> hence inherently limited to only as many bits as the word-length supports.

The limit has nothing to do with word size. The bits are passed as unsigned 
char arrary.

Here is the piece expelling the limit:
 select() can monitor only file descriptors numbers that are less than
   FD_SETSIZE; poll(2)  
and epoll(7)  do not have 
this limitation.  See
   BUGS.
FD_SETSIZE is typically 1024.

Barry

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Puzzling difference between lists and tuples

[Grant Edwards]

On 2020-09-18, Chris Angelico  wrote:
> On Fri, Sep 18, 2020 at 10:53 AM Grant Edwards
> wrote:
>>
>> On 2020-09-17, MRAB  wrote:
>> >> The only time the parentheses are required for tuple building is when
>> >> they would otherwise not be interpreted that way:
>> >>
>> > They're needed for the empty tuple, which doesn't have a comma.
>> >
>> >> some_func('first', 'second')   # some_func called with two str args
>> >>
>> >> some_func(('first', 'second')) # some_func called with one tuple arg
>>
>> Yea, the syntax for tuple literals has always been a bit of an ugly
>> spot in Python.  If ASCII had only had one more set of open/close
>> brackets...
>
> ...then I'd prefer them to be used for sets, actually. I think the
> set/dict collision is more weird than the tuple/grouping one.
>
> Now, if only ASCII had *two* more sets of open/close brackets...

Rigth. I had forgotten about sets -- they're still a "new" feature in
my mind, and I rarely seem to run into situations where I use them.

There must be a few more sets of brackets in Unicode...

--
Grant


-- 
https://mail.python.org/mailman/listinfo/python-list

importlib changes from py3.7 to py3.8

[Eko palypse]

With python3.7 I'm able to use

_module = importlib.import_module('app.plugins.plugin_dir.plugin_name')

succesfully but with python 3.8 this results in 

  File "D:\ProgramData\Python\Python38_64\lib\importlib\__init__.py", line 127, 
in import_module
return _bootstrap._gcd_import(name[level:], package, level)
  File "", line 1014, in _gcd_import
  File "", line 991, in _find_and_load
  File "", line 961, in _find_and_load_unlocked
  File "", line 219, in _call_with_frames_removed
  File "", line 1014, in _gcd_import
  File "", line 991, in _find_and_load
  File "", line 961, in _find_and_load_unlocked
  File "", line 219, in _call_with_frames_removed
  File "", line 1014, in _gcd_import
  File "", line 991, in _find_and_load
  File "", line 975, in _find_and_load_unlocked
  File "", line 671, in _load_unlocked
  File "", line 779, in exec_module
  File "", line 916, in get_code
  File "", line 846, in source_to_code
  File "", line 219, in _call_with_frames_removed
ValueError: source code string cannot contain null bytes

Any idea what happened here?

Thank you
Eren
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Puzzling difference between lists and tuples

[Greg Ewing]

On 19/09/20 6:49 am, Grant Edwards wrote:


There must be a few more sets of brackets in Unicode...


Quite a lot, actually. The character browser in MacOSX is currently
showing me 17, not including the ones that are built up from
multiple characters.

Some of them could be easily confused with normal brackets, but
there are a few interesting ones, such as LEFT/RIGHT WHITE/BLACK
LENTICULAR BRACKET and LEFT/RIGHT TORTOISE SHELL BRACKET.

〖 〗【 】〔 〕
--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Making Python the Best App Dev Platform

[gMail]

Hello, All

We all know Python is a wonderful environment, with fantastic packages for App 
Dev like Flask, SQLAlchemy and Django.  Now let’s make it the only reasonable 
choice for building database-oriented systems.

I’d like to introduce 2 new capabilities:

Build a web app in 10 minutes: this generator 
 builds basic multi-page, 
multi-table apps based on Flask AppBuilder and SQLAlchemy; here’s an article 


40x reduction in backend code: this rules engine 
 uses spreadsheet-like rules to 
automate SQLAlchemy multi-table derivation and constraint logic.  The readme 
and samples show how 5 rules equate to 200 lines of code.

The potential value is substantial: a 40X is reduction is not only helpful for 
our projects, but has the potential to expand our community.

The second project is in-progress.  If you have feedback on the general value 
or particular features, I’d love to hear from you.  And if you’d like to help, 
super!

Best,
Val

-- 
https://mail.python.org/mailman/listinfo/python-list