[issue43572] "Too many open files" on macOS buildbot
Matt Billenstein added the comment: I don't see that error message - where are you seeing it? -- ___ Python tracker <https://bugs.python.org/issue43572> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43572] "Too many open files" on macOS buildbot
Matt Billenstein added the comment: In any case - I've raised file/process limits on the buildbot. -- ___ Python tracker <https://bugs.python.org/issue43572> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43572] "Too many open files" on macOS buildbot
Matt Billenstein added the comment: Ah, word, was looking at stdout -- 256 is pretty low, raised it to 200k... -- ___ Python tracker <https://bugs.python.org/issue43572> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44290] x86-64 macOS 3.x buildbot build failed with: No such file or directory: '/Users/buildbot/buildarea/3.x.billenstein-macos/build/target/include/python3.11d/pyconfig.h'
Matt Billenstein added the comment: I rebuilt it a while ago which could have caused this - can you re-run the affected build? thx m On Wed, Jun 02, 2021 at 06:08:16PM +, Ned Deily wrote: > > Ned Deily added the comment: > > I can't reproduce that failure with that checkout and I'm not even entirely > sure where that error is coming from. My guess it that something went wrong > during the previous build that resulted in the connection lost (a system > crash perhaps?) that left the build area or the file system in an unusual > state. > > @mattbillenstein, could you check the buildbot and perhaps delete the build > directory? > > -- > nosy: +mattbillenstein > > ___ > Python tracker > <https://bugs.python.org/issue44290> > ___ -- ___ Python tracker <https://bugs.python.org/issue44290> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44290] x86-64 macOS 3.x buildbot build failed with: No such file or directory: '/Users/buildbot/buildarea/3.x.billenstein-macos/build/target/include/python3.11d/pyconfig.h'
Matt Billenstein added the comment: Hmm, digging, afaict the buildbot-worker is up and running - something on the master? I just updated my setup to use python3 instead of python2... On Wed, Jun 02, 2021 at 06:24:23PM +, Ned Deily wrote: > > Ned Deily added the comment: > > Hmm, I just tried to restart it from the builbot page but the build request > is just sitting there ATM. > > https://buildbot.python.org/all/#/builders/366 > > -- > > ___ > Python tracker > <https://bugs.python.org/issue44290> > ___ -- ___ Python tracker <https://bugs.python.org/issue44290> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44290] x86-64 macOS 3.x buildbot build failed with: No such file or directory: '/Users/buildbot/buildarea/3.x.billenstein-macos/build/target/include/python3.11d/pyconfig.h'
Matt Billenstein added the comment: Probably at 321 where it lost connection - I would have shut it down and wiped the buildarea. I'm not sure what's going on now, I'm walking back versions of buildbot-worker and they seem to never connect to the master... m On Wed, Jun 02, 2021 at 07:25:49PM +, Ned Deily wrote: > > Ned Deily added the comment: > > Can you say at what point you did the upgrade, i.e. between which two builds > here? > > https://buildbot.python.org/all/#/builders/366 > > -- > > ___ > Python tracker > <https://bugs.python.org/issue44290> > ___ -- ___ Python tracker <https://bugs.python.org/issue44290> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44290] x86-64 macOS 3.x buildbot build failed with: No such file or directory: '/Users/buildbot/buildarea/3.x.billenstein-macos/build/target/include/python3.11d/pyconfig.h'
Matt Billenstein added the comment: Hmm, yeah, it did seem to me like the server was wedged or something... glad to see it going. M -- Matt Billenstein m...@vazor.com > On Jun 2, 2021, at 6:13 PM, STINNER Victor wrote: > > > STINNER Victor added the comment: > >> Hmm, I just tried to restart it from the builbot page but the build request >> is just sitting there ATM. >> https://buildbot.python.org/all/#/builders/366 > > Not sure what happened. I restarted the buildbot server manually. I cancelled > the old manual build and scheduled a new manual build which is now running. > > -- > > ___ > Python tracker > <https://bugs.python.org/issue44290> > ___ -- ___ Python tracker <https://bugs.python.org/issue44290> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44290] x86-64 macOS 3.x buildbot build failed with: No such file or directory: '/Users/buildbot/buildarea/3.x.billenstein-macos/build/target/include/python3.11d/pyconfig.h'
Matt Billenstein added the comment: So, I'd been trying various things before the master restart and I was using python3 supplied by homebrew - I decided I didn't like that dependency and switched it back to /usr/bin/python3 as supplied by the system [1]. I did a rebuild on 325 a couple more times and both of those failed in the same way - now I've switched back to homebrew python3 and compile succeeded this time... I'll let this bake, seems pretty strange changing the interpreter running the buildbot would matter - everything else should be the same however. m 1: mattb@macbook-pro:~ $ /usr/bin/python3 Python 3.8.2 (default, Apr 8 2021, 23:19:18) [Clang 12.0.5 (clang-1205.0.22.9)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> ^D mattb@macbook-pro:~ $ /usr/local/bin/python3 Python 3.9.5 (default, May 4 2021, 03:36:27) [Clang 12.0.0 (clang-1200.0.32.29)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> On Thu, Jun 03, 2021 at 04:20:40AM +, Ned Deily wrote: > > Ned Deily added the comment: > > Alas, it looks like build 325 failed in the same way as build 322: > > https://buildbot.python.org/all/#/builders/366/builds/325 > > I wonder if there might be a Makefile race condition. > > -- > resolution: out of date -> > stage: resolved -> > status: closed -> open > > ___ > Python tracker > <https://bugs.python.org/issue44290> > ___ -- ___ Python tracker <https://bugs.python.org/issue44290> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44290] x86-64 macOS 3.x buildbot build failed with: No such file or directory: '/Users/buildbot/buildarea/3.x.billenstein-macos/build/target/include/python3.11d/pyconfig.h'
Matt Billenstein added the comment: I have been stopping it since the initial problem - there’s something wonky with using the system python on macos afaict... M -- Matt Billenstein m...@vazor.com > On Jun 3, 2021, at 4:39 AM, STINNER Victor wrote: > > > STINNER Victor added the comment: > > "So, I'd been trying various things before the master restart (...) I'll let > this bake, seems pretty strange changing the interpreter running the > buildbot would matter - everything else should be the same however." > > If you can, try to stop your buildbot client when you upgrade your system. > System upgrades always caused hiccups on buildbots, it's ok. It's just > suprising when I got an error and don't know that the system was upgraded. > > I close again the issue ;-) > > -- > resolution: -> fixed > stage: -> resolved > status: open -> closed > > ___ > Python tracker > <https://bugs.python.org/issue44290> > ___ -- ___ Python tracker <https://bugs.python.org/issue44290> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30703] regrtest hangs on the master branch
Matt Billenstein added the comment: I've been debugging this and I can repro on El Capitan on a different machine as well -- it's an infrequent hang, I've been running in a loop: mattb@mattb-mbp:~/src/misc/cpython master$ for i in $(seq 1000); do echo "Run: $i -- $(date)"; time./python.exe -m test --timeout 180 test_multiprocessing_forkserver || break; sleep 10; done And in this case after ~30 successful runs: Run: 31 -- Tue Jun 20 05:12:19 PDT 2017 Run tests sequentially 0:00:00 load avg: 4.28 [1/1] test_multiprocessing_forkserver Timeout (0:03:00)! Thread 0x7fffa1b3a3c0 (most recent call first): File "/Users/mattb/src/misc/cpython/Lib/selectors.py", line 415 in select File "/Users/mattb/src/misc/cpython/Lib/multiprocessing/connection.py", line 916 in wait File "/Users/mattb/src/misc/cpython/Lib/multiprocessing/popen_fork.py", line 45 in wait File "/Users/mattb/src/misc/cpython/Lib/multiprocessing/process.py", line 121 in join File "/Users/mattb/src/misc/cpython/Lib/test/_test_multiprocessing.py", line 431 in test_many_processes File "/Users/mattb/src/misc/cpython/Lib/unittest/case.py", line 605 in run File "/Users/mattb/src/misc/cpython/Lib/unittest/case.py", line 653 in __call__ File "/Users/mattb/src/misc/cpython/Lib/unittest/suite.py", line 122 in run File "/Users/mattb/src/misc/cpython/Lib/unittest/suite.py", line 84 in __call__ File "/Users/mattb/src/misc/cpython/Lib/unittest/suite.py", line 122 in run File "/Users/mattb/src/misc/cpython/Lib/unittest/suite.py", line 84 in __call__ File "/Users/mattb/src/misc/cpython/Lib/unittest/suite.py", line 122 in run File "/Users/mattb/src/misc/cpython/Lib/unittest/suite.py", line 84 in __call__ File "/Users/mattb/src/misc/cpython/Lib/test/support/__init__.py", line 1772 in run File "/Users/mattb/src/misc/cpython/Lib/test/support/__init__.py", line 1896 in _run_suite File "/Users/mattb/src/misc/cpython/Lib/test/support/__init__.py", line 1936 in run_unittest File "/Users/mattb/src/misc/cpython/Lib/test/libregrtest/runtest.py", line 168 in test_runner File "/Users/mattb/src/misc/cpython/Lib/test/libregrtest/runtest.py", line 169 in runtest_inner File "/Users/mattb/src/misc/cpython/Lib/test/libregrtest/runtest.py", line 137 in runtest File "/Users/mattb/src/misc/cpython/Lib/test/libregrtest/main.py", line 374 in run_tests_sequential File "/Users/mattb/src/misc/cpython/Lib/test/libregrtest/main.py", line 454 in run_tests File "/Users/mattb/src/misc/cpython/Lib/test/libregrtest/main.py", line 530 in _main File "/Users/mattb/src/misc/cpython/Lib/test/libregrtest/main.py", line 504 in main File "/Users/mattb/src/misc/cpython/Lib/test/libregrtest/main.py", line 573 in main File "/Users/mattb/src/misc/cpython/Lib/test/__main__.py", line 2 in File "/Users/mattb/src/misc/cpython/Lib/runpy.py", line 85 in _run_code File "/Users/mattb/src/misc/cpython/Lib/runpy.py", line 193 in _run_module_as_main real3m0.115s user0m5.804s sys 0m0.683s I also ran ~600 passes overnight on this test on an Ubuntu 16.04 machine without a hang, so it seems to be OSX specific. -- nosy: +mattbillenstein ___ Python tracker <http://bugs.python.org/issue30703> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30703] regrtest hangs on the master branch
Matt Billenstein added the comment: It consistently takes between ~61 and ~73 seconds with this setup. -- ___ Python tracker <http://bugs.python.org/issue30703> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30703] test_multiprocessing_forkserver hangs on the master branch
Matt Billenstein added the comment: Cool -- do you need me to do something more to help debug this? -- ___ Python tracker <http://bugs.python.org/issue30703> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30703] test_multiprocessing_forkserver hangs on the master branch
Matt Billenstein added the comment: I don't see anything odd -- it runs for awhile and then times out once it's deadlocked: 0:03:18 load avg: 3.20 [224] test_multiprocessing_forkserver test_many_processes (test.test_multiprocessing_forkserver.WithProcessesTestProcess) ... ok test_many_processes (test.test_multiprocessing_forkserver.WithThreadsTestProcess) ... skipped 'test not appropriate for threads' -- Ran 2 tests in 0.849s OK (skipped=1) 0:03:18 load avg: 3.11 [225] test_multiprocessing_forkserver test_many_processes (test.test_multiprocessing_forkserver.WithProcessesTestProcess) ... ok test_many_processes (test.test_multiprocessing_forkserver.WithThreadsTestProcess) ... skipped 'test not appropriate for threads' -- Ran 2 tests in 0.865s OK (skipped=1) 0:03:19 load avg: 3.11 [226] test_multiprocessing_forkserver test_many_processes (test.test_multiprocessing_forkserver.WithProcessesTestProcess) ... Timeout (0:00:30)! Thread 0x7fff790c1000 (most recent call first): File "/Users/mattb/src/misc/cpython/Lib/selectors.py", line 415 in select File "/Users/mattb/src/misc/cpython/Lib/multiprocessing/connection.py", line 916 in wait File "/Users/mattb/src/misc/cpython/Lib/multiprocessing/popen_fork.py", line 45 in wait File "/Users/mattb/src/misc/cpython/Lib/multiprocessing/process.py", line 121 in join File "/Users/mattb/src/misc/cpython/Lib/test/_test_multiprocessing.py", line 431 in test_many_processes File "/Users/mattb/src/misc/cpython/Lib/unittest/case.py", line 605 in run File "/Users/mattb/src/misc/cpython/Lib/unittest/case.py", line 653 in __call__ File "/Users/mattb/src/misc/cpython/Lib/unittest/suite.py", line 122 in run File "/Users/mattb/src/misc/cpython/Lib/unittest/suite.py", line 84 in __call__ File "/Users/mattb/src/misc/cpython/Lib/unittest/suite.py", line 122 in run File "/Users/mattb/src/misc/cpython/Lib/unittest/suite.py", line 84 in __call__ File "/Users/mattb/src/misc/cpython/Lib/unittest/suite.py", line 122 in run File "/Users/mattb/src/misc/cpython/Lib/unittest/suite.py", line 84 in __call__ File "/Users/mattb/src/misc/cpython/Lib/unittest/runner.py", line 176 in run File "/Users/mattb/src/misc/cpython/Lib/test/support/__init__.py", line 1896 in _run_suite File "/Users/mattb/src/misc/cpython/Lib/test/support/__init__.py", line 1936 in run_unittest File "/Users/mattb/src/misc/cpython/Lib/test/libregrtest/runtest.py", line 168 in test_runner File "/Users/mattb/src/misc/cpython/Lib/test/libregrtest/runtest.py", line 169 in runtest_inner File "/Users/mattb/src/misc/cpython/Lib/test/libregrtest/runtest.py", line 137 in runtest File "/Users/mattb/src/misc/cpython/Lib/test/libregrtest/main.py", line 374 in run_tests_sequential File "/Users/mattb/src/misc/cpython/Lib/test/libregrtest/main.py", line 454 in run_tests File "/Users/mattb/src/misc/cpython/Lib/test/libregrtest/main.py", line 530 in _main
[issue30703] test_multiprocessing_forkserver hangs on the master branch
Matt Billenstein added the comment: Yes, I'll email you the details. -- ___ Python tracker <http://bugs.python.org/issue30703> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30703] Non-reentrant signal handler (test_multiprocessing_forkserver hangs)
Matt Billenstein added the comment: Verified this by letting the test_many_processes loop overnight (master@42bc8beadd49)-- 34k passes later and no hangs. Nice work! -- ___ Python tracker <http://bugs.python.org/issue30703> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31805] support._is_gui_available() hangs on x86-64 Sierra 3.6/3.x buildbot
Matt Billenstein added the comment: Did some debugging by sticking a couple prints in support.is_gui_available() and it appears everything is fine up until the call to app_services.SetFrontProcess() -- this is returning error code -606 which is defined in: /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/CarbonCore.framework/Versions/A/Headers/MacErrors.h: appIsDaemon = -606, /*app is BG-only, and launch flags disallow this*/ What this means exactly or how to fix it, I don't know... -- ___ Python tracker <https://bugs.python.org/issue31805> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31805] support._is_gui_available() hangs on x86-64 Sierra 3.6/3.x buildbot
Matt Billenstein added the comment: Note, that's running ./python.exe Lib/tests/test_tk.py from a Terminal when logged into the gui... -- ___ Python tracker <https://bugs.python.org/issue31805> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31380] test_undecodable_filename() in Lib/test/test_httpservers.py broken on APFS
Matt Billenstein added the comment: Reflected on the High Sierra buildbot now: http://buildbot.python.org/all/#/builders/14/builds/162 -- nosy: +mattbillenstein ___ Python tracker <https://bugs.python.org/issue31380> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32517] test_read_pty_output() of test_asyncio hangs on macOS 10.13.2 (darwin 17.3.0)
Matt Billenstein added the comment: I don't see a conflict in the uids: mattb@mattb-mbp2:~ $ id -u buildbot 506 mattb@mattb-mbp2:~ $ id -u _timed 266 mattb@mattb-mbp2:~ $ grep _timed /etc/passwd _timed:*:266:266:Time Sync Daemon:/var/db/timed:/usr/bin/false mattb@mattb-mbp2:~ $ grep 506 /etc/passwd mattb@mattb-mbp2:~ $ Buildbot is started via launchd: mattb@mattb-mbp2:~ $ sudo cat /Library/LaunchDaemons/net.buildbot.slave.plist http://www.apple.com/DTDs/PropertyList-1.0.dtd";> Label net.buildbot.slave UserName buildbot WorkingDirectory /Users/buildbot/buildarea ProgramArguments /Users/buildbot/bin/run_slave.sh StandardOutPath twistd.log StandardErrorPath twistd.log KeepAlive SessionCreate -- ___ Python tracker <https://bugs.python.org/issue32517> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32517] test_read_pty_output() of test_asyncio hangs on macOS 10.13.2 (darwin 17.3.0)
Matt Billenstein added the comment: Note, I can repro running it by hand from the cli. And I cannot repro on 3.x, only 3.6 on the same machine. -- ___ Python tracker <https://bugs.python.org/issue32517> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
