Le Sun, Aug 25, 2024 at 10:01:00AM +0200, Landry Breuil a écrit :
> Le Sun, Aug 25, 2024 at 09:36:26AM +0200, Landry Breuil a écrit :
> > Le Sun, Aug 25, 2024 at 03:30:56AM -0400, A Tammy a écrit :
> > >
> > > On 8/25/24 3:07 AM, Landry Breuil wrote:
> > > > Le Thu, Aug 22, 2024 at 10:04:45AM +0200, Landry Breuil a écrit :
> > > >> Le Thu, Aug 22, 2024 at 09:40:23AM +0200, Landry Breuil a écrit :
> > > >>> Le Wed, Aug 21, 2024 at 06:11:32PM +0200, Kirill A. Korinsky a écrit :
> > > >>>> On Wed, 21 Aug 2024 17:16:11 +0200,
> > > >>>> Landry Breuil <lan...@openbsd.org> wrote:
> > > >>>>> Le Wed, Aug 21, 2024 at 05:00:21PM +0200, Landry Breuil a écrit :
> > > >>>>>> Le Wed, Aug 21, 2024 at 03:49:45PM +0200, Kirill A. Korinsky a
> > > >>>>>> écrit :
> > > >>>>>>> ports@,
> > > >>>>>>>
> > > >>>>>>> Here a ping from another diff from my
> > > >>>>>> can you explain the dance about dropping/copying tests in pre-test
> > > >>>>>> ?
> > > >>>>>> why was that needed ? isn't doing MODPY_PYTEST_ARGS = --forked
> > > >>>>>> enough ?
> > > >>>>> now that i've tested, this fails with a rather strange error
> > > >>>>> (strange as
> > > >>>>> in "i dont understand why moving tests/ around helps") on all test
> > > >>>>> files:
> > > >>>>>
> > > >>>> and I confirm that this is the problem I am overlooking by moving
> > > >>>> tests.
> > > >> it feels we're getting close.. but not yet.
> > > > this version with a symlink for the rust lib works, and it's probably
> > > > what im
> > > > going to commit. fixing the tests requiring a throwaway synapse test
> > > > instance
> > > > is left for future work.
> > > >
> > > > Index: Makefile
> > > > ===================================================================
> > > > RCS file: /cvs/ports/net/synapse/Makefile,v
> > > > diff -u -r1.82 Makefile
> > > > --- Makefile 31 Jul 2024 16:01:52 -0000 1.82
> > > > +++ Makefile 25 Aug 2024 07:04:41 -0000
> > > > @@ -72,10 +72,20 @@
> > > > TEST_DEPENDS = ${FULLPKGNAME}:${BUILD_PKGPATH} \
> > > > devel/py-mock${MODPY_FLAVOR} \
> > > > devel/py-parameterized${MODPY_FLAVOR} \
> > > > + devel/py-test-forked${MODPY_FLAVOR} \
> > > > www/py-jwt${MODPY_FLAVOR}
> > > >
> > > > do-configure:
> > > > @${MODCARGO_configure}
> > > > +
> > > > +MODPY_PYTEST_ARGS = --forked
> > > > +
> > > > +# some tests fail, but they need a previously running synapse process,
> > > > not just a database
> > > > +# E synapse.storage.prepare_database.UpgradeDatabaseException:
> > > > Uninitialised database:
> > > > +# run the main synapse process to prepare the database schema before
> > > > starting worker processes.
> > > > +# make sure that the rust library is found
> > > > +pre-test:
> > > > + ln -sf
> > > > ${WRKSRC}/build/lib.openbsd-${OSREV}-${ARCH}-cpython-${MODPY_MAJORMINOR}/synapse/synapse_rust.abi3.so
> > > > ${WRKSRC}/synapse/
> > > >
> > > > # to generate rust modules.inc:
> > > > # make modcargo-gen-crates and modcargo-gen-crates-licenses
> > >
> > >
> > > I encountered almost exactly the same kind of bug for devel/py-uvloop.
> > > The solution was to move or delete the original sources folder, like so
> > > -
> > > http://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/devel/py-uvloop/Makefile?rev=1.2&content-type=text/x-cvsweb-markup
> > >
> > > pre-test:
> > > cd ${WRKSRC} && test -d uvloop && mv uvloop uvloop-bk || test -d
> > > uvloop-bk
> > >
> > > This was because python tries to find the package to import in the local
> > > folder before it tries to find it in the PYTHONPATH environment vars
> > > locations.
> > >
> > > Because the python tests run after the fake target has been completed,
> > > the move should be fine and shouldn't affect the PLIST generation.
> >
> > yeah, i also figured that out after finding how nix runs the tests in
> > their synapse package, they rm -Rf synapse before starting tests.
> >
> > interestingly, i also realized that upstream doesn't run tests via
> > pytest, but via twisted.trial, which might work around the "needs a
> > running instance of synapse first" issue.
> >
> > currently trying with:
> >
> > # this symlink makes sure that the rust library is found, since by default
> > the source is used to run the tests
> > pre-test:
> > ln -sf
> > ${WRKSRC}/build/lib.openbsd-${OSREV}-${ARCH}-cpython-${MODPY_MAJORMINOR}/synapse/synapse_rust.abi3.so
> > ${WRKSRC}/synapse/
> >
> > do-test:
> > cd ${MODPY_TEST_DIR} && ${SETENV} ${ALL_TEST_ENV} ${MODPY_BIN} -m
> > twisted.trial tests
> >
> > this produces a more verbose output with all tests names printed, but maybe
> > more tests succeed.
>
> running tests with twisted.trial doesnt require pytest-forked, and only
> 3 tests fail:
>
> tests.handlers.test_user_directory.UserDirectoryTestCase.test_search_punctuation
> twisted.trial.unittest.FailTest: Sequences differ: ['@user-a2:test'] !=
> ['@user-a1:test']
>
> tests.storage.test_room_search.EventSearchInsertionTest.test_null_byte
> twisted.trial.unittest.FailTest: 0 != 1
>
> tests.storage.test_room_search.MessageSearchTest.test_sqlite_search
> twisted.trial.unittest.FailTest: 0 != 1 : expected 'quick brown' to match
> 'the quick brown fox jumps over the lazy dog'
>
> FAILED (skips=303, failures=3, successes=3162)
>
> the other skipped tests are for missing deps (or unsupported/not yet
> working tests):
> - OIDC
> - saml2
> - pyICU
> - jaeger_client
> - Postgres
> - authlib
>
> so.. i don't know which is best. Keep using pytest --forked as it's more
> integrated within the portstree, or run the tests the same way upstream
> does ?
comparing the results with pytest:
=========================== short test summary info ============================
FAILED
tests/app/test_openid_listener.py::FederationReaderOpenIDListenerTests::test_openid_listener_0
FAILED
tests/app/test_openid_listener.py::FederationReaderOpenIDListenerTests::test_openid_listener_1
FAILED
tests/app/test_openid_listener.py::FederationReaderOpenIDListenerTests::test_openid_listener_2
FAILED
tests/app/test_openid_listener.py::FederationReaderOpenIDListenerTests::test_openid_listener_3
FAILED
tests/handlers/test_user_directory.py::UserDirectoryTestCase::test_search_punctuation
FAILED
tests/replication/test_federation_ack.py::FederationAckTestCase::test_federation_ack_sent
FAILED
tests/storage/test_rollback_worker.py::WorkerSchemaTests::test_not_upgraded_current_schema_version_with_outstanding_deltas
FAILED
tests/storage/test_rollback_worker.py::WorkerSchemaTests::test_not_upgraded_old_schema_version
FAILED
tests/storage/test_rollback_worker.py::WorkerSchemaTests::test_rolling_back
FAILED
tests/storage/test_room_search.py::EventSearchInsertionTest::test_null_byte
FAILED tests/storage/test_room_search.py::MessageSearchTest::test_sqlite_search
ERROR tests/federation/transport/server/test__base.py::test_disconnect
ERROR tests/http/test_servlet.py::test_disconnect
ERROR tests/replication/http/test__base.py::test_disconnect
ERROR tests/test_server.py::test_disconnect
= 11 failed, 3154 passed, 303 skipped, 205 warnings, 4 errors in 3730.21s
(1:02:10) =
Landry
