Re: [PATCH v4] app/test: add support for skipping tests
Thanks, this should help greatly going forward in the community lab. As it relates to our arm64 unit testing, I will give it a few days (or longer if needed) for next branches to rebase off of main and then re-enable arm64 unit testing with the eal_flags_file_prefix_autotest added to the skipped list. David explained to me on slack that this patch would not likely be a candidate for backporting, so of course LTS will be excluded.
Re: [PATCH v6 1/3] eal: add x86 cpuid support for monitorx
Recheck-request: iol-unit-amd64-testing Failed for service_autotest on windows. I'm doing a retest to see if it's reproducible.
Re: [PATCH v4] app/test: add support for skipping tests
On Wed, Oct 4, 2023 at 11:13 AM Aaron Conole wrote: > Patrick Robb writes: > > > Thanks, this should help greatly going forward in the community lab. > > > > As it relates to our arm64 unit testing, I will give it a few days (or > longer if needed) for next branches to rebase off of > > main and then re-enable arm64 unit testing with the > eal_flags_file_prefix_autotest added to the skipped list. David > > explained to me on slack that this patch would not likely be a candidate > for backporting, so of course LTS will be > > excluded. > > This is in testing area, and maybe it can be considered as an exception > if it allows for improved LTS testing. CC'd Kevin. > > Hello, Yes, backporting would be ideal from a CI perspective because without it we can't run arm64 testing on LTS tests. But I know there are other considerations which also have to be weighed. David also has a patch[1] which should resolve the underlying issue which introduces the failures on the unit test we want to skip. If that patch is accepted, and backported, fixing our original problem with unit testing on our arm testbeds, that's another solution, at least for this specific unit test issue. It would still be nice to have this feature in case we need it otherwise. [1] https://patches.dpdk.org/project/dpdk/patch/20230821085806.3062613-4-david.march...@redhat.com/
Re: [PATCH] eal: remove return from functions declared void return type
Recheck-request: iol-broadcom-Functional Sorry, this series was affected by an infra failure today.
Re: [PATCH v4] app/test: add support for skipping tests
On Mon, Oct 9, 2023 at 4:03 PM Patrick Robb wrote: > >> Hello, > > Yes, backporting would be ideal from a CI perspective because without it > we can't run arm64 testing on LTS tests. But I know there are other > considerations which also have to be weighed. > > David also has a patch[1] which should resolve the underlying issue which > introduces the failures on the unit test we want to skip. If that patch is > accepted, and backported, fixing our original problem with unit testing on > our arm testbeds, that's another solution, at least for this specific unit > test issue. > > It would still be nice to have this feature in case we need it otherwise. > > [1] > https://patches.dpdk.org/project/dpdk/patch/20230821085806.3062613-4-david.march...@redhat.com/ > > Hi. just to close the loops on this, yes David's aforementioned patch did resolve the unit test failure which was preventing us from running fast-tests on our arm64 test beds. But, it is not (yet, at least) backported for LTS releases. Even if it were, having Bruce's patch here backported would mean the CI testing approach could be common across releases in situations where testcases have to be skipped. Anyways, whether it's possible or "worth it" is ultimately down to the community's bandwidth, but I didn't want to let the conversation lapse without an update, and raising what the benefits would be. In any case, thanks again Bruce for the rework, it's a great addition.
Re: [PATCH 2/2] build: fail if explicitly requested driver is unbuildable
ARM Ampere server test fails on this patch are lab infra-failures (I did some updates on the server yesterday) and they can be ignored.
Re: [PATCH v13 8/8] net/ntnic: adds socket connection to PMD
ARM Ampere server test fails on this patch are lab infra-failures (I did some updates on the server yesterday) and they can be ignored.
Re: [PATCH] maintainers: update email address
Recheck-request: iol-unit-amd64-testing A maintainers file change should not cause eal_flags_misc_autotest to fail, so I'm queuing a retest for this series.
Community CI Meeting Minutes - July 11, 2024
# July 11, 2024 Attendees 1. Patrick Robb 2. Juraj Linkeš 3. Alex Chapman 4. Tomas Durovec 5. Jeremy Spewock 6. Adam Hassick 7. Manit Mahajan 8. Dean Marx # Minutes = General Announcements * CFP for DPDK Summit closes July 21 * There was a call with AWS people to gauge interest in DPDK testing on their platform. * Their primary interest is in the functional testing * Performance testing is not ideal for them as their platform has a deep tech stack which may scale differently according to deployment conditions, so it’s hard to publish perf numbers which will hold true across all possible deployments. = CI Status - UNH-IOL Community Lab * Pending checks appear to be working fine so far: * Next step is deploying a watchdog which will report to the UNH team every time a patchseries crosses the 24 hour threshold without all its pending checks being superseded by PASS/FAIL. This watchdog script is written and ready to deploy. * Xueming Li (23.11 LTS maintainer) noticed that 23.11-staging branch was not triggering a test when he amended the latest commit and force pushed. UNH team did a little digging and found that the commits in question were appearing on the dpdk.org git server, but not the GitHub DPDK mirror. Our webhook for triggering LTS-staging runs is through GitHub, so this caused the failure to trigger CI. David is helping troubleshoot the GitHub mirror but this is not resolved yet. * Retests v2: * Targeting to deploy this by Monday, July 15 * Once this is deployed at UNH, make sure all scripts are submitted to dpdk-ci and help other labs deploy the updates for the retest framework (Zhoumin from Loongson has indicated interest) * OVS: * There was some discussion on the mailing list about OVS compile testing at UNH lab. This testing has been enabled, runs from the dpdk-latest branch on OVS, and should be catching and series which break ovs compile. - Intel Lab * Patrick will contact them in the future to ask about supporting the retest framework - Github Actions * None - Loongarch Lab * None = DTS Improvements & Test Development * The MTU setting bug for mlx5 which Jeremy reported is being fixed in this patch: https://patches.dpdk.org/project/dpdk/patch/20240708105931.1842134-1-dsosnow...@nvidia.com/ * DTS group had a call to sync a few days ago: * Testpmd context manager: Now split out from the scatter testsuite and submitted as an independent testsuite. Has some reviews and was tested at UNH. * Juraj Linkeš is going to take a look at this series and the improved interactive shell output gathering series, and will follow up with Thomas afterwards requesting a merge * Patrick Robbwill ping David on Slack asking about adding a DTS-next branch which the DTS group can use for framework patches * UNH team is going to focus a lot of their time over the next week to testing and reviewing the testsuites which are “ready,” and seeing if those can make RC3. * Blocklist * Mac_filter * Vlan * Jumboframes * Queue_start_stop/Dynamic_queue (this is one where we will want to add a capability for stopping queues, which can come in the next release) * “Is there a list of Test Suites currently being worked on, as to not do the same ones? I may have missed it in the last meeting.” -Alex * Working tracking spreadsheet: https://docs.google.com/spreadsheets/d/1KrAS0c08x16RddzmYm2RDR93lRYxI9_owfk-9sz6iaI/edit?gid=0#gid=0 * Should always make a ticket when starting a new testsuite: https://bugs.dpdk.org/buglist.cgi?quicksearch=component%3Adts&list_id=8059 = Any other business * The DTS calls are now rescheduled, happening at 13:00 UTC on Thursdays. * Next Meeting: July 25, 2024
Re: [PATCH v7 01/21] net/ntnic: add ethdev and makes PMD available
Hello Zhoumin, It looks like Loongarch CI failed to apply this patch, but it worked at the other labs and locally for Serhii when they were checked out to next-net. Maybe your CI did not choose next-net, the right branch? I remember you saying your CI is based on a fork of the dpdk-ci repo. Perhaps the pw_maintainers_cli.py script for choosing the right branch to apply on, is outdated in your fork. https://git.dpdk.org/tools/dpdk-ci/tree/tools/pw_maintainers_cli.py Can you take a look? Thanks. On Fri, Jul 12, 2024 at 5:49 AM Serhii Iliushyk wrote: > > Add initial ntnic ethdev skeleton and register PCI probe functions > Update documentation: Device description and feature list > > Signed-off-by: Serhii Iliushyk > --- > .mailmap | 1 + > MAINTAINERS| 7 > doc/guides/nics/features/ntnic.ini | 8 + > doc/guides/nics/index.rst | 1 + > doc/guides/nics/ntnic.rst | 39 > doc/guides/rel_notes/release_24_07.rst | 10 ++ > drivers/net/meson.build| 1 + > drivers/net/ntnic/meson.build | 18 ++ > drivers/net/ntnic/ntnic_ethdev.c | 49 ++ > 9 files changed, 134 insertions(+) > create mode 100644 doc/guides/nics/features/ntnic.ini > create mode 100644 doc/guides/nics/ntnic.rst > create mode 100644 drivers/net/ntnic/meson.build > create mode 100644 drivers/net/ntnic/ntnic_ethdev.c > > diff --git a/.mailmap b/.mailmap > index 552d79eaa6..aad8c552f6 100644 > --- a/.mailmap > +++ b/.mailmap > @@ -1307,6 +1307,7 @@ Sergey Madaminov > Sergey Mironov > Sergey Temerkhanov > Sergio Gonzalez Monroy > +Serhii Iliushyk > Seth Arnold > Seth Howell > Shachar Beiser > diff --git a/MAINTAINERS b/MAINTAINERS > index 533f707d5f..0359368981 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -855,6 +855,13 @@ F: drivers/net/octeon_ep/ > F: doc/guides/nics/features/octeon_ep.ini > F: doc/guides/nics/octeon_ep.rst > > +Napatech ntnic > +M: Christian Koue Muf > +M: Serhii Iliushyk > +F: drivers/net/ntnic/ > +F: doc/guides/nics/ntnic.rst > +F: doc/guides/nics/features/ntnic.ini > + > NVIDIA mlx4 > M: Matan Azrad > M: Viacheslav Ovsiienko > diff --git a/doc/guides/nics/features/ntnic.ini > b/doc/guides/nics/features/ntnic.ini > new file mode 100644 > index 00..9ceb75a03b > --- /dev/null > +++ b/doc/guides/nics/features/ntnic.ini > @@ -0,0 +1,8 @@ > +; > +; Supported features of the 'ntnic' network poll mode driver. > +; > +; Refer to default.ini for the full list of available PMD features. > +; > +[Features] > +Linux= Y > +x86-64 = Y > diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst > index 7bfcac880f..c14bc7988a 100644 > --- a/doc/guides/nics/index.rst > +++ b/doc/guides/nics/index.rst > @@ -53,6 +53,7 @@ Network Interface Controller Drivers > nfb > nfp > ngbe > +ntnic > null > octeon_ep > octeontx > diff --git a/doc/guides/nics/ntnic.rst b/doc/guides/nics/ntnic.rst > new file mode 100644 > index 00..249d83d511 > --- /dev/null > +++ b/doc/guides/nics/ntnic.rst > @@ -0,0 +1,39 @@ > +.. SPDX-License-Identifier: BSD-3-Clause > +Copyright(c) 2023 Napatech A/S > + > +NTNIC Poll Mode Driver > +== > + > +The NTNIC PMD provides poll mode driver support for Napatech smartNICs. > + > + > +Design > +-- > + > +The NTNIC PMD is designed as a pure user-space driver, and requires no > special > +Napatech kernel modules. > + > +The Napatech smartNIC presents one control PCI device (PF0). NTNIC PMD > accesses > +smartNIC PF0 via vfio-pci kernel driver. Access to PF0 for all purposes is > +exclusive, so only one process should access it. The physical ports are > located > +behind PF0 as DPDK port 0 and 1. > + > + > +Supported NICs > +-- > + > +- NT200A02 2x100G SmartNIC > + > +- FPGA ID 9563 (Inline Flow Management) > + > +All information about NT200A02 can be found by link below: > +https://www.napatech.com/products/nt200a02-smartnic-inline/ > + > +Limitations > +~~~ > + > +Kernel versions before 5.7 are not supported. Kernel version 5.7 added > vfio-pci > +support for creating VFs from the PF which is required for the PMD to use > +vfio-pci on the PF. This support has been back-ported to older Linux > +distributions and they are also supported. If vfio-pci is not required kernel > +version 4.18 is supported. > diff --git a/doc/guides/rel_notes/release_24_07.rst > b/doc/guides/rel_notes/release_24_07.rst > index e50afed0d5..332a959618 100644 > --- a/doc/guides/rel_notes/release_24_07.rst > +++ b/doc/guides/rel_notes/release_24_07.rst > @@ -154,6 +154,16 @@ New Features > >Added an API that allows the user to reclaim the defer queue with RCU. > > +* **Added Napatech ntnic experimental PMD driver.** > + > + * Added the experimental PMD driver > + > +- Ability to initialize the
Re: [PATCH v7 01/21] net/ntnic: add ethdev and makes PMD available
Hi Min Zhou, I am seeing that commit for next-net: https://git.dpdk.org/next/dpdk-next-net/commit/?id=a6c3ec342ee105e322ffdb21e810cdfd38455c62 If you try to manually apply it on next-net, does it work? Pasting the logs from our apply process below for context: ``` Trying to checkout branch: origin/next-net-for-main Checked out to next-net-for-main (a6c3ec342ee105e322ffdb21e810cdfd38455c62) Applying patch... Applying: net/ntnic: add ethdev and makes PMD available Applying: net/ntnic: add logging implementation Applying: net/ntnic: add minimal initialization for PCI device Applying: net/ntnic: add NT utilities implementation Applying: net/ntnic: add VFIO module Applying: net/ntnic: add basic eth dev ops to ntnic Applying: net/ntnic: add core platform structures Applying: net/ntnic: add adapter initialization Applying: net/ntnic: add registers and FPGA model for NapaTech NIC Applying: net/ntnic: add FPGA modules for initialization Applying: net/ntnic: add FPGA initialization functionality Applying: net/ntnic: add support of the NT200A0X smartNIC Applying: net/ntnic: add startup and reset sequence for NT200A0X Applying: net/ntnic: add clock profile for the NT200A0X smartNIC Applying: net/ntnic: add link management skeleton Applying: net/ntnic: add link 100G module ops Applying: net/ntnic: add generic NIM and I2C modules Applying: net/ntnic: add QSFP support Applying: net/ntnic: add QSFP28 support Applying: net/ntnic: add GPIO communication for NIMs Applying: net/ntnic: add physical layer control module Running test build... The Meson build system ```
Re: [PATCH V2] examples/pipeline: fix include path for rte_log.h
On Wed, Feb 14, 2024 at 3:17 PM Patrick Robb wrote: > > > > On Wed, Feb 14, 2024 at 3:00 PM Dumitrescu, Cristian > wrote: >> >> >> >> [Cristian] >> Yes, you are right, we do have a DTS test suite for the pipeline library. >> >> It would be great to run it automatically as part of the CI testing. Just a >> word of caution though: I am not the owner of those tests, and also not >> intimately familiar with them, so it might be hard to find volunteers to >> setup the test suite, that the only problem that I see unfortunately. But we >> should try and see if there are any issues at all. >> >> Thanks, >> Cristian > > > Okay, the setup doesn't look too burdensome anyhow so we should be alright > with just the team here at UNH. I'll let you know how it goes. Sorry I realized today I never provided an update here. Back in February I did wire a testbed according to the testplan, insalled the dependencies and gave it a run, which was not passing. It would have been possible to pick apart the testsuite, manually running the example app to see whether the issue laid in the testbed setup, testsuite implementation, or somewhere else. But, it would have taken time and the premise of this effort was that it would be a quick setup, so I gave up as our efforts should go towards making additions to new DTS now, instead of debugging old DTS. I'm sorry that there isn't pipeline coverage in CI currently though - something to target going forward.
Re: [PATCH v7 01/21] net/ntnic: add ethdev and makes PMD available
On Mon, Jul 15, 2024 at 10:38 PM zhoumin wrote: > > Hi Patrick, > > Thanks for giving the link of commit `a6c3ec342ee1`. > > However I cannot checkout this commit in the next-net repository because > the commit ID is not exist in the history of next-net repository. Could > you find it? > > It seems that the commit `a6c3ec342ee1` is nearly identical to the > commit `62edcfd6ea3c` except the minor change in the subjects as following: > > https://git.dpdk.org/next/dpdk-next-net/commit/?id=62edcfd6ea3c61639df68e4a315046d09f462e8c > Thanks Min Zhou, I see what you mean. I guess Ferruh amended a commit and force pushed to the next-net branch? Otherwise I'm not sure what happened. In any case, it looks like for the v9 submission you checked out to a valid commit on next-net, so I think we're good. Thanks for updating your dpdk-ci repo!
Re: [PATCH] common/mlx5: remove unneeded field when modify RQ table
Rerunning the CI Test for Broadcom Performance, which appears to be a false failure.
Re: Community CI Meeting Minutes - June 27, 2024
On Thu, Jun 27, 2024 at 4:52 PM Patrick Robb wrote: > # > Minutes > > = > General Announcements > * DPDK Summit in Montreal will be September 24-25: > https://www.dpdk.org/event/dpdk-summit-2024/ >* CFP closes July 21 Correction, CFP closes July 31, not 21. https://events.linuxfoundation.org/dpdk-summit/program/cfp/
DTS WG Meeting Minutes - July 18, 2024
July 18, 2024
#
Attendees
* Patrick Robb
* juraj.lin...@pantheon.tech
* paul.szczepa...@arm.com
* Jeremy Spewock
* Dean Marx
* Nicholas Pratte
* Alex Chapman
#
Minutes
=
General Announcements
* DPDK CFP deadline is 07/31
* UNH is going to propose a 10 minute talk
* Going to improve the statistics.txt which DTS outputs. Should be
able to largely do the same thing as the json files which old DTS
outputs.
* https://bugs.dpdk.org/show_bug.cgi?id=1363
* Patrick Robbto comment with any suggestions
* UNH is starting to experiment with the flow API and writing
testsuite(s) for it.
* Having some difficulty checking device capabilities. There is a
testpmd validate function which can be run for flow rules, but
behavior from NICs after applying flow rules seems unreliable - we
need to dig more and figure out whether our implementation is poor,
testpmd support for flow api is poor, driver flow rule validation is
lacking, or something else.
=
Patch discussions
* Thomas has indicated he will look at the DTS patches which are
ready(docs gen, testpmd context manager, interactive shell output
gathering) this week for RC3
* Docs generation
* Thomas is going to provide a review
* The part where we link the DTS api docs with the Doxygen landing
page needs to be reviewed, so there is a link from DPDK docs to the
DTS docs.
* Testpmd context manager
* Has been reviewed by everyone and is final
* Improve interactive shell output gathering
* Ready
* Capabilities patch
* Almost ready, but Juraj
* needs to add support for getting capabilities from show port info
* Show rxq info vs show rx_queue offload. Which is needed for
our testcase?
* There was an email send in the Spring about checking for
scatter capability
* Juraj is looking into the scatter capabilities more:
* Jeremy: The first testcase should be based on what the port
natively supports (maybe show rxq info {port} {queue}), and the second
testcase should be based on whether the scatter capability can be
offloaded to the NIC (show port {port} rx_offload capabilities)
* It should be possible to check capabilities for the flow API
too by using the testpmd flow rule validate command. Once we are
confident with the rte_flow testsuite, we can add this part
* Queue_Region testsuite:
* Tests some comments in testpmd for the i40e driver
* This uses a private API, and is exclusively for i40e
* The driver feature is not supported in DPDK
* It makes sense to avoid working on this testsuite for now, but we
can come back to it
* Blocklist:
* UNH needs to provide a tested-by tag and reviews tag once it’s
ran on our hardware
* Mac Filter:
* Jeremy has provided a review, reviews from others requested
* Dual VLAN
* Basically the same as the single VLAN testsuite which Dean has
submitted, but for 2 VLANs
* There was a testcase for extended VLANs in old DTS. And a comment
that says that extended VLANs makes filtering “work normally” on
i40e... it’s not clear what this means, but the VLAN strip, insert,
and filter functions should work without extended VLAN range.
* TPID support can come with a future version of this testsuite,
but would include implementation of a capability check
* Speed Capabilities:
* Alex is looking at this testsuite, but it relies on defining a
testsuite config file, something not needed for all other testsuites
written so far
* Dynamic queue is out waiting for review
* Jumbo frames
* Waiting on Ferruh to get back on the issue Nick was facing
* Hoping that he will be able to expand more on what
–max-pkt-len is trying to do and why it doesn’t completely work.
* The command seems to be shaving off certain number of bytes
from the MTU (seemingly in an attempt to account for different PMDs
supporting different ethernet overheads)
=
Bugzilla discussions
* None
=
Any other business
* Next meeting Aug 1, 2024
Community CI Meeting Minutes - July 25, 2024
#
July 25, 2024
Attendees
1. Patrick Robb
2. Jeremy Spewock
3. Juraj Linkeš
4. Aaron Conole
5. Adam Hassick
6. Dean Marx
7. Luca Vizzarro
8. Manit Mahajan
9. Paul Szczepanek
10. Tomas Durovec
#
Minutes
=
General Announcements
* Retests v2:
* When passing in a branch name, make sure it aligns with what
would be output by pw_maintainers_cli.py (i.e. “next-net” instead of
“next-net-for-main”)
* Depends-on:
* Reminder: UNH folks are working on adding depends-on to the
patchwork project itself, with a v1 submission up so far.
* Integer based series IDs vs series urls vs message IDs: will
either be patch urls or message ids to indicate the dependency.
* Aaron: Message ids are better for folks who are working from
their email clients and don’t want to have to load up patchwork.
* DPDK Summit Presentations:
* CFP deadline is Jul 31, 2024
* Luca has made a submission for: How to run DTS testsuites, how to
write a testsuite from scratch. Going to do an overview of the APIs
available for test writers.
* Patrick will do a submission for UNH Lab updates, including new
ci tools for the community, testing for projects which consume dpdk
(ovs, spdk), how (new) DTS will be used in the lab going forward.
=
CI Status
-
UNH-IOL Community Lab
* Retest Framework now supports the “rebase={branch}” parameter, which
will re-apply the series on a given branch before triggering retests
* The needed updates to create_series_artifact.py have been
submitted from Adam, and applied on the dpdk-ci repo by Aaron.
* Patrick Robbto work with other labs on making use of these updates
* The new AMD & Intel x86 servers are mounted, cabled etc. in the new
DPDK server rack. We’re moving over some of the NICs from the old
servers now, setting up the DTS configurations, etc.
* Running from Ubuntu 24.04
* AMD has reached out for donation agreement for donating 3 additional
servers to the lab.
-
Intel Lab
* None
-
Github Actions
* Upcoming work: Cirrus CI
-
Loongarch Lab
* Patrick needs to reach out to Min Zhou and start up the process for
adding recheck support
=
DTS Improvements & Test Development
* next-dts branch has been approved by tech board, with Juraj as maintainer.
* Testpmd context manager is merged to main
* Alex had a bugfix for the git revision flag merged:
https://patchwork.dpdk.org/project/dpdk/patch/20240719153445.277851-1-alex.chap...@arm.com/
* Jeremy has rebased the improvements for interactive shell output
gathering, and this should be applied.
* Scatter capability:
* Show rxq info: shows scatter on if MTU will not fit in a message
buf, shows off otherwise.
* Show rx_offload capability: This shows the offload capability,
regardless of any runtime configuration
* Mellanox appears to require that the –enable-scatter flag be
passed, others do not have this requirement
=
Any other business
* Patrick Robbneeds to add Tomas Durovec to the DTS call (he is only
invited to the CI call)
Re: How does CI system get updated?
Hi Stephen,
This is a UNH Lab system.
We review our systems for updates once every 4 months. The idea is we
do it early in each DPDK release's development cycle. So, we update
Dockerfiles (for container environments), we apply updates where
needed to persistent systems (for VMs, or baremetal servers).
Obviously the Mingw version for the windows system was a check we have
not been doing. Thank you for spotting this and letting us know.
We will apply the update and let you know when it's ready.
On Thu, Jul 25, 2024 at 11:03 AM Stephen Hemminger
wrote:
>
>
>
> This warning is due to a very old version of Mingw installed in CI system.
>
> 20 line log output for Windows Server 2019 (dpdk_mingw64_compile):
> In file included from ..\lib\net/rte_ip.h:21,
> from ../lib/net/rte_dissect.c:20:
> C:/mingw64/mingw64/x86_64-w64-mingw32/include/ws2tcpip.h:447:63: note:
> expected 'PVOID' {aka 'void *'} but argument is of type 'const uint8_t *'
> {aka 'const unsigned char *'}
> WINSOCK_API_LINKAGE LPCSTR WSAAPI InetNtopA(INT Family, PVOID pAddr, LPSTR
> pStringBuf, size_t StringBufSize);
> ~~^
> ../lib/net/rte_dissect.c:292:29: error: passing argument 2 of 'inet_ntop'
> discards 'const' qualifier from pointer target type
> [-Werror=discarded-qualifiers]
> inet_ntop(AF_INET6, ip6_hdr->dst_addr, dbuf, sizeof(dbuf));
> ~~~^~
> In file included from ..\lib\net/rte_ip.h:21,
> from ../lib/net/rte_dissect.c:20:
> C:/mingw64/mingw64/x86_64-w64-mingw32/include/ws2tcpip.h:447:63: note:
> expected 'PVOID' {aka 'void *'} but argument is of type 'const uint8_t *'
> {aka 'const unsigned char *'}
> WINSOCK_API_LINKAGE LPCSTR WSAAPI InetNtopA(INT Family, PVOID pAddr, LPSTR
> pStringBuf, size_t StringBufSize);
> ~~^
>
> It was fixed upstream in Mingw 4 years ago.
Re: How does CI system get updated?
Okay I understand better now how we ended up with an older mingw64
version. The DPDK Docs for windows compile direct folks over to
(https://sourceforge.net/projects/mingw-w64/files/) to get the
prebuilt binaries, but the latest toolchain published there is Mingw64
v8.*, whereas the current version is v11.*. So, when we upgraded to
the "latest" published version, we upgraded to that v8.* from years
ago. If you look at the mingw64 website downloads page
(https://www.mingw-w64.org/downloads/), it directs people over to
winlibs.com to download the prebuilt binaries for v11.
I have replaced the Windows Server 2019 CI VM's old mingw64 binaries
with the new (v11.*) ones downloaded from winlibs.com, and I see that
Stephen's patch now passes the compile test. I can issue a retest for
your series once I am all done making the update for the server 2022
machine too.
I guess this also raises the question of whether the DPDK docs for the
windows mingw64 compile process should be updated to point to
winlibs.com instead of sourceforge.net (only has the source code).
https://doc.dpdk.org/guides/windows_gsg/build_dpdk.html#option-2-mingw-w64-toolchain
On Thu, Jul 25, 2024 at 3:06 PM Patrick Robb wrote:
>
> Hi Stephen,
>
> This is a UNH Lab system.
>
> We review our systems for updates once every 4 months. The idea is we
> do it early in each DPDK release's development cycle. So, we update
> Dockerfiles (for container environments), we apply updates where
> needed to persistent systems (for VMs, or baremetal servers).
> Obviously the Mingw version for the windows system was a check we have
> not been doing. Thank you for spotting this and letting us know.
>
> We will apply the update and let you know when it's ready.
>
>
> On Thu, Jul 25, 2024 at 11:03 AM Stephen Hemminger
> wrote:
> >
> >
> >
> > This warning is due to a very old version of Mingw installed in CI system.
> >
> > 20 line log output for Windows Server 2019 (dpdk_mingw64_compile):
> > In file included from ..\lib\net/rte_ip.h:21,
> > from ../lib/net/rte_dissect.c:20:
> > C:/mingw64/mingw64/x86_64-w64-mingw32/include/ws2tcpip.h:447:63: note:
> > expected 'PVOID' {aka 'void *'} but argument is of type 'const uint8_t *'
> > {aka 'const unsigned char *'}
> > WINSOCK_API_LINKAGE LPCSTR WSAAPI InetNtopA(INT Family, PVOID pAddr, LPSTR
> > pStringBuf, size_t StringBufSize);
> > ~~^
> > ../lib/net/rte_dissect.c:292:29: error: passing argument 2 of 'inet_ntop'
> > discards 'const' qualifier from pointer target type
> > [-Werror=discarded-qualifiers]
> > inet_ntop(AF_INET6, ip6_hdr->dst_addr, dbuf, sizeof(dbuf));
> > ~~~^~
> > In file included from ..\lib\net/rte_ip.h:21,
> > from ../lib/net/rte_dissect.c:20:
> > C:/mingw64/mingw64/x86_64-w64-mingw32/include/ws2tcpip.h:447:63: note:
> > expected 'PVOID' {aka 'void *'} but argument is of type 'const uint8_t *'
> > {aka 'const unsigned char *'}
> > WINSOCK_API_LINKAGE LPCSTR WSAAPI InetNtopA(INT Family, PVOID pAddr, LPSTR
> > pStringBuf, size_t StringBufSize);
> > ~~^
> >
> > It was fixed upstream in Mingw 4 years ago.
DTS WG Meeting Minutes - August 1, 2024
August 1, 2024 # Attendees * Patrick Robb * Jeremy Spewock * Nicholas Pratte * Juraj Linkeš * Alex Chapman * Luca Vizzarro # Minutes = General Announcements * DPDK 24.07 is released * DPDK Summit CFP deadline has passed. Luca submitted a talk for setting up DTS and running testsuites. Patrick submitted a talk for CI Labs/community update and new CI testing coverage, tools, etc. available to DPDK developers. * Draft of DTS 24.11 roadmap: https://docs.google.com/document/d/1Rcp1-gZWzGGCCSkbEsigrd0-NoQmknv6ZS7V2CPdgFo/edit * Do we want to add DTS items to release notes? * What is the process for building the release notes? Should the people working on DTS aggregate the updates and submit them? Patrick Robbshould check the website for the release notes process, or email the dev mailing list asking. * Dpdk-next-dts has been set up: https://git.dpdk.org/next/dpdk-next-dts/ * We should continue to submit patches like normal. Juraj will apply the patches to next-dts as needed. * Once the patches are in next DTS for-main branch, they can be pulled into DPDK main at any time by Thomas/David * Some AWS engineers are invited to the CI call for next week. * Will talk about reporting publicly using their internal tooling * Will talk about reporting using dpdk-test from AWS * Will (possibly) talk about DTS at AWS = Patch discussions * Adding support for DTS creating device VFs: * Shouldn’t need any additions to the config file. * There can be a capability associated with this, i.e. this testsuite requires VFs, then we run “dpdk-devbind.py -s” and parse out the output for available VFs. * Method for creating the VFs can return the list of VFs which can be stored in the testsuite object as VF ports * Ipv4_reassembly: not eth_dev, will not be implemented in new DTS. * Patrick Robbneeds to strike through this row, and give Luca edit access on the spreadsheet * We need to add more testsuites to the list. Dean Marxand Patrick Robbshould sync on this to discuss the new list that Dean put together, and find some more cases. * We often ship testsuites patches and testpmd shell additions together, and the testpmd shell addition gets locked up for an extended period of time as the testsuite gets reviews. If we spit out the testsuite additions and the framework additions, we can merge the framework additions quickly into the dts staging branch. For now, we can ship testsuite patches and their respective framework updates together, and split them out if we reach the 2nd half of a release cycle and we need to start moving. = Bugzilla discussions * Looking into submitting a bugzilla ticket for mlx5 NICs not distinguishing between having a bad checksum and having no checksum in testpmd verbose output. = Any other business * Next meeting Aug 15, 2024
Re: [PATCH v2 0/7] record and rework component dependencies
I saw there was one fail for this series for the Marvell MVNETA container. I thought I should also mention that the build did fail when our automation tried to run DTS on the Octeon CN10K DPU (which is also Marvell). So, that is why you won't get a report for the CN10K DTS results. Let me know if you can't get what you need from the MVNETA build logs failure, and I can manually get more info about why this won't compile. Cheers.
Re: [OS-Team] [dpdklab] Re: [PATCH] version: 24.11-rc0
Thanks I will merge the change disabling ABI checks at UNH at start of day tomorrow. On Thu, Aug 8, 2024 at 8:56 AM David Marchand wrote: > > On Thu, Aug 8, 2024 at 2:00 PM Thomas Monjalon wrote: > > > > 08/08/2024 10:03, David Marchand: > > > Start a new release cycle with empty release notes. > > > > > > The ABI version becomes 25.0. > > > The map files are updated to the new ABI major number (25). > > > The ABI exceptions are dropped and CI ABI checks are disabled because > > > compatibility is not preserved. > > > > > > Signed-off-by: David Marchand > > Acked-by: Thomas Monjalon > > Applied, thanks. > > Heads up to CI: ABI checks must be disabled during v24.11 release. > > > -- > David Marchand >
Community CI Meeting Minutes - August 8, 2024
# August 8, 2024 Attendees 1. Patrick Robb 2. Juraj Linkeš 3. Aaron Conole 4. Alex Chapman 5. Dean Marx 6. Jeremy Spewock 7. Luca Vizzarro 8. Nicholas Pratte 9. Paul Szczepanek 10. Manit Mahajan # Minutes = General Announcements * Testing presentations at DPDK Summit: * UNH Lab Update and CI tooling updates talk was approved. Patrick and Adam Hassick will give this talk * Luca’s proposal for a DTS talk was approved, but the remote presentation option was rejected. So, Patrick and (Nick or Dean) will give this presentation. * AWS CI Testing initiative status? * What testing tooling/frameworks can AWS people use, for CI testing and reporting to the mailing list? * 1. They mentioned they have some internal DPDK test tools * 2. dpdk-test/meson test from the DPDK repo * 3. DTS (would be most appropriate in a couple months once more testsuites are merged to DPDK and we can figure out running the framework in a cloud environment) * What utilities do they need in order to setup CI? * Each lab has a different automation framework setup (UNH uses Jenkins, for instance) but the dpdk-ci repo (https://git.dpdk.org/tools/dpdk-ci/) contains bash/python scripts which can be used to stand up the CI process regardless of the automation tool in use: * downloading patches from DPDK patchwork (download-patch.sh) * UNH also wrote a python equivalent for downloading patchseries, which we can publish to dpdk-ci if there is interest. * Making a guess of what the correct DPDK branch to apply on is, based on files modified (https://git.dpdk.org/tools/dpdk-ci/tree/tools/pw_maintainers_cli.py) * (if needed) Apply patches to DPDK with git-pw and create a tarball of the resulting DPDK artifact to test on (https://git.dpdk.org/tools/dpdk-ci/tree/tools/create_series_artifact.py) * Send test report to the DPDK test-report mailing list (https://git.dpdk.org/tools/dpdk-ci/tree/tools/send-patch-report.sh) * And more utilities, which may be needed depending on the specific infra/process you decide to stand up * There are some people wanting to visit the UNH lab. Need to resurrect the mailing list conversation. * Most people will probably try to leave the morning of the 26th and drive to UNH (5 hours), and can arrive early afternoon at UNH = CI Status - UNH-IOL Community Lab * Windows Server testbeds: Tyler from Microsoft let us know that we can discontinue usage of the 2019 server. So, we will only run from server 2022 now, we just need to verify we are running all 3 compile toolchains on server 2022. As a reminder, they are: * LLVM 14.0.0 (or later) and Microsoft MSVC linker. * Microsoft Visual Studio 2022 (any edition). * The MinGW-w64 10.0 (or later) toolchain (native). * One more note about Windows compile jobs is that the prebuilt mingw64 binaries are not longer available at Sourceforge. It is necessary to download them at https://winlibs.com/ * Depends-on support on Patchwork: submitted a v2 for the series * https://patchwork.ozlabs.org/project/patchwork/list/?submitter=88738 * Pending reporting: * This is working, but we have gotten complaints about spamming the mailing list with too many reports. The reason is that we send a new report every time a testing job finishes for a particular environment (so we add a new row to the results table for a patchwork context). The community has requested that we change to a model of “send a report when testing for a patchwork context begins, then as tests finish, only send a new report if the overall test status for the context has changed, then send a report when all testing for the patchwork context is complete.” We are making these updates now. * Testbed maintenance: Updating baremetal systems, firmware, VMs, containers definitions via dockerfiles, distro upgrades. We do this at the start of every release cycle. * DPDK Dashboard: Have made some improvements to how we query for test results, and started caching some requests we make to our results REST API, which have improved performance quite a bit. Still not lightning fast, but will no longer get timeout errors trying to load the results detail view for a patchseries etc. - Intel Lab * Juraj’s DTS API doc generation patch failed Intel’s CI, due to “-- exclude=doc/* *” build parameter being used. They have stated this is done to provide CI stability, and suggested in this case maintainers make an exception and merge the patch without a pass from Intel lab. Conversation is ongoing on the mailing list here, but Juraj
Re: [PATCH] net/bonding: add user callback for bond xmit policy
Recheck-request: iol-marvell-Functional Putting in a retest for this.
DTS WG Meeting Minutes - August 15, 2024
#
August 15, 2024
Attendees
* Patrick Robb
* Jeremy Spewock
* Alex Chapman
* Juraj Linkeš
* Tomas Durovec
* Dean Marx
* Luca Vizzarro
* Paul Szczepanek
* Nicholas Pratte
#
Minutes
=
General Discussion
* DTS Roadmap:
https://docs.google.com/document/d/1Rcp1-gZWzGGCCSkbEsigrd0-NoQmknv6ZS7V2CPdgFo/edit
* Will email out after this meeting
* Speakers are all signed up for the CI and DTS talks at DPDK Summit
=
Patch discussions
* Testpmd shell method names: should they align with existing testpmd
runtime commands? I.e. should the “flow create” runtime command be
implemented via a method named flow_create_*() or the more english
intuitive create_flow_*()
* One option is to implement both, and have one method call the other
* This potentially creates confusion as people read different
testsuites and see different functions used, not realizing they may be
the same
* The group agrees it is best to name methods in a human readable
intuitive way… so like create_flow_*() from the example above.
* Testpmd verbose parser
* If we read port from testpmd to identify packets, they must have
a tcp/udp layer, which may be limiting. If, for whatever reason,
packets for a testsuite cannot be built with a l4, individual
testsuites may have to check based on src mac address, checksum etc.
* In almost all cases, packets can be build with a l4
* Checksum offload suite is submitted
* Dependency on the existing testpmd verbose parser
* RX side testcases work fine, but TX side behavior is not aligning
with what is described in the testsuite, so feedback on this is
appreciated
* Checksum offload command
* Csum set {layer name} hw {port number}
* Returns sctp offload is not supported
* TCP/UDP packets are working
* Port assignment:
* Physical ports are defined in the nodes conf section, then port
ids are referred to in the testrun config
* Also includes splitting the nodes and testrun configs into different files
* Discussion on ticket regarding having a conf directory to contain these
* Still some work to be done removing unneeded configuration from conf.yaml
* VXLAN-GPE testsuite is now canceled as the feature is removed as of DPDK 24.07
* API Docs
* Juraj needs reviews and testing
* UNH people please rebuild the docs and provide your experience
* Should specifically test meson install
* Aim is to make it simple to use (and it is)
* It builds with DPDK docs
* L2fwd
* Jeremy provided a review, more people at UNH please run this and
provide feedback
* When reviewing people should also review the dependency - add
pktgen and testpmd change series
* Tomas and Juraj have begun work on producing the testrun results json
=
Bugzilla discussions
* None
=
Any other business
* Next meeting Aug 29, 2024
Community CI Meeting Minutes - August 22, 2024
# August 22, 2024 Attendees 1. Patrick Robb 2. Tomas Durovec 3. Dean Marx 4. Luca Vizzarro 5. Jeremy Spewock 6. Ali Alnubani 7. Juraj Linkeš 8. Paul Szczepanek # Minutes = General Announcements * DPDK Summit Montreal: September 24-25 * Some Summit attendees will be visiting the UNH Community Lab on September 26. = CI Status - UNH-IOL Community Lab * openssl_asym_autotest driver test is failing on opensuse Leap * Submitted a bugzilla ticket with testlogs:https://bugs.dpdk.org/show_bug.cgi?id=1523 * Disabling this in CI testing in the interim * New Community Lab Servers: * NVIDIA ConnectX6-Lx and ConnectX7 NICs were moved to the new servers. The single core performance testing appears good so far (much better than with the old servers, and having rough parity with the performance reports NVIDIA publishes on dpdk.org) * Patrick is experimenting with some more of the AMD specific optimization system configs: * https://doc.dpdk.org/guides/linux_gsg/amd_platform.html * https://www.amd.com/system/files/documents/58017-amd-epyc-9004-tg-data-plane-dpdk.pdf * Linux Foundation has provided legal approval for AMD to donate 3 additional used servers to the Community Lab, so those will be arriving soon * Deployed the update for sending fewer emails to the test-report mailing list (now we only send an extra email when the overall test label status updates, or when testing for a patchwork label is 100% done) * DTS Check format check: * Going to run that in CI, should be able to do this by next week * Also going to start running meson check format with the same process - Intel Lab * None - Github Actions * None - Loongarch Lab * None = DTS Improvements & Test Development * MTU update testpmd methods: * Nick will submit a standalone patch for this, Jeremy and Juraj will depend on this * Luca’s pktgen and testpmd change patch includes an implementation for starting and stopping ports. Everyone should use this method once the series is reviewed and merged. http://patches.dpdk.org/project/dpdk/patch/20240806124642.2580828-5-luca.vizza...@arm.com/ * VF creation: * Host must have sr-iov setup * PF is bound to the kernel driver (os_driver in dts) * Is it important to be non-destructive towards existing VFs when DTS starts? * People may not expect for DTS to be touching existing system configuration, and could cause confusion for people who run DTS in terms of their system state after running DTS * This is only relevant for bifurcated drivers (mlx5_core) * Interfaces need to be up (broadcom) * https://bugs.dpdk.org/show_bug.cgi?id=1500 * Pending Testsuites: * Some tests require adding functionality to testpmd * Interrupt_pmd test plan is written from l3fwd-power which has some hooks for checking wakeups, would need to patch testpmd to fetch the same info * https://doc.dpdk.org/dts/test_plans/interrupt_pmd_test_plan.html * MTU Update is assigned to Alex * There is probably some discussion on the mailing list about this suite from years ago, when it was returning inconsistent behavior across NICs based on assumptions around byte allocations for VLANs * Might be the exact same issue as arose when the jumboframes testsuite was written a couple months ago * Patrick will search for the old mail threads * Patrick and UNH team need to add more testsuite ideas to the tracking spreadsheet: https://docs.google.com/spreadsheets/d/1KrAS0c08x16RddzmYm2RDR93lRYxI9_owfk-9sz6iaI/edit?gid=0#gid=0 * Dts-next branch. * Paul has been joining the DPDK release meetings, with ARM updates, and can give dts-next updates too (what is being merged, what are the important considerations). * We should summarize the high level for the patches submitted to the next-dts branch every week, and then Paul can bring that information to the release meeting if needed. * Next-dts summary: * It is not being used yet, but we are going to start now with the patches which are on patchwork * So, for example, Luca’s pktgen and testpmd updates series would be one of the first things which Juraj will add * Juraj (Next-DTS Maintainer) will be on vacation next week, but this will be used after * DTS Docs: * Dependency issue: pyyaml was a missing dependency, causing docs build to fail * .toml file for dependencies uses distro name for py
Re: [PATCH v1] net/ice: fix incorrect reading of PHY timestamp
Recheck-request: iol-marvell-Functional
On Fri, Aug 23, 2024 at 7:56 AM Soumyadeep Hore
wrote:
>
> In E830 adapters, PHY timestamp for Tx packets should be read once
> the ready status of PHY timestamp registers is 1.
>
> Fixes: 881169950d80 ("net/ice/base: implement initial PTP support for E830")
> Cc: sta...@dpdk.org
>
> Signed-off-by: Soumyadeep Hore
> ---
> drivers/net/ice/base/ice_ptp_hw.c | 68 ---
> 1 file changed, 44 insertions(+), 24 deletions(-)
>
> diff --git a/drivers/net/ice/base/ice_ptp_hw.c
> b/drivers/net/ice/base/ice_ptp_hw.c
> index 004f659eae..41367105b2 100644
> --- a/drivers/net/ice/base/ice_ptp_hw.c
> +++ b/drivers/net/ice/base/ice_ptp_hw.c
> @@ -5526,6 +5526,27 @@ ice_ptp_port_cmd_e830(struct ice_hw *hw, enum
> ice_ptp_tmr_cmd cmd,
>lock_sbq);
> }
>
> +/**
> + * ice_get_phy_tx_tstamp_ready_e830 - Read Tx memory status register
> + * @hw: pointer to the HW struct
> + * @port: the PHY port to read
> + * @tstamp_ready: contents of the Tx memory status register
> + *
> + */
> +static int
> +ice_get_phy_tx_tstamp_ready_e830(struct ice_hw *hw, u8 port, u64
> *tstamp_ready)
> +{
> + u64 hi;
> + u32 lo;
> +
> + lo = rd32(hw, E830_PRTMAC_TS_TX_MEM_VALID_L);
> + hi = (u64)rd32(hw, E830_PRTMAC_TS_TX_MEM_VALID_H) << 32;
> +
> + *tstamp_ready = hi | lo;
> +
> + return 0;
> +}
> +
> /**
> * ice_read_phy_tstamp_e830 - Read a PHY timestamp out of the external PHY
> * @hw: pointer to the HW struct
> @@ -5539,10 +5560,30 @@ ice_ptp_port_cmd_e830(struct ice_hw *hw, enum
> ice_ptp_tmr_cmd cmd,
> static int
> ice_read_phy_tstamp_e830(struct ice_hw *hw, u8 lport, u8 idx, u64 *tstamp)
> {
> - u32 hi_addr = E830_HIGH_TX_MEMORY_BANK(idx, lport);
> - u32 lo_addr = E830_LOW_TX_MEMORY_BANK(idx, lport);
> + u32 hi_addr, lo_addr;
> u32 lo_val, hi_val, lo;
> - u8 hi;
> + u8 hi, ret;
> + u64 start_time, curr_time;
> + u64 tstamp_ready = 0;
> +
> + start_time = rte_get_timer_cycles() / (rte_get_timer_hz() / 1000);
> +
> + /* To check the ready status of HY Timestamp register for fetching
> timestamp */
> + while (!(tstamp_ready & BIT_ULL(0))) {
> + ret = ice_get_phy_tx_tstamp_ready_e830(hw, lport,
> &tstamp_ready);
> + if (ret) {
> + PMD_DRV_LOG(ERR, "Failed to get phy ready for
> timestamp");
> + return -1;
> + }
> + curr_time = rte_get_timer_cycles() / (rte_get_timer_hz() /
> 1000);
> + if (curr_time - start_time > 1000) {
> + PMD_DRV_LOG(ERR, "Timeout to get phy ready for
> timestamp");
> + return -1;
> + }
> + }
> +
> + hi_addr = E830_HIGH_TX_MEMORY_BANK(idx, lport);
> + lo_addr = E830_LOW_TX_MEMORY_BANK(idx, lport);
>
> lo_val = rd32(hw, lo_addr);
> hi_val = rd32(hw, hi_addr);
> @@ -5558,27 +5599,6 @@ ice_read_phy_tstamp_e830(struct ice_hw *hw, u8 lport,
> u8 idx, u64 *tstamp)
> return 0;
> }
>
> -/**
> - * ice_get_phy_tx_tstamp_ready_e830 - Read Tx memory status register
> - * @hw: pointer to the HW struct
> - * @port: the PHY port to read
> - * @tstamp_ready: contents of the Tx memory status register
> - *
> - */
> -static int
> -ice_get_phy_tx_tstamp_ready_e830(struct ice_hw *hw, u8 port, u64
> *tstamp_ready)
> -{
> - u64 hi;
> - u32 lo;
> -
> - lo = rd32(hw, E830_PRTMAC_TS_TX_MEM_VALID_L);
> - hi = (u64)rd32(hw, E830_PRTMAC_TS_TX_MEM_VALID_H) << 32;
> -
> - *tstamp_ready = hi | lo;
> -
> - return 0;
> -}
> -
> /* Device agnostic functions
> *
> * The following functions implement shared behavior common to both E822/E823
> --
> 2.43.0
>
Re: [PATCH v3 00/12] Align ICE shared code with Base driver
Recheck-request: iol-marvell-Functional On Fri, Aug 23, 2024 at 6:51 AM Soumyadeep Hore wrote: > > Updating the latest shared code patches to ICE base driver. > > --- > v3: > - Addressed comments givn by reviewer > --- > v2: > - Addressed comments given by reviewer > - Corrected errors in Camel Case > --- > > Dan Nowlin (2): > net/ice: correct Tx Scheduler AQ command RD bit for E825C > net/ice: support optional flags in signature segment header > > Fabio Pricoco (1): > net/ice: update iteration of TLVs in Preserved Fields Area > > Jacob Keller (1): > net/ice: avoid reading past end of PFA > > Norbert Zulinski (2): > net/ice: updates for ptp init in E825C > net/ice: update PTP init > > Oleg Akhrem (1): > net/ice: address compilation errors > > Paul Greenwalt (3): > net/ice: add new tag definitions > net/ice: fix link speed for 200G > net/ice: update E830 50G branding strings > > Przemyslaw Gierszynski (1): > net/ice: add support for FEC auto-detect for E830 > > Yogesh Bhosale (1): > net/ice: use correct format specifiers for unsigned ints > > drivers/net/ice/base/ice_adminq_cmd.h | 2 +- > drivers/net/ice/base/ice_cgu_regs.h | 19 + > drivers/net/ice/base/ice_common.c | 66 > drivers/net/ice/base/ice_ddp.c| 31 +++- > drivers/net/ice/base/ice_ddp.h| 5 +- > drivers/net/ice/base/ice_devids.h | 12 +-- > drivers/net/ice/base/ice_hw_autogen.h | 14 > drivers/net/ice/base/ice_nvm.c| 36 ++--- > drivers/net/ice/base/ice_ptp_consts.h | 75 ++ > drivers/net/ice/base/ice_ptp_hw.c | 106 +- > drivers/net/ice/base/ice_ptp_hw.h | 21 + > drivers/net/ice/ice_ethdev.c | 6 +- > 12 files changed, 285 insertions(+), 108 deletions(-) > > -- > 2.43.0 >
Re: [PATCH v3 00/10] dts: add hello world testcase
sion/{ => remote}/ssh_session.py (65%)
>> delete mode 100644 dts/framework/remote_session/remote_session.py
>> create mode 100644 dts/framework/test_result.py
>> create mode 100644 dts/framework/test_suite.py
>> create mode 100644 dts/framework/testbed_model/dpdk.py
>> create mode 100644 dts/framework/testbed_model/hw/__init__.py
>> create mode 100644 dts/framework/testbed_model/hw/cpu.py
>> create mode 100644 dts/framework/testbed_model/hw/virtual_device.py
>> create mode 100644 dts/framework/testbed_model/sut_node.py
>> create mode 100644 dts/test_plans/hello_world_test_plan.rst
>> create mode 100644 dts/tests/TestSuite_hello_world.py
>>
>> --
>> 2.30.2
>>
>>
Tested-by: Patrick Robb
--
Patrick Robb
Technical Service Manager
UNH InterOperability Laboratory
21 Madbury Rd, Suite 100, Durham, NH 03824
www.iol.unh.edu
Re: [PATCH v1] dts: add Dockerfile
t-get -y install --no-install-recommends \
> > +vim emacs git
> > diff --git a/dts/README.md b/dts/README.md new file mode 100644 index
> > 00..fd9f32595c
> > --- /dev/null
> > +++ b/dts/README.md
> > @@ -0,0 +1,55 @@
> > +# DTS Environment
> > +The execution and development environments for DTS are the same, a
> > +[Docker](https://docs.docker.com/) container defined by our
> > [Dockerfile](./Dockerfile).
> > +Using a container for the development environment helps with a few
> > things.
> > +
> > +1. It helps enforce the boundary between the DTS environment and the
> > TG/SUT, something
> > + which caused issues in the past.
> > +2. It makes creating containers to run DTS inside automated tooling much
> > easier, since
> > + they can be based off of a known-working environment that will be
> > updated as DTS is.
> > +3. It abstracts DTS from the server it is running on. This means that
> the bare-
> > metal os
> > + can be whatever corporate policy or your personal preferences
> dictate,
> > and DTS does
> > + not have to try to support all distros that are supported by DPDK CI.
> > +4. It makes automated testing for DTS easier, since new dependencies
> > +can be sent in with
> > + the patches.
> > +5. It fixes the issue of undocumented dependencies, where some test
> > suites require
> > + python libraries that are not installed.
> > +6. Allows everyone to use the same python version easily, even if they
> are
> > using a
> > + distribution or Windows with out-of-date packages.
> > +7. Allows you to run the tester on Windows while developing via Docker
> for
> > Windows.
> > +
> > +## Tips for setting up a development environment
> > +
> > +### Getting a docker shell
> > +These commands will give you a bash shell inside the container with all
> > +the python dependencies installed. This will place you inside a python
> > +virtual environment. DTS is mounted via a volume, which is essentially a
> > symlink from the host to the container.
> > +This enables you to edit and run inside the container and then delete
> > +the container when you are done, keeping your work.
> > +
> > +```shell
> > +docker build --target dev -t dpdk-dts .
> > +docker run -v $(pwd)/..:/dpdk -it dpdk-dts bash $ poetry install $
> > +poetry shell ```
> > +
> > +### Vim/Emacs
> > +Any editor in the ubuntu repos should be easy to use, with vim and
> > +emacs already installed. You can add your normal config files as a
> > +volume, enabling you to use your preferred settings.
> > +
> > +```shell
> > +docker run -v ${HOME}/.vimrc:/root/.vimrc -v $(pwd)/..:/dpdk -it
> > +dpdk-dts bash ```
> > +
> > +### Visual Studio Code
> > +VSCode has first-class support for developing with containers. You may
> > +need to run the non-docker setup commands in the integrated terminal.
> > +DTS contains a .devcontainer config, so if you open the folder in
> > +vscode it should prompt you to use the dev container assuming you have
> > +the plugin installed. Please refer to [VS Development Containers
> > +Docs](https://code.visualstudio.com/docs/remote/containers)
> > +to set it all up.
> > +
> > +### Other
> > +Searching for '$IDE dev containers' will probably lead you in the right
> > direction.
> > --
> > 2.30.2
>
> Tested-by: Patrick Robb
--
Patrick Robb
Technical Service Manager
UNH InterOperability Laboratory
21 Madbury Rd, Suite 100, Durham, NH 03824
www.iol.unh.edu
Observed issues with the FIPS sample app
Hello all, At the UNH Community Lab, we are now running CI testing for cryptodev validation using the FIPS sample application. In setting up testing we have run into issues with the sample app documentation being out of date. In places it points to dependency versions which do not work with the sample app, and we are also seeing the sample application failing to run on some of the supported test vectors. We hope to start a conversation about these issues with interested community members responsible for maintaining cryptodev functions in DPDK and the FIPS sample application. If desired, we could produce an updated setup guide according to how we’ve set up our environment. But, others who are more involved in developing the sample app may want to provide their own input. We are grateful for any feedback to us or development of the sample app which may come out of this conversation. A synopsis of our experience using the FIPS sample app is below. *Issues with current documentation for IPsec-mb library:* According to the current test plan documentation for how to utilize the DPDK FIPS Validation Application ( https://git.dpdk.org/tools/dts/tree/test_plans/fips_cryptodev_test_plan.rst ), it is recommended that you pull from the intel-ipsec-mb github repository and then checkout a specific commit (noted as “latest working commit”). The commit it recommends using in this documentation is a few commits after the version tagged “v0.50” however, when trying to build DPDK using this version of the library, it reports that it does not support any version older than “1.0.0”. When using the documented version of the library, you also do not get access to the virtual crypto device that is used in the examples on this page. There is also the DPDK user guide for the application ( *https://doc.dpdk.org/guides/sample_app_ug/fips_validation.html* <https://doc.dpdk.org/guides/sample_app_ug/fips_validation.html>) and this guide does not list any of those same prerequisites, but when building DPDK without said prerequisites, you receive the driver warning *crypto/ipsec_mb: missing dependency, "libIPSec_MB”.* There is a different crypto device used in the examples on this user guide (*crypto_aesni_mb*) but if you do build the IPsec library from github, on the “0.50” version you don’t get access to this device. However, if you instead build the IPsec library on the latest tagged version (v1.3) you can run the sample application using the crypto device that is mentioned on this page. Even with being able to run the sample application on the latest version, not all supported algorithms pass, there are a few algorithms listed as supported that return a failed verdict. It is unclear which steps should be followed and which parts are accurate as it seems some pieces of each form of documentation are true while others are outdated. *Other things to note:* There are other algorithms that pass on some hosts, but fail on others. Using an ubuntu 20.04 container, on some hosts, validation of AES-GCM algorithms works and returns a passing verdict. However, running inside the same container on our production hosts, these algorithms fail with the message “CipherText was not present in the TestCase.” There was recently a *patch* <http://mails.dpdk.org/archives/test-report/2023-February/350635.html> put out to fix the AES-GCM validation, however we are seeing it fail to compile with DPDK. *All algorithm testing listed below was performed using the “crypto_aesni_mb” virtual device on version 1.3 of the ipsec library* *List of working algorithms*: - AES-CBC - AES-CMAC - AES-CTR - AES-GMAC - HMAC-SHA1 - TDES-CBC *List of failing algorithms*: - AES-GCM - “CipherText missing from TestCase” - This failure message is found in the verdict returned from the ACVP API - AES-XTS - “ACVP-AES-XTS-2.0: General exception. Contact service provider.” - This failure happens before reaching DPDK application so likely a NIST problem - SHA-1 - “Digests do not match” - This failure message is found in the verdict returned from the ACVP API - Passes some test cases but fails others - SHA-256 - “Digests do not match” - This failure message is found in the verdict returned from the ACVP API - Like SHA-1, fails some but passes others - TDES-ECB - “USER1: Failed to get capability for cdev 0” - “USER1: Error -22: test block” - These errors are encountered during the validation phase (i.e., when the vectors are being run through the DPDK sample application) *Untested algorithms*: - RSA - ECDSA -- Patrick Robb Technical Service Manager UNH InterOperability Laboratory 21 Madbury Rd, Suite 100, Durham, NH 03824 www.iol.unh.edu
Re: [PATCH v2] vhost: fix madvise arguments alignment
(vq->desc, len, true);
> > > - mem_set_dump(vq->avail, len, true);
> > > - mem_set_dump(vq->used, len, true);
> > > vq->access_ok = true;
> > >
> > > VHOST_LOG_CONFIG(dev->ifname, DEBUG, "mapped address desc:
> %p\n", vq->desc);
> > > @@ -1230,7 +1268,8 @@ vhost_user_mmap_region(struct virtio_net *dev,
> > > region->mmap_addr = mmap_addr;
> > > region->mmap_size = mmap_size;
> > > region->host_user_addr = (uint64_t)(uintptr_t)mmap_addr +
> mmap_offset;
> > > - mem_set_dump(mmap_addr, mmap_size, false);
> > > + region->alignment = alignment;
> > > + mem_set_dump(mmap_addr, mmap_size, false, alignment);
> > >
> > > if (dev->async_copy) {
> > > if (add_guest_pages(dev, region, alignment) < 0) {
> > > @@ -1535,7 +1574,6 @@ inflight_mem_alloc(struct virtio_net *dev, const
> char *name, size_t size, int *f
> > > return NULL;
> > > }
> > >
> > > - mem_set_dump(ptr, size, false);
> > > *fd = mfd;
> > > return ptr;
> > > }
> > > @@ -1566,6 +1604,7 @@ vhost_user_get_inflight_fd(struct virtio_net
> **pdev,
> > > uint64_t pervq_inflight_size, mmap_size;
> > > uint16_t num_queues, queue_size;
> > > struct virtio_net *dev = *pdev;
> > > + uint64_t alignment;
> > > int fd, i, j;
> > > int numa_node = SOCKET_ID_ANY;
> > > void *addr;
> > > @@ -1628,6 +1667,8 @@ vhost_user_get_inflight_fd(struct virtio_net
> **pdev,
> > > dev->inflight_info->fd = -1;
> > > }
> > >
> > > + alignment = get_blk_size(fd);
> > > + mem_set_dump(addr, mmap_size, false, alignment);
> > > dev->inflight_info->addr = addr;
> > > dev->inflight_info->size = ctx->msg.payload.inflight.mmap_size =
> mmap_size;
> > > dev->inflight_info->fd = ctx->fds[0] = fd;
> > > @@ -1744,10 +1785,10 @@ vhost_user_set_inflight_fd(struct virtio_net
> **pdev,
> > > dev->inflight_info->fd = -1;
> > > }
> > >
> > > - mem_set_dump(addr, mmap_size, false);
> > > dev->inflight_info->fd = fd;
> > > dev->inflight_info->addr = addr;
> > > dev->inflight_info->size = mmap_size;
> > > + mem_set_dump(addr, mmap_size, false, get_blk_size(fd));
> > >
> > > for (i = 0; i < num_queues; i++) {
> > > vq = dev->virtqueue[i];
> > > @@ -2242,6 +2283,7 @@ vhost_user_set_log_base(struct virtio_net **pdev,
> > > struct virtio_net *dev = *pdev;
> > > int fd = ctx->fds[0];
> > > uint64_t size, off;
> > > + uint64_t alignment;
> > > void *addr;
> > > uint32_t i;
> > >
> > > @@ -2280,6 +2322,7 @@ vhost_user_set_log_base(struct virtio_net **pdev,
> > >* fail when offset is not page size aligned.
> > >*/
> > > addr = mmap(0, size + off, PROT_READ | PROT_WRITE, MAP_SHARED,
> fd, 0);
> > > + alignment = get_blk_size(fd);
> > > close(fd);
> > > if (addr == MAP_FAILED) {
> > > VHOST_LOG_CONFIG(dev->ifname, ERR, "mmap log base
> failed!\n");
> > > @@ -2296,7 +2339,7 @@ vhost_user_set_log_base(struct virtio_net **pdev,
> > > dev->log_addr = (uint64_t)(uintptr_t)addr;
> > > dev->log_base = dev->log_addr + off;
> > > dev->log_size = size;
> > > - mem_set_dump(addr, size, false);
> > > + mem_set_dump(addr, size + off, false, alignment);
> > >
> > > for (i = 0; i < dev->nr_vring; i++) {
> > > struct vhost_virtqueue *vq = dev->virtqueue[i];
> >
>
>
--
Patrick Robb
Technical Service Manager
UNH InterOperability Laboratory
21 Madbury Rd, Suite 100, Durham, NH 03824
www.iol.unh.edu
Re: [v1, 00/10] fips_validation application improvements
Hello Akhil and Gowrishankar, We saw the same issue with running the fips sample app under CI testing here at the UNH Community Lab: http://mails.dpdk.org/archives/test-report/2023-February/350635.html. We reported a warn because it failed on the compilation stage (as opposed to a failure of the actual sample app run). On the other hand, we are excited to see this patch again with the compilation part resolved. I sent an email to the dev mailing list a few weeks ago for our fips sample app CI testing regarding where we could and could not provide test vector coverage, and it appears this patch series may resolve our ciphertext issue with AES-GCM test vector and more issues with the sample app. So - looking forward to seeing a patch like this being merged when stable! Best, Patrick Robb On Tue, Feb 28, 2023 at 2:39 AM Akhil Goyal wrote: > Hi Gowrishankar, > > > > > > > Subject: [v1, 00/10] fips_validation application improvements > > > > > > > > This patch series adds support for SHA3, SHAKE, AES-CCM JSON test > vectors > > > > and fixes existing algorithms to support NIST test vectors. > > > > > > > > Gowrishankar Muthukrishnan (10): > > > > examples/fips_validation: fix MCT output for SHA > > > > examples/fips_validation: add SHA3 validation > > > > examples/fips_validation: fix integer parse in test case > > > > examples/fips_validation: add SHAKE validation > > > > examples/fips_validation: add CCM JSON validation > > > > examples/fips_validation: add ECDSA keygen support > > > > examples/fips_validation: add SHA3 algorithms in ECDSA test > > > > examples/fips_validation: fix AES GCM validation tests > > > > examples/fips_validation: fix AES XTS to read seq number > > > > examples/fips_validation: add extra space in JSON buffer > > > > > > > > doc/guides/sample_app_ug/fips_validation.rst | 7 +- > > > > examples/fips_validation/fips_validation.c| 31 ++- > > > > examples/fips_validation/fips_validation.h| 10 +- > > > > .../fips_validation/fips_validation_ccm.c | 132 > > > > .../fips_validation/fips_validation_ecdsa.c | 56 + > > > > .../fips_validation/fips_validation_gcm.c | 12 +- > > > > .../fips_validation/fips_validation_hmac.c| 8 + > > > > .../fips_validation/fips_validation_sha.c | 91 ++-- > > > > .../fips_validation/fips_validation_xts.c | 13 +- > > > > examples/fips_validation/main.c | 196 > +- > > > > 10 files changed, 467 insertions(+), 89 deletions(-) > > > > > > > > -- > > > > 2.25.1 > > > > > > Series-acked-by: Brian Dooley > > > > Series Applied to dpdk-next-crypto > The series is showing compilation issues, please fix it. The series is > removed from the tree. > > > -- Patrick Robb Technical Service Manager UNH InterOperability Laboratory 21 Madbury Rd, Suite 100, Durham, NH 03824 www.iol.unh.edu
Re: [EXT] Re: [v1, 00/10] fips_validation application improvements
Hi Akhil, One 2023 goal for UNH is implementing an email based retesting framework. Once that work is completed, you will be able to trigger a retest yourself under circumstances where waiting for dependent patches is needed. On Tue, Feb 28, 2023 at 10:02 AM Akhil Goyal wrote: > Hi Patrick, > > > > The issue reported by CI in below link is not an issue, as the patchset > was dependent on another patch which is already merged. Now we are > observing a new issue which is coming only on CentOS I believe. > > CI reports are not useful in case there are dependent patches. There > should be a way maintainer/developer can retrigger the CI as required when > other patches are merged. > > > > This issue got skipped as I personally do not test on CentOS and CI > results are not meaningful when there were dependent patches. > > Below is the log for the compilation issue observed now on TOT when these > patches are applied which is not visible in the below link. > > > > > OS: CentOS79-64 > > > Target: x86_64-native-linuxapp-gcc > > > FAILED: examples/dpdk-fips_validation.p/fips_validation_main.c.o > > > gcc -Iexamples/dpdk-fips_validation.p -Iexamples -I../examples - > > > Iexamples/fips_validation -I../examples/fips_validation > -I../examples/common - > > > I. -I.. -Iconfig -I../config -Ilib/eal/include -I../lib/eal/include - > > > Ilib/eal/linux/include -I../lib/eal/linux/include -Ilib/eal/x86/include - > > > I../lib/eal/x86/include -Ilib/eal/common -I../lib/eal/common -Ilib/eal > -I../lib/eal > > > -Ilib/kvargs -I../lib/kvargs -Ilib/metrics -I../lib/metrics > -Ilib/telemetry - > > > I../lib/telemetry -Ilib/mempool -I../lib/mempool -Ilib/ring > -I../lib/ring -Ilib/net - > > > I../lib/net -Ilib/mbuf -I../lib/mbuf -Ilib/ethdev -I../lib/ethdev > -Ilib/meter - > > > I../lib/meter -Ilib/cmdline -I../lib/cmdline -Ilib/cryptodev > -I../lib/cryptodev - > > > Ilib/rcu -I../lib/rcu -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra > - > > > Werror -O3 -include rte_config.h -Wcast-qual -Wdeprecated -Wformat - > > > Wformat-nonliteral -Wformat-security -Wmissing-declarations -Wmissing- > > > prototypes -Wnested-externs -Wold-style-definition -Wpointer-arith > -Wsign- > > > compare -Wstrict-prototypes -Wundef -Wwrite-strings -Wno-missing-field- > > > initializers -D_GNU_SOURCE -march=native -DUSE_OPENSSL - > > > DALLOW_EXPERIMENTAL_API -MD -MQ examples/dpdk- > > > fips_validation.p/fips_validation_main.c.o -MF examples/dpdk- > > > fips_validation.p/fips_validation_main.c.o.d -o examples/dpdk- > > > fips_validation.p/fips_validation_main.c.o -c > ../examples/fips_validation/main.c > > > ../examples/fips_validation/main.c: In function 'fips_mct_shake_test': > > > ../examples/fips_validation/main.c:2438:5: error: dereferencing > type-punned > > > pointer will break strict-aliasing rules [-Werror=strict-aliasing] > > > (*(uint16_t *)rightmost % range); > > > ^ > > > > > > Regards, > > Akhil > > > -- > > Hello Akhil and Gowrishankar, > > > > We saw the same issue with running the fips sample app under CI testing > here at the UNH Community Lab: > http://mails.dpdk.org/archives/test-report/2023-February/350635.html > <https://urldefense.proofpoint.com/v2/url?u=http-3A__mails.dpdk.org_archives_test-2Dreport_2023-2DFebruary_350635.html&d=DwMFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=DnL7Si2wl_PRwpZ9TWey3eu68gBzn7DkPwuqhd6WNyo&m=QswegPR1LvDRH4f_2rvo_BvAfrFOBGZPVPGXC7pPlKn2Q99LMoJlfwv4ZZ0QxRu-&s=Ozm6byIlAfWxebW0mSomWcCW-0s59wZNJ7QED4-Tjcs&e=>. > We reported a warn because it failed on the compilation stage (as opposed > to a failure of the actual sample app run). > > > > On the other hand, we are excited to see this patch again with the > compilation part resolved. I sent an email to the dev mailing list a few > weeks ago for our fips sample app CI testing regarding where we could and > could not provide test vector coverage, and it appears this patch series > may resolve our ciphertext issue with AES-GCM test vector and more issues > with the sample app. So - looking forward to seeing a patch like this being > merged when stable! > > > > Best, > > Patrick Robb > > > > > > > > > > On Tue, Feb 28, 2023 at 2:39 AM Akhil Goyal wrote: > > Hi Gowrishankar, > > > > > > > Subject: [v1, 00/10] fips_validation application improvements > > > > > > > > This patch series adds support for SHA3, SHAKE, AES-CCM JSON test > vectors > > > >
Re: [PATCH v6 10/10] doc: update dts setup and test suite cookbook
Tested-by: Patrick Robb On Fri, Mar 3, 2023 at 5:25 AM Juraj Linkeš wrote: > Document how to configure and run DTS. > Also add documentation related to new features: SUT setup and a brief > test suite implementation cookbook. > > Signed-off-by: Juraj Linkeš > --- > doc/guides/tools/dts.rst | 165 ++- > 1 file changed, 163 insertions(+), 2 deletions(-) > > diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst > index daf54359ed..ebd6dceb6a 100644 > --- a/doc/guides/tools/dts.rst > +++ b/doc/guides/tools/dts.rst > @@ -1,5 +1,5 @@ > .. SPDX-License-Identifier: BSD-3-Clause > -Copyright(c) 2022 PANTHEON.tech s.r.o. > +Copyright(c) 2022-2023 PANTHEON.tech s.r.o. > > DPDK Test Suite > === > @@ -56,7 +56,7 @@ DTS runtime environment or just plain DTS environment > are used interchangeably. > > > Setting up DTS environment > --- > +~~ > > #. **Python Version** > > @@ -93,6 +93,167 @@ Setting up DTS environment >poetry install >poetry shell > > +#. **SSH Connection** > + > + DTS uses Python pexpect for SSH connections between DTS environment > and the other hosts. > + The pexpect implementation is a wrapper around the ssh command in the > DTS environment. > + This means it'll use the SSH agent providing the ssh command and its > keys. > + > + > +Setting up System Under Test > + > + > +There are two areas that need to be set up on a System Under Test: > + > +#. **DPDK dependencies** > + > + DPDK will be built and run on the SUT. > + Consult the Getting Started guides for the list of dependencies for > each distribution. > + > +#. **Hardware dependencies** > + > + Any hardware DPDK uses needs a proper driver > + and most OS distributions provide those, but the version may not be > satisfactory. > + It's up to each user to install the driver they're interested in > testing. > + The hardware also may also need firmware upgrades, which is also left > at user discretion. > + > +#. **Hugepages** > + > + There are two ways to configure hugepages: > + > + * DTS configuration > + > + You may specify the optional hugepage configuration in the DTS > config file. > + If you do, DTS will take care of configuring hugepages, > + overwriting your current SUT hugepage configuration. > + > + * System under test configuration > + > + It's possible to use the hugepage configuration already present on > the SUT. > + If you wish to do so, don't specify the hugepage configuration in > the DTS config file. > + > + > +Running DTS > +--- > + > +DTS needs to know which nodes to connect to and what hardware to use on > those nodes. > +Once that's configured, DTS needs a DPDK tarball and it's ready to run. > + > +Configuring DTS > +~~~ > + > +DTS configuration is split into nodes and executions and build targets > within executions. > +By default, DTS will try to use the ``dts/conf.yaml`` config file, > +which is a template that illustrates what can be configured in DTS: > + > + .. literalinclude:: ../../../dts/conf.yaml > + :language: yaml > + :start-at: executions: > + > + > +The user must be root or any other user with prompt starting with ``#``. > +The other fields are mostly self-explanatory > +and documented in more detail in > ``dts/framework/config/conf_yaml_schema.json``. > + > +DTS Execution > +~ > + > +DTS is run with ``main.py`` located in the ``dts`` directory after > entering Poetry shell:: > + > + usage: main.py [-h] [--config-file CONFIG_FILE] [--output-dir > OUTPUT_DIR] [-t TIMEOUT] > + [-v VERBOSE] [-s SKIP_SETUP] [--tarball TARBALL] > + [--compile-timeout COMPILE_TIMEOUT] [--test-cases > TEST_CASES] > + [--re-run RE_RUN] > + > + Run DPDK test suites. All options may be specified with the > environment variables provided in > + brackets. Command line arguments have higher priority. > + > + options: > + -h, --helpshow this help message and exit > + --config-file CONFIG_FILE > + [DTS_CFG_FILE] configuration file that > describes the test cases, SUTs > + and targets. (default: conf.yaml) > + --output-dir OUTPUT_DIR, --output OUTPUT_DIR > + [DTS_OUTPUT_DIR] Output directory where dts > logs and results are > + saved. (default: output) &g
Re: [PATCH] drivers: Convert uses of RTE_LOG_DP to use RTE_LOG_DP_LINE
Recheck-request: iol-intel-Functional I saw this failed in CI testing for the DTS mtu_update test but suspect it was an infra failure as it happened across 3 distinct patches in a short span of time. So, triggering a retest.
Re: [PATCH 00/13] net/ionic: miscellaneous fixes and improvements
Recheck-request: iol-intel-Functional I saw this failed in CI testing for the DTS MTU_Update test but suspect it was an infra failure as it happened across 3 distinct patches in a short span of time. So, triggering a retest.
Re: [PATCH v7 00/19] Replace uses of RTE_LOGTYPE_PMD
Recheck-request: iol-intel-Functional I saw this failed in CI testing for the DTS MTU_Update test but suspect it was an infra failure as it happened across 3 distinct patches in a short span of time. So, triggering a retest.
Re: [PATCH V2] examples/pipeline: fix include path for rte_log.h
Hi Aaron/Cristian,
On Wed, Feb 14, 2024 at 11:25 AM Aaron Conole wrote:
> Ferruh Yigit writes:
>
> > On 2/13/2024 5:38 PM, Cristian Dumitrescu wrote:
> >> When rte_log.h was moved to a new directory, the include path was not
> >> updated for the generated C code produced by the pipeline library,
> >> which results in build failure for this code.
> >>
> >> Fixes: 09ce41310930 ("log: separate logging functions out of EAL")
> >> Cc: sta...@dpdk.org
> >>
> >
> > Hi Cristian,
> >
> > How can I verify the fix? Can you please list the required steps?
>
> I guess maybe (?) with the pipeline DTS case, but I'm not sure that
> would be sufficient.
>
> > And I wonder how this skipped the testing, I guess v23.11 released with
> > this defect. Is there a gap in the CI or internal build/test scripts?
>
> I don't know that softnic driver is used in the lab. Actually, would
> DTS suite even have triggered this issue? I'm not sure if there is a
> set of tests which covers the case. Maybe Patrick can confirm about the
> pipeline test?
>
So, based on what Cristian stated (passing in any of the .cli files when
starting the pipeline example app would show it is fixed), yes I assume the
DTS testsuite would have caught this, as I can see the testsuite does do
that. But, yes it's also true that the pipeline testsuites are not run at
UNH or the Intel Lab (the two labs which publicly report DTS results), so
that's how this gets through CI Testing. It is not possible (testing
capacity wise) to run every testsuite, and I don't think there has been
conversation between the lab and our vendor contacts about this specific
coverage (at least not while I've been working here).
However, based on the physical testplan requirements (4 10G tester ports to
4 10G DUT ports), we could bring the testsuite online if there is
interest(and the testsuite hasn't broken since it dropped in 2020). One of
our Intel testbeds which we run currently has exactly that NIC topology,
and it also doesn't have bad testing capacity concerns as compared to some
other testbeds. Let me know if there is an interest in this coverage and
I'll make a ticket for the team to take a look. At a minimum we could dry
run the framework on the testbed I'm thinking of and provide feedback,
which I suppose would take only a couple minutes/hours. Christian let me
know if that's a value - it's a low barrier of entry to dry run.
https://git.dpdk.org/tools/dts/tree/test_plans/pipeline_test_plan.rst
Re: [PATCH V2] examples/pipeline: fix include path for rte_log.h
On Wed, Feb 14, 2024 at 3:00 PM Dumitrescu, Cristian < cristian.dumitre...@intel.com> wrote: > > > [Cristian] > Yes, you are right, we do have a DTS test suite for the pipeline library. > > It would be great to run it automatically as part of the CI testing. Just > a word of caution though: I am not the owner of those tests, and also not > intimately familiar with them, so it might be hard to find volunteers to > setup the test suite, that the only problem that I see unfortunately. But > we should try and see if there are any issues at all. > > Thanks, > Cristian > Okay, the setup doesn't look too burdensome anyhow so we should be alright with just the team here at UNH. I'll let you know how it goes.
DTS WG Meeting Minutes - February 14, 2024
February 14, 2024 # Attendees * Patrick Robb * Juraj Linkeš * Thomas Monjalon * Gregory Etelson * Juraj Linkes * Luca Vizzarro * Nicholas Pratte # Agenda * Additions to the agenda * Patch discussions * DTS Developer documentation * Bugzilla discussion # Minutes = General Announcements * Added developers for DTS: Nick from UNH is starting on DTS now, and 1-2 more people from UNH will be starting on this project in the near future. * The first thing Nick will do is build the DTS API docs from Juraj’s patch and provide a review * He will be joining the meetings going forward * Recordings of these meetings can be seen at the below url, by clicking on the meeting date, then the 3 dots, then share meeting recording: https://projectadmin.lfx.linuxfoundation.org/project/a094105osNmAAI = Patch discussions * Docs improvements: Luca’s patch was accepted * Thomas noted that the schema for DPDK build options was probably not well thought out, and just imported from old dts. * Noting I remember Juraj mentioned once that cross compile targets probably are not needed 1. This is not a priority - we should wait and see if there is interest later. 2. https://bugs.dpdk.org/show_bug.cgi?id=1360 3. There are some unused values which can be removed 4. Lcores value: change from empty string to “all” (if not specifying a specific list of cores) 5. Port topology: should this be a part of the conf schema? 1. Some testsuites may allow for different port topologies and should be configurable, others may have specific requirements which should be built into the capabilities assessment per testsuite 6. Ports: If we specify port 0 on machine 1 connects to port 0 on machine 2, we don’t then need to specify port 0 on machine 2 connects to port 0 on machine 1 - it’s redundant configuration (and introduces possible setup human error) * Improved error messages: * Intent for Luca’s patch is to improve patch reporting, as well as log errors from remote sessions * Scatter * Per conversation at the previous meeting, adding a 2nd testcase to the testsuite (one will include the rx offload testpmd flag, one will not) * Ran into a type error yesterday from the xmlrpc client when running on one of the community lab’s testbeds, but will continue debugging 1. Error is about a list not being callable? Jeremy Spewock will send the error and latest diff to Juraj on Slack * When running a suite in which testpmd is started for 1 testcase, then stopped, then started again for subsequent testcases, this can lead to a timeout. The timeout comes from paramiko. * Device capabilities: Only source of truth for collecting capabilities should be any information we can gather via testpmd * Needs to build a way to mark testsuites with capabilities * Will build a list of capabilities per device * Dockerfile patch: * Patrick Robbneeds to apply this, run from container on one of our baremetal servers, and ask Thomas to merge * API Docs: * Needs reviews - Nick will provide one * Juraj: Small patch for storing the output from remote commands (strips the whitespace) * Testcase blocking: When a testcase fails, everything under it is blocked. This patch is final from Juraj’s point of view, and Jeremy has provided some comments. * More reviews are needed. Patrick and Nick can do reviews, also Luca possibly. = Bugzilla: * If we can assign configuration file bugzillas to two people, they can provide reviews for one another * We will discuss bugzilla tickets more tomorrow at the CI meeting # Any other business * Juraj is on vacation 4th of March to 29 March.
Rescheduling DPDK CI Meetings
Hello all, As discussed in the previous CI testing meeting, it will be advantageous for us to shift the (every other week) DPDK CI meetings forward or backwards 1 week to get it onto a different cadence. That will allow us to have the DTS meetings 1 week, and the CI meetings on the off week, instead of both on the same week, which should yield more valuable conversations. So the idea is to have tomorrow's CI meeting, then have another meeting next Thursday (2/22), then go back to every other week. So the 2/29 meeting would be cancelled, we would have a 3/7 meeting, a 3/21 meeting, etc. I am hoping this will work well for those of you out there who attend these meetings, but please let me know! If there are no objections, I will request that Nathan modify the calendar event. Thanks.
Re: Rescheduling DPDK CI Meetings
Hi, This is finalized. The next CI meeting will be next Thursday. Then we go back to every other week. On Wed, Feb 14, 2024 at 4:20 PM Patrick Robb wrote: > Hello all, > > As discussed in the previous CI testing meeting, it will be advantageous > for us to shift the (every other week) DPDK CI meetings forward or > backwards 1 week to get it onto a different cadence. > > That will allow us to have the DTS meetings 1 week, and the CI meetings on > the off week, instead of both on the same week, which should yield more > valuable conversations. > > So the idea is to have tomorrow's CI meeting, then have another meeting > next Thursday (2/22), then go back to every other week. So the 2/29 meeting > would be cancelled, we would have a 3/7 meeting, a 3/21 meeting, etc. > > I am hoping this will work well for those of you out there who attend > these meetings, but please let me know! If there are no objections, I will > request that Nathan modify the calendar event. Thanks. > >
Re: [PATCH] doc: update minimum Linux kernel version
+Gao, DaxueX +Mcnamara, John Hello, As you say the Community Lab dropped main and next-* testing for RHEL7 when the requirement for a C11 compliant compiler was added last year, so we should already be good to go. We don't have any centos7 testing (centos8 was the first centos introduced for testing at the Community Lab). Adding Daxue and John so they are aware the Intel Lab will want to (probably) drop the centos7 testing by 24.07. Thanks! On Fri, Feb 16, 2024 at 12:42 PM Stephen Hemminger < step...@networkplumber.org> wrote: > On Fri, 16 Feb 2024 09:29:47 +0100 > Morten Brørup wrote: > > > The system requirements in the Getting Started Guide [1] says: > > > > Kernel version >= 4.14 > > The kernel version required is based on the oldest long term stable > kernel available at kernel.org when the DPDK version is in development. > > Compatibility for recent distribution kernels will be kept, notably > RHEL/CentOS 7. > > > > [1]: https://doc.dpdk.org/guides/linux_gsg/sys_reqs.html#system-software > > > > If we consider it API breakage to change that, we have to wait until the > 24.11 release. > > For future DPDK LTS releases, we should be more careful about what we > claim to support. And again: If we claim to support something, people > expect it to be tested in CI. > > > > Disregarding the API breakage by stopping support for a system we claim > to support... RHEL7 testing was changed to LTS only [2], that should > probably have been applied to CentOS 7 too. > > > > [2]: > https://inbox.dpdk.org/dev/CAJvnSUBcq3gznQD4k=krQ+gu2OxTxA2YJBc=J=ltidfxqgg...@mail.gmail.com/ > > > > This patch is too late for 24.03 release, by the time the next one happens, > we can drop CentOS 7 as well as the old kernel. >
Re: DTS testpmd and SCAPY integration
Hi Gregory,
I apologize for my delayed response. I have now taken a look through the
code. I do think a number of the design decisions you made for the unit
test framework can be carried over to DTS - in fact we have some tickets
currently for refactoring DTS to this end. Namely:
-Breaking out test agent setup config and test config into separate
files
-Reducing the amount of config values which need to be set by the user,
lowering the barrier for entry for running the testing framework.
-Add some level of testsuite specific YAML which can be mapped to the
testsuite its written for (just for some values, not explicit commands)
But, I figure what you are asking about here really is the design of
setting phases in YAML and writing the scapy/testpmd commands there.
Although I (and I think the DTS group broadly) agree that this system does
provide a good user experience for quickly writing new testsuites, there
are other reasons (testsuite flexibility, platform/TG agnosticism) we don't
want to provide the 100% YAML option for a testsuite. We discussed the idea
of providing two paths for writing a testsuite (YAML file only, or YAML +
python testsuite file), but honestly I don't think it's a good idea to
split our efforts (particularly early on). If we want to consider building
support for a YAML testsuite approach for simple func testsuites in the
(far) future we can discuss it, but that is an unknown. I think our efforts
now need to remain on the python testsuites.
Still, seeing your project has been a big inspiration. When we can abstract
away some boilerplate, or use YAML to override default behavior, we want to
do it. But we still plan to handle sending commands to TG/SUT from the
python testsuite files. I think you are helping us a lot with your ideas,
even if the end implementation is a little different in approach.
Note: incorporating a function(s) to load different mlnx device firmware as
a part of the testsuite is an interesting touch. Do you mind explaining why
it is made a part of the testing framework, as opposed to a pre-step for
the user to complete ahead of running the framework? Does managing loading
the firmware this way make device firmware development and testing it
against DPDK easier, or is there another reason? Thanks.
On Wed, Feb 14, 2024 at 12:27 PM Gregory Etelson
wrote:
> Hello Patrick,
>
> Did you have time to check the Unit Test design ?
> Do you think it can be used for short functional DTS tests ?
>
> Regards,
> Gregory
>
> --
> *From:* Gregory Etelson
> *Sent:* Wednesday, January 31, 2024 09:43
> *To:* Patrick Robb
> *Cc:* Gregory Etelson ; Jeremy Spewock <
> jspew...@iol.unh.edu>; NBU-Contact-Thomas Monjalon (EXTERNAL) <
> tho...@monjalon.net>; Honnappa Nagarahalli ;
> Juraj Linkeš ; Paul Szczepanek <
> paul.szczepa...@arm.com>; Yoan Picchi ; Luca
> Vizzarro ; c...@dpdk.org ; dev@dpdk.org
> ; nd ; Maayan Kashani ;
> Asaf Penso
> *Subject:* Re: DTS testpmd and SCAPY integration
>
> Hello Patrick,
>
> > External email: Use caution opening links or attachments
> > Thank you for sharing Gregory. I did not get an opportunity to look
> through the code today, but I did run
> > through the presentation. A few points I noted:
> > 1. The presentation shows an example testpmd testcase for creating a
> flow rule, and then shows a
> > validation step in which standard out is compared against the expected
> string ("flow rule x created") and
> > we can conclude whether we are able to create flow rules. Are you also
> sending packets according to the
> > flow rules and validating that what is sent/received corresponds to the
> expected behavior of the flow
> > rules? When I look at the old DTS framework, and an example flow rules
> testsuite
> > (https://doc.dpdk.org/dts/test_plans/rte_flow_test_plan.html) which we
> want feature parity with, I think
> > that validation for this testing framework needs to primarily rely on
> comparing packets sent and packets
> > received.
>
> The unit test infrastructure validates flow rule creation and
> a result produced by that flow.
> Flow result is triggered by a packet.
> However, flow result validation does not always can be done by testing a
> packet.
> Unit test implements 2 flow validation methods.
>
> The first validation method tests testpmd output triggered by a test
> packet.
>
> Example: use the MODIFY_FIELD action to copy packet VLAN ID to flow TAG
> item.
> Flow tag is internal flow resource. It must be validated in DPDK
> application.
>
> Test creates 2 flow rules:
>
> Rule 1: use MODIFY_FILED to copy packet VLAN ID to flow TAG item
> pattern eth / vlan / end \
> actions modify_field op set dst_type tag ... src_type vlan_id .
Email based retest request process: proposal for new pull/re-apply feature
Hi all, I want to poll the CI group and dev community about a proposed feature addition to the CI retest request framework. Currently, you can respond to a patchseries or patch email, requesting a retest like so: Recheck-request: iol-compile-amd64-testing, iol-unit-amd64-testing Labs who have added this functionality (UNH and the GitHub Robot) will then trigger retests according to the contexts provided, using the ORIGINAL dpdk artifact they produced at the time when the patch was submitted. This is useful for requesting a retest on a patch when you believe a failure may have been an infra failure or spurious. It is not useful if you believe the tree your patch was applied on was in a bad state when your patch was submitted, and you would like for your patch to be re-applied on the current tip of the branch. A few people have suggested we add this feature (re-apply to tip of branch and retest). So, we should probably add an option allowing people to indicate they want this behavior instead of the "default" retest. Ferruh emailed me about this a while ago and proposed the following syntax, which I am okay with: Recheck-request,attribute=value: ... So a practical example would look like: Recheck-request,pull=True: iol-sample-apps-testing, iol-compile-amd64-testing, github-robot Also, I believe that although we should still require people to include the contexts they're requesting a retest for for posterity (so we can refer back to it), I think if someone includes the pull keyword, ALL labs should trigger retests for ALL tests. The reason for this is I don't think we should display results side by side on a patchseries which are coming from distinct DPDK artifacts. Readers may assume (rightly, in my opinion) that when they're looking at a results table for a patchseries, those results are all coming from identical DPDK artifacts, and not from distinct DPDK artifacts produced at different times, from different commits. What do you all think? Thanks.
Re: Email based retest request process: proposal for new pull/re-apply feature
On Tue, Feb 20, 2024 at 1:12 PM Aaron Conole wrote: > > Why not something like: > > Recheck-request: [attribute-list],[test-list]... > > For example, then we can do: > > Recheck-request: rebase=[identifier], > > where identifier is a branch specifier (or the word 'latest')? > I hadn't thought about the option of allowing branch specifiers. Agree that allowing a human correction process for the pw_maintainer_cli.py script choosing the wrong branch sounds helpful. My original idea was offering 2 options (test original artifact, or re-apply on latest). Do we want to support for checking out to a specific commit and re-applying there? I figured that would not be worth it (too much of a niche case), but your comments are making me reconsider. > > That lets us fixup if the branch picker script guessed a wrong branch. > > Just spit-balling on syntax. > > > That said, I agree - if a rebase has been requested, all tests need to > be rerun. Maybe we should consider that the test labels should be added > with a run number or something? Or we could also include that the run > is a rerun. That way for labs that don't currently support the recheck > request framework, we can easily tell that they weren't re-tried. > > so re-report with a modified test label? That is good in that it shows the behavior more clearly. But, it also means we will not overwrite any fails. So the fail will still be there, and the patchwork patch page will grow a huge table. Maybe this is fine. Also raises the point of getting more coverage for the retest framework at other labs. I will email Min Zhou regarding how he uses the dpdk-ci project for the loongson build jobs and see how well that can integrate with the get_reruns.py script.
Re: [PATCH v8 0/7] dts: Port scatter suite over
Tested-by: Patrick Robb I ran this testsuite with a bnxt_en NIC at the Community Lab. I also spoke with Jeremy about the state of this patch today. He wants to add a second testcase to the suite for testing the scattered packets hardware offload (--enable-scatter flag in testpmd). But, he still has questions about querying ethdev for capabilities and writing the testcase around that, so that testcase cannot be submitted for this release. It will come in as a separate patch. So, from what I can tell Juraj has completed his review and this is the final v of this patchseries. On Wed, Jan 10, 2024 at 9:43 AM wrote: > From: Jeremy Spewock > > v8: > > Address comments by making minor docstring adjustments. > > Jeremy Spewock (7): > dts: add startup verification and forwarding modes to testpmd shell > dts: limit EAL parameters to DPDK apps and add parameters to all apps > dts: add optional packet filtering to scapy sniffer > dts: add pci addresses to EAL parameters > dts: allow configuring MTU of ports > dts: add scatter to the yaml schema > dts: add pmd_buffer_scatter test suite > > dts/framework/config/conf_yaml_schema.json| 3 +- > dts/framework/exception.py| 7 + > dts/framework/remote_session/testpmd_shell.py | 149 +- > dts/framework/test_suite.py | 15 +- > dts/framework/testbed_model/linux_session.py | 8 + > dts/framework/testbed_model/os_session.py | 9 ++ > dts/framework/testbed_model/sut_node.py | 28 +++- > dts/framework/testbed_model/tg_node.py| 14 +- > .../traffic_generator/__init__.py | 7 +- > .../capturing_traffic_generator.py| 22 ++- > .../testbed_model/traffic_generator/scapy.py | 27 > dts/tests/TestSuite_pmd_buffer_scatter.py | 132 > 12 files changed, 407 insertions(+), 14 deletions(-) > create mode 100644 dts/tests/TestSuite_pmd_buffer_scatter.py > > -- > 2.43.0 > >
Re: [PATCH v4] dts: add Dockerfile
Tested-by: Patrick Robb The approach for mounting in ssh keys and managing the test engine environment has been agreed upon by the CI group, and this patch has been tested by a few people at the Community Lab. I believe it is ready to be merged. On Tue, Jan 16, 2024 at 2:18 PM wrote: > From: Juraj Linkeš > > The Dockerfile defines development and CI runner images. > > Signed-off-by: Juraj Linkeš > Signed-off-by: Jeremy Spewock > > >
Re: [EXT] Re: [PATCH v2 8/8] crypto/ipsec_mb: set and use session ID
On Tue, Jun 20, 2023 at 5:44 AM Fangming Fang wrote: > v1.4 release on the arm repo breaks DPDK > We have reverted the Arm library version to 1.3.0 to work around this > issue according to Pablo's suggestion. The library fixing this issue is > tagged as SECLIB-IPSEC-2023.06.20. > > Thanks, > Fangming > > IMPORTANT NOTICE: The contents of this email and any attachments are > confidential and may also be privileged. If you are not the intended > recipient, please notify the sender immediately and do not disclose the > contents to any other person, use it for any purpose, or store or copy the > information in any medium. Thank you. > Does anyone know if there are plans to bring the arm ipsec-mb repo to v1.4, or plans to advance any of the other ideas which were put forth in this conversation? DPDK 24.03 will require ipsec 1.4 or greater, and there is a patch pending to that effect. We do run some testing at the Community Lab on ARM which installs the ipsec-mb library, and runs ZUC and Snow3G autotests with their respective crypto vdev pmds. As noted here the the build fails from the v1.4 tag on arm, and the SECLIB-IPSEC-2023.06.20 cannot be used for 24.03. If it won't be possible to install ipsec-mb on ARM for use with 24.03 I will have to take that test coverage offline. I just want to check to see if there is a workaround I've missed or some planned work on the horizon which will unblock this. Thanks.
Re: [EXT] [PATCH v2 8/8] crypto/ipsec_mb: set and use session ID
Okay. In terms of lab operations, I'll disable the ARM ZUC/Snow3G tests for main and next-*, but leave enabled for LTS. Once we can use v1.4 for ARM, will re-enable everything. Thanks. On Wed, Feb 21, 2024 at 12:10 AM Honnappa Nagarahalli < honnappa.nagaraha...@arm.com> wrote: > > > > On Feb 20, 2024, at 11:01 PM, Patrick Robb wrote: > > > > > > > > On Tue, Jun 20, 2023 at 5:44 AM Fangming Fang > wrote: > > v1.4 release on the arm repo breaks DPDK > > We have reverted the Arm library version to 1.3.0 to work around this > issue according to Pablo's suggestion. The library fixing this issue is > tagged as SECLIB-IPSEC-2023.06.20. > > > > Thanks, > > Fangming > > > > IMPORTANT NOTICE: The contents of this email and any attachments are > confidential and may also be privileged. If you are not the intended > recipient, please notify the sender immediately and do not disclose the > contents to any other person, use it for any purpose, or store or copy the > information in any medium. Thank you. > > > > Does anyone know if there are plans to bring the arm ipsec-mb repo to > v1.4, or plans to advance any of the other ideas which were put forth in > this conversation? DPDK 24.03 will require ipsec 1.4 or greater, and there > is a patch pending to that effect. > > > > We do run some testing at the Community Lab on ARM which installs the > ipsec-mb library, and runs ZUC and Snow3G autotests with their respective > crypto vdev pmds. As noted here the the build fails from the v1.4 tag on > arm, and the SECLIB-IPSEC-2023.06.20 cannot be used for 24.03. If it won't > be possible to install ipsec-mb on ARM for use with 24.03 I will have to > take that test coverage offline. I just want to check to see if there is a > workaround I've missed or some planned work on the horizon which will > unblock this. Thanks. > I am not sure about the current plans, we will get back on this soon. > > IMPORTANT NOTICE: The contents of this email and any attachments are > confidential and may also be privileged. If you are not the intended > recipient, please notify the sender immediately and do not disclose the > contents to any other person, use it for any purpose, or store or copy the > information in any medium. Thank you. >
Re: [EXT] [PATCH v2 8/8] crypto/ipsec_mb: set and use session ID
On Wed, Feb 21, 2024 at 4:50 AM Power, Ciara wrote: > > Hi folks, > > We had based the ipsec-mb version bump to 1.4 on both intel ipsec-mb and > arm ipsec-mb supporting that version, so both could still use the Ipsec-mb > SW PMDs. > I based the arm support from the repo main branch ( > https://gitlab.arm.com/arm-reference-solutions/ipsec-mb/-/tree/main) > It's version is 1.4 and looks like some of the code changes from > intel-ipsec-mb 1.4 have been merged in (e.g. imb_set_session function which > is 1.4+). > I know there was issues before with it being tagged as 1.4 too soon - is > that still the case, or is the main branch up to date now with > intel-ipsec-mb 1.4? > > Thanks, > Ciara > Hi Ciara, I see what you mean and did a test on an arm container. I used tip of main for ipsec-mb, applied the v1.4 requirement patch to dpdk, built dpdk and was able to run the ZUC autotest with the zuc vdev pmd. root@8aad8acde315:/opt/dpdk/build# meson test cryptodev_sw_zuc_autotest --test-args="--no-huge --no-pci --vdev=crypto_zuc" ninja: no work to do. ninja: Entering directory `/opt/dpdk/build' ninja: no work to do. 1/1 DPDK:driver-tests / cryptodev_sw_zuc_autotestOK 0.22s Ok: 1 Expected Fail: 0 Fail: 0 Unexpected Pass:0 Skipped:0 Timeout:0 Usually when we install dependencies we go from a tag which is stable. We'll update the dpdk-ci container image template engine to clone the repo and then checkout to this commit which is known to be good: cef9b8924e3ff52f2c5a3703860008c27ee723c4. And our automated testing can remain online in this way. Arm folks, I'm not sure what your plans are but at some point in the docs I think pages like: https://doc.dpdk.org/guides/cryptodevs/zuc.html https://doc.dpdk.org/guides/cryptodevs/snow3g.html should be updated to reference this commit, or a newly introduced tag, to reflect that the SECLIB-IPSEC-2023.06.20 tag will no longer work for 24.03 as it is ipsec 1.3. Thanks for the help.
Re: [PATCH 0/7] bnxt bug fixes
https://patchwork.dpdk.org/project/dpdk/patch/20240208171330.31139-8-ajit.khapa...@broadcom.com/ Hi Ajit, So you know, this series did fail in CI testing, and now that it is merged to main, all patchseries are failing DTS at EAL. Performance testing is also offline for the same reason. I will run the niccli utility to upgrade broadcom NIC firmware on the system and give it a rerun. Otherwise, let me know what you think. This occurs on our ARM Ampere server, with the BCM57414 NetXtreme-E 10Gb/25Gb 21:02:08 [00;36m TestScatter: Test Case test_scatter_mbuf_2048 Begin [0m 21:02:08 dut.arm-ampere-dut.dpdklab.iol.unh.edu: x86_64-native-linux-gcc/app/dpdk-testpmd -l 1,2 -n 4 -a 0007:01:00.0 -a 0007:01:00.1 --file-prefix=dpdk_2978774_20240222015854 -- -i --mbcache=200 --mbuf-size=1024 --portmask=0x1 --max-pkt-len=9000 --port-topology=loop --tx-offloads=0x8000 [0m 21:04:14 [01;31m TestScatter: Test Case test_scatter_mbuf_2048 Result FAILED: TIMEOUT on x86_64-native-linux-gcc/app/dpdk-testpmd -l 1,2 -n 4 -a 0007:01:00.0 -a 0007:01:00.1 --file-prefix=dpdk_2978774_20240222015854 -- -i --mbcache=200 --mbuf-size=1024 --portmask=0x1 --max-pkt-len=9000 --port-topology=loop --tx-offloads=0x8000 [0m 21:04:14 [01;31m TestScatter: EAL: Detected CPU lcores: 160 21:04:14 EAL: Detected NUMA nodes: 2 21:04:14 EAL: Detected static linkage of DPDK 21:04:14 EAL: Multi-process socket /var/run/dpdk/dpdk_2978774_20240222015854/mp_socket 21:04:14 EAL: Selected IOVA mode 'VA' 21:04:14 EAL: VFIO support initialized 21:04:14 EAL: Using IOMMU type 1 (Type 1) 21:04:14 EAL: Probe PCI driver: net_bnxt (14e4:16d7) device: 0007:01:00.0 (socket 1) 21:04:14 Segmentation fault (core dumped) I'll let you know how reruns go. On Thu, Feb 8, 2024 at 4:51 PM Ajit Khaparde wrote: > On Thu, Feb 8, 2024 at 9:13 AM Ajit Khaparde > wrote: > > > > Patchset with bug fixes for bnxt PMD. > > Patch based against the > > next-net-brcm for-next-net branch. > > > > Please apply. > Patchset merged to the dpdk-next-brcm for-next-net branch. > Thanks > > > > > Ajit Khaparde (4): > > net/bnxt: modify locking for representor Tx > > net/bnxt: refactor VNIC context cleanup > > net/bnxt: update consumer index of NQ regularly > > net/bnxt: update RSS algorithm capability > > > > Damodharam Ammepalli (1): > > net/bnxt: cleanup vnic ref count > > > > Kishore Padmanabha (1): > > net/bnxt: avoid seg fault in Tx queue release > > > > Shuanglin Wang (1): > > net/bnxt: adjust session name on multi host system > > > > drivers/net/bnxt/bnxt.h| 2 +- > > drivers/net/bnxt/bnxt_ethdev.c | 14 +++- > > drivers/net/bnxt/bnxt_hwrm.c | 16 +++-- > > drivers/net/bnxt/bnxt_irq.c| 26 +++--- > > drivers/net/bnxt/bnxt_reps.c | 6 ++-- > > drivers/net/bnxt/bnxt_txq.c| 8 - > > drivers/net/bnxt/bnxt_txq.h| 1 + > > drivers/net/bnxt/bnxt_txr.c| 13 +++ > > drivers/net/bnxt/bnxt_txr.h| 4 ++- > > drivers/net/bnxt/tf_ulp/bnxt_ulp.c | 58 ++ > > 10 files changed, 125 insertions(+), 23 deletions(-) > > > > -- > > 2.39.2 (Apple Git-143) > > >
Community CI Meeting Minutes - February 22, 2024
February 22, 2024 # Attendees 1. Patrick Robb 2. Ali Alnubani 3. Paul Szczepanek 4. Juraj Linkeš 5. Luca Vizzarro # Minutes = General Announcements * Aaron is polling the tech board for feedback on what server hardware is most needed in the community lab going forward. Some ideas are: * AMD CPUs * RISC-V CPUs * Better PCI-E generation slots which will allow us to use newer NICs * He is visiting UNH today so we can work on starting a proposal and plan * 24.03 RC1 has been released this morning * Retest framework: Email has been sent out with the proposed syntax and approach for retests in which users want to request their patch be re-applied on tip of branch = CI Status - UNH-IOL Community Lab * Hardware Refresh: * NVIDIA CX7: * Had some minor improvements in the performance results, but still debugging with Ali and NVIDIA * Can possibly replaced the cx5 with another cx7, and do forwarding between 2 nics, solving 1 bandwidth bottleneck * Will work on resolving the lower speeds seen on the CX6 first - this could be related to using a different board and CPU with different clock rate etc. * This server has a broadwell CPU * Patrick Robb Make sure that Ali and Gal are included for the initial feedback thread for server refresh and what is most needed * Bringing the NVIDIA testbed offline today for a few hours so Ali and Bing can do some debugging on the mlx5 failure on the CX5 yesterday * Arm IPSEC-MB Library: Had to move to running from tip of main on the ARM ipsec repo to enablev1.4 - just storing the commit hash right now but ARM will publish a new tag for a known good version soon. * Wathsala will be doing the new tag soon - Intel Lab * None - Github Actions * We plan to have a maintenance window either Thus the 22nd, or sometime next week to cover migrating to the original server. This will involve upgrading the base os for both the host and the VM. Michael will send out the notice on the day it happens letting everyone know of the downtime. * We don't expect that the downtime will last too long, less than a day. We will likely recover the workflows shortly after that. - Loongarch Lab * Patrick pinged Zhoumin about adding retest framework support to the Loongson lab * UNH willing to assist - not sure right now what is possible/not possible based on how the loongson lab stood up their automation. They do use the dpdk-ci repo tools. = DTS Improvements & Test Development * Dockerfile patch can be merged - Thomas has been pinged about this on slack * Scattered packets patch: * Patrick tested this on a bnxt_en NIC, and it worked fine * When gathering device capabilities, the scatter capability is always off on mlnx nics, even when including the scatter offload flag with testpmd * Juraj is going to send to Patrick and Jermey a summary of what he has learned about passing the scatter flag and how DPDK handles it. And what he has learned about querying for this capability. * For now, not including the scatter offload flag testcase with this testsuite, only submitting the testcase which is a direct port from old dts * Luca is going to review this today. He is also running it on a MLNX nic. * Juraj: Interestingly, on the Intel NICs it is on whether you include the --enable-scatter flag or not, but the Mellanox NICs don't have it set to on in either case * Ali: Have you tried "--enable-scatter --tx-offloads=0x8000" * Jeremy Spewockwill try this * For next Wednesday, We are going to have to have discussions for putting together the 24.07 DTS roadmap since Juraj will be on vacation in March and we won’t be able to have the conversation then. * Patrick will put together some ethdev suite ideas which the new DTS employees at UNH can start on * We will also review bugzilla tickets then, assign more tickets if needed * Patch for the testcast blocking: * There is a bug (it does not include the smoke tests in the list of suites to run), so Juraj will be sending a new version * Gregory reached out to see whether his framework’s approach could be used for simple DTS cases. Juraj and Patrick read the code. There are some good ideas we are bringing into the framework, but not the phase based yaml approach which translates scapy and testpmd commands to testsuites. * XMLRPC Server: Jeremy foun
Re: [PATCH v3] examples/ipsec-secgw: fix cryptodev to SA mapping
Recheck-request: iol-broadcom-Performance
This patch should not fail a performance test in CI - checking with a rerun
now.
On Mon, Feb 26, 2024 at 5:25 AM Radu Nicolau wrote:
> There are use cases where a SA should be able to use different cryptodevs
> on
> different lcores, for example there can be cryptodevs with just 1 qp per
> VF.
> For this purpose this patch relaxes the check in create lookaside session
> function.
> Also add a check to verify that a CQP is available for the current lcore.
>
> Fixes: a8ade12123c3 ("examples/ipsec-secgw: create lookaside sessions at
> init")
> Cc: sta...@dpdk.org
> Cc: vfia...@marvell.com
>
> Signed-off-by: Radu Nicolau
> Tested-by: Ting-Kai Ku
> Acked-by: Ciara Power
> Acked-by: Kai Ji
> ---
> v3: check if the cryptodev are not of the same type
>
> examples/ipsec-secgw/ipsec.c | 25 -
> 1 file changed, 20 insertions(+), 5 deletions(-)
>
> diff --git a/examples/ipsec-secgw/ipsec.c b/examples/ipsec-secgw/ipsec.c
> index f5cec4a928..b59576c049 100644
> --- a/examples/ipsec-secgw/ipsec.c
> +++ b/examples/ipsec-secgw/ipsec.c
> @@ -288,10 +288,21 @@ create_lookaside_session(struct ipsec_ctx
> *ipsec_ctx_lcore[],
> if (cdev_id == RTE_CRYPTO_MAX_DEVS)
> cdev_id = ipsec_ctx->tbl[cdev_id_qp].id;
> else if (cdev_id != ipsec_ctx->tbl[cdev_id_qp].id) {
> - RTE_LOG(ERR, IPSEC,
> - "SA mapping to multiple cryptodevs
> is "
> - "not supported!");
> - return -EINVAL;
> + struct rte_cryptodev_info dev_info_1, dev_info_2;
> + rte_cryptodev_info_get(cdev_id, &dev_info_1);
> +
> rte_cryptodev_info_get(ipsec_ctx->tbl[cdev_id_qp].id,
> + &dev_info_2);
> + if (dev_info_1.driver_id == dev_info_2.driver_id) {
> + RTE_LOG(WARNING, IPSEC,
> + "SA mapped to multiple cryptodevs
> for SPI %d\n",
> + sa->spi);
> +
> + } else {
> + RTE_LOG(WARNING, IPSEC,
> + "SA mapped to multiple cryptodevs
> of different types for SPI %d\n",
> + sa->spi);
> +
> + }
> }
>
> /* Store per core queue pair information */
> @@ -908,7 +919,11 @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct
> ipsec_ctx *ipsec_ctx,
> continue;
> }
>
> - enqueue_cop(sa->cqp[ipsec_ctx->lcore_id], &priv->cop);
> + if (likely(sa->cqp[ipsec_ctx->lcore_id]))
> + enqueue_cop(sa->cqp[ipsec_ctx->lcore_id],
> &priv->cop);
> + else
> + RTE_LOG(ERR, IPSEC, "No CQP available for lcore
> %d\n",
> + ipsec_ctx->lcore_id);
> }
> }
>
> --
> 2.34.1
>
>
Re: [PATCH v4 1/2] crypto/ipsec_mb: bump minimum IPsec Multi-buffer version
Recheck-request: iol-unit-arm64-testing This ran before we had swapped out the old arm v1.2 ipsec container for our new arm v1.4 ipsec container. We have now made that change, so this retest should report a pass. Sorry for the inconvenience. On Fri, Feb 23, 2024 at 7:21 AM Sivaramakrishnan Venkat < venkatx.sivaramakrish...@intel.com> wrote: > SW PMDs increment IPsec Multi-buffer version to 1.4. > A minimum IPsec Multi-buffer version of 1.4 or greater is now required. > > Signed-off-by: Sivaramakrishnan Venkat > > Acked-by: Ciara Power > Acked-by: Pablo de Lara > --- > v4: > - 24.03 release notes updated to bump minimum IPSec Multi-buffer >version to 1.4 for SW PMDs. > v2: > - Removed unused macro in ipsec_mb_ops.c > - set_gcm_job() modified correctly to keep multi_sgl_job line > - Updated SW PMDs documentation for minimum IPSec Multi-buffer version > - Updated commit message, and patch title. > --- > doc/guides/cryptodevs/aesni_gcm.rst | 3 +- > doc/guides/cryptodevs/aesni_mb.rst | 3 +- > doc/guides/cryptodevs/chacha20_poly1305.rst | 3 +- > doc/guides/cryptodevs/kasumi.rst| 3 +- > doc/guides/cryptodevs/snow3g.rst| 3 +- > doc/guides/cryptodevs/zuc.rst | 3 +- > doc/guides/rel_notes/release_24_03.rst | 4 + > drivers/crypto/ipsec_mb/ipsec_mb_ops.c | 23 --- > drivers/crypto/ipsec_mb/meson.build | 2 +- > drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 165 > drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h | 9 -- > 11 files changed, 17 insertions(+), 204 deletions(-) > > diff --git a/doc/guides/cryptodevs/aesni_gcm.rst > b/doc/guides/cryptodevs/aesni_gcm.rst > index f5773426ee..dc665e536c 100644 > --- a/doc/guides/cryptodevs/aesni_gcm.rst > +++ b/doc/guides/cryptodevs/aesni_gcm.rst > @@ -85,7 +85,8 @@ and the external crypto libraries supported by them: > 18.05 - 19.02 Multi-buffer library 0.49 - 0.52 > 19.05 - 20.08 Multi-buffer library 0.52 - 0.55 > 20.11 - 21.08 Multi-buffer library 0.53 - 1.3* > - 21.11+ Multi-buffer library 1.0 - 1.5* > + 21.11 - 23.11 Multi-buffer library 1.0 - 1.5* > + 24.03+ Multi-buffer library 1.4 - 1.5* > = > > \* Multi-buffer library 1.0 or newer only works for Meson but not Make > build system. > diff --git a/doc/guides/cryptodevs/aesni_mb.rst > b/doc/guides/cryptodevs/aesni_mb.rst > index b2e74ba417..5d670ee237 100644 > --- a/doc/guides/cryptodevs/aesni_mb.rst > +++ b/doc/guides/cryptodevs/aesni_mb.rst > @@ -146,7 +146,8 @@ and the Multi-Buffer library version supported by them: > 19.05 - 19.08 0.52 > 19.11 - 20.08 0.52 - 0.55 > 20.11 - 21.08 0.53 - 1.3* > - 21.11+ 1.0 - 1.5* > + 21.11 - 23.11 1.0 - 1.5* > + 24.03+ 1.4 - 1.5* > == > > \* Multi-buffer library 1.0 or newer only works for Meson but not Make > build system. > diff --git a/doc/guides/cryptodevs/chacha20_poly1305.rst > b/doc/guides/cryptodevs/chacha20_poly1305.rst > index 9d4bf86cf1..c32866b301 100644 > --- a/doc/guides/cryptodevs/chacha20_poly1305.rst > +++ b/doc/guides/cryptodevs/chacha20_poly1305.rst > @@ -72,7 +72,8 @@ and the external crypto libraries supported by them: > = > DPDK version Crypto library version > = > - 21.11+ Multi-buffer library 1.0-1.5* > + 21.11 - 23.11 Multi-buffer library 1.0-1.5* > + 24.03+ Multi-buffer library 1.4-1.5* > = > > \* Multi-buffer library 1.0 or newer only works for Meson but not Make > build system. > diff --git a/doc/guides/cryptodevs/kasumi.rst > b/doc/guides/cryptodevs/kasumi.rst > index 0989054875..a8f4e6b204 100644 > --- a/doc/guides/cryptodevs/kasumi.rst > +++ b/doc/guides/cryptodevs/kasumi.rst > @@ -87,7 +87,8 @@ and the external crypto libraries supported by them: > = > 16.11 - 19.11 LibSSO KASUMI > 20.02 - 21.08 Multi-buffer library 0.53 - 1.3* > - 21.11+ Multi-buffer library 1.0 - 1.5* > + 21.11 - 23.11 Multi-buffer library 1.0 - 1.5* > + 24.03+ Multi-buffer library 1.4 - 1.5* > = > > \* Multi-buffer library 1.0 or newer only works for Meson but not Make > build system. > diff --git a/doc/guides/cryptodevs/snow3g.rst > b/doc/guides/cryptodevs/snow3g.rst > index 3392932653..46863462e5 100644 > --- a/doc/guides/cryptodevs/snow3g.rst > +++ b/doc/guides/cryptodevs/snow3g.rst > @@ -96,7 +96,8 @@ and the external crypto libraries supported by them: > = > 16.04 - 19.11 LibSSO SNOW3G > 20.02 - 21.08 Multi-buffer library 0.53 - 1.3* > - 21.11+ Multi-buffer lib
Re: [EXT] Re: [PATCH v2 8/8] crypto/ipsec_mb: set and use session ID
On Mon, Feb 26, 2024 at 6:24 PM Wathsala Wathawana Vithanage < wathsala.vithan...@arm.com> wrote: > Hi Patrick, > > I tried mainline earlier today it compiles and links fine. However, > build failed on > > v1.4. We are working on tagging the mainline, until then please continue > > working on mainline. > > > A new tag SECLIB-IPSEC-2023.10.13 has been created from the tip of arm > ipsec-mb git repo. > Please use this tag going forward, it has been tested and works as > expected. > > Thanks Wathsala - will do!
Re: [PATCH v2] dts: strip whitespaces from stdout and stderr
Acked-by: Patrick Robb
Re: [PATCH] crypto/mlx5: add max segment assert
The Community Lab had an infra failure this morning and some patches
including yours were affected with false failures. The issue is now
resolved and we are rerunning the tests in question for all patches
submitted today.
On Fri, Mar 1, 2024 at 7:43 AM Suanming Mou wrote:
> Currently, for multi-segment mbuf, before crypto WQE an extra
> UMR WQE will be introduced to build the contiguous memory space.
> Crypto WQE uses that contiguous memory space key as input.
>
> This commit adds assert for maximum supported segments in debug
> mode in case the segments exceed UMR's limitation.
>
> Signed-off-by: Suanming Mou
> Acked-by: Matan Azrad
> ---
> drivers/crypto/mlx5/mlx5_crypto_gcm.c | 6 ++
> 1 file changed, 6 insertions(+)
>
> diff --git a/drivers/crypto/mlx5/mlx5_crypto_gcm.c
> b/drivers/crypto/mlx5/mlx5_crypto_gcm.c
> index 8b9953b46d..fc6ade6711 100644
> --- a/drivers/crypto/mlx5/mlx5_crypto_gcm.c
> +++ b/drivers/crypto/mlx5/mlx5_crypto_gcm.c
> @@ -441,6 +441,9 @@ mlx5_crypto_gcm_get_op_info(struct mlx5_crypto_qp *qp,
> op_info->digest = NULL;
> op_info->src_addr = aad_addr;
> if (op->sym->m_dst && op->sym->m_dst != m_src) {
> + /* Add 2 for AAD and digest. */
> + MLX5_ASSERT((uint32_t)(m_dst->nb_segs + m_src->nb_segs +
> 2) <
> + qp->priv->max_klm_num);
> op_info->is_oop = true;
> m_dst = op->sym->m_dst;
> dst_addr = rte_pktmbuf_mtod_offset(m_dst, void *,
> op->sym->aead.data.offset);
> @@ -457,6 +460,9 @@ mlx5_crypto_gcm_get_op_info(struct mlx5_crypto_qp *qp,
> op_info->need_umr = true;
> return;
> }
> + } else {
> + /* Add 2 for AAD and digest. */
> + MLX5_ASSERT((uint32_t)(m_src->nb_segs) + 2 <
> qp->priv->max_klm_num);
> }
> if (m_src->nb_segs > 1) {
> op_info->need_umr = true;
> --
> 2.34.1
>
>
Re: [PATCH] crypto/mlx5: add virtual function device ID
The Community CI Testing Lab had an infra failure this morning and some
patches including yours were affected with false failures. The issue is now
resolved and we are rerunning the tests in question for all patches
submitted today.
On Fri, Mar 1, 2024 at 7:31 AM Suanming Mou wrote:
> This adds the virtual function device ID to the list of
> supported NVIDIA devices that run the MLX5 compress PMD.
>
> Signed-off-by: Suanming Mou
> Acked-by: Matan Azrad
> ---
> drivers/crypto/mlx5/mlx5_crypto.c | 4
> 1 file changed, 4 insertions(+)
>
> diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
> b/drivers/crypto/mlx5/mlx5_crypto.c
> index 4bac723c8b..26bd4087da 100644
> --- a/drivers/crypto/mlx5/mlx5_crypto.c
> +++ b/drivers/crypto/mlx5/mlx5_crypto.c
> @@ -465,6 +465,10 @@ static const struct rte_pci_id
> mlx5_crypto_pci_id_map[] = {
> RTE_PCI_DEVICE(PCI_VENDOR_ID_MELLANOX,
> PCI_DEVICE_ID_MELLANOX_BLUEFIELD3)
> },
> + {
> + RTE_PCI_DEVICE(PCI_VENDOR_ID_MELLANOX,
> + PCI_DEVICE_ID_MELLANOX_CONNECTXVF)
> + },
> {
> .vendor_id = 0
> }
> --
> 2.34.1
>
>
Re: [PATCH v4 0/7] test case blocking and logging
The Community CI Testing Lab had an infra failure this morning and some patches including yours were affected with false failures. The issue is now resolved and we are rerunning the tests in question for all patches submitted today. On Fri, Mar 1, 2024 at 5:55 AM Juraj Linkeš wrote: > We currently don't record test case results that couldn't be executed > because of a previous failure, such as when a test suite setup failed, > resulting in no executed test cases. > > In order to record the test cases that couldn't be executed, we must > know the lists of test suites and test cases ahead of the actual test > suite execution, as an error could occur before we even start executing > test suites. > > In addition, the patch series contains two refactors and one > improvement. > > The first refactor is closely related. The dts.py was renamed to > runner.py and given a clear purpose - running the test suites and all > other orchestration needed to run test suites. The logic for this was > not all in the original dts.py module and it was brought there. The > runner is also responsible for recording results, which is the blocked > test cases are recorded. > > The other refactor, logging, is related to the first refactor. The > logging module was simplified while extending capabilities. Each test > suite logs into its own log file in addition to the main log file which > the runner must handle (as it knows when we start executing particular > test suites). The runner also handles the switching between execution > stages for the purposes of logging. > > The one aforementioned improvement is in unifying how we specify test > cases in the conf.yaml file and in the environment variable/command line > argument. > > v2: > Rebase and update of the whole patch. There are changes in all parts of > the code, mainly improving the design and logic. > Also added the last patch which improves test suite specification on the > cmdline. > > v3: > Improved variables in _get_test_suite_class along with docstring. > Fixed smoke test suite not being added into the list of test suites to > be executed. > > v4: > Added the checking of CamelCase convention when discovering test cases. > Also added additional test stages. The stages were split into setup and > teardown. > > Juraj Linkeš (7): > dts: convert dts.py methods to class > dts: move test suite execution logic to DTSRunner > dts: filter test suites in executions > dts: reorganize test result > dts: block all test cases when earlier setup fails > dts: refactor logging configuration > dts: improve test suite and case filtering > > doc/guides/tools/dts.rst | 14 +- > dts/framework/config/__init__.py | 36 +- > dts/framework/config/conf_yaml_schema.json| 2 +- > dts/framework/dts.py | 338 - > dts/framework/logger.py | 246 +++--- > dts/framework/remote_session/__init__.py | 6 +- > .../interactive_remote_session.py | 6 +- > .../remote_session/interactive_shell.py | 6 +- > .../remote_session/remote_session.py | 8 +- > dts/framework/runner.py | 711 ++ > dts/framework/settings.py | 188 +++-- > dts/framework/test_result.py | 565 -- > dts/framework/test_suite.py | 239 +- > dts/framework/testbed_model/node.py | 11 +- > dts/framework/testbed_model/os_session.py | 7 +- > .../traffic_generator/traffic_generator.py| 6 +- > dts/main.py | 9 +- > dts/pyproject.toml| 3 + > dts/tests/TestSuite_os_udp.py | 2 +- > dts/tests/TestSuite_smoke_tests.py| 2 +- > 20 files changed, 1375 insertions(+), 1030 deletions(-) > delete mode 100644 dts/framework/dts.py > create mode 100644 dts/framework/runner.py > > -- > 2.34.1 > >
Re: [PATCH v5 0/4] add pointer compression API
The Community CI Testing Lab had an infra failure this morning and some patches including yours were affected with false failures. The issue is now resolved and we are rerunning the tests in question for all patches submitted today. On Fri, Mar 1, 2024 at 6:16 AM Morten Brørup wrote: > > From: Konstantin Ananyev [mailto:konstantin.anan...@huawei.com] > > Sent: Thursday, 22 February 2024 17.16 > > > > > For some reason your email is not visible to me, even though it's in > the > > > archive. > > > > No worries. > > > > > > > > On 02/11/202416:32,Konstantin Ananyev konstantin.v.ananyev wrote: > > > > > > > From one side the code itself is very small and straightforward, > > from > > other side - it is not clear to me what is intended usage for it > > > > within DPDK and it's applianances? > > > > Konstantin > > > > > > The intended usage is explained in the cover email (see below) and > > demonstrated > > > in the test supplied in the following patch - when sending arrays of > > pointers > > > between cores as it happens in a forwarding example. > > > > Yes, I saw that. The thing is that test is a 'synthetic' one. > > My question was about how do you expect people to use it in more > realistic > > scenarios? > > Let say user has a bunch of mbuf pointers, possibly from different > mempools. > > How he can use this API: how to deduce the base pointer for all of them > and > > what to > > do if it can't be done? > > I share Konstantin's concerns with this feature. > > If we want to compress mbuf pointers in applications with a few mbuf > pools, e.g. an mbuf pool per CPU socket, the compression algorithm would be > different. > > I would like to add: > If we want to offer optimizations specifically for applications with a > single mbuf pool, I think it should be considered in a system-wide context > to determine if performance could be improved in more areas. > E.g. removing the pool field from the rte_mbuf structure might free up > space to move hot fields from the second cache line to the first, so the > second cache line rarely needs to be touched. (As an alternative to > removing the pool field, it could be moved to the second cache line, only > to be used if the global "single mbuf pool" is NULL.) > > On the other hand, I agree that pointer compression can be useful for some > applications, so we should accept it. > > However, pointer compression has nothing to do with the underlying > hardware or operating system, so it does not belong in the EAL (which is > already too bloated). It should be a separate library. > > > > > > On 01/11/2023 18:12, Paul Szczepanek wrote: > > > > > > > This patchset is proposing adding a new EAL header with utility > functions > > > > that allow compression of arrays of pointers. > > > > > > > > When passing caches full of pointers between threads, memory > containing > > > > the pointers is copied multiple times which is especially costly > between > > > > cores. A compression method will allow us to shrink the memory size > > > > copied. > > > > > > > > The compression takes advantage of the fact that pointers are usually > > > > located in a limited memory region (like a mempool). We can compress > them > > > > by converting them to offsets from a base memory address. > > > > > > > > Offsets can be stored in fewer bytes (dictated by the memory region > size > > > > and alignment of the pointer). For example: an 8 byte aligned pointer > > > > which is part of a 32GB memory pool can be stored in 4 bytes. The > API is > > > > very generic and does not assume mempool pointers, any pointer can be > > > > passed in. > > > > > > > > Compression is based on few and fast operations and especially with > vector > > > > instructions leveraged creates minimal overhead. > > > > > > > > The API accepts and returns arrays because the overhead means it > only is > > > > worth it when done in bulk. > > > > > > > > Test is added that shows potential performance gain from > compression. In > > > > this test an array of pointers is passed through a ring between two > cores. > > > > It shows the gain which is dependent on the bulk operation size. In > this > > > > synthetic test run on ampere altra a substantial (up to 25%) > performance > > > > gain is seen if done in bulk size larger than 32. At 32 it breaks > even and > > > > lower sizes create a small (less than 5%) slowdown due to overhead. > > > > > > > > In a more realistic mock application running the l3 forwarding dpdk > > > > example that works in pipeline mode on two cores this translated > into a > > > > ~5% throughput increase on an ampere altra. > > Which burst size was used to achieve this ~5% throughput increase? > > > > > > > > > v2: > > > > * addressed review comments (style, explanations and typos) > > > > * lowered bulk iterations closer to original numbers to keep runtime > short > > > > * fixed pointer size warning on 32-bit arch > > > > v3: > > > > * added 16-bit versions of compression functions and tests > > > > * ad
Re: [PATCH v5] net/i40e: add diagnostic support in TX path
The Community CI Testing Lab had an infra failure this morning and some patches including yours were affected with false failures. The issue is now resolved and we are rerunning the tests in question for all patches submitted today. On Fri, Mar 1, 2024 at 5:25 AM Bruce Richardson wrote: > On Fri, Mar 01, 2024 at 09:44:21AM +, Mingjin Ye wrote: > > Implemented a Tx wrapper to perform a thorough check on mbufs, > > categorizing and counting invalid cases by types for diagnostic > > purposes. The count of invalid cases is accessible through xstats_get. > > > > Also, the devarg option "mbuf_check" was introduced to configure the > > diagnostic parameters to enable the appropriate diagnostic features. > > > > supported cases: mbuf, size, segment, offload. > > 1. mbuf: check for corrupted mbuf. > > 2. size: check min/max packet length according to hw spec. > > 3. segment: check number of mbuf segments not exceed hw limitation. > > 4. offload: check any unsupported offload flag. > > > > parameter format: "mbuf_check=" or "mbuf_check=[,]" > > eg: dpdk-testpmd -a :81:01.0,mbuf_check=[mbuf,size] -- -i > > > > Signed-off-by: Mingjin Ye > > --- > > v2: remove strict. > > --- > > v3: optimised. > > --- > > v4: rebase. > > --- > > v5: fix ci error. > > --- > > doc/guides/nics/i40e.rst | 13 +++ > > drivers/net/i40e/i40e_ethdev.c | 138 - > > drivers/net/i40e/i40e_ethdev.h | 28 ++ > > drivers/net/i40e/i40e_rxtx.c | 153 +++-- > > drivers/net/i40e/i40e_rxtx.h | 2 + > > 5 files changed, 326 insertions(+), 8 deletions(-) > > > > diff --git a/doc/guides/nics/i40e.rst b/doc/guides/nics/i40e.rst > > index 15689ac958..bf1d1e5d60 100644 > > --- a/doc/guides/nics/i40e.rst > > +++ b/doc/guides/nics/i40e.rst > > @@ -275,6 +275,19 @@ Runtime Configuration > > > >-a 84:00.0,vf_msg_cfg=80@120:180 > > > > +- ``Support TX diagnostics`` (default ``not enabled``) > > + > > + Set the ``devargs`` parameter ``mbuf_check`` to enable TX > diagnostics. For example, > > + ``-a 18:01.0,mbuf_check=`` or ``-a > 18:01.0,mbuf_check=[,...]``. Also, > > + ``xstats_get`` can be used to get the error counts, which are > collected in > > + ``tx_mbuf_error_packets`` xstats. For example, ``testpmd> show port > xstats all``. > > + Supported cases: > > + > > + * mbuf: Check for corrupted mbuf. > > + * size: Check min/max packet length according to hw spec. > > + * segment: Check number of mbuf segments not exceed hw limitation. > > + * offload: Check any unsupported offload flag. > > + > > Hi Mingjin, > > please see the changes made to the equivalent doc (and commit-log) updates > for iavf when I applied that earlier patch to next-net-intel. This patch > should be updated to match that. Changes were pretty basic, but still > useful, for example, aligning line breaks to punctuation. > > Thanks, > /Bruce > > PS: This feedback applies to the net/ice patch too. >
Re: [PATCH v3] net/ice: add diagnostic support in TX path
The Community CI Testing Lab had an infra failure this morning and some
patches including yours were affected with false failures. The issue is now
resolved and we are rerunning the tests in question for all patches
submitted today.
On Fri, Mar 1, 2024 at 5:11 AM Mingjin Ye wrote:
> Implemented a Tx wrapper to perform a thorough check on mbufs,
> categorizing and counting invalid cases by types for diagnostic
> purposes. The count of invalid cases is accessible through xstats_get.
>
> Also, the devarg option "mbuf_check" was introduced to configure the
> diagnostic parameters to enable the appropriate diagnostic features.
>
> supported cases: mbuf, size, segment, offload.
> 1. mbuf: check for corrupted mbuf.
> 2. size: check min/max packet length according to hw spec.
> 3. segment: check number of mbuf segments not exceed hw limitation.
> 4. offload: check any unsupported offload flag.
>
> parameter format: "mbuf_check=" or "mbuf_check=[,]"
> eg: dpdk-testpmd -a :81:01.0,mbuf_check=[mbuf,size] -- -i
>
> Signed-off-by: Mingjin Ye
> ---
> v2: rebase.
> ---
> v3: Modify comment log.
> ---
> doc/guides/nics/ice.rst | 13 +++
> drivers/net/ice/ice_ethdev.c | 108 +++-
> drivers/net/ice/ice_ethdev.h | 23 +
> drivers/net/ice/ice_rxtx.c | 158 ---
> drivers/net/ice/ice_rxtx.h | 20 +
> 5 files changed, 311 insertions(+), 11 deletions(-)
>
> diff --git a/doc/guides/nics/ice.rst b/doc/guides/nics/ice.rst
> index bafb3ba022..d1aee811b3 100644
> --- a/doc/guides/nics/ice.rst
> +++ b/doc/guides/nics/ice.rst
> @@ -257,6 +257,19 @@ Runtime Configuration
>As a trade-off, this configuration may cause the packet processing
> performance
>degradation due to the PCI bandwidth limitation.
>
> +- ``Tx diagnostics`` (default ``not enabled``)
> +
> + Set the ``devargs`` parameter ``mbuf_check`` to enable TX diagnostics.
> For example,
> + ``-a 18:01.0,mbuf_check=`` or ``-a
> 18:01.0,mbuf_check=[,...]``.
> + Also, ``xstats_get`` can be used to get the error counts, which are
> collected in
> + ``tx_mbuf_error_packets`` xstats. For example, ``testpmd> show port
> xstats all``.
> + Supported cases:
> +
> + * mbuf: Check for corrupted mbuf.
> + * size: Check min/max packet length according to hw spec.
> + * segment: Check number of mbuf segments not exceed hw limitation.
> + * offload: Check any unsupported offload flag.
> +
> Driver compilation and testing
> --
>
> diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
> index 72e13f95f8..d28e205375 100644
> --- a/drivers/net/ice/ice_ethdev.c
> +++ b/drivers/net/ice/ice_ethdev.c
> @@ -12,6 +12,7 @@
> #include
>
> #include
> +#include
>
> #include "eal_firmware.h"
>
> @@ -34,6 +35,7 @@
> #define ICE_HW_DEBUG_MASK_ARG "hw_debug_mask"
> #define ICE_ONE_PPS_OUT_ARG "pps_out"
> #define ICE_RX_LOW_LATENCY_ARG"rx_low_latency"
> +#define ICE_MBUF_CHECK_ARG "mbuf_check"
>
> #define ICE_CYCLECOUNTER_MASK 0xULL
>
> @@ -49,6 +51,7 @@ static const char * const ice_valid_args[] = {
> ICE_ONE_PPS_OUT_ARG,
> ICE_RX_LOW_LATENCY_ARG,
> ICE_DEFAULT_MAC_DISABLE,
> + ICE_MBUF_CHECK_ARG,
> NULL
> };
>
> @@ -319,6 +322,14 @@ static const struct ice_xstats_name_off
> ice_stats_strings[] = {
> #define ICE_NB_ETH_XSTATS (sizeof(ice_stats_strings) / \
> sizeof(ice_stats_strings[0]))
>
> +static const struct ice_xstats_name_off ice_mbuf_strings[] = {
> + {"tx_mbuf_error_packets", offsetof(struct ice_mbuf_stats,
> + tx_pkt_errors)},
> +};
> +
> +#define ICE_NB_MBUF_XSTATS (sizeof(ice_mbuf_strings) / \
> + sizeof(ice_mbuf_strings[0]))
> +
> static const struct ice_xstats_name_off ice_hw_port_strings[] = {
> {"tx_link_down_dropped", offsetof(struct ice_hw_port_stats,
> tx_dropped_link_down)},
> @@ -2061,6 +2072,58 @@ handle_pps_out_arg(__rte_unused const char *key,
> const char *value,
> return 0;
> }
>
> +static int
> +ice_parse_mbuf_check(__rte_unused const char *key, const char *value,
> void *args)
> +{
> + char *cur;
> + char *tmp;
> + int str_len;
> + int valid_len;
> +
> + int ret = 0;
> + uint64_t *mc_flags = args;
> + char *str2 = strdup(value);
> + if (str2 == NULL)
> + return -1;
> +
> + str_len = strlen(str2);
> + if (str_len == 0) {
> + ret = -1;
> + goto err_end;
> + }
> +
> + /* Try stripping the outer square brackets of the parameter
> string. */
> + str_len = strlen(str2);
> + if (str2[0] == '[' && str2[str_len - 1] == ']') {
> + if (str_len < 3) {
> + ret = -1;
> + goto err_end;
> + }
> + valid_len = str_len - 2;
> + memmove(str2, str2 + 1, valid_len
Re: [PATCH] net/mlx5: set correct priority for meter policy
The Community CI Testing Lab had an infra failure this morning and some
patches including yours were affected with false failures. The issue is now
resolved and we are rerunning the tests in question for all patches
submitted today.
On Fri, Mar 1, 2024 at 3:46 AM Shun Hao wrote:
> Currently a meter policy's flows are always using the same priority for
> all colors, so the red color flow might be before green/yellow ones.
> This will impact the performance cause green/yellow packets will check
> red flow first and got miss, then match green/yellow flows, introducing
> more hops.
>
> This patch fixes this by giving the same priority to flows for all
> colors.
>
> Fixes: 363db9b00f ("net/mlx5: handle yellow case in default meter policy")
> CC: sta...@dpdk.org
>
> Signed-off-by: Shun Hao
> Acked-by: Bing Zhao
> Acked-by: Matan Azrad
> ---
> drivers/net/mlx5/mlx5_flow_dv.c | 41 +++--
> 1 file changed, 24 insertions(+), 17 deletions(-)
>
> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c
> b/drivers/net/mlx5/mlx5_flow_dv.c
> index 18f09b22be..f1584ed6e0 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -17922,9 +17922,8 @@ __flow_dv_create_policy_matcher(struct rte_eth_dev
> *dev,
> }
> }
> tbl_data = container_of(tbl_rsc, struct mlx5_flow_tbl_data_entry,
> tbl);
> - if (priority < RTE_COLOR_RED)
> - flow_dv_match_meta_reg(matcher.mask.buf,
> - (enum modify_reg)color_reg_c_idx, color_mask,
> color_mask);
> + flow_dv_match_meta_reg(matcher.mask.buf,
> + (enum modify_reg)color_reg_c_idx, color_mask, color_mask);
> matcher.priority = priority;
> matcher.crc = rte_raw_cksum((const void *)matcher.mask.buf,
> matcher.mask.size);
> @@ -17975,7 +17974,6 @@ __flow_dv_create_domain_policy_rules(struct
> rte_eth_dev *dev,
> int i;
> int ret = mlx5_flow_get_reg_id(dev, MLX5_MTR_COLOR, 0, &flow_err);
> struct mlx5_sub_policy_color_rule *color_rule;
> - bool svport_match;
> struct mlx5_sub_policy_color_rule *tmp_rules[RTE_COLORS] = {NULL};
>
> if (ret < 0)
> @@ -18011,10 +18009,9 @@ __flow_dv_create_domain_policy_rules(struct
> rte_eth_dev *dev,
> /* No use. */
> attr.priority = i;
> /* Create matchers for colors. */
> - svport_match = (i != RTE_COLOR_RED) ? match_src_port :
> false;
> if (__flow_dv_create_policy_matcher(dev, color_reg_c_idx,
> MLX5_MTR_POLICY_MATCHER_PRIO, sub_policy,
> - &attr, svport_match, NULL,
> + &attr, match_src_port, NULL,
> &color_rule->matcher, &flow_err)) {
> DRV_LOG(ERR, "Failed to create color%u matcher.",
> i);
> goto err_exit;
> @@ -18024,7 +18021,7 @@ __flow_dv_create_domain_policy_rules(struct
> rte_eth_dev *dev,
> color_reg_c_idx, (enum rte_color)i,
> color_rule->matcher,
> acts[i].actions_n, acts[i].dv_actions,
> - svport_match, NULL, &color_rule->rule,
> + match_src_port, NULL, &color_rule->rule,
> &attr)) {
> DRV_LOG(ERR, "Failed to create color%u rule.", i);
> goto err_exit;
> @@ -18907,7 +18904,7 @@ flow_dv_meter_hierarchy_rule_create(struct
> rte_eth_dev *dev,
> struct {
> struct mlx5_flow_meter_policy *fm_policy;
> struct mlx5_flow_meter_info *next_fm;
> - struct mlx5_sub_policy_color_rule
> *tag_rule[MLX5_MTR_RTE_COLORS];
> + struct mlx5_sub_policy_color_rule *tag_rule[RTE_COLORS];
> } fm_info[MLX5_MTR_CHAIN_MAX_NUM] = { {0} };
> uint32_t fm_cnt = 0;
> uint32_t i, j;
> @@ -18941,14 +18938,22 @@ flow_dv_meter_hierarchy_rule_create(struct
> rte_eth_dev *dev,
> mtr_policy = fm_info[i].fm_policy;
> rte_spinlock_lock(&mtr_policy->sl);
> sub_policy = mtr_policy->sub_policys[domain][0];
> - for (j = 0; j < MLX5_MTR_RTE_COLORS; j++) {
> + for (j = 0; j < RTE_COLORS; j++) {
> uint8_t act_n = 0;
> - struct mlx5_flow_dv_modify_hdr_resource
> *modify_hdr;
> + struct mlx5_flow_dv_modify_hdr_resource
> *modify_hdr = NULL;
> struct mlx5_flow_dv_port_id_action_resource
> *port_action;
> + uint8_t fate_action;
>
> - if (mtr_policy->act_cnt[j].fate_action !=
> MLX5_FLOW_FATE_MTR &&
> - mtr_policy->act_cnt[j].fate_action !=
> MLX5_FLOW_FATE
Depends-on patchseries support via git-pw or patchwork
Hi all, As some of you know from discussions at DPDK CI meetings, Adam from UNH is writing a script which leverages git-pw, and takes as arguments a patch series patchwork id, patchwork project, and pw token, and produces a project artifact for CI testing purposes. Starting in January we will use it for applying patches to DPDK and creating our dpdk.tar.gz artifacts for testing. And, we will submit it to the dpdk-ci repo. Anyways, when we originally discussed the idea, Thomas suggested that we implement the depends-on functionality by contributing to the git-pw project, as opposed to implementing the depend-on support in the create artifact script itself. Adam did create a github issue on the git-pw project in order to poll the community for interest in this feature, and one of the patchwork maintainers chimed in to suggest that rather than implementing the feature on the client side via git-pw, it should simply be implemented for patchwork itself. That way if it's patchwork server side and exposed via the api, other client side tools like pwclient can also receive the benefits. I just wanted to flag this on the ci mailing list so that anyone with thoughts could submit them on the Github issue, which you can find here: https://github.com/getpatchwork/git-pw/issues/71 Thanks Adam for pushing this effort forward.
Re: [PATCH] doc: update minimum Linux kernel version
On Thu, Jan 11, 2024 at 4:18 AM Morten Brørup wrote: > > I wonder if any automated tests are using this specific kernel version, or > if we are only testing using the distros' native kernels. @Aaron? > For UNH, we generally run from the distros' native kernels. Any exceptions are not for testing older kernel versions, so we don't have anything running from 4.14 in our testing right now. If running some automated testing for, say, the minimum supported kernel version at any point in time is a value to anyone, certainly they can speak up and we can discuss adding that.
Re: [PATCH] doc: update minimum Linux kernel version
On Thu, Jan 11, 2024 at 2:27 PM Morten Brørup wrote: > *From:* Patrick Robb [mailto:pr...@iol.unh.edu] > *Sent:* Thursday, 11 January 2024 20.02 > > > > On Thu, Jan 11, 2024 at 4:18 AM Morten Brørup > wrote: > > > I wonder if any automated tests are using this specific kernel version, or > if we are only testing using the distros' native kernels. @Aaron? > > > > For UNH, we generally run from the distros' native kernels. Any exceptions > are not for testing older kernel versions, so we don't have anything > running from 4.14 in our testing right now. > > > > If running some automated testing for, say, the minimum supported kernel > version at any point in time is a value to anyone, certainly they can speak > up and we can discuss adding that. > > > > > > When the documentation specifies a minimum required kernel version, it > implicitly claims that DPDK works with that kernel version. > > So we should either test with the specified kernel version (which requires > significant effort to set up, so I’m not going to ask for it!), or add a > big fat disclaimer/warning that DPDK is not tested with the mentioned > kernel version, and list the kernel versions actually tested. > Well, adding one system which moves with the minimum supported kernel version probably wouldn't be too onerous, so I will add a ticket noting this as a community request. On the other hand, one of the reasons we're moving to running more and more of our testing from containers is so that we don't have to do as much VM "babysitting" and our testing environment is more cleanly defined. Obviously that doesn't apply in this case, so we'd set up a custom VM for this testing job specifically. But, as an LTS kernel version reaches EOL approximately 1x per year, it shouldn't be too bad. > > > As an appliance vendor, we build our systems from scratch, including the > bootloader, kernel and file systems. We don’t use any of the distro stuff. > > Having information about well tested kernel versions would be helpful when > choosing kernel version for our appliances. I suppose other appliance > vendors also use their own hardened/purpose-built kernel versions, and > would consider this information useful for their decision process too. > > > Maybe the best thing we can do without a significant overhaul of our process is to more explicitly display the kernel version for a system when reporting results. If others chime in and there is big interest here, we can go further.
Re: Minutes of Techboard Meeting, 2024-01-10
On Tue, Jan 16, 2024 at 11:45 AM Honnappa Nagarahalli < honnappa.nagaraha...@arm.com> wrote: > > 3. How well does DPDK run on Hyperscaler environments? > - Can UNH take the task of running DPDK on Hyperscaler environments? UNH > seems to have lot of things already. > Sorry, I could not make techboard this week as I had a conflict from another project. I would like to reiterate that in principle an effective way to start addressing any gaps is to use testing to better understand the landscape as is. At the same time, yes this conversation did not reach maturity during 2023, so there aren't hyperscaler testing work items in 2024's Community Lab SOW. Written SOW items always have to take precedence, but we will continue to accept ad-hoc community requests like always, and engage in conversations for what's next for testing in DPDK. We had an email chain going with Robin/Honnappa about the most needed test coverage in this space, etc. but it died out. I remember Honnappa mentioning the importance of measuring multi core scalability performance, running l2fwd l3fwd sample apps etc. I think Robin may host some calls in 2024 to take feedback on this. We would still need a plan on funding for the cloud instances used, list of vNICs to provide testing coverage for, a test plan, etc. So, realistically today we are very far off, but if the conversation on this continues to develop, we can engage and define better if/how the Community Lab could play a role.
Community CI Meeting Minutes - January 18, 2024
January 18, 2024 # Attendees 1. Lincoln Lavoie 2. Thomas Monjalon 3. Aaron Conole 4. Jeremy Spewock 5. Paul Szczepanek 6. Ali Alnubani 7. Juraj Linkeš # Minutes = General Announcements * First 2024 DTS WG meeting was held yesterday, and the minutes are here: https://docs.google.com/document/d/1pG_NGuwYgPuovwIfhvcs9u8PNYIJuInsFr0GeTUIU4k/edit?usp=sharing * Patrick Robb will publish the meeting minutes = CI Status - UNH-IOL Community Lab * Octeon CN106XX: Patrick worked with Hiral this week about the process for SDK rebuild. This will have to happen regularly, but we now have a VM setup which will serve as the place to rebuild SDK and act as TFTP server for the Octeon board. * After that we need to iron out switching rootfs to ubuntu, setting up DTS on the tester, validating this works fine in a CI context (Phanendra from Marvell has approved of the concept). * The Intel server arrived in the mail yesterday. We will mount the system this week and begin setup. As a reminder this now unblocks: * E810 testing for Intel * Traffic gen for the CX7 testing (this server will act as TG) * Traffic gen for the Octeon CN106xx board * The new create dpdk artifact for ci testing script is in production at UNH, and Adam is submitting the V3 patchseries for this to dpdk-ci today * There is an update from the patchwork maintainer about supporting Depends-on via patchwork. I encourage everyone to read his full thoughts on the issue below, but shorthand conclusions are: * He prefers to only support Depends-on on a series basis, not a series or patch basis. * Patrick will update the CI testing thread on this topic, but this may require the DPDK community agreeing to a new approach. From looking through the dev mailing list, it seems that series dependency is the typical use, but there are also some examples of developers using patch dependency. * Keep the same syntax which allows for depends-on: patch, but in reality translate this to depends on that patch’s series * He will need some help for the effort. We can ping the DPDK community to look for volunteers. Or, it can possibly become a community ask for development at the DPDK Community Lab. Looking at it at a high level I don’t think the scope will be too bad. * Full thread: https://github.com/getpatchwork/git-pw/issues/71 * Arm-Ampere still needs a kernel rebuild for the QAT card * Standing up ts-factory testing framework * Adam did the first cx-5 run on an ARM system, and will do an Intel XL710 run on an ARM system today. Both will be published on Oktet Lab’s Bublik for review. * Unlike our ARM testbeds, our x86 servers are single server TG/DUT testbeds, which breaks an assumption in ts-factory. My view on how to proceed is bring testing online with what works now (arm), then review the value with the community, and choose a strategy for x86 based on what we learn from running this at the lab. * Andrew from Oktet labs has added some missing components to GitHub for the data visualization tool (Bublik), and has finished categorizing the tests for the XL710 and the expected results for that NIC * Old DTS patches: * We noticed that as DPDK has grown, the compile time also increases and the old compile timeout for the DUT is no longer valid on some slower systems. Jeremy will submit a patch extending the timeout. * Once the cx7 is online, patrick will submit the patch adding the cx6 and cx7 to the NIC registry in DTS * Lincoln saw the email thread about DPDK failing to build on FreeBSD 14. We are only testing FreeBSD 13 in the lab right now, so we will add coverage for 14. - Intel Lab * John says there is still no new person who can act as a contact. There have been further staffing changes at Intel, so it may be hard to get a contact soon, but Patrick will keep asking. - Github Actions * Cirrus CI: Adding support for this to the 0 day robot. Aaron submitted a patch for polling for Cirrus ci status in ovs. * They are getting a server to migrate the VM over to, so there should be minimal downtime associated with the hardware moving from Westford to another state. - Loongarch Lab * None = DTS Improvements & Test Development * scapy/templated yaml for tests: * One motivation is for making writing simple tests in minimal time (like 3 minutes). Should we aim for a target of writing a testsuite in minutes, not h
Fwd: DTS WG Meeting Minutes - January 17, 2024
I forgot to CC the dev mailing list -- Forwarded message - From: Patrick Robb Date: Thu, Jan 18, 2024 at 2:52 PM Subject: DTS WG Meeting Minutes - January 17, 2024 To: Cc: , NBU-Contact-Thomas Monjalon (EXTERNAL) < tho...@monjalon.net>, Juraj Linkeš January 17, 2024 # Attendees * Juraj Linkeš * Gregory Etelson * Honnappa Nagarahalli * Jeremy Spewock * Luca Vizzarro * Paul Szczepanek # Agenda * Additions to the agenda * Patch discussions * DTS Developer documentation * 24.03 roadmap # Minutes = Additions to the agenda * Nothing = YAML test suites * Greg wanted to automate his testing; started with test writtens in Python, but was not scalable; easily understandable by newcomers. * The idea is to take an application, send commands (interactive input), collect output and compare with expected strings. * The code was available as of two months ago, but no longer is (private on GitHub). Greg may be able to share it once taking care of it in his company. * Gregory submitted an idea for writing test suites in yaml, which just passes values into a templated testpmd testsuite. * Do we want to support a secondary way of writing test suites? * Will this be usable for both functional and performance testing? * Will this coexist well with the current method? * The current method also aims to be minimalistic and intuitive * Coexistence makes sense as the yaml approach may not be able to cover more complicated cases * What are any limitations which this places on the testing framework? If there aren’t major downsides, then the benefits in terms of quickly adding new testpmd testsuites seems significant. * The traffic generator can't be configured here, we need an abstraction that works for all traffic generators; we can mark the test cases as functional/performance though, which could be enough * We can only specify test-specific testpmd cmdline options; shouldn't be a problem, but we have to keep in mind that configuration such as cores and pci addresses are configured elsewhere (the testbed configuration) * Using specific strings in testpmd is harder to maintain (if the same config is used in multiple places) * Are the phases for both setup/teardown and test cases? This could complicate results recording * Can we easily specify multiple test cases? I.e. we have a test method and we want to test different input combinations (the inputs could just be the number of cores/packet size for performance tests)
Re: DTS testpmd and SCAPY integration
Thanks for summarizing Juraj. As we discussed at the CI meeting yesterday, we should get together next week to workshop some rough testsuites based on the ideas which have been proposed above, and discussed in the DTS meeting and CI testing meeting. Just to get something on the calendar I scheduled a zoom for next wednesday at 2pm UTC, but yall let me know if you prefer something different. All are welcome of course. https://unh.zoom.us/j/94747183133 On Thu, Jan 18, 2024 at 7:32 AM Juraj Linkeš wrote: > Hi folks, > > Let me summarize the yesterday's discussion in a few keys points: > >- Greg's proposal aims at simplicity and is useful mainly for test >cases which can be written in a few minutes. More complex test cases are >not suitable for the YAML approach. >- The above implies that the YAML based test cases would be supported >alongside the existing approach. This fast way to implement simple test >cases would likely be a valuable addition. >- The big picture idea behind the YAML test cases is to take an >application with interactive input, send commands, collect output and >compare the output with expected string(s). >- Greg may be able to make the code available and may assess how to >integrate it with DTS. > > Regards, > Juraj > > On Mon, Jan 8, 2024 at 6:36 PM Honnappa Nagarahalli < > honnappa.nagaraha...@arm.com> wrote: > >> >> >> > >> > Hello Honnappa, >> > >> > [snip] >> > >> > > Hi Gregory, >> > >I do not fully understand your proposal, it will be helpful to >> join the DTS >> > meetings to discuss this further. >> > > >> > >> > Agree, let's discuss the proposal details during the DTS meeting. >> > >> > > YAML has wide support built around it. By using our own text format, >> we will >> > have to build the parsing support etc ourselves. >> > > >> > > However, YAML is supposed to be easy to read and understand. Is it >> just a >> > matter for getting used to it? >> > > >> > >> > I selected YAML for 2 reasons: >> >* Plain and intuitive YAML format minimized test meta data. >> > By the meta data I refer to control tags and markup characters >> > that are not test commands. >> >* YAML has Python parser. >> I have mis-understood your proposal. I agree with your above comments. >> +1 for the proposal. >> >> > >> > Regards, >> > Gregory >> > >> > >> >> -- Patrick Robb Technical Service Manager UNH InterOperability Laboratory 21 Madbury Rd, Suite 100, Durham, NH 03824 www.iol.unh.edu
Re: DTS testpmd and SCAPY integration
Thank you for sharing Gregory. I did not get an opportunity to look through
the code today, but I did run through the presentation. A few points I
noted:
1. The presentation shows an example testpmd testcase for creating a flow
rule, and then shows a validation step in which standard out is compared
against the expected string ("flow rule x created") and we can conclude
whether we are able to create flow rules. Are you also sending packets
according to the flow rules and validating that what is
sent/received corresponds to the expected behavior of the flow rules? When
I look at the old DTS framework, and an example flow rules testsuite (
https://doc.dpdk.org/dts/test_plans/rte_flow_test_plan.html) which we want
feature parity with, I think that validation for this testing framework
needs to primarily rely on comparing packets sent and packets received. In
any case, there may be some testsuites which can be written which are small
enough in scope that validating by standard out in this way may be
appropriate. I'm not sure but we should keep our options open.
2. If the implementation overhead is not too significant for the
configuration step in the DTS execution a "--fast" option like you use may
be a good improvement for the framework. In your mind, is the main benefit
A. reduced execution time, B. reduced user setup time (don't have to write
full config file) or C. Something else?
Thanks for making this available to use so we can use it as a reference in
making DTS better. :)
On Sun, Jan 28, 2024 at 8:44 AM Gregory Etelson wrote:
> Hello,
>
> I've uploaded the code snapshot and functional description here:
>
> https://drive.google.com/drive/folders/1cHrPwx4fUJ6ibUCtHd4kNKsrmmvQvvOj?usp=drive_link
>
> Regards,
> Gregory
> --
> *From:* Jeremy Spewock
> *Sent:* Tuesday, January 23, 2024 20:26
> *To:* NBU-Contact-Thomas Monjalon (EXTERNAL)
> *Cc:* Honnappa Nagarahalli ; Gregory
> Etelson ; Juraj Linkeš ;
> Paul Szczepanek ; Yoan Picchi <
> yoan.pic...@foss.arm.com>; Patrick Robb ; Luca
> Vizzarro ; c...@dpdk.org ; dev@dpdk.org
> ; nd
> *Subject:* Re: DTS testpmd and SCAPY integration
>
> *External email: Use caution opening links or attachments*
>
>
> On Tue, Jan 23, 2024 at 3:50 AM Thomas Monjalon
> wrote:
>
> 23/01/2024 04:42, Honnappa Nagarahalli:
> > > 08/01/2024 13:10, Luca Vizzarro:
> > > > Your proposal sounds rather interesting. Certainly enabling DTS to
> > > > accept YAML-written tests sounds more developer-friendly and should
> > > > enable quicker test-writing. As this is an extra feature though – and
> > > > a nice-to-have, it should definitely be discussed in the DTS meetings
> > > > as Honnappa suggested already.
> > >
> > > I would not classify this idea as "nice-to-have".
> > > I would split this proposal in 2 parts:
> > > 1/ YAML is an implementation alternative.
> > > 2/ Being able to write a test with a small number of lines,
> > > reusing some commands from existing tools,
> > > should be our "must-have" common goal.
> > >
> > > Others have mentioned that YAML may not be suitable in complex cases,
> and
> > > that it would be an additional language for test writing.
> > > I personnaly think we should focus on a single path which is easy to
> read and
> > > maintain.
> >
> > I think we are digressing from the plan we had put forward if we have to
> go down this path.
> > We should understand what it means by going down the YAML format.
> > Also, what would happen if there is another innovation in 3 months?
>
> There is a misunderstanding here.
> I suggest to take this proposal as an example of the simplicity to reach.
> But I agree with you it is more reasonnable to continue with the Python
> path.
>
>
> I agree that it would be easier to develop both the framework and some of
> the more complex test cases in python and think sticking with Python is a
> good route.
>
> I think taking this proposal and using it as an example of something we
> could work towards is also a great outlook because it gives a more
> structured goal instead of the idea being "let's just make it as simple as
> we can."
>
>
>
> > We already have scatter-gather test suite ported to DPDK repo and has
> undergone review in the community.
> >
> > In the last meeting we went through a simple test case. Is it possible
> to write the scatter-gather test case in YAML and see how they compare?
>
> After the latest CI meeting we thought about writing a simple text case
> in Python with some virtual functions which would abst
DTS WG Meeting Minutes - January 31, 2024
January 31, 2024 # Attendees * Patrick Robb * Juraj Linkeš * Paul Szczepanek * Jeremy Spewock * Luca Vizzarro * Thomas Monjalon # Agenda * Additions to the agenda * Patch discussions * DTS Developer documentation * 24.03 roadmap # Minutes = General Announcements * Added developers for DTS: Nick from UNH is starting on DTS now, and 1-2 more people from UNH will be starting on this project in the near future. = Patch discussions * Dockerfile patch is updated to fix poetry installation and add documentation for using SSH keys. * We are not going to COPY in the keys to the image from the Dockerfile. New approach is volume mounting ~/.ssh to the container at Docker run, and readme is provided explaining this process * Patrick Robb will run this, add a tag on the patch, and request a merge from Thomas * Scatter is submitted and has been reviewed by Juraj, looking for more reviews from community members to get that patch moving. * Can have 1 test for no offload, and 1 test for including offload 1. If scatter is not available, we should expect a failure in the second case. 2. Feature is underdocumented currently 3. We can add a patch adding comments documenting the feature. All lines in ethdev.h should be commented, so it is a “bug” that additions from early dpdk days are not well documented. (RTE_ETH_RX_OFFLOAD_SCATTER) 4. doc/guides/nics/features.rst - this file documents the feature matrix, what the feature does and how we determine if the feature is available. * Docstring patch: What is the latest status of the needed meson changes? Has Bruce provided any feedback? * Bruce said he was OK with the patch, but Juraj is not * The docs don't look as good as they probably could. * The current process generates the docs in two steps: one generates the .rst inputs for Sphinx (using the sphinx-apidoc tool), the second step then generates the actual html docs from the .rst file. * Juraj is looking into removing the sphinx-apidoc step from meson and having a semi-manual way to produce the .rst files: running the sphinx-apidoc locally the first time (or when there's a big change to file structure) and adding new files manually (which is very easy copy-paste with just a few changes). * The second approach gives us much better control over how the docs look and how we structure the pages. * Do we want to generate docs from test suites? This is likely an item for the future. * Action item: Try to generate the docs, give feedback on how it looks, give feedback on maintenance. * Blocked test case results recording: In progress, needs rebase on top of docstrings and finishing touches. * DTS Documentation from Luca. * Has been merged by Thomas now. * We might want to move around some documentation so it is more logically partitioned for readers. * Juraj is ok with the patch, but Thomas wants bigger changes. We need to agree on what we do in this patch and what in other patches. * Possibly related are these bugzilla items: 1. DTS: Split testbed and test configuration 2. DTS: Rename the execution section/stage 3. DTS Config improvements * On one more item that came up during the review: 1. DTS: add support for externally compiled DPDK = The original DTS patches * Thomas pinged Intel people to check whether there can be a final round of DTS patches merged, or some minimal support for that repo going forward. There are some tens or hundreds of patches which were submitted after Lijuan Tu (old maintainer) left Intel, which are just languishing in the dts mailing list. * Jeremy has a patch extending timeout for DTS which he can submit, but only if there will be more merging coming up * Patrick has some patch(es) adding cx6 and cx7 NICs to old DTS. Technically our testbeds running now are forked. When setup is 100% done for both NICs, intending to circle back and add these NICs to upstream DTS if merging is a possibility. New/Open feature requests * How do we want to handle feature requests for the framework going forward? We have a lot of needed features dumped into bugzilla now (thank you Juraj), but how do we prioritize, filter through what is most important for each release? * Patrick suggests: incorporate bugzilla tickets into release roadmaps, assign them to individuals, and set action items at each DTS meeting for bugzilla tickets. https://bugs.dpdk.org/buglist.cgi?bug_status=__open__&component=dts&product=DPDK * We can also run through all the tickets during meetings, and set the “importance” value f
Community CI Meeting Minutes - February 1, 2024
February 1, 2024 # Attendees 1. Patrick Robb 2. Paul Szczepanek 3. Thomas Monjalon 4. Juraj Linkes 5. Ali Alnubani 6. Aaron Conole # Minutes = General Announcements * Per mailing list/slack conversation, an important feature to add for the email based retest framework is to trigger a retest, but instead of retesting the original DPDK artifact created for the patchseries, re-apply on latest and retest. Ferruh suggests we include an optional argument “pull”, so if “pull=true” we re-apply on latest, if the argument is omitted or pull=false we do the normal behavior. * Some community members running automation are still pulling from the dpdk.org git server and need to move to github. People who don’t move will get blacklisted, and then they will move quickly. * Making periodic test results more visible: * Would be helpful to list the testing labels matrix for periodic “main” testing on the dpdk.org testing page (or a page under testing). * After main, we should add all branches, and should display datetime of test and branch name * Can also extend this to per patch testing. If we only include the latest patch tested, at least that shows all the labels tested. * Would be helpful to have a more detailed description. * We can do this in small steps, and just start with periodic testing. * Is there a problem with testing RFCs? UNH and GHA are not doing it, but should start. It does present a value to maintainers/developers and it probably does not present a testing capacity problem (most come early in a release cycle.) = CI Status - UNH-IOL Community Lab * Automation for TS-Factory testing is now online at UNH * Runs on ARM servers, for the XL710 NIC and MLNX CX5 * Runs 1x/month * Posts results to https://ts-factory.io/bublik/v2/runs * UNH team is still working on provisioning the new server donated by Intel * Requesting reviews/comments on Adam’s create artifact script (can be used in ci to leverage the pw_maintainers_cli.py tool for creating DPDK artifacts from new patchseries): https://inbox.dpdk.org/ci/20240118234120.29256-1-ahass...@iol.unh.edu/ * Rebuilding containers for all testing which has an ipsec-mb library dependency (fips, zuc, snow3g) due to the minimal version increasing to 1.4: https://inbox.dpdk.org/dev/20240130142412.2047550-1-venkatx.sivaramakrish...@intel.com/ - Intel Lab * No updates from John - Github Actions * Will be (within the next week) migrating the VM which runs the robot. The disruption should be about one half day. - Loongarch Lab * None = DTS Improvements & Test Development * We held a meeting yesterday for this. * We have someone at UNH starting now on this project (Nicholas Pratte), so we will see if he can start joining these meetings * Should update the bugzilla query for DTS to include the priority column so we can sort by that = Any other business * In the near future, we should change the week which ci meetings fall on, so that they are on the off week of the DTS meetings * Thomas will start saving the ci meeting minutes on the website * Next Meeting: February 15, 2024
Re: [PATCH v1 2/4] dts: unify class inheritance from object
Reviewed-by: Patrick Robb
Re: [PATCH v1 3/4] dts: unify super calls
Reviewed-by: Patrick Robb
Re: [PATCH v1 2/2] dts: Add missing docstring from XML-RPC server
Recheck-request: github-robot
Re: [PATCH v2 1/2] deque: add multi-thread unsafe double ended queue
Hi Ali, Wathsala reached out asking how the checkpatch CI check can be updated so that this series passes checkpatch. If building the dictionary is a 1 time operation for you, you may have to apply this patch and re-run devtools/build-dict.sh so that the new dictionary is in place for a V3 of this series. It looks like these dictionary exceptions are submitted quite rarely. But, if it becomes more common in the future you could look at adding a step to your automation which produces a new dictionary every time you run checkpatch, based on any additions to the exception list which came with the patch. But it's probably not worth the effort with the low volume of word exception additions. Thanks.
DTS WG Meeting Minutes - April 24, 2024
# Attendees * Patrick Robb * Juraj Linkeš * Paul Szczepanek * Luca Vizzarro * Nicholas Pratte # Minutes = General Announcements * What does the path to project wide adoption look like for DTS? * Tech board laid out at the end of 2023 what the process should look like for building DTS adoption in DPDK * 1. Current DTS developers start writing testsuites for dpdk library X, demonstrating the framework (easily and effectively) supports testsuites for the library. If any gaps in the framework are discovered, submit the needed DTS framework patches for writing testsuites for dpdk library X. * 1.1 As this process gets under way, it’s important to start working with library maintainers to validate the new testsuites, and start to build the loose framework for what broad adoption will look like. * 1.2 To this end, UNH developers are supposed to write ethdev testsuites in 2024. The current timeline is that techboard will assess ethdev testsuite support in DTS at the end of 2024. * Paul notes that Luca too will be writing some ethdev suites in the future * Group should determine the “shortlist” of ethdev testsuite * Patrick Robbto the full list down for the next bi-weekly dts meeting * Always make a bugzilla ticket per testsuite before doing any work * please make a ticket for jumboframes: https://bugs.dpdk.org/show_bug.cgi?id=1421 * 2. When DTS developers indicate DTS support for library X is ready, tech board will take it up as a discussion point and give that an up/down vote. And “up” vote will set the new policy that when DPDK developers submit a DPDK patch to library X, they are now required to also submit an associated testsuite/testcase addition which provides some test coverage for their new feature. * 2.1 DPDK CI labs will “pick up” the new testsuite additions alongside the new patches and run across their hardware. * Jeremy has been on another project for a few weeks, but that’s wrapped up now and he’s back to focusing on DTS = Patch discussions * Hugepages patch: * V4 is submitted with some small improvements recommended by Juraj: https://patchwork.dpdk.org/project/dpdk/patch/20240418161026.2839-1-npra...@iol.unh.edu/ * Juraj will send a review tag * Jumboframes: Nick is writing this now, V1 should be coming soon * Bugzilla: https://bugs.dpdk.org/show_bug.cgi?id=1421 * Testpmd show port info/show port stats: https://patchwork.dpdk.org/project/dpdk/list/?series=31729 * Testpmd statefullness / params class * V2 will be forthcoming here * Jeremy can give a review on the V2 now that’s he’s back to DTS world… * Jeremy should assess for his patch whether show port info should be used as a tool to check capabilities * Skip test cases based on capabilities * Jeremy still needs to provide feedback from his new scatter testsuite rebased off of this patch * Replace XML-RPC server with scapy shell: * Coming soon to a mailing list near you. * API-Docs generation patch * Review are needed for the latest version * Jeremy Spewockplease review * Nicholas Pratteplease build the docs again and provide another tested-by * Renaming “Execution” * Idea is to rename to testrun * As it is a tiny patch it can be applied soon * Juraj’s patch to remove OS-UDP suite * We want to remove the code which was added only to support the os-udp suite. This includes the code which allows for network configuration in the os. But, we may need some of this for scapy on the TG. * Bugzilla: https://bugs.dpdk.org/show_bug.cgi?id=1414 * Juraj submitted some small patches with cleanups like changing implicit object inheritance… etc. * Interactive shells: * By default, the interactive shells are not run with admin privileges. For shells handling any DPDK apps, it may make sense to run with admin privilege. = Bugzilla discussions * Had some discussion on 1360 relating to how users should define node ports for an execution: https://bugs.dpdk.org/show_bug.cgi?id=1360 * It’s not really execution specific, as the execution has things which we are configuring for a specific testrun. There will never be a different value for a specific port on a node, so this information should not go into the execution config block. * Individual nodes do not need to have the full port links stored as attributes. The port links can be created by the testsuite alone. * So, going forward the SUTNode and TGNode should contain ONLY local pci addresses, and these nodes are not aware of the full links * Store a list of “NICs” in the node config block, then specify the
Re: [PATCH] net/af_packet: cache align Rx/Tx structs
Recheck-request: iol-compile-amd64-testing The DPDK Community Lab updated to the latest Alpine image yesterday, which resulted in all Alpine builds failing. The failure is unrelated to your patch, and this recheck should remove the fail on Patchwork, as we have disabled Alpine testing for now.
Re: [PATCH v2] vhost: cleanup resubmit info before inflight setup
Recheck-request: iol-compile-amd64-testing The DPDK Community Lab updated to the latest Alpine image yesterday, which resulted in all Alpine builds failing. The failure is unrelated to your patch, and this recheck should remove the fail on Patchwork, as we have disabled Alpine testing for now.
Re: [RFC] net/af_packet: make stats reset reliable
Recheck-request: iol-compile-amd64-testing The DPDK Community Lab updated to the latest Alpine image yesterday, which resulted in all Alpine builds failing. The failure is unrelated to your patch, and this recheck should remove the fail on Patchwork, as we have disabled Alpine testing for now.
Re: [PATCH] lib/vhost: add flag for async connection in client mode
Recheck-request: iol-compile-amd64-testing The DPDK Community Lab updated to the latest Alpine image yesterday, which resulted in all Alpine builds failing. The failure is unrelated to your patch, and this recheck should remove the fail on Patchwork, as we have disabled Alpine testing for now.
Re: [PATCH] eal/linux: enhanced error handling for affinity
Recheck-request: iol-compile-amd64-testing The DPDK Community Lab updated to the latest Alpine image yesterday, which resulted in all Alpine builds failing. The failure is unrelated to your patch, and this recheck should remove the fail on Patchwork, as we have disabled Alpine testing for now.
Re: [PATCH 10/10] doc: update nfp document
Recheck-request: iol-compile-amd64-testing The DPDK Community Lab updated to the latest Alpine image yesterday, which resulted in all Alpine builds failing. The failure is unrelated to your patch, and this recheck should remove the fail on Patchwork, as we have disabled Alpine testing for now.
Re: [v1 1/1] docs: af_xdp device plugin repo update
Recheck-request: iol-compile-amd64-testing The DPDK Community Lab updated to the latest Alpine image yesterday, which resulted in all Alpine builds failing. The failure is unrelated to your patch, and this recheck should remove the fail on Patchwork, as we have disabled Alpine testing for now.
Re: [RFC 0/4] malloc type argument cleanup (part 1)
Recheck-request: iol-compile-amd64-testing The DPDK Community Lab updated to the latest Alpine image yesterday, which resulted in all Alpine builds failing. The failure is unrelated to your patch, and this recheck should remove the fail on Patchwork, as we have disabled Alpine testing for now.
Re: [PATCH] ethdev: document that stats reset APIs are not thread-safe
Recheck-request: iol-compile-amd64-testing The DPDK Community Lab updated to the latest Alpine image yesterday, which resulted in all Alpine builds failing. The failure is unrelated to your patch, and this recheck should remove the fail on Patchwork, as we have disabled Alpine testing for now.
Re: [PATCH] bpf/xdp: disable on 32bit x86
Recheck-request: iol-compile-amd64-testing The DPDK Community Lab updated to the latest Alpine image yesterday, which resulted in all Alpine builds failing. The failure is unrelated to your patch, and this recheck should remove the fail on Patchwork, as we have disabled Alpine testing for now.
Re: [v1 1/1] MAINTAINERS: add another AF_XDP maintainer
Recheck-request: iol-compile-amd64-testing The DPDK Community Lab updated to the latest Alpine image yesterday, which resulted in all Alpine builds failing. The failure is unrelated to your patch, and this recheck should remove the fail on Patchwork, as we have disabled Alpine testing for now.
Re: [RFC v2 0/6] Improve EAL bit operations API
Recheck-request: iol-compile-amd64-testing The DPDK Community Lab updated to the latest Alpine image yesterday, which resulted in all Alpine builds failing. The failure is unrelated to your patch, and this recheck should remove the fail on Patchwork, as we have disabled Alpine testing for now.
Re: [PATCH] bus/pci: fix build with musl 1.2.4 / Alpine 3.19
On Mon, Apr 29, 2024 at 6:01 AM David Marchand wrote: > Following an upgrade of musl, pread64/pwrite64 wrappers are not provided > anymore. Switch to POSIX pread/pwrite. > > Bugzilla ID: 1422 > Cc: sta...@dpdk.org > > Signed-off-by: David Marchand > > Tested-by: Patrick Robb We will re-enable Alpine compile in our CI testing once this patch hits mainline
Re: [PATCH v4 0/3] dts: API docs generation
On Mon, Apr 29, 2024 at 9:49 AM Jeremy Spewock wrote: > > > The patchset contains the .rst sources which Sphinx uses to generate the > > html pages. These were first generated with the sphinx-apidoc utility > > and modified to provide a better look. The documentation just doesn't > > look that good without the modifications and there isn't enough > > configuration options to achieve that without manual changes to the .rst > > files. This introduces extra maintenance which involves adding new .rst > > files when a new Python module is added or changing the .rst structure > > if the Python directory/file structure is changed (moved, renamed > > files). This small maintenance burden is outweighed by the flexibility > > afforded by the ability to make manual changes to the .rst files. > > > > We can merge this patch when: > > 1. We agree on the approach with manually modifying the files. > > This approach is, in my opinion, much better than just generating the > > .rst files every time, > > +1 for manually modifying .rst files. The .rst files are very simple, > and I think the added flexibility to change headers or tweak things as > needed is a big benefit over just auto-generating and not having as > much control. Additionally, if it just so happens that the > auto-generated file looks fine and the developer doesn't want to make > changes, they can still just generate it themselves and it fits right > in, so this approach still works with the other regardless. > > +1
Re: Run unit tests with C++ too
On Sun, Apr 28, 2024 at 3:46 AM Mattias Rönnblom wrote: > It would be great if the unit test suite (app/test/*) was compiled (and > run) using a C++ (C++11) compiler as well. At least, if such is available. > Sure, the UNH Lab can try this. > > With the current state of affairs, header file macros or functions are > not verified to be functional (or even valid) C++. > > "C is a subset of C++", which was never true, is becoming less and less so. > > If all unit tests aren't valid C++, maybe one could start with an "opt > in" model. > Okay, so basically run the fast-test suite, record all that don't pass, submit a bugzilla ticket stating which unit tests are not valid on a certain c++ compiler, then bring CI Testing online using the valid subset of fast-tests. This should work. > > A drawback of this is that the unit tests need to be both valid C and > valid C++. >
Re: Run unit tests with C++ too
On Tue, Apr 30, 2024 at 4:13 PM Mattias Rönnblom
wrote:
> On 2024-04-30 15:52, Patrick Robb wrote:
> >
> >
> > On Sun, Apr 28, 2024 at 3:46 AM Mattias Rönnblom > <mailto:hof...@lysator.liu.se>> wrote:
> >
> > It would be great if the unit test suite (app/test/*) was compiled
> (and
> > run) using a C++ (C++11) compiler as well. At least, if such is
> > available.
> >
> >
> > Sure, the UNH Lab can try this.
> >
> >
> > With the current state of affairs, header file macros or functions
> are
> > not verified to be functional (or even valid) C++.
> >
> > "C is a subset of C++", which was never true, is becoming less and
> > less so.
> >
> > If all unit tests aren't valid C++, maybe one could start with an
> "opt
> > in" model.
> >
> >
> > Okay, so basically run the fast-test suite, record all that don't pass,
> > submit a bugzilla ticket stating which unit tests are not valid on a
> > certain c++ compiler, then bring CI Testing online using the valid
> > subset of fast-tests. This should work.
> >
>
> Sounds good.
>
> Just to be clear: the above includes extending the DPDK build system to
> build the app/test/dpdk-test binary in two versions: one C and one C++,
> so that anyone can run the C++ tests locally as well. Correct?
>
Okay, so now I am understanding this is not yet available. When I responded
this morning I was figuring that c++ compiler support was available and I
simply wasn't aware, and that we could quite easily set cc={some c++
compiler}, meson would pick it up, and we would be able to build DPDK and
then run unit tests in this manner in CI testing.
I didn't mean to suggest we would submit patches extending the build system
to this end. That's probably a little out of scope for what we try to
accomplish at the Community Lab.
But if the aforementioned build system support is added, of course we are
willing to add that as a build environment for unit tests and report those
respective results.
Community CI Meeting Minutes - May 2, 2024
# May 2, 2024 Attendees 1. Patrick Robb 2. Juraj Linkeš 3. Aaron Conole 4. Paul Szczepanek 5. Luca Vizzarro # Minutes = General Announcements * New servers at UNH Lab: * Patrick has an initial quote from our server vendor for the new Dell servers coming into the lab. He requested some small modification and a new quote which is pending. Once this is finalized we will send to Aaron Conole and if he approves, make the purchase. * Small adjustment: Switching CPU on Xeon DUT to a similar model, but with more offload devices (QAT, etc.). Bruce from Intel acked this and it is actually slightly cheaper. * Quote not yet ready with Supermicro for the ARM Grace server, but we’re working with one of their sales reps. = CI Status - UNH-IOL Community Lab * FreeBSD 14: * Cody has noted that not all unit tests pass on our new FreeBSD 14 VM. He submitted an update on Bugzilla. We are bringing this machine online now only for build testing. * OvS DPDK testing: * Adam finished bugfixing for OvS compile, but we are writing the jenkinsfiles still * Code coverage reports added to our dashboard: https://dpdkdashboard.iol.unh.edu/results/dashboard/code-coverage * Aaron requested we dry run the driver tests and perf tests testsuites alongside fast tests. That’s on our radar but still in queue * Patrick Robbshould “announce” the code coverage report page on dpdkdashboard, so the maintainers and developers take note. Can do this on the CI and dev mailing lists. * NVIDIA: Installed an additional CX7 on the DUT server at UNH (thanks Gal and Ali for sending). Saw an increase in MPPS but still not matching published results from NVIDIA. * Patrick can put more cycles into troubleshooting the system setup, the testpmd params etc. * This may be a case where the CPU hardware is also a factor (these are on older broadwell servers), so this is a place where the new servers coming into UNH may be beneficial - Intel Lab * Rajesh can be contacted for Intel lab needs going forward * There was a request on the CI mailing list for Intel lab to discontinue CI Testing on centos 7: https://inbox.dpdk.org/ci/CAJvnSUB8ikROiizW+iS=h0-GdTjh2a2UnawJFpEF=g-oppr...@mail.gmail.com/T/#t * GCC 4.8.5 is no longer supported in DPDK - Github Actions * Did migrate the GH robot from its temp machine to permanent home. There were some issues with sending report emails, but Aaron doesn’t think any patches were missed. If they were, the Recheck-request function allows for users to initiate a backfill of results. * Roughly April 25-26 - Loongarch Lab * None = DTS Improvements & Test Development * Paul/Luca indicated interest in writing testsuites. Patrick will produce an updated list of ethdev testsuites to port to the new framework, and we will pick some next Wednesday at the next DTS meeting. * Juraj also indicated interest in also writing some, but he will be on vacation next week, so we’ll select 2-3 for him. * patches: * Juraj: small cleanup of remote session close, moving the function and removing the force parameter. * There is agreement here, so Juraj will ping Thomas for an apply * Juraj: Rename execution to test run * There is agreement here, so Juraj will ping Thomas for an apply * Nick: hugepage is done and acked by Juraj * Requsting a review from Luca * Otherwise, there is agreement here, so Juraj will ping Thomas for an apply * Nick: Working on Jumboframes testsuite patch right now. * Luca: * From_dict methods and typing: Convert all the classes factory methods used as `@staticmethod` to`@classmethod`. Update mypy so the self type is recognized as valid. Mypy update can be in a separate patch which will (hopefully) get applied quickly as there will be basically nothing to review. https://bugs.dpdk.org/show_bug.cgi?id=1433 * CI: It would be worth running mypy in CI. Luca is running it locally and is seeing some things have been missed. UNH CI should run the /devtools/dts-check-format.sh, and if black, linters, etc. find any issues it will result in a non-zero exit code, and we can report a fail based on this. * Patrick Robbwill make a ticket for this at UNH and get it in queue * Can name it dts_lint * Only run it if /dts/* are modified. = Any other business * Next Mee
DPDK code coverage reports dashboard page
Hi DPDK Developers, I wanted to announce to the mailing lists that the UNH Community Lab recently rolled out a new page on our dashboard where you can find code coverage reports for DPDK unit tests. https://lab.dpdk.org/results/dashboard/code-coverage It provides some unit test code coverage reports for lib/* and drivers/*. You can download and extract the test coverage artifacts, and open up the styled html page by navigating to out/coveragereport/index.html. We run these coverage reports 1x/month, on the first day of the month. Ideally these can serve as a reference for how many lines of code are touched by unit tests, but also how coverage evolves over time. Some improvements we have in queue: -Add driver-tests and perf-tests testsuites. Currently we are only running fast-tests and a few individual testcases from driver-tests. -embed the test report (with styling) in the lab dashboard page, so people don't have to download and open the page locally. We are building gcov/lcov targets according to the documentation here: https://mesonbuild.com/Unit-tests.html Questions or suggestions on how to improve the coverage reports are welcome. Thanks, -DPDK Community Lab team
Re: [PATCH v5 00/45] use stdatomic API
Recheck-request: iol-broadcom-Performance On Mon, May 6, 2024 at 1:58 PM Tyler Retzlaff wrote: > > This series converts all non-generic built atomics to use the rte_atomic > macros that allow optional enablement of standard C11 atomics. > > Use of generic atomics for non-scalar types are not converted in this > change and will be evaluated as a part of a separate series. > Specifically conversion of lib/lpm and drivers/x/cnxk will be addressed > in a separate series to address use of generics. > > v5: > * add missing RTE_ATOMIC for ``struct channel_info.status`` field. > > v4: > * rebase after merge of move alignment attribute on types for MSVC, > no other changes. > > v3: > * event/dsw, wrap all lines <= 80 chars, align arguments to > opening parenthesis. > * event/dlb2, wrap changed lines <= 80 chars, remove comments > referencing gcc __atomic built-ins. > * bus/vmbus, remove comment referencing gcc atomic built-ins, > fix mistake where monitor_mask was declared RTE_ATOMIC(uint32_t), > fix mistake where pending was not declared RTE_ATOMIC(uint32_t), > remove now unnecessary cast to __rte_atomic of pending (since > the field is now properly declare RTE_ATOMIC). > > v2: > * drop the net/sfc driver from the series. the sfc driver > uses generic __atomic_store not handled by the current macros. > the cases where generic __atomic_xxx are used on objects that > can't be accepted by __atomic_xxx_n will be addressed in a > separate series. > > Tyler Retzlaff (45): > net/mlx5: use rte stdatomic API > net/ixgbe: use rte stdatomic API > net/iavf: use rte stdatomic API > net/ice: use rte stdatomic API > net/i40e: use rte stdatomic API > net/hns3: use rte stdatomic API > net/bnxt: use rte stdatomic API > net/cpfl: use rte stdatomic API > net/af_xdp: use rte stdatomic API > net/octeon_ep: use rte stdatomic API > net/octeontx: use rte stdatomic API > net/cxgbe: use rte stdatomic API > net/gve: use rte stdatomic API > net/memif: use rte stdatomic API > net/thunderx: use rte stdatomic API > net/virtio: use rte stdatomic API > net/hinic: use rte stdatomic API > net/idpf: use rte stdatomic API > net/qede: use rte stdatomic API > net/ring: use rte stdatomic API > vdpa/mlx5: use rte stdatomic API > raw/ifpga: use rte stdatomic API > event/opdl: use rte stdatomic API > event/octeontx: use rte stdatomic API > event/dsw: use rte stdatomic API > dma/skeleton: use rte stdatomic API > crypto/octeontx: use rte stdatomic API > common/mlx5: use rte stdatomic API > common/idpf: use rte stdatomic API > common/iavf: use rte stdatomic API > baseband/acc: use rte stdatomic API > net/txgbe: use rte stdatomic API > net/null: use rte stdatomic API > event/dlb2: use rte stdatomic API > dma/idxd: use rte stdatomic API > crypto/ccp: use rte stdatomic API > common/cpt: use rte stdatomic API > bus/vmbus: use rte stdatomic API > examples: use rte stdatomic API > app/dumpcap: use rte stdatomic API > app/test: use rte stdatomic API > app/test-eventdev: use rte stdatomic API > app/test-crypto-perf: use rte stdatomic API > app/test-compress-perf: use rte stdatomic API > app/test-bbdev: use rte stdatomic API > > app/dumpcap/main.c | 12 +- > app/test-bbdev/test_bbdev_perf.c | 183 > + > app/test-compress-perf/comp_perf_test_common.h | 2 +- > app/test-compress-perf/comp_perf_test_cyclecount.c | 4 +- > app/test-compress-perf/comp_perf_test_throughput.c | 10 +- > app/test-compress-perf/comp_perf_test_verify.c | 6 +- > app/test-crypto-perf/cperf_test_latency.c | 6 +- > app/test-crypto-perf/cperf_test_pmd_cyclecount.c | 10 +- > app/test-crypto-perf/cperf_test_throughput.c | 10 +- > app/test-crypto-perf/cperf_test_verify.c | 10 +- > app/test-eventdev/test_order_atq.c | 4 +- > app/test-eventdev/test_order_common.c | 5 +- > app/test-eventdev/test_order_common.h | 8 +- > app/test-eventdev/test_order_queue.c | 4 +- > app/test-eventdev/test_perf_common.h | 6 +- > app/test/test_bpf.c| 46 -- > app/test/test_distributor.c| 114 ++--- > app/test/test_distributor_perf.c | 4 +- > app/test/test_func_reentrancy.c| 28 ++-- > app/test/test_hash_multiwriter.c | 16 +- > app/test/test_hash_readwrite.c | 74 - > app/test/test_hash_readwrite_lf_perf.c | 88 +- > app/test/test_lcores.c | 25 +-- > app/test/test_lpm_perf.c | 14 +- > app/test/test_mcslock.c| 12 +- > app/test/test_mempool_perf.c | 9 +- > app/test/test_pflock.c
Updated DTS 24.07 Roadmap
When we sent this out originally we missed some patches Luca Vizzarro has been working on for this release, so here is an updated version. Thanks Luca. 1) Write ethdev testsuites: Jumboframes: https://git.dpdk.org/tools/dts/tree/test_plans/jumboframes_test_plan.rst Mac Filter: https://git.dpdk.org/tools/dts/tree/test_plans/mac_filter_test_plan.rst Dynamic Queue: https://git.dpdk.org/tools/dts/tree/test_plans/dynamic_queue_test_plan.rst 2) Replace XMLRPC server with Scapy interactive shell: https://bugs.dpdk.org/show_bug.cgi?id=1374 3) Configuration schema updates: DTS Hugepages refactor: https://patchwork.dpdk.org/project/dpdk/patch/20240409172811.27866-1-npra...@iol.unh.edu/ Cut down total configuration options in schema (not all we originally added are needed): https://bugs.dpdk.org/show_bug.cgi?id=1360 Split testbed and test configuration to different files: https://bugs.dpdk.org/show_bug.cgi?id=1344 4)API Docs generation: https://patchwork.dpdk.org/project/dpdk/list/?series=30877 5)Skip test cases based on testbed capabilities: https://patches.dpdk.org/project/dpdk/patch/20240301155416.96960-1-juraj.lin...@pantheon.tech/ Bugzilla: https://bugs.dpdk.org/show_bug.cgi?id=1351 6)Rename the execution section/stage: Bugzilla: https://bugs.dpdk.org/show_bug.cgi?id=1355 7)Add support for externally compiled DPDK: Bugzilla: https://bugs.dpdk.org/show_bug.cgi?id=136 [ADDED] 8) Add common data structure to handle command line params for EAL and testpmd: https://patches.dpdk.org/project/dpdk/list/?series=31897 [ADDED] 9) Update mypy and backport new self type: https://patches.dpdk.org/project/dpdk/list/?series=31920 [ADDED] 10) testpmd: show port info and show port stats support and parsing utility: https://patches.dpdk.org/project/dpdk/list/?series=31898 [ADDED] 11) Error message and usage improvements for debugging: https://patches.dpdk.org/project/dpdk/list/?series=31924
