This method was discussed briefly several months back, but I feel it may be time to discuss, finalize, and move forward.
THIS WILL REQUIRE ALL PACKAGE MAINTAINERS TO CHANGE THEIR BEHAVIOR. It should be a small change: just the addition of a few lines to each setup.hint, but the new method will only be useful if all package maintainers are happy with the change and are willing to ensure the new method is followed. Currently, I store all package information in a large XML document which I update by hand whenever a new package is ITPd, an existing package is updated, or some other change must be made against the live repository. XSL transformations convert this XML document into several other formats, including the plaintext PPL, MPL, and make files to automate downloading, reviewing, and pushing packages into the live repository. If nothing else, the XSL transformations on the XML document take a lot of CPU time, which makes it slow to push packages, generate the PPL, etc. Additionally, package information is spread between setup.hint files and the XML file, making it hard for other Cygwin contributors to keep everything synchronized. The proposal from several months back was to extend the format of the setup.hint files to include *all* package-related information. Currently, setup.hint files only contain information necessary to build the master setup.ini file--that is, information that would show up in setup.exe, or is otherwise needed to prepare new packages for distribution by setup.exe. In order to ease the in-place transition, it was agreed that the non- setup.exe information should be stored as specially-formatted comments in setup.hint. That is, information that is not used to generate setup.ini will stay hidden from the utilities that generate setup.ini (so these utilities will not need to be modified ahead of time). One proposal was to use the sharp/bang notation traditionally used to denote executable script files, leaving all other lines beginning with a sharp (aka hash or pound) as un-interpreted comments. Since these files are not truly executable files (and can not be made executable without modifying the utilities that generate setup.ini), it will be important to ensure that no setup.hint begins with a sharp/bang special comment. For example: release/foomaster/setup.hint sdesc: "The Finite Object Organization visualization tool" ldesc: "The Finite Object Organization visualization tool allows visual navigation of organizations of finite objects." requires: cygwin libfoo category: Utils #!maintainer Daniel Reed #!bin http://www.example.net/products/foomaster/Cygwin/foomaster-%{version}-%{release}.tar.bz2 #!src http://www.example.net/products/foomaster/Cygwin/foomaster-%{version}-%{release}-src.tar.bz2 #!hint http://www.example.net/products/foomaster/Cygwin/foomaster.hint release/foomaster/libfoo/setup.hint sdesc: "The Finite Object Organization library" ldesc: "The Finite Object Organization library provides service routines to organize finite objects." category: Libs #!maintainer Daniel Reed #!related-to foomaster #!bin http://www.example.net/products/foomaster/Cygwin/libfoo-%{version}-%{release}.tar.bz2 #!src http://www.example.net/products/foomaster/Cygwin/libfoo-%{version}-%{release}-src.tar.bz2 #!hint http://www.example.net/products/foomaster/Cygwin/libfoo.hint release/foomaster/libfoo-devel/setup.hint sdesc: "libfoo development headers" ldesc: "The Finite Object Organization library provides service routines to organize finite objects." requires: libfoo category: Devel Libs external-source: libfoo #!related-to foomaster #!bin http://www.example.net/products/foomaster/Cygwin/libfoo-devel-%{version}-%{release}.tar.bz2 #!hint http://www.example.net/products/foomaster/Cygwin/libfoo-devel.hint The procedure [on my end] for pushing new packages would be: 1) Wait for the announcement that a new package is available. 2) Grab the %{hint} file from the URL in the existing .hint file. This will allow a package maintainer to move URL bases around with the only backward- compatibility requirement being that a stub .hint file be left in the prior location. (If the value of %{hint} changes between the existing .hint and the newly-downloaded .hint, step 2 will be repeated at least once.) 3) Using URL bases in the new .hint, grab the %{bin} and %{src} packages. 4) Perform automated sanity checks against the bin package. 5) Push the new hint, bin, and src packages into the live repository. After step 5, the automatic utilities will notice the new package and regenerate setup.ini with the new .hint's information. I envision the syntax for a push to be something as simple as: cd release/.util; make upgrade NAME=foomaster CURR=2.0-3 PREV=1.3 %{bin}, %{src}, and any other URLs will be evaluated with %{version} = 2.0 and %{release} = 3 in this case. If CURR had been simply 2.0, %{version} would be 2.0 and %{release} would default to 1. PREV would only be used to tell the script which previous version to leave; all others would be removed. If no CURR is provided, the script will expect the to-be-downloaded .hint file to include at least an appropriate curr: and possibly a prev: line. This will be the prefered procedure when upgrading a package (to minimize the chance of the package coordinator manually typing the wrong value). The syntax would then simply be: cd release/.util; make upgrade NAME=foomaster Or possibly even: cd release/foomaster; make -f ../netrel.mk upgrade For pending packages--those packages that have been created for Cygwin but have not been formally accepted into the Cygwin distribution--a non-release/ repository will be maintained in a non-live location. In addition to holding operational values like #!bin and #!maintainer, the .hint files stored in non-release/ will include values like #!vote, #!problem, #!review, and any other information required to generate the PPL. Once a package has been formally accepted these values may be kept for historical purposes but, as the package maintainer will control the contents of the .hint at that point, they may be replaced with other values or removed altogether. (In practice, these PPL-specific values may be held in a separate file in the same directory and simply merged with setup.hint for processing.) To propose a new package, one would simply send the proposed initial contents of the package's .hint file to the cygwin-apps mailing list. An automated script can be used to periodically scan the archives looking for these messages. If the .hint section of the email is guaranteed to always begin with "@ packagename", no other special formatting should be required for these proposal messages. That is, the following would be acceptable for automatic purposes: From: Daniel Reed <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] Subject: hi Hi. This is a cool package. @ foomaster curr: 1.3 sdesc: "The Finite Object Organization visualization tool" ldesc: "The Finite Object Organization visualization tool allows visual navigation of organizations of finite objects." requires: cygwin libfoo category: Utils #!bin http://www.example.net/products/foomaster/Cygwin/foomaster-%{version}-%{release}.tar.bz2 #!src http://www.example.net/products/foomaster/Cygwin/foomaster-%{version}-%{release}-src.tar.bz2 #!hint http://www.example.net/products/foomaster/Cygwin/foomaster.hint And this one! @ libfoo curr: 1.1.5 sdesc: "The Finite Object Organization library" ldesc: "The Finite Object Organization library provides service routines to organize finite objects." category: Libs #!bin http://www.example.net/products/foomaster/Cygwin/libfoo-%{version}-%{release}.tar.bz2 #!src http://www.example.net/products/foomaster/Cygwin/libfoo-%{version}-%{release}-src.tar.bz2 #!hint http://www.example.net/products/foomaster/Cygwin/libfoo.hint And this! @ libfoo-devel sdesc: "libfoo development headers" ldesc: "The Finite Object Organization library provides service routines to organize finite objects." requires: libfoo category: Devel Libs #!bin http://www.example.net/products/foomaster/Cygwin/libfoo-devel-%{version}-%{release}.tar.bz2 #!hint http://www.example.net/products/foomaster/Cygwin/libfoo-devel.hint Thanks. However, keep in mind that this example is intended to show the strength of the message format, specifically its resilience to human neglect. It is advised that proposal messages should adhere to existing Cygwin standards for informative Subject content and introductory paragraphing. If no #!maintainer is supplied, the value will be pulled from the From field. For messages containing multiple .hint file contents, all packages after the first will be treated as "related-to" the first unless an explicit #!related-to is given. #!related-to may be provided and left blank to force the subsequent package to reside at the top level of the repository. In the interests of simplicity, it may make sense to allow the @ line to contain a path instead of just a package name. That is, "@ libfoo-devel" followed by "#!related-to foomaster" could be simplified to just "@ foomaster/libfoo-devel". The final, live setup.hint would return to "@ libfoo-devel" in the foomaster/libfoo-devel/ directory. If no #!src is provided, external-source will be required in the final .hint. If no external-source is provided when one is needed, the field will be set to the first "complete" package preceeding the current one. That is, in the above example, libfoo-devel will have an external-source of libfoo. If a fourth package, libfoo-demo, was provided without #!src nor external-source, it too would be given an external-source of libfoo (skipping over libfoo-devel). For truly binary-only packages (such as packages composed completely of shell scripts), #!src may be provided and left blank. Long URLs and other data that exceed 80 characters will have to run past the 80th column; it will be an error to split long lines. Furthermore, "@" and "#!" must start in the first column; it will be an error to start any line in a .hint file with any leading spacing. Every effort will be made to ensure that extra spaces between the #!-section and the value, as well as any trailing spaces are ignored. I have not decided on an optimal machine-readable format for announcing updates. If the hint URL (#!hint) has not changed, all that is needed for automation purposes is the new CURR and possibly a new PREV. It may be easiest in practice to allow package updates to be announced by again posting the .hint file in the above format. However, if the package maintainer's online .hint file contains curr: and prev: entries, even that information is not strictly required in the announce message; simply announcing that a new version is available would be sufficient. This would make it easy to construct an automatic web form or special-purpose email address for package maintainers to cause new versions to be downloaded, auto-reviewed, and pushed to the live repository without any human intervention. If this proposal is accepted, I will begin writing the required backend utilities. Once they have been written, the exact non-setup.exe syntax can be finalized and the documentation can be updated. Offhand, I believe only the setup.hint section of http://cygwin.com/setup.html will need to be updated. I think this is a good proposal, but I am open to any suggestions. Maintaining the PPL and assorted support files has become very time- consuming lately; I am hopeful this new method will allow us to redistribute some of the coordination load and help streamline Cygwin maintenance. -- Daniel Reed <[EMAIL PROTECTED]> http://naim-users.org/nmlorg/ http://naim.n.ml.org/ 1832 Savior214: that sucks that one day your just gonna die and all that work you did learning stuff just gets a rm -rf
