On Sat, May 26, 2012 at 11:46:13AM +1000, Brian May wrote:
> On 25 May 2012 16:40, Ron <r...@debian.org> wrote:
> > Do you really need this persistently after gitpkg exits, or can you
> > actually do what you need with it from a gitpkg hook before it exits?
>
> The problem with doing it from a hook is that I don't want to do the
> exact same action every time I run gitpkg.
>
> For example:
>
> * Sometimes I just want to check out the source code.
> * Sometimes I want to do a binary build (e.g. for Debian upload).
> * Sometimes I want to do a source only build (e.g. for Ubuntu PPA upload).
> * Sometimes I want to do a binary build in a schroot of my choosing.
> * Sometimes I want to do a binary build in a schroot of chosen by package.
> * I might want to run Lintian on the result.
> * etc
>
> These builds will create files that can conflict with each other. So
> for the Ubuntu PPA upload for example, I have a script that will
> create a temporary directory, copy/checkout everything into this
> directory, build a source only package, upload it to my PPA, and then
> delete the temporary directory when finished.
>
> While building a source only package is a useful action, I wouldn't
> want to build a source only package every time I do an export. So I
> wouldn't want to make it a permanent configuration change. In fact, I
> could be doing a full binary build at the same time in another window
> - changing a global setting could upset other processes already
> running.
>
> If I could specify the value for DEB_DIR *and* gitpkg.exit-hook from
> my script, I think I would be mostly happy.
Can you tolerate your hook being interactive?
There's no reason you can't add a 'generic' exit hook, that either:
a) Prompts you about what additional actions to do after export.
b) Just does them automatically based on some other environment
setting of your own.
That hook might then execute any of several scripted options that
make up your usual set of 'automatic' actions.
For the sort of things you mention above, that seems very much in the
spirit of the hooks being used for 'local admin' automation. ie. there
is no reason another user couldn't export things from your repo without
your hook - and there is likewise no reason you couldn't also use your
hook with other people's repos. Since it assumes nothing about the repo
itself, and requires nothing special from it. It's just a list of the
actions to do after the source package is exported.
I do something like this with the cowpoke hook. I usually make it ask
me before sending the package off to the buildd - since I don't always
want to do that immediately or at all. (and cowpoke itself takes care
of some of the higher levels you describe above, like running lintian
and a debdiff against the previous package etc.)
> > Right. That's kind of why "correctly building the source package"
> > really shouldn't ever depend on any hooks. Mostly they should be
> > considered "reserved for the local user", who might do anything with
> > them that they please to connect a source package export with *their*
> > other automation, such as running sanity checks or passing the package
> > off to a buildd of their choice etc.
>
> There seem to be two types of hooks:
That's roughly true, but ...
> type A: Those that are required to correctly build the source package,
> e.g. pristine-tar, quilt-patches. These vary depending on the package
> being built, and as such, having them as git config settings makes
> perfect sense. They should get executed every time gitpkg is called,
> regardless of who/what is calling it.
These kind of go against the original spirit of what was intended for
the hooks, but for people who want these things, that's a tradeoff they
need to choose for themselves. The thing that keeps them within it
though, is that they are only required to build a specific *form* of
package. ie. if you don't have them installed, and you export the
same repo, you will still get a valid package of the same source, that
builds exactly the same binary packages.
The *only* difference is, you'll get a format 1.0 style monolithic
diff without the quilt hook instead of the split up patch series.
Or you'll get a tarball re-exported from the upstream branch, instead
the pristine-tar hacked one - so it will have the same source, just
not the same checksum. (and if you already have the old orig tarball
present in the export dir, or fetch it with wget from a hook, then it
will re-use that and give you identical results to what p-t would too).
So yes, they are required to get an 'identical' source package.
But not to correctly export a valid source package, with identical
"after patching" source, and build another copy of that version.
> type B: Those that do extra things after the source code has been
> built, e.g. building the binary packages, lintian checks, etc. These
> should not be run when gitpkg is invoked from a script, and the exact
> requirements may vary with every run.
This is the part where I wonder if you're trying to work around something
that you could actually make work for you :)
Instead of invoking gitpkg from a script that modifies or overrides these
hooks, possibly the functionality of your script should actually go *in*
them.
It's a fairly subtle difference, but since the hooks are set on a local
repo working copy (not on a package, or in a central repo), every user
is free to set them to whatever they please. So if you want them to not
run, you simply don't set them - but if you have a script that would
override them, possibly that script itself should be the hook you set
for your repo.
The hooks are just scripts, and be be as complicated and offer as many
options as you please. The main advantage over your external script
though is they automatically have access to all the information that
gitpkg knows about the package version and dirs etc. without needing
to do tricks to get that information back out again.
You could still set just one hook script, but have it do different
things based on whatever conditions you care to define it to recognise.
> > Can you give me an outline of what your script needs to do before and
> > after gitpkg does its bit? I really would like to avoid there being
> > anything that is "impossible" to do with it, that isn't already quite
> > impossible for fundamental reasons which have nothing to do with gitpkg.
> > So we've tried to expose most things and let you hook in at all the
> > places where they are first known, or where you might do some action
> > based on them before the next logical step.
>
> Something like:
>
> TMPDIR=$(mktemp -d)
> trap "rm -rf -- '$TMPDIR'" EXIT HUP INT TERM
> DEB_DIR=$TMPDIR gitpkg $VERSION
> cd $TMPDIR/blah/blah
> debuild -i -S -uc -us
> dput ppa-natty:brian-microcomaustralia/django blah_source.changes
>
> (the last three lines could go into a gitpkg.exit-hook hook)
Yeah, the parts after the gitpkg call will be easiest to do in an exit
hook, since you have all the relevant information available to do them
in it.
And the TMPDIR part, you could actually do from an admin-config-hook ...
That runs immediately before gitpkg actually starts any real work, and
lets you override any of the gitpkg configuration options. ie. it also
runs after the git-config options are read, so anything you change in
this hook can override them as well.
Which means you can mktemp and set DEB_DIR there, and since the hook
is simply sourced into gitpkg, even set up your trap to remove it :)
> > The main thing we should try to avoid is creating a repo where the
> > source package can only be exported correctly from it if you have
> > exactly the right hooks and other scripts to do that configured.
> > But any other higher level operations that do things with the source
> > once exported we should support as fully as it is possible to do.
>
> As explained about, some repositories will need the right hooks setup
> (type A), however these shouldn't affect any automatic scripting.
>
> What might be a problem is if there is a type B hook that does
> something that is relied on by a calling script. Or does something
> that is not expected.
This _shouldn't_ be a problem, mostly because the hooks are entirely
local to a given working copy of a repo. So generally speaking the
person working in that will know if they have hooks set or are calling
a script, and should know enough about both those things to know if
they are somehow incompatible with one another. There shouldn't really
be any surprises there for the sort of thing we are talking about here.
If the hooks or the scripts were doing something funky to modify the
source while building the package this might be more amusing, but for
the sort of pre- and post- export automation we've discussed so far,
there shouldn't be any reason for conflicts.
> In a practical sense, I suspect type B hooks will always (?) be
> gitpkg.exit-hook, and as such, if the calling script can override this
> value it can have control over what happens.
The exit-hook is the only one that runs after the source has been built,
so by your definition of 'type B', that's the only one :)
It's conceivable that the lines may be a bit more blurred than that though.
For instance you could run a check in the deb-export-hook that makes sure
your changelog doesn't have the dist as UNRELEASED or something similar,
and abort if it does. Or do similar sanity checks on upstream source too.
In general it's basically not safe to assume anything about what other
people have in their hooks, but should be perfectly safe to set your own
to whatever you want them to be. (the corollary of which means it's
perfectly safe to ignore what other people have in their hooks :)
You have total control over them for your own working copies of your own
repos, so you can pick whatever combination of hooks and scripts is right
for that particular repo.
> I hope this explanation helps :-)
Yes, very much so, thanks! Just to be perfectly clear, I'm not at all
trying to prescribe how you should do this - the above is basically just
me looking at the options we already have to do what you've told me you
need. Mostly so that if we do need to add something extra to gitpkg,
I'm perfectly clear on what is needed and how it should work and/or might
be used.
But if your TMPDIR example above is roughly all you need, then it does
look like you can do it with an exit-hook and an admin-config-hook, that
either have some interactive decision smarts in them, or that respond
differently based on some environment vars of their own which you set
before calling gitpkg.
Then your top level interface can either just be gitpkg itself, or a
(set of) very light wrapper(s) that just do(es) something like:
MY_BUILDD="PPA"
gitpkg $@
And have your hooks switch on $MY_BUILDD etc. for each variation that
you might want to run.
Does that sound like it will work, or do we still need something more?
Best,
Ron
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
