Re: Using the Taskcluster index to find builds

2015-12-02 Thread Chris AtLee
In this case, latest is just latest from wherever. I agree that l10n nightlies should be under 'nightly' as well. On Wed, Dec 2, 2015 at 3:04 PM, Axel Hecht wrote: > On 12/1/15 3:48 PM, Chris AtLee wrote: > >> Localized builds should be at e.g. >> gecko.v2.mozilla-central.latest.firefox-l10n.win

Re: Using the Taskcluster index to find builds

2015-12-02 Thread Axel Hecht
On 12/1/15 3:48 PM, Chris AtLee wrote: Localized builds should be at e.g. gecko.v2.mozilla-central.latest.firefox-l10n.win32-opt And yes, once we've got the naming structure nailed down, wget-en-US should change to use the index. I would expect l10n nightlies to be under nightly? How does one

Re: Using the Taskcluster index to find builds

2015-12-02 Thread Ted Mielczarek
On Wed, Dec 2, 2015, at 09:52 AM, Benjamin Smedberg wrote: > In order to correlate telemetry data, I need a time series of all > mozilla-central nightly builds (with version and buildid). It's > important that when there are multiple nightlies on a given date, that I > get all of them. You shou

Re: Using the Taskcluster index to find builds

2015-12-02 Thread Benjamin Smedberg
Where is the right place to ask questions about this and file bugs? mozilla.dev.builds? I have a series of use-cases that I need to solve, and it's still very unclear to me whether taskcluster is the right solution for these, or how I'd solve them. Here are a few examples: In order to correlat

Re: Using the Taskcluster index to find builds

2015-12-01 Thread Chris AtLee
One approach we've taken when considering changes to the routes used is to play in the 'garbage.' prefix. You can see the results of earlier experiments there: https://tools.taskcluster.net/index/#garbage/garbage Regarding your proposal, I find the word 'nightly' overloaded, and it needs more con

Re: Using the Taskcluster index to find builds

2015-12-01 Thread Julien Wajsberg
hi, Because we have an index, it's now very easy to add new routes. I think it would be a lot more user-friendly to have an index that starts with the product name ("firefox" for example). For example: "gecko.v2.firefox.win64-opt.nightly.latest" instead of "gecko.v2.mozilla-central.nightly.latest

Re: Using the Taskcluster index to find builds

2015-12-01 Thread Chris AtLee
You're right in that we can't change the expiration after the fact, but we can copy all of the artifacts to new tasks with the longer expiration. On Tue, Dec 1, 2015 at 9:53 AM, Ryan VanderMeulen wrote: > What does that mean for jobs that have already run? My understanding is > that we can't cha

Re: Using the Taskcluster index to find builds

2015-12-01 Thread Ryan VanderMeulen
What does that mean for jobs that have already run? My understanding is that we can't change the expiration after the fact for them? Though I guess that it's not an issue as long as we fix bug 1145300 prior to shutting off publishing to archive.m.o? I just want to avoid any gaps in nightly bui

Re: Using the Taskcluster index to find builds

2015-12-01 Thread Chris AtLee
The expiration is currently set to one year, but we can (and should!) change that for nightlies. That work is being tracked in https://bugzilla.mozilla.org/show_bug.cgi?id=1145300 On Mon, Nov 30, 2015 at 7:00 PM, Ryan VanderMeulen wrote: > On 11/30/2015 3:43 PM, Chris AtLee wrote: > >> The RelEn

Re: Using the Taskcluster index to find builds

2015-12-01 Thread Chris AtLee
Localized builds should be at e.g. gecko.v2.mozilla-central.latest.firefox-l10n.win32-opt And yes, once we've got the naming structure nailed down, wget-en-US should change to use the index. On Tue, Dec 1, 2015 at 5:22 AM, Axel Hecht wrote: > I haven't found localized builds and their assets by

Re: Using the Taskcluster index to find builds

2015-12-01 Thread Axel Hecht
I haven't found localized builds and their assets by glancing at things. Are those to come? Also, I suspect we should rewrite wget-en-US? Or add an alternative that's index-bound? Axel On 11/30/15 9:43 PM, Chris AtLee wrote: The RelEng, Cloud Services and Taskcluster teams have been doing a

Re: Using the Taskcluster index to find builds

2015-11-30 Thread Ryan VanderMeulen
On 11/30/2015 3:43 PM, Chris AtLee wrote: The RelEng, Cloud Services and Taskcluster teams have been doing a lot of work behind the scenes over the past few months to migrate the backend storage for builds from the old "FTP" host to S3. While we've tried to make this as seamless as possible, the

Re: Using the Taskcluster index to find builds

2015-11-30 Thread Nicholas Alexander
On Mon, Nov 30, 2015 at 12:43 PM, Chris AtLee wrote: > The RelEng, Cloud Services and Taskcluster teams have been doing a lot of > work behind the scenes over the past few months to migrate the backend > storage for builds from the old "FTP" host to S3. While we've tried to make > this as seamles

Re: Using the Taskcluster index to find builds

2015-11-30 Thread Gijs Kruitbosch
FWIW, I used this for "diditland" - http://www.gijsk.com/blog/2015/11/did-it-land/ and https://gijsk.github.io/diditland/ . It was a significant improvement over scraping archive.mozilla.org's HTML pages for a month, finding the right folder for a nightly, and then scraping that for the right j

Using the Taskcluster index to find builds

2015-11-30 Thread Chris AtLee
The RelEng, Cloud Services and Taskcluster teams have been doing a lot of work behind the scenes over the past few months to migrate the backend storage for builds from the old "FTP" host to S3. While we've tried to make this as seamless as possible, the new system is not a 100% drop-in replacement