El Capitan Buildbot

Post by Sean Farley
Since the public beta of El Capitan has been out for a while, is there
any chance we can get a buildbot set up so that we're ready for the
release?

In the past, we haven't set up a build server for an OS until it public release. It's conceivable that something would change between now and final release that would necessitate rebuilding packages...

Jeremy Lavergne

2015-08-16 21:22:25 UTC

I suspect being available at release (even if rebuilding) is far better than waiting 2+ weeks.

Post by Ryan Schmidt
In the past, we haven't set up a build server for an OS until it public release. It's conceivable that something would change between now and final release that would necessitate rebuilding packages...

Sean Farley

2015-08-16 21:21:31 UTC

Post by Sean Farley
Since the public beta of El Capitan has been out for a while, is there
any chance we can get a buildbot set up so that we're ready for the
release?

Well, it's a buildbot. We could just, you know, rebuild things. In the
recent past, the main delay has been allocating the resources for a new
buildbot, so I figured getting the ball rolling now might help getting
the buildbot sooner.

I think this would be a great time to get reports on which packages have
trouble building on El Capitan before the main release.

Mojca Miklavec

2015-08-17 09:03:16 UTC

Post by Sean Farley
Since the public beta of El Capitan has been out for a while, is there
any chance we can get a buildbot set up so that we're ready for the
release?

Well, it's a buildbot. We could just, you know, rebuild things. In the
recent past, the main delay has been allocating the resources for a new
buildbot, so I figured getting the ball rolling now might help getting
the buildbot sooner.
I think this would be a great time to get reports on which packages have
trouble building on El Capitan before the main release.

I don't want to be pessimistic, but I will be enormously happy if
we'll get the buildbot up and running at all.
Given how long the 10.6 and 10.7 buildbots have been broken ...

And yes, it would make a lot of sense to start the process now if
there is anyone at Apple who is willing and able to look into it.

Mojca

Wahlstedt Jyrki

2015-08-17 10:05:05 UTC

Post by Mojca Miklavec
And yes, it would make a lot of sense to start the process now if
there is anyone at Apple who is willing and able to look into it.

I’d
like to see this, too, but then, we haven’t a release for El Capitan yet, have we!?

!
! Jyrki

Jeremy Lavergne

2015-08-17 13:15:38 UTC

While the beta is a public release, it is under NDA.

"Don’t blog, post screen shots, tweet, or publicly post information about the public beta software, and don't discuss the public beta software with or demonstrate it to others who are not in the Apple Beta Software Program. If Apple has publicly disclosed technical information about the public beta software, it is no longer considered confidential."
https://beta.apple.com/sp/betaprogram/faq

Sadly, one must say this means a buildslave is not yet a viable option—even though the beta OS release is open the public.

Post by Wahlstedt Jyrki
I’d
like to see this, too, but then, we haven’t a release for El Capitan yet, have we!?

Sean Farley

2015-08-17 17:18:11 UTC

Post by Jeremy Lavergne
While the beta is a public release, it is under NDA.
"Don’t blog, post screen shots, tweet, or publicly post information about the public beta software, and don't discuss the public beta software with or demonstrate it to others who are not in the Apple Beta Software Program. If Apple has publicly disclosed technical information about the public beta software, it is no longer considered confidential."
https://beta.apple.com/sp/betaprogram/faq
Sadly, one must say this means a buildslave is not yet a viable option—even though the beta OS release is open the public.

Darn. Could we at least allocate another buildslave with Yosemite so
that when El Capitan is release, we could at least upgrade right away?

Jeremy Lavergne

2015-08-17 18:18:40 UTC

That's a good idea. We could actually schedule downtime for the Yosemite box to be cloned then :-o (we could clear out all artifacts to possibly speed up the process).

But once it's released: how quickly could someone update the OS on this new buildslave?

It's still a blackbox of hope.

Post by Sean Farley
Darn. Could we at least allocate another buildslave with Yosemite so
that when El Capitan is release, we could at least upgrade right away?

Sean Farley

2015-08-17 19:20:11 UTC

Post by Jeremy Lavergne
That's a good idea. We could actually schedule downtime for the Yosemite box to be cloned then :-o (we could clear out all artifacts to possibly speed up the process).
But once it's released: how quickly could someone update the OS on this new buildslave?
It's still a blackbox of hope.

Tears of unfathomable sadness :'(

Arno Hautala

2015-08-17 14:23:20 UTC

Post by Mojca Miklavec
I don't want to be pessimistic, but I will be enormously happy if
we'll get the buildbot up and running at all.
Given how long the 10.6 and 10.7 buildbots have been broken ...
And yes, it would make a lot of sense to start the process now if
there is anyone at Apple who is willing and able to look into it.

Given the time it's taken in the past to get a build slave set up and
the difficulty with fixing errors when is it worthwhile to start
looking at setting up a slave that's under closer control?
This could run the gamut from a hosted VM to a spare machine that
someone is willing to keep in a closet.

I already archive all my packages on a home NAS and, if everyone was
trusted on the Internet, MacPorts could just accept package uploads
from anyone to be shared to all. Conceptually, there's no real
difference with any of the committers (maybe just the management team
or elder council) being able to share packages and someone from the
team being able to configure the official build slave.

I suppose I'd be personally happy just running a VM slave on my NAS.
Though that doesn't contribute anything to anyone else and not
everyone has the means to host their own buildbot. Are the buildbots
just using MPAB?

--
arno s hautala /-| ***@alum.wpi.edu

pgp b2c9d448

Joshua Root

2015-08-17 23:33:48 UTC

Post by Arno Hautala

Running your own buildbot instance purely for CI testing purposes is
fine. But...

Post by Arno Hautala
I already archive all my packages on a home NAS and, if everyone was
trusted on the Internet, MacPorts could just accept package uploads
from anyone to be shared to all. Conceptually, there's no real
difference with any of the committers (maybe just the management team
or elder council) being able to share packages and someone from the
team being able to configure the official build slave.

Our hosting is pretty tightly configured such that packages.macports.org
won't accept uploads from outside the Apple firewall (or even the
macosforge VLAN). That probably won't change purely due to Apple infosec
policies, but also shouldn't change just yet because buildbot (at least
the version we're currently using) doesn't have the ability to use TLS
when uploading files from the slaves to the master.

Post by Arno Hautala
I suppose I'd be personally happy just running a VM slave on my NAS.
Though that doesn't contribute anything to anyone else and not
everyone has the means to host their own buildbot. Are the buildbots
just using MPAB?

Yes, this is the config:
<https://trac.macports.org/browser/contrib/buildbot>

Technically, if you hosted your archives somewhere publicly accessible,
signed them, and published the public key, others could decide that they
trust you, and add the URL to their archive_sites.conf and the key to
their pubkeys.conf.

- Josh

Ryan Schmidt

2015-08-18 02:34:40 UTC

Post by Sean Farley
Since the public beta of El Capitan has been out for a while, is there
any chance we can get a buildbot set up so that we're ready for the
release?

Well, it's a buildbot. We could just, you know, rebuild things.

We've never had the situation arise before, so we don't have a procedure in place for accomplishing that, so I'd rather not risk getting into that situation.

Post by Sean Farley
In the
recent past, the main delay has been allocating the resources for a new
buildbot, so I figured getting the ball rolling now might help getting
the buildbot sooner.

This is true, and I've put in a request:

https://trac.macports.org/ticket/48609

Post by Sean Farley
I think this would be a great time to get reports on which packages have
trouble building on El Capitan before the main release.

Perhaps, but after a buildbot build server is initially set up, it takes weeks to build all ports. The resulting log would surely be gigabytes in size. I doubt anyone would be interested in downloading and analyzing that.

If you have 10.11 beta installed, I recommend you install the ports you maintain, and work on fixing any problems you encounter (both with your ports directly or with their dependencies, with proper maintainer approval, of course). If everyone does this in advance of the 10.11 release, that'll be a great help.

Sean Farley

2015-08-18 02:59:49 UTC

Post by Sean Farley
Since the public beta of El Capitan has been out for a while, is there
any chance we can get a buildbot set up so that we're ready for the
release?

Well, it's a buildbot. We could just, you know, rebuild things.

We've never had the situation arise before, so we don't have a procedure in place for accomplishing that, so I'd rather not risk getting into that situation.

So maybe we should create a procedure for that?

Post by Sean Farley
In the
recent past, the main delay has been allocating the resources for a new
buildbot, so I figured getting the ball rolling now might help getting
the buildbot sooner.

https://trac.macports.org/ticket/48609

Thanks!

Post by Sean Farley
I think this would be a great time to get reports on which packages have
trouble building on El Capitan before the main release.

I definitely would.

Post by Ryan Schmidt
If you have 10.11 beta installed, I recommend you install the ports you maintain, and work on fixing any problems you encounter (both with your ports directly or with their dependencies, with proper maintainer approval, of course). If everyone does this in advance of the 10.11 release, that'll be a great help.

Sure, but as a fair warning to others, El Capitan broke things[1] so I
needed to install 10.11 on an extra computer.

[1] http://superuser.com/questions/935756/mac-os-el-capitan-10-11-not-able-to-connect-to-wifiwpa-2-enterprise

Joshua Root

2015-08-18 04:46:24 UTC