[asterisk-dev] Git Migration

Tue Sep 16 21:52:10 CDT 2014

On Tue, Sep 16, 2014 at 5:01 PM, Russell Bryant <russell at russellbryant.net>
wrote:

> On Tue, Sep 16, 2014 at 3:48 PM, Matthew Jordan <mjordan at digium.com>
> wrote:
>
>> "And there was much rejoicing"
>>
>
> \o/
>
>
>> But seriously, we all know that a lot of people have wanted to move to
>> Git for some time. For the record, everyone at Digium has wanted to move
>> the project to Git for some time. I swore to myself that we wouldn't do
>> another Standard release on Subversion - after we spent at least six weeks
>> mucking around with merge conflicts during Asterisk 12 - and with Asterisk
>> 14 looming ever closer, the time is now to start getting something done on
>> this.
>>
>> So!
>>
>> To that end, a page on the wiki has been made with some initial thoughts:
>>
>> https://wiki.asterisk.org/wiki/display/AST/Git+Migration
>>
>> To summarize:
>>  * A comparison of management platforms has been done. Barring a giant
>> catastrophe or some insane limitation, we're going to go simple here and
>> stick with gitolite. Reasoning is on the wiki page.
>>  * The first thing to migrate is _not_ the Asterisk project, but the
>> Asterisk Test Suite. That will allow us (or force us) to deal with some of
>> the tooling and process issues, which will make it easier to tackle
>> Asterisk.
>>
>> I'm sure there are a lot of opinions about all of this, and if you have
>> thoughts on technical or process hurdles we may be running into, I'd love
>> to hear it. Just remember that like many other things in life and
>> development, there's a lot of ways to manage your source code. You may
>> really, really, _REALLY_ like the way Project X does it, and you may think
>> that the way we are proposing it is clearly inferior. That's great, you may
>> be right. But in the interest of this not dragging on for another 5 years,
>> I'd like to keep any discussions focussed on getting things done while not
>> shooting ourselves in our virtual feet.
>>
>
> So, I realize this is pretty much exactly the opposite of what you just
> said, but I wanted to offer some comments on the infrastructure.  :-)  I'm
> really not deeply invested in what is chosen.  My interest here is just to
> provide an overview of another option in the interest of exploring options.
>

So my "no bike-shedding please" innuendo was definitely not meant to cut
off all conversation. This is a pretty important move, and whatever we
choose to go to we're going to live with for at least some time - I
certainly hope we don't change tools out every other year.

So, to set what I hope are a few guidelines:
(1) I know this is a subject with a lot of opinions, and coming to a
concurrence on something that is opinion based is tough. Voice opinions
after thinking, "would I be willing to spend time working on this"?
(2) This is a project that can easily suck several developers for a long
time if the scope of it expands substantially. The scope should hopefully
be kept as tight as possible, as we will have a lot of re-tooling to do
once the git move occurs (all CI, code review, releasing, etc.)
(3) The Asterisk project has several hard requirements, a few softer ones,
that have to be met:
  * Code submitted has to be assigned to a CLA. We have to verify that
people proposing patches have acknowledged that they were licensed to
submit said patch in some fashion.
  * Issue tracking has to be done in JIRA (large investment, huge cost in
moving to anything else right now).
  * Code must be reviewed.
  * Any substitute for Review Board, Bamboo, or other existing tools should
have obvious benefits over the existing tools.

>
> For the last few years, most of my time has been going into OpenStack [1].
>  We use git and I have become a big fan of our workflow and infrastructure.
>  It's all open source and reusable.
>
> From a high level, all patches go to a code review system.  *Every* patch
> must be peer reviewed (usually by 2 people, but that's a policy decision).
>  *Every* patch must also pass tests.  Once a patch passes both tests and
> peer review, it is automatically merged into the repository.
>
> I *love* that workflow for several reasons.  If it's appealing, it's
> probably much easier to do it now while you're doing a big switch anyway.
>  If you're not sold, I'm certainly not hurt.  Like I said, I just wanted to
> offer info.  The current plan will be less up front setup for sure.
>

I am a huge fan of this.

We've been moving (slowly but surely) towards this model. Code is reviewed
more often than not, and with Crowd integration, anyone can propose a
patch. Tests are written for the vast majority of new work. We *do* still
have a problem with bouncing tests - and while Bamboo may point the finger
more at 12+ currently (due to a current spat with native RTP bridges,
direct media, and transfers) - the bouncing is actually more pernicious in
many ways with the 1.8/11 branches. We'd have to solve that problem if we
move to this model.

I don't consider that a bad thing at all.

>
> If you're a hands on kind of person, browse http://review.openstack.org/
> for open code reviews.  You can also see patches going through CI pipelines
> on http://status.openstack.org/zuul//
>
> The major tools involved are:
>
>  - gerrit for code review and repository management [2][3]
>  - jenkins for CI [4]
>  - Zuul, A CI job scheduler that automates running things in response to
> events on gerrit. [5]
>  - CGit, repo hosting [6][7]
>
>
This would replace Review Board and Bamboo as well, so we'll need to
consider the effort involved with that.

> Everything we use is managed via puppet and all of the configuration is in
> git.  It's designed to be reusable.  The folks that run it have documented
> how to re-use it [8] and are quite friendly.  You can find them in
> #openstack-infra on freenode.
>
>
Awesome. I'll do that - in particular, I'm interested what is involved in
the recurring maintenance of gerrit/Jenkins/Zuul.

>
> [1] http://www.openstack.org
> [2] https://code.google.com/p/gerrit/
> [3] https://review.openstack.org/
> [4] http://jenkins-ci.org/
> [5] http://ci.openstack.org/zuul.html
> [6] http://ci.openstack.org/git.html
> [7] http://git.openstack.org
> [8] http://ci.openstack.org/running-your-own.html
>
> I'll also try to answer the fields of the comparison chart on the wiki
> page:
>
> -- Web View
>
> Yes.
>
> -- Project Management
>
> This would replace existing usage of bamboo and reviewboard.  It does not
> include issue tracking.  Keeping that is would be fine.
>
> -- Protected Branches
>
> Gerrit supports permissions on a per branch basis.
>
> -- Rewriting history
>
> Not sure the intent here ... wanting to make that can be avoided?
>
> In this system, the merges are automated so you can't accidentally do a
> bad push.  An admin can force reset a repo if needed, of course.
>
>
I like that 'automated' prevents bad pushes, but I can see how that would
be the case. If anything, the fact that you can assign people into roles,
with permissions per roles, fits fairly closely with the existing model
that we have already.

> -- Team repos
>
> I'd recommend just using your own account on github or whatever.
>
> -- Git hooks
>
> For what, exactly?  It's probably easier to discuss the problem that needs
> to be solved.
>

In general, there are a few use cases for either git hooks or web hooks (or
something else, such as the Gerrit event stream):
(1) Auto-close issues in JIRA
(2) Auto-close reviews in Review Board (but clearly not an issue in this
case)
(3) Prevent pushes from private branches going to public branches. This
would have to be done a pre-hook in some fashion.

I think there are various ways to achieve all three of these, although
admittedly the 'private branch' one is the most tricky and annoying.

> -- Web hooks
>
> Again, it's probably worth discussing the use case.  Gerrit has an event
> stream.  That event stream includes merges.  Tools to do things in
> responses to merges (which could be running a web hook) listen and react.
>
> -- Performance
>
> Yes.  :-)
>
> There's lots of stats I could dig up, but as one example, I track code
> review stats for a subset of projects on review.openstack.org  In the
> last year, for that subset, there have been 266,127 code reviews done (avg
> > 700 per day).  That gives some sense of scale.  From a quick glance at
> the CI status page (http://status.openstack.org/zuul/), it has been
> launching about 500-600 jobs per hour today.
>
> -- Process Recommendation
>
> I discussed this a good bit above, but I'm happy to answer questions.
>

Wiki page updated!

You make a lot of good points, and I think it's worth honestly exploring
this option. It does look like it would be more work, but as you pointed
out, now is the time to do it - and it may not be substantially more than
what we're already going to have to do. We'll have to take a deeper look at
Gerrit and find out.

Thanks for weighing in on this!

Matt

-- 
Matthew Jordan
Digium, Inc. | Engineering Manager
445 Jan Davis Drive NW - Huntsville, AL 35806 - USA
Check us out at: http://digium.com & http://asterisk.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20140916/fc9eb944/attachment-0001.html>