[asterisk-biz] Res: 2008 Predictions
Russell Bryant
russell at digium.com
Fri Jan 4 12:54:18 CST 2008
Trixter aka Bret McDanel wrote:
> You never actually countered his argument, there were smoke and mirrors
> to avoid it, but never actually counter it.
>
> Lets review:
> you "guessed" that 1/3 were community patches, this means the other
> percentages are guesses too
> you "processed" issues - that doesnt mean that the patches were applied
It is rare that an issue that contains a patch gets closed without commit of the
actual patch, or a similar patch that provides equivalent functionality. I
would go as far as to say this is the case for 95% of patch submissions. The
times that issues are closed without commit are cases such as the patch is not
functional and the submitter has not addressed the issues that have been raised.
If you can point out any individual cases where you are not happy with the
outcome, I would be happy to address it individually. Also, I would very much
prefer recent examples. Our handling of bugs.digium.com has come a _long_ way
over the past year.
> Now I dont know if those numbers are accurate or not, I havent actually
> gotten real stats on it. I do know that the assertion made was
> challenged and focus diverted, which makes me personally suspect the
> challenge. Its like the chewbacca defense when you state you disagree
> with something and talk around the issue instead of being direct about
> it.
I certainly did not mean to divert the focus. I was simply trying to provide
the information on issues handled to the best of my ability. The percentages
are certainly guesses. However, as the person who as closed the 2nd most issues
on bugs.digium.com, behind Mark Spencer, I feel that I have a _very_ good feel
for the mix of things that get posted there. However, I invite others that have
spent a lot of time working on there to post their feelings, as well.
>> that a lot of this type of work isn't very visible, unless you really go looking
>> in the svn commit history,
>
> SVN keeps a revision history, that would have instantly said how many
> commits were performed last year. While it wouldnt be a direct 1:1 for
> the bug tracker, since someone may commit stuff multiple times during
> the course of their work, there may be commits that dont have an issue
> in the bug system, it would help to justify that there were close to
> 3000 bugs that were resolved.
Sure, I can do that.
For the 1.4 branch, which has a policy of bug-fix only commits, there were 1748
commits in 2007. My guess was ... ( 2960 * (2 / 3) * .95 ) is about 1875.
(Also, keep in mind there may have been issues that only affected Asterisk 1.2
on the bug tracker, as we were fully supporting that for most of the year.)
~/src/asterisk/1.4$ svn log -r 49096:95577 --xml | grep '<logentry' | wc -l
1748
For Asterisk trunk, that includes bug fixes, as well as new features, there were
3345 commits in 2007. So, that is also in line with my claim that processing
almost 3000 issues resulted in commits for most of them. The additional commits
are merging in new features, performance enhancements, or other code cleanups
that did not go through bugs.digium.com.
~/src/asterisk/trunk$ svn log -r 49092:95578 --xml | grep '<logentry' | wc -l
3345
As you said, while this does not provide an exact correlation to issues closed
on bugs.digium.com, it does provide a pretty good rough picture.
> Now the bug tracking software probably has the ability to sort based on
> date ranges and resolutions so that could probably get better numbers
> for how many (based on the guess 5% of 2/3 of almost 3000 or about 100
> that were closed without commit)
It does have some information. But, unfortunately, the resolution type is not
correctly set for a number of issues. We made a change to mantis at some point
to make sure that we set the resolution properly when closing an issue, but that
was at some point in the middle of the year.
In any case, I made an attempt to get an even closer look.
First I got a CSV export of all issues that have the following criteria, which
is the best I could do with the data available.
1) Are closed
2) Were last updated in 2007
The number comes out to be a total of 3370 issues. It includes issues that were
closed at some point before 2007, but received some sort of comment in 2007...
Here is the breakdown:
--------------------------------------------
duplicate: 116
fixed: 1918
no change required: 419
not fixable: 49
open: 324
reopened: 27
suspended: 254
unable to reproduce: 122
won't fix: 141
Total: 3370
The ones that were closed with commit:
fixed: 1918
Percent of total: 56.9 %
The ones closed without commit:
no change required: 419
not fixable: 49
won't fix: 141
unable to reproduce: 122
suspended: 254
duplicate: 116
Percent of total: 32.7 %
The ones with an ambiguous resolution:
open: 324
reopened: 27
Percent of total: 10.4 %
------------------------------------------------
Now, I did some more analysis on the issues closed without commit (minus the
duplicate issues).
Here are some tables of resolution type versus severity, including numbers for
each that indicate how many of these issues actually had a patch associated with
them:
(Determining that an issue had a patch is simply done by seeing if the word
"patch" is in the title. We do a pretty good job of making sure that [patch] is
in the issue title when a patch is posted. So, it should be accurate enough for
the sake of this discussion ...)
no_change_required ...
"block": 8
-- had patch: 1
"crash": 39
-- had patch: 1
"feature": 34
-- had patch: 7
"major": 130
-- had patch: 3
"minor": 178
-- had patch: 15
"text": 4
-- had patch: 1
"trivial": 10
-- had patch: 2
"tweak": 16
-- had patch: 2
not_fixable ...
"block": 3
-- had patch: 1
"crash": 6
-- had patch: 0
"feature": 1
-- had patch: 0
"major": 14
-- had patch: 0
"minor": 21
-- had patch: 0
"trivial": 1
-- had patch: 0
"tweak": 3
-- had patch: 0
won't_fix ...
"block": 2
-- had patch: 0
"crash": 9
-- had patch: 0
"feature": 36
-- had patch: 12
"major": 29
-- had patch: 2
"minor": 55
-- had patch: 10
"text": 2
-- had patch: 1
"trivial": 6
-- had patch: 3
"tweak": 2
-- had patch: 0
unable_to_reproduce ...
"block": 5
-- had patch: 0
"crash": 56
-- had patch: 1
"major": 31
-- had patch: 0
"minor": 46
-- had patch: 4
"trivial": 1
-- had patch: 0
"tweak": 1
-- had patch: 0
suspended ...
"block": 7
-- had patch: 0
"crash": 40
-- had patch: 0
"feature": 50
-- had patch: 25
"major": 64
-- had patch: 3
"minor": 77
-- had patch: 14
"text": 3
-- had patch: 1
"trivial": 6
-- had patch: 0
"tweak": 7
-- had patch: 2
Total number of issues that
- were last updated in 2007
- are closed
- had a patch
==> 111, 3.3% of all issues
Total number of issues with a patch: 806
--> Percent closed without commit, 111 / 806 = 13.8 %
So, by these rough calculations, about 1 in 8 patches don't make it into the
tree for some reason or another. However, I promise you that you do _not_ want
every patch that we get to be merged. Some patches recreate functionality that
is already there. Others are not up to our quality standards. We work very
hard to maintain a consistent architecture with all new things that get added,
and some submissions do not fit that architecture. Again, if anyone has
individual situations they are concerned with, I would be happy to address them
individually.
-----------------------------------------------
> As I do not have information to the contrary but have seen bugs get
> comments in the bug system saying "happy birthday" when they are over a
> year and a patch has been constantly ported forward and ready to go, I
> kinda lean towards the they dont get committed side.
As I said, and have hopefully demonstrated to some degree, we have come a _long_
way in the past year with our handling of bugs.digium.com. I know that we have
definitely had issues with patch turnaround, but the situation has improved and
will continue to do so. However, I also do not deny that there are still cases
where it takes us longer than I am happy with to handle patch submissions.
However, my main point is that we _are_ committing patches - a _lot_ of patches.
And, even as the project grows and we are getting more and more submissions, we
are growing the development team as best we can to keep up with it.
--
Russell Bryant
Senior Software Engineer
Open Source Team Lead
Digium, Inc.
More information about the asterisk-biz
mailing list