[asterisk-dev] About asterisk development plans

[John Lange]

On Wed, 2009-03-18 at 17:55 -0600, Steve Murphy wrote:

> Believe me, if this was all that was needed, I'd have implemented this
> two years ago!
> The trouble is, that the times change from test to test. And
> therefore, you'll not
> be able to diff CDR's from test to test.

Ya, not just time stamps but sip channel ids etc. I admit I didn't think
about that but ...

>  Some interesting scripts could be written
> to just verify that one timepoint is equal to another, that one
> timepoint is between
> two others, perhaps on other CDR's, but the element of reproducibility
> is not there.

With a little more scripting could you use "cut" or "awk" (or whatever)
to remove the unique parts of the CDR?

Perhaps my feeling on this is a bit different but I don't think the test
suite needs to inherently "know" what success looks like, it only needs
to know that the result it got this time was the same as the result it
got last time.

We're regression testing and that to me means assuring that this version
works the same as the previous version. Us humans are the ones
responsible for verifying that the result we're checking against works
the way it should. The test suite is responsible for verifying it's the
same going forward.

In the event that a code fix changes a result the test suite would be
updated after the humans have verified it's working. There need not be
any "intelligence" in the result checking.

-- 
John Lange
http://www.johnlange.ca