Regressions and testing

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Regressions and testing

Damjan Jovanovic
Hi

It seems to me that regressions that we've had with releases recently could
have been prevented with judicious testing.

We do already have considerable tests in our tree, but they are never run.

I've committed a patch that fixes use of junit with hamcrest in both gbuild
and dmake, and should allow us to run "subsequent tests" after a build.

Damjan
Reply | Threaded
Open this post in threaded view
|

Re: Regressions and testing

Patricia Shanahan
The more automated testing we can do, the better. I have a possible
patch for one of the regressions. What do I need to do to run the
"subsequent tests"?

On 11/9/2017 9:46 PM, Damjan Jovanovic wrote:

> Hi
>
> It seems to me that regressions that we've had with releases recently could
> have been prevented with judicious testing.
>
> We do already have considerable tests in our tree, but they are never run.
>
> I've committed a patch that fixes use of junit with hamcrest in both gbuild
> and dmake, and should allow us to run "subsequent tests" after a build.
>
> Damjan
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Regressions and testing

Damjan Jovanovic
In a module:
OOO_SUBSEQUENT_TESTS=1 build
Results go to main/solver/420/.../workdir/JunitTests for gbuild at least.

There's a Perl script main/solenv/bin/subsequent_tests or something like
that which runs the tests in all modules. It's currently broken at the
first line in the file, but even when that's fixed, it complains about the
gbuild module stack being corrupt.

Subsequent tests have limits and may not help much though. The richest and
most thorough tests that can check content round tripped through saving and
reloading, with screenshots, are the bvt and fvt tests in the tests/
directory (same level as main/). Please search for my previous emails where
I explained more.

On Friday, November 10, 2017, Patricia Shanahan <[hidden email]> wrote:

> The more automated testing we can do, the better. I have a possible patch
> for one of the regressions. What do I need to do to run the "subsequent
> tests"?
>
> On 11/9/2017 9:46 PM, Damjan Jovanovic wrote:
>
>> Hi
>>
>> It seems to me that regressions that we've had with releases recently
>> could
>> have been prevented with judicious testing.
>>
>> We do already have considerable tests in our tree, but they are never run.
>>
>> I've committed a patch that fixes use of junit with hamcrest in both
>> gbuild
>> and dmake, and should allow us to run "subsequent tests" after a build.
>>
>> Damjan
>>
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Regressions and testing

Damjan Jovanovic
We already have a bvt test that writes a spreadsheet with a chart:
https://wiki.openoffice.org/wiki/QA/BVT


On Friday, November 10, 2017, Damjan Jovanovic <[hidden email]> wrote:

> In a module:
> OOO_SUBSEQUENT_TESTS=1 build
> Results go to main/solver/420/.../workdir/JunitTests for gbuild at least.
>
> There's a Perl script main/solenv/bin/subsequent_tests or something like
> that which runs the tests in all modules. It's currently broken at the
> first line in the file, but even when that's fixed, it complains about the
> gbuild module stack being corrupt.
>
> Subsequent tests have limits and may not help much though. The richest and
> most thorough tests that can check content round tripped through saving and
> reloading, with screenshots, are the bvt and fvt tests in the tests/
> directory (same level as main/). Please search for my previous emails where
> I explained more.
>
> On Friday, November 10, 2017, Patricia Shanahan <[hidden email]
> <javascript:_e(%7B%7D,'cvml','[hidden email]');>> wrote:
>
>> The more automated testing we can do, the better. I have a possible patch
>> for one of the regressions. What do I need to do to run the "subsequent
>> tests"?
>>
>> On 11/9/2017 9:46 PM, Damjan Jovanovic wrote:
>>
>>> Hi
>>>
>>> It seems to me that regressions that we've had with releases recently
>>> could
>>> have been prevented with judicious testing.
>>>
>>> We do already have considerable tests in our tree, but they are never
>>> run.
>>>
>>> I've committed a patch that fixes use of junit with hamcrest in both
>>> gbuild
>>> and dmake, and should allow us to run "subsequent tests" after a build.
>>>
>>> Damjan
>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>