[PROPOSAL] On Reproducibility of Defects

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[PROPOSAL] On Reproducibility of Defects

Dennis E. Hamilton-2
I have a suggestion with regard to reproducibility of reported defects.

First, it is important to reproduce using the same release and platform as the reporter if at all possible.  

If the user has a configuration that is not available for confirming yet the bug can't be reproduced with the current released version on the same platform, that should be reported so that others can take a look.  Also, the reporter should have the opportunity to see if they can deepen the information about their case.

If the incident can't be reproduced with a development build, be certain that it can be reproduced with the current release on the same platform.  If not, treat that as if it is the previous case.  

If it is a defect that is confirmed for the current release and not reproducible for an in-development release, please confirm the defect and set the target for fix to the coming version.  If there is a workaround until an update is released, provide that.

 - Dennis

PS: I have no thoughts on when such an issue should be marked resolved.  I would recommend waiting until the next release has a release candidate, at the earliest, and there is a final check.  (These are good regression checkers.)

PPS: It is easy for users to see similarities in their own experience and add their report as a comment to a current issue.  The only way I know to determine that is to ask for more detail and if there is indication of a clear difference, create a new issue and lead the user to it.



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: [PROPOSAL] On Reproducibility of Defects

Dennis E. Hamilton-2
Update at the end ...

-----Original Message-----
From: Dennis E. Hamilton [mailto:[hidden email]]
Sent: Thursday, September 17, 2015 14:22
To: [hidden email]
Subject: [PROPOSAL] On Reproducibility of Defects

I have a suggestion with regard to reproducibility of reported defects.

First, it is important to reproduce using the same release and platform as the reporter if at all possible.  

If the user has a configuration that is not available for confirming yet the bug can't be reproduced with the current released version on the same platform, that should be reported so that others can take a look.  Also, the reporter should have the opportunity to see if they can deepen the information about their case.

If the incident can't be reproduced with a development build, be certain that it can be reproduced with the current release on the same platform.  If not, treat that as if it is the previous case.  

If it is a defect that is confirmed for the current release and not reproducible for an in-development release, please confirm the defect and set the target for fix to the coming version.  If there is a workaround until an update is released, provide that.

 - Dennis

PS: I have no thoughts on when such an issue should be marked resolved.  I would recommend waiting until the next release has a release candidate, at the earliest, and there is a final check.  (These are good regression checkers.)

<orcmid>
  Duhh.  The obvious time to mark it resolved is when the reporter(s) confirms that the problem does not occur with a new release.  
</orcmid>

PPS: It is easy for users to see similarities in their own experience and add their report as a comment to a current issue.  The only way I know to determine that is to ask for more detail and if there is indication of a clear difference, create a new issue and lead the user to it.



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [PROPOSAL] On Reproducibility of Defects

Kay Schenk-2
On Thu, Sep 17, 2015 at 7:17 PM, Dennis E. Hamilton <[hidden email]>
wrote:

> Update at the end ...
>
> -----Original Message-----
> From: Dennis E. Hamilton [mailto:[hidden email]]
> Sent: Thursday, September 17, 2015 14:22
> To: [hidden email]
> Subject: [PROPOSAL] On Reproducibility of Defects
>
> I have a suggestion with regard to reproducibility of reported defects.
>
> I've numbered the steps for easier reference.

-- a definite "yes" on this one --

1.First, it is important to reproduce using the same release and platform
> as the reporter if at all possible.
>
> -- I'm a little confused by this explanation. By "user" you mean QA
volunteer? --


> 2. If the user has a configuration that is not available for confirming
> yet the bug can't be reproduced with the current released version on the
> same platform, that should be reported so that others can take a look.
> Also, the reporter should have the opportunity to see if they can deepen
> the information about their case.
>
> -- and for this one,  "previous case is #1 or #2? --

3. If the incident can't be reproduced with a development build, be certain

> that it can be reproduced with the current release on the same platform.
> If not, treat that as if it is the previous case.
>
> If it is a defect that is confirmed for the current release and not
> reproducible for an in-development release, please confirm the defect and
> set the target for fix to the coming version.  If there is a workaround
> until an update is released, provide that.
>
>  - Dennis
>

And...we have a wiki page for QA information at :
https://wiki.openoffice.org/wiki/Quality_Assurance

I would guess that a good number of our QA volunteers are pretty
well-versed in general QA practices including some of the processes you've
described here, but it would be nice to include the points above, and other
matters as well,  on the wiki page at some point so folks don't need to
search the resource links we have there. This would also help with
outlining processes specific to Apache OpenOffice.

Thanks for taking this on.


> PS: I have no thoughts on when such an issue should be marked resolved.  I
> would recommend waiting until the next release has a release candidate, at
> the earliest, and there is a final check.  (These are good regression
> checkers.)
>
> <orcmid>
>   Duhh.  The obvious time to mark it resolved is when the reporter(s)
> confirms that the problem does not occur with a new release.
> </orcmid>
>
> PPS: It is easy for users to see similarities in their own experience and
> add their report as a comment to a current issue.  The only way I know to
> determine that is to ask for more detail and if there is indication of a
> clear difference, create a new issue and lead the user to it.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>


--
-------------------------------------------------------------------------------------------------
MzK

“The journey of a thousand miles begins with a single step.”
                                                          --Lao Tzu