[DISCUSS] Release 4.2: General Topics

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[DISCUSS] Release 4.2: General Topics

Kay Schenk@apache.org
Hello all--
I think it would be valuable to discuss some general issues/ideas with
the upcoming 4.2 release. My plan is to keep this general discussion "in
play" until Sat, Sept 3, then do a summary with what was agreed to.

WARNING: This is quite long!

*PRIORITIES*

1. Update the localization.

We've had quite a bit of work by the localization folks since the 4.1.1
release. This was the last release, in 2014-08-21 to import localization
updates. Currently, it seems we might also add 3 new languages: Uyghur,
Sinhala, and Icelandic with the 4.2 release. This would include both UI
translations and Help translations.

We need volunteers to lead this endeavor. I, personally, don't know
anything about this process. This is a very high priority and it would
be good to port translations over to our main repository as soon as
possible for testing.

2. Update Java requirement from Java 1.5 to *at least* Java 1.7

I am rather adamant that we change our building requirement to Java 1.7
for all platforms. I will be changing that in our Building Guide today.
Java 1.5 went out of support by Oracle in November, 2009. We use OpenJDK
but I'm sure updates for Java 1.5 through that channel are also no
longer available.
Even Java 1.7 has reached end of life by Oracle for public support as of
April, 2015. To avoid undue issues for some of our current users on
older platforms, I am "OK" with java 1.7. I am fairly confident ALL
users can obtain this for whatever platform they are using.

3. Issues for inclusion

We need to include submitted/tested patches since 4.0.x. This should not
include UI changes which would need to undergo a much longer test period.

The ones I've identified are:


https://bz.apache.org/ooo/buglist.cgi?bug_status=CONFIRMED&bug_status=ACCEPTED&bug_status=REOPENED&bug_status=RESOLVED&f1=product&f2=component&f3=attachments.ispatch&f4=attachments.description&o1=notsubstring&o2=notsubstring&o4=substring&order=Importance&query_format=advanced&resolution=---&v1=ui&v2=ui&v4=patch&version=4.0.0&version=4.0.0-dev&version=4.0.1&version=4.0.1-dev&version=4.1.0&version=4.1.0-beta&version=4.1.0-dev&version=4.1.1&version=4.1.2&version=4.1.2-dev&version=4.2.0-dev

Due to the fact that I actually have trouble identifying patches in BZ,
this query may not be inclusive, so please feel free to do your own
investigation.

Additionally, issue 127068, involving analytics on our source code would
surely be worth investigating.

https://bz.apache.org/ooo/show_bug.cgi?id=127068

You might also see commits involving code that are related to other
issues that are not on the above query.

*BUILDBOTS AND CONFIGURATION*

1. Move to different buildbots?

I will be forwarding a communication I had this morning from
infrastructure concerning our current issues with our buildbots and a
possible solution. You will see that the Linux32 and Linux64 buildbots
are not even the same version of Ubuntu. We could move to Ubuntu 14 for
both these Linux buildbots where we would also have more control over
what's installed on them.
We need a volunteer to lead this effort.

2. Configuration Issues
Add, at least the ant version we're checking for in our configuration is
not the version recommended in our Building Guide.

*PRODUCTION ENVIRONMENT*

For all our past distributions, we've had our own production environment
if you will. This means the end user binaries were produced on AOO
developer equipment, and these developers took responsibility for
signing the binaries and getting them uploaded to SourceForge.  It has
been suggested that we use the ASF buildbots to produce our binaries
with this release. My feeling is that unless we can "move" to a new
buildbot environment that is more consistent with our two Linux
distributions, we can'd to this.

The issues with using an AOO production environment vs ASF is this:
* it is much easier to script signing of binaries and move them to
SourceForge on AOO supplied production equipment.
* if we use ASF buildbot output, the binaries need to be downloaded to
some other location by developers for signing, computing checksums, etc.
There is no direct shell access to the buildbot machine that I am aware
of for transfer purposes.

Andrea has volunteered to set up a production environment for us. SEE:
http://markmail.org/message/b4dbjdeu4llczqwt

We need PMC members to volunteer with this effort if we decide to
continue with the AOO production environment.


--- the end for now --

This is probably enough for now. More coming over the next few days.
--
----------------------------------------
Kay Schenk
Apache OpenOffice

"Things work out best for those who make
 the best of the way things work out."
                         -- John Wooden

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Release 4.2: General Topics

Don Lewis-2
Here is my list:

  * Update bundled redland / rasqal / raptor to fix a CVE.  This is
    currently in progress.  The FreeBSD port has been using the system
    versions of the latest and greatest.

  * Updated bundled serf to fix a CVE and a WebDAV issue.  The FreeBSD
    port has been using the system version of the latest and greatest.

  * [low priority] Update bundled vigra.  This version is no longer
    available upstream.  It is used by the basebmp module, but I don't
    know what it is used for or how to test it.  I believe only the
    headers are used.  The FreeBSD port uses the latest and greatest
    version and I haven't heard of any problems.

  * [low priority] Updated bundled CoinMP.  There has been an API
    change, which prevents the FreeBSD port from using the system
    version.  Difficulty unknown ...

  * Fix the easy to fix compiler warnings.  In the case of unused
    private class member variables, deleting them would cause an ABI
    change because the object size would shrink.  If that is a problem
    the alternate would be to mark them unused.  I think there are only
    a small number of these, but they get reported many times.

  * Handle the Wiki Publisher and PDF Import extensions as integrated
    extensions:
    <https://wiki.openoffice.org/wiki/Setup_-_Bundled_Extensions>
    I've done some work on this in the FreeBSD port, but I haven't
    tackled the installer part for other platforms.

  * Fix a problem that I stumbled across involving extensions that can
    cause an infinite crash / recover loop.  I think I can generate an
    alert when I detect the problem.  I'm not sure if I can use an
    existing alert message or will need a new one.  If the latter, then
    there will be some translation work required.

  * Give network connection failures a more informative error message.
    I stumbled across this when tracking down a problem that was
    confined to FreeBSD.


Someone should fix the broken spellcheck problem.  It generates a lot of
complaints from our users.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Release 4.2: General Topics

Rory O'Farrell
On Wed, 24 Aug 2016 00:01:22 -0700 (PDT)
Don Lewis <[hidden email]> wrote:

> Here is my list:
>
>   * Update bundled redland / rasqal / raptor to fix a CVE.  This is
>     currently in progress.  The FreeBSD port has been using the system
>     versions of the latest and greatest.
>
>   * Updated bundled serf to fix a CVE and a WebDAV issue.  The FreeBSD
>     port has been using the system version of the latest and greatest.
>
>   * [low priority] Update bundled vigra.  This version is no longer
>     available upstream.  It is used by the basebmp module, but I don't
>     know what it is used for or how to test it.  I believe only the
>     headers are used.  The FreeBSD port uses the latest and greatest
>     version and I haven't heard of any problems.
>
>   * [low priority] Updated bundled CoinMP.  There has been an API
>     change, which prevents the FreeBSD port from using the system
>     version.  Difficulty unknown ...
>
>   * Fix the easy to fix compiler warnings.  In the case of unused
>     private class member variables, deleting them would cause an ABI
>     change because the object size would shrink.  If that is a problem
>     the alternate would be to mark them unused.  I think there are only
>     a small number of these, but they get reported many times.
>
>   * Handle the Wiki Publisher and PDF Import extensions as integrated
>     extensions:
>     <https://wiki.openoffice.org/wiki/Setup_-_Bundled_Extensions>
>     I've done some work on this in the FreeBSD port, but I haven't
>     tackled the installer part for other platforms.
>
>   * Fix a problem that I stumbled across involving extensions that can
>     cause an infinite crash / recover loop.  I think I can generate an
>     alert when I detect the problem.  I'm not sure if I can use an
>     existing alert message or will need a new one.  If the latter, then
>     there will be some translation work required.
>
>   * Give network connection failures a more informative error message.
>     I stumbled across this when tracking down a problem that was
>     confined to FreeBSD.
>
>
> Someone should fix the broken spellcheck problem.  It generates a lot of
> complaints from our users.
>

The "broken spellcheck problem" seems largely to be caused by over hasty close-down of the computer before cached files (both hardware and software caches) are properly written to disk.  User education is the answer - educate them to be patient in closedown and wait for disk activity to finish before power-off.

--
Rory O'Farrell <[hidden email]>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Release 4.2: General Topics

Don Lewis-2
On 24 Aug, Rory O'Farrell wrote:
> On Wed, 24 Aug 2016 00:01:22 -0700 (PDT)
> Don Lewis <[hidden email]> wrote:

>> Someone should fix the broken spellcheck problem.  It generates a lot of
>> complaints from our users.
>>
>
> The "broken spellcheck problem" seems largely to be caused by over hasty close-down of the computer before cached files (both hardware and software caches) are properly written to disk.  User education is the answer - educate them to be patient in closedown and wait for disk activity to finish before power-off.

Maybe using fsync() to flush things out faster when shutting down the
app and keeping a backup copy of the config would help avoid the
problem.   We shouldn't just silently fail when something is corrupt.

I've even experienced this myself on my FreeBSD desktop which stays on
24x7 and has an UPS.  It took me quite a while to notice spellcheck
wasn't working.  I just thought my spelling was unusually good ;-)


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: [DISCUSS] Release 4.2: General Topics

Dennis E. Hamilton
The problem of incomplete shutdown is very serious.  It can result in damaged and completely lost in-progress work, even though apparently saved, and other damage.  The whole problem of shutdown, Quickstarter, and failed auto-recovery is a big ball of snakes that is worthy of a maintenance release all on its own.

With regard to spell-checking, the usual symptom is that suddenly all words are redlined.  Having it stop checking probably goes unnoticed more often, as it did for Don.

With regard to spell-checking, there is a promise to provide a profile-fix tool for Windows,
<https://bz.apache.org/ooo/show_bug.cgi?id=121930#c96>.  That is a bit buried (but look at the issue it is tied to).

I will open a separate task for this now.

At some point, installation of such a tool should at least be part of the built distribution.

So there are three steps:

 1. Get the tool out there as something users can install and use.

 2. Incorporate the tool in our binary distributions.

 3. Fix the shut-down problems we have that are so overwhelming and so damaging to the work of our users.

Steps (1) will require QA work and L10N work in order to reach the greatest number of users.  English will only work for 45% of our users.  It should work better for end users than the instructions that are being provided repetitively each time someone stumbles on this and manages to report it.  

This is clearly a triage situation.  We need something now, we need something about best practices, such as having Quickstarter OFF by default at least on Windows, and also doing something about auto-recovery failures.  And we can't wait for laborious software fixes for which there is no foreseeable developer availability and then production of subsequent full-binary distributions.

Speaking of triage, the targeting of Windows reaches 87% of our users based on platform downloads.  Remedies there are assisted by the fact that cmd.exe (the console processor) and command-line utilities are available on all versions of Windows that AOO supports.  In addition, every install of AOO includes a functional version of Python in the OpenOffice 4 code.  Accomplishing the provision of side-car utilities for other platforms will depend on committers who are devoted to those platforms stepping forward.

 - Dennis

PS: The 4.1.2-patch1 Hotfix will roll to general distribution in the next 24-48 hours.  We have English language covered and there is provision for more L10N localization.  However, the only one completed at this time is Nederland (nl, Dutch) for the Windows version.  A Deutsch (de, German) one is in the works.  To get better coverage, the priority additional L10N-s should be French, Italian, and Spanish.  That gets us to 75% of the users by downloaded languages.  To get to 90% add Japanese, Russian, Polish, (Dutch), and Portuguese.  Chinese would be a great bonus and, of course, we would appreciate any other localizations that AOO enthusiasts and committers provide.

> -----Original Message-----
> From: Don Lewis [mailto:[hidden email]]
> Sent: Wednesday, August 24, 2016 00:38
> To: [hidden email]
> Cc: [hidden email]; [hidden email];
> [hidden email]
> Subject: Re: [DISCUSS] Release 4.2: General Topics
>
> On 24 Aug, Rory O'Farrell wrote:
> > On Wed, 24 Aug 2016 00:01:22 -0700 (PDT)
> > Don Lewis <[hidden email]> wrote:
>
> >> Someone should fix the broken spellcheck problem.  It generates a lot
> of
> >> complaints from our users.
> >>
> >
> > The "broken spellcheck problem" seems largely to be caused by over
> hasty close-down of the computer before cached files (both hardware and
> software caches) are properly written to disk.  User education is the
> answer - educate them to be patient in closedown and wait for disk
> activity to finish before power-off.
>
> Maybe using fsync() to flush things out faster when shutting down the
> app and keeping a backup copy of the config would help avoid the
> problem.   We shouldn't just silently fail when something is corrupt.
>
> I've even experienced this myself on my FreeBSD desktop which stays on
> 24x7 and has an UPS.  It took me quite a while to notice spellcheck
> wasn't working.  I just thought my spelling was unusually good ;-)
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Release 4.2: General Topics

Andrea Pescetti-2
In reply to this post by Kay Schenk@apache.org
Resending to 3 lists... I suggest to have a "canonical" reply-to to the
dev list for the next messages. Andrea

Andrea Pescetti wrote:

> On 23/08/2016 Kay Schenk wrote:
>> WARNING: This is quite long!
>
> And the discussion was even longer, but I'll start with answering this one.
>
> And I'll first note that:
>
> 1) Work is not starting now. We have years of code already committed and
> not shown in previous releases.
>
> 2) Like for every release, we make plans but at a certain point we have
> to cut the release and this "wishlist" is thus a tentative guideline.
>
>> *PRIORITIES*
>> 1. Update the localization.
>> We've had quite a bit of work by the localization folks since the 4.1.1
>> release. This was the last release, in 2014-08-21 to import localization
>> updates. Currently, it seems we might also add 3 new languages: Uyghur,
>> Sinhala, and Icelandic with the 4.2 release. This would include both UI
>> translations and Help translations.
>
> Last translations import were done in 4.1.0 and not 4.1.1 (if I recall
> correctly); but this is a minor detail. There are no new languages to be
> expected in 4.2.0: we have new languages in Pootle, but I don't think
> any of them is ready enough for being released (this may of course
> improve with time). So in short 4.2.0 means that we can add strings to
> the code, which means we can make them available to translators, which
> in turn means we can (we have to) update all translations.
>
>> We need volunteers to lead this endeavor. I, personally, don't know
>> anything about this process.
>
> I'm slowly working on this but I still have something to find/learn.
> I've sent the l10n list a mail sending that I'm planning to test a first
> import in early September - just to test the process.
>
>> 2. Update Java requirement from Java 1.5 to *at least* Java 1.7
>> I am rather adamant that we change our building requirement to Java 1.7
>> for all platforms. I will be changing that in our Building Guide today.
>
> Is there a real reason for it? I see this like saying (this is just an
> example, not to be taken literally) "we drop support for Windows XP
> since it's old and unsupported". In short: if we need work to drop Java
> 1.5 then we have clear advantages in raising our requirement to 1.7,
> otherwise we can simply drop the requirement saying "we won't explicitly
> test compatibility with Java < 1.7"; but in that case we must provide
> ways to obtain a compatible JRE for all the 4 supported platforms.
>
>> 3. Issues for inclusion
>> We need to include submitted/tested patches since 4.0.x. This should not
>> include UI changes which would need to undergo a much longer test period.
>
> The version number is not a detail. We call it 4.2.0 since UI changes
> are allowed. On the other hand, we don't have to include all patches;
> actually, seeing all the code that already went in, I would be more on
> the conservative side here.
>
>> Additionally, issue 127068, involving analytics on our source code would
>> surely be worth investigating.
>> https://bz.apache.org/ooo/show_bug.cgi?id=127068
>
> These are automatically found defects, good for easy fixes but probably
> not really important.
>
> I'd rather suggest that we give some attention to the 4.1.2 regressions,
> especially this one (the only one so far):
> https://bz.apache.org/ooo/show_bug.cgi?id=126622
>
>> *BUILDBOTS AND CONFIGURATION*
>> 1. Move to different buildbots?
>
> Not needed. A "nice to have" if they standardize it, but buildbots (I
> mean, the Linux version they use) are not so relevant for a release.
>
>> 2. Configuration Issues
>> Add, at least the ant version we're checking for in our configuration is
>> not the version recommended in our Building Guide.
>
> The this is a bug in configure, needs its own issue and must be checked.
>
>> *PRODUCTION ENVIRONMENT* ...
>> It has
>> been suggested that we use the ASF buildbots to produce our binaries
>> with this release.
>
> The ASF buildbots and releases cover two different fields. I've been
> misunderstood from time to time, but just to make it clear: I would
> never want that we use the buildbots for releasing (at least for Linux),
> since you want a recent Linux on buildbots and on old Linux on the
> release VM (where this VM is hosted can be deferred to a separate thread).
>
>> Andrea has volunteered to set up a production environment for us. SEE:
>> http://markmail.org/message/b4dbjdeu4llczqwt
>
> I see that discussion has been misunderstood. I'll reply there. It
> suffices to say, here, that I'm not suggesting to use buildbots for the
> release builds. Which basically means I agree with your point of view in
> this respect.
>
> Regards,
>    Andrea.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Release 4.2: General Topics

Andrea Pescetti-2
In reply to this post by Don Lewis-2
On 24/08/2016 Don Lewis wrote:
>    * Updated bundled serf to fix a CVE and a WebDAV issue.  The FreeBSD
>      port has been using the system version of the latest and greatest.

We've been discussing this separately. This will make the build system
even more complex, but it does have advantages and brings to completion
work started with 4.1.2.

>    * Handle the Wiki Publisher and PDF Import extensions as integrated
>      extensions:
>      <https://wiki.openoffice.org/wiki/Setup_-_Bundled_Extensions>
>      I've done some work on this in the FreeBSD port, but I haven't
>      tackled the installer part for other platforms.

Can you elaborate on this? For example the fact that they shouldn't ping
for updates looks problematic (not so bad, but it will require a
reasonable release pace). And actually the document suggests that
bundled dictionaries are handled as integrated extensions, which is not
the case at the moment - and this allows users to get, for example, a
fresh English dictionary when they wish.

> Someone should fix the broken spellcheck problem.  It generates a lot of
> complaints from our users.

We've investigated it a length; it is uncommon (it seems common due to
the huge number of users) and like Rory and Dennis wrote this is better
fixed with an after-the-fact fix: a tool (which ideally would become a
menu item or a first start option) that allows to start with a clean
profile.

Regards,
   Andrea.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]