[DISCUSSION] Build environment future

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

[DISCUSSION] Build environment future

Peter Kovacs-2
Hello Damjan and all


I would like to re-discuss our current plan. Hoping to gain a common view.

Current state is mostly we use gmake, there are still some difficult to
migrate dmake projects. And we use Ant for java.

The plan is not to stop at the dmake -> gmake conversion but to move on
to scons, removing as much dependencies as we can. Right?

I would like to set the target to build everything to Ant, removing as
much dependencies we can.


My arguments are mostly that Ant is supported by most when not all IDEs
and I would really like to have an IDE as working environment, and my
hope is that it is easier maybe to integrate an Ant build environment
then a scons or gmake environment.

 I think this would give us a better base then the plan above. So what
was the arguments against Ant again?


All the Best

Peter


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] Build environment future

Mechtilde Stehmann-2
Hello Peter and all,

Am 30.10.19 um 06:25 schrieb Peter Kovacs:
> Hello Damjan and all
>
>
> I would like to re-discuss our current plan. Hoping to gain a common view.
>
> Current state is mostly we use gmake, there are still some difficult to
> migrate dmake projects. And we use Ant for java.

Are gmake or rather dmake only for building with Java? and scons?
>
> The plan is not to stop at the dmake -> gmake conversion but to move on
> to scons, removing as much dependencies as we can. Right?

For building on Linux IMO it is easier to remove the shipped
dependencies because there are also part of the distributions we build on.

But we need them for building on Windows, don't we.
>
> I would like to set the target to build everything to Ant, removing as
> much dependencies we can.

What does everything means? only Java stuff? or also the C++ stuff?

> My arguments are mostly that Ant is supported by most when not all IDEs
> and I would really like to have an IDE as working environment, and my
> hopegma is that it is easier maybe to integrate an Ant build environment
> then a scons or gmake environment.
>
>  I think this would give us a better base then the plan above. So what
> was the arguments against Ant again?

see my questions above
>
>
> All the Best
>
> Peter

Kind regards

--
Mechtilde Stehmann
## Apache OpenOffice
## Freie Office Suite für Linux, MacOSX, Windows
## Debian Developer
## PGP encryption welcome
## F0E3 7F3D C87A 4998 2899  39E7 F287 7BBA 141A AD7F


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] Build environment future

Patricia Shanahan
In reply to this post by Peter Kovacs-2
As a user of the build system, I think we have too many build
strategies, and should not add another one unless it is absolutely
essential to do so. Of course I would prefer an IDE, but another round
of build system changes is far too high a price to pay.

Given that most is on gmake, I would much prefer to stick with gmake.

If we keep changing our collective mind about the build system we will
never have a consistent system, but will always have some modules on
each of several systems.

On 10/29/2019 10:25 PM, Peter Kovacs wrote:

> Hello Damjan and all
>
>
> I would like to re-discuss our current plan. Hoping to gain a common view.
>
> Current state is mostly we use gmake, there are still some difficult to
> migrate dmake projects. And we use Ant for java.
>
> The plan is not to stop at the dmake -> gmake conversion but to move on
> to scons, removing as much dependencies as we can. Right?
>
> I would like to set the target to build everything to Ant, removing as
> much dependencies we can.
>
>
> My arguments are mostly that Ant is supported by most when not all IDEs
> and I would really like to have an IDE as working environment, and my
> hope is that it is easier maybe to integrate an Ant build environment
> then a scons or gmake environment.
>
>   I think this would give us a better base then the plan above. So what
> was the arguments against Ant again?
>
>
> All the Best
>
> Peter
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

--
This email has been checked for viruses by AVG.
https://www.avg.com


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] Build environment future

Branko Čibej
In reply to this post by Peter Kovacs-2
On 30.10.2019 06:25, Peter Kovacs wrote:

> Hello Damjan and all
>
>
> I would like to re-discuss our current plan. Hoping to gain a common view.
>
> Current state is mostly we use gmake, there are still some difficult to
> migrate dmake projects. And we use Ant for java.
>
> The plan is not to stop at the dmake -> gmake conversion but to move on
> to scons, removing as much dependencies as we can. Right?


As a far-too-long-time user of Scons (in Serf), let me just add a
caution: Scons is very, very broken and not at all well maintained.
Writing a truly cross-platform, cross-toolchain SConstruct file for
anything other than really trivial cases amounts to writing a *lot* of
platform-specific Python code to make the builds work.

If you're already moving to gmake (without autotools, and I expect using
Cygwin on Windows), then I suggest you finish the transition, then leave
well enough alone.

-- Brane

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] Build environment future

Jim Jagielski
In reply to this post by Peter Kovacs-2
Is the build environment really enough of a failure that we need to spend time on redoing it again?

Now if people want to work on that, I'm fine... we are all volunteers and work on stuff that interests us, but certainly, if people are *looking* for things to do, improving the actual code itself might be more worthwhile that rejiggering the build setup.

> On Oct 30, 2019, at 1:25 AM, Peter Kovacs <[hidden email]> wrote:
>
> Hello Damjan and all
>
>
> I would like to re-discuss our current plan. Hoping to gain a common view.
>
> Current state is mostly we use gmake, there are still some difficult to
> migrate dmake projects. And we use Ant for java.
>
> The plan is not to stop at the dmake -> gmake conversion but to move on
> to scons, removing as much dependencies as we can. Right?
>
> I would like to set the target to build everything to Ant, removing as
> much dependencies we can.
>
>
> My arguments are mostly that Ant is supported by most when not all IDEs
> and I would really like to have an IDE as working environment, and my
> hope is that it is easier maybe to integrate an Ant build environment
> then a scons or gmake environment.
>
>  I think this would give us a better base then the plan above. So what
> was the arguments against Ant again?
>
>
> All the Best
>
> Peter
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] Build environment future

Marcus (OOo)
In reply to this post by Patricia Shanahan
Am 30.10.19 um 08:06 schrieb Patricia Shanahan:

> As a user of the build system, I think we have too many build
> strategies, and should not add another one unless it is absolutely
> essential to do so. Of course I would prefer an IDE, but another round
> of build system changes is far too high a price to pay.
>
> Given that most is on gmake, I would much prefer to stick with gmake.
>
> If we keep changing our collective mind about the build system we will
> never have a consistent system, but will always have some modules on
> each of several systems.

strong +1

Otherwise the probability and risk is high that someone start with
another build system and does not bring it to the end. Then we have just
another partly working one - and the complexity is reaching new heights
that nobody wants.

Marcus



> On 10/29/2019 10:25 PM, Peter Kovacs wrote:
>> Hello Damjan and all
>>
>>
>> I would like to re-discuss our current plan. Hoping to gain a common
>> view.
>>
>> Current state is mostly we use gmake, there are still some difficult to
>> migrate dmake projects. And we use Ant for java.
>>
>> The plan is not to stop at the dmake -> gmake conversion but to move on
>> to scons, removing as much dependencies as we can. Right?
>>
>> I would like to set the target to build everything to Ant, removing as
>> much dependencies we can.
>>
>>
>> My arguments are mostly that Ant is supported by most when not all IDEs
>> and I would really like to have an IDE as working environment, and my
>> hope is that it is easier maybe to integrate an Ant build environment
>> then a scons or gmake environment.
>>
>>   I think this would give us a better base then the plan above. So what
>> was the arguments against Ant again?


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] Build environment future

Peter Kovacs-3
In reply to this post by Jim Jagielski
Thanks all for the value able feedback. It shows how difficult it is to
get the right direction.

I think Ant is easier then gmake. But maybe this is only a treachery
impression. And we do use both systems. (Next to dmake somehow, but I
think at least there is no mix between dmake and gmake. Or I got it
wrong. ;) )

Okay, I will try to support gmake for now.


Important is in the first step the difficulty drops to build OpenOffice.
And I try to slice everything down into smaller to swallowable pieces.
Only small turns.

Since most topics I have are architectural issues.

On 30.10.19 14:01, Jim Jagielski wrote:

> Is the build environment really enough of a failure that we need to spend time on redoing it again?
>
> Now if people want to work on that, I'm fine... we are all volunteers and work on stuff that interests us, but certainly, if people are *looking* for things to do, improving the actual code itself might be more worthwhile that rejiggering the build setup.
>
>> On Oct 30, 2019, at 1:25 AM, Peter Kovacs <[hidden email]> wrote:
>>
>> Hello Damjan and all
>>
>>
>> I would like to re-discuss our current plan. Hoping to gain a common view.
>>
>> Current state is mostly we use gmake, there are still some difficult to
>> migrate dmake projects. And we use Ant for java.
>>
>> The plan is not to stop at the dmake -> gmake conversion but to move on
>> to scons, removing as much dependencies as we can. Right?
>>
>> I would like to set the target to build everything to Ant, removing as
>> much dependencies we can.
>>
>>
>> My arguments are mostly that Ant is supported by most when not all IDEs
>> and I would really like to have an IDE as working environment, and my
>> hope is that it is easier maybe to integrate an Ant build environment
>> then a scons or gmake environment.
>>
>>  I think this would give us a better base then the plan above. So what
>> was the arguments against Ant again?
>>
>>
>> All the Best
>>
>> Peter
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] Build environment future

Damjan Jovanovic
In reply to this post by Peter Kovacs-2
Hi

Ant already works for some Java modules, though at a module-only level. I
did get gbuild to extract dependency information from Ant, and build Ant
modules in the right order. For C++ though, I don't see how Ant is
possible. Even if we do get it to call the C++ compiler, it doesn't support
parallel builds. Also Ant is quite old, its build.xml file format wasn't
designed to be verifiable by an XML schema.

I should have some time for more AOO development in a week or so. I've been
stuck on porting the "config" build API to gbuild (which seems to be some
sort of API for XML configuration of UNO components, and schemas and XSLT
code to verify them at build time - main/solenv/inc/tg_config.mk); once
that's ported, 9 more modules should be portable to gbuild. That has been
particularly difficult, and it seems to affect post-build packaging too.

Another 8 modules relate to Windows. I'll have to setup up a fuller Windows
build environment for those, and I wonder if they need updating - several
of them use .NET 2.

It was quite encouraging to see that on Win64, several modules that didn't
build in dmake, started building once they were converted to gbuild.

In other news, I've been playing around with Wine, to try and cross-compile
the Windows version of AOO without Windows. I helped fix a Wine bug that
was crashing Cygwin scripts during installation and making the install take
hours (
https://source.winehq.org/git/wine.git/commit/2e53f8bccb65d112e5e341586c730094950fe6c3).
Cygwin installed nicely on Wine, bash builtin commands started working.
Then I ran into a bug where Cygwin expects user APCs to be called during
DLL loading, and Wine's DLL loading doesn't call any wait functions that
would trigger calling APCs, so a thread that would be created by an APC
doesn't get created, so fork() hangs waiting for that thread to suspend all
other threads and copy their memory to the child process (which is the only
way to fork() properly on Windows). Patching Cygwin to create that thread
directly (without using APCs) gets it further, fork() creates the child
process, but then fails an assertion and aborts because some data structure
in the child process is not at the expected address. If we can get a
working Windows build environment set up in Wine, we should hopefully
reduce build times for the Windows build from the current 5-8 hours, closer
to the 90 minutes it takes for the *nix build. AOO itself installed and
worked in Wine since OpenOffice.org version 1, so we should be able to both
build and test in Wine.

Of course, there is also the question of how well the Platform SDK and
other dependencies work on Wine. But I am quite optimistic. Cygwin is
unique in messing around with low-level operating system implementation
details. It even binary-patches some functions in NTDLL.DLL to speed up
getting the current working directory (which is apparently slow on Windows).



On Wed, Oct 30, 2019 at 7:25 AM Peter Kovacs <[hidden email]> wrote:

> Hello Damjan and all
>
>
> I would like to re-discuss our current plan. Hoping to gain a common view.
>
> Current state is mostly we use gmake, there are still some difficult to
> migrate dmake projects. And we use Ant for java.
>
> The plan is not to stop at the dmake -> gmake conversion but to move on
> to scons, removing as much dependencies as we can. Right?
>
> I would like to set the target to build everything to Ant, removing as
> much dependencies we can.
>
>
> My arguments are mostly that Ant is supported by most when not all IDEs
> and I would really like to have an IDE as working environment, and my
> hope is that it is easier maybe to integrate an Ant build environment
> then a scons or gmake environment.
>
>  I think this would give us a better base then the plan above. So what
> was the arguments against Ant again?
>
>
> All the Best
>
> Peter
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] Build environment future

Don Lewis-2
I'd like to see build.pl go away and switch to gmake at the top level.
build.pl is bloated with a lot of stuff that we don't use and gets in
the way of efficient parallel builds.  Sometimes we don't have N modules
that we can build at the same time, so it would be nice to be able to
use the extra CPU cores to start more parallel compiles in the modules
that we are able to build.  Other times modules can be single-threaded,
and it would be nice to start building more modules.  This is possible
using gmake at the top level as well as the module level.

Generating the top-level Makefile with the module dependencies should be
fairly easy.  I think the big missing piece is the deliver stuff for the
non-gbuild modules.

On  5 Nov, Damjan Jovanovic wrote:

> Hi
>
> Ant already works for some Java modules, though at a module-only level. I
> did get gbuild to extract dependency information from Ant, and build Ant
> modules in the right order. For C++ though, I don't see how Ant is
> possible. Even if we do get it to call the C++ compiler, it doesn't support
> parallel builds. Also Ant is quite old, its build.xml file format wasn't
> designed to be verifiable by an XML schema.
>
> I should have some time for more AOO development in a week or so. I've been
> stuck on porting the "config" build API to gbuild (which seems to be some
> sort of API for XML configuration of UNO components, and schemas and XSLT
> code to verify them at build time - main/solenv/inc/tg_config.mk); once
> that's ported, 9 more modules should be portable to gbuild. That has been
> particularly difficult, and it seems to affect post-build packaging too.
>
> Another 8 modules relate to Windows. I'll have to setup up a fuller Windows
> build environment for those, and I wonder if they need updating - several
> of them use .NET 2.
>
> It was quite encouraging to see that on Win64, several modules that didn't
> build in dmake, started building once they were converted to gbuild.
>
> In other news, I've been playing around with Wine, to try and cross-compile
> the Windows version of AOO without Windows. I helped fix a Wine bug that
> was crashing Cygwin scripts during installation and making the install take
> hours (
> https://source.winehq.org/git/wine.git/commit/2e53f8bccb65d112e5e341586c730094950fe6c3).
> Cygwin installed nicely on Wine, bash builtin commands started working.
> Then I ran into a bug where Cygwin expects user APCs to be called during
> DLL loading, and Wine's DLL loading doesn't call any wait functions that
> would trigger calling APCs, so a thread that would be created by an APC
> doesn't get created, so fork() hangs waiting for that thread to suspend all
> other threads and copy their memory to the child process (which is the only
> way to fork() properly on Windows). Patching Cygwin to create that thread
> directly (without using APCs) gets it further, fork() creates the child
> process, but then fails an assertion and aborts because some data structure
> in the child process is not at the expected address. If we can get a
> working Windows build environment set up in Wine, we should hopefully
> reduce build times for the Windows build from the current 5-8 hours, closer
> to the 90 minutes it takes for the *nix build. AOO itself installed and
> worked in Wine since OpenOffice.org version 1, so we should be able to both
> build and test in Wine.
>
> Of course, there is also the question of how well the Platform SDK and
> other dependencies work on Wine. But I am quite optimistic. Cygwin is
> unique in messing around with low-level operating system implementation
> details. It even binary-patches some functions in NTDLL.DLL to speed up
> getting the current working directory (which is apparently slow on Windows).
>
>
>
> On Wed, Oct 30, 2019 at 7:25 AM Peter Kovacs <[hidden email]> wrote:
>
>> Hello Damjan and all
>>
>>
>> I would like to re-discuss our current plan. Hoping to gain a common view.
>>
>> Current state is mostly we use gmake, there are still some difficult to
>> migrate dmake projects. And we use Ant for java.
>>
>> The plan is not to stop at the dmake -> gmake conversion but to move on
>> to scons, removing as much dependencies as we can. Right?
>>
>> I would like to set the target to build everything to Ant, removing as
>> much dependencies we can.
>>
>>
>> My arguments are mostly that Ant is supported by most when not all IDEs
>> and I would really like to have an IDE as working environment, and my
>> hope is that it is easier maybe to integrate an Ant build environment
>> then a scons or gmake environment.
>>
>>  I think this would give us a better base then the plan above. So what
>> was the arguments against Ant again?
>>
>>
>> All the Best
>>
>> Peter
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] Build environment future

Jim Jagielski
Before we start reworking the build environment to pieces, could we look at possibly doing a 4.2.0 release 1st?
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]