A more sane way to build - SCons, deCygwination and other hopes

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

A more sane way to build - SCons, deCygwination and other hopes

Damjan Jovanovic
Hi

After days of failing to add a few new simple features to gbuild, I've now
reached my limits, and have begun experimenting with the SCons build system
instead.

It's starting to work. Having ported some of gbuild.mk, LinkTarget.mk and
platform/freebsd.mk to SCons, as well as a module's local gbuild files, I
can now compile files and (badly) link them into libraries, and clean the
build.

SCons is an advanced next-generation build system like gbuild, with high
level declarative syntax in Python, support for multiple modules that build
in parallel, header dependencies, file change detection through MD5 sums of
contents instead of timestamps so rebuilds are faster, etc. It builds C,
C++, Objective C, Java, Fortran, D, the sorely necessary Flex and Yacc that
gbuild doesn't support. It supports tons of platforms and compilers,
including OS/2. It's maintainable and usable, can print debugging info such
as dependency trees, and is generally pleasant to work with. We've
discussed it before on this list, but I never got to trying it until now.

Virtually everything gbuild does, is already done by SCons, and often done
much better. The syntax for SCons files is similar/related to gbuild's
syntax, so an automated conversion from gbuild to SCons might even be
possible.

So far, I have defines, includes and C[XX]FLAGS working. I've structured it
a bit better, with platform-specific files in classes that only provide
variables to slot in higher up, instead of one giant set of globally
mutable global variables like in gbuild. Linking still needs a bit of work.
Windows is next on the list, as Cygwin will be the ultimate test of whether
we can use it.

I am very hopeful. SCons has a long history and is used by other projects,
while only OO.o derivatives use gbuild. It's well documented. It supports
features we need which gbuild doesn't, like library version names and
flex/yacc. It's supposed to work on Windows, both inside and outside of
Cygwin and could help us build without Cygwin some day. It supports many
versions of Visual Studio and could help us in upgrading to a new one. We
will need to add custom builders to SCons for our custom file formats like
IDL, but it's just Python, not my favorite language but infinitely better
than GNU make.

It would help if someone could review my changes, as I am not very familiar
with Python and there might be better ways to write some of the code. I'll
post a patch for review after some further development.

Thank you
Damjan
Reply | Threaded
Open this post in threaded view
|

Re: A more sane way to build - SCons, deCygwination and other hopes

Peter Kovacs-3
sounds great from what you write. Lets try SCons build system.
Where can I find your changes so I can help? Have you checked them into
trunk?

I have some expreience with python, which will come in handy.

Meanwhile I read Scons Documentation.

Am Samstag, den 02.12.2017, 10:05 +0200 schrieb Damjan Jovanovic:

> Hi
>
> After days of failing to add a few new simple features to gbuild,
> I've now
> reached my limits, and have begun experimenting with the SCons build
> system
> instead.
>
> It's starting to work. Having ported some of gbuild.mk, LinkTarget.mk
> and
> platform/freebsd.mk to SCons, as well as a module's local gbuild
> files, I
> can now compile files and (badly) link them into libraries, and clean
> the
> build.
>
> SCons is an advanced next-generation build system like gbuild, with
> high
> level declarative syntax in Python, support for multiple modules that
> build
> in parallel, header dependencies, file change detection through MD5
> sums of
> contents instead of timestamps so rebuilds are faster, etc. It builds
> C,
> C++, Objective C, Java, Fortran, D, the sorely necessary Flex and
> Yacc that
> gbuild doesn't support. It supports tons of platforms and compilers,
> including OS/2. It's maintainable and usable, can print debugging
> info such
> as dependency trees, and is generally pleasant to work with. We've
> discussed it before on this list, but I never got to trying it until
> now.
>
> Virtually everything gbuild does, is already done by SCons, and often
> done
> much better. The syntax for SCons files is similar/related to
> gbuild's
> syntax, so an automated conversion from gbuild to SCons might even be
> possible.
>
> So far, I have defines, includes and C[XX]FLAGS working. I've
> structured it
> a bit better, with platform-specific files in classes that only
> provide
> variables to slot in higher up, instead of one giant set of globally
> mutable global variables like in gbuild. Linking still needs a bit of
> work.
> Windows is next on the list, as Cygwin will be the ultimate test of
> whether
> we can use it.
>
> I am very hopeful. SCons has a long history and is used by other
> projects,
> while only OO.o derivatives use gbuild. It's well documented. It
> supports
> features we need which gbuild doesn't, like library version names and
> flex/yacc. It's supposed to work on Windows, both inside and outside
> of
> Cygwin and could help us build without Cygwin some day. It supports
> many
> versions of Visual Studio and could help us in upgrading to a new
> one. We
> will need to add custom builders to SCons for our custom file formats
> like
> IDL, but it's just Python, not my favorite language but infinitely
> better
> than GNU make.
>
> It would help if someone could review my changes, as I am not very
> familiar
> with Python and there might be better ways to write some of the code.
> I'll
> post a patch for review after some further development.
>
> Thank you
> Damjan

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: A more sane way to build - SCons, deCygwination and other hopes

Jim Jagielski
In reply to this post by Damjan Jovanovic
Playing devils advocate, does it make sense to introduce
Yet Another Build System at this stage?

Also, there are a few projects that I know of that
use SCons. In general, one of the most popular common
threads related to them are "Why the hell are you using SCons?" :)

> On Dec 2, 2017, at 3:05 AM, Damjan Jovanovic <[hidden email]> wrote:
>
> Hi
>
> After days of failing to add a few new simple features to gbuild, I've now
> reached my limits, and have begun experimenting with the SCons build system
> instead.
>
> It's starting to work. Having ported some of gbuild.mk, LinkTarget.mk and
> platform/freebsd.mk to SCons, as well as a module's local gbuild files, I
> can now compile files and (badly) link them into libraries, and clean the
> build.
>
> SCons is an advanced next-generation build system like gbuild, with high
> level declarative syntax in Python, support for multiple modules that build
> in parallel, header dependencies, file change detection through MD5 sums of
> contents instead of timestamps so rebuilds are faster, etc. It builds C,
> C++, Objective C, Java, Fortran, D, the sorely necessary Flex and Yacc that
> gbuild doesn't support. It supports tons of platforms and compilers,
> including OS/2. It's maintainable and usable, can print debugging info such
> as dependency trees, and is generally pleasant to work with. We've
> discussed it before on this list, but I never got to trying it until now.
>
> Virtually everything gbuild does, is already done by SCons, and often done
> much better. The syntax for SCons files is similar/related to gbuild's
> syntax, so an automated conversion from gbuild to SCons might even be
> possible.
>
> So far, I have defines, includes and C[XX]FLAGS working. I've structured it
> a bit better, with platform-specific files in classes that only provide
> variables to slot in higher up, instead of one giant set of globally
> mutable global variables like in gbuild. Linking still needs a bit of work.
> Windows is next on the list, as Cygwin will be the ultimate test of whether
> we can use it.
>
> I am very hopeful. SCons has a long history and is used by other projects,
> while only OO.o derivatives use gbuild. It's well documented. It supports
> features we need which gbuild doesn't, like library version names and
> flex/yacc. It's supposed to work on Windows, both inside and outside of
> Cygwin and could help us build without Cygwin some day. It supports many
> versions of Visual Studio and could help us in upgrading to a new one. We
> will need to add custom builders to SCons for our custom file formats like
> IDL, but it's just Python, not my favorite language but infinitely better
> than GNU make.
>
> It would help if someone could review my changes, as I am not very familiar
> with Python and there might be better ways to write some of the code. I'll
> post a patch for review after some further development.
>
> Thank you
> Damjan


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: A more sane way to build - SCons, deCygwination and other hopes

Peter Kovacs-3
Am Samstag, den 02.12.2017, 09:19 -0500 schrieb Jim Jagielski:
> Playing devils advocate, does it make sense to introduce
> Yet Another Build System at this stage?
I see it as for the search for the future build system. There is always
the discussion popping up what to use.
Gbuild seem to make some troubles. We have looked into Maven with no
good result.
So far satisfactory outcome. So you as the release team still use the
old one with the known difficulties.

I think we should give it a try. Well buts that me, as a noob I want to
try always all options and then decide where my favours are.
>
> Also, there are a few projects that I know of that
> use SCons. In general, one of the most popular common
> threads related to them are "Why the hell are you using SCons?" :)

Do they bash or is there some serious arguments behind that?- I am
always interested in negative opinion as long as they have substance.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: A more sane way to build - SCons, deCygwination and other hopes

Damjan Jovanovic
In reply to this post by Jim Jagielski
On Sat, Dec 2, 2017 at 4:19 PM, Jim Jagielski <[hidden email]> wrote:

> Playing devils advocate, does it make sense to introduce
> Yet Another Build System at this stage?
>
>
I anticipated this question.

Firstly, we don't only have dmake and gbuild as our build systems. We also
use Ant, meta-build tools like build.pl and co, Microsoft's nmake for
main/icu on Windows, and probably more. The existing number of tools
doesn't negatively affect development, so why should 1 more?

Secondly SCons can replace ./configure, so in all all-SCons world we would
eliminate 3 tools, not just 2.

As for "sense . .. at this stage", we've hit several walls with gbuild at
this stage:
* It can't version libraries in the form of reg3.dll vs reg.so linking to
reg.so.3 on *nix. Most of gbuild was written with the assumption libraries
on *nix always end with .so. Even when we dangerously weaken those
assumptions, and badly hack it, it doesn't deliver the link...
* It already broke certain mixtures of build settings, eg. I think you
can't both debug build and use precompiled headers on Windows, CFLAGS gets
lost somewhere...
* Nobody knows gbuild. The syntax is atrocious. It uses obscure features of
GNU make. We can't debug it. It takes days of work to investigate/fix any
problem with it, work that could be better spent on a this vast project
with few development resources.


> Also, there are a few projects that I know of that
> use SCons. In general, one of the most popular common
> threads related to them are "Why the hell are you using SCons?" :)
>
>
I'd like to see those discussions.

SCons as opposed to what?

Damjan
Reply | Threaded
Open this post in threaded view
|

Re: A more sane way to build - SCons, deCygwination and other hopes

Jim Jagielski

> On Dec 2, 2017, at 1:59 PM, Damjan Jovanovic <[hidden email]> wrote:
>
> On Sat, Dec 2, 2017 at 4:19 PM, Jim Jagielski <[hidden email]> wrote:
>
>> Playing devils advocate, does it make sense to introduce
>> Yet Another Build System at this stage?
>>
>>
> I anticipated this question.

:)

>
> Secondly SCons can replace ./configure, so in all all-SCons world we would
> eliminate 3 tools, not just 2.

./configure seems to work fine.

>
> * Nobody knows gbuild.

And more people know SCons?? :)

My point is that likely more people know make and gmake
and its obscure features than SCons.

What I don't want is some hybrid monstrosity like we
have now. Nor do I want work on the build system to hold
off release of 4.2.0.

Just my own 2c and all the above IMHO


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: A more sane way to build - SCons, deCygwination and other hopes

Marcus (OOo)
Am 02.12.2017 um 23:13 schrieb Jim Jagielski:

>
>> On Dec 2, 2017, at 1:59 PM, Damjan Jovanovic <[hidden email]> wrote:
>>
>> On Sat, Dec 2, 2017 at 4:19 PM, Jim Jagielski <[hidden email]> wrote:
>>
>>> Playing devils advocate, does it make sense to introduce
>>> Yet Another Build System at this stage?
>>>
>>>
>> I anticipated this question.
>
> :)
>
>>
>> Secondly SCons can replace ./configure, so in all all-SCons world we would
>> eliminate 3 tools, not just 2.
>
> ./configure seems to work fine.
>
>>
>> * Nobody knows gbuild.
>
> And more people know SCons?? :)
>
> My point is that likely more people know make and gmake
> and its obscure features than SCons.
>
> What I don't want is some hybrid monstrosity like we
> have now. Nor do I want work on the build system to hold
> off release of 4.2.0.

relating to the last sentence:
For this there is the fantastic invention of branches. ;-)

OK, seriously:
At the moment we are talking more about a 4.2.0 release than all other
posts in the past together. Therefore we should think of when the best
time point is for creating a new 4.2.0 branch. IMHO it is coming really
closer.

Then we can go on on Trunk with improving the build system without any
effects for 4.2.0.

My 2ct.

Marcus


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: A more sane way to build - SCons, deCygwination and other hopes

Don Lewis-2
In reply to this post by Damjan Jovanovic
On  2 Dec, Damjan Jovanovic wrote:

> On Sat, Dec 2, 2017 at 4:19 PM, Jim Jagielski <[hidden email]> wrote:
>
>> Playing devils advocate, does it make sense to introduce
>> Yet Another Build System at this stage?
>>
>>
> I anticipated this question.
>
> Firstly, we don't only have dmake and gbuild as our build systems. We also
> use Ant, meta-build tools like build.pl and co, Microsoft's nmake for
> main/icu on Windows, and probably more. The existing number of tools
> doesn't negatively affect development, so why should 1 more?
>
> Secondly SCons can replace ./configure, so in all all-SCons world we would
> eliminate 3 tools, not just 2.
>
> As for "sense . .. at this stage", we've hit several walls with gbuild at
> this stage:
> * It can't version libraries in the form of reg3.dll vs reg.so linking to
> reg.so.3 on *nix. Most of gbuild was written with the assumption libraries
> on *nix always end with .so. Even when we dangerously weaken those
> assumptions, and badly hack it, it doesn't deliver the link...

Other than for the benefit of extensions, I don't see much point in
versioning the libraries because they are otherwise private.  If add
versioning so that it is more obvious that an old extension won't work
with a new version of AOO, are we disiplined enough to do the
appropriate version bumps?

> * It already broke certain mixtures of build settings, eg. I think you
> can't both debug build and use precompiled headers on Windows, CFLAGS gets
> lost somewhere...

That should be fixable, but ...

> * Nobody knows gbuild. The syntax is atrocious. It uses obscure features of
> GNU make. We can't debug it. It takes days of work to investigate/fix any
> problem with it, work that could be better spent on a this vast project
> with few development resources.

Agreed 1000% here.

If you are doing a build with bundled python, how do you bootstrap?  If
you are on Windows and don't have python, how do you run the SCons
equivalent of configure and then use SCons to build python?

And a more recent question near and dear to my heart, how easy is it to
do per-target optimization flag overrides?


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: A more sane way to build - SCons, deCygwination and other hopes

Peter Kovacs-3


On 03.12.2017 01:57, Don Lewis wrote:

> On  2 Dec, Damjan Jovanovic wrote:
>> * Nobody knows gbuild. The syntax is atrocious. It uses obscure features of
>> GNU make. We can't debug it. It takes days of work to investigate/fix any
>> problem with it, work that could be better spent on a this vast project
>> with few development resources.
> Agreed 1000% here.
>
> If you are doing a build with bundled python, how do you bootstrap?  If
> you are on Windows and don't have python, how do you run the SCons
> equivalent of configure and then use SCons to build python?
Today you need perl. So we replace one language with another.
I think the benefit is that Scons is a generic Buildsystem that is
maintained by a team.
So creating a valid build environment may be much simpler then the
version we have today.
But this depends how we use the tool.

Configure and bootstrap is up to us. We could stick with the current
process. (Replace only the build command)
We could include the current process within Scons. Or we could integrate
both into Scons using Scons provided tools.
While I am not sure how to do the last step.
What I have seen if we integrate it into Scon, we can define our own
Flags to control the behaviour. (Switch Bootstrap on, or not)
> And a more recent question near and dear to my heart, how easy is it to
> do per-target optimization flag overrides?
Scons thinks in build Environments, which are Enviroment Objects. Each
Environment contains build flags.

An Enviroment could Fetch Values from another enviroment, but we can
also expand values, replace values or clone them. There are also ways to
Append  and to Prepend values (with the option do do this unique).
An Environment can be created having default values.
We can define Issue handling on environment level. If we know
combinations do not work or something is missing.

We can also define multiple Enviroments. However a build config can only
be associated with one build environment.

I hope this gives an Idea what posilities we have with Scons. How we
design our build process is up to us.
I would think that we have one global environment that feeds from
Templates like Debug and release. Each Module has then a specific own
Environment which feeds from the global ones.

HTH
Peter


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: A more sane way to build - SCons, deCygwination and other hopes

Andrea Pescetti-2
In reply to this post by Jim Jagielski
On 02/12/2017 Jim Jagielski wrote:
> What I don't want is some hybrid monstrosity like we
> have now. Nor do I want work on the build system to hold
> off release of 4.2.0.

This is surely a good point. Actually, the wide delay between 4.1.0 and
the future 4.2.0 is due in big part to build changes. Most welcome, but
also time consuming.

There is a reason, though, that makes Damjan's current effort worth
investigating in all cases: we had to put a large patch (for a bugfix
involving work with remote files) on hold for 4.1.2 since that would
have required updating serf, which in turn would have implied that we
added support for SCons as they have switched to it for their recent
releases. I can dig up the issue if you need, just ask if interested.
Having a way to build SCons-based packages would already be a step
forward. And yes, it would be "just another tool" in this case, but with
so many dependencies we must be able to integrate many build systems.

Regards,
   Andrea.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: A more sane way to build - SCons, deCygwination and other hopes

Damjan Jovanovic
In reply to this post by Don Lewis-2
On Sunday, December 3, 2017, Don Lewis <[hidden email]> wrote:

> On  2 Dec, Damjan Jovanovic wrote:
> > On Sat, Dec 2, 2017 at 4:19 PM, Jim Jagielski <[hidden email]> wrote:
> >
> >> Playing devils advocate, does it make sense to introduce
> >> Yet Another Build System at this stage?
> >>
> >>
> > I anticipated this question.
> >
> > Firstly, we don't only have dmake and gbuild as our build systems. We
> also
> > use Ant, meta-build tools like build.pl and co, Microsoft's nmake for
> > main/icu on Windows, and probably more. The existing number of tools
> > doesn't negatively affect development, so why should 1 more?
> >
> > Secondly SCons can replace ./configure, so in all all-SCons world we
> would
> > eliminate 3 tools, not just 2.
> >
> > As for "sense . .. at this stage", we've hit several walls with gbuild at
> > this stage:
> > * It can't version libraries in the form of reg3.dll vs reg.so linking to
> > reg.so.3 on *nix. Most of gbuild was written with the assumption
> libraries
> > on *nix always end with .so. Even when we dangerously weaken those
> > assumptions, and badly hack it, it doesn't deliver the link...
>
> Other than for the benefit of extensions, I don't see much point in
> versioning the libraries because they are otherwise private.  If add
> versioning so that it is more obvious that an old extension won't work
> with a new version of AOO, are we disiplined enough to do the
> appropriate version bumps?
>
> > * It already broke certain mixtures of build settings, eg. I think you
> > can't both debug build and use precompiled headers on Windows, CFLAGS
> gets
> > lost somewhere...
>
> That should be fixable, but ...
>
> > * Nobody knows gbuild. The syntax is atrocious. It uses obscure features
> of
> > GNU make. We can't debug it. It takes days of work to investigate/fix any
> > problem with it, work that could be better spent on a this vast project
> > with few development resources.
>
> Agreed 1000% here.
>
> If you are doing a build with bundled python, how do you bootstrap?  If
> you are on Windows and don't have python, how do you run the SCons
> equivalent of configure and then use SCons to build python?
>
>
I am currently doing it by installing Python natively in Windows,
installing SCons there through pip, and then calling that scons.bat from
Cygwin.

And, beyond my wildest expectations, SCons in this setup successfully
compiled and linked a module, both inside Cygwin AND OUTSIDE IT!!! :) :)


> And a more recent question near and dear to my heart, how easy is it to
> do per-target optimization flag overrides?
>
>
C[XX]FLAGS are specified to SCons like all other settings, as arrays of
strings. You can specify common settings in the environment object and can
add to those or replace them on individual SharedObjects.


> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

Damjan
Reply | Threaded
Open this post in threaded view
|

Re: A more sane way to build - SCons, deCygwination and other hopes

Don Lewis-2
In reply to this post by Damjan Jovanovic
On  2 Dec, Damjan Jovanovic wrote:

> * It already broke certain mixtures of build settings, eg. I think you
> can't both debug build and use precompiled headers on Windows, CFLAGS gets
> lost somewhere...

I figured out that problem and discovered that when using precompiled
headers with Visual C++, the optimization flags need to be consistent
between the precompiled headers and when the source files are compiled.

After getting past that, I found that when debugging enabled the
argument to -Fd must also be consistent between compiling the headers
and compiling the sources.  The first problem there is that when
compiling the sources the value of $(PDBFILE) is somehow getting lost.
I'm not sure what is going wrong there, but I have a workaround.  The
next problem is that when building a module with multiple libraries, the
value of $(PDBFILE) has the library name built in as in:
  $W/LinkTarget/pdb/Library/<libname>.lib.pdb
That means that the headers need to be compiled separately for each
library with the matching value of $(PDBFILE) and the resulting compiled
headers need to be stashed in a separate, per-library place instead of
  $W/PrecompiledHeader/debug/precompiled_<module>.hxx.{pch,pch.obj}
This is pretty baked into the the gbuild framework due to the use of
globals.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: A more sane way to build - SCons, deCygwination and other hopes

Carl Marcum
In reply to this post by Damjan Jovanovic
On 12/02/2017 03:05 AM, Damjan Jovanovic wrote:

> Hi
>
> After days of failing to add a few new simple features to gbuild, I've now
> reached my limits, and have begun experimenting with the SCons build system
> instead.
>
> It's starting to work. Having ported some of gbuild.mk, LinkTarget.mk and
> platform/freebsd.mk to SCons, as well as a module's local gbuild files, I
> can now compile files and (badly) link them into libraries, and clean the
> build.
>
> SCons is an advanced next-generation build system like gbuild, with high
> level declarative syntax in Python, support for multiple modules that build
> in parallel, header dependencies, file change detection through MD5 sums of
> contents instead of timestamps so rebuilds are faster, etc. It builds C,
> C++, Objective C, Java, Fortran, D, the sorely necessary Flex and Yacc that
> gbuild doesn't support. It supports tons of platforms and compilers,
> including OS/2. It's maintainable and usable, can print debugging info such
> as dependency trees, and is generally pleasant to work with. We've
> discussed it before on this list, but I never got to trying it until now.
>
> Virtually everything gbuild does, is already done by SCons, and often done
> much better. The syntax for SCons files is similar/related to gbuild's
> syntax, so an automated conversion from gbuild to SCons might even be
> possible.
>
> So far, I have defines, includes and C[XX]FLAGS working. I've structured it
> a bit better, with platform-specific files in classes that only provide
> variables to slot in higher up, instead of one giant set of globally
> mutable global variables like in gbuild. Linking still needs a bit of work.
> Windows is next on the list, as Cygwin will be the ultimate test of whether
> we can use it.
>
> I am very hopeful. SCons has a long history and is used by other projects,
> while only OO.o derivatives use gbuild. It's well documented. It supports
> features we need which gbuild doesn't, like library version names and
> flex/yacc. It's supposed to work on Windows, both inside and outside of
> Cygwin and could help us build without Cygwin some day. It supports many
> versions of Visual Studio and could help us in upgrading to a new one. We
> will need to add custom builders to SCons for our custom file formats like
> IDL, but it's just Python, not my favorite language but infinitely better
> than GNU make.
>
> It would help if someone could review my changes, as I am not very familiar
> with Python and there might be better ways to write some of the code. I'll
> post a patch for review after some further development.
>
> Thank you
> Damjan
>

Hi Damjan,

Please don't take this as a suggestion.

I admire your dedication to finding a solution to our build tool issues
and all your work.

This is only curiosity.

Have you looked at Gradle at all?
It uses Apache Groovy for scripting rather than Python and may be a more
familiar syntax.

I use it almost exclusively now for my other (much much smaller Java and
Groovy) projects.

I know it's used for Java, and C/C++ but not sure of the pros/cons vs.
the others you have looked into.

Best regards,
Carl

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: A more sane way to build - SCons, deCygwination and other hopes

Damjan Jovanovic
In reply to this post by Peter Kovacs-3


On Sat, Dec 2, 2017 at 10:42 AM, Peter Kovacs <[hidden email]> wrote:
sounds great from what you write. Lets try SCons build system.
Where can I find your changes so I can help? Have you checked them into
trunk?


Please find a patch with my current SCons changes attached.

Currently only main/fileaccess builds, on Windows and FreeBSD, and doesn't even build completely as I haven't done the "ComponentTarget" yet. You run "scons" in main/ or "scons -u" in main/fileaccess. AOO has to already be built.

 
I have some expreience with python, which will come in handy.


That's great to hear. Please let me know what you think about the structure, classes, should we use globals and soenv and SCon's Environment, etc. There is no easier time to make changes than at the beginning.

Should we make a separate branch for SCons or develop in trunk? It doesn't alter any existing files so it shouldn't interfere with our existing build.
 
Meanwhile I read Scons Documentation.

Am Samstag, den 02.12.2017, 10:05 +0200 schrieb Damjan Jovanovic:
> Hi
>
> After days of failing to add a few new simple features to gbuild,
> I've now
> reached my limits, and have begun experimenting with the SCons build
> system
> instead.
>
> It's starting to work. Having ported some of gbuild.mk, LinkTarget.mk
> and
> platform/freebsd.mk to SCons, as well as a module's local gbuild
> files, I
> can now compile files and (badly) link them into libraries, and clean
> the
> build.
>
> SCons is an advanced next-generation build system like gbuild, with
> high
> level declarative syntax in Python, support for multiple modules that
> build
> in parallel, header dependencies, file change detection through MD5
> sums of
> contents instead of timestamps so rebuilds are faster, etc. It builds
> C,
> C++, Objective C, Java, Fortran, D, the sorely necessary Flex and
> Yacc that
> gbuild doesn't support. It supports tons of platforms and compilers,
> including OS/2. It's maintainable and usable, can print debugging
> info such
> as dependency trees, and is generally pleasant to work with. We've
> discussed it before on this list, but I never got to trying it until
> now.
>
> Virtually everything gbuild does, is already done by SCons, and often
> done
> much better. The syntax for SCons files is similar/related to
> gbuild's
> syntax, so an automated conversion from gbuild to SCons might even be
> possible.
>
> So far, I have defines, includes and C[XX]FLAGS working. I've
> structured it
> a bit better, with platform-specific files in classes that only
> provide
> variables to slot in higher up, instead of one giant set of globally
> mutable global variables like in gbuild. Linking still needs a bit of
> work.
> Windows is next on the list, as Cygwin will be the ultimate test of
> whether
> we can use it.
>
> I am very hopeful. SCons has a long history and is used by other
> projects,
> while only OO.o derivatives use gbuild. It's well documented. It
> supports
> features we need which gbuild doesn't, like library version names and
> flex/yacc. It's supposed to work on Windows, both inside and outside
> of
> Cygwin and could help us build without Cygwin some day. It supports
> many
> versions of Visual Studio and could help us in upgrading to a new
> one. We
> will need to add custom builders to SCons for our custom file formats
> like
> IDL, but it's just Python, not my favorite language but infinitely
> better
> than GNU make.
>
> It would help if someone could review my changes, as I am not very
> familiar
> with Python and there might be better ways to write some of the code.
> I'll
> post a patch for review after some further development.
>
> Thank you
> Damjan

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

aoo-scons-patch.txt (29K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: A more sane way to build - SCons, deCygwination and other hopes

Patricia Shanahan
If it turns out to only make things even more complicated, adding yet
another build tool, how hard would it be to remove it from trunk?

On 12/6/2017 6:10 PM, Damjan Jovanovic wrote:
...
> Should we make a separate branch for SCons or develop in trunk? It doesn't
> alter any existing files so it shouldn't interfere with our existing build.

---
This email has been checked for viruses by AVG.
http://www.avg.com


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: A more sane way to build - SCons, deCygwination and other hopes

Damjan Jovanovic
Globally, removing SCons should only entail:
rm -rf site_scons SConstruct */SConscript

Modules that get ported from gbuild to SCons would also require replacement
of their main/<module>/prj/makefile.mk with a gbuild copy of that file
(which is the same in all gbuild modules), and if their gbuild makefiles
were deleted, they need to be restored.

Modules that get ported from dmake to SCons have extensive precompiled
header changes and symbol visibility changes in source code
(SAL_DLLPUBLIC_EXPORT), so "svn revert" is probably the only option for
them.

Be more optimistic though. SCons works better than expected. And I've even
made a preliminary parser/converter for gbuild's module makefiles, that can
convert them into SCons build files, so a fast accurate automated
conversion might be possible 😉, as least for some modules.

On Thu, Dec 7, 2017 at 5:32 AM, Patricia Shanahan <[hidden email]> wrote:

> If it turns out to only make things even more complicated, adding yet
> another build tool, how hard would it be to remove it from trunk?
>
> On 12/6/2017 6:10 PM, Damjan Jovanovic wrote:
> ...
>
>> Should we make a separate branch for SCons or develop in trunk? It doesn't
>> alter any existing files so it shouldn't interfere with our existing
>> build.
>>
>
> ---
> This email has been checked for viruses by AVG.
> http://www.avg.com
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: A more sane way to build - SCons, deCygwination and other hopes

Peter Kovacs-3
In reply to this post by Damjan Jovanovic
Hi Damjan,

Do you have any reason not to use the environment class [2]? I think it
does the same as the Plattformclass you created.
On this overview ocument [1] you see in section 3.1 you see the
architecture of Scons.
I think we should try to use as much preconfigs as possble and minimize
customization as much as possible.

All the best
Peter

[1] http://scons.org/doc/production/PDF/scons-design.pdf
[2]
http://scons.org/doc/production/HTML/scons-user.html#chap-environments

Am 07.12.2017 03:10 schrieb Damjan Jovanovic:

> On Sat, Dec 2, 2017 at 10:42 AM, Peter Kovacs <[hidden email]>
> wrote:
>
>> sounds great from what you write. Lets try SCons build system.
>> Where can I find your changes so I can help? Have you checked them
>> into
>> trunk?
>
> Please find a patch with my current SCons changes attached.
>
> Currently only main/fileaccess builds, on Windows and FreeBSD, and
> doesn't even build completely as I haven't done the "ComponentTarget"
> yet. You run "scons" in main/ or "scons -u" in main/fileaccess. AOO
> has to already be built.
>
>> I have some expreience with python, which will come in handy.
>
> That's great to hear. Please let me know what you think about the
> structure, classes, should we use globals and soenv and SCon's
> Environment, etc. There is no easier time to make changes than at the
> beginning.
>
> Should we make a separate branch for SCons or develop in trunk? It
> doesn't alter any existing files so it shouldn't interfere with our
> existing build.
>
>> Meanwhile I read Scons Documentation.
>>
>> Am Samstag, den 02.12.2017, 10:05 +0200 schrieb Damjan Jovanovic:
>>> Hi
>>>
>>> After days of failing to add a few new simple features to gbuild,
>>> I've now
>>> reached my limits, and have begun experimenting with the SCons
>> build
>>> system
>>> instead.
>>>
>>> It's starting to work. Having ported some of gbuild.mk [1],
>> LinkTarget.mk
>>> and
>>> platform/freebsd.mk [2] to SCons, as well as a module's local
>> gbuild
>>> files, I
>>> can now compile files and (badly) link them into libraries, and
>> clean
>>> the
>>> build.
>>>
>>> SCons is an advanced next-generation build system like gbuild,
>> with
>>> high
>>> level declarative syntax in Python, support for multiple modules
>> that
>>> build
>>> in parallel, header dependencies, file change detection through
>> MD5
>>> sums of
>>> contents instead of timestamps so rebuilds are faster, etc. It
>> builds
>>> C,
>>> C++, Objective C, Java, Fortran, D, the sorely necessary Flex and
>>> Yacc that
>>> gbuild doesn't support. It supports tons of platforms and
>> compilers,
>>> including OS/2. It's maintainable and usable, can print debugging
>>> info such
>>> as dependency trees, and is generally pleasant to work with.
>> We've
>>> discussed it before on this list, but I never got to trying it
>> until
>>> now.
>>>
>>> Virtually everything gbuild does, is already done by SCons, and
>> often
>>> done
>>> much better. The syntax for SCons files is similar/related to
>>> gbuild's
>>> syntax, so an automated conversion from gbuild to SCons might
>> even be
>>> possible.
>>>
>>> So far, I have defines, includes and C[XX]FLAGS working. I've
>>> structured it
>>> a bit better, with platform-specific files in classes that only
>>> provide
>>> variables to slot in higher up, instead of one giant set of
>> globally
>>> mutable global variables like in gbuild. Linking still needs a
>> bit of
>>> work.
>>> Windows is next on the list, as Cygwin will be the ultimate test
>> of
>>> whether
>>> we can use it.
>>>
>>> I am very hopeful. SCons has a long history and is used by other
>>> projects,
>>> while only OO.o derivatives use gbuild. It's well documented. It
>>> supports
>>> features we need which gbuild doesn't, like library version names
>> and
>>> flex/yacc. It's supposed to work on Windows, both inside and
>> outside
>>> of
>>> Cygwin and could help us build without Cygwin some day. It
>> supports
>>> many
>>> versions of Visual Studio and could help us in upgrading to a new
>>> one. We
>>> will need to add custom builders to SCons for our custom file
>> formats
>>> like
>>> IDL, but it's just Python, not my favorite language but
>> infinitely
>>> better
>>> than GNU make.
>>>
>>> It would help if someone could review my changes, as I am not
>> very
>>> familiar
>>> with Python and there might be better ways to write some of the
>> code.
>>> I'll
>>> post a patch for review after some further development.
>>>
>>> Thank you
>>> Damjan
>>
>>
> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>
>
>
> Links:
> ------
> [1] http://gbuild.mk
> [2] http://freebsd.mk
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: A more sane way to build - SCons, deCygwination and other hopes

Pedro Giffuni
In reply to this post by Damjan Jovanovic
Hello everyone!

Just my $0.02

> Hi
>
> After days of failing to add a few new simple features to gbuild, I've now
> reached my limits, and have begun experimenting with the SCons build system
> instead.
>
> It's starting to work. Having ported some of gbuild.mk, LinkTarget.mk and
> platform/freebsd.mk to SCons, as well as a module's local gbuild files, I
> can now compile files and (badly) link them into libraries, and clean the
> build.

I am fine with bringing yet another build system, specially since we
need this one to update Apache serf.

Of course the objective should still be to remove dmake, so my request
to Damjan, would be to move python ASAP to  gbuild. I can understand us
needing gbuild and Scons but we should be able to build SCons, and any
other build system, without dmake.

Regards,

Pedro.

Reply | Threaded
Open this post in threaded view
|

Re: A more sane way to build - SCons, deCygwination and other hopes

Peter Kovacs-3
Hi Pedro,


On 08.12.2017 15:06, Pedro Giffuni wrote:

> Hello everyone!
>
> Just my $0.02
>
>> Hi
>>
>> After days of failing to add a few new simple features to gbuild,
>> I've now
>> reached my limits, and have begun experimenting with the SCons build
>> system
>> instead.
>>
>> It's starting to work. Having ported some of gbuild.mk, LinkTarget.mk
>> and
>> platform/freebsd.mk to SCons, as well as a module's local gbuild
>> files, I
>> can now compile files and (badly) link them into libraries, and clean
>> the
>> build.
>
> I am fine with bringing yet another build system, specially since we
> need this one to update Apache serf.
>
> Of course the objective should still be to remove dmake, so my request
> to Damjan, would be to move python ASAP to  gbuild. I can understand
> us needing gbuild and Scons but we should be able to build SCons, and
> any other build system, without dmake.
The objective should be to move complete. My dream would be to make the
transformation automatically. For that we should focus to have the
tools, and then parse the information we need from the files we have.
It will take some time, thought till it realy works and we fixed all
bugs from the transformation.

My 2 cents how to do this.-

All the best
Peter

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]