The new SDBC-JDBC bridge in Java

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

The new SDBC-JDBC bridge in Java

Damjan Jovanovic
Hi

As of revision 1814552, I've rewritten our formerly C++ based SDBC-JDBC
bridge driver in Java. The SDBC-JDBC bridge is there to allows AOO to
access databases through Java's JDBC drivers.

This port originally aimed at fixing serious issues I discovered in the old
C++ implementation, where there is apparently memory corruption on method
IDs that causes methods to be called on wrong Java objects. In the process
of porting I've also discovered and fixed many other problems. For example:
* The C++ code was calling Java code through the Java Invocation API, so it
was using strings to look up method names. This defeats type checking, and
in some cases was looking for methods with typos in their names or that
don't even exist. My current bridge only uses Java to Java calls that are
always strongly type checked and correct.
* Conversions between bytes and chars were broken: n bytes were being
copied verbatim into memory for char arrays n chars long in Java (Java's
char is 16 bits) so the second half of the array was untouched and the
first half had wrong chars, and non-existent classes like
java.io.CharArrayInputStream were being created. Real classes that do
exist, and proper character set conversions, are now used.
* Chained Java exceptions are now converted to chained UNO exceptions via
the commonly used and clearer getCause() chaining dimension, not the
java.sql.SQLException's getNextException() chaining dimension.

The new driver is, as seen externally, completely interchangeable with the
old one. Service names are identical, implementation names are identical,
supported interfaces are identical, the configuration schema is identical,
initialization arguments are identical, even logging channels and messages
logged are identical - client code using the old driver should be able to
use the new one transparently, and not only should everything that worked
before still work now, but some things that didn't work before might also
begin to work now.

Having said that, the driver does push UNO to its limits, and might reveal
other bugs in UNO bridging, Base, other AOO components, and the C++ runtime
environment. I already discovered AOO sometimes crashes on FreeBSD due to
issues in converting exceptions from Java to C++ (missing exception RTTI
causes a segfault), but such bugs should be fixed in main/bridges and/or
the Clang project.

There are still some cleanups I want to do, like making sure nulls strings
are never passed or returned to UNO, cleaning up exception handling, doing
a comparison of every C++ method with every Java method to see if I missed
anything, and so on, which is why the old C++ code has been left in the
tree for now even though it's no longer built. In the meanwhile, please
feel free to start testing :).

Oh and my original memory-corruption-induced IllegalClassChangeError is
gone - my PostgreSQL SDBC driver already works better with the new
SDBC-JDBC bridge :).

Damjan
Reply | Threaded
Open this post in threaded view
|

Re: The new SDBC-JDBC bridge in Java

Matthias Seidel
Hi Damjan,

On my release machine (Win 10) it stops with:

---
make: *** [C:/Source/aoo/main/solenv/gbuild/UnoApiTarget.mk:165:
/cygdrive/c/Source/aoo/main/solver/420/wntmsci12.pro/workdir/UnoApiPartTarget/offapi/com/sun/star/sdb/ParameterSubstitution.urd]
Error 1
make: *** Waiting for unfinished jobs....
dmake:  Error code 2, while making 'all'

1 module(s):
        offapi
need(s) to be rebuilt

Reason(s):

ERROR: error 65280 occurred while making
/cygdrive/c/Source/aoo/main/offapi/prj

When you have fixed the errors in that module you can resume the build
by running:

        build --from offapi
---

I will try to force a build on our buildbot to cross check.

Regards, Matthias


Am 08.11.2017 um 05:34 schrieb Damjan Jovanovic:

> Hi
>
> As of revision 1814552, I've rewritten our formerly C++ based SDBC-JDBC
> bridge driver in Java. The SDBC-JDBC bridge is there to allows AOO to
> access databases through Java's JDBC drivers.
>
> This port originally aimed at fixing serious issues I discovered in the old
> C++ implementation, where there is apparently memory corruption on method
> IDs that causes methods to be called on wrong Java objects. In the process
> of porting I've also discovered and fixed many other problems. For example:
> * The C++ code was calling Java code through the Java Invocation API, so it
> was using strings to look up method names. This defeats type checking, and
> in some cases was looking for methods with typos in their names or that
> don't even exist. My current bridge only uses Java to Java calls that are
> always strongly type checked and correct.
> * Conversions between bytes and chars were broken: n bytes were being
> copied verbatim into memory for char arrays n chars long in Java (Java's
> char is 16 bits) so the second half of the array was untouched and the
> first half had wrong chars, and non-existent classes like
> java.io.CharArrayInputStream were being created. Real classes that do
> exist, and proper character set conversions, are now used.
> * Chained Java exceptions are now converted to chained UNO exceptions via
> the commonly used and clearer getCause() chaining dimension, not the
> java.sql.SQLException's getNextException() chaining dimension.
>
> The new driver is, as seen externally, completely interchangeable with the
> old one. Service names are identical, implementation names are identical,
> supported interfaces are identical, the configuration schema is identical,
> initialization arguments are identical, even logging channels and messages
> logged are identical - client code using the old driver should be able to
> use the new one transparently, and not only should everything that worked
> before still work now, but some things that didn't work before might also
> begin to work now.
>
> Having said that, the driver does push UNO to its limits, and might reveal
> other bugs in UNO bridging, Base, other AOO components, and the C++ runtime
> environment. I already discovered AOO sometimes crashes on FreeBSD due to
> issues in converting exceptions from Java to C++ (missing exception RTTI
> causes a segfault), but such bugs should be fixed in main/bridges and/or
> the Clang project.
>
> There are still some cleanups I want to do, like making sure nulls strings
> are never passed or returned to UNO, cleaning up exception handling, doing
> a comparison of every C++ method with every Java method to see if I missed
> anything, and so on, which is why the old C++ code has been left in the
> tree for now even though it's no longer built. In the meanwhile, please
> feel free to start testing :).
>
> Oh and my original memory-corruption-induced IllegalClassChangeError is
> gone - my PostgreSQL SDBC driver already works better with the new
> SDBC-JDBC bridge :).
>
> Damjan
>


smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: The new SDBC-JDBC bridge in Java

Damjan Jovanovic
Hi Matthias

Was it a clean rebuild?

Can you post the full output of "build --from offapi"?

Maybe it needs DOS line endings in ParameterSubstitution.idl?

Damjan

On Wed, Nov 8, 2017 at 2:52 PM, Matthias Seidel <[hidden email]>
wrote:

> Hi Damjan,
>
> On my release machine (Win 10) it stops with:
>
> ---
> make: *** [C:/Source/aoo/main/solenv/gbuild/UnoApiTarget.mk:165:
> /cygdrive/c/Source/aoo/main/solver/420/wntmsci12.pro/
> workdir/UnoApiPartTarget/offapi/com/sun/star/sdb/ParameterSubstitution.urd
> ]
> Error 1
> make: *** Waiting for unfinished jobs....
> dmake:  Error code 2, while making 'all'
>
> 1 module(s):
>         offapi
> need(s) to be rebuilt
>
> Reason(s):
>
> ERROR: error 65280 occurred while making
> /cygdrive/c/Source/aoo/main/offapi/prj
>
> When you have fixed the errors in that module you can resume the build
> by running:
>
>         build --from offapi
> ---
>
> I will try to force a build on our buildbot to cross check.
>
> Regards, Matthias
>
>
> Am 08.11.2017 um 05:34 schrieb Damjan Jovanovic:
> > Hi
> >
> > As of revision 1814552, I've rewritten our formerly C++ based SDBC-JDBC
> > bridge driver in Java. The SDBC-JDBC bridge is there to allows AOO to
> > access databases through Java's JDBC drivers.
> >
> > This port originally aimed at fixing serious issues I discovered in the
> old
> > C++ implementation, where there is apparently memory corruption on method
> > IDs that causes methods to be called on wrong Java objects. In the
> process
> > of porting I've also discovered and fixed many other problems. For
> example:
> > * The C++ code was calling Java code through the Java Invocation API, so
> it
> > was using strings to look up method names. This defeats type checking,
> and
> > in some cases was looking for methods with typos in their names or that
> > don't even exist. My current bridge only uses Java to Java calls that are
> > always strongly type checked and correct.
> > * Conversions between bytes and chars were broken: n bytes were being
> > copied verbatim into memory for char arrays n chars long in Java (Java's
> > char is 16 bits) so the second half of the array was untouched and the
> > first half had wrong chars, and non-existent classes like
> > java.io.CharArrayInputStream were being created. Real classes that do
> > exist, and proper character set conversions, are now used.
> > * Chained Java exceptions are now converted to chained UNO exceptions via
> > the commonly used and clearer getCause() chaining dimension, not the
> > java.sql.SQLException's getNextException() chaining dimension.
> >
> > The new driver is, as seen externally, completely interchangeable with
> the
> > old one. Service names are identical, implementation names are identical,
> > supported interfaces are identical, the configuration schema is
> identical,
> > initialization arguments are identical, even logging channels and
> messages
> > logged are identical - client code using the old driver should be able to
> > use the new one transparently, and not only should everything that worked
> > before still work now, but some things that didn't work before might also
> > begin to work now.
> >
> > Having said that, the driver does push UNO to its limits, and might
> reveal
> > other bugs in UNO bridging, Base, other AOO components, and the C++
> runtime
> > environment. I already discovered AOO sometimes crashes on FreeBSD due to
> > issues in converting exceptions from Java to C++ (missing exception RTTI
> > causes a segfault), but such bugs should be fixed in main/bridges and/or
> > the Clang project.
> >
> > There are still some cleanups I want to do, like making sure nulls
> strings
> > are never passed or returned to UNO, cleaning up exception handling,
> doing
> > a comparison of every C++ method with every Java method to see if I
> missed
> > anything, and so on, which is why the old C++ code has been left in the
> > tree for now even though it's no longer built. In the meanwhile, please
> > feel free to start testing :).
> >
> > Oh and my original memory-corruption-induced IllegalClassChangeError is
> > gone - my PostgreSQL SDBC driver already works better with the new
> > SDBC-JDBC bridge :).
> >
> > Damjan
> >
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: The new SDBC-JDBC bridge in Java

Matthias Seidel
Hi Damjan,

Am 08.11.2017 um 14:36 schrieb Damjan Jovanovic:
> Hi Matthias
>
> Was it a clean rebuild?

Yes, a new build after "dmake clean". I can additionally try a complete
checkout.
At the moment I am not able to force a build from our aoo-win7 buildbot.

>
> Can you post the full output of "build --from offapi"?

$ build --from offapi
build -- version: 1775979


=============
Building module offapi
=============

Entering /cygdrive/c/Source/aoo/main/offapi/prj

cd .. && make -s -r -j1   && make -s -r deliverlog
[ build IDL ] offapi/com/sun/star/sdb/ParameterSubstitution.urd
make: *** [C:/Source/aoo/main/solenv/gbuild/UnoApiTarget.mk:165:
/cygdrive/c/Source/aoo/main/solver/420/wntmsci12.pro/workdir/UnoApiPartTarget/offapi/com/sun/star/sdb/ParameterSubstitution.urd]
Error 1
dmake:  Error code 2, while making 'all'

1 module(s):
        offapi
need(s) to be rebuilt

Reason(s):

ERROR: error 65280 occurred while making
/cygdrive/c/Source/aoo/main/offapi/prj

When you have fixed the errors in that module you can resume the build
by running:

        build --from offapi

>
> Maybe it needs DOS line endings in ParameterSubstitution.idl?

Sounds familiar, we once had a problem with line endings. Built only on
older versions of cygwin.

Regards, Matthias

>
> Damjan
>
> On Wed, Nov 8, 2017 at 2:52 PM, Matthias Seidel <[hidden email]>
> wrote:
>
>> Hi Damjan,
>>
>> On my release machine (Win 10) it stops with:
>>
>> ---
>> make: *** [C:/Source/aoo/main/solenv/gbuild/UnoApiTarget.mk:165:
>> /cygdrive/c/Source/aoo/main/solver/420/wntmsci12.pro/
>> workdir/UnoApiPartTarget/offapi/com/sun/star/sdb/ParameterSubstitution.urd
>> ]
>> Error 1
>> make: *** Waiting for unfinished jobs....
>> dmake:  Error code 2, while making 'all'
>>
>> 1 module(s):
>>         offapi
>> need(s) to be rebuilt
>>
>> Reason(s):
>>
>> ERROR: error 65280 occurred while making
>> /cygdrive/c/Source/aoo/main/offapi/prj
>>
>> When you have fixed the errors in that module you can resume the build
>> by running:
>>
>>         build --from offapi
>> ---
>>
>> I will try to force a build on our buildbot to cross check.
>>
>> Regards, Matthias
>>
>>
>> Am 08.11.2017 um 05:34 schrieb Damjan Jovanovic:
>>> Hi
>>>
>>> As of revision 1814552, I've rewritten our formerly C++ based SDBC-JDBC
>>> bridge driver in Java. The SDBC-JDBC bridge is there to allows AOO to
>>> access databases through Java's JDBC drivers.
>>>
>>> This port originally aimed at fixing serious issues I discovered in the
>> old
>>> C++ implementation, where there is apparently memory corruption on method
>>> IDs that causes methods to be called on wrong Java objects. In the
>> process
>>> of porting I've also discovered and fixed many other problems. For
>> example:
>>> * The C++ code was calling Java code through the Java Invocation API, so
>> it
>>> was using strings to look up method names. This defeats type checking,
>> and
>>> in some cases was looking for methods with typos in their names or that
>>> don't even exist. My current bridge only uses Java to Java calls that are
>>> always strongly type checked and correct.
>>> * Conversions between bytes and chars were broken: n bytes were being
>>> copied verbatim into memory for char arrays n chars long in Java (Java's
>>> char is 16 bits) so the second half of the array was untouched and the
>>> first half had wrong chars, and non-existent classes like
>>> java.io.CharArrayInputStream were being created. Real classes that do
>>> exist, and proper character set conversions, are now used.
>>> * Chained Java exceptions are now converted to chained UNO exceptions via
>>> the commonly used and clearer getCause() chaining dimension, not the
>>> java.sql.SQLException's getNextException() chaining dimension.
>>>
>>> The new driver is, as seen externally, completely interchangeable with
>> the
>>> old one. Service names are identical, implementation names are identical,
>>> supported interfaces are identical, the configuration schema is
>> identical,
>>> initialization arguments are identical, even logging channels and
>> messages
>>> logged are identical - client code using the old driver should be able to
>>> use the new one transparently, and not only should everything that worked
>>> before still work now, but some things that didn't work before might also
>>> begin to work now.
>>>
>>> Having said that, the driver does push UNO to its limits, and might
>> reveal
>>> other bugs in UNO bridging, Base, other AOO components, and the C++
>> runtime
>>> environment. I already discovered AOO sometimes crashes on FreeBSD due to
>>> issues in converting exceptions from Java to C++ (missing exception RTTI
>>> causes a segfault), but such bugs should be fixed in main/bridges and/or
>>> the Clang project.
>>>
>>> There are still some cleanups I want to do, like making sure nulls
>> strings
>>> are never passed or returned to UNO, cleaning up exception handling,
>> doing
>>> a comparison of every C++ method with every Java method to see if I
>> missed
>>> anything, and so on, which is why the old C++ code has been left in the
>>> tree for now even though it's no longer built. In the meanwhile, please
>>> feel free to start testing :).
>>>
>>> Oh and my original memory-corruption-induced IllegalClassChangeError is
>>> gone - my PostgreSQL SDBC driver already works better with the new
>>> SDBC-JDBC bridge :).
>>>
>>> Damjan
>>>
>>
>>


smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: The new SDBC-JDBC bridge in Java

Matthias Seidel
In reply to this post by Damjan Jovanovic
Hi Damjan,

I was able to force the Windows buildbot and this is the detailed error log:

https://ci.apache.org/projects/openoffice/buildlogs/win/main/offapi/wntmsci12.pro/misc/logs/prj.txt

Regards, Matthias


Am 08.11.2017 um 14:36 schrieb Damjan Jovanovic:

> Hi Matthias
>
> Was it a clean rebuild?
>
> Can you post the full output of "build --from offapi"?
>
> Maybe it needs DOS line endings in ParameterSubstitution.idl?
>
> Damjan
>
> On Wed, Nov 8, 2017 at 2:52 PM, Matthias Seidel <[hidden email]>
> wrote:
>
>> Hi Damjan,
>>
>> On my release machine (Win 10) it stops with:
>>
>> ---
>> make: *** [C:/Source/aoo/main/solenv/gbuild/UnoApiTarget.mk:165:
>> /cygdrive/c/Source/aoo/main/solver/420/wntmsci12.pro/
>> workdir/UnoApiPartTarget/offapi/com/sun/star/sdb/ParameterSubstitution.urd
>> ]
>> Error 1
>> make: *** Waiting for unfinished jobs....
>> dmake:  Error code 2, while making 'all'
>>
>> 1 module(s):
>>         offapi
>> need(s) to be rebuilt
>>
>> Reason(s):
>>
>> ERROR: error 65280 occurred while making
>> /cygdrive/c/Source/aoo/main/offapi/prj
>>
>> When you have fixed the errors in that module you can resume the build
>> by running:
>>
>>         build --from offapi
>> ---
>>
>> I will try to force a build on our buildbot to cross check.
>>
>> Regards, Matthias
>>
>>
>> Am 08.11.2017 um 05:34 schrieb Damjan Jovanovic:
>>> Hi
>>>
>>> As of revision 1814552, I've rewritten our formerly C++ based SDBC-JDBC
>>> bridge driver in Java. The SDBC-JDBC bridge is there to allows AOO to
>>> access databases through Java's JDBC drivers.
>>>
>>> This port originally aimed at fixing serious issues I discovered in the
>> old
>>> C++ implementation, where there is apparently memory corruption on method
>>> IDs that causes methods to be called on wrong Java objects. In the
>> process
>>> of porting I've also discovered and fixed many other problems. For
>> example:
>>> * The C++ code was calling Java code through the Java Invocation API, so
>> it
>>> was using strings to look up method names. This defeats type checking,
>> and
>>> in some cases was looking for methods with typos in their names or that
>>> don't even exist. My current bridge only uses Java to Java calls that are
>>> always strongly type checked and correct.
>>> * Conversions between bytes and chars were broken: n bytes were being
>>> copied verbatim into memory for char arrays n chars long in Java (Java's
>>> char is 16 bits) so the second half of the array was untouched and the
>>> first half had wrong chars, and non-existent classes like
>>> java.io.CharArrayInputStream were being created. Real classes that do
>>> exist, and proper character set conversions, are now used.
>>> * Chained Java exceptions are now converted to chained UNO exceptions via
>>> the commonly used and clearer getCause() chaining dimension, not the
>>> java.sql.SQLException's getNextException() chaining dimension.
>>>
>>> The new driver is, as seen externally, completely interchangeable with
>> the
>>> old one. Service names are identical, implementation names are identical,
>>> supported interfaces are identical, the configuration schema is
>> identical,
>>> initialization arguments are identical, even logging channels and
>> messages
>>> logged are identical - client code using the old driver should be able to
>>> use the new one transparently, and not only should everything that worked
>>> before still work now, but some things that didn't work before might also
>>> begin to work now.
>>>
>>> Having said that, the driver does push UNO to its limits, and might
>> reveal
>>> other bugs in UNO bridging, Base, other AOO components, and the C++
>> runtime
>>> environment. I already discovered AOO sometimes crashes on FreeBSD due to
>>> issues in converting exceptions from Java to C++ (missing exception RTTI
>>> causes a segfault), but such bugs should be fixed in main/bridges and/or
>>> the Clang project.
>>>
>>> There are still some cleanups I want to do, like making sure nulls
>> strings
>>> are never passed or returned to UNO, cleaning up exception handling,
>> doing
>>> a comparison of every C++ method with every Java method to see if I
>> missed
>>> anything, and so on, which is why the old C++ code has been left in the
>>> tree for now even though it's no longer built. In the meanwhile, please
>>> feel free to start testing :).
>>>
>>> Oh and my original memory-corruption-induced IllegalClassChangeError is
>>> gone - my PostgreSQL SDBC driver already works better with the new
>>> SDBC-JDBC bridge :).
>>>
>>> Damjan
>>>
>>
>>


smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: The new SDBC-JDBC bridge in Java

Damjan Jovanovic
Thank you. There is nothing in that log but "Error 1".

It builds perfectly on FreeBSD. I'll try a Windows build too but it could
take a while.

The file has *nix style line endings like every other IDL file.

Any other ideas?

Regards
Damjan


On Wed, Nov 8, 2017 at 5:50 PM, Matthias Seidel <[hidden email]>
wrote:

> Hi Damjan,
>
> I was able to force the Windows buildbot and this is the detailed error
> log:
>
> https://ci.apache.org/projects/openoffice/buildlogs/
> win/main/offapi/wntmsci12.pro/misc/logs/prj.txt
>
> Regards, Matthias
>
>
> Am 08.11.2017 um 14:36 schrieb Damjan Jovanovic:
> > Hi Matthias
> >
> > Was it a clean rebuild?
> >
> > Can you post the full output of "build --from offapi"?
> >
> > Maybe it needs DOS line endings in ParameterSubstitution.idl?
> >
> > Damjan
> >
> > On Wed, Nov 8, 2017 at 2:52 PM, Matthias Seidel <
> [hidden email]>
> > wrote:
> >
> >> Hi Damjan,
> >>
> >> On my release machine (Win 10) it stops with:
> >>
> >> ---
> >> make: *** [C:/Source/aoo/main/solenv/gbuild/UnoApiTarget.mk:165:
> >> /cygdrive/c/Source/aoo/main/solver/420/wntmsci12.pro/
> >> workdir/UnoApiPartTarget/offapi/com/sun/star/sdb/
> ParameterSubstitution.urd
> >> ]
> >> Error 1
> >> make: *** Waiting for unfinished jobs....
> >> dmake:  Error code 2, while making 'all'
> >>
> >> 1 module(s):
> >>         offapi
> >> need(s) to be rebuilt
> >>
> >> Reason(s):
> >>
> >> ERROR: error 65280 occurred while making
> >> /cygdrive/c/Source/aoo/main/offapi/prj
> >>
> >> When you have fixed the errors in that module you can resume the build
> >> by running:
> >>
> >>         build --from offapi
> >> ---
> >>
> >> I will try to force a build on our buildbot to cross check.
> >>
> >> Regards, Matthias
> >>
> >>
> >> Am 08.11.2017 um 05:34 schrieb Damjan Jovanovic:
> >>> Hi
> >>>
> >>> As of revision 1814552, I've rewritten our formerly C++ based SDBC-JDBC
> >>> bridge driver in Java. The SDBC-JDBC bridge is there to allows AOO to
> >>> access databases through Java's JDBC drivers.
> >>>
> >>> This port originally aimed at fixing serious issues I discovered in the
> >> old
> >>> C++ implementation, where there is apparently memory corruption on
> method
> >>> IDs that causes methods to be called on wrong Java objects. In the
> >> process
> >>> of porting I've also discovered and fixed many other problems. For
> >> example:
> >>> * The C++ code was calling Java code through the Java Invocation API,
> so
> >> it
> >>> was using strings to look up method names. This defeats type checking,
> >> and
> >>> in some cases was looking for methods with typos in their names or that
> >>> don't even exist. My current bridge only uses Java to Java calls that
> are
> >>> always strongly type checked and correct.
> >>> * Conversions between bytes and chars were broken: n bytes were being
> >>> copied verbatim into memory for char arrays n chars long in Java
> (Java's
> >>> char is 16 bits) so the second half of the array was untouched and the
> >>> first half had wrong chars, and non-existent classes like
> >>> java.io.CharArrayInputStream were being created. Real classes that do
> >>> exist, and proper character set conversions, are now used.
> >>> * Chained Java exceptions are now converted to chained UNO exceptions
> via
> >>> the commonly used and clearer getCause() chaining dimension, not the
> >>> java.sql.SQLException's getNextException() chaining dimension.
> >>>
> >>> The new driver is, as seen externally, completely interchangeable with
> >> the
> >>> old one. Service names are identical, implementation names are
> identical,
> >>> supported interfaces are identical, the configuration schema is
> >> identical,
> >>> initialization arguments are identical, even logging channels and
> >> messages
> >>> logged are identical - client code using the old driver should be able
> to
> >>> use the new one transparently, and not only should everything that
> worked
> >>> before still work now, but some things that didn't work before might
> also
> >>> begin to work now.
> >>>
> >>> Having said that, the driver does push UNO to its limits, and might
> >> reveal
> >>> other bugs in UNO bridging, Base, other AOO components, and the C++
> >> runtime
> >>> environment. I already discovered AOO sometimes crashes on FreeBSD due
> to
> >>> issues in converting exceptions from Java to C++ (missing exception
> RTTI
> >>> causes a segfault), but such bugs should be fixed in main/bridges
> and/or
> >>> the Clang project.
> >>>
> >>> There are still some cleanups I want to do, like making sure nulls
> >> strings
> >>> are never passed or returned to UNO, cleaning up exception handling,
> >> doing
> >>> a comparison of every C++ method with every Java method to see if I
> >> missed
> >>> anything, and so on, which is why the old C++ code has been left in the
> >>> tree for now even though it's no longer built. In the meanwhile, please
> >>> feel free to start testing :).
> >>>
> >>> Oh and my original memory-corruption-induced IllegalClassChangeError is
> >>> gone - my PostgreSQL SDBC driver already works better with the new
> >>> SDBC-JDBC bridge :).
> >>>
> >>> Damjan
> >>>
> >>
> >>
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: The new SDBC-JDBC bridge in Java

Damjan Jovanovic
Apparently I left out that IDL file in my last commit. Sorry. The latest
SVN should build now.

On Wed, Nov 8, 2017 at 6:41 PM, Damjan Jovanovic <[hidden email]> wrote:

> Thank you. There is nothing in that log but "Error 1".
>
> It builds perfectly on FreeBSD. I'll try a Windows build too but it could
> take a while.
>
> The file has *nix style line endings like every other IDL file.
>
> Any other ideas?
>
> Regards
> Damjan
>
>
> On Wed, Nov 8, 2017 at 5:50 PM, Matthias Seidel <
> [hidden email]> wrote:
>
>> Hi Damjan,
>>
>> I was able to force the Windows buildbot and this is the detailed error
>> log:
>>
>> https://ci.apache.org/projects/openoffice/buildlogs/win/
>> main/offapi/wntmsci12.pro/misc/logs/prj.txt
>>
>> Regards, Matthias
>>
>>
>> Am 08.11.2017 um 14:36 schrieb Damjan Jovanovic:
>> > Hi Matthias
>> >
>> > Was it a clean rebuild?
>> >
>> > Can you post the full output of "build --from offapi"?
>> >
>> > Maybe it needs DOS line endings in ParameterSubstitution.idl?
>> >
>> > Damjan
>> >
>> > On Wed, Nov 8, 2017 at 2:52 PM, Matthias Seidel <
>> [hidden email]>
>> > wrote:
>> >
>> >> Hi Damjan,
>> >>
>> >> On my release machine (Win 10) it stops with:
>> >>
>> >> ---
>> >> make: *** [C:/Source/aoo/main/solenv/gbuild/UnoApiTarget.mk:165:
>> >> /cygdrive/c/Source/aoo/main/solver/420/wntmsci12.pro/
>> >> workdir/UnoApiPartTarget/offapi/com/sun/star/sdb/ParameterSu
>> bstitution.urd
>> >> ]
>> >> Error 1
>> >> make: *** Waiting for unfinished jobs....
>> >> dmake:  Error code 2, while making 'all'
>> >>
>> >> 1 module(s):
>> >>         offapi
>> >> need(s) to be rebuilt
>> >>
>> >> Reason(s):
>> >>
>> >> ERROR: error 65280 occurred while making
>> >> /cygdrive/c/Source/aoo/main/offapi/prj
>> >>
>> >> When you have fixed the errors in that module you can resume the build
>> >> by running:
>> >>
>> >>         build --from offapi
>> >> ---
>> >>
>> >> I will try to force a build on our buildbot to cross check.
>> >>
>> >> Regards, Matthias
>> >>
>> >>
>> >> Am 08.11.2017 um 05:34 schrieb Damjan Jovanovic:
>> >>> Hi
>> >>>
>> >>> As of revision 1814552, I've rewritten our formerly C++ based
>> SDBC-JDBC
>> >>> bridge driver in Java. The SDBC-JDBC bridge is there to allows AOO to
>> >>> access databases through Java's JDBC drivers.
>> >>>
>> >>> This port originally aimed at fixing serious issues I discovered in
>> the
>> >> old
>> >>> C++ implementation, where there is apparently memory corruption on
>> method
>> >>> IDs that causes methods to be called on wrong Java objects. In the
>> >> process
>> >>> of porting I've also discovered and fixed many other problems. For
>> >> example:
>> >>> * The C++ code was calling Java code through the Java Invocation API,
>> so
>> >> it
>> >>> was using strings to look up method names. This defeats type checking,
>> >> and
>> >>> in some cases was looking for methods with typos in their names or
>> that
>> >>> don't even exist. My current bridge only uses Java to Java calls that
>> are
>> >>> always strongly type checked and correct.
>> >>> * Conversions between bytes and chars were broken: n bytes were being
>> >>> copied verbatim into memory for char arrays n chars long in Java
>> (Java's
>> >>> char is 16 bits) so the second half of the array was untouched and the
>> >>> first half had wrong chars, and non-existent classes like
>> >>> java.io.CharArrayInputStream were being created. Real classes that do
>> >>> exist, and proper character set conversions, are now used.
>> >>> * Chained Java exceptions are now converted to chained UNO exceptions
>> via
>> >>> the commonly used and clearer getCause() chaining dimension, not the
>> >>> java.sql.SQLException's getNextException() chaining dimension.
>> >>>
>> >>> The new driver is, as seen externally, completely interchangeable with
>> >> the
>> >>> old one. Service names are identical, implementation names are
>> identical,
>> >>> supported interfaces are identical, the configuration schema is
>> >> identical,
>> >>> initialization arguments are identical, even logging channels and
>> >> messages
>> >>> logged are identical - client code using the old driver should be
>> able to
>> >>> use the new one transparently, and not only should everything that
>> worked
>> >>> before still work now, but some things that didn't work before might
>> also
>> >>> begin to work now.
>> >>>
>> >>> Having said that, the driver does push UNO to its limits, and might
>> >> reveal
>> >>> other bugs in UNO bridging, Base, other AOO components, and the C++
>> >> runtime
>> >>> environment. I already discovered AOO sometimes crashes on FreeBSD
>> due to
>> >>> issues in converting exceptions from Java to C++ (missing exception
>> RTTI
>> >>> causes a segfault), but such bugs should be fixed in main/bridges
>> and/or
>> >>> the Clang project.
>> >>>
>> >>> There are still some cleanups I want to do, like making sure nulls
>> >> strings
>> >>> are never passed or returned to UNO, cleaning up exception handling,
>> >> doing
>> >>> a comparison of every C++ method with every Java method to see if I
>> >> missed
>> >>> anything, and so on, which is why the old C++ code has been left in
>> the
>> >>> tree for now even though it's no longer built. In the meanwhile,
>> please
>> >>> feel free to start testing :).
>> >>>
>> >>> Oh and my original memory-corruption-induced IllegalClassChangeError
>> is
>> >>> gone - my PostgreSQL SDBC driver already works better with the new
>> >>> SDBC-JDBC bridge :).
>> >>>
>> >>> Damjan
>> >>>
>> >>
>> >>
>>
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: The new SDBC-JDBC bridge in Java

Matthias Seidel
Am 08.11.2017 um 17:48 schrieb Damjan Jovanovic:
> Apparently I left out that IDL file in my last commit. Sorry. The latest
> SVN should build now.

No problem, I will start a new build immediately!

Thanks!

>
> On Wed, Nov 8, 2017 at 6:41 PM, Damjan Jovanovic <[hidden email]> wrote:
>
>> Thank you. There is nothing in that log but "Error 1".
>>
>> It builds perfectly on FreeBSD. I'll try a Windows build too but it could
>> take a while.
>>
>> The file has *nix style line endings like every other IDL file.
>>
>> Any other ideas?
>>
>> Regards
>> Damjan
>>
>>
>> On Wed, Nov 8, 2017 at 5:50 PM, Matthias Seidel <
>> [hidden email]> wrote:
>>
>>> Hi Damjan,
>>>
>>> I was able to force the Windows buildbot and this is the detailed error
>>> log:
>>>
>>> https://ci.apache.org/projects/openoffice/buildlogs/win/
>>> main/offapi/wntmsci12.pro/misc/logs/prj.txt
>>>
>>> Regards, Matthias
>>>
>>>
>>> Am 08.11.2017 um 14:36 schrieb Damjan Jovanovic:
>>>> Hi Matthias
>>>>
>>>> Was it a clean rebuild?
>>>>
>>>> Can you post the full output of "build --from offapi"?
>>>>
>>>> Maybe it needs DOS line endings in ParameterSubstitution.idl?
>>>>
>>>> Damjan
>>>>
>>>> On Wed, Nov 8, 2017 at 2:52 PM, Matthias Seidel <
>>> [hidden email]>
>>>> wrote:
>>>>
>>>>> Hi Damjan,
>>>>>
>>>>> On my release machine (Win 10) it stops with:
>>>>>
>>>>> ---
>>>>> make: *** [C:/Source/aoo/main/solenv/gbuild/UnoApiTarget.mk:165:
>>>>> /cygdrive/c/Source/aoo/main/solver/420/wntmsci12.pro/
>>>>> workdir/UnoApiPartTarget/offapi/com/sun/star/sdb/ParameterSu
>>> bstitution.urd
>>>>> ]
>>>>> Error 1
>>>>> make: *** Waiting for unfinished jobs....
>>>>> dmake:  Error code 2, while making 'all'
>>>>>
>>>>> 1 module(s):
>>>>>         offapi
>>>>> need(s) to be rebuilt
>>>>>
>>>>> Reason(s):
>>>>>
>>>>> ERROR: error 65280 occurred while making
>>>>> /cygdrive/c/Source/aoo/main/offapi/prj
>>>>>
>>>>> When you have fixed the errors in that module you can resume the build
>>>>> by running:
>>>>>
>>>>>         build --from offapi
>>>>> ---
>>>>>
>>>>> I will try to force a build on our buildbot to cross check.
>>>>>
>>>>> Regards, Matthias
>>>>>
>>>>>
>>>>> Am 08.11.2017 um 05:34 schrieb Damjan Jovanovic:
>>>>>> Hi
>>>>>>
>>>>>> As of revision 1814552, I've rewritten our formerly C++ based
>>> SDBC-JDBC
>>>>>> bridge driver in Java. The SDBC-JDBC bridge is there to allows AOO to
>>>>>> access databases through Java's JDBC drivers.
>>>>>>
>>>>>> This port originally aimed at fixing serious issues I discovered in
>>> the
>>>>> old
>>>>>> C++ implementation, where there is apparently memory corruption on
>>> method
>>>>>> IDs that causes methods to be called on wrong Java objects. In the
>>>>> process
>>>>>> of porting I've also discovered and fixed many other problems. For
>>>>> example:
>>>>>> * The C++ code was calling Java code through the Java Invocation API,
>>> so
>>>>> it
>>>>>> was using strings to look up method names. This defeats type checking,
>>>>> and
>>>>>> in some cases was looking for methods with typos in their names or
>>> that
>>>>>> don't even exist. My current bridge only uses Java to Java calls that
>>> are
>>>>>> always strongly type checked and correct.
>>>>>> * Conversions between bytes and chars were broken: n bytes were being
>>>>>> copied verbatim into memory for char arrays n chars long in Java
>>> (Java's
>>>>>> char is 16 bits) so the second half of the array was untouched and the
>>>>>> first half had wrong chars, and non-existent classes like
>>>>>> java.io.CharArrayInputStream were being created. Real classes that do
>>>>>> exist, and proper character set conversions, are now used.
>>>>>> * Chained Java exceptions are now converted to chained UNO exceptions
>>> via
>>>>>> the commonly used and clearer getCause() chaining dimension, not the
>>>>>> java.sql.SQLException's getNextException() chaining dimension.
>>>>>>
>>>>>> The new driver is, as seen externally, completely interchangeable with
>>>>> the
>>>>>> old one. Service names are identical, implementation names are
>>> identical,
>>>>>> supported interfaces are identical, the configuration schema is
>>>>> identical,
>>>>>> initialization arguments are identical, even logging channels and
>>>>> messages
>>>>>> logged are identical - client code using the old driver should be
>>> able to
>>>>>> use the new one transparently, and not only should everything that
>>> worked
>>>>>> before still work now, but some things that didn't work before might
>>> also
>>>>>> begin to work now.
>>>>>>
>>>>>> Having said that, the driver does push UNO to its limits, and might
>>>>> reveal
>>>>>> other bugs in UNO bridging, Base, other AOO components, and the C++
>>>>> runtime
>>>>>> environment. I already discovered AOO sometimes crashes on FreeBSD
>>> due to
>>>>>> issues in converting exceptions from Java to C++ (missing exception
>>> RTTI
>>>>>> causes a segfault), but such bugs should be fixed in main/bridges
>>> and/or
>>>>>> the Clang project.
>>>>>>
>>>>>> There are still some cleanups I want to do, like making sure nulls
>>>>> strings
>>>>>> are never passed or returned to UNO, cleaning up exception handling,
>>>>> doing
>>>>>> a comparison of every C++ method with every Java method to see if I
>>>>> missed
>>>>>> anything, and so on, which is why the old C++ code has been left in
>>> the
>>>>>> tree for now even though it's no longer built. In the meanwhile,
>>> please
>>>>>> feel free to start testing :).
>>>>>>
>>>>>> Oh and my original memory-corruption-induced IllegalClassChangeError
>>> is
>>>>>> gone - my PostgreSQL SDBC driver already works better with the new
>>>>>> SDBC-JDBC bridge :).
>>>>>>
>>>>>> Damjan
>>>>>>
>>>>>
>>>
>>>


smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: The new SDBC-JDBC bridge in Java

Kay Schenk-2
In reply to this post by Damjan Jovanovic
These changes seemed to have fixed the errors I was having about a week ago
while building with the "connectivity" module. Still on Linux-32. I hadn't
had time to investigate when I saw these new changes. I did a clean build
AND a new configuration from scratch and the build is fine. No in depth
testing as yet. Thank you. Damjen.

On Wed, Nov 8, 2017 at 8:48 AM, Damjan Jovanovic <[hidden email]> wrote:

> Apparently I left out that IDL file in my last commit. Sorry. The latest
> SVN should build now.
>
> On Wed, Nov 8, 2017 at 6:41 PM, Damjan Jovanovic <[hidden email]>
> wrote:
>
> > Thank you. There is nothing in that log but "Error 1".
> >
> > It builds perfectly on FreeBSD. I'll try a Windows build too but it could
> > take a while.
> >
> > The file has *nix style line endings like every other IDL file.
> >
> > Any other ideas?
> >
> > Regards
> > Damjan
> >
> >
> > On Wed, Nov 8, 2017 at 5:50 PM, Matthias Seidel <
> > [hidden email]> wrote:
> >
> >> Hi Damjan,
> >>
> >> I was able to force the Windows buildbot and this is the detailed error
> >> log:
> >>
> >> https://ci.apache.org/projects/openoffice/buildlogs/win/
> >> main/offapi/wntmsci12.pro/misc/logs/prj.txt
> >>
> >> Regards, Matthias
> >>
> >>
> >> Am 08.11.2017 um 14:36 schrieb Damjan Jovanovic:
> >> > Hi Matthias
> >> >
> >> > Was it a clean rebuild?
> >> >
> >> > Can you post the full output of "build --from offapi"?
> >> >
> >> > Maybe it needs DOS line endings in ParameterSubstitution.idl?
> >> >
> >> > Damjan
> >> >
> >> > On Wed, Nov 8, 2017 at 2:52 PM, Matthias Seidel <
> >> [hidden email]>
> >> > wrote:
> >> >
> >> >> Hi Damjan,
> >> >>
> >> >> On my release machine (Win 10) it stops with:
> >> >>
> >> >> ---
> >> >> make: *** [C:/Source/aoo/main/solenv/gbuild/UnoApiTarget.mk:165:
> >> >> /cygdrive/c/Source/aoo/main/solver/420/wntmsci12.pro/
> >> >> workdir/UnoApiPartTarget/offapi/com/sun/star/sdb/ParameterSu
> >> bstitution.urd
> >> >> ]
> >> >> Error 1
> >> >> make: *** Waiting for unfinished jobs....
> >> >> dmake:  Error code 2, while making 'all'
> >> >>
> >> >> 1 module(s):
> >> >>         offapi
> >> >> need(s) to be rebuilt
> >> >>
> >> >> Reason(s):
> >> >>
> >> >> ERROR: error 65280 occurred while making
> >> >> /cygdrive/c/Source/aoo/main/offapi/prj
> >> >>
> >> >> When you have fixed the errors in that module you can resume the
> build
> >> >> by running:
> >> >>
> >> >>         build --from offapi
> >> >> ---
> >> >>
> >> >> I will try to force a build on our buildbot to cross check.
> >> >>
> >> >> Regards, Matthias
> >> >>
> >> >>
> >> >> Am 08.11.2017 um 05:34 schrieb Damjan Jovanovic:
> >> >>> Hi
> >> >>>
> >> >>> As of revision 1814552, I've rewritten our formerly C++ based
> >> SDBC-JDBC
> >> >>> bridge driver in Java. The SDBC-JDBC bridge is there to allows AOO
> to
> >> >>> access databases through Java's JDBC drivers.
> >> >>>
> >> >>> This port originally aimed at fixing serious issues I discovered in
> >> the
> >> >> old
> >> >>> C++ implementation, where there is apparently memory corruption on
> >> method
> >> >>> IDs that causes methods to be called on wrong Java objects. In the
> >> >> process
> >> >>> of porting I've also discovered and fixed many other problems. For
> >> >> example:
> >> >>> * The C++ code was calling Java code through the Java Invocation
> API,
> >> so
> >> >> it
> >> >>> was using strings to look up method names. This defeats type
> checking,
> >> >> and
> >> >>> in some cases was looking for methods with typos in their names or
> >> that
> >> >>> don't even exist. My current bridge only uses Java to Java calls
> that
> >> are
> >> >>> always strongly type checked and correct.
> >> >>> * Conversions between bytes and chars were broken: n bytes were
> being
> >> >>> copied verbatim into memory for char arrays n chars long in Java
> >> (Java's
> >> >>> char is 16 bits) so the second half of the array was untouched and
> the
> >> >>> first half had wrong chars, and non-existent classes like
> >> >>> java.io.CharArrayInputStream were being created. Real classes that
> do
> >> >>> exist, and proper character set conversions, are now used.
> >> >>> * Chained Java exceptions are now converted to chained UNO
> exceptions
> >> via
> >> >>> the commonly used and clearer getCause() chaining dimension, not the
> >> >>> java.sql.SQLException's getNextException() chaining dimension.
> >> >>>
> >> >>> The new driver is, as seen externally, completely interchangeable
> with
> >> >> the
> >> >>> old one. Service names are identical, implementation names are
> >> identical,
> >> >>> supported interfaces are identical, the configuration schema is
> >> >> identical,
> >> >>> initialization arguments are identical, even logging channels and
> >> >> messages
> >> >>> logged are identical - client code using the old driver should be
> >> able to
> >> >>> use the new one transparently, and not only should everything that
> >> worked
> >> >>> before still work now, but some things that didn't work before might
> >> also
> >> >>> begin to work now.
> >> >>>
> >> >>> Having said that, the driver does push UNO to its limits, and might
> >> >> reveal
> >> >>> other bugs in UNO bridging, Base, other AOO components, and the C++
> >> >> runtime
> >> >>> environment. I already discovered AOO sometimes crashes on FreeBSD
> >> due to
> >> >>> issues in converting exceptions from Java to C++ (missing exception
> >> RTTI
> >> >>> causes a segfault), but such bugs should be fixed in main/bridges
> >> and/or
> >> >>> the Clang project.
> >> >>>
> >> >>> There are still some cleanups I want to do, like making sure nulls
> >> >> strings
> >> >>> are never passed or returned to UNO, cleaning up exception handling,
> >> >> doing
> >> >>> a comparison of every C++ method with every Java method to see if I
> >> >> missed
> >> >>> anything, and so on, which is why the old C++ code has been left in
> >> the
> >> >>> tree for now even though it's no longer built. In the meanwhile,
> >> please
> >> >>> feel free to start testing :).
> >> >>>
> >> >>> Oh and my original memory-corruption-induced IllegalClassChangeError
> >> is
> >> >>> gone - my PostgreSQL SDBC driver already works better with the new
> >> >>> SDBC-JDBC bridge :).
> >> >>>
> >> >>> Damjan
> >> >>>
> >> >>
> >> >>
> >>
> >>
> >>
> >
>



--
----------------------------------------------------------------------
MzK

"Only the truth will save you now."
                         -- Ensei Tankado, "Digital Fortress"