Changing values in matrix which is parameter

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Changing values in matrix which is parameter

Regina Henschel
Hi all,

in method LINEST I get pMatX from the stack. That is a reference to the
matrix MatX. I calculate the means column wise getting a matrix MeanX
with associated reference pMeanX. In next step I calculate
        pMatX->GetDouble(i,j) - pMeanX->GetDouble(i,0)

Can I overwrite the values in matrix MatX with this results? Or must the
values of the matrix MatX remain unchanged by the method LINEST?

Does there exist a guide for handling of matrices in the interpreter?

Kind regards
Regina







---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Changing values in matrix which is parameter

Eike Rathke-6
Hi Regina,

On Sunday, 2010-08-08 20:38:21 +0200, Regina Henschel wrote:

> in method LINEST I get pMatX from the stack. That is a reference to
> the matrix MatX. I calculate the means column wise getting a matrix
> MeanX with associated reference pMeanX. In next step I calculate
> pMatX->GetDouble(i,j) - pMeanX->GetDouble(i,0)
>
> Can I overwrite the values in matrix MatX with this results? Or must
> the values of the matrix MatX remain unchanged by the method LINEST?

Use a new matrix created with ScInterpreter::GetNewMat(), a matrix
argument may be a matrix constant as in LINEST(...;{1|2|3|4};...)

> Does there exist a guide for handling of matrices in the interpreter?

Unfortunately not.

  Eike

--
 OOo Calc core developer. Number formatter stricken i18n transpositionizer.
 Signature key 0x87F8D412 : 2F58 5236 DB02 F335 8304  7D6C 65C9 F9B5 87F8 D412
 OpenOffice.org Engineering at Oracle: http://blogs.sun.com/GullFOSS
 --
 Please don't send mail to the old [hidden email] account that I used for mailing
 lists, it phased out. Use [hidden email] instead. Thanks.

attachment0 (200 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Changing values in matrix which is parameter

Regina Henschel
Hi Eike,

Eike Rathke schrieb:

> Hi Regina,
>
> On Sunday, 2010-08-08 20:38:21 +0200, Regina Henschel wrote:
>
>> in method LINEST I get pMatX from the stack. That is a reference to
>> the matrix MatX. I calculate the means column wise getting a matrix
>> MeanX with associated reference pMeanX. In next step I calculate
>> pMatX->GetDouble(i,j) - pMeanX->GetDouble(i,0)
>>
>> Can I overwrite the values in matrix MatX with this results? Or must
>> the values of the matrix MatX remain unchanged by the method LINEST?
>
> Use a new matrix created with ScInterpreter::GetNewMat(), a matrix
> argument may be a matrix constant as in LINEST(...;{1|2|3|4};...)
>

I have not considered that in my tests, fortunately I ask here.

When X and Y are the given matrices and deltaX and deltaY those matrices
where the means are subtracted, then the needed calculation for the
coefficients B (in column arrangement) is in case of "with constant"
B = (deltaX' * deltaX)^{-1} * deltaX' * deltaY
and in case of "without constant"
B = (X' * X)^{-1} * X' * Y
where A' means transpose(A) and A^{-1} means inverse(A). So using deltaX
instead of X would have been handy.

The current implementation avoids to copy the matrices X or Y, although
that would ease and shorten the code. I thought there must be a reason
for that and tried to avoid it too. I hesitate to copy X or Y, because
they might have a huge number of lines. Statistical investigations with
more then 1000 samples are commonly. The number of variables (=columns
of X) is often 1 or at least under 10. So products as X'X are much smaller.

I will try only using new matrices for products like X'X as it is done
in the current implementation. That will result in a lot of nearly
identical multiplication methods, but I think, that is better than
copying the matrices.

Kind regards
Regina




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Changing values in matrix which is parameter

Eike Rathke-6
Hi Regina,

On Thursday, 2010-08-12 16:38:33 +0200, Regina Henschel wrote:

> The current implementation avoids to copy the matrices X or Y,
> although that would ease and shorten the code. I thought there must
> be a reason for that and tried to avoid it too. I hesitate to copy X
> or Y, because they might have a huge number of lines.

Indeed.

> I will try only using new matrices for products like X'X as it is
> done in the current implementation. That will result in a lot of
> nearly identical multiplication methods, but I think, that is better
> than copying the matrices.

To avoid both, having overly complicated code and copying large
matrices, we could introduce a const/non-const flag in ScMatrix,
defaulted to const and set to non-const by GetNewMat(), but to const
again when stored in ScFormulaResult. The usual case is that a matrix
was constructed from a range reference by the interpreter and not as
a const array or formula cell result. Then having
a ScMatrix::CloneIfConst() method could just return the current matrix
if non-const and would have to actually clone only in rare const cases.

Just an idea..

  Eike

--
 OOo Calc core developer. Number formatter stricken i18n transpositionizer.
 Signature key 0x87F8D412 : 2F58 5236 DB02 F335 8304  7D6C 65C9 F9B5 87F8 D412
 OpenOffice.org Engineering at Oracle: http://blogs.sun.com/GullFOSS
 --
 Please don't send mail to the old [hidden email] account that I used for mailing
 lists, it phased out. Use [hidden email] instead. Thanks.

attachment0 (200 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Changing values in matrix which is parameter

Regina Henschel
Hi Eike,

Eike Rathke schrieb:

> Hi Regina,
>
> On Thursday, 2010-08-12 16:38:33 +0200, Regina Henschel wrote:
>
>> The current implementation avoids to copy the matrices X or Y,
>> although that would ease and shorten the code. I thought there must
>> be a reason for that and tried to avoid it too. I hesitate to copy X
>> or Y, because they might have a huge number of lines.
>
> Indeed.
>
>> I will try only using new matrices for products like X'X as it is
>> done in the current implementation. That will result in a lot of
>> nearly identical multiplication methods, but I think, that is better
>> than copying the matrices.
>
> To avoid both, having overly complicated code and copying large
> matrices, we could introduce a const/non-const flag in ScMatrix,
> defaulted to const and set to non-const by GetNewMat(), but to const
> again when stored in ScFormulaResult. The usual case is that a matrix
> was constructed from a range reference by the interpreter and not as
> a const array or formula cell result. Then having
> a ScMatrix::CloneIfConst() method could just return the current matrix
> if non-const and would have to actually clone only in rare const cases.
>
> Just an idea..

The problem with constant parameters is already actual :(
http://www.openoffice.org/issues/show_bug.cgi?id=113879

Enter the formula
=LOGEST({2|3|5|8|13};{2|4|6|8|10})
Set the cursor into the formula range to see the formula in the input line.
Recalculate some times.
Notice, that the values in the first parameter changes.

This is done in the method ScInterpreter::CheckMatrix in the statement
     if ( _bLOG )
     {
         for (SCSIZE nElem = 0; nElem < nCountY; nElem++)
         {
             const double fVal = pMatY->GetDouble(nElem);
             if (fVal <= 0.0)
             {
                 PushIllegalArgument();
                 return false;
             }
             else
                 pMatY->PutDouble(log(fVal), nElem);   <---- here
         } // for (nElem = 0; nElem < nCountY; nElem++)
     } // if ( _bRKP )

Kind regards
Regina

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Changing values in matrix which is parameter

Eike Rathke-6
Hi Regina,

On Saturday, 2010-08-14 00:18:38 +0200, Regina Henschel wrote:

> The problem with constant parameters is already actual :(
> http://www.openoffice.org/issues/show_bug.cgi?id=113879

Thanks for catching. I'll see if I can easily introduce the mentioned
CloneIfConst() approach or have to simply fix this with an unconditional
Clone().

  Eike

--
 OOo Calc core developer. Number formatter stricken i18n transpositionizer.
 Signature key 0x87F8D412 : 2F58 5236 DB02 F335 8304  7D6C 65C9 F9B5 87F8 D412
 OpenOffice.org Engineering at Oracle: http://blogs.sun.com/GullFOSS
 --
 Please don't send mail to the old [hidden email] account that I used for mailing
 lists, it phased out. Use [hidden email] instead. Thanks.

attachment0 (200 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Changing values in matrix which is parameter

Regina Henschel
Hi Eike,

-----Original Message-----
> Date: Mon, 16 Aug 2010 16:24:23 +0200
> Subject: Re: [sc-dev] Changing values in matrix which is parameter
> From: Eike Rathke <[hidden email]>
> To: [hidden email]

> Hi Regina,
>
> On Saturday, 2010-08-14 00:18:38 +0200, Regina Henschel wrote:
>
> > The problem with constant parameters is already actual :(
> > http://www.openoffice.org/issues/show_bug.cgi?id=113879
>
> Thanks for catching. I'll see if I can easily introduce the mentioned
> CloneIfConst() approach or have to simply fix this with an
> unconditional Clone().
>

Isn't the total length of a formula restricted to 255 characters? If so,
a constant matrix cannot be huge and cloning is harmless.

Does your suggestion mean, that changing the values in the parameter
matrices in the implementation of the function would then be always
possible? Or will there be a property of pMatY which I have to test
inside the implementation for to decide, whether I can change it or have
to use a clone?

Kind regards
Regina






---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Changing values in matrix which is parameter

Eike Rathke-6
Hi Regina,

On Monday, 2010-08-16 22:40:49 +0200, Regina Henschel wrote:

> > Thanks for catching. I'll see if I can easily introduce the mentioned
> > CloneIfConst() approach or have to simply fix this with an
> > unconditional Clone().
>
> Isn't the total length of a formula restricted to 255 characters?

No, the length of one symbol/token was limited to 255 characters,
nowadays is 1024. A token is an operator, a function name, a literal
string, ... Each element of a constant matrix is a token. The maximum
total length of a formula is 512 tokens.

> If so, a constant matrix cannot be huge and cloning is harmless.

An unconditional clone would also clone any range reference that was
converted to a matrix. This is the case we'd like to avoid.

> Does your suggestion mean, that changing the values in the parameter
> matrices in the implementation of the function would then be always
> possible? Or will there be a property of pMatY which I have to test
> inside the implementation for to decide, whether I can change it or have
> to use a clone?

What I have in mind would be a method
ScMatrixRef ScMatrix::CloneIfConst()
that could be called as

    ScMatrixRef pNew = pMatY->CloneIfConst();
    ... modify pNew ...
    pMatY = pNew;

If pMatY was mutable, the call wouldn't clone the matrix but just add
a new reference and modifying pNew would actually modify pMatY. The
ScMatrix ctor would always flag a matrix as immutable and only
ScInterpreter::GetNewMat() would set that to mutable, and the final
result matrix would be set to immutable again in case it is used as
input for another expression. However, the case of array context needs
to be evaluated, as multiple calls to a function that expects mixed
scalar and matrix parameters of course should not modify the matrix
multiple times. If there is no easy solution to that I'll refrain from
changing things for OOo3.3 and just clone the matrix.

  Eike

--
 OOo Calc core developer. Number formatter stricken i18n transpositionizer.
 Signature key 0x87F8D412 : 2F58 5236 DB02 F335 8304  7D6C 65C9 F9B5 87F8 D412
 OpenOffice.org Engineering at Oracle: http://blogs.sun.com/GullFOSS
 --
 Please don't send mail to the old [hidden email] account that I used for mailing
 lists, it phased out. Use [hidden email] instead. Thanks.

attachment0 (200 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Changing values in matrix which is parameter

Eike Rathke-6
In reply to this post by Eike Rathke-6
Hi Regina,

On Monday, 2010-08-16 16:24:23 +0200, Eike Rathke wrote:

> > The problem with constant parameters is already actual :(
> > http://www.openoffice.org/issues/show_bug.cgi?id=113879
>
> Thanks for catching. I'll see if I can easily introduce the mentioned
> CloneIfConst() approach or have to simply fix this with an unconditional
> Clone().

You may want to use the changeset fa8436bc0f37 in your further
development, that implements the CloneIfConst() approach I mentioned.

http://hg.services.openoffice.org/cws/calc58/changeset/fa8436bc0f37

  Eike

--
 OOo Calc core developer. Number formatter stricken i18n transpositionizer.
 Signature key 0x87F8D412 : 2F58 5236 DB02 F335 8304  7D6C 65C9 F9B5 87F8 D412
 OpenOffice.org Engineering at Oracle: http://blogs.sun.com/GullFOSS
 --
 Please don't send mail to the old [hidden email] account that I used for mailing
 lists, it phased out. Use [hidden email] instead. Thanks.

attachment0 (200 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Changing values in matrix which is parameter

Regina Henschel
Hi Eike,

Eike Rathke schrieb:

> Hi Regina,
>
> On Monday, 2010-08-16 16:24:23 +0200, Eike Rathke wrote:
>
>>> The problem with constant parameters is already actual :(
>>> http://www.openoffice.org/issues/show_bug.cgi?id=113879
>>
>> Thanks for catching. I'll see if I can easily introduce the mentioned
>> CloneIfConst() approach or have to simply fix this with an unconditional
>> Clone().
>
> You may want to use the changeset fa8436bc0f37 in your further
> development, that implements the CloneIfConst() approach I mentioned.
>
> http://hg.services.openoffice.org/cws/calc58/changeset/fa8436bc0f37

I have applied it to my work now and it works fine :) But I need
ScMatrixRef pNewY = pMatY->CloneIfConst(); not only in case _bLOG, so I
had to move it to outside the if-statement.

kind regards
Regina

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]