A refactoring proposal

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

A refactoring proposal

Patricia Shanahan
Without going into details here, some recently fixed security issues
have related to the use of fixed size arrays without bounds checks.

In general, that is not a very robust programming practice. It depends
on careful checking in the source code to prevent array overflow.

I suggest a project to replace raw arrays with Standard Template Library
classes as appropriate. All accesses should be through safe functions
such as std::array::at. In some cases we could replace a limited size
but large array with e.g. a std::vector that can start small and grow
only as needed.

This matches nicely with my observations of volunteers. We are not
getting many people with the skills and experience to dive into a
very large body of code and debug it. We are getting students and
early career programmers who could work on something like this. It
might also be a viable Google Summer of Code project.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: A refactoring proposal

Matthias Seidel
Hi Patricia,

I am not a programmer, but looking at the screen when building AOO I see
a lot of messages that make me think that the code could need a review...

Your suggestion sounds very reasonable to me!

Kind regards, Matthias


Am 17.06.2017 um 19:52 schrieb Patricia Shanahan:

> Without going into details here, some recently fixed security issues
> have related to the use of fixed size arrays without bounds checks.
>
> In general, that is not a very robust programming practice. It depends
> on careful checking in the source code to prevent array overflow.
>
> I suggest a project to replace raw arrays with Standard Template Library
> classes as appropriate. All accesses should be through safe functions
> such as std::array::at. In some cases we could replace a limited size
> but large array with e.g. a std::vector that can start small and grow
> only as needed.
>
> This matches nicely with my observations of volunteers. We are not
> getting many people with the skills and experience to dive into a
> very large body of code and debug it. We are getting students and
> early career programmers who could work on something like this. It
> might also be a viable Google Summer of Code project.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: A refactoring proposal

Patricia Shanahan
Another useful project would indeed be warning removal. The reason for
putting bounds checking ahead of it is that I have not yet seen a
security bug that would have been fixed by warning removal.

On 6/17/2017 3:01 PM, Matthias Seidel wrote:

> Hi Patricia,
>
> I am not a programmer, but looking at the screen when building AOO I see
> a lot of messages that make me think that the code could need a review...
>
> Your suggestion sounds very reasonable to me!
>
> Kind regards, Matthias
>
>
> Am 17.06.2017 um 19:52 schrieb Patricia Shanahan:
>> Without going into details here, some recently fixed security issues
>> have related to the use of fixed size arrays without bounds checks.
>>
>> In general, that is not a very robust programming practice. It depends
>> on careful checking in the source code to prevent array overflow.
>>
>> I suggest a project to replace raw arrays with Standard Template Library
>> classes as appropriate. All accesses should be through safe functions
>> such as std::array::at. In some cases we could replace a limited size
>> but large array with e.g. a std::vector that can start small and grow
>> only as needed.
>>
>> This matches nicely with my observations of volunteers. We are not
>> getting many people with the skills and experience to dive into a
>> very large body of code and debug it. We are getting students and
>> early career programmers who could work on something like this. It
>> might also be a viable Google Summer of Code project.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: A refactoring proposal

Peter Kovacs-3
In reply to this post by Patricia Shanahan
+1

I have a question thought. In Tour of C++ (Bjarn Stroustrup) its
recommended to use array instead of buildin arrays and only to use
Arrays if we know the amount of elements (constexpr).

buildin arrays example:  Circle myarray[10]
array example: array<Circle,10> myarray

so std::array::at is equal to the arrayexample?
sorry if the question is dumb. I am not sure if what I know is the same
what you talk about.
I am in the "early carrier" category. lol.

All the best
Peter

Am 17.06.2017 um 19:52 schrieb Patricia Shanahan:

> Without going into details here, some recently fixed security issues
> have related to the use of fixed size arrays without bounds checks.
>
> In general, that is not a very robust programming practice. It depends
> on careful checking in the source code to prevent array overflow.
>
> I suggest a project to replace raw arrays with Standard Template Library
> classes as appropriate. All accesses should be through safe functions
> such as std::array::at. In some cases we could replace a limited size
> but large array with e.g. a std::vector that can start small and grow
> only as needed.
>
> This matches nicely with my observations of volunteers. We are not
> getting many people with the skills and experience to dive into a
> very large body of code and debug it. We are getting students and
> early career programmers who could work on something like this. It
> might also be a viable Google Summer of Code project.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: A refactoring proposal

Patricia Shanahan
I don't know. I have not read that particular book. If the code
snippet includes "using namespace std" then "array" means "std::array".

I have a time gap in my C++ experience - I used it professionally in the
1980's, and now I'm coming back to it for AOO. I don't think the
Standard Template Library came into wide use until the 1990's.

At this point I think we should use the STL for the next layer of data
structures above raw arrays, including arrays with bounds checking. I
have not heard of a competing library and I do not like to roll my own
if I can possibly avoid it.


On 6/17/2017 3:50 PM, Peter Kovacs wrote:

> +1
>
> I have a question thought. In Tour of C++ (Bjarn Stroustrup) its
> recommended to use array instead of buildin arrays and only to use
> Arrays if we know the amount of elements (constexpr).
>
> buildin arrays example:  Circle myarray[10]
> array example: array<Circle,10> myarray
>
> so std::array::at is equal to the arrayexample?
> sorry if the question is dumb. I am not sure if what I know is the same
> what you talk about.
> I am in the "early carrier" category. lol.
>
> All the best
> Peter
>
> Am 17.06.2017 um 19:52 schrieb Patricia Shanahan:
>> Without going into details here, some recently fixed security issues
>> have related to the use of fixed size arrays without bounds checks.
>>
>> In general, that is not a very robust programming practice. It depends
>> on careful checking in the source code to prevent array overflow.
>>
>> I suggest a project to replace raw arrays with Standard Template Library
>> classes as appropriate. All accesses should be through safe functions
>> such as std::array::at. In some cases we could replace a limited size
>> but large array with e.g. a std::vector that can start small and grow
>> only as needed.
>>
>> This matches nicely with my observations of volunteers. We are not
>> getting many people with the skills and experience to dive into a
>> very large body of code and debug it. We are getting students and
>> early career programmers who could work on something like this. It
>> might also be a viable Google Summer of Code project.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: A refactoring proposal

Peter Kovacs-4
The book says it is in <array>.
The book is about C++11, providing a quick reference on C++.
I cam across when I saw this youtube video:
https://www.youtube.com/watch?v=86xWVb4XIyE
And since I learned C++ in the 90ies, but never used it I thought it is
a good invest.

Am 18.06.2017 um 01:06 schrieb Patricia Shanahan:

> I don't know. I have not read that particular book. If the code
> snippet includes "using namespace std" then "array" means "std::array".
>
> I have a time gap in my C++ experience - I used it professionally in the
> 1980's, and now I'm coming back to it for AOO. I don't think the
> Standard Template Library came into wide use until the 1990's.
>
> At this point I think we should use the STL for the next layer of data
> structures above raw arrays, including arrays with bounds checking. I
> have not heard of a competing library and I do not like to roll my own
> if I can possibly avoid it.
>
>
> On 6/17/2017 3:50 PM, Peter Kovacs wrote:
>> +1
>>
>> I have a question thought. In Tour of C++ (Bjarn Stroustrup) its
>> recommended to use array instead of buildin arrays and only to use
>> Arrays if we know the amount of elements (constexpr).
>>
>> buildin arrays example:  Circle myarray[10]
>> array example: array<Circle,10> myarray
>>
>> so std::array::at is equal to the arrayexample?
>> sorry if the question is dumb. I am not sure if what I know is the
>> same what you talk about.
>> I am in the "early carrier" category. lol.
>>
>> All the best
>> Peter
>>
>> Am 17.06.2017 um 19:52 schrieb Patricia Shanahan:
>>> Without going into details here, some recently fixed security issues
>>> have related to the use of fixed size arrays without bounds checks.
>>>
>>> In general, that is not a very robust programming practice. It depends
>>> on careful checking in the source code to prevent array overflow.
>>>
>>> I suggest a project to replace raw arrays with Standard Template
>>> Library
>>> classes as appropriate. All accesses should be through safe functions
>>> such as std::array::at. In some cases we could replace a limited size
>>> but large array with e.g. a std::vector that can start small and grow
>>> only as needed.
>>>
>>> This matches nicely with my observations of volunteers. We are not
>>> getting many people with the skills and experience to dive into a
>>> very large body of code and debug it. We are getting students and
>>> early career programmers who could work on something like this. It
>>> might also be a viable Google Summer of Code project.
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: A refactoring proposal

Patricia Shanahan
In that case, we are talking about the same class. std::array is
defined in header <array>. See
http://en.cppreference.com/w/cpp/container/array

On 6/17/2017 9:58 PM, Peter Kovacs wrote:

> The book says it is in <array>.
> The book is about C++11, providing a quick reference on C++.
> I cam across when I saw this youtube video:
> https://www.youtube.com/watch?v=86xWVb4XIyE
> And since I learned C++ in the 90ies, but never used it I thought it is
> a good invest.
>
> Am 18.06.2017 um 01:06 schrieb Patricia Shanahan:
>> I don't know. I have not read that particular book. If the code
>> snippet includes "using namespace std" then "array" means "std::array".
>>
>> I have a time gap in my C++ experience - I used it professionally in the
>> 1980's, and now I'm coming back to it for AOO. I don't think the
>> Standard Template Library came into wide use until the 1990's.
>>
>> At this point I think we should use the STL for the next layer of data
>> structures above raw arrays, including arrays with bounds checking. I
>> have not heard of a competing library and I do not like to roll my own
>> if I can possibly avoid it.
>>
>>
>> On 6/17/2017 3:50 PM, Peter Kovacs wrote:
>>> +1
>>>
>>> I have a question thought. In Tour of C++ (Bjarn Stroustrup) its
>>> recommended to use array instead of buildin arrays and only to use
>>> Arrays if we know the amount of elements (constexpr).
>>>
>>> buildin arrays example:  Circle myarray[10]
>>> array example: array<Circle,10> myarray
>>>
>>> so std::array::at is equal to the arrayexample?
>>> sorry if the question is dumb. I am not sure if what I know is the
>>> same what you talk about.
>>> I am in the "early carrier" category. lol.
>>>
>>> All the best
>>> Peter
>>>
>>> Am 17.06.2017 um 19:52 schrieb Patricia Shanahan:
>>>> Without going into details here, some recently fixed security issues
>>>> have related to the use of fixed size arrays without bounds checks.
>>>>
>>>> In general, that is not a very robust programming practice. It depends
>>>> on careful checking in the source code to prevent array overflow.
>>>>
>>>> I suggest a project to replace raw arrays with Standard Template
>>>> Library
>>>> classes as appropriate. All accesses should be through safe functions
>>>> such as std::array::at. In some cases we could replace a limited size
>>>> but large array with e.g. a std::vector that can start small and grow
>>>> only as needed.
>>>>
>>>> This matches nicely with my observations of volunteers. We are not
>>>> getting many people with the skills and experience to dive into a
>>>> very large body of code and debug it. We are getting students and
>>>> early career programmers who could work on something like this. It
>>>> might also be a viable Google Summer of Code project.
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [hidden email]
>>>> For additional commands, e-mail: [hidden email]
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: A refactoring proposal

Peter Kovacs-4
Awesome! Thanks for the Clarification.

Am 18.06.2017 um 10:19 schrieb Patricia Shanahan:

> In that case, we are talking about the same class. std::array is
> defined in header <array>. See
> http://en.cppreference.com/w/cpp/container/array
>
> On 6/17/2017 9:58 PM, Peter Kovacs wrote:
>> The book says it is in <array>.
>> The book is about C++11, providing a quick reference on C++.
>> I cam across when I saw this youtube video:
>> https://www.youtube.com/watch?v=86xWVb4XIyE
>> And since I learned C++ in the 90ies, but never used it I thought it
>> is a good invest.
>>
>> Am 18.06.2017 um 01:06 schrieb Patricia Shanahan:
>>> I don't know. I have not read that particular book. If the code
>>> snippet includes "using namespace std" then "array" means "std::array".
>>>
>>> I have a time gap in my C++ experience - I used it professionally in
>>> the
>>> 1980's, and now I'm coming back to it for AOO. I don't think the
>>> Standard Template Library came into wide use until the 1990's.
>>>
>>> At this point I think we should use the STL for the next layer of data
>>> structures above raw arrays, including arrays with bounds checking. I
>>> have not heard of a competing library and I do not like to roll my own
>>> if I can possibly avoid it.
>>>
>>>
>>> On 6/17/2017 3:50 PM, Peter Kovacs wrote:
>>>> +1
>>>>
>>>> I have a question thought. In Tour of C++ (Bjarn Stroustrup) its
>>>> recommended to use array instead of buildin arrays and only to use
>>>> Arrays if we know the amount of elements (constexpr).
>>>>
>>>> buildin arrays example:  Circle myarray[10]
>>>> array example: array<Circle,10> myarray
>>>>
>>>> so std::array::at is equal to the arrayexample?
>>>> sorry if the question is dumb. I am not sure if what I know is the
>>>> same what you talk about.
>>>> I am in the "early carrier" category. lol.
>>>>
>>>> All the best
>>>> Peter
>>>>
>>>> Am 17.06.2017 um 19:52 schrieb Patricia Shanahan:
>>>>> Without going into details here, some recently fixed security issues
>>>>> have related to the use of fixed size arrays without bounds checks.
>>>>>
>>>>> In general, that is not a very robust programming practice. It
>>>>> depends
>>>>> on careful checking in the source code to prevent array overflow.
>>>>>
>>>>> I suggest a project to replace raw arrays with Standard Template
>>>>> Library
>>>>> classes as appropriate. All accesses should be through safe functions
>>>>> such as std::array::at. In some cases we could replace a limited size
>>>>> but large array with e.g. a std::vector that can start small and grow
>>>>> only as needed.
>>>>>
>>>>> This matches nicely with my observations of volunteers. We are not
>>>>> getting many people with the skills and experience to dive into a
>>>>> very large body of code and debug it. We are getting students and
>>>>> early career programmers who could work on something like this. It
>>>>> might also be a viable Google Summer of Code project.
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: [hidden email]
>>>>> For additional commands, e-mail: [hidden email]
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [hidden email]
>>>> For additional commands, e-mail: [hidden email]
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: A refactoring proposal

Carl Marcum
In reply to this post by Patricia Shanahan
On 06/17/2017 01:52 PM, Patricia Shanahan wrote:

> Without going into details here, some recently fixed security issues
> have related to the use of fixed size arrays without bounds checks.
>
> In general, that is not a very robust programming practice. It depends
> on careful checking in the source code to prevent array overflow.
>
> I suggest a project to replace raw arrays with Standard Template Library
> classes as appropriate. All accesses should be through safe functions
> such as std::array::at. In some cases we could replace a limited size
> but large array with e.g. a std::vector that can start small and grow
> only as needed.
>
> This matches nicely with my observations of volunteers. We are not
> getting many people with the skills and experience to dive into a
> very large body of code and debug it. We are getting students and
> early career programmers who could work on something like this. It
> might also be a viable Google Summer of Code project.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

That sounds like a good idea.

Do you think the calls may be common enough to find with a search to get
a list of files to look in?

Thanks,
Carl

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: A refactoring proposal

Patricia Shanahan

On 6/18/2017 5:51 AM, Carl Marcum wrote:

> On 06/17/2017 01:52 PM, Patricia Shanahan wrote:
>> Without going into details here, some recently fixed security issues
>> have related to the use of fixed size arrays without bounds checks.
>>
>> In general, that is not a very robust programming practice. It depends
>> on careful checking in the source code to prevent array overflow.
>>
>> I suggest a project to replace raw arrays with Standard Template Library
>> classes as appropriate. All accesses should be through safe functions
>> such as std::array::at. In some cases we could replace a limited size
>> but large array with e.g. a std::vector that can start small and grow
>> only as needed.
>>
>> This matches nicely with my observations of volunteers. We are not
>> getting many people with the skills and experience to dive into a
>> very large body of code and debug it. We are getting students and
>> early career programmers who could work on something like this. It
>> might also be a viable Google Summer of Code project.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
> That sounds like a good idea.
>
> Do you think the calls may be common enough to find with a search to get
> a list of files to look in?

Array access, including unchecked index access to STL array-like
structures, can be identified by the use of '[.*]' bracketed
expressions. STL access with bounds checking uses function call syntax
instead of overloaded array access syntax.

That is going to get a lot of hits so some prioritization is needed.

I suggest working first on files that have been culprits in
array-related security problems we have fixed. We know those use fixed
size arrays and were written or edited by people who did not always
check the bounds. I would then work out to other files in the same
modules, and then anything involved in building internal structures from
input files.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Loading...