PDA

View Full Version : What is maximu Lscript Array size?



sami
02-07-2013, 10:24 PM
Searched docs and mikeg's site, but can't seem to find the maximum size of arrays in Lscript. I'm hitting the limit, but don't know what that limit is or how to get around it.

thanks!

sami
02-08-2013, 03:11 AM
I had more of a play and got arrays (at least ones containing just integers) to be as big as 10,000,000 elements. But I can't seem to get decent performance filling the arrays if they are dynamic. If I just keep popping items into a dynamic array after about 200,000 elements each subsequent element you add to the array gets progressively slower and slower to add. It's almost like it is re-sorting the array everytime you add an element even though I'm adding them to the end (appending them I thought?)...?

However if I allocate the space ahead of time by defining the array to a really big size to begin with like this:

var myArray[10000000];

then filling the elements is very fast. Why is this? Are dynamic arrays just much more inefficient? What do you suggest if you don't know how big the array is - just over allocate and make a giant array that could fill what you need?

evenflcw
02-08-2013, 05:40 AM
I think you answered your own question...!

(Personally I imagine lscript reallocates and copies all data whenever you index out of bounds.)

Sensei
02-08-2013, 07:44 PM
I would say it's pretty normal.

Reallocation, copying old one to new one, and freeing old one is how dynamic array work. The more elements to copy, the slower working.

What can you do with it?

You can create your own implementation of dynamic array.
Make class that will have additional parameters: allocated size, chunk size.

Then append()
should check whether index is > allocated size, if so, allocate new table with allocated size+chunk size
if it's < then just add element at index (table has enough space to hold it).

chunk size can be f.e. 16, so every 16 entries there will be reallocation,
if f.e. 256, so every 256 elements added there will be reallocation,
and so on.

sami
02-08-2013, 08:14 PM
I would say it's pretty normal.

Reallocation, copying old one to new one, and freeing old one is how dynamic array work. The more elements to copy, the slower working.


Perhaps, but behind the scenes in other languages, they compensate for this and dynamic array appending is not as slow as in LScript.



What can you do with it?

You can create your own implementation of dynamic array.
Make class that will have additional parameters: allocated size, chunk size.

Then append()
should check whether index is > allocated size, if so, allocate new table with allocated size+chunk size
if it's < then just add element at index (table has enough space to hold it).

chunk size can be f.e. 16, so every 16 entries there will be reallocation,
if f.e. 256, so every 256 elements added there will be reallocation,
and so on.

Nice technique!

I keep forgetting that because LScript is a procedural language - we have to keep creating our own functions to OOP-ize it and make it easier to program with. Your idea is good.

I'm already creating OOP "objects" by using named @sequences for "property" values and creating functions in the form ObjectType_create() as constructors, and then using nested dynamic arrays as "collections" of "objects" for encapsulation of objects-- this technique will make my OO-style data-structures perform better when adding items to collections which already have hundreds of thousands of elements. thanks!

Once Python can do everything LScript can and if I don't care about backwards compatibility to old LW versions, then I won't have to jump through these hoops anymore....

Sensei
02-08-2013, 08:18 PM
Nice technique!


Every programmer on the planet knows this..

IMHO Lscript and Python are for artist, that don't know how to develop real program. You should use C/C++.. Compiled plugin works thousands times faster..

sami
02-08-2013, 08:28 PM
Every programmer on the planet knows this..

IMHO Lscript and Python are for artist, that don't know how to develop real program. You should use C/C++.. Compiled plugin works thousands times faster..

Take a compliment ;)

Of course it is an obvious design pattern. You are absolutely correct. But using Lscript, one gets into bad habits because your hands are so tied with the language you start thinking constrained - that was my point and you reminded me of a better approach.

The benefit of LScript over C++ is that on any machine with a text editor, I can quickly slap something together without having to have a full visual studio IDE and annoyances of setting up my dev environment every time. Plus I don't have to set it up on a Mac machine for multiple platform compiling. I still think LSript is a good choice if the task can be achieved fast enough. but you are right that there gets a point where with the hoops you have to jump through, you might as well do it in a real language like C++.