Effectively use Ta3D::String

Everything related to the code /
Tout ce qui touche au code
Post Reply
milipili
Posts: 545
Joined: Thu Nov 02, 2006 8:52 am
Location: Paris (France)
Contact:

Effectively use Ta3D::String

Post by milipili » Fri May 22, 2009 7:35 pm

It was true with the old Ta3D::String, and it is still true (and for any class actually, not just strings). Just to recall that using the operator `+` with string is quite expansive. This is not obvious so here is an example.

Assuming that we have a dummy function `foo(const String& s)`, this code is more efficient

Code: Select all

int i = 42;
float f = 0.23f;
foo(String() << "Some value : " << i << ", another one : " << f)
than

Code: Select all

int i = 42;
float f = 0.23f;
foo("Some value : " + i + ", another one : " + f)
In the second example, 3 strings will be created instead of a single one, which implies 2 object instanciations + 2 malloc + 2 free for nothing in the case of strings.


Another example, when using string in loops. Assuming we have a list of string (std::vector<String>), merely a list of filename to load

Code: Select all

for (std::vector<String>::const_iterator i = l.begin(); i != l.end(); ++i)
{
    String absoluteFileName = path + *i;
    // Do something
}
The better way for doing it is the following :

Code: Select all

String absoluteFileName;
const std::vector<String>::const_iterator end = l.end();
for (std::vector<String>::const_iterator i = l.begin(); i != end; ++i)
{
    absoluteFileName.clear();
    absoluteFileName << path << *i;
    // Do something 
}
In the most cases, the second example will use a one malloc + one free (may be two, depanding the length of the string). In the first example there is a malloc + free for each iteration (plus in this example another one due to the operator `+`).

The worst scenario : 2 malloc + 1 free for the first example, (N+1) malloc + (N+1) free where N = l.size() (+1 for the use of the operator +)
Damien Gerard
Ta3d & Yuni Developer

User avatar
zuzuf
Administrateur - Site Admin
Posts: 3281
Joined: Mon Oct 30, 2006 8:49 pm
Location: Toulouse, France
Contact:

Re: Effectively use Ta3D::String

Post by zuzuf » Fri May 22, 2009 9:06 pm

This is the problem with temporary objects. Note that there is the same problem in Lua and this can really slow scripts down if we don't pay attention to it : if you write some Lua code like you would write it in C++ then there is a good chance it could run 10 times faster ! (I tested this with simple vector objects).

Optimizing string operations becomes important now that most of the loading process is taken by string processing ...
=>;-D Penguin Powered

milipili
Posts: 545
Joined: Thu Nov 02, 2006 8:52 am
Location: Paris (France)
Contact:

Re: Effectively use Ta3D::String

Post by milipili » Fri May 22, 2009 10:05 pm

:D
Damien Gerard
Ta3d & Yuni Developer

xpoy
Posts: 669
Joined: Mon Sep 22, 2008 3:55 am

Re: Effectively use Ta3D::String

Post by xpoy » Sat May 23, 2009 6:45 pm

Really good advice.
But how to imporve lua?What can I do...

milipili
Posts: 545
Joined: Thu Nov 02, 2006 8:52 am
Location: Paris (France)
Contact:

Re: Effectively use Ta3D::String

Post by milipili » Sat May 23, 2009 7:25 pm

We can follow the way Homeworld 2 or Wow used.
But We will see it later. There is a lot of things to do before that.
Damien Gerard
Ta3d & Yuni Developer

User avatar
zuzuf
Administrateur - Site Admin
Posts: 3281
Joined: Mon Oct 30, 2006 8:49 pm
Location: Toulouse, France
Contact:

Re: Effectively use Ta3D::String

Post by zuzuf » Sat May 23, 2009 8:26 pm

I experimented objects with 2 states: temporary and storage. So when you do operations on any of them you get a temporary object which is reused when possible, then when you need to store the result, you just convert it to a stored object (it replaces the metatable with a one containing functions that create new objects when needed instead of reusing existing ones).

There is probably a better way to do this (Lua is very flexible), but since there are lots of other things to do we should see that later.
=>;-D Penguin Powered

milipili
Posts: 545
Joined: Thu Nov 02, 2006 8:52 am
Location: Paris (France)
Contact:

Re: Effectively use Ta3D::String

Post by milipili » Sat May 23, 2009 10:03 pm

As I can see, there is :)
But first of all i am fixing all segV
Damien Gerard
Ta3d & Yuni Developer

User avatar
zuzuf
Administrateur - Site Admin
Posts: 3281
Joined: Mon Oct 30, 2006 8:49 pm
Location: Toulouse, France
Contact:

Re: Effectively use Ta3D::String

Post by zuzuf » Sun May 24, 2009 11:24 am

It seems that Yuni Strings don't support this:

Code: Select all

String i("whatever you want");
i = i;

milipili
Posts: 545
Joined: Thu Nov 02, 2006 8:52 am
Location: Paris (France)
Contact:

Re: Effectively use Ta3D::String

Post by milipili » Mon May 25, 2009 9:12 am

It is an "undefined behavior" (that's why there is an assert). std::string accepts this since it uses a reference count and not directly the buffer.
May be it should be supported. I don't know. Is it something really important to you ?
Damien Gerard
Ta3d & Yuni Developer

User avatar
zuzuf
Administrateur - Site Admin
Posts: 3281
Joined: Mon Oct 30, 2006 8:49 pm
Location: Toulouse, France
Contact:

Re: Effectively use Ta3D::String

Post by zuzuf » Tue May 26, 2009 11:13 pm

no, it's not really important, I just noticed it when I found several lines like:

Code: Select all

someStringVector[i] = someStringVectorThatDoesNotAppearToBeTheSame[e];
with i == e ...
I used several times the fact that the old String used to support it to simplify the code a bit, but since it crashes the program all this will be gone soon
=>;-D Penguin Powered

xpoy
Posts: 669
Joined: Mon Sep 22, 2008 3:55 am

Re: Effectively use Ta3D::String

Post by xpoy » Fri May 29, 2009 9:49 am

:o

Code: Select all

someStringVector[i] = someStringVectorThatDoesNotAppearToBeTheSame[e];
This bug is creative.
But I didn't found the EIP in log when crash. :)

User avatar
zuzuf
Administrateur - Site Admin
Posts: 3281
Joined: Mon Oct 30, 2006 8:49 pm
Location: Toulouse, France
Contact:

Re: Effectively use Ta3D::String

Post by zuzuf » Fri May 29, 2009 6:43 pm

I found a difference in the behavior of Yuni String::explode : it doesn't keep empty elements, but TA3D uses empty elements to fill its menus. Also TA3D expects elements to be trimmed.

I could write a work around but the explode function is widely used so this would really be ugly.
It would be better to write another function similar to explode but which behaves as expected, or add parameters to explode (which is probably even better).
=>;-D Penguin Powered

xpoy
Posts: 669
Joined: Mon Sep 22, 2008 3:55 am

Re: Effectively use Ta3D::String

Post by xpoy » Fri May 29, 2009 7:11 pm

Is it helpful in load time? :)
I just want a simple addtion,when crashed,loged the crash Eip,that the "pointer code",it will make bug fix eazyly.

milipili
Posts: 545
Joined: Thu Nov 02, 2006 8:52 am
Location: Paris (France)
Contact:

Re: Effectively use Ta3D::String

Post by milipili » Fri May 29, 2009 8:31 pm

String::explode : Patch applied :)
Damien Gerard
Ta3d & Yuni Developer

xpoy
Posts: 669
Joined: Mon Sep 22, 2008 3:55 am

Re: Effectively use Ta3D::String

Post by xpoy » Sat May 30, 2009 8:12 pm

Good job!good job man!
Cool!
I couldn't say that the tone!Your job just made me pop to LOOOOL from very MAD( with some helpless guy).

huangyi
Posts: 10
Joined: Thu Mar 11, 2010 1:28 pm

Re: Effectively use Ta3D::String

Post by huangyi » Thu Mar 11, 2010 1:55 pm

why not use std::string::operator +=

User avatar
zuzuf
Administrateur - Site Admin
Posts: 3281
Joined: Mon Oct 30, 2006 8:49 pm
Location: Toulouse, France
Contact:

Re: Effectively use Ta3D::String

Post by zuzuf » Sat Mar 13, 2010 3:11 pm

huangyi wrote:why not use std::string::operator +=
Well ... because we don't use std::string but Yuni::String which supports more string operations and supports UTF-8 which is used almost everywhere in the engine :)
=>;-D Penguin Powered

huangyi
Posts: 10
Joined: Thu Mar 11, 2010 1:28 pm

Re: Effectively use Ta3D::String

Post by huangyi » Sat Mar 13, 2010 6:20 pm

Thank you for answer.
I should have scaned source code before I ask. sorry.

Post Reply

Who is online

Users browsing this forum: No registered users and 44 guests