unit scripting

Everything related to the code /
Tout ce qui touche au code
Post Reply
User avatar
zuzuf
Administrateur - Site Admin
Posts: 3281
Joined: Mon Oct 30, 2006 8:49 pm
Location: Toulouse, France
Contact:

unit scripting

Post by zuzuf » Thu Jun 18, 2009 7:19 pm

I've just noticed Lua VMs are much bigger than I expected : 84 kB if you load all the Lua libraries but since units don't need everything I cut it down to 53kB by loading only what's required.

Anyway this is still way too much since 4000 units would require 207MB only to store the Lua VMs. So I'll redesign the thing to use only one VM that'll run all the unit scripts. The major change for unit scripts will be that they'll have to share the same environment, so they'll run from data stored in specific tables instead of their own global one. There won't be any thread safety problem since all unit scripts are run from the same thread and will use their own tables.
=>;-D Penguin Powered

User avatar
Balthazar
Moderator
Posts: 2055
Joined: Wed Nov 01, 2006 4:31 pm
Location: Russian Federation
Contact:

Re: unit scripting

Post by Balthazar » Thu Jun 18, 2009 7:36 pm

Yeah, even now 512 MB of RAM are not everyone have.

xpoy
Posts: 669
Joined: Mon Sep 22, 2008 3:55 am

Re: unit scripting

Post by xpoy » Thu Jun 18, 2009 7:58 pm

Good Job!
This is cool solve
And that xon.dll expand VM or some other unit scripts' data will post in here(if got some more, :P)

User avatar
zuzuf
Administrateur - Site Admin
Posts: 3281
Joined: Mon Oct 30, 2006 8:49 pm
Location: Toulouse, France
Contact:

Re: unit scripting

Post by zuzuf » Sat Jun 20, 2009 11:55 am

I reduced the memory needs of Lua scripts, now I can run a 100 s 100 arm guardian simulation with only 440 MB of RAM being used (1000 vs 1000 required 1GB). This is still too much and there is still a bug I don't understand:
running tests on the AimPrimary function of ARMGUARD, I noticed lots of threads were allocated (and since they're cached once allocated they're not freed until the unit dies, which normally is not a problem since there should be very few threads) and running. Normally AimPrimary should kill all similar threads using the signal mechanisms, but it appears it doesn't work ... (which is strange because according to logs I've added a bit everywhere threads are actually killed :evil: and recycled).

Technically it seems that set_signal_mask changes the signal mask of the running thread which is later reverted to 0 for yet unknown reason, so it never gets killed before it returns :evil: resulting in 100 threads (sometimes more) running the same function at the same time and which shouldn't be there, and this is only for ONE UNIT :shock: !

If you run functions that do not rely on the signal mechanism, then it works better since it doesn't allocate all those threads (1000 arm solar panels use much less memory because of this).
=>;-D Penguin Powered

User avatar
zuzuf
Administrateur - Site Admin
Posts: 3281
Joined: Mon Oct 30, 2006 8:49 pm
Location: Toulouse, France
Contact:

Re: unit scripting

Post by zuzuf » Sat Jun 20, 2009 7:00 pm

ok, it's fixed ... well fixed but slow. The signal stuffs isn't slow itself, what is slow is creating threads when you use signals because Lua yielded coroutines cannot be stopped to be recycled :evil: so TA3D has to destroy them and replace them with new ones when it finds a yielded coroutine in its thread cache :cry: .

Hopefully there are other ways to implement exclusion mechanisms, remember unit scripts run sequentially (since they share the same Lua VM they'll never run parallelized), so it's easy to implement a lock (just do like COB/BOS scripts do in statechg.h ...). It works very well and it's much faster. So our Lua scripts are working again :mrgreen: and needs much less memory :P .

Since this needs testing I am going to make a new package with this new Lua engine.

So now I can run a 2000 vs 2000 game with less than 570MB RAM on my 64bits system :D

NB: now all units share the same Lua environment, this may be used later to do some nice things (unit communication, remote control - imagine you can hack enemy units and make them aim where you want :mrgreen: - , etc ...)
=>;-D Penguin Powered

User avatar
Balthazar
Moderator
Posts: 2055
Joined: Wed Nov 01, 2006 4:31 pm
Location: Russian Federation
Contact:

Re: unit scripting

Post by Balthazar » Sat Jun 20, 2009 7:35 pm

We are looking for the new binaries. Great progress indeed :P

User avatar
zuzuf
Administrateur - Site Admin
Posts: 3281
Joined: Mon Oct 30, 2006 8:49 pm
Location: Toulouse, France
Contact:

Re: unit scripting

Post by zuzuf » Sat Jun 20, 2009 8:23 pm

I am uploading the new package (it took me some time because I wanted to build it with GCC 4.2 instead of GCC 3.4, just to know if there is a noticeable difference on windows, so I had to recompile all the code which is SLOW in a virtual machine)
=>;-D Penguin Powered

xpoy
Posts: 669
Joined: Mon Sep 22, 2008 3:55 am

Re: unit scripting

Post by xpoy » Sun Jun 21, 2009 2:09 pm

What's diffrent for TA3D that GCC 4.2 instead GCC 3.4?

User avatar
zuzuf
Administrateur - Site Admin
Posts: 3281
Joined: Mon Oct 30, 2006 8:49 pm
Location: Toulouse, France
Contact:

Re: unit scripting

Post by zuzuf » Sun Jun 21, 2009 2:21 pm

I think GCC 4.2 is better at optimizing the generated machine code, it's also more paranoid regarding C++ syntax :D (but it's a good thing it forces you to write cleaner code :P) but the later won't change anything on the windows binary since GCC 4 is now the default on most Linux systems and it's the version I use. But since GCC 4 is not part of official mingw install package I wanted to know if it was still very experimental or not and if it was worth it :)
=>;-D Penguin Powered

Post Reply

Who is online

Users browsing this forum: No registered users and 29 guests