096.4 & memory problem

orii · Post by **orii** » Sat Feb 10, 2007 4:24 am

I've got a problem with Pol 096, which services 2 realms at the momment, and loads about 1500 scripts during the start.
When the server is up, operating system notifies, that memory takes away about 800MB, and when the players are connected, about 1GB, and this is where the real problem starts. Every uptime hour causes increase of occupied memory for about 100MB, in spite of almost constant number of nps's, pc's and items. In spite of all, we quickly approach 2GB memory limit (windows 2k3), and a server crushes.

I wander, if you could show me the best way of tracking that memory leak, or if I set in my boot.ini file /3GB paramter, server will achieve additional 1GB of memory, which will give us some more time to server crash.

SMJ · Post by **SMJ** » Sat Feb 10, 2007 4:15 pm

First thing to do: Upgrade from 096.4 to 096.5 and see if there's still a problem.

orii · Post by **orii** » Sat Feb 10, 2007 4:32 pm

there is a bug with stairs & etc
..o, and nothing about memory leak in core-changes.

OldnGrey · Post by **OldnGrey** » Sat Feb 10, 2007 4:39 pm

I ran 096 in its various releases for over a year in test and was most impressed with the way it didn't grow in RAM usage. I could easily leave the server up for 1 week without the ram getting our of control.

I'd have to suggest it isn't the core causing your problems.

What's your problem with stairs? The height of the player has been changed a couple of times iirc. In 097 for instance I have had to raise the ceiling in a few players to allow players to walk. I can't remember if 096.6 is going to include a change for char height to fix a problem reported by CWO, but I think that's not coming out for a while, being paradoxically tied to 097 and the 097 distro being released at the same time as the 097 core.

orii · Post by **orii** » Sat Feb 10, 2007 6:06 pm

stairs are not a problem. memory leak is a problem ;p
..and i dont know where is problem (core or script)

i just searching for a tools or advice to find where is 1GBram.

orii · Post by **orii** » Sun Feb 11, 2007 12:31 am

this :

Code: Select all

use uo;
use polsys;
use file;

program txtlogerscript(who)
    who := who;
    var pdict  := dictionary;
    LogToFile(":commandlog:logs/scripts.log","=============================================================");
    foreach thg in (polcore().all_scripts)
        pdict[thg.name] := Cint(pdict[thg.name]) + thg.var_size;
    endforeach
    foreach thg in (pdict.keys())
        LogToFile(":commandlog:logs/scripts.log","["+thg+"]["+pdict[thg]+"]");
    endforeach
endprogram

not show any problem in scripts, so where?

orii · Post by **orii** » Thu Feb 15, 2007 6:27 am

or maybe, please, anyone can say - what is that:

Remaining script objectimps: xxxx

last line after shutdown.

orii · Post by **orii** » Mon Feb 19, 2007 3:34 am

really noone know what is that:
"Remaining script objectimps xxx"
? :>

some screenshot
1) after restart

2) after several hours

1269884 (main memory)
1916928 (VM size)

Bracco · Post by **Bracco** » Sun Mar 04, 2007 2:18 pm

i'm sorry to bring up this post, but did you find any solution?

orii · Post by **orii** » Tue Mar 06, 2007 3:42 pm

no, sorry
i think-there is still problem with packethook-memory-leak
but...without packethook my shard is totaly unplayble, i can`t test this.

Post by **Shinigami** » Wed Mar 07, 2007 4:00 am

1. Please check your script base for usage of AppendConfigFile!

2. Do u modifiy Config files using external applications while server is running?

Shinigami

orii · Post by **orii** » Fri Mar 09, 2007 3:01 am

1 ) no, with AppendConfigFile i have only script "staticsexport/import/check" etc. (i don`t remember when i used this last time - meany years ago ;p - so i think it`s not a problem)
1b) i have many scripts with datafile & logfile
2) i have external aplication : this aplication have access to data/accounts.txt (r/o)

Bracco · Post by **Bracco** » Fri Mar 09, 2007 3:26 am

that should be fine, but instead there must be some other thing that causes that high memory usage... i have the same problem, and just can't figure out what's causing this

OldnGrey · Post by **OldnGrey** » Fri Mar 09, 2007 3:50 am

You can always try the crude methods.

What about on a test shard: does ram usage grow without players, if you remove the npcs files and turn off the spawner does it still grow? Can you reproduce the things that do cause ram usage to grow? Are there any situations where you can stop it growing - turn off packages etc.

I would expect RAM usage to grow slightly over time due to scripts loading and staying resident. Even unloadcfg has little effect on my RAM, certainly nothing to worry about.

Checked script.log, datafile.log?

The packet hooks in the 097 distro don't cause problems on my shard either. I guess it comes down to trying to get it to misbehave on a test shard.

Bracco · Post by **Bracco** » Fri Mar 09, 2007 3:59 am

OldnGrey wrote:You can always try the crude methods.

What about on a test shard: does ram usage grow without players, if you remove the npcs files and turn off the spawner does it still grow? Can you reproduce the things that do cause ram usage to grow? Are there any situations where you can stop it growing - turn off packages etc.

I would expect RAM usage to grow slightly over time due to scripts loading and staying resident. Even unloadcfg has little effect on my RAM, certainly nothing to worry about.

Checked script.log, datafile.log?

The packet hooks in the 097 distro don't cause problems on my shard either. I guess it comes down to trying to get it to misbehave on a test shard.

i am using 097 RC2, ram usage doesn't grow without players, i checked script.log, pol.log, debug.log and found nothing strange.
to me it seems like nearly every script that runs does not release all the memory it used... so the more players, the more script runs, the more memory grows, and grows faster

i remember this problem was introduced with one of the first 096 betas, but seems like something similar was in there from before (see my post in bug reports 097)

imho, it could be related to the script caching system or something like that

orii · Post by **orii** » Fri Mar 09, 2007 4:12 am

same here, more players = more memory disapering

Bracco · Post by **Bracco** » Thu Mar 15, 2007 6:52 am

orii wrote:no, sorry
i think-there is still problem with packethook-memory-leak
but...without packethook my shard is totaly unplayble, i can`t test this.

after intensive testing i came up with the conclusion that you are right... seems a packethook leak. i am almost sure, tried disabling them on local test machine and ram usage didn't grow. gonna test it on live shard and come back with results

tekproxy · Post by **tekproxy** » Thu Mar 15, 2007 10:33 am

I could write a quick script to connect to a server and login to a large number of accounts, if any of the core devs thought it would be useful for testing. I may do it later just for kicks to see if the distro has the same problem.

Bracco · Post by **Bracco** » Thu Mar 15, 2007 11:06 am

tekproxy wrote:I could write a quick script to connect to a server and login to a large number of accounts, if any of the core devs thought it would be useful for testing. I may do it later just for kicks to see if the distro has the same problem.

it's not needed... it seems that the leak happens just for the existance of an hook (the hook must run to leak ofc), regardless of what code it has inside...

right now i'm testing a 0x11 SendStatus packethook, even if it's empty (only a return 0) it leaks

gonna see if ReceiveFunction leaks also

btw, later i'll post it in bug forums

OldnGrey · Post by **OldnGrey** » Thu Mar 15, 2007 3:09 pm

I've gone back to a 33 hour restart for pol097.
None of the leaks showed up on the test shard, but it's amazing how all my testing was useless as soon as I went live with players. I have quite a few packethooks too.

Not surprising really. We got used to 095 over its many years of service. 096/7 is a new learning experience. It's surprisingly good and easy to maintain so far, but we shouldn't be surprised.