PenUltima Online Forum Index Official Core: 096.7
Official Core: 097 2008-02-26
Donate towards the POL web hosting bill!
 POL Home   FAQ   Search    Memberlist   Usergroups    Register    Profile   Log in to check your private messages   Log in
Script that iterates over all items on server?
Goto page Previous  1, 2
 
Post new topic   Reply to topic    PenUltima Online Forum Index -> General Help 096
Display posts from previous:   

Author Message
MontuZ
Distro Developer


Joined: 10 Feb 2006
Posts: 297
Location: Myrtle Beach, South Carolina

PostPosted: Sun Feb 18, 2007 5:25 pm    Post subject: Reply with quote

Okay, Barb. Cool

Author Message
Yukiko



Joined: 02 Feb 2006
Posts: 1094
Location: Southern Central USA

PostPosted: Mon Feb 19, 2007 11:57 pm    Post subject: Reply with quote

I never ignore the "Check Engine Light" in my car. Everytime that light comes on I always pull over and look under the hood. So far my engine has always been there.

Author Message
Gnafu



Joined: 02 Feb 2006
Posts: 88
Location: Livorno, Italy

PostPosted: Thu Feb 22, 2007 6:27 pm    Post subject: Reply with quote

That script is expected to go runaway because is wanted to do lots of things.
So obviously it can freeze the server.
To be sure to avoid errors, run it without pcs online.

I'm thinking if it's possible to iterate through the items directly in the storage file.... Question

Author Message
Barbeirosa



Joined: 01 Feb 2007
Posts: 43

PostPosted: Thu Feb 22, 2007 8:09 pm    Post subject: Reply with quote

Gnafu wrote:
That script is expected to go runaway because is wanted to do lots of things.
So obviously it can freeze the server.
To be sure to avoid errors, run it without pcs online.

Exactly, Gnafu!

Gnafu wrote:
I'm thinking if it's possible to iterate through the items directly in the storage file.... Question

It's certainly possible to write something to iterate through, parse, modify and save the altered data in the save files. However, I think it's easier (and safer - you needn't worry about errors in your parsing code) to simply set up POL specially to run with no connections, and simply do all the work and then do a world save and shutdown. Either way, you have to shut down POL and do this work. Doing it within POL itself gives you the benefit of POL's own parsing and saving of its data.

That said, for the times I have decided to write a program that parsed and modified save data, it goes MUCH faster than POL can do it, for some types of things. Probably playing with properties on items isn't a big deal, but one loop I had once done to set/erase properties on all the accounts in the game was actually taking a second or two per account simply to set and/or erase a handful of account properties. I think that's an issue unique to accounts and account properties, however, and does not affect any other POL objects. (perhaps because account changes are persisted to the file immediately?)

Author Message
Datus



Joined: 19 Sep 2006
Posts: 58

PostPosted: Tue Feb 27, 2007 11:01 pm    Post subject: Reply with quote

I once had the check engine light on in my car for about a month. I hooked up my homegrown OBDII scanner and discovered it was just a faulty O2 sensor. The sucker was siezed onto the intake manifold so I had to take a torch to that mother ... I installed the new one with that anti-sieze grease stuff...

Anyway my car is not a performance car and is a piece of crap ... just thought I could provide a useless analogy ...

Maybe sleeps are like that anti-sieze grease ... or maybe they're like my crappy car ... in any case, forget about riced out cars and get yourself one of these babies!

http://en.wikipedia.org/wiki/Lamborghini_Murci%C3%A9lago

Author Message
mr bubbles



Joined: 18 Jan 2007
Posts: 91

PostPosted: Wed Feb 28, 2007 1:04 am    Post subject: Reply with quote

That reminds me i gotta book my car in too. The check engine light has been on for about 3 months now Razz Havn't had the time and it is more than likely just faulty wiring like urs... I hope anyway Smile

Author Message
Yukiko



Joined: 02 Feb 2006
Posts: 1094
Location: Southern Central USA

PostPosted: Thu Mar 01, 2007 3:03 am    Post subject: Reply with quote

Mr Bubbles, try inserting some sleep function calls in there. That should make that check engine light go out.

Author Message
tekproxy
Distro Developer


Joined: 06 Apr 2006
Posts: 350
Location: Nederland, Texas

PostPosted: Thu Mar 01, 2007 11:17 am    Post subject: Reply with quote

SleepMS(2) works fine, as well.

And Austin probably wrote that right off the top of his head. Give him a break Very Happy

Author Message
MontuZ
Distro Developer


Joined: 10 Feb 2006
Posts: 297
Location: Myrtle Beach, South Carolina

PostPosted: Fri Mar 02, 2007 4:47 am    Post subject: Reply with quote

tekproxy wrote:
SleepMS(2) works fine, as well.

And Austin probably wrote that right off the top of his head. Give him a break Very Happy


Hehe. Yeah, I was just messing with him. He also had about a thousand spelling errors too that I didn't point out. Probably sober when he did it...

Author Message
Barbeirosa



Joined: 01 Feb 2007
Posts: 43

PostPosted: Fri Mar 02, 2007 6:51 am    Post subject: Reply with quote

tekproxy wrote:
SleepMS(2) works fine, as well.

And Austin probably wrote that right off the top of his head. Give him a break Very Happy


I think Austin wrote it exactly correctly. It would have been a mistake to have put sleeps in that script. But I guess it's easier to just assume someone else made a mistake, rather than even consider the possibility that you might actually be wrong.

Author Message
MontuZ
Distro Developer


Joined: 10 Feb 2006
Posts: 297
Location: Myrtle Beach, South Carolina

PostPosted: Fri Mar 02, 2007 8:16 pm    Post subject: Reply with quote

Barb, it's a joke.

Calm down, Very Happy

Author Message
Barbeirosa



Joined: 01 Feb 2007
Posts: 43

PostPosted: Fri Mar 02, 2007 8:42 pm    Post subject: Reply with quote

MontuZ wrote:
Barb, it's a joke.

Calm down, Very Happy


You first? Razz Laughing Very Happy Surprised Cool Cool Razz

Nah.. hell with calming down. Just start getting things right, and I'll be happy! Very Happy

Author Message
Gnafu



Joined: 02 Feb 2006
Posts: 88
Location: Livorno, Italy

PostPosted: Sun Mar 04, 2007 12:50 pm    Post subject: Reply with quote

Idea from storageWipe.src:

Code:

program storagewipe()
   
   set_critical(0);//without list line it runs fast, goes RunAway, makes megalag.
   var endnum, npctest;
   
   npctest := CreateNpcFromTemplate( "banker", 5445, 1153, 0  ,  0  , _DEFAULT_REALM );
   endnum := npctest.serial;
   RevokePrivilege( npctest, "invul" );
   SetObjProperty(npctest, "guardkill", 1);
   ApplyRawDamage( npctest, 510 );

   var i;
   for (i := 1; i < endnum; i:= i + 1)
       sleepms(250);//without list line it runs fast, goes RunAway, makes megalag.
       // DO WHAT YOU WANT TO DO HERE
   endfor

endprogram


The npc creation is used to get the last serialnumber to stop to.
The template must not necessary "banker".
Can it be usefull ?

Author Message
Barbeirosa



Joined: 01 Feb 2007
Posts: 43

PostPosted: Mon Mar 05, 2007 12:57 am    Post subject: Reply with quote

A couple notes about that code:

First, it should run critical. Why is simple: If not, other NPCs could be created which would go beyond that number. Perhaps it won't matter for the purpose that script was made for, though.

Second, though; that script will only iterate through Mobile serial numbers - not item serial numbers, which are all higher. Serial numbers for mobiles (including PCs and NPCs) start at 0x1, but serials for items start at roughly 1 billion. (0x40000000). If you want to cycle through ITEMS, you would set critical, create an item and get its serial, then iterate from 0x40000000 to its serial. However, even that won't work properly if you have a large and long-running shard, which has rolled over on the serial numbers (the serial number is a 32 bit integer, it seems by the way it's been split, so the max would be roughly 0x80000000). If it's remotely possible your shard has rolled over on serial numbers, code like the above would not at all be useful.

Third, there will be massive ranges of serials that do not exist due to item/npc destruction, resulting in lots of wasted effort trying to find objects that don't exist. Depending on what you are doing, why you are doing it, and when you are doing it through, that might not be a problem. But I think that a methodical way of going through all the items that do exist, such as Austin's code, is much better, because it avoids the overhead of trying to find literally millions, tens-of-millions or even hundreds-of-millions of items that do not exist.

And then, let's go back to that sleepms() thing; this is again why sleeps are a BAD idea in this type of script: You really need to run this sort of thing critical - get it done fast and have NOTHING else going on at all on the shard in the meantime.

A shard can easily go through, say, 200 million item serial numbers in a couple of years. How can I be so sure? Because I've seen it myself, first hand. If you use something like the below, and cycle instead from 0x40000000 to the last serial, and you have 200 million items to go through, with a sleepms(250) on each item, let's do some math:

200,000,000 items *0.250 seconds/item = 50,000,000 seconds of sleeping, alone (not counting actual code running, nor scheduling time)
50,000,000 seconds = 833,333.33... minutes
833,333.33... minutes = 13,888.88... hours
13,888.88... hours = 578.703... days.

So, let's be clever and reduce it to only sleepms(1). Those who know anything about this sort of thing know that you'll always get a much longer sleep than 1 ms from this - more likely you'll get something from 2-10ms, with a good round number being 3ms, depending on your system. For the sake of argument, let's aim right at the bottom of that range and say you'll get an entirely perfect outcome every time of 2ms of actual delay. To get our final number then, we simply divide our last result by 125:

578.703... days / 125 = 4.629... days


So... assuming that code ran perfectly, the sleeps alone on even a small shard that's been running a couple years would take over four-and-a-half days. That's not even including the actual code running.

Post new topic   Reply to topic    PenUltima Online Forum Index -> General Help 096 All times are GMT - 4 Hours
Goto page Previous  1, 2
Page 2 of 2

 




Powered by phpBB © 2001, 2005 phpBB Group :: Theme & Graphics by GHS & Scott E. Royalty