Merchant Archive

Thread: Database 101

shadowdefender
Tue Aug 10, 2004 8:45 am
#14

I am a Microsoft database administrator and software developer, I know for a fact that if SOE does 1 thing on their list of merchant changes their performance will improve at least 85%.

Remove the vendors from those that do not have the skills to maintain them. 85% of problems gone! It really is that simple.

They should do this first... load test etc., then if that doesn't work (but it will) they should then implement vendor limits. I am not against vendor limits btw - I am against the ludicrously low limits they proposed. As a Master Tailor you cannot have a nice stock with 100 - 200 items, even at 660 that is stretching it. If they went up to 1500 items for Master that would be 100% better than what they are proposing.



Desire Storm-Shadow'fyre-JuliaChilds
Mayor of Stargate & Dancer-AFLAC Guild Leader - Swordsman-Tailor & Architect & Merchant
Wanderhome Starport Crawl 4/3/05 - Don't Miss It!!! - Stargate

JutMan
Tue Aug 10, 2004 9:13 am
#15

Databases 102 (Advanced)

1. Databases have auto-grow feature typically based on a % or static number.
2. You can have multiple databeses spread across multiple servers to enhance speed and spread the storage requirements.. (Ever login right after the servers come up and the bazaars are not working but the game is?)
3. Databases are typically "indexed" to speed data retreival.
4. Each planet and it's storage are probably completely different databases lightly tied together through "join" statements when using the Bazaar. All database requests other than the Bazaar do not need to span servers. (You can only have a bank deposit box on one server at a time, Structures cannot be queried from a different server, charcters database is attached to a independant server queriable by all servers)
5. The "Vendor" database is probably the biggest because of the number of items so it is suspect. Vendor database is probably also a "per server" database with most utilization Coronet, Naboo, & Dantooine.

The "purge" of cancelled accounts will reduce the Charcter, Structure, and Item Databases. The Vendor database it "current activity" only so cannot be purged. The only way it can be reduced is by the current test server implementation of Merchant "nerf". If the Vendor database is NOT server independant then it is an even bigger problem.

Estimate:
1000 Vendors
1000 Items
10K per recods
/ 1025 (for MB conversion)
Equates to 9.7 GB Database per Planet assuming the Vendor database is "per server"
If it is NOT per planet it would be 97 GB while large is managable.



Taywen O'Shay~ Master Bio-Engineer / Master Chef~
Molukai~ Bothan Jedi Knight ~

~TRGA~ Always have been.. and will be
Kyrien
Tue Aug 10, 2004 9:25 am
#16

Lol, Now that's funny....


Kyrien



Kyrien - Crafting Process Engineer
Wryrur - Big Furry Ranger

Kyrien's Crafting and Tools
Crafting, Repair & Survey Tools
Crafting Stations and Factories
Sparta Architecture Shop (-1864 -5810)
Startsider - Tatooine - Sparta
Shann0w
Wed Aug 11, 2004 5:23 pm
#17






shadowdefender wrote:
I am a Microsoft database administrator and software developer, I know for a fact that if SOE does 1 thing on their list of merchant changes their performance will improve at least 85%.

Remove the vendors from those that do not have the skills to maintain them. 85% of problems gone! It really is that simple.

They should do this first... load test etc., then if that doesn't work (but it will) they should then implement vendor limits. I am not against vendor limits btw - I am against the ludicrously low limits they proposed. As a Master Tailor you cannot have a nice stock with 100 - 200 items, even at 660 that is stretching it. If they went up to 1500 items for Master that would be 100% better than what they are proposing.



The biggest complaint I've had about SWG devs since beta is their approach to changes. They don't look to see if a firecracker or even a hand grenade will do the trick, they go straight tomultiple 1 million megaton fusion bombs.

Zed-TORA
Wed Aug 11, 2004 5:39 pm
#18






Baelin_Radiant wrote:

SOE makes roughly $4,500,000 a month($54,000,000 per year) on SWG. How much database does that buy?







Actually they are making about 18 million a month (256 million a year) based on the Figure of 1.2 millon active accounts given at fan fest..... Less expence still equals Holy begebers...



As for the rest of this conversation i understand a bit about DB programing structure and performance. Until anyone can make an argument that SWG has more lines than google in there DB you can take all the DB theory and throw it away. In a billion+ line DB at google (More like 10's of billions) a simple query will yield a result in 2 seconds or less with hundreds of thousands of simultaneous queries being executed at the same time. The excuse that it is not possible to have a data base this large run effective is a crap argument.




It's amazing what a man can accomplish when he's not worried about who will get the credit.
Zed
TORA
Dsabre
Wed Aug 11, 2004 5:49 pm
#19

vendor storage isn't whats taking up most of the space...it may help contribute to it...but what it ultimately comes down to is "unique items".

MMO's are ALL based on the same basic concepts first seen in MUD's, MOO's, MUSH's, etc...(for anyone that doesn't know what these are...think text based EQ, where each "zone" is a room...and no distance other than number of rooms away, etc).

in these MUD's, and such...there is almost always something called a "donation pit"...this is where all the junk that people want to get rid of...but not necessarily destroy goes via a simple "donate x" command. these pits can grow to HUGE sizes depending on how stable the MUD is, or how popular it is.

this is sort of like an SWG vendor (without the ability to categorize, or sort tho). thousands and thousands of items can be found in some of these pits...and MUD's typically don't have more than 5 MB of storage space...so how is it that they could have no problems with these pits...but SWG have problems with vendors...

simple...unique items.

each item that is created must be issued a serial number...and is probably stored in a table set aside for items...most likely broken down by item type (weapon, food, armor, etc). when crafter A makes a run of 100 VK's (if this isn't possible sorry...not a WS myself) there is 1 new entry into the dB (this is the shortened and widely used way to say database). this detailed table would contain things like stats...damage, ham costs, etc....but again 1 entry only for all 100 of these items. when these items are distributed to a vendor a less detailed table entry would be needed...simply serial #, vendor ID, and a few other things...these entries are ultimately a lot smaller than the more detailed table with the stats.

same thing with MUD's...and likely EQ...each item has set stats, and has its own detailed entry...doesn't matter that 150000 players have 1 or more of these...1 entry in that detailed table is all it gets.

problems happen when unique items are created...in MUD's this would be most common when "restringing" an item so its more personalized. In SWG this would come in the form of DoT's, skill mods, and enhanced items in general. Even something so simple as renaming a backpack, or inserting a CA/AA requires that the item be treated as "unique".



essentially the argument that vendor item caps will alleviate dB strain is true...but only in the sense that less "loot" vendors will exist to sell/store those unique items (once their destroyed...their entry can be deleted from the table). arguments like "take out all the broken datapads" won't help alleviate server strain...to do that would be like 1 person deleting 1 of their enhanced NS lances...wouldn't help much at all.

if they really wanted to cut dB size...make the character purge cover all accounts deactivated for more than 3 months...1 character file probably takes up more space than a mall full of stocked vendors (gotta figure in that most characters have a significant amount of loot, or the last of a particular crafted item).
Reaperss
Wed Aug 11, 2004 5:50 pm
#20

they said at fan fest they have 1.2 million accounts?



Ugo Deathbringer
-X Master Droid Engineer, X Master Merchant, X Master Artisan,
Don't hate da playa or da game, hate the Devs
Warn you we did, listen you did not, now screwed we all are
Aeja
Wed Aug 11, 2004 6:23 pm
#21

Well Im an unemployed CIS major anyone able to get me a job will work fairly cheap .
rexan
Wed Aug 11, 2004 6:38 pm
#22






Dsabre wrote:
vendor storage isn't whats taking up most of the space...it may help contribute to it...but what it ultimately comes down to is "unique items".






Unique items are not the problem since each player has a limited amount of storage space. Each player is limited to 10 lots, 1 bank, +personal inventory, droids.

1000 (10 lots at 100 items)

110 (60 items personal inv plus a 50 item backback)

100 (saftey deposit box at the bank)

50 (5 droids with max storage)

------

1260 items total per character



Any database should be able to handle this amount of data per player.


The problem will come when you break this limit by using vendors for unlimited storage. Especially since vendors can currently be exploited because they require no skill points to maintain.





Rexan Ryu
Master Smuggler
Flurry Server
Zed-TORA
Wed Aug 11, 2004 6:59 pm
#23






Reaperss wrote:

they said at fan fest they have 1.2 million accounts?






Yup... That number came from the lucas arts rep there i belive . The numbers were fairly insaine like at any given time there were 15 thousand people logged in or something like that and peek hours it went way higher. Im sure someone else that was at fan fest wrote the numbers down. At the time i really didnt care i had canceled my account was just there with my wife and some guildies.



It's amazing what a man can accomplish when he's not worried about who will get the credit.
Zed
TORA
Reaperss
Wed Aug 11, 2004 7:02 pm
#24

that can't be an accurate number, I think that would be more than EQ which they would have bragged about all over the place. It has to be 400,000 - 500,000 max.



Ugo Deathbringer
-X Master Droid Engineer, X Master Merchant, X Master Artisan,
Don't hate da playa or da game, hate the Devs
Warn you we did, listen you did not, now screwed we all are
Zed-TORA
Wed Aug 11, 2004 7:37 pm
#25






Reaperss wrote:

that can't be an accurate number, I think that would be more than EQ which they would have bragged about all over the place. It has to be 400,000 - 500,000 max.






Ya but i thought in EQ you could have more than 1 char per server and OH BTW There first month of sales was 290.000 that is in a press release.
SWG is bigger than EQ and before SWG was Lineage 1 if you go to there web page at one point lineage had over 1 million accounts world wide.... EQ WAS the largest selling 3D mmo cause lineage was 2d like ultima..



It's amazing what a man can accomplish when he's not worried about who will get the credit.
Zed
TORA
Zed-TORA
Wed Aug 11, 2004 7:41 pm
#26






Zed-TORA wrote:





Reaperss wrote:

that can't be an accurate number, I think that would be more than EQ which they would have bragged about all over the place. It has to be 400,000 - 500,000 max.






Ya but i thought in EQ you could have more than 1 char per server and OH BTW There first month of sales was 290.000 that is in a press release.
SWG is bigger than EQ and before SWG was Lineage 1 if you go to there web page at one point lineage had over 1 million accounts world wide.... EQ WAS the largest selling 3D mmo cause lineage was 2d like ultima..




I think that EQ claims or claimed the highest amount of Individual users not most accounts.


And since we are having a history lesson and yes i know ultima was a 2d game but ANYONE could have a vendor and if i remember and its been a long time there was no cap at all on vendors there and im assuming from what i know about data in the end its all just bits and bytes in a DB regardless of the dimension





It's amazing what a man can accomplish when he's not worried about who will get the credit.
Zed
TORA
Page 2 of 3