Merchant Archive

Thread: The REAL Database Issue Found?

Naufragus
Tue Mar 09, 2004 9:49 am
#1

Read Blunderheartsgun thread in the "IN LIVE" forum...



if he has his information correct it appears that the database tracks ALL resource and subcomponent stats on items after the final combine. It doenst just track the items final stats but those of everything used to build it.


So lets say i make a fusiion reactor...The database is tracking the statsof the ore, metal and chemuse to make the final combine, then the resources and stats of the wall modules, the resources and stats of the harvesting mechinism and the generater turbine and then there is the manufacturing mechanism which takes other subcomponents to build whoese resources and stats must be tracked... so just this one item could easily generate over 50 entries that must be tracked FOREVER...i can only shudder at how many entries msut be saved for something like armor.


This sounds like to me that this is what is bogging down the database....not items on vendors


Thoughts?
rexan
Tue Mar 09, 2004 12:43 pm
#2

Wow, thats incredibly short sighted of them i would think.


Do you have a link?





Rexan Ryu
Master Smuggler
Flurry Server
Haruspex77
Wed Mar 10, 2004 4:38 pm
#3







rexan wrote:


Do you have a link?




I do, thestatement is in:


http://forums.station.sony.com/swg/board/message?board.id=weaponsmith&message.id=34558&highlight=crate+fwg5#M34558


Note that it relates only to crates, not to every item. Apparently the crate implementation is that "When you pull an item out of a crate it basically does an "insta-craft" on the spot." While TH has been known to get things wrong, this is an explantion of how the "can't get pre-nerf guns out of crates" bug worked, and that bug was fixed.


One practical implication of that information is that if you are going to stock something up that is about to be nerfed, take them all out of crates. The guns now apparently come out of the crate nerfed.


Clearly the implementation of the game is far more complex than we ever expected. That is not a good thing.

Undergrid
Wed Mar 10, 2004 5:18 pm
#4

This is how I understand what TH said (IMO)

Crates don't actually have items in them, simply a record of the number of items and the stats of that item. When you pull something out of a crate it creates an item using the stats in that record, it doesn't actually combine resources into components and again into the final crate item doing expermentation etc in the process otherwise you'd never get the same item out of a crate twice. As a software engineer this is how I'd do it anyway, any other way is just stupid.





Undergrid (Chimaera)
Master Smuggler / Master Pistoleer / Teras Kasi Master / Master Pilot / Rebel Colonel

Haruspex77
Wed Mar 10, 2004 5:53 pm
#5






Undergrid wrote:
This is how I understand what TH said (IMO)

Crates don't actually have items in them, simply a record of the number of items and the stats of that item. When you pull something out of a crate it creates an item using the stats in that record, it doesn't actually combine resources into components and again into the final crate item doing expermentation etc in the process otherwise you'd never get the same item out of a crate twice. As a software engineer this is how I'd do it anyway, any other way is just stupid.




That would make sense, but apparently there is more to it than that. The evidence is in the bug he was talking about, where the crate could not create the item because the "schematic" no longer existed. It had been redesigned as part of the pistol nerf.


Now apparently, there is some detail about the item that is not visible to the user in the data that the crate uses. Some of this might well relate to what was used in its creation, as that was the only difference between the old and new versions other than statistics.


I suppose another possibility is that there were some checks put in at the creation, like the checking of petsfor validity whencalled, that the item failed. I would hope that they had learned their lesson with the BE pet fiascos, and didn't do that.

JTGAlpha
Wed Mar 10, 2004 6:03 pm
#6

This is also how we have such a rich crafting system where resource quality matters. Resource quality is probably WHY these things must be maticulously tracked. I could be wrong of course, I'm just guessing here. However it seems to me that while this IS a complicated system it is also this complexity which allows for the varied goods on the market, and the player run economy. The resources have to change periodically. Schematics then become obsolete creating a need for NEW schematics made from different resources. Without this complicated system we are all swinging the same +2 damage sword of buggery, instead of Jiist's DX2 or Hephaestos's Pre-Nerf Krayyt FWG5.



Dayasi Vo'Boda CEO of SCUM PA.
Founders of Agrilatia in the Agrilat Swamps Of Corellia (Intrepid).
Taking Scum and Villainy to a new Level to Serve YOU.
Remember: SCUM does it dirty
Another Horseman of the Smuggling Apocolypse

SueDenim
Wed Mar 10, 2004 9:40 pm
#7

Mentally reverse-engineering what the Oracle database design might be here....

It could have one of two explanations based on the explanation:

1. the "horrible design" hypothesis - every time you pull something out of a crate, it literally crafts it from scratch, looking up every single separate ingredient, the resource qualities of each, etc. This would be really bad, as once a schematic is created, you know its final stats, and that's all you *need* to know. Once the schematic's made, we should quit caring about the conductivity of the Thoranium Steel that went into it, etc. And there should be just one record the database needs to consult - the stats of a given schematic with a particular serial number that's on the crate.

2. the "not too awful" hypothesis - the "insta-crafting" process does, in fact, just look at the stats of that serial numbered schematic. *But*, it might choke on the "This schematic isaversionof This base item."

I could see something like this happening. It might be that there's a table of "Base Weapon Schematics." And the "FWG5 Pistol Mark 1" had a unique id number of, say, 15, in that table. But later, they nerf the FWG5 and create a new base schematic for it, the "FWG5 Pistol Mark 2," which has a unique id number of, 36, and delete the Mark 1 record. And the crate might be looking for "schematic #15" and not finding it, and thus choking because #15 no longer exists. This explanation would be plausible, and indicative of some bad practices, but not outrageously incompetent ones - basically, the story would be that they didn't anticipate one particular consequence of changing a "base" schematic.



Bindi Kicklighter
Kicky Fashions of Kor Vella (-3403, 1395)
Visit our "Hall of Justice" display!
Kor Vella, Corellia, Lowca
Mohr
Thu Mar 11, 2004 5:18 am
#8






SueDenim wrote:
Mentally reverse-engineering what the Oracle database design might be here....

It could have one of two explanations based on the explanation:

1. the "horrible design" hypothesis - every time you pull something out of a crate, it literally crafts it from scratch, looking up every single separate ingredient, the resource qualities of each, etc. This would be really bad, as once a schematic is created, you know its final stats, and that's all you *need* to know. Once the schematic's made, we should quit caring about the conductivity of the Thoranium Steel that went into it, etc. And there should be just one record the database needs to consult - the stats of a given schematic with a particular serial number that's on the crate.

2. the "not too awful" hypothesis - the "insta-crafting" process does, in fact, just look at the stats of that serial numbered schematic. *But*, it might choke on the "This schematic isaversionof This base item."

I could see something like this happening. It might be that there's a table of "Base Weapon Schematics." And the "FWG5 Pistol Mark 1" had a unique id number of, say, 15, in that table. But later, they nerf the FWG5 and create a new base schematic for it, the "FWG5 Pistol Mark 2," which has a unique id number of, 36, and delete the Mark 1 record. And the crate might be looking for "schematic #15" and not finding it, and thus choking because #15 no longer exists. This explanation would be plausible, and indicative of some bad practices, but not outrageously incompetent ones - basically, the story would be that they didn't anticipate one particular consequence of changing a "base" schematic.





If Number 2 is the hypothesis then this could be fixed via a simple SQL Statement


"Update ItemTable Set SchematicID = 36 Where SchematicID = 15"


I know things are never this simple but I would say I have two hypothesis for this fix.


1. the "horrible designer" hypothesis - they just don't know how to do the above


2. the "lazy designer" hypoethesis - they just don't feel like doing this


I know when I have lost items in the past (that were crated) due to inventory bugs the CSRs wanted to know all the materials that were in the items or whether I still had the schematic. At the time I thought this was a somewhat odd request since to refund the item I would have thought it would have been easier to "create" the item. In light of ThunderGuts post this might make more sense.


SueDenim
Thu Mar 11, 2004 8:13 am
#9

The overall signs, and my real-world experience, suggest to me that SOE hired outside consultants for their Oracle work (a very common practice, and often a wise one), but got stuck with really incompetent and lazy ones (which are also not all that uncommon in the field.) All the earmarks of this sort of dynamic are there, and it wouldn't surprise me if the answers given to SOE people (possibly not Oracle-savvy enough to supervise the consultants well) by the consultants are usually "No. It'd be impossible to change this on our end. It's the users' fault, they're exploiting the database in ways you didn't tell us to anticipate. You have to make the database smaller by destroying lots more items, that's the only solution."

When the real answer is probably "We [EDIT]ed up the database in the design stage. And fixing an Oracle database that's got a fundamentally screwed-up design is really time-consuming and expensive and disruptive. It's also usually *far* superior to doing and endless stream of band-aid fixes like we're suggesting to you. But if you took that path, we're probably not competent enough to do the job right the second time any more than we were the first time. So you'd probably fire us and hire someone competent. Which would suck for us, because we've got a good gig here. So we'll tell you the band-aid fixes are the only technically feasible solutions, and hope our gravy train doesn't get disrupted for a long, long time."



Bindi Kicklighter
Kicky Fashions of Kor Vella (-3403, 1395)
Visit our "Hall of Justice" display!
Kor Vella, Corellia, Lowca
SueDenim
Thu Mar 11, 2004 8:19 am
#10



Mohr wrote:


I know when I have lost items in the past (that were crated) due to inventory bugs the CSRs wanted to know all the materials that were in the items or whether I still had the schematic. At the time I thought this was a somewhat odd request since to refund the item I would have thought it would have been easier to "create" the item. In light of ThunderGuts post this might make more sense.






Actually, I think from the "CSR refund" perspective, this might make sense. If I understand how they do refunds (and I may not, this is based on second-and-third-hand knowledge), they just refund based on the market value of the resources that went into crafting the item (presumably with some multiplier.) Which isn't necessarily unreasonable. In a player-run economy, estimating, or creating a tool for CSRs to use to estimate, "fair market value" for each of the thousands of items that can be made, all with various permutations of stats, would be really difficult. But estimating fair market value of *resources* - that's a more manageable task. If they do it right, it's a reasonable compromise, and wouldn't necessarily imply that insanely awful database design is the rationale forcing that compromise.



Bindi Kicklighter
Kicky Fashions of Kor Vella (-3403, 1395)
Visit our "Hall of Justice" display!
Kor Vella, Corellia, Lowca
PepeGSay
Thu Mar 11, 2004 11:58 am
#11






Naufragus wrote:

Read Blunderheartsgun thread in the "IN LIVE" forum...



if he has his information correct it appears that the database tracks ALL resource and subcomponent stats on items after the final combine. It doenst just track the items final stats but those of everything used to build it.


So lets say i make a fusiion reactor...The database is tracking the statsof the ore, metal and chemuse to make the final combine, then the resources and stats of the wall modules, the resources and stats of the harvesting mechinism and the generater turbine and then there is the manufacturing mechanism which takes other subcomponents to build whoese resources and stats must be tracked... so just this one item could easily generate over 50 entries that must be tracked FOREVER...i can only shudder at how many entries msut be saved for something like armor.


This sounds like to me that this is what is bogging down the database....not items on vendors


Thoughts?






This statement devines a whole lot... and while it may be logically true I would bet the design is far different than you imagine. for instance:

The database is tracking the statsof the ore

More likely it is tracking that it was made from a resource called 'X'. There is an entry in the DB for every resource that ever existed, and from there it getsthe stats. In addition every item has a serial number, so your crafted item was made with X,Y,Z sub components, those are in the DB somewherer too with all the particulars. But your schem for the fusion doesn't duplicate it all it just references it.


Net result they may track a bunch of items for the crated items, but it is not quite as bad as you think.. and might actually improve performance in some ways.



Tiquor Canche - Intrepid
Stim City: Naboo, Moenia 5326 -4084 Meds & A Ton of Other Merchants
Proud Member of SIN (Strength In Numbers)

+20 Medical, Combat Medical & Chef Experimentation
"When your HAM is looking shi&&y, you'll be glad you shopped at Stim City"
Merchant Email Processor available at http://www.thedotcommakesitfunny.com/swg/ Now on Alpha 11.

Targetam
Sun Mar 14, 2004 1:56 am
#12

A small amount of additional evidence for you to consider ...


A couple of months ago, whent eh Architect class was revamped, and we went from BER 7 to BER 10 Heavy harvesters, something very odd happened to the Fusions - the ones that were in backpacks shot up to BER of 18, for the most part. Then I found one that had a BER of 25. Apparently, the architect actually experimented that fusion all the way up to the maximum, and the SQL script that SoE used to update the "backpack" schematics really goofed up


I love having a 400,000 unit hoper size, for example.


The point, though, is that material quality and experimentation *had* to be retained, somewhere, for that 25 BER fusion to exist. And it had to be there from back in the day when most architects took the lower quality stuff because they could get identical results with it that they could get from the godd stuff.






Arget Cotusa <Sanct>
a.k.a. Vock Opib <Sanct> Retired 2004-12-19 (12-point Master WS) (11-point Master Architect)
d/b/a Zaracorp Weapons and Architecture
Please contact Palo for Weaponsmith, Carra for Architect orders.

Page 1 of 1
Previous Next