Bio Engineer Archive

I know the reason we dont have a special bag for dna is because it takes up so much room in the database but dont see a reason for this. I had an idea (its pretty simple so I am sure someone else has had it) about reducing the amount of storage space dna takes up in the database.

The way I see it there should be 2 types of dna, one type is sampled from wild creatures which needs to take up very little space and one type sampled from BE creatures which does need 2 take up space. From the dev responce I think that dna at the moment stores each and every value we see but I was wondering if there was a better way.

The way I see it there are 2 ways to reduce the amount of data used in storing wild creature dna.

a) Every type of sample from the same creature could be made exactly the same, all the same stats and the same serial number For example say you sample from a rancor, there are 5 types of dna you could get, VHQ, HQ, AA, A and Poor. Why not have a single master entry in the database for each of the 5 qualities of sample, then if you get a VHQ sample all that needs to be stored is the wild creature dna id, the rancor id, and a number from 1 to 5. This means all VHQ samples will be exactly the same, with the same serial num so they can stack in inventory (and even be used in facories!)

So in short: For each creature there are 5 master samples, one for each quality type All sampled dna references a master sample and carries no other stats All samples at the same quality from the same animal are exactly the same so they stack

b) More like the system we have at the moment, you have one master sample in the database for rancor dna, when you sample you make a roll from 1 to 100. A result of 1-20 will give you a poor sample, 20-40 will give average etc etc. When the resulting sample is stored in the database it is stored as follows: . This would be more like the current system (all VHQ samples will not be the same). When it comes to the stats of the dna you just sampled, you take the result of the roll you made while sampling and use it as a percentage against the master dna sample. For example say to made a roll of 98% then you take every stat from the master sample and multiply it by .98 to get the stats of your sample.

So in short: For each creature there is 1 master sample All dna references the master sample along with a percentage All samples at the same quality from the same animal are still different We'll need a dna storage bag of some form, which will only take wild creature dna, NOT dna sampled from BE creature

You should be able to implement either of the above systems fairly easily, mark all current dna as BE creature dna and any new dna you get is marked as wild creature dna. You just make a new type of object for wild creature dna and change the current object from dna to BE creature dna. Like I said the idea is fairly simple, I though this up during the long scilences during a sampling session last night.

I'd love to hear what u guys think and maybe this could be passed to the devs?

PinZip

---

PinZip wrote:
I know the reason we dont have a special bag for dna is because it takes up so much room in the database but dont see a reason for this. I had an idea (its pretty simple so I am sure someone else has had it) about reducing the amount of storage space dna takes up in the database.

The way I see it there should be 2 types of dna, one type is sampled from wild creatures which needs to take up very little space and one type sampled from BE creatures which does need 2 take up space. From the dev responce I think that dna at the moment stores each and every value we see but I was wondering if there was a better way.

The way I see it there are 2 ways to reduce the amount of data used in storing wild creature dna.



a) Every type of sample from the same creature could be made exactly the same, all the same stats and the same serial number For example say you sample from a rancor, there are 5 types of dna you could get, VHQ, HQ, AA, A and Poor. Why not have a single master entry in the database for each of the 5 qualities of sample, then if you get a VHQ sample all that needs to be stored is the wild creature dna id, the rancor id, and a number from 1 to 5. This means all VHQ samples will be exactly the same, with the same serial num so they can stack in inventory (and even be used in facories!)

So in short: For each creature there are 5 master samples, one for each quality type All sampled dna references a master sample and carries no other stats All samples at the same quality from the same animal are exactly the same so they stack


Yuck! This would take alot of the fun of getting samples, and place pet making one step closer to crafting anything else. Kinda like having all weaponsmith resources have the same stats. You'd end up with just a few different recipes that were worth anything.


b) More like the system we have at the moment, you have one master sample in the database for rancor dna, when you sample you make a roll from 1 to 100. A result of 1-20 will give you a poor sample, 20-40 will give average etc etc. When the resulting sample is stored in the database it is stored as follows: . This would be more like the current system (all VHQ samples will not be the same). When it comes to the stats of the dna you just sampled, you take the result of the roll you made while sampling and use it as a percentage against the master dna sample. For example say to made a roll of 98% then you take every stat from the master sample and multiply it by .98 to get the stats of your sample.

So in short: For each creature there is 1 master sample All dna references the master sample along with a percentage All samples at the same quality from the same animal are still different We'll need a dna storage bag of some form, which will only take wild creature dna, NOT dna sampled from BE creature


This is better, but it's pretty much what already being done.

You should be able to implement either of the above systems fairly easily, mark all current dna as BE creature dna and any new dna you get is marked as wild creature dna. You just make a new type of object for wild creature dna and change the current object from dna to BE creature dna. Like I said the idea is fairly simple, I though this up during the long scilences during a sampling session last night.

I'd love to hear what u guys think and maybe this could be passed to the devs?

PinZip

---

DNA doesn't take up much space anyway. Broken down we have 9 resists, 10 stats, 2 special attacks, a name, and a quality to store on any given DNA sample. All of these can be stored as a single integer and referenced from a list on the client machine (your PC). That gives us 46 bytes of data to store int he base table per sample. I know that it all adds up, but the storage requirements just aren't that big. It's not the storage of this data, it's the moving it around from server to client and back that's the problem. Bandwidth is another problem altogether. And with this it's not even the bandwidth that the client has. It's just having to deal with all the people on at once on the app server side of things that's going to get messy.
Even with this set aside, if you have a master table that you do lookups from you still need a lookup table to know who has how much of what. Depending on how this table is implemented, it can be faster access to just store it in one table (you can get it all into a few blocks on disk and in the SGA).
Ok, DBA hat off head again, back to dumb user mode. Why not just have DNA samples as data in the data pad instead of items in inventory? We could even have a special tool that only gets used for DNA and creature making. THis would allow us to store a bunch of DNA in droids.

---

PlainWhiteSocks wrote:

Yuck! This would take alot of the fun of getting samples, and place pet making one step closer to crafting anything else. Kinda like having all weaponsmith resources have the same stats. You'd end up with just a few different recipes that were worth anything.

---

Not really. all weaponsmith resources stack so long as they are the same type. when you buy 1k of resources you don't actually get 1000 seperate items, u get a stack of 1000. I have been led to believe, the way things are now, the stats of your DNA is within a range of the 'average' for that critter.

I personally would have DNA stackable in a similar fashion to above, but not *quite* the same...

I would have quality dependant on skills, rather than skills+luck. ie a Novice will get V low Qual, Master will get VHQ.

Then, all creatures share the exact same DNA template as their siblings. ie all rancors have the same stats, which is different to all merekDNA (which is the same for all mereks) etc.

This would allow stacking of DNA from the same creatures with only minor modifications to code and DB.

If they really wanted to be clever, they couldmake the DNA shift like normal resource spawns, so it doesn't stack for ever.

The reason they don't allow DNA stacking at the moment is that it would take up more space in the DB. Well both of these solutions would stop that issue.

---

PlainWhiteSocks wrote:
DNA doesn't take up much space anyway...

---

---

LloydPickering wrote:

---

PlainWhiteSocks wrote:

Yuck! This would take alot of the fun of getting samples, and place pet making one step closer to crafting anything else. Kinda like having all weaponsmith resources have the same stats. You'd end up with just a few different recipes that were worth anything.

---

Not really. all weaponsmith resources stack so long as they are the same type. when you buy 1k of resources you don't actually get 1000 seperate items, u get a stack of 1000. I have been led to believe, the way things are now, the stats of your DNA is within a range of the 'average' for that critter.

I personally would have DNA stackable in a similar fashion to above, but not *quite* the same...

I would have quality dependant on skills, rather than skills+luck. ie a Novice will get V low Qual, Master will get VHQ.

Then, all creatures share the exact same DNA template as their siblings. ie all rancors have the same stats, which is different to all merek DNA (which is the same for all mereks) etc.

This would allow stacking of DNA from the same creatures with only minor modifications to code and DB.

If they really wanted to be clever, they could make the DNA shift like normal resource spawns, so it doesn't stack for ever.

The reason they don't allow DNA stacking at the moment is that it would take up more space in the DB. Well both of these solutions would stop that issue.

Lloyd Pickering

Master Bio-Engineer

Chimaera

6276, 1923 Sapphire Lake, Naboo

---

I knew I wouldn't get my point across well with that statement. hmmmm. If all DNA samples from a specific creature are the same then your recipes would always make the same exact pets with the same stats every time. Eventually it would get boring. Instead of trying to get good high stat samples of creatures you'd just get samples. It would take some of the challenge away. As it is today, using the same recipe for two pets, the final stats for the pets can vary quite a bit (+/-1k of ham, +/-.1 to hit, +/-30 points of damage). In the system you proposed the stats would always be the same.

What I meant with the weaponsmith example, was what if they never changed the stats on normal resources spawns? So all xyz aluminum had xyz stats, and all abc steel had abc stats , and it never changed. Maybe it's just a bad analogy.

---

LloydPickering wrote:

There are plenty of ways around this however. My suggestion is just one.

---

Well the blood sample suggestion a while back was probably the best - this is where you would get a blood sample from an animal and keep it. Whenever you want to get a DNA sample from it you could. Obviously eventually this supply would perish and you would need a new sample.

Message Edited by ArthurDentOnBria on 11-10-2004 11:36 AM

---

ArthurDentOnBria wrote:

But then the implement a crazy loot system in JTL which generates an amount of "crap" that is just staggering. I think soon junky ship parts will take up like 99% of the bazar.

Message Edited by ArthurDentOnBria on 11-10-2004 11:36 AM

---

This quite funny Arthur....

I have been using my yacht to store space loot. One player I took up in the yacht exclaimed, "Your yacht looks like a flying Autozone! Could you point me to the Windshield wipers sectionplease?"

Honestly, I really don't think Database storage is as high a priority as one would expect....regardless of what we have been told by the devs. I think this is one of those areas of IT that will have to splatter like a giantgrasshopper on your motorcycle helmet for it to even get on the radar of things that need more attention.It would be a fair assumption to think thefolks responsible, not necessarily the devs,for this sort of thing are keeping an eye out forover-utilization. If they aren't, .....then at some point there will be a new job posting on SOE website for Database Director.

As for the original poster...

I thinkmost ideas for streamliningstorage area move in the right direction. The ability to put like DNA in a factory and crank out a certain number of the same generic DNA templates does soundappealing at times....but then, if I want to crank things out of a factory, tissues always seems to suffice. One of the lovely things about our profession is the very complex clone system. I am oddly pleased at times with the way it works and frustrated as hell other times.

However, I can certainly appreciate your thoughts in improvinga process.

---

Bendi_James wrote:

---

ArthurDentOnBria wrote:

But then the implement a crazy loot system in JTL which generates an amount of "crap" that is just staggering. I think soon junky ship parts will take up like 99% of the bazar.

Message Edited by ArthurDentOnBria on 11-10-2004 11:36 AM

---

This quite funny Arthur....

I have been using my yacht to store space loot. One player I took up in the yacht exclaimed, "Your yacht looks like a flying Autozone! Could you point me to the Windshield wipers section please?"

Honestly, I really don't think Database storage is as high a priority as one would expect....regardless of what we have been told by the devs. I think this is one of those areas of IT that will have to splatter like a giant grasshopper on your motorcycle helmet for it to even get on the radar of things that need more attention. It would be a fair assumption to think the folks responsible, not necessarily the devs, for this sort of thing are keeping an eye out for over-utilization. If they aren't, .....then at some point there will be a new job posting on SOE website for Database Director.

As for the original poster...

I think most ideas for streamlining storage are a move in the right direction. The ability to put like DNA in a factory and crank out a certain number of the same generic DNA templates does sound appealing at times....but then, if I want to crank things out of a factory, tissues always seems to suffice. One of the lovely things about our profession is the very complex clone system. I am oddly pleased at times with the way it works and frustrated as hell other times.

However, I can certainly appreciate your thoughts in improving a process.

---

My first responce to any change to a database at work is "No". I imagine that if I worked there as a DBA that I'd give the same answer. Sometimes this attitude gets me into a little hot water, and I know it has sent a few projects to their deaths in early stages. Maybe they have something like that going on at SOE. At any rate I still think it's a bandwidth issue instead of a storage issue, but it's much easier to explain as a storage issue.

---

ArthurDentOnBria wrote:

There have been lots and lots and lots of threads about dna management proposals. I think the most popular ideas involved some type of system by which you aquire some sort of generic stackable thing (blood or whatever) for optimal storage, and then you extract the unique dna samples at the time when you need to use it, so you get the diversity and the storage both.

Wow! I must have missed the post on this one. Excellent idea.

I wanna be a bloody BE

Well the idea I posted would realy take up 0 bandwidh when you compare it to say character models, simple store the master samples on the clients hard drive once and from then on your talking about a few bytes for each dna sample which should only be transmitted when u make a sample or log in. Now compare that to walking into a busy city with 10s of players on screen (hell a long buff line can easly hold 10ppl) and now you have to transfer all the data for character appearance, and this happens WAY more often than sampling dna. As for not messing with a live database I would be as unwilling to do it as any sane person, the difference is I'd make sure my database model wasn't going to need to be changed BEFORE it became a live database. The idea above shouldn't require that much change to the code, its only one extra inventory item type which is much like all other resources in the game.

As for PlainWhiteSocks I dont understand what you mean when you say that this will lead to us being more like WS or that we'll be left with only a few good recipes, the variations between samples of the same grade are pretty minimal at the moment. I dont have or have never seen any recipes that state "you'll need a VHQ sample of rancor dna with a hardiness over (value) Also data storage and data transmission are very related, time the difference between downloading a 1k file and a 1GB file see for yourself. And you cant comment on the performance difference on different forms data storage until you know what kind of database they are running even tho the performance of the database for dna sampling is a bit of a mute point because it isn't used very often, compared to all the other actions which use the database. You optimise the most used actions first, who wants to optimise the 80% of the program thats used 20% of the time when you have 20% thats used 80% of the time?

---

PinZip wrote:
Well the idea I posted would realy take up 0 bandwidh when you compare it to say character models, simple store the master samples on the clients hard drive once and from then on your talking about a few bytes for each dna sample which should only be transmitted when u make a sample or log in. Now compare that to walking into a busy city with 10s of players on screen (hell a long buff line can easly hold 10ppl) and now you have to transfer all the data for character appearance, and this happens WAY more often than sampling dna. As for not messing with a live database I would be as unwilling to do it as any sane person, the difference is I'd make sure my database model wasn't going to need to be changed BEFORE it became a live database. The idea above shouldn't require that much change to the code, its only one extra inventory item type which is much like all other resources in the game.

As for PlainWhiteSocks I dont understand what you mean when you say that this will lead to us being more like WS or that we'll be left with only a few good recipes, the variations between samples of the same grade are pretty minimal at the moment. I dont have or have never seen any recipes that state "you'll need a VHQ sample of rancor dna with a hardiness over (value) Also data storage and data transmission are very related, time the difference between downloading a 1k file and a 1GB file see for yourself. And you cant comment on the performance difference on different forms data storage until you know what kind of database they are running even tho the performance of the database for dna sampling is a bit of a mute point because it isn't used very often, compared to all the other actions which use the database. You optimise the most used actions first, who wants to optimise the 80% of the program thats used 20% of the time when you have 20% thats used 80% of the time?

---

Actually I have no recipes that call for hardiness, but I have several recipes that require fortitude above a specific value in a particular critter to get armor. In fact I know that at least one recipe was posted here. Look around for it. The stats vary enough to give at least a 1k ham difference on a given recipe. Even the old VD-HP recipe can have a 1k difference in health between two creatures. While I do agree that the creature the dna came from has more effect than the individual stats of the creature compared to another of the same type, I still hold that the stats of an individual creature are important. Of the top of my head I know that the lowest hardiness MH sample I've gotten is between 600 and 610 (I think 609). The highest was between 750 and 760 (I think 757). That's a pretty big jump. It equates to around a 1700 health difference. Moving to the quality of the sample, it's used in a bunch of recipes to get a desired set of specials to come through.


Did you read my other posts?
I never said it would make us more like a WS. I tried to clear that up in a another post on this thread. Sorry, it was probably a bad analogy, badly written.

I stated I that I thought it's a bandwidth problem not a storage problem in another post in this thread. Splitting hairs? Maybe. Yeah, they're related, but when I ran some tests to see what kind of bandwidth this game was using I found it's not using much at all. It uses a bunch on load, then very little after that. (I did the tests to see if I could listen to shoutcast radio while playing.

As for the database they're running, I thought there is a post by SOE stating they were running on Oracle. Maybe this is not the case, but I was going on that. If I'm wrong then well heck, not the first time. *shrug* As to the actual implementation, well just like any other moron I can make wild guesses. It's even kinda fun.

I agree the percentage that DNA is used compared to the rest of the system is very low. I also agree tuning to the age old 80/20 formula is a good idea to start with.

-[Quote Pinzip]-
"As for not messing with a live database I would be as unwilling to do it as any sane person, the difference is I'd make sure my database model wasn't going to need to be changed BEFORE it became a live database."

Dude, show me the place where I can work without making changes to a live database, and I'll burn a path to their front door, resume in hand. People just keep changing the specs after implementation. At least once a week I look at someone that comes into my office and say "You wanna do what!?!?" Seems like everywhere I've worked the changes were handed down from managment to some poor developer who has to deal with some control freak DBA like me. (muhuhahaha the database is mine) In the end I can make reccomendations and I usually have final say in the structure of the DB, but either I make the changes, or I do the old sql>ALTER SYSTEM UPDATE RESUME IMMEDIATE
and man, I hate that command.

sheesh I can hardly believe I've typed this much. I'll shuttup now and go back in my hole.
Peace out dude.