What I want is a nothing less than a complete reference to everything published about Mystara in all products. And I don't want it on a web page somewhere - I want it in a database so I can ask it questions like, "What products describe the Savage Coast?" or "What modules are set in the Principalities of Glantri?" or "In what countries is the Immortal Halav worshipped?"
Tall order? Ok, it might be a bit of work, but far less than, say, Thorf's Secret Project. It's certainly doable. That's why I like Mystara, in fact - it's actually conceivable that I could read and catalogue everything published about it. Imagine doing that with The Forgotten Realms!
Here's how I plan to do it. I've marked some things I'd like feedback on in bold.
Technology:
The Mystaran Gazetteer is a Postgresql database, and a web frontend written in Ruby on Rails to add new items or query it. You'll only have access to update or add to the database if you have a login and password, but everyone will be able to query it. (And I'll make all the source code and database contents public, so if you want to just set up your own copy you can do that too.) To start with I'll be doing all the updates myself, mainly because it's fun, but if I decide it's too much work for one person I'll start giving out access passes.
What Gets Indexed:
For now, I'll just be doing the RPG sourcebooks (not the novels or computer games) that are included in these lists:
http://www.mystaranet.jamm.com/vaults/html/prodlist.html
http://www.mystaranet.jamm.com/vaults/html/mystfaq.html
http://www.geocities.com/TimesSquare/Dungeon/2967/prodlist.html
http://home.flash.net/~brenfrow/ddindex.htm
To my knowledge, that includes all BD&D and AD&D 2nd Ed products ever published. I will not (yet) be including 3rd Ed stuff or fan-created work. (If fan work is ever added, it will be clearly marked.)
This is just to keep the amount of work down. I plan to do the rest someday.
Each individual publication will be held in a Products table, with its name, a brief description and the system that it's written for (OD&D - for the original Blackmoor booklet and similar; BD&D; AD&D1; AD&D2; D&D3 - I don't plan to distinguish between 3.0 and 3.5).
Yeah, But What Gets Indexed?
All the "information" will be held in two tables: Concepts, which holds individual items of information, and "ConceptRefs", which links a concept to a product. For instance, the city of Specularum is a Concept, and its entry in the Concepts table will include its name, a brief description, and its location (Karameikos, another Concept). Specularum is described in both GAZ1: The Grand Duchy of Karameikos and the module B6: The Veiled Society (among others), so there will be two ConceptRef entries, one for each publication.
As well as the Concept and the Product it's described in, each ConceptRef will have a "level" field describing how important this reference is. For instance, GAZ3: The Principalities of Glantri is the main resource for Glantri, so it will have a level of "Main"; Glantri is visited briefly in the course of Champions of Mystara, so this would have a level of "Minor".
Concepts include:
- Places: Planes, Planets, Continents, Regions (conceptual groupings of countries, such as the Known World and the Savage Coast), Countries, Provinces, Cities, Features (natural, such as mountains and lakes), and Structures (artificial, such as famous buildings or monuments).
Each Place has a "parent" or "enclosing" Place, so we can get a full address. For instance Specularum would be Prime Plane (plane) -> Mystara (planet) -> Brun (continent) -> The Known World (region) -> The Grand Duchy of Karameikos (country) -> Specularum (city). Notice you can skip parts of the heirarchy; Specularum isn't in a Province (maybe it should be; I'll figure it out when I get around to indexing GAZ1). The Great School of Magic (a Structure) would be in Glantri City (a City), but Castle Amber (also a Structure) would be directly in The Principalities of Glantri (a Country) since it's not part of any city and we don't know what province it's in. Undecided: I plan to group seas in with Region, since "The Sea of Dread" and "The Known World" seem like about the same level of specificity to me. Should I add a separate Sea subtype instead?
The ConceptRef for a place includes some Yes/No questions: HasMap (does this product include a map of the Place) and OnMap (does this product include a map of the enclosing Place, with the location of the Place marked?) For instance, the ConceptRef linking X2 and Castle Amber would have HasMap "true" since X2 contains a map of the castle, but OnMap "false" since it doesn't contain a map of Glantri with the castle's location marked.
- People: PC's (ie. sample characters), NPC's (the vast majority) and Immortals. They will also have a location, which is filled in only if there is one place the character is almost always found. (Emperor Thincol is in Thyatis City 90% of the time, so he'd have it filled in; Bargle the Infamous would probably have the Black Eagle Barony - a province - as his location since it's his home base, although he can really be found anywhere in Karameikos and possibly beyond; Immortals could be found anywhere in the multiverse, so they wouldn't have a location at all.)
ConceptRef's for people include a HasStats flag, to say whether the product contains game statistics or just a description.
- Things: Artifacts (Immortal-level magic), MagicItems (notable mortal magic - not just every +1 sword), and NormalItems (especially valuable treasures, artworks and tomes, heirlooms, etc.) Like People, they have a Location if they are strongly associated with one place, and not otherwise.
Also like People, their ConceptRef includes HasStats, which is "true" if the product gives game mechanics for them and "false" otherwise.
- Race: This one's a bit trickier. Each Concept I've listed so far has a fixed list of subtypes (such as Artifact, MagicItem, NormalItem for Things). Race has a potentially unlimited number. Each notable clan or grouping will get an entry in the Race table, with their actual race as the subtype, and their home territory as the location. For instance, Clan de Belcadiz and Clan Erewan will have subtype Elf and location Glantri. Many, many products mention things like local orc tribes, so there will be entries like "Local Karameikos Orcs" as well.
Undecided: Should I have separate subtypes for Orc, Goblin, etc, or a single Humanoids subtype? Also undecided: should I really bother to list Orc for every module that includes orcs as a wandering monster? True, it's information - yes, Orcs live in this area, because you can run into them - but there are an awful lot of them and it's not as interesting as an actual named Orc tribe.
- Religion: Each Immortal is a subtype, and any piece of significant information relating to them gets its own entry. So, for Vanya, I would have one Person entry (name "Vanya", type "Person", subtype "Immortal", with a description of Vanya's personal and physical characteristics) and several Religion entries (name "Thyatian Vanya cult", type "Religion", subtype "Vanya", location "Thyatis"; name "Heldannic Vanya cult", type "Religion", subtype "Vanya", location "Heldann").
- Themes: Anything else that I feel is important or fun to note goes under themes. For instance, there would be a Criminals subtype for things like the Veiled Society and the Minrothad Thieves Guild; Nithia and Blackmoor subtypes; Thyatis and Alphatia subtypes for areas with Thyatian or Alphatian Conquests in their history; a subtype for famous Wars; etc.
Undecided: I haven't decided exactly how to lay out these themes yet. Is a 3-level heirarchy enough (type "Theme", subtype "Nithia", entry "ruins on the highlands in Ylaruam") or should I set up a more generic heirarchy (Theme -> Ancient Civilizations -> Nithia -> highland ruins; Theme -> Ancient Civilizations -> Blackmoor -> The Radiance)? And how would I even do such a heirarchy?
Whew. I'm getting tired. Tomorrow I'll post the database schema so you can see in more detail how this all fits together. In the meantime, here are some more things I haven't figured out yet:
- What do I do about changes over time (the renaming of Karameikos, the creation of the Darokin crater - hell, Castle Amber returning)?
- What about structures that exist in multiple places simultaneously (actually, I can't really think of any right now, but I'm sure I'll find one soon)?
- Like I mentioned, I'll have a list of Products and a ConceptRef linking an entry in the Product table with an entry in the Concept table. What do I do about variant products? I don't want to have to repeat all the ConceptRefs for a product which is only slightly different. (This is a complicated question: the green-cover and orange-cover versions of B1 are clearly variants and their contents are almost, but not exactly, identical. What about GAZ1: The Kingdom of Karameikos and Karameikos: Kingdom of Adventure? What about Champions of Mystara and the original Voyage of the Princess Ark articles? What about B1-9: In Search of Adventure? I'm tempted to skip the whole mess and just have lots of ConceptRefs showing that the same concepts are described in the exact same ways in different products.)
The Future, Conan?
This is what I'm going to start out with. (I'll probably skimp heavily on the Themes for the time being.) There's more info that could be added. Off the top of my head, each Place could include a link to Thorf's map of that place, and each Product could include a cover scan. These can be easily added later. Anything else cool you think should be indexed?
What's Finished
I have a rudimentary UI which I've used to input B1 - In Search of Adventure, which was fairly easy since it doesn't have much meat. I'm currently thinking about how to lay out the UI better to make it easier to input the data for more complex products. I've also already noticed a few deficiencies in my database structure that I want to correct before I put in more data.
That reminds me: What should I do about old modules like B1 that don't fit anywhere? It's put in Mystara by convention because it's part of the D&D line, but it doesn't mention any part of the Known World: it just has a few names of old adventurers and the concept of "barbarians to the north". I can make an educated guess that it goes in Karameikos (and the barbarians are Ylari tribesmen, which doesn't quite fit) or northern Ylaruam or Darokin (with the barbarians as Ethangars or Northern Reachmen), but based on the weirdness of the sample characters' names I'd like to put it in Ghyr: the tone seems to match XL1 quite well. (I believe B1-9 put it in Karameikos, but it also excised a lot of the background info that made it fit better with Ghyr - see what I mean about variants being hard to deal with?) So should I just make a decision and say "It's in Ghyr", or leave its location unknown?
Anyway, my next step will be to add authentication and a viewer so I can make this public and let people look at it without worrying about them ruining all my hard work. Then I'll fix up the database, then extend the UI some more, and then start throwing data on the thing like a madman!
My own project is on stand-by until early September as I am handing in my thesis in less than 48 hours
and after that I am going to Greece for a while to relax and recover 
:-) Jesper
- For Races, consider two levels of entry, one general and one specific. This would allow DBM for both general stuff like 'humanoids' or 'lizardkind' and specific entries like 'elf', 'dwarf', 'orc', or 'goblin' as well. Humans are something of a wildcard here - an entry of Human could be a specific value under a general category of 'PC races', or general with individual human ethnicities ('Traladaran', 'Glantrian', 'Pearl Islander') as the specific slot.
- Changes over time are pretty easily handled with StartDate and EndDate fields. General Torenal of Alphatia, for instance, might have StartDate 960 AC (or whatever, don't have books on my person for his birth year) and EndDate 1004 AC (due to his unfortunate demise during WotI). It's then pretty trivial to specify a year and screen out anything with a later StartDate or earlier EndDate. Data records that substantially change get new database entries, with possibly some pointer set up to note the successor (so "Principality of Blackhill", EndDate 1006 AC, might have a pointer set up to the Great Crater afterwards.)
Good luck! :D
The Mystaran Gazetteer is a Postgresql database, and a web frontend written in Ruby on Rails to add new items or query it.
Wow.....ummmmm. Hmmmmmm. Are you sure you want to go to all this effort? I see some issues here. I don't think you have thought this completely through.
Wouldn't the Mystara community be better off with a PDF of descriptive information (maybe some game stats) rather than a Postgresql database/Ruby on Rails? Someone would have to host it, right? That's alot of internet traffic.
Also, you're not going to do the 3e/3.5e conversions (yet). I don't see how it's going help if that isn't done. The current version of the game (as much as I really don't want to say it) should be the target.
What about the copyrights?
(Darn it! I've opened such a bucket of worms!!!)
--Ray.
- For Races, consider two levels of entry, one general and one specific. This would allow DBM for both general stuff like 'humanoids' or 'lizardkind' and specific entries like 'elf', 'dwarf', 'orc', or 'goblin' as well. Humans are something of a wildcard here - an entry of Human could be a specific value under a general category of 'PC races', or general with individual human ethnicities ('Traladaran', 'Glantrian', 'Pearl Islander') as the specific slot.
Yeah, I was thinking about that. I'm not sure how to get that to work with my current database schema, though. I'll post more details on that tomorrow. (Yeah, I know I said that yesterday.)
Changes over time are pretty easily handled with StartDate and EndDate fields.
Hey, good idea. I'll add those right away.
Wow.....ummmmm. Hmmmmmm. Are you sure you want to go to all this effort? I see some issues here. I don't think you have thought this completely through.
Yes, because I find it fun. The minute I find it more work then fun, I'll stop.
Wouldn't the Mystara community be better off with a PDF of descriptive information (maybe some game stats) rather than a Postgresql database/Ruby on Rails? Someone would have to host it, right? That's alot of internet traffic.
This isn't the big of a community. I think I can handle it, or easily find hosting that can.
I already have an RTF file here. I noticed while I was making it that I was entering a lot of information in multiple places, and it was hard to keep them in sync - sometimes I'd enter some data in one place and forget to enter it in another. (For instance, I wanted a list of "Elven Clans" so that I wouldn't have to search the whole thing to get a list of every elven clan mentioned, but I also wanted to list the local elven clans under each country.) So I realized that it would be much more efficient to make a database, and then I only have to input each clan once, and I can ask it either for a view of all elven clans or a view of everything it knows about a given country.
If I want to publish a PDF (or, more like, an HTML page) with stats, I can just write a script to pull it from the database. But a nice relational database is clearly the right format to store this information at the root.
Also, you're not going to do the 3e/3.5e conversions (yet). I don't see how it's going help if that isn't done. The current version of the game (as much as I really don't want to say it) should be the target.
Help what? I'm not doing this for players of 3rd Ed, I'm doing this for me. I just thought the community might be interested. If you're not, go make your own project.
What about the copyrights?
Hmm, good question. I don't think that's a problem, since I'm just describing what each book contains, not duplicating the information. (I'm going to be noting whether each product contains stats or maps, for instance, but not actually listing those stats.) The idea is to give you a complete list of what you have to buy in PDF to get all the info on a certain topic - I can't see how that's not fair use.
Thorf's maps may have a copyright problem, but I haven't made any provision to link them in yet, so I'll deal with that when the time comes.
Undecided: Should I have separate subtypes for Orc, Goblin, etc, or a single Humanoids subtype? Also undecided: should I really bother to list Orc for every module that includes orcs as a wandering monster? True, it's information - yes, Orcs live in this area, because you can run into them - but there are an awful lot of them and it's not as interesting as an actual named Orc tribe.
This really is a cool project. I'm not entirely sure if it's any less ambitious than my own project, though - you have a lot to work through! :D
Regarding the wandering monster issue, doesn't your reference importance scale handle that already? A reference for a monster as a simple wandering monster is a "minor" (or less) reference. This isn't so important for orcs, as you said, but it should be done for the rarer or more interesting wandering monsters, probably.
--Ray.
Thorf's maps may have a copyright problem, but I haven't made any provision to link them in yet, so I'll deal with that when the time comes.
As far as the copyright goes with my maps, or any other fan maps of Mystara for that matter, I really don't know where we stand. Obviously I have used the original copyrighted material as primary sources, and half of my work is making direct copies of those sources using my own art work (which itself is copied from/based on TSR art work), but I really don't know exactly what that means.
However, it seems to me that, as dedicated fans of a dead world, it's not really an issue we need to deal with. As long as we aren't posting direct scans of the original TSR books, I don't think Wizards of the Coast will mind what we do. Especially since our motivations are not at all profit-based.
All tables include an "id" column as primary key, which is a unique integer. Any column named foo_id is an integer which is a foreign key into another
table. No column can be null unless marked "optional".
Aside: any column named foo_ids points to one or more foreign keys - ok, you can't really do that. The real way to have a many-to-many relationship involves making an extra table, but that'd just clutter up the list. I you know anything about DB's, you know what I'm using a shorthand for, and if you don't, just pretend it means a list of id's instead of just one. As far as you know, that works fine.
Products
The first tables describe physical products.
systems
This table contains each different rule set I might be indexing products for.
short_name (ie. BD&D, AD&D2)
full_name: (ie. Basic Dungeons & Dragons, Advanced Dungeons & Dragons 2nd Edition)
description: paragraph format description (optional)
formats
This table contains each physical format of product.
name: (ie. box set, module, web download)
description: paragraph format description (optional)
products
Finally the main product list!
key: the brief product code number, or an informal equivalent (ie. X4, GAZ3, RC)
name: the full name (ie. Master of the Desert Nomads, The Principalities of Glantri, D&D Rules Cyclopedia)
description: paragraph format description (optional)
format_id: the format of the product (ie. module, gazeteer, hardback)
system_ids: all systems this product supports (ie. usually BD&D, but some modules will include BD&D and AD&D2)
product_sources
Places where a product is available online. I don't think I'll bother listing actual prices since they can change. I also won't bother making a complete list, since there are lots of PDF sellers - this table's entirely optional.
product_id: the product available at this site
url: self-explanatory
is_free: a boolean describing whether it's a free download or a pay PDF
So, for instance, B1 - In Search of the Unknown has:
systems:
id: 1
short_name: "BD&D"
full_name: "Basic Dungeons & Dragons"
description: "The "basic" line of the D&D/AD&D split, originally published in the Basic/Expert/Companion/Master sets."
formats:
id: 1
name: "module"
description: "An adventure packaged in a folder with slipcover and stapled pages."
products:
id: 1
key: "B1"
name: "In Search of the Unknown"
description: "This package (a cover folder with maps and descriptive booklet within) forms a complete module for use with DUNGEONS & DRAGONS Basic Set. It is especially designed as an instructional aid for beginning Dungeon Masters and players, specifically created to enable new Dungeon Masters to initiate play with a minimum of preparation.
In addition to descriptive and situational material, this module also includes special informational sections giving: background history and legends, listings of possible monsters and treasures and how to place them, a list of adventuring characters, tips on various aspects of play for the Dungeon Master, and helpful advice for starting players."
format_id: 1
system_ids: 1
product_sources
id: 1
product_id: 1
url: "http://www.rpgnow.com/product_info.php?products_id=1113&"
is_free: false
id: 2
product_id: 1
url: "http://www.rpgunited.com/product/default/tsr9023.html"
is_free: false
Of course, the "systems" and "formats" entries will be reused for other products, so there shouldn't be many of them; each product will get exactly one row in the "products" table, and possibly several rows in "product_sources". (Normally I'd just include one, but I did two to illustrate this time.)
Ok, on to the concepts - oops, I'm late for something. Gotta go.
Tables Describing Products
The rules system a product is written for (ie. Basic D&D, AD&D2, D&D 3.0/3.5):
So concepts now looks like this:
Note that something on these lines was done for Ravenloft (see http://www.kargatane.com/sotk/rldb/rldb.shtml ) on the then-official fan website, so it should be ok from the copyright point of view (at least, AFAIK no one raised any issue with the RL catalogue).