nezroy was here

Universal Dye Kit v1.2 Released

2010-07-08T00:41:00.001-06:00

I've released v1.2 of the Universal Dye Kit mod for Dragon Age. This is an item that allows custom tinting to be applied to most armor and weapons. Use in conjunction with my Online Tint Creator to customize your armor and weapon colors or color coordinate your entire party.

Among other changes, this version adds the UDK to Lem, the vendor in Leliana's Song. It also improves compatibility with The Winter Forge.

Armor of the Black Fox v2.0 Released

2010-07-08T00:47:00.000-06:00

I've released v2.0 of the Armor of the Black Fox mod for Dragon Age. This mod adds a set of armor and weapons to the game. These custom items were once owned by the notorious and dashing Orlesian rogue known as the Black Fox.

There were quite a few changes in this version. The big ones are the name change (previously called "Leliana Item Set"), the addition of male variants for the armor, and support for the Universal Dye Kit.

Warden Shields v2.0 Released

2010-07-08T00:44:00.000-06:00

I've released v2.0 of my Warden Shields mod for Dragon Age. This mod adds a set of shields forged to match the Warden Commander armor set.

The major change in this version is to add support for the Universal Dye Kit. I also removed the runic shield variants since that mod now comes with its own matching shields.

Leliana Item Set v1.5 Released

2010-05-07T00:14:00.006-06:00

I was thinking recently it might be smart to cross-post other stuff I do more often to my blog. So in that vein, I have released v1.5 of my Leliana Item Set mod, an armor and weapon mod I created for Dragon Age: Origins. This release adds female dwarf support and an important bug fix.

Ripping CDs the nezroy Way

2010-02-12T07:08:00.006-07:00

I would never claim I'm OCD; like schizophrenia, it is a disorder far too often misunderstood and trivialized to its pop-culture interpretation, much to the disservice of its sufferers. However, there's no question that I have my own fair share of bizarre quirks and compulsions. For instance, only recently did I stop "tracking" the number of strokes while brushing my teeth (10 per zone, and 30 zones in my mouth... I could draw a diagram, if you're interested). Or the fact that when I listen to music on a player that shows numerical volume levels, I can only stand to listen at volumes that are prime numbers. I like to think of it as a perfectionist issue; ask my wife, though, and I suspect these traits might rather be described as ridiculous, weird, annoying, or just plain dumb.

The most common way this idiosyncrasy of mine tends to manifest is in the way I approach any discrete task. I will start with a firm set of idealistic requirements that are often impractical or downright trivial, yet serve to completely stall any progress for lack of viable options or willingness to settle or compromise on my part. And even if I don't start out with any pre-built roadblocks, by the time I'm finished with my initial research phase, I'll probably have managed to acquire quite a few lofty and barely informed hang-ups. I suppose the fact that I start most tasks with a "research phase" is odd-enough. Am I supposed to be using 10d box nails or common nails? Would a 12d nail be better? Maybe it'll get put outside in the rain for 10 minutes one day... shouldn't I really be using galvanized nails for this project? Do we have any 12d common galvanized nails? No?! Sorry honey, I can't fix this right now, it would be WRONG! Honey, put down the hammer! You don't even KNOW what kind of nail that is, do you?!

So as you might imagine, something as littered with perfectionist pitfalls as ripping my music collection to disk can be practically paralyzing. It's a task I've been meaning to get to for quite a while; actually, ever since our CDs were stolen from the car 7 years ago. And yet any time I've started working on it, the lack of a solution that meets my requirements has prevented me from making useful progress. If it couldn't be done "the nezroy way", then dammit, it wasn't getting done at all.

In this case, I didn't actually think my requirements were *that* unreasonable:

the ripped format should be loss-less, because I'm not doing all this work just to get an imperfect audio copy
the ripped format should be an open standard, because... well, really there's no particularly justifiable reason for this one, leave me alone
the ripped format should be natively playable on at least one decent digital audio player because I am NOT transcoding all this crap again
the ripped format should, at minimum, be playable on windows media player with a trivial amount of work
the process must produce a "pretty" library with good tagging
the process must be simple enough that I can remember how to do it the same way next time I buy a CD; which means I'm not going to put up with a command-line, batch file, or scripting solution for doing this... nothing personal, I love Linux, bash, and what have you, but come on, it's 2010 now and I've become old and irritable
ideally, the music library it generates should be easy to use, for some nebulous definition of easy

Given that list, it really breaks down into three independent variables to resolve. The format, the process, and the player. Unfortunately, every time I've looked into this in the past, the output of "the format" and the input of "the process" never matched up. Which is to say, none of the formats I wanted to use had decent tools, and handling that impedance mismatch made the process too difficult to easily reproduce. Rather than compromise on format or do some extra work with the tools, I mostly just let our CD collection gather dust (and scratches). I am thrilled to report, however, that this great new decade has brought with it a process that finally meets all of my requirements. Yay!

Problem the first; the format. Picking the format has always been relatively easy for me, if only because there aren't many options that even come close to my criteria. Loss-less restricts it to something like WAV, WMA, WavPack, FLAC, or any number of other obscure and irrelevant choices. WAV is simply too large and inefficient, and doesn't have decent standardized tagging support to boot. WMA loss-less has decent compression and good tagging, but it's not truly open, though it was "standard enough" that I was willing to consider it. WavPack is fully open and has good compression, but is not widely supported in comparison. Lastly, FLAC has good compression, good tagging, is completely open, and has a lot of support.

In the end I've always come back to FLAC for this, for a number of nuanced reasons. As an open project it's part of the Xiph Foundation, the backers of Ogg Vorbis and a whole suite of open multimedia formats. This provides considerable long-term viability in comparison to some other options. As a format, FLAC achieves its compression ratios without cutting corners, which is also a big plus compared to some other formats. In particular, it decompresses extremely fast, which translates directly to battery life when it comes to portable devices, and its streaming and seeking features are unmatched by other loss-less formats. Lastly, it's trivial to add Windows Media Player support for FLAC via the DirectShow filters provided by Xiph.

Problem the second; the process. This, of course, has always been the real stickler. My unwillingness to compromise on the format has meant that finding tools that would let me rip and tag to a FLAC library has historically been very difficult. Even grudgingly considering WMA didn't yield a lot of good options either. Recently, however, I was finally able to achieve my ideal FLAC ripping process with a collection of 3 tools. None of them are particularly new, but apparently I had just never managed to get them all together in the same room before:

The ripper of choice for many folks seems to be Exact Audio Copy (though CDex is also a popular choice). EAC transcodes to FLAC (among other formats), supports freedb metadata, and does a good job of producing pretty music library files with good tagging. It's also highly configurable, which means you can accommodate just about any tagging/naming convention you care to think up.

That said, the real niche of EAC is that it specializes in producing digitally exact copies of the CD. This can make it much slower than other rippers, but that is the trade-off for knowing that your rip is clean and glitch-free. The issues surrounding this are surprisingly complex, but suffice to say that even though a CD contains pure digital data, a CD player reading an audio CD tends to be very imprecise. Highly specialized glitch- and gap-fixing hardware in consumer CD players minimizes the audio impact of scratches and missed data, but these will show up glaringly in your digital audio library if left unhandled. It's actually quite time-consuming to get a digitally exact copy of a scratched CD, even though it might play and sound just find on a consumer player. Other ripping software tends to simply ignore the issue entirely, but EAC addresses it head-on with a host of features that guarantee accurate rips.

The other heavy-hitter in my toolset is foobar2000, whose primary function is actually as a music player. However, its tagging and library management features are phenomenal. Anything that EAC misses in tagging can easily be cleaned up with foobar2000. You can view and organize your library based not just on folder structure but also extracted tagging metadata, and you can apply precise tag tweaks to entire swaths of your library based on searches and filters of that metadata. You can also rename and move the file structure around en-masse based on tagging data as well, and any number of other similar tasks.

As an added bonus, I've actually replaced the use of Windows Media Player with foobar2000 for my default music player. It's just so easy to use and does such a good job of managing and exposing my music library that it became a no-brainer.

The last tool I use is really a non-essential item, but it provides that last little bit of polish. It's the Album Art Downloader, which searches a ton of different online sources to allow you to pick and choose front and back cover art, insert and artist art, etc. for the CDs you've ripped. It's not really a necessary part of my library or ripping process, but it's definitely a "nice to have" feature that I find to be worth using. It's managed to uncover album art for some of the most esoteric and oddball CDs I have, which makes it a great tool in my book.

Problem the third; the digital audio player. Having never gotten this far before due to the issues with process, I wasn't sure what to expect. However, it turns out that there are so many digital audio players on the market these days that it's not hard to find something that meets all of my obscure requirements. And FLAC actually has a decent level of hardware support, making the choices plentiful. Since the player is primarily for my wife, it's really her needs that came into play here. She likes her devices tiny and simple, and she hates the very concept of iTunes and iPods with a passion. Her preferred UI is a play button, and her preferred choice of library management is a USB drive or memory card and good 'ole Windows file copy. She also listens to the radio a lot, which is an esoteric but, happily, accomodatable request.

In the end we decided on the SanDisk Sansa Clip+, which is a cute little player that has exactly those features mentioned. Native FLAC support, an FM radio, a tiny screen and easy to use interface, and it connects to her PC via USB and appears as a simple USB drive to drop music onto. As an added bonus, it also has a voice recoding mode which turned out to be a great perk. With 4GB of internal memory for CDN$50, it was hard to beat. And it has a MicroSD slot which can handle another 16GB.

So there you have it; everything you need to know to rip CDs the "nezroy way"! Now if you'll excuse me, I have a lot of disc swapping to attend to.

The Kindle Seduced Me

2010-01-03T04:29:00.026-07:00

I have a secret I must now confess: I'm a bit of a gadget Luddite. This may come as a shock to anyone who knows me, since I'm a software developer by trade and have, at last count, 6 active computers in my household (and enough parts for a couple more). But while I am an unabashed computer geek, I've really just never been able to get into gadgets. My cellphone is more than two years old and weighs about as much as a small toaster oven. It answers calls, and it makes calls, and that's about all I expect from it. The screen cracked 6 months back and I didn't really care. I've never asked it to try and find the mythical intarweb, and it has obliged my lack of curiosity by resetting itself randomly in the middle of calls, or any time I try to add a contact to the address book.

I do have an iPod nano, but it sits unused in my "stuff I should really find a place for" bin at the corner of my desk. Every once in a while I catch a glimpse of its blank little screen staring at me forlornly, but I simply look away. Each time I notice it, I promise myself that we'll spend more time together. Maybe go out for a stroll so I can listen to its finely crafted playlist, or at least plug it into the USB port and fire up iTunes for a couple of minutes. But we both know that it was an unwanted child, and I'll never have the time it wants from me. I even flirted with a PSP once, which regaled me with GTA and some crazy but fun marble game... for a while. But eventually the spark died, and I found myself leaving the house without it more and more often. It finally got to the point that I could no longer bear the recriminating stare of its darkened face, so I relented and gave it away to a new owner who could give it the attention it deserved.

So I was surprised, even shocked, to feel within me a growing lust for the Amazon Kindle after learning that the global version had finally been released in Canada. It makes sense, I suppose... this gadget is totally my type. I imagine that if it were a woman, it would be a librarian. Slim, smart, and bookish; complete with horn-rimmed glasses, a tightly wound bun, and a plain, gray, knit sweater. Not like that free-spirited hippy iPod, or that pretentious urban chic PSP. This was a woman... er, I mean, a gadget, that I really wanted to get my hands on.

This funny feeling didn't even begin with the Kindle. My attention was first drawn to modern ebook readers by a chance internet encounter with Sony's PRS-300. I knew about their existence in general, of course; I remember reading about the first generation Kindle when it launched. But something about the PRS-300 (and its big touchy brother, the Sony PRS-600) piqued my interest anew. Enough so that I decided to head to Best Buy to take a look. After all, the Christmas season was fast approaching, and I had a feeling that Santa hadn't yet figured out what to bring me.

And I fell in love!

More precisely, I fell in love with the e-paper screen. The moment I laid eyes on it, I knew I had to have one. I yearned for its clarity, focus, and contrast. The choice had been made for me... a modern ebook reader would be mine before the holidays were holidayed! All that remained was to decide which one it should be. I'd like to report that I was an informed consumer who spent weeks researching the best option for my ebook reader. Scouring over internet reviews, creating charts of pros and cons, and consulting the Oracle of Delphi for advice. But alas, that's not really how it happened.

The PRS-300 display model I saw in Best Buy was extremely slow and unresponsive, and had a 5" screen which simply felt a bit too small for me. The PRS-600 was more snappy, but the display was, in a word, fuzzy. It is a result of the touch screen overlay, so I've been told, but it simply did not appeal to me next to the clarity of the PRS-300. Not to mention the PRS-600 cost far more than I wanted to spend, especially for touch-screen technology that I knew I would never use. Simply put, neither would suffice to satiate my growing desire.

I then rushed home, thought to myself "Hey, didn't Amazon make one of these? I think they sell a lot of books", looked up the Kindle, and found to my delight that the global version had just been released in Canada not two weeks prior. Thus invited into its welcoming embrace, I noticed the price: $260, with free shipping. The ecstasy now brimming, I was already hovering over the bright yellow "buy me!" button. The last thing I remember, before succumbing to its seductive overtures and making the plunge, was a bit about free wireless connectivity to Wikipedia. Everything after that is a blur, though I distinctly remember "Don't panic!" being murmured in sweet, dulcet tones. (Turns out that was probably the audio reader feature, something I didn't learn about until long after our relationship had been consummated).

The Kindle arrived shortly thereafter, but in repentance for giving in so impulsively, I forced myself to wait four weeks before I opened it. It simply wouldn't do to walk around with the thing where the kids could see it, then wrap it back up and put it under the tree from Santa on Christmas morn. I'm pretty sure that would have raised pointed questions regarding Christmas mythology that I didn't really want to answer this holiday. Explaining how Santa got into our house when we don't have a fireplace or a chimney was a taxing enough endeavor for me this season. But Christmas morning did finally arrive, and at last Santa had delivered into my hands the object of my desire; an Amazon Kindle ebook reader.

As a gadget, the ebook reader has a singular purpose. It lets you read books. Since I love to read books, and have hauled hundreds of kilograms of paperbacks with me on over a half-dozen moves to prove it, this seems like a perfect match. And the Kindle is very good at handling the mechanics for reading a book. First, and most importantly, is the e-paper screen. The best way to relate what the Kindle's e-paper looks like is to simply have you find a nearby paperback, open up the cover, and look at a printed page. Yes, that's really what modern e-paper looks like. It's that good. It's clear, precise, high contrast, and has no glare. Some devices with a touch-screen overlay can have a fuzzy display or a bit of glare, but that's not the e-paper's fault. I'll try to illustrate with a photo, but really, it's just like reading paper.

The e-paper display is the distinguishing feature of the modern ebook reader, and it's the reason they will always have a place in the world. It doesn't matter how many tablets Apple announces this year, or how small my laptop gets or how long its battery lasts -- I will never go back to reading copious amounts of text on an LCD or OLED display ever again. I've no doubt that "gadget people" have no problem reading their ebooks on laptops, tablets, or what have you. But for the rest of us there will be no competition; reading on any display that emits light is awkward and eye straining, and reading on e-paper isn't. As far as I'm concerned, e-paper will be a defining feature of mass market ebook readers... once all the other ebook issues are sorted out, of course. But I'm getting ahead of myself.

Beyond its beautiful screen, the Kindle is a simple device. It has a basic keyboard for note-taking and a couple of buttons for navigation and interaction. The device in total is a little bit larger than a standard paperback, but not by much. The screen is 6" diagonal, which is noticeably smaller than the reading surface of a paperback page. For my reading speed, however, it has been large enough and hasn't bothered me in the slightest. Several other readers have a 5" screen, which I think would have been just a little bit too small for me.

Along with my Kindle I also ordered the Amazon Kindle Leather Cover, which is visible in the photos. The Kindle has two slots with mechanical latches on the left side, so it locks securely into a compatible cover (such as that one), leaving no worry at all that it will accidentally slip out. With the leather cover the Kindle is absolutely comfortable to hold, and it feels just like holding a paperback. I have read it for hours in many positions -- sitting, standing, and lying down -- with no discomfort at all. In some positions it's even more convenient than reading a paperback, as you can lay it flat or hold it with one hand and the pages don't start flipping around on you. That particular cover can also fold back on itself without stretching or damaging it, and it takes up little more room than the Kindle itself, so it's suitable for all positions. I suspect that even without the leather cover it would be comfortable to hold, but it fits so nicely and provides such good protection that I've never undressed my Kindle.

My Kindle, fully dressed, weighs in at 462g. In contrast, the paperback in the photos, which is a relatively short book, comes in at 103g. One of the larger paperbacks I have, James Clavell's Noble House (1370 pages), weighs in at 560g. So the Kindle is a bit on the heavy side, but even with my cover it's still not as heavy as the larger paperbacks, and the weight has not once been a comfort issue for me.

Reading a book on the Kindle couldn't be simpler. When you click on either of the two large and well-placed "Next Page" buttons, it displays the next page. The screen takes about a half second to a second to refresh, which does not seem to interrupt my reading flow. A speed-reader might have problems, but I'm no slouch and it doesn't bother me. A "Prev Page" button on the left does what you'd expect, though it would have been nice to have a "Prev Page" button on the right side too, though that's a minor nitpick at most. You can adjust the text size and margins through a decent range, so just about anyone should be able to find a comfortable page setup for reading.

When you are finished reading, you simply put the Kindle down. That is, once you've gotten over the ingrained urge to look for the page number to open to the next time you pick it up. You know an ebook reader is doing something right when you have to constantly remind yourself that you DON'T need to put a bookmark in it before you put it down. After a few minutes of inactivity the Kindle goes into "sleep" mode automatically, which doesn't seem to tax the battery at all, giving you a couple of weeks per charge if you have the wireless turned off (once the e-paper is "printed", it takes no power for it to continue displaying whatever it is displaying). Of course this varies with how many pages you view, but I've used it several hours a day and it's easily gone more than a week. When you pick up the Kindle again (and wake it from sleep, if it has gone to sleep), it will simply start you with the book and page you were last reading.

The Kindle remembers what page you were on for every other book you were reading as well. You simply click the "Home" button to get to the main screen, which lists your books, and select whichever book you want to read. It will automatically take you to the last page read, so you can read multiple books concurrently without losing your place. The home menu also shows you how far along you are in each book, and while reading a book a progress indicator on the bottom of the display shows you how far along you are as well. Because an ebook has no concept of pages, the Kindle shows you your current "location", which corresponds roughly to a paragraph or sub-paragraph level. It makes for an easy way to quickly jump back and forth in a book; about as easy as flipping to a specific page number in a physical book. You can also bookmark a location and even annotate a book, which is great for academic reading or just remembering interesting stuff. And with 2G of memory, which translates to a couple thousand books, it's pretty hard to run out of interesting stuff.

The Kindle has some other features that aren't relevant to its primary task of letting me read books, but I've found most of them useful, without being intrusive.

The built-in dictionary; when you highlight any word on the page, a small window pops up at the bottom of the display with a definition. You can hit the enter key to expand it to a full page view with the complete definition, and hitting the back button takes you right back to your spot in the book. It's a simple feature that is well executed, and though I don't use it all that often, it's extremely convenient when I do need it.
Text-to-speech; the Kindle can read any text on the page to you with a speech synthesizer (well, almost, but I'll talk about that later). I don't use this myself, but I imagine this could be extremely useful to a lot of people.
MP3 player; along with the audio reading feature, it can also play MP3s. Or so the docs say. I haven't tried it yet for fear of sending my iPod into a jealous rage. Plus I'm sure it can't be good for the battery life.
Free wireless to Wikipedia; the "Whispernet" that Amazon uses to connect to its store for book purchases is just an Edge/3G wireless deal they've worked out with various providers. The fact that it's free to browse Wikipedia on the Whispernet is stupendously awesome, if only because I can pretend to be an intergalactic restaurant critic. I actually haven't used it very much; it's a bit slow and I had to turn off images to make it very usable. But still, it's free and in a pinch it would do the trick nicely. The biggest drawback is simply that turning on and extensively using the wireless drops the battery life down to a few days.
Multiple Kindle sharing; you can have up to 6 Kindles registered to your account, all of which can have copies of the ebooks you have purchased. This means you can share your ebook library among multiple devices for a family or group. It's certainly a selling point for me, as I'm thrilled that I can get a Kindle for my wife without us having to worry about buying the same ebooks more than once.
More than just ebooks; when the Kindle is plugged into your computer, it shows up as a standard USB drive. You can copy over any format the Kindle supports to read on your Kindle. The most interesting of these formats is PDF, which means you can read things like manuals, reports, etc. It's not just for ebooks!

I'm not a "Kindlenatic", as I hear some folks have been dubbed, and I won't say that the Kindle is unequivocally the best ebook reader on the market. I haven't done the research to support such a ridiculous claim, and even if I had, I don't believe there's a single ebook reader to meet every need. It's not a perfect ebook reader; for instance, even basic 802.11 wi-fi would have been a welcome addition. However, the Kindle is a very solid product; one that I'm happy with for the price-point at which I acquired it. And more importantly, one that meets my needs quite well.

So the real question is, if the Kindle is so awesome, why hasn't it reached a mass audience already? I certainly don't think the answer is price; at $260 it's already worth every penny. No doubt ebook readers will see more penetration as they hit the sub-$200 and eventually sub-$100 ranges, but I don't think that's really the problem area. And the ebook prices are certainly not an issue either; at anywhere from $10 to $2 for the average ebook, it's not as though you feel you're being gouged. No, unfortunately the biggest issues are the problems surrounding ebooks themselves.

For instance, I mentioned that the Kindle can read an ebook out loud, but in order to appease the publishers, who were convinced that this would damage their audio book sales, the ebook can set a flag to disable this feature. This is farily typical of the publisher perception of "those damned ebooks!" (as I imagine them shouting). More problematic, there is still no single standard ebook format and, critically, no central ebook standard exchange system for handling formats, DRM, and search/discovery that multiple portals and retailers can tie into (basically an ISBN on steroids for ebooks). I use the free Calibre software to manage my ebook library and convert between formats, which works extremely well, but it's still an annoying extra step that would instantly alienate your average mass market user. And even the Amazon store, which is arguably the largest single ebook retailer out there, doesn't have a LOT of the books I've wanted to read. I'd guess at least 80% of my searches for ebooks on Amazon have been unfulfilled, which is simply depressing. I want to pay them money to read their wonderful books on my Kindle, but I can't because publishers still haven't gotten with the program yet. Sure, there's plenty of great stuff over at Project Gutenberg, but that doesn't help me if I'm looking for something modern.

Even those publishers that embrace the ebook concept are still left to decide which of the half dozen or so ebook retailers and formats they are going to support, meaning you can sometimes find an ebook you want, but it's not one that you can (easily) USE on your particular device. It's not as bleak as I may make it sound, but it's still frustrating that it's not even better. Luckily I can commiserate with the folks at the MobileRead site, which is a fantastic resource for learning about the ebook world and all its issues.

Ultimately, I'm convinced ebook readers are eventually going to hit a tipping point and become ubiquitous mass market devices, just like the datapads from Star Trek. It will take some major improvements in ebook accessibility to make that happen, but I'm hoping this future is closer than it looks. Until you can buy the vast majority of books you want in ebook format, without having to leave the confines of your ebook reader, without ever worrying about arcane ebook formats or if the DRM is going to work on your device or not, it won't be a mass audience device. Part of the reason I went with a Kindle is because I suspect Amazon, more than any other company, has the clout and desire to pull publishers, kicking or screaming, into this vision of the future. Unfortunately, I suspect it may take many years and several bitter legal battles before we get there.

Fashion Mash Fate

2010-01-01T14:57:00.015-07:00

It's been a while since I've talked about Fashion Mash, the website project that occupied several years of our life. It's with mixed emotion that I can now report its eventual fate: we sold Fashion Mash to Apply Marketing, who manage, among other things, the OpenFashion website.

Of course I'm thrilled that we managed to find a new home for the site, and for the more than 800 users who stuck with us since our launch back in 2007. I have no doubt that Apply Marketing will be able to help the site realize its full potential. I'm certain that the sale was the right thing for the site and for us, and I wish the new owners the best of luck in moving Fashion Mash forward.

As confident as I am in the choice to sell and in the new owners we found, I'm equally disheartened about the events that led to this eventual outcome. It had become painfully clear over the last year or so that we (mezamashii and I) no longer had the time, energy, or resources necessary to give the site all that it deserved. I've spent a lot of time thinking about what went wrong and what went right with this project, as it is an invaluable learning experience that we hope will enable us to lay the foundations of success for our future projects.

From the very beginning of this project we did a lot to stack the deck against ourselves without realizing just how important some of these factors would end up being. The timing and choice to open Heliostorm and begin the Fashion Mash project was precipitated in part by a major life event. Now this is not that uncommon, and I have no doubt that an unforeseen kick in the pants has been the last bit of necessary motivation to get many would-be-entrepreneurs started down their own paths.

In our case, however, it was not something "simple" like a downsizing job loss. Instead our catalyst was that the children we had been parenting in foster care for nearly two years, and working to adopt, ultimately went on to live with a blood relative in something of a last-minute change of heart beyond our control. While we were happy that the kids found their "forever family" with a blood relative (something we feel is extremely important), it was still emotionally devastating for us. There's no question that the grief we experienced over this event was a hindrance to our early progress on this project, and it certainly impacted our rational decision making processes.

Had this been our only obstacle, I suppose we might still have made good on our efforts. But at the time we were "in for a penny, in for a pound", and decided that this was a good time for several other pending life changes as well. So we liquidated our assets, sold our house, and moved. Not just to a new place or a new city, but to a new country. With our U-Haul packed and itemized, we headed north across the border into Canada, winding up in my birth city, Edmonton. In retrospect this was certainly the right decision for our family as a whole, but it was not necessarily the best decision for starting our business.

This choice had a lot of unanticipated knock-on effects for starting our company and beginning the Fashion Mash project. First were some of the direct effects, such as the practical problems with starting and running a business in a country where the legal, social, and cultural foundations surrounding employment and business are very different from those with which we were familiar. While Canada and the US are oft compared and the differences minimized, I can assure you that the similarities are superficial only. Under the hood there are a LOT of important differences, and many of the practical things we had learned from our years working in the US had to be re-evaluated.

Another direct effect was that we had shortened our runway. While attending a local entrepreneurial course at the Anderson Career Training Institute, an excellent analogy which I found particularly endearing was presented to us. The startup phase of a new business was compared to a plane taking off. Your starting capital, resources, energy, opportunities, and initial conditions are your runway. If you can't get your business airborne (self-sufficient) by the end of your runway, disaster is the probable outcome. By moving to Canada in general and Edmonton in particular, we effectively doubled our cost of living and about halved the length of our runway.

The foreshortening of our resources was not the only unexpected limitation we encountered financially. A word of advice for those looking to hop the border -- your credit history does not immigrate with you. And you probably aren't a student any more. The practical implications this has for starting a new business are profound. Our first credit card was a $300 secured card; we were not able to graduate up to an unsecured credit line (with a paltry $1000 limit) for a YEAR. I won't even go into the farcical details surrounding the Catch-22 of trying to sign a rental lease when you don't have a bank account, and getting a bank account when you don't have an address. (It's hilarious now, of course, but in the deep February winter, living out of the Jasper ave. HoJo with 3 cats, it wasn't quite as funny).

The last direct effect was that my wife was unable to work legally in Canada for pretty much the entire life of our company and the project. Which isn't to say that she didn't do an insane amount of work for us, but the roles she could take on were often limited by the fact that she could not legally be a director or even employee of the company. Unfortunately her immigration was not pushed through until long after we had floundered off the end of our runway and changed focus in our life. Of course the immigration process itself is something we probably could have done much more efficiently too, but that is a topic for another day.

Beyond these direct, practical obstacles, there were also some fuzzy factors that made things difficult as well. By moving to a new country we had effectively cut ourselves off from all of the support of our family and friends. We had distanced ourselves from all of our business and professional contacts and the networks we had established over the years. Though we made significant effort to set down roots and network in our new home, the simple fact is that these things take a lot of time. This was further hampered by some of the cultural and social differences we had not accounted for. As a result of this, our runway was not only shorter than it could have been, but far bumpier as well.

And yet, for all of these mistakes, I think we still could have made it, but for what I believe to be our most fatal choice. Fashion Mash was not the only project we wanted to execute under the Heliostorm banner. We were constantly thinking about what we would get to next, and not spending enough time thinking about what we were doing right now. Even more importantly, Fashion Mash was not even our favourite project idea, and our choice to implement it instead of our other ideas had more to do with our perceptions of marketability and feasibility than with an essential passion for the concept. And I'm convinced that this lack of passion, more than anything else, led to the site's eventual fate under our stewardship. At every turn it sapped our energy and killed our momentum, even when we managed to fight through the many other obstacles and produce a compelling product with avid and interested early adopters.

Though I have focused on many of the negatives, there were also a lot of things I think we executed very well with this project. And the failures are certainly educational, and I think invaluable for our future goals. Already my wife has leveraged the lessons learned with her follow-up business; a successful family daycare that has been running self-sufficiently for more than a year.

If we'd had the necessary passion we could have fought through anything to finish our takeoff roll. With a longer, smoother runway we might have pulled it off regardless. But together these two factors conspired to bring our first entrepreneurial project to a dignified but less than spectacular close. I've no doubt, however, that I'll be back in the pilot's seat.

Technology is Neat, or How I Learned to Love the Virtual Machine

2009-01-14T01:53:00.018-07:00

In my last post I talked about my current development environment, and one key aspect of this environment is the use of virtualization and virtual machines. While virtualization is not a new concept (to me or to the world), it has only recently crossed into the realm of day-to-day utility in my life, and I find the entire topic to be one of those eye-catching technological wonders that throws me back to the first time I turned on a VIC-20 or started hacking away at an Osborne. In short, it makes me smile and think, "Hey... that's really neat!".

Virtualization can mean a lot of things, but lately it's been the new, trendy way of talking about what used to be lumped in with emulation. At the technical and semantic level, they are not the same thing: in emulation, the hardware you present inside the VM is entirely abstracted and is often not the same as the physical hardware of the host (e.g. emulating a Nintendo-64 on PC hardware), whereas in virtualization the guest machine sees the architecture of the host and probably even has direct access to some of its bits and pieces. From a practical standpoint, the most common end-goal for both is the same: inside a "host" OS running on real, physical hardware, create one or more "guest virtual machines" (VMs or guests) that look just like real, physical machines to anything that runs on them. The practical upshot of all this is that one piece of physical hardware can be running multiple OS instances, not even necessarily the same OS, and they all think they are running on their own little piece of hardware without a care in the world.

Again, this is nothing new. Virtualization has existed as a practical reality since at least 1972. My first negative experience with it was VMWare, and my first positive experience with it was Xen. But only recently has it truly become a useful tool for my day-to-day tasks, for a number of different reasons.

Virtualization used to be very slow and buggy in the x86 world, which is one of the major reasons that I stayed away from it. VMWare was always such a let-down for me; what they accomplished with the hardware at hand is really pretty amazing, but for day-to-day use it was far too slow and unstable for my tastes. However, hardware support for virtualization finally entered the mainstream x86 processor market (AMD and Intel CPUs) around 2007, which has opened up a world of stable and fast virtualization options.

Another limitation had always been hardware resources, and specifically memory. I've rarely had enough RAM to comfortably run my regular OS, let alone a couple of tag-along virtual machines. It's not always been just about price, but also the 4GB RAM barrier of a 32-bit OS. But with Vista 64-bit and the current prices for RAM, I've finally been able to afford and actually use a surplus of cheap memory; far more than my OS currently needs, and more than enough to handle the requirements of several VMs simultaneously.

Lastly, there seems to be a wellspring of VM options these days. VMWare has come out with a free version, Microsoft has entered the fray, and there are a number of other choices. With all of these options, I was able to find one that met my needs: Sun's VirtualBox. It's free (hey, what can I say, I'm cheap). Setting up a VM is easy, running multiple VMs is stable, and it "just works". It supports seamless mouse and keyboard integration with the guest machine, so the window running the guest OS behaves almost identically to every other window on my desktop. It supports a network mode that gives the guest OS full visibility on my internal network through promiscuous use of my NIC (sounds kinky, I know, but it isn't). And it can share files and folders on my host OS with the guest OS through an embedded file-share device that behaves just like a network mount.

With all of these forces combined, I've suddenly started using virtualization all over the place. As I already talked about in detail, I've been using it to run a guest Linux webserver VM on my Vista host in order to serve up the web apps I'm actively developing. I also use it to run a Windows XP image in order to test with IE6.

Once I started with this setup, I found other handy uses for my little virtual machines. For instance, my day-job requires that I spend a lot of time connected to various corporate VPNs, and these have an annoying tendency to kill my regular network connectivity. Also, a lot of them don't work very well on Vista; especially not 64-bit Vista. No problem now, though; I have an XP VM dedicated just to VPNs. It has all of my VPNs configured and I can connect anywhere I need without killing my host OS network. Even better, when I go on the road, I simply copy over the VM image to my laptop and run it there, without having to worry about maintaining two sets of configurations or worrying about OS compatibility.

And perhaps the most endearing and important aspect of it all is that it's just really neat! Running a Linux webserver in a little window on my Vista PC as I edit the files it serves up in real-time on my host OS while simultaneously testing these changes using IE6 on another version of Windows running inside another little window... well, that's just fundamentally cool. And I don't even take advantage of all the other nifty things you can do with virtualization, like migrating a running virtual machine from one physical machine to another without interruption, or scaling physical resources on demand between multiple VMs. With all of this cool techno-wizardry going on, it's easy to see why I finally learned to love the virtual machine.

Fallout 3 Modding - Primer

2009-02-16T18:04:00.022-07:00

In this post I want to spend some time talking about the fundamentals of Fallout 3 modding; the engine and how it loads mods, the relevant files and file types involved, and the various tools and their purposes. This primer is not intended to serve as a practical modding guide; it will not necessarily teach you how to do anything just yet; we will not be building any tutorial mod in this post or even opening any of the tools. What it will do is set the ground-work for understanding how modding works, all of the files and locations involved, and should provide a much better place to start from when approaching all of the many practical tutorials already available on the web. This guide will also, as an added bonus, provide a relatively comprehensive list of the tools you will need (and explanations of why you will need them), as well as links to additional resources where you can get your feet wet with practical applications.

Engines, Masters, and Plugins, Oh My!

The engine Bethesda used for Fallout 3 is the Gamebryo engine. This is not the only game to use this engine; in addition to other games out there, Bethesda has used progressive versions of this engine for Morrowind and Oblivion as well as Fallout 3. The details I will discuss in this series are most relevant to Fallout 3 modding, but a large host of details would work equally well for Oblivion modding, which uses a very similar version of the Gamebryo engine.

The first thing to know about modding Fallout 3 is how the engine loads the game world. There are two types of data files; "Master Files" with the suffix ".esm", and "Plugin Files" with the suffix ".esp". In general, the engine usually loads only ONE master file, and then loads any additional plugin files that are available and configured. Modders are usually working with the plugin ".esp" files. It is possible for the engine to load more than one master file, and some mods do need to be masters (such as the case of a total conversion), but these cases are few and far between so I'm not going to discuss them here.

In Fallout 3, the entire game world is loaded from a master file that came with the game called "Fallout3.esm", which is located in the "Data" directory of your Fallout 3 installation (probably something like C:\Program Files\Bethesda Softworks\Fallout 3). This file contains or references everything necessary to load the game world; all the layouts, NPCs, quests, dialog, models, items, etc. After this master file has been loaded, the engine can then load any additional plugins that it finds in this same "Data" directory (files ending in ".esp"). You can choose which plugin files the game should load using the "Data Files" option in the Fallout 3 launcher.

In essence, plugins modify the data in the master file that is first loaded. The plugin can add new elements, remove existing elements, or modify existing elements. We'll go into more detail about how this is done later, because it is important, but it suffices right now to know that the master file contains the original game world, and the plugin file you create contains all of your changes to the original game world.

The last thing to note is that the master file and the plugin files are essentially identical in content and structure. The concept of a "master file" is more logistical than technical; it just indicates that the file has a self-contained and complete description of a world, and it is the file which should be loaded first to form the base against which all additional plugins make modifications. A plugin file can also be this "self-contained" if so desired, though realistically we often want to use plugins to make changes to the existing world rather than defining an all-new world, because, hey, that would be a lot of work :)

Resource Data

You may have noticed earlier that I said the master file "contains or references" everything needed for the game world. What a master file or plugin file "contains" is the logical description of the world and its elements, but all of the "heavy hitting" items such as game models, textures, sound resources, and so on are actually stored in a different place. So the plugin file (and the original master file) may store information about, for example, an NPC -- that it exists, what its name is, where it is located -- but some of the information about the NPC in the plugin file will be references to external elements, such as which model and texture to use and which voice file to use for its spoken dialog.

These references are defined in the plugin file, but the actual model data, texture data, and sound data live in one of two places. The first place they may live is in a ".bsa" file, which probably stands for "Bethesda Softworks Archive" or something equally witty. These ".bsa" files are also located in the "Data" directory alongside the master and plugin files, and you will notice that the original Fallout 3 game comes with several of these archive files. These contain all of the original game resources, and are significantly larger than the master or plugin files.

The second place that this data can live is in subdirectories within the "Data" directory. These subdirectories contain individual resource files, but you won't see any in a fresh Fallout 3 installation because all of the original game resources live in the ".bsa" archives instead. However, you can put new resources, such as new models or textures, in subdirectories within the "Data" directory for your plugin to reference, and the game will load these just as it does the resources in the archives, allowing you to add completely new content in this way.

In fact, if you were to look inside the ".bsa" archive files that come with Fallout 3, you will see that the archives are really just a collection of resource files organized in a directory tree. It is possible to extract these archives into a separate location so that you can look at all of the resources available in the game, and even use them as a basis to make new resources for you plugin, e.g. by taking an existing model and applying a new texture. You would then put the new resource into a subdirectory within the "Data" directory and reference it in your plugin. The engine would then load up the referenced resource and display it in game!

Further, it is even possible to override the resource files that are contained within the original game archive files, by placing an identically named resource file in an identically named directory tree within the "Data" directory. To make sure this is what you intend, Fallout 3 additionally requires a special ArchiveInvalidation.txt file be created in the "Data" directory that lists all of the things in the archive that you want to override with copies in the local directory tree. However, there is a tool that makes this unnecessary, so I won't describe the details of messing with the ArchiveInvalidation.txt file, but it's useful to know at least that much about that process when it comes to troubleshooting.

Finally, you might be interested in what type of resources are available. Links to tools for working with these will be mentioned later in the tools section:

sound formats include wav, mp3, and ogg; additionally, lip files are used for dialog lip synching
all textures are stored in the standard dds, using DXT1 compression
models are stored in the nif format, which is also used for save-games and some other things
lots of other things, like skeletons and animations and other bits of data that I'm not going to enumerate here

Plugin File Structure

In theory, you rarely have to worry about the underlying structure of the plugin or master files, because this is all handled transparently by the editor tools for you. In practice, unfortunately, there are often conflicts and issues that can be most simply and easily discovered and resolved by low-level inspection and editing of the plugin files. Knowing about these fundamentals will very likely save you (and your users) hours of frustration in the future.

As mentioned earlier, master and plugin files are quite similar. In fact, when it comes to the underlying structure, they are basically identical, except for a few flags that mark a master file as a master. Simply put, these files are collections of records. A record is an entry in the file that has a type, some data, and possibly some child (or sub) records. So a master file or plugin file is simply a collection of records and subrecords, nested as deeply as needed, each containing data about the element it describes.

Every item, NPC, quest, script, location, effect, and any other element of the game world is contained and described by one of these records. The type defines what the data for that record contains; the SCPT record data contains a script, the QUST record data contains information about a quest, and so on. Some record types are quite complex, and they need additional subrecords in order to completely describe the content because the record data itself is not sufficient or dynamic enough. This makes implicit sense; if you have a record that describes a complex entity like a room, for example, you would expect that it would have a bunch of subrecords, one for each object contained inside that room. And it does.

Reaching back to the discussion on plugin loading, you may remember that first the master file is loaded, and then all plugin files are loaded. At a low-level, what this means is that all of the records in the master file are loaded, and then all of the records and modifications to existing records are loaded from the plugin files. This is how a plugin is able to modify or delete existing content, rather than just adding new content; the plugin can instruct the engine to modify or even delete records that have been loaded previously.

This record-based approach gives plugins their power and flexibility, but it can also cause some problems. If you make modifications to the same records (i.e. elements within the master file) as a different plugin, you may have conflicts with that plugin. It is also very easy to accidentally modify or delete records from the master file unintentionally, which is generally referred to as a "dirty" mod. This happens routinely when you are in the editor looking at existing content from the master file, and accidentally delete or move an item without noticing. Not only can this cause annoyances within the original game world completely unrelated to the scope of your plugin, but it can also cause unintentional conflicts with other plugins that would not ordinarily conflict with your mod.

When this happens, the only way to undo your unintentional changes are to find the DELE records in your plugin that tell the engine to delete the other record loaded from the master file, or else to find the records in your plugin that are making modifications to the master file records that you didn't really want to change (such as giving it a new position or rotation, a ridiculously easy thing to do by accident in the editor).

To track down records and items for these (and other) troubleshooting purposes, it's helpful to know that every record has a FormID. The FormID is a unique hex identifier for that record. You will use this quite often in scripts or when looking through the records of your plugin for specific changes. These FormIDs are created automatically by the editor tool, though you will have to set a "prefix" so that the FormIDs in your mod are unique from the FormIDs in the master file and everyone else's plugin.

In addition to the FormID, many elements in a plugin also have an EditorID. The EditorID is something that can be set within the editor and is a useful way to unique identify items within scripts and references without having to figure out the FormID. I mention this mainly because it is important to know the difference when troubleshooting, but some tutorials use the terms interchangeably, which can make that confusing. Interestingly enough, EditorIDs are in fact defined by an EDID record, which will be a subrecord of the item to which it applies.

Tools and Resources

One thing to note before going into the tools and resources is that the Fallout 3 engine is just a newer version of the Oblivion engine. If you cannot find a specific resource, tool, or tutorial about how to do something in Fallout 3, it may be possible to find equivalent information for doing that same task in Oblivion. Since it is (mostly) the same engine and even a similar editor, the information you learn from an Oblivion tutorial transposes quite nicely to Fallout 3.

Before installing the tools, you may want to re-install Fallout 3 to a simple base directory, like "C:\Fallout3". This makes your file paths much simpler and easier to find, and it can also prevent some known-issues with Vista and UAC when operating out of the Program Files directory. This is entirely up to your discretion, however.

The Garden of Eden Construction Kit (G.E.C.K.)

Get it here.

The G.E.C.K. is the official editor for Fallout 3 and is the modder's bread and butter tool. It is where you do everything you need to edit the plugin files; create and modify items, quests, NPCs, rooms, layouts, etc.

FO3 Archive Utility

Get it here.

This archive utility lets you inspect and extract files from the ".bsa" archives. This allows you to extract resources from the original archives so that you can re-use original game components. You will almost certainly want to do this extraction before starting on your mod. Even if all you want to do is re-use existing resources, the editor does not know how to select these items out of the ".bsa" archives. Instead, you must select it from the extracted resources, and the editor then strips off the root of the resource path (everything before the "Data" directory part). When the engine loads, it then uses this relative path to lookup the resource in the ".bsa" files (or on disk, if a copy exists in one of the subdirectories in the "Data" directory and the necessary override has been configured). So the engine loads the resource from the archive, but the editor cannot select it from the archive, hence the need to extract even for simple re-use.

It is not recommended that you extract these to the Fallout 3 "Data" directory, as you will then be potentially overriding resources within the ".bsa" files, which you don't want to do (except in the specific cases when you do want to do that, but you don't want to be doing it in the general sense for every single resource). Since I installed Fallout 3 to "C:\Fallout3", and I put all of my tools in "C:\Fallout3Tools", it made sense for me to extract the original game archives to "C:\Fallout3Tools\Data" to mirror the original structure, but you can put them in any location that makes sense for you.

This tool will also let you create new ".bsa" archive files, which is a great way to distribute any new resources for your mod in a single self-contained way, instead of requiring the end-user to extract all your resources into individual subdirectories within the "Data" directory.

ArchiveInvalidation Invalidator

Get it here.

As you may recall during the discussion on resources, it was necessary to use an ArchiveInvalidation.txt file if you wanted to override resources within the ".bsa" archives with local copies in the "Data" directory. This tool makes maintaining the ArchiveInvalidation.txt file unnecessary, and instead forces Fallout 3 to check in the "Data" directory first and use anything it finds there to override the ".bsa" archive versions.

Remember that if your mod overrides built-in resources, the end-user will also need to install this tool (or update their ArchiveInvalidation.txt file) in order to use your mod properly.

Games for Windows Disabler

Get it here.

Bethesda chose to include Games for Windows Live as part of the Fallout 3 package, which has been something of a controversial issue. I'll remain safely neutral on the point here, except to point out that during mod development you will be potentially loading and reloading Fallout 3 dozens or hundreds of times. This tool will disable the use of Games for Windows Live within Fallout 3, making startup faster and trouble-shooting potentially easier.

NifSkope

Get it here.

The NIF model format is somewhat proprietary to the Gamebryo engine, though there are plugins to work with it in Blender and other tools. That said, the de-facto tool for viewing and editing NIF files directly is NifSkope, which lets you preview model files and even make simple changes (such as re-texturing).

DDS plugins

For Gimp.
For Photoshop.

The Direct-Draw Surface format is relatively common, and the above links provide plugins for the two most common graphics packages that you would use to edit these textures, which is the most likely use-case. However, there are many more DDS tools available out there; viewers, editors, etc., if you need to do something else with them.

Fallout Mod Manager (FOMM)

Get it here.

Installing and managing mods is not intensely difficult, but many users like to have a tool that handles the task for them. The FOMM is probably the de-facto standard mod manager that end-users have, so it makes sense to be familiar with its features and to make your mod FOMM-friendly.

FO3Edit

Get it here.

The G.E.C.K. editor is a great tool, but when you need to get into the nitty-gritty record-level details of your mod to resolve conflicts or clean up dirty modifications, you'll want to use the FO3Edit tool to manage the low-level details.

Resources

G.E.C.K. wiki - the official wiki for the G.E.C.K. tool and for modders
Fallout 3 Forums - the official Fallout 3 forums at Bethesda's website
Vault Tec Labs - a Fallout 3 modding wiki at wikia
Fallout 3 Nexus - a site with Fallout 3 mods and utilities

Posts in this series:

Fallout 3 Modding - Introduction
Fallout 3 Modding - Primer

All of my posts about Fallout 3 modding.

The Reality of Edmonton Urban Sprawl

2009-12-15T12:17:00.029-07:00

I was listening in on the Edmonton City Council's public hearing regarding the proposed west and southeast LRT routes, when someone brought up that old bugaboo about Edmonton being quite possibly the worst major city in the world for urban density. You don't have to look far to find this claim... the first paragraph on Wikipedia will do the trick:

At 684 km², the City of Edmonton covers an area larger than Chicago, Philadelphia, Toronto, or Montreal. Edmonton has one of the lowest urban population densities in North America, about 9.4% that of New York City.

This claim has always bugged me; not because it's strictly untrue or even fundamentally invalid, but mainly because it's spiritually misleading. To show why, let's start with a naive comparison of city population density for some of the cities mentioned. This is easily determined by dividing each city's population by its area, using the numbers from Wikipedia:

City	Density	Area
Edmonton	1067.2/km²	684km²
Calgary	1360.2/km²	726km²
Toronto	3973.5/km²	630km²
Chicago	4707.3/km²	606km²
New York City	6887.1/km²	1214km²

This looks pretty bad; 27% of Toronto's density and only 15% of New York City's density. With numbers like that, we must all be living on acreages in the middle of downtown Edmonton! Of course, a quick look at a couple of maps will tell us why these numbers are not really that helpful for comparing density in these cities.

We can see in the top two images (Chicago and Toronto respectively) that the city limits don't even come close to encompassing the actual urban population of those cities. Let's see how Edmonton compares...

Hmm... those city limits seem to include quite a bit of farmland, and don't reflect the urban/suburban population distribution all that closely. You can see that the entire northeast quadrant is basically empty, not to mention the fringe of farmland around the west and south sides.

Further, you might notice the green strip of land and the river running through the heart of Edmonton. Heading on over to trusty Wikipedia, we find that between the river valley park system and local neighborhood parks, Edmonton has over 111km² of parkland. With a city area of 684km², this means more than 16% of the city area is being used for green space. In contrast, the famed Chicago Park District, the largest urban park system in the United States, is approximately 30km², or only 5% of its city area.

A more enlightened approach might be to then compare the "metropolitan" area of Edmonton with those of other cities. Using metro numbers, we get a table that looks like:

Metro	Density	Area	% metro pop outside city limits
Edmonton CMA	109.9/km²	9418km²	30%
Calgary CMA	224.1/km²	5083km²	13%
Toronto CMA	866.0/km²	5904km²	51%
Golden Horseshoe Core	642.5/km²	10097km²	61%
Chicago Metro	347.5/km²	28163km²	71%
New York City Metro	1092.0/km²	17405km²	56%

This seems like a perfectly valid idea, but it again falls short in implementation. The numbers used for the comparison for Canadian cities is the "Census Metropolitan Area (CMA)", a census apparatus defined by Statistics Canada. The numbers for the US cities are basically whatever Wikipedia came up with, and I'm sure the methodology doesn't line up 1 to 1. In fact, even between Canadian CMAs, comparisons are difficult. The Edmonton CMA is the largest CMA in Canada, and yet only 30% of this CMA's population lives outside the city limits of Edmonton (which is a refreshing number to help put our sprawl issues in perspective).

In contrast, the Toronto CMA doesn't even match the area known as the Greater Toronto Area, and arguably the more valid comparison would be the "Golden Horseshoe Core", which encompasses a total area equivalent to the Edmonton CMA. The Golden Horseshoe effectively supports the Toronto urban centre, and 61% of this population is "sprawled" outside of Toronto city limits. In fact, if we were to run the numbers on the extended Golden Horseshoe, which contains about 25% of Canada's population and which is arguably essential to the city of Toronto, we see density drop off to 256.7/km².

The point of all these numbers and comparisons isn't to prove anything specific, or to argue that Edmonton doesn't have more sprawl than it probably needs. It's simply to point out that a blind comparison of density stats tells us very little about the actual density and sprawl issues of any particular city. It also reminds us that the incredibly high population densities of metro centres like Toronto or New York are artifacts of huge populations spread across entire regions, without which the central cities would simply cease to function.

As a prairie city wrapped around a river, the issues Edmonton faces with sprawl are different than many of the cities we are often compared to. We have a strongly identified central core (well, two of them), a really good public transit system for a city of this size -- and if you don't believe this, stop comparing your city to New York or Toronto, and instead try to find a US city in the 750k range that can match Edmonton transit -- and a surprisingly consolidated set of suburbs. From a completely personal perspective, I've lived in cities 1/3 of the size of Edmonton with end-to-end commute times that are worse than anything I've encountered here.

I certainly think Edmonton could use more density, and I'm glad to see that sprawl is a continuing concern. However, I also think that Edmonton is more dense than people realize, and that the overall situation is not as dire as some believe. It's nice to see that with our hyper-sensitivity to sprawl, it seems unlikely that it will ever be left to get truly out of hand.

On a final note, I think that one of the reasons the sprawl issue is both frustrating and exciting is because it means that the average Edmontonian is not content with simply living in a Houston or a Denver. We compare ourselves to cultural giants of cities; Toronto, Chicago, New York. We feel our transit system is inadequate because we don't have the London underground, or that our density is suspect because we don't have a New York skyline. I'm glad that we hold ourselves up to this incredibly high standard. I hope that we never stop striving to be a world-class place to live -- it's what I love about this city.

Fallout 3 Modding - Introduction

2009-02-15T11:01:00.000-07:00

I have recently decided to create a mod for Fallout 3. Since I am just starting out, I don't plan to talk about the details of the mod just yet. However, the project is relatively ambitious and there's no guarantee that it will ever see the light of day, so I wanted to write a couple of posts about the experience so that some utility may be gleaned from the endeavor, regardless of the eventual outcome. These posts will probably vary between simple curiosities to full-fledged tutorials; hopefully they will all share at least one thing in common, and that is to be in some way be helpful to other Fallout 3 modders and modder-hopefuls.

This will not be my first experience modding this particular engine, since it is the same engine used by the game Oblivion. I have already written a mod for Oblivion, which was in many ways a much more technically challenging mod due to what it was trying to accomplish and the ways in which it pushed the engine. Its coolest feature was arguable, but its second-coolest feature was definitely the fact that the mannequins introduced by the mod would disappear from the game after a few days time, taking all of your best armor and weapons with them. I like to pretend this was intentional; that my Oblivion character was sitting on a hoard of stolen loot inside her castle keep, cackling gleefully under the moonlight as armies of magical mannequins quietly converged on her lair in the depths of the night, dumbly ferrying the stolen booty to their evil master. In reality, of course, it was just a bug, and one I never got around to fixing at that.

The point of this digression is not just to reminisce about my past glories, but also to plan out at least a few of these posts. While I should have known what I was doing going into this mod, I found myself struggling to remember most of the important details. Further, while there were many practical "how-to" tutorials on specific topics, there were very few tutorials discussing the basic details of how mods function and interact with the engine, which are things that I had to learn the hard way and which are just now slowly percolating back into my working memory. While these practical tutorials are probably very useful for some people, I find myself to be more of an abstract, theoretical learner. A step-by-step guide does very little to help me out if it does not explain the "why" behind each step.

Keeping this in mind, I want to write a primer that tackles two main points. First, I want to talk about the fundamentals behind modding this engine, because I feel this is an area that is most lacking within the existing knowledge-base. Knowing about the basics of the file layout, structure, and data resources used by the engine make a lot of common problems much easier to understand and resolve. This will be somewhat technical and abstract, and is intended to introduce all of the basics to someone who knows nothing about modding Oblivion or Fallout 3. Second, I want to build on that basis to provide a comprehensive list of tools required to work with each of these fundamental areas. Instead of just randomly downloading tools and blundering through a "My First Dungeon" tutorial, I hope that this combination will give you a firm understanding of what each of the tools does and why you will need it.

After that, we'll just see where things go. As I make progress on the mod I will probably be making additional posts about the various things I run into. I'm sure at least some of them will be interesting :)

Posts in this series:

Fallout 3 Modding - Introduction
Fallout 3 Modding - Primer

All of my posts about Fallout 3 modding.

Rotation Tribulation

2007-09-08T22:22:00.006-06:00

I love spending hours tracking down obscure API issues that turn my carefully crafted code into a crap machine. It's always exciting, and in the end I feel a healthy sense of accomplishment for having exhausted hours of intense study in order to make something work. The way it was supposed to work in the first place. If you can't guess already, I just had another one of those wonderful sessions with PIL -- the Python Imaging Library.

Now, in it's defence, this is pretty much the first issue I've run into with PIL. Both functionally and documentation-wise, it has been exceptional all around. So I can't really complain TOO much. That won't stop me from complaining just a little, though.

All I wanted to do was rotate an image. And while PIL happily obliged with a handy "rotate" function all rearing to go, the result was... well, it was crap. It looked like something you might expect from a CS student who had just learnt matrices and was busy rotating pixels gleefully, completely oblivious to the swath of destructive artifacting in their wake.

Here are the important bits of the rotate API documentation in the PIL handbook:

im.rotate(angle, filter=NEAREST, expand=0) => image

Returns a copy of an image rotated the given number of degrees counter clockwise around its centre.

The filter argument can be one of NEAREST (use nearest neighbour), BILINEAR (linear interpolation in a 2x2 environment), or BICUBIC (cubic spline interpolation in a 4x4 environment). If omitted, or if the image has mode "1" or "P", it is set to NEAREST.

The expand argument, if true, indicates that the output image should be made large enough to hold the rotated image. If omitted or false, the output image has the same size as the input image.

Here's the tricky part it doesn't mention: if you actually use the expand option, it will completely ignore any filter you provide, and simply use the NEAREST method. Which might as well be named UGLY, because that's what you get. Oh, and nevermind the fact that the actual argument isn't even filter at all, but resample (and yes, that matters when trying to pass a named argument in Python).

The fix? Trivial, once you've spent a few hours wondering why none of the filters make any difference and poked your way through the code to find out it's happily ignoring your settings entirely (no... I'm not bitter... why do you ask?). Simply pretend the "expand" option doesn't exist, and instead manually expand the image size prior to rotating so that it can hold the rotated image. A little bit of trig and you're good as rain. Then you can happily rotate away with expand turned off, using the BICUBIC filter, which looks oh so pretty.

Meez

2007-03-29T21:06:00.004-06:00

Meez is a large part of what we planned to do with Avatude. One less thing to worry about there, frees me up for the rest of our projects! :) Maybe I'll just start from that as a base and do the other Avatude parts on top of it, without worrying about custom avatars at this point. Here's my Meez:

What do you do with a drunken... website?

2009-01-08T10:37:00.010-07:00

I run the Fashion Mash website, together with mezamashii, and lately I have been spending a lot of time thinking about where to go from here.

The reality is that we made a lot of mistakes during the development of Fashion Mash. We started down a promising path with a utility concept, but wandered astray into a fashion-based world that we had very strictly meant to avoid. Even the name was, in retrospect, a poor and misleading choice. This was never meant to be a fashion site but a wardrobe utility.

As a result of our many misjudgements, the site became something of an unwanted child, a poor lost project with differing taste and opinions from its parent creators. It has been neglected and left to fend for its own for many months as our attentions turned to projects more exciting or practical.

The thing is, though, that in spite of our downright abusive neglect towards this poor little website, it has still managed to keep puttering. We routinely get new signups and there is active, though somewhat anemic, community participation. There are frankly so many things wrong with the website in its current state that I'm amazed it still manages to pick up several users a week.

All of which leaves me in a bit of a quandary. The site itself still has quite a lot of untapped potential. It may never be the next Facebook, but it is certainly a concept that people seem to find attractive and, at the moment, there's only a smattering of competition. I'm certain that with a major overhaul based on all that we've learned so far, the turnover and community participation rates could be greatly improved. Of course, that would be a lot of work, and without the interest to drive that effort it becomes a mere drudgery.

And that, I suppose, is the real problem. We've fallen out of love with the site itself, and our obligations to it feel like exactly that; a sense of forlorn duty rather than one of excited passion.

As I often do in these cases, I turned to the Lucky Honu Oracle in hope of inspiration. While I have no belief in its mystical capacity, I find the presentation of a presumptuously authoritative answer to be an excellent jump start to the introspective process. My gut reactions to its prophecies frequently tell me a lot about my true feelings on the matter.

I asked the I Ching Oracle: What should we do with Fashion Mash? Fix it up or let it lie?

37. Chia Jen - Family Put the family or group first. Take the mother figure into account when making decisions.

What does the Oracle hold for you? http://luckyhonu.com/

Unfortunately, this hasn't yet opened up any new wellspring of insight into my problem. Devote energy into fixing the site and hope the spark rekindles, or let it lie dormant and let bygones be bygones?

I wonder what the legal precedent is for divorcing a website.

My Development Environment

2009-01-13T17:50:00.005-07:00

My answer to a recent stackoverflow question got me to thinking about the long and painful process I went through in creating a development environment that met all of my idiosyncratic needs.

The question asks people what kind of setup they use for their development environments. My environment for building web applications and other personal projects went through a lot of iterations before I finally settled on one that met all of my needs and did not irritate me in any discernible fashion. Some of the personal requirements that I slowly identified as I went through this process were:

I need Windows: First and foremost, I'm a PC gamer. Additionally, my day-job requires a variety of Windows applications, so I need a functional Windows desktop no matter what the underlying task. Also, many of my personal projects are built in Visual Studio.
I need Linux: I'm also an open-source web developer and so I need Linux for many of those projects too. I use a Linux server for production deployments and it helps to have a functional Linux development server that matches that environment as closely as possible.
I treat personal projects like real ones: For even the simplest of personal webpages, I like to have a development environment separate from my production environment. I even like to have a test environment too, but that's not generally as critical. While it may seem like overkill for a lot of basic tasks, it enforces good habits and has saved my bacon too many times to count.
I need to edit locally: While I have nothing against network mounts, and in fact use a really cool NAS device for a ton of handy things, I've always found the random network delays an annoyance when editing text files. I'm a compulsive saver and every slight delay drives me nuts.
I don't like messing with character encodings: DOS newlines, UNIX newlines, UTF vs. standard ANSI... I'm very picky about my character encodings. I want files deployed to my production Linux environment to have UNIX newlines with UTF-8 encodings, no exceptions.
I'm power-conscious: At the end of the day, I want to be able to turn off as many of my electronic devices as possible, as easily as possible.
I like to share: I need to be able to share up-to-the-second changes in development with my wife and co-developer, with no more hassle than looking at the changes myself.

Of course, there's no single magic bullet that resolved all of these needs at once. Many of my setups excelled in some areas while falling down in others. I had several setups that were "good enough", but the slight deficiencies constantly irritated me. In some ways these were the worst, because I always felt guilty about spending time trying to improve the setup when I should have been working on code instead. But I did, finally, settle on a setup that meets all my needs and has no apparent annoyances. Without further delay, I present it to you!

The Host OS

I tried dual-booting various versions of Windows with a Linux desktop environment for a long time -- I've toyed on and off with that kind of setup for almost a decade now. But the maintenance overhead of a dual-boot system has always been a drawback to me, as well as the complexity and potential (often realized) for boot-sector SNAFUs. I've also tried Linux as my primary desktop with virtualized Windows environments, but that's always been a problem for gaming and performance concerns.

In the end, I finally settled on Windows Vista, 64-bit. 64-bit allows me to take advantage of all 4GB of my RAM (and more soon, I hope!), and I can play all of the latest and greatest PC gaming titles without problem. Further, because it is my host OS and not just a VM, I always feel like I'm getting "all the bang for my buck" when it comes to gaming or other intensive Windows tasks, like Visual Studio compiles.

The Dev Server

Finding a good choice for my Linux development server has likewise been a long and often twisting road. I've gone from dedicated servers to using my desktop as my development environment and all the way back again, before finally settling on the option that now works for me. I run a Linux (Ubuntu) server as a virtual machine (VM) on my desktop host, using Sun's VirtualBox.

Running a local VM for my development server has solved a lot of problems for me. I can edit files locally on Windows but have the guest OS serve them up without any hassle. The VM instance has its own IP, so other computers on my network can access it just like a physical host. It's easy to start and stop at will, and shuts down when I turn off my desktop at night. It also lets me experiment very easily; I can create a new server image (or clone my existing one) at any time to try out something risky or dramatic, with far less pain and suffering than in the past. It also lets me take advantage of my desktop hardware for something other than just gaming, which makes the cost of my inevitable and continuous desktop upgrades a little easier to justify. This allows me to concentrate my precious hardware funds into one machine.

The Editor

I can't even count the number of editors I've tried over the years. The one I had been using most recently was jEdit. It was powerful, had lots of optional plug-ins, and was cross-platform, which tended to come up a lot as I switched my environment around ad nauseam. But it never handled character encodings in a way that I liked, and it was always a little slow and a little clunky due to its Java implementation. I've always had problems with the damn installer too, which is a trite thing to get hung up on I suppose, but there it is.

Lately I've been using Notepad++ which is fast, lightweight, and handles character encodings exactly the way I want. I can force the default encoding to UNIX newlines and UTF-8 for all new files, and never worry about it again. It clearly indicates which encoding a file is using in case I need to convert something, and has no problem displaying any of the standard encodings I regularly come across. It has excellent syntax highlighting, but it doesn't have a lot of plug-in options, which has actually been a good thing for me. I'm no longer tempted to try all these stupid editor features which seem cool at first but ultimately become a waste of time and don't really address the problem I'm trying to solve.

So, there you have it. My ideal development environment, finally realized after almost a decade of experimentation. So... maybe now I should get on with the task of actually writing some code, eh?

EVE Online - Anathema

2009-04-13T16:31:00.002-06:00

A friend of mine recently posted some screenshots of his favorite mounts, including an Amarri Anathema from EVE Online. Not to be outdone, I had to dredge up a screenshot of my Anathema as well :)

Torque Engine Demo

2007-02-09T12:55:00.002-07:00

For no particular reason I took a Fraps capture of the Torque Game Engine demo application and put it on Google Video. And here it is.

How Not to Upgrade

2007-05-08T12:37:00.002-06:00

I love doing stupid things. No, really, I mean that with all my heart. Every time I do something stupid and manage to recover from it without too much pain, I feel as if some part of my mandatory universal stupidity quota has just been filled. My normal sense of complacency is repealed, I pay better attention to all the buttons I normally push without thinking, and I become hyper-vigilant to avoid doing anything glaringly dumb. For a while.

The latest stupid thing I did happened while upgrading to Ubuntu Feisty. Actually, I upgraded several hosts without a hitch, so really, power to the people at Ubuntu for making such clean upgrade paths (and, I suppose, to the Debian folks for the underlying package manager on which it all depends). Unfortunately, when I pulled the big handle on my primary server and X11 platform, I wasn't paying enough attention. The upgrade rolled around just fine, updating old packages and removing obsolete ones, while I blindly agreed to every prompt that popped up.

The time to restart comes along and... oh, what's this? What do you mean you can't find a kernel?

It's not fair to say that it couldn't find the kernel, exactly. The kernel loaded just fine. Unfortunately the kernel is a Xen Hypervisor, which points to another kernel to use for the Dom0 machine instance which runs my actual server. And that kernel was nowhere to be found. After a bit of digging it became painfully obvious that one of the obsolete packages I had blindly thrown out was, in fact, my precompiled Xen generic x86 kernel.

Oops.

Thankfully I have lots of other internet connected machines at my disposal, and it was a quick matter to grab an Ubuntu Live-CD and boot into a root shell on my server from there. Another few minutes of fiddling and I had apt-get'ted myself the latest Xen generic x86 kernel package. One last smidgen of editing in the GRUB list, and all was well once again.

So other than my heart skipping a beat or ten, this was happily a very painless act of stupidity. At my current quota-filling rate, I expect I'll be posting something eerily similar to this in, oh, another 5 months or so. I can only hope it'll be as easy to fix as this one was.

Running Max Media Manager With nginx

2007-08-05T17:21:00.002-06:00

So I've been mucking around with a variety of webservers due to a number of performance concerns with Apache. I first landed on lighttpd, but it had some ambiguous 404 handling that confused Google. From there I decided to try out nginx, which seems to be a popular webserver among, um... Russian spam and porn sites. That does seem a little sketchy at first, but its hard to imagine a site with higher traffic loads than a Russian porn site. For that matter, its hard to imagine a webmaster with less tolerance for crappy software than the kind of Russian mafia lord that might be running said site. So, really, this all seems to be a good sign for nginx!

While getting Drupal to run in a FastCGI environment is well documented, I also need to be able to run my Openads server (AKA Max Media Manager), a setup which proved to be a little trickier. Ultimately, it boiled down to figuring out and providing all of the necessary headers that MMM uses to setup its web environment. It makes a number of unusual decisions based on obscure things like, for example, the SERVER_PORT being set -- if it's not, you won't get a default http:// protocol. So here's my sample nginx server config which, at the moment, seems to work great with MMM running as a standalone FastCGI process.

server {
    listen       80;
    server_name  ads.example.com;
    access_log   logs/adserver-access.log  combined;
 
    location / {
        root  /root/to/adserver/www;
        index index.php;
    }

    location ~ .php$ {
        fastcgi_pass   127.0.0.1:8080;
        fastcgi_index  index.php;
        fastcgi_param  SCRIPT_FILENAME  /root/to/adserver/www$fastcgi_script_name;
        fastcgi_param  DOCUMENT_ROOT    /root/to/adserver/www;
        fastcgi_param  PHP_SELF         $fastcgi_script_name;
        fastcgi_param  REQUEST_URI      $request_uri;
        fastcgi_param  QUERY_STRING     $query_string;
        fastcgi_param  REQUEST_METHOD   $request_method;
        fastcgi_param  CONTENT_TYPE     $content_type;
        fastcgi_param  CONTENT_LENGTH   $content_length;
        fastcgi_param  SERVER_NAME      ads.example.com;
        fastcgi_param  HTTP_HOST        ads.example.com;
        fastcgi_param  SERVER_PORT      80;
        fastcgi_param  REMOTE_ADDR      $remote_addr;
    }
}

Service Providers

2007-02-10T20:51:00.001-07:00

Recently moved the majority of our hosting to slicehost. Xen based virtual servers; very nice stuff and quite happy with them so far.

Similarly, we moved (or are in the process of moving :) all of our DNS services over to EasyDNS. Also quite pleased with their services to date.

The Oracle is mocking me!

2007-02-10T21:56:00.001-07:00

Stupid uppity Oracle making me do more work... *grumble*...

I asked the I Ching Oracle: Should I fix questions with apostrophes? '
	59. Huan - Dispersion You can succeed in dispersing even the greatest obstacle, provided you sort through chaos to find the source and clear the blockage.
What does the Oracle hold for you? http://luckyhonu.com/iching/oracle

How to Procrastinate Effectively

2007-03-17T10:36:00.001-06:00

I am a procrastinator. There, I said it. I am willing to embrace my true nature!

Four years ago I wrote a novel in what amounts to about three weekends. Three years ago I finished several major edits of that novel in what amounted to a few more weekends. And then... I sat on it. For three years. Blocked, in theory, on the process of writing a ten page synopsis. The kind of school assignment you got every week or two, and were happy about because they told you to use 1" margins and double-space everything, making it that much easier.

Was I truly unable to write a ten page book report after having already written 100,000 words and plowing through three major edits? What in the world was going on?

I've read several articles recently that talk about procrastination as a manifestation of a deep-rooted fear of failure. At least for me, that's pretty accurate. Oh, sure, I can be obsessive and perfectionist at times, but that's not my real problem. Unfortunately, this fear of failure gives me not just one, but two modes of procrastination. The first is that I have a hard time getting started on anything, for fear that it's going to turn out like crap. The second is that I have a hard time finishing anything, for fear that I'll then have to reveal it to someone else and they will see that it turned out like crap.

In the middle ground, between starting and finishing, I tend to putter along quite well. Actually, I can get downright obsessive, what with the classic all-nighters and marathon sessions -- be it writing, coding, or what have you. It's just around the edges where I tend to need that extra motivation. And if I don't get it, well... I end up sitting on a finished novel for three years.

Luckily, I have found a cure -- or at least, a viable cure for me. I quit my job and started living off my dwindling savings account. This has been surprisingly effective at getting me motivated, for a couple of reasons. First, the ever-running calculation in my head of time left to live on remaining money is pretty inspirational all by itself. But just as relevant, for me, is that deep-down, a large part of me thinks that what I should be doing is going out and looking for a new job, post-haste! Get off my lazy butt and start submitting resumes!

Of course, that sounds like a lot of effort, not to mention the crushing possibility of being turned down and unable to find decent employment. Heck, stress like that, it's enough to make me want to procrastinate on it, and do pretty much anything *except* look for a job. And suddenly, writing a ten page synopsis starts to look pretty appealing by comparison. The worst outcome there is an impersonal rejection letter in six months or so. That's far easier than an in-your-face rejection at a job interview. For that matter, even starting a whole new company and working on all kinds of projects that have piled up over the years sounds more appealing than that kind of gross employment failure.

So I embrace my nature as a procrastinator. Maybe a better person could change outright, conquering their procrastination demons once and for all! Me? I'll just settle for outwitting myself -- especially since I know that's not hard to do.