Danny Tuppeny

Is Google Code Readying Git Support?

2010-03-19T18:11:00Z

It looks like Google Code could be working on adding Git support (at last)! I've been following the native Git support issue in the issue tracker and some interesting comments have appeared today:

(8) sussman: Tell you what... as soon as git can do HTTP reasonably, let us know, and we can look into the issue of git on googlecode. :-)
(12) rickvug: Git over HTTP is now more efficient as of version 1.6.6. For more information see http://progit.org/2010/03/04/smart-http.html. I guess this is now something to look into? :)
(13) sussman: We're very aware of this, since it was a googler who wrote the new HTTP protocol. :-)

The fact that Google are "very aware" (not just aware, but very aware!) and one of their own staff has had a hand in this, I wouldn't be at all surprised if Git support showed up soon.

Watch this space! (or, the issue tracker!)

(Solved) Intermittent Server."CreateObject Failed" 'ASP 0177 : 8000ffff' Error Creating .NET COM Components

2010-03-14T11:32:00Z

Over the past year or so we've been wrestling with an intermittent error from classic ASP when trying to instantiate .NET components with Server.CreateObject. Everything works fine 90% of the time, and now and then we'll start seeing this error:

Server object error 'ASP 0177 : 8000ffff' 
Server.CreateObject Failed 
<FileName>.asp, line <LineNumber>
8000ffff

Once this error starts happening, it generally persists until we move the application to another application pool or restart IIS. Recycling the app pool does not fix the problem.

Unfortunately, the error message and number are fairly generic, so it's been pretty tricky to track down. There are lots of people with very similar issues posting all over the web that have solved them in various ways. None of the solutions we found have ever worked for us. Most of them were people seeing the error all the time (eg. permissions errors), but we found very few people seeing our behaviour where the code would just randomly stop working.

After many, many months of searching, we found some reports of similar behaviour being caused by installing an Internet Explorer 7 patch. Many people were rolling the patch back with some success. Rolling back patches doesn't seem like the best thing in the world, so we've always avoided the complications that go with it.

The Solution

Eventually, we found the solution. It's in this Microsoft knowledgebase article KB945701. The problem appears to be a failure to read some IE-related values from the registry. The hotfix on the page above (which is included in the latest service pack, so you probably do not need the hotfix) adds the ability to ignore these errors by setting a registry key:

Locate and then click the following registry subkey:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Internet Explorer\MAIN\
FeatureControl\FEATURE_IGNORE_ZONES_INITIALIZATION_FAILURE_KB945701
Note If the FEATURE_IGNORE_ZONES_INITIALIZATION_FAILURE_KB945701 subkey does not exist, you must manually create it.
Right-click FEATURE_IGNORE_ZONES_INITIALIZATION_FAILURE_KB945701, point to New, and then click DWORD Value.
Type w3wp.exe to name the new registry entry, and then press ENTER.
Right-click w3wp.exe, and then click Modify.
In the Value data box, type 1, and then click OK.

After setting this registry key, a simple app pool restart will apply the change. No longer will your .NET COM components randomly stop working with no real solution except shuffling application pools!

Wanted: Artist for iPhone Game

2010-03-09T22:30:00Z

I posted this on Buzz, but thought it worth also repeating here. I'm working on a few iPhone games, but I'm no good when it comes to graphics. I'm looking for someone that might be able to do the graphics for me. One of the games involves a penguin with an umbrella, falling from the sky :-)

Edit: Got a designer signed up. Time to start coding!

Google Buzz - What's it all About?

2010-02-11T22:09:00Z

Like every man and his dog, I've been playing around with Google Buzz the last few days. So far it's looking quite good. It's not entirely accurate to call it a Twitter clone, but it certainly works in a very similar way.

If you'd like to have your blog posts (or other external content) posted on Twitter, you'd need to find a service that uses the Twitter API to post messages. Google Buzz works the opposite way, and Google will pull updates from other services (Twitter, Flickr, Blogs, Youtube, Google Chat, Picasa, Google Reader) and show them in your Buzz feed.

Currently, there's no built-in way to send your Buzz updates back to these services, though there are a few 3rd party services and workarounds to make this work.

DeWitt Clinton posted an interesting Buzz (?) about where Google Buzz is heading. It focuses on being open and interoperable. Rather than a closed system like Twitter, where you can only share with other Twitter users, Google are promoting an open system where different hosts can exchange messages with each other, like how other systems like Email and Google Wave work.

I have high hopes for Google Buzz. For too long have we put up with Twitter's fail whale and erratic API. It's time for their monopoly to end and everyone to play nicely together.

If you're giving Google Buzz a try, you can follow me using the button in the right column of my Google Profile page. If not, you can still find my ramblings on Twitter.

World of Warcraft Addon Updaters - What Do You Use and Why?

2010-01-28T21:21:00Z

This post is for the World of Warcraft players among you. More specifically, those that use Addons. I'm interested in whether you use an addon updater (that is, software to keep your addons up-to-date). If so, which one, and what you do and don't like about it.

I don't want to get into a debate about Curse, WoWInterface and WoWMatrix (that ship has sailed, and it's been "discussed" more than enough), so please try to avoid mentioning that ;-)

Feedback is encouraged in the comments, the World of Warcraft Forums, or tweeted @NewsWoW.

Open Sourcing Google Wave Notifier

2010-01-22T17:44:00Z

Yesterday I finally got around to do something I've been planning on doing for a number of weeks. I uploaded Google Wave Notifier to Google Code. From today, Google Wave Notifier is Open Source!

In hindsight, I should have done this much sooner. The app is now very stable and contains all the features that I planned to implement (and more!). I'm still getting lots of feature requests, and I really wish I could implement them all. However, in reality, I just don't have time. By sharing the code with the world, all of these features need not go unimplemented!

I still intend to work through bugs and some feature requests myself, though it's unlikely to be at the rate of previous releases. With help from the community, hopefully we'll still see regular releases and new functionality.

For more info, or to download the source, please visit the Google Wave Notifier page on Google Code.

Generic 301 Redirection Script for Google App Engine

2010-01-17T11:02:00Z

Although this post is about writing a redirect script for App Engine, it doesn't require that any of the sites are hosted on App Engine, so it could be useful to you even if you're hosting .NET websites elsewhere, but need to handle redirecting old domains.

If you believe what Derek Says, I change my domain every 5 minutes. While this might be a slight exaggeration, I have moved many domains recently and needed to deal with the usual problems this brings: search engine rankings and existing inbound links.

301 vs 302 Redirects

The most common type of redirect you'll see on the web is a "302 Temporary Redirect". This is what most frameworks will output when you redirect (eg. Response.Redirect() in ASP/.NET or self.redirect() in App Engine). The "Temporary" part of this redirect means the redirect is a one-off and will not always be served. This is handy for example, for redirecting a user back to a page after logging in to your site. Since this page may be different each time, the redirect is not fixed.

The other type of redirect, 301, is a "Permanent Redirect". Most frameworks support these redirects without outputting your own headers. Eg. in App Engine you can use self.redirect(url, permanent=True). This type of redirect means the redirect will always occur to the same link, and that clients are free to bypass your script and assume it will always go to the same place. This is the redirect that we want to use when moving domains. It tells search engines that this page has moved, permanently, and they may associate any rankings for the old page, with the new page.

Generic App Engine Redirect Script

So, now we know what type of redirect to use, it's time to build a script to handle our redirects. If you have multiple domains like me, it makes sense to write a script that can handle them all in one go. I've decided to set up a new App Engine app with the sole purpose of redirects for all my domains from this point forward.

Since I recently decided to move everything from dantup.me.uk to dantup.com, I had a few different domains to redirect. blog.dantup.me.uk needs to map to blog.dantup.com, wavenotifier.dantup.me.uk needs to map to wavenotifier.dantup.com and all of the unused domains (eg. dantup.com, www.dantup.com, tuppeny.com, etc.) need to map to the root of my blog.

App.yaml Setup

If we're gong to be redirecting all incoming requests, we need to route all requests through our script. This is why it's important to use a new App Engine app rather than piggy-back onto an existing one. We'll see up our App.yaml file to route all requests into a script called main.py.

application: myapp-redir
version: 1
runtime: python
api_version: 1

handlers:

- url: /.*
 script: main.py

Next we need to define a way to hold all of the data we'll need to perform our redirects. As well as the old domain and the new domain, we need to know whether to map urls from the request onto the new domain, or just redirect to the root. Eg., when I moved my blog from blog.dantup.me.uk, I wanted blog.dantup.me.uk/mypost to redirect to blog.dantup.com/mypost. However I want tuppeny.com/anything to just redirect to the root, blog.dantup.com.

A dictionary seems to be a good way to store this data because we can perform lookups on the domain quickly, and we can store the new domain and a boolean for the url mapping as a tuple.

# Old Domain: New Domain, Map urls (else redirects to root)
urls = {
    'www.dantup.com': ('blog.dantup.com', False),
    'www.dantup.me.uk': ('blog.dantup.com', False),
    'www.tuppeny.com': ('blog.dantup.com', False),
    'dantup-redir.appspot.com': ('blog.dantup.com', False),
    'blog.dantup.me.uk': ('blog.dantup.com', True),
    'feeds.dantup.me.uk': ('feeds.dantup.com', True),
    'wavenotifier.dantup.me.uk': ('wavenotifier.dantup.com', True),
    'wavenotifier.tuppeny.com': ('wavenotifier.dantup.com', True),
    'go.dantup.me.uk': ('go.dantup.com', True),
    'go.tuppeny.com': ('go.dantup.com', True),
}

In addition to this mapping, we should declare a default domain, so if any requests make it to our script that don't have a mapping, we can redirect there. We'll use a 302 and also log and email this, since it's probably a mistake.

DEFAULT_URL = 'http://blog.dantup.com/'

This is all looking a little complicated, so it makes sense to build in a way to test our mappings without having to set up lots of entries in the hosts file. I've decided to declare a boolean that enables/disables testing. When testing is enabled, if you navigate to /test then it will output a bunch of URLs and the locations they'll redirect to. We'll keep a list of URLs to test in the code:

ALLOW_TEST = True

test_urls = [
    'http://www.dantup.me.uk',
    'http://www.dantup.me.uk/',
    'http://www.dantup.me.uk/blah',
    'http://www.dantup.com',
    'http://www.dantup.com/',
    'http://www.dantup.com/blah',
    'http://www.tuppeny.com',
    'http://www.tuppeny.com/',
    'http://www.tuppeny.com/blah',
    'http://blog.dantup.me.uk',
    'http://blog.dantup.me.uk/',
    'http://blog.dantup.me.uk/2010/mytest',
    'http://feeds.dantup.me.uk',
    'http://feeds.dantup.me.uk/',
    'http://feeds.dantup.me.uk/2010/mytest',
    'http://wavenotifier.dantup.me.uk',
    'http://wavenotifier.dantup.me.uk/',
    'http://wavenotifier.dantup.me.uk/2010/mytest',
    'http://wavenotifier.tuppeny.com',
    'http://wavenotifier.tuppeny.com/',
    'http://wavenotifier.tuppeny.com/2010/mytest',
    'http://go.dantup.me.uk',
    'http://go.dantup.me.uk/',
    'http://go.dantup.me.uk/mytest',
    'http://go.tuppeny.com',
    'http://go.tuppeny.com/',
    'http://go.tuppeny.com/mytest',
]

Now we've set the data up, it's time to write the code to handle the redirects. To allow for easy testing, we'll first create a method that takes a URL and returns where it should map to. This will be called by both the tests and the real redirects.

def get_redirect_url(url):
    scheme, netloc, path, query, fragment = urlparse.urlsplit(url)
	
    # Discard any port number from the hostname
    netloc = netloc.split(':', 1)[0]
	
    # Fix empty paths to be just '/' for consistency
    if path == '':
        path = '/'
	
    # Check if we have a mapping for this domain
    if netloc in urls:
        # Grab the redirect info tuple
        redirect_info = urls[netloc]
        # Root redirects
        if not redirect_info[1]:
            return 'http://' + redirect_info[0] + '/'
        # Redirects with paths
        else:
            return urlparse.urlunsplit(['http', redirect_info[0], path, query, fragment])
    # No mapping, so return None
    else:
        return None

This code is fairly straight forward. It uses our mappings dictionary to look up the domain to redirect to, and whether to include the path information. Next we need to write the code that actually handles incoming requests. This will check whether test mode is enabled, and if the request is '/test'. If so, it will output a table using out list of test URLs above. Otherwise it will call the same method, but actually perform a redirect. If we couldn't match a domain, we'll use a 302 redirect to the default URL, and send an email/log.

    def get(self):
        # If we're allowed to test (eg. local), and requested /test, then output the test
        if ALLOW_TEST and self.request.path == '/test':
            self.response.out.write('<h1>testing</h1>')
            self.response.out.write('<table>')
            for test_url in test_urls:
                self.response.out.write('<tr><td>' + test_url + '</td><td> </td><td>' + get_redirect_url(test_url) + '</td></tr>')
            self.response.out.write('</table>')

        # Otherwise, just go ahead and redirect
        else:
            # Perform redirect
            url = get_redirect_url(self.request.url)
			
            if url:
                logging.info('Redirecting ' + self.request.url + ' to ' + url);
                self.redirect(url, permanent=True)

            else:
                # Log that we didn't know what this was, and redirect to a good default
                logging.error('Unable to redirect this url: ' + self.request.url);
                mail.send_mail_to_admins(
                    sender='"DanTup Redirect" <myemail@mydomain.com>',
                    subject='Redirect Script Error',
                    body='Unable to redirect this url: ' + self.request.url
                )

                # Don't do permanent (301), since we don't know what this is.
                # Move it into the dictionary above if needed
                self.redirect(DEFAULT_URL)

There's a lot of code there, but it should be fairly simple to understand. We handle the test mode by just spitting out a table of our test URLs and the redirects. We can then look over this manually to make sure everything looks correct before going live. Otherwise we work out the redirect for the current request and redirect. If no URL was found, we log and email the attempt, and redirect to the default URL. When the email comes through, we can then add the domain we missed to the mappings dictionary and specify how it should be handled.

Naked Domains on App Engine

You'll notice that "naked" versions of my domains are missing from the script. This is because App Engine doesn't support naked domains, so these are all set up as redirects in my registrars control panel. They support 301 redirects with the same URL mapping options (eg. redirect all to root, or copy the path).

Conclusion

It didn't take much to write a simple generic redirect script, and now we can handle redirects for all domains in the future. This simply needs setting up on App Engine and any number of domains pointing at it. It's worth noting that you can point multiple domains from different Google Apps accounts at the same App Engine app. There is no requirement to use App Engine for hosting your sites in order for this script to be used. The fact that blog.dantup.com is hosted on App Engine doesn't change anything. You could redirect to an Azure site if you wished! Though you probably wouldn't want to ;-)

Full Listing

For convenience, you can get a copy of the full script.

Google App Engine Benchmarks - db.put() Performance

2010-01-16T20:23:00Z

Over the past few weeks as I've been using Google App Engine, I've come across people requesting benchmarks so they can compare App Engine performance to other solutions before they try it out. I don't really think comparing Google App Engine and it's Datastore to something like Azure and SQL Server is all that useful (because you'd generally structure things very different on each platform), but either way, it's interesting to see how things perform.

As well as comparing numbers with other platforms, I think it's worthwhile for App Engine developers to know how the different APIs perform (eg. the difference between fetching something from Memcache and the Datastore). Over the next few blog posts, I'm hoping to provide some numbers I gathered on the production App Engine servers. Please bear in mind that App Engine is still considered a "preview" and as things evolve, performance may change (hopefully for the better!).

The first set of data I gathered was on the performance of db.put(), and more specifically, the difference between calling db.put() multiple times with single entities vs calling it with a whole bunch of entities at once. It's very easy to call db.put() multiple times in a single request, but it's usually trivial to change your code to save all the entities in a single call. I thought that illustrating this difference with some pretty graphs might encourage people to use batch operations.

As always, the size and shape of your data will affect your timings. In my sample I used a small entity with only three string properties (and a key_name). You can grab a copy of the data I gathered in CSV format: App Engine db.Put() Benchmarks

db.put() Performance

Each test was run 10 times, and both the mean and median values plotted on a chart. I did this for varying numbers of entities from 10 to 500 (since db.put() has a limit of 500 entities).

As expected, in all cases, the batch method out-performs calling db.put() many times. Both operations scale very linearly (note the first 3 points are not increments of 100, which is why the line doesn't appear straight). For very small numbers (eg. 10 or less) the results are very similar, but as the number of entities increases, it becomes more important to batch up your requests.

It's worth noting that when you call db.put() with multiple entities, they are not combined into a transaction. If one of the writes fails, an error is raised, but any entities that have already been saved are not rolled back. If you want to update multiple entities are part of a transaction, you must do this the usual way by giving them the same parent and using run_in_transaction.

db.put() Performance Consistency

Since each test was run 10 times, I had enough data to draw a chat showing the consistency of the db.put() performance. The closer a line is to being completely horizontal, the more consistent the write performance is.

Individual db.put() Performance

Batch db.put() Performance

As you can see, the calls are fairly consistent, though the more entities you're saving, the bigger the variance. Although from the graph it looks like the batch calls are less consistent, they graph is drawn at a different scale. The batch calls vary my up to 1.5 seconds, whereas the individual calls vary by up to 5 seconds!

I hope to run some more benchmarks over the coming weeks showing the difference between other APIs such as using Memcache to avoid going to the Datastore on every request.

New Domain - DanTup.com - Please Update Your Links!

2010-01-09T12:24:00Z

After much discussion and debate, I've decided to move all of my sites from the dantup.me.uk domain to dantup.com. The .me.uk domain came free with some hosting and was never really intended to become my "main" domain, but it did. It looked like Google were ranking me lower in Google.com vs Google.co.uk because they believed my content to be more relevant to the UK, which isn't really the case.

Over the last few hours I've been slowly migrating things and setting up redirects. An unfortunate side effect is that you may get duplicate posts in my feed (again). Apologies for that, but once everything is moved over, hopefully it's now here to stay :-)

Although not necessary (old URLs will 301-redirect to new ones), if you have any links to any of my sites/feed, it'd be nice if you could update them to the .com to avoid additional redirects. Here's the new URLs:

Blog: http://blog.dantup.com/
Feed: http://feeds.dantup.com/DanTup
Wave Notifier: http://wavenotifier.dantup.com/

It looks like some things that I've moved from one Google service to another (eg. Feedburner to App Engine) are taking some time to make it through the Google network, so the old domains may behave strangle until that's done. Please let me know if you notice anything strange.

Entity Groups, Contention and Transactions in Google App Engine

2010-01-06T17:16:00Z

When I started learning Google App Engine, I misunderstood a fairly fundamental part of the datastore - Entity Groups. The documentation is not very clear, and over the weeks I've seen many questions asked in videos and forums that suggest I'm not the only one that misunderstood. I thought it was worth a blog post to explain.

Entity Groups are not Tables!

The misunderstanding is that people think of entity groups like table in SQL. This is not the case. If you create two entities that are of the same kind, by default, they do not belong to the same entity group. Unless you specifically choose to put them in the same group, all of your entities will be in a seperate entity group. This means if you have 1,000 entities of the same kind, you have 1,000 entity groups containing one entity each! This is not a bad thing, you shouldn't put things into the same entity groups unless you need to, since updates to an entity lock the whole entity group.

Putting Entities in the Same Entity Group

All entity groups have a root entity. You can't put two entities into the same entity group without one of them being the root entity (parent). To put two entities into the same entity group, you set the parent property, like this:

# Create the company
my_company = Company(name='Danny\'s Company')
my_company.put()

# Create the employee
me = Employee(
    parent=my_company,
    name='Danny Tuppeny'
)
me.put()

Because updates lock an entire entity group, you should only use them if you need to. If you're not going to need to update an employee and its company in the same transaction, you don't need to put them in the same entity group.

Entity Keys Include Their Parent Entity Keys

Something worth noting, is that the key for an entity includes the key of it's parent. That means you can do some clever things for performance purposes, such as "Relation Index Entities" as described by Brett Slatkin in this video. Brett creates child entities for querying and converts their keys to their parents keys using the parent() method to then batch fetch the entities themselves.

Hopefully this clears things up a little. If it still leaves questions (or I've missed anything), please leave a comment below and I'll try to update the post.

Profiling Google App Engine with Appstats

2010-01-03T17:49:00Z

As mentioned in my previous post, I've been keeping an eye on my logs after putting this blog live on App Engine. One thing that wasn't so easy to do with App Engine was monitor the datastore calls being made (like I would with Microsoft's SQL Server Profiler).

Luckily for us, Google added some API Hooks into App Engine and with some help from Nick Johnson and Jens Scheffler I managed to cobble together some scripts that allowed me to identify and solve my problems.

Today, by accident, I came across a library that completely blew my hacked scripts out of the water...

Guido van Rossum's Appstats

Guido van Rossum (a Software Engineer at Google, and the author of the Python programming language!) has released a library called Appstats that hooks and monitors the API calls your app makes and presents them in tables and graphs to aid profiling. Setting it up in your app is a breeze (especially if you already use util.run_wsgi_app to run your app).

I got it up and running on my blog via the dev server (though it works equally well on the production servers) to see how well it works. After a few clicks around the app, I navigated to /stats to see what it came up with.

The Appstats Dashboard

The Appstats dashboard is broken into three sections. The first shows stats for each API method you've called. The second shows stats by URL. The third shows recent requests. Each line in these tables can be expanded for a breakdown of the numbers, as shown below.

Request Statistics

Once you click on a request, you'll get a lot of information about the API calls made.

The first thing shown is a timeline showing not only how long each call took, but also when it started and finished (the will help identify large amounts of time being spent outside of the API calls). In the above example you can see that the most expensive part of this request was a RunQuery call. This call fetches comments for a given post when the posts HTML is not in memcache. All other lookups performed are either memcache Gets or db get_by_key_name calls, which are very fast in comparison.

If you expand one of the API calls you'll see a breakdown of what the call involved (both the request and response). You can specify how much text is included in these tables (such as the request/response) by changing the Appstats options.

Finally there's a summary table showing all API calls for the given request, much the same as on the dashboard, though this one includes timings.

Appstats looks to be a very valuable tool for any App Engine developer, and I look forward to seeing what comes out of the team in the future!

Caching App Engine Datastore Results in Memcache

2010-01-01T20:58:00Z

Since moving my blog to Google App Engine a few days ago, I've been keeping a close eye on the logs. This is my first app engine project that's using the datastore, so I wanted to make sure I hadn't done anything silly and I wasn't getting a large number of timeouts. Although it's probably overkill for my blog, to learn the APIs I use memcache to avoid hitting the datastore lots for the same data.

The caching I've implemented is fairly basic. When it generates a page using Django templates, the entire page contents are put into memcache, using a key that includes the page URL. If somebody else requests the same page, I can serve the entire thing from memcache without any processing overhead. This works well because there is no dynamic (different per user) content on any pages (unless you're an admin, but then memcache is bypassed entirely). When a comment is posted or a post is modified, the entire cache is wiped out. This is done to ensure comment counts on list pages are always up-to-date. Since posts and comments happen very infrequently on this blog (a handful per day at most) this shouldn't be an issue.

So, after putting things live, I noticed that some of my page hits were still quite expensive (in terms of CPU) with the caching. It turned out, that when someone visited a page that wasn't in the cache, I had to execute 3 or 4 datastore queries. One to get the Tag or Archive you were viewing (this step wasn't needed for homepage of direct post pages), one for the post(s), one for the tag list and one for the archive list. Since the tags and archive list don't change much, these could be cached across the application, rather than just per-page, which would result in much faster loading of uncached pages (since they would now be only 1-2 datastore hits).

So, I implemented this extra caching as follows, and all was good.

# TAGS: Check cache first
tags = memcache.get('tags')

# TAGS: Get and cache if not there
if not tags:
    logging.info('Saving tags to cache')
    tags = Tag.all().order('name_lower')
    memcache.set('tags', tags, CACHE_TIME)
else:
    logging.info('Got tags from cache')
	
# ARCHIVE: Check cache first
archive = memcache.get('archive')

# ARCHIVE: Get and cache if not there
if not archive:				
    logging.info('Saving archive to cache')
    archive = Archive.all().order('-date')
    memcache.set('archive', archive, CACHE_TIME)
else:
    logging.info('Got archive from cache')

Or so I thought. Although I tested the code worked with no errors, there's no nice "Datastore Profiler" like the SQL Server Profile I'm used to in Microsoft-land, so I assumed everything was good.

Today, it occurred to me that the CPU values for my uncached page views was still pretty high. I was seeing up to half a second CPU time for what should be a single datastore query. This seemed pretty high even for my inefficient coding!

It turns out, I made a rookie mistake. The problem is with this code right here:

archive = Archive.all().order('-date')
memcache.set('archive', archive, CACHE_TIME)

This code does not cache the entities returned by the query. Rather, it caches the query. That means every time I grabbed it from cache and gave it into Django, the query was being executed as the template being parsed!

A quick change to call fetch() forces the query to be executed and now the entities are cached instead.

archive = Archive.all().order('-date').fetch(1000)
memcache.set('archive', archive, CACHE_TIME)

I'm not so keen on the hard-coded 1000, but until I can find a better way to force the query to execute, it'll do the job. My next job is to write some better hooks to allow better profiling of the datastore so I can better test for this kind of issue in future!

Update

After posting in the App Engine group, I was direct to the article Efficient Model Memcaching on Nick Johnson's blog, which shows a better way, avoiding some of the overhead of caching Model instances directly.

Serving Different Ads/Content Based on a Users Location (Geo-Targeting)

2009-12-31T14:55:00Z

You probably haven't noticed, but this blog serves up different ads depending on where you're visiting from. Or at least, it'll serve Amazon UK ads if you're near the UK, and Amazon US ads otherwise. Serving up US ads to UK visitors (and vice versa) is pretty pointless, and I've always tried to avoid showing any ads unless they're relevant and at least targeted to the right country.

There are a number of ways to determine where your visitors are coming from, so I spent some time yesterday trying to find the most reliable way (and preferably one that didn't involve having a huge IP database sat alongside my site!). After much hacking and testing, I found what I believe to be the best way. Google.

Google has a JavaScript loader API, which allows developers to load JavaScript libraries from Google with various benefits. That's not really what we're interested in though, it has something more exciting:

google.loader.ClientLocation

It appears that you do not need an API key to use the JavaScript loader, you can simply reference it at http://www.google.com/jsapi. If you look at the JavaScript served up (which is incredibly fast), you'll see something like this:

google.loader.ClientLocation = { "latitude":50.123, "longitude":-2.876, "address": { "city":"Liverpool", "region":"Merseyside", "country":"United Kingdom", "country_code":"GB" } };

Not only do you get the country, but you get the county, city and even lat/lon pair. For me, the location given was within 2-3 miles of where I live, so if you wanted, you could really localise your ads!

On this site, the country is just sent to a script that will serve up some ads based on keywords I've tagged against a post. You might wish to be a bit more exciting and show your users places or people nearby. This could be especially useful for mobile applications/sites, though be sure to read any associated terms and conditions before using it!

Google Wave Notifier Now in 15 Languages!

2009-12-31T14:50:00Z

When I started writing Google Wave Notifier last month, it was really only for my own use. I wouldn't use Google Wave if I had no way of knowing when I had messages (and keeping a browser open is not an acceptable solution!) so I hacked something together to login to Wave and parse the JavaScript objects.

I decided others might benefit from the app, so I cleaned it up a little, gave it an icon and published it on the web. I had no idea that it would get close to 10,000 downloads in little over a month! Nor did I expect so many people to send in translations!

So here we are 6 weeks down the line, and the app has been translated into 15 different languages!

English
Czech
Danish
Dutch
French
German
Hungarian
Italian
Polish
Portuguese
Romanian
Russian
Spanish
Swedish
Turkish

So I'd like to say thanks to everyone that's helped; using, translating and finding bugs in the app. What was a small script written for personal use has turned into something quite useful!

For more information or to download, see the Google Wave Notifier website!

Redirecting Requests from appid.appspot.com to a Custom Domain

2009-12-30T12:31:00Z

If you're running your app engine project on a custom domain (like this blog), you're probably not so happy that people can still access your app at http://appid.appspot.com.

When I started working on my blog, I realised this might be an issue, and did some investigating into how I could stop it. I found solution on SackOverflow that seemed to do what I needed, so I set it up and got it working.

Not long after implementing this code, I found a few problems:

SSL is not supported on custom domains
Cron jobs fail when Google invokes them with an appspot.com address and you serve a 301

So with some tweaks, I've managed to get this working as required. The only annoyance is the hard-coded '/admin' check. This is to support cron jobs, task queues etc., which are all protected ("login: admin" in app.yaml). They must work with an appspot.com address, because Google doesn't seem to follow the redirect when invoking them. It's possible you could do an IP address check here, but I'm not sure how consistent cron/task queue IP addresses are.

The code is called like this:

def main():
    dantup.run_app([
        ("/\d*", RootHandler),
        ("/feed", FeedHandler),
        ("/tags/.*", TagHandler),
        ("/archive/.*", ArchiveHandler),
        ("/.*", PostHandler)
    ])

And dantup.py looks like this:

from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app

def run_app(url_mapping):
    application = webapp.WSGIApplication(url_mapping, debug=True)
    application = redirect_from_appspot(application)
    run_wsgi_app(application)

def redirect_from_appspot(wsgi_app):
    """Handle redirect to my domain if called from appspot (and not SSL)"""
    from_server = "dantup-blog.appspot.com"
    to_server = "blog.dantup.me.uk"

    def redirect_if_needed(env, start_response):
	
        # If we're calling on the appspot address, and we're not SSL (SSL only works on appspot)
        if env["HTTP_HOST"].endswith(from_server) and env["HTTPS"] == "off":
		
            # Parse the URL
            import webob, urlparse
            request = webob.Request(env)
            scheme, netloc, path, query, fragment = urlparse.urlsplit(request.url)
            url = urlparse.urlunsplit([scheme, to_server, path, query, fragment])
			
            # Exclude /admin calls, since they're used by Cron, TaskQueues and will fail if they return a redirect
            if not path.startswith('/admin'):
                # Send redirect
                start_response("301 Moved Permanently", [("Location", url)])
                return ["301 Moved Peramanently", "Click Here %s" % url]
	
        # Else, we return normally
        return wsgi_app(env, start_response)

    return redirect_if_needed

Hopefully you may find this useful. If you encounter any problems with it, please let me know!

1-2-3-Testing. Is This Thing On? New Blog!

2009-12-29T17:54:00Z

There's nothing like testing a system after it's gone live!

After lots of frantic hacking over the last few weeks, the blog engine I've been writing for Google App Engine is up and running. It's not complete, but the important things (frontend, feed, posting, etc.) are done so it's at least usable.

Apologies if you just got a flood of old messages in your feed reader. I did investigate keeping the Blogger IDs to avoid this, but it seemed more hassle than it was worth, so I'll forgive you for clicking "Mark All as Read" this one time ;-)

I'll post more info (and code) on the blog as I get time, but now I'm off to sort some food out after a hard day's hacking!

If you notice anything messed up in the transition, please do let me know!

From .NET to Python on App Engine: Building a Blog on Google App Engine

2009-12-24T21:50:00Z

Over the past few weeks I've been playing with Google App Engine. I find the best way to learn a new language/framework/platform is to just jump in and write something in/on it. So that's what I'm doing. I've decided to write my blog in Python for Google App Engine.

As I may have blogged about in the past, I wrote a blog engine in Microsoft ASP.NET MVC not so long ago with the aim of moving away from Blogger. It was around 90% complete when I abandoned it for a variety of reasons (one being Azure pricing).

It's entirely possible the Google App Engine blog engine will also be abandoned, but since the hosting is free it at least stands a good chance of seeing the light of day! It'll also make an interesting comparison to the ASP.NET MVC version.

I started writing code a few nights ago, and currently the blog stands at 159 non-blank lines. I'm actually quite impressed with how little code I've had to write to get up and running. Currently there's no back-end, but the displaying of posts, comments, tags and archives are all working. Here's a quick screenshot to prove it exists! :)

Over the coming weeks I'll blog about how I've built it (including code), the pitfalls and the the experience of moving from .NET and C# to Python and App Engine!

Downloading/Exporting App Engine Logs

2009-12-20T16:53:00Z

Although I've been playing with App Engine for quite a few weeks now, I only found out yesterday how I can download the logs from App Engine for parsing locally. There's no export option in the dashboard, nor any option in the Windows launcher. However, you can do this yourself with appcfg.py.

If, like me, you've only used the launcher up until now, and never done anything with Python outside of GAE, you might find the thought of running python scripts a little scary. Don't worry, it actually turns out to be very easy. I'm hoping when you installed the Launcher you ticked the "Add to PATH" option... :o)

The command you want to run looks something like this:

appcfg.py request_logs appname/ output.txt

You should replace "appname" with the name of your app. This is the folder the contains the app.yaml file for the app you wish to get logs for. That means you need to be in the correct directory (or provide a full path). Output.txt is obviously where the logs should be written.

This command will retrieve everything from the last day. This might not be what you want, so you can increase this with the num_days argument.

appcfg.py --num_days=5 request_logs appname/ output.txt

Note that you can also specify 0 to get all logs:

appcfg.py --num_days=0 request_logs appname/ output.txt

And finally, if you want all logs that you haven't already downloaded, you can use --append, which will scan the last line of the existing file, and download anything since:

appcfg.py --append request_logs appname/ output.txt

However, you'll find this doesn't work on Windows, and you'll end up with a load of duplicate entries. This is in the bugtracker, but I don't know if/when it'll be fixed. For now, I'm having to just download everything (and I'm not sure if this is counting towards my bandwidth quota!).

Another option worth noting, is include_vhost, which will include the hostname used, so you can seperate requests for different versions of your app (or different custom domains). You can use this like so:

appcfg.py --num_days=0 --include_vhost request_logs appname/ "Logs/Logs.txt"

And that's all there is to it. Now you can create nice pretty charts in Microsoft Excel (since Google Spreadsheets sucks!) showing how well (or badly, in my case) your app is doing!

Increased App Engine Quotas, for Free?

2009-12-19T13:45:00Z

While writing my comparison of Azure and App Engine pricing yesterday, I had a thought about how to increase some of your quotas without actually paying anything. I'm not sure whether Google would consider this "gaming the system" and stamp on you (and I certainly have no need to do it with my traffic levels), but I thought I'd post it in the interest of sharing.

If you check the App Engine Quotas page, you'll see the differences between the free quota, and the billing enabled quota. For example, if you don't have billing enabled, you may serve 1,300,000 requests per day. If you have billing enabled, you get 43,000,000 requests per day (other limits, such as CPU/bandwidth still apply).

So, if you've taken a look at how the billing works, you'll know that you get to decide exactly how your budget is split between resources. The minimum budget you can set is $1/day.

Consider what would happen if you enabled billing, but assigned the whole budget to something you know you won't exceed (eg., if you don't use memcache, assign it to memcache). Your app will be given the higher "billing enabled" quotas, but it won't cost you a penny!

As I said earlier - I've no idea whether Google will frown upon this behaviour, so if you do it, it's at your own risk!

Microsoft Windows Azure vs Google App Engine: Pricing

2009-12-18T14:40:00Z

As a .NET developer, I was quite excited to hear about Windows Azure. It sounded like a less painful version of Amazon's EC2, supporting .NET (less painful in terms of server management!). When I saw the pricing, it didn't look too bad either. That was, until I realised that their "compute hour" referred to an hour of your app running, not an hour of actual CPU time. Wow. This changes things. To keep a single web role running, you're looking at $0.12/hour = $2.88/day = $20.16/week = $86.40/month.

Anyone that's bought hosting for a small site/app recently will know that this is not particularly cheap!

So, recently I've been playing around with Google App Engine. It has this massive problem called Python (and an even bigger one called Java ;o)), but it's such a nice framework/engine to work with that I've somehow overlooked this and started coding with it. There's so much to like about it. Everything is so simple to deploy, and it scales "out of the box". Want Cron jobs? No worries, specify them in a file in your app, and when you deploy, App Engine will pick them up and schedule them. Want to queue up work to process later so that your pages return faster? Task queues do just that. What's more, you get a ridiculous free quota every day. It may be Python, but this sounds tempting, no?

So, I thought it'd be interesting to compare the costs of App Engine vs Azure. I understand this isn't really a like-for-like comparison, but both can achieve the same sort of things, and while all programmers will have a preferred language/framework (I'm no exception), many can be swayed by a cool framework or hosting.

First off, let's compare what you get for free. Bear in mind that Azure is free until the end of January, but since this is a CTP and won't end soon, I'm going to exclude it. Google's free quota currently has no time restrictions.

	Windows Azure	Google App Engine
CPU Hours	-	6.5hrs/day
Bandwidth (out)	-	1GB/day
Bandwidth (in)	-	1GB/day
Storage (DB)	-	1GB
Storage Transactions	-	10,368,000/day

Well, for free, I think we have a clear winner. If you can run your website/app within the limits above, then you can do it for free with Google. It's worth mentioning that Google let you have 10 apps per account (though you may not balance a single site/app across app instances - they are specifically for separate projects).

But what if you're bigger than that? What if you get Slashdotted or Dugg a lot? You might find you quickly break out of the free limits. How do prices compare?

	Windows Azure	Google App Engine
CPU	$0.12/hour	$0.10/hour
Note that MS bill "per hour app is running" whereas Google bill "per CPU hour consumed"
Bandwidth (out)	$0.15/GB	$0.12/GB
Bandwidth (in)	$0.10/GB	$0.10/GB
Storage (Files)	$0.15/GB/month	Pricing unavailable (Blobstore)
Storage (DB)	$9.99/month (SQL Server up to 1GB)	$0.15/GB/month ($0.005/GB/day)
Storage Transactions	$0.01/10,000	?
I can't find any prices for Google App Engine storage transactions, so it's possible there is no charge (though a limit of 140,000,000/day applies)

Well, that's interesting. I was going to try and calculate at what point Azure would become cheaper, but looking at those prices, it just isn't going to happen. Now it's worth pointing out that not all of the comparisons are fair. Google bill per actual CPU hour (so if nobody visits your site, it's not costing you), whereas Microsoft are billing for each hour your app is live and able to respond. There's also a significant difference between SQL Server and App Engine Datastore (and depending on what you're doing, one will have advantages over the other).

I really hope Microsoft re-evaluate their pricing for small apps. It's too expensive to play around with small prototypes at those prices, whereas Google's offering will let me get started completely free, until my app is churning a considerable amount of traffic, and even then, it'll work our cheaper for the same processing/transfer.

Sorry Microsoft. I love .NET and Visual Studio, but Google App Engine is just so easy and cheap that it's going to be my "toy of choice" for my hobby coding for the immediate future!

Should I move my Blog to Google App Engine?

2009-12-13T17:10:00Z

Over the years, I've written (and re-written) many blog engines with the intention of hosting my blog on my own code. I'm a programmer, that's what we do. We don't like the thought of other people writing our HTML!

If you've been following my blog over the last twelve months, you'll probably know what my language/framework of choice would be when writing a web app, based on my previous posts...

As a .NET developer by trade, I'm experienced with C#, .NET, ASP.NET MVC etc. and it's always been the logical choice. There's nothing I like to write more than C#/.NET

So why are you hosted on Blogger?

An interesting question :) Unfortuantely the answer is not interesting. The answer is simply: Hosting. Windows hosting is a pain in the ass. I've had many bad Windows hosts that have dodgy control panels, poor performance or a lack of required features. Most of this is addressed with IIS 7 because we can now do most things via XML config files, but it still leaves one issue: Price. Prices seem to vary wildly, and it's such a pain to set up that I'm often put off signing up in case it turns out to be a lemon. This is the reason I have a fully working .NET MVC blog engine ready to roll in my Documents folder that's never made it to the big cloud in the sky.

What about Azure?

A few months ago I was really excited about Azure. It looked ideal for what I wanted, and looking at how EC2 is priced, I suspected it would be cheap if you're only getting a few hundred visitors per day. Then Microsoft announced the prices... To run a no-to-low traffic blog would cost more than getting a virtual server! On top of that, it seemed quite hacky to get MVC working, and it's still only in preview.

Enter, Google App Engine

While playing around in Google Wave, I downloaded the Google App Engine SDK and had a play. While Python isn't exactly my language of choice (I've never written it before!), it struck me as quite a good deal. Five million page views per month... for free! It's hosted on Google's infrastructure and scales well, and there's no way I'd hit those limits anytime soon.

So, now I'm having thoughts... Do I become a hacky Python programmer, putting aside all the .NET experience I have, for free hosting with Microsoft's biggest competitor?

If anyone has any experience with running blogs on Google App Engine, I'd be very interested in hearing from them. Should I run with it, or should I stop being tight-fisted and just pay for some Windows hosting?

To be continued!

Google Wave Notifier now in Spanish, Russian, Swedish

2009-12-05T11:47:00Z

With the help of some users, yesterday I added support for Spanish, Russian and Swedish languages to Google Wave Notifier. All parts of the interface are translated and I'll keep adding languages if people are prepared to help with the translations :)

If you'd like to help translate into your native language, please visit the translation page.

Google Wave Notifier Update

2009-11-28T23:45:00Z

Apologies if this is all I seem to be blogging about lately, but it's the only interesting thing I've been doing!

I've just pushed out a small update to my Google Wave Notifier application (v0.6). You can read about it here.

New features include the ability to play the Windows "New Mail" sound when you get new messages, the ability to change the interval it polls for new messages and some general tidying up and bug fixes!

Google Wave Notifier Update - Automatic Update Checking

2009-11-20T13:46:00Z

I wasn't planning on pushing out an update to my Google Wave Notifier application so soon, but I realised that without automatic update notification, people will just get stuck on the old version and never update.

So, I've pushed out v0.4 which will notify you when an update to the app is available. Other than that it just has some minor tweaks (such as a busy icon while it's checking).

If you find it useful, let your friends on Wave know about it!

Click here for the Google Wave Notifier website.

Google Wave Notifier v0.3 Public Release

2009-11-20T11:09:00Z

Edit: Google Wave Notifier has been released now. You can find the latest posts here.

After some thorough testing, I've decided to publicly release the Google Wave Notifier I've been working on recently. Thanks to everyone that helped test it over the last few days.

Because Blogger doesn't support attachments, I've created a download page on Google Sites.

As always, please post your suggestions or bugs to the bug tracker. I hope you might find this useful!

Click here for the Google Wave Notifier Download Page

Balloons! - The World Just Got Smaller!

2009-11-19T20:57:00Z

Note: As I post this, Balloons! has been reduced to £0.59 for the paid version. Get it while it lasts!

A few weeks ago I was lucky enough to participate in the beta of Balloons! for iPhone. Since it's now available to buy, I thought I'd share my thoughts on the app here.

Balloons! is purely for entertainment, though it doesn't really fall into the category of games. The basic idea is that you pick a balloon and attach a message and/or photo to the label, and launch it into the air. The balloon will travel around the globe (electronically, of course) to be caught by other Balloons! users. When someone catches your balloon, they may attach their own message and photo and relaunch it into the sky for someone else to catch.

I've included a few screenshots below, and although you can see the level of polish Balloons! has, unfortunately you can't see the subtle animations that really bring the app to life. The official website has a video that might give you a better idea of how it looks.

The Balloons! menu screen.

When you launch a balloon, you get to choose how it looks from a selection ranging from normal round balloons to heart and bear-shaped balloons.

Catching a balloon is easy. You can catch balloons from all over the world, though realistically, they have to have been travelling for some time to get from one side of the globe to the other!

The Balloon Tracker allows you to see all balloons you've launched, along with the messages and photos that have been attached to them. This is only available in the paid version.

If you click on a balloon in the Balloon Tracker, you get to see the original message/photo, along with any other messages added to it.

Paid vs Free

Like most apps on the App Store, there's a free version and a paid version of Balloons! With the exception of ads, the only difference is the "Balloon Tracker", which lets you see all balloons you've launched, and the messages that have been added to them. It's well worth paying for this additional feature, or you'll be left wondering whether anyone has caught and added messages to your balloons!

Wanted: Beta Testers for Google Wave Notifier Application

2009-11-19T18:31:00Z

Edit: Google Wave Notifier has been released now. You can find the latest posts here.

Those of you lucky enough to have a Google Wave account have probably noticed there's no official app to notify you of new waves. Google Talk doesn't do it, GMail Notifier doesn't do it, and there's no official Google Wave Notifier.

Yesterday I put together a quick app that will check for new messages in waves and light up a system tray icon (along with showing a notification balloon) when new messages are found. Before I release it into the wild, I'm trying to find a few people to help me test it. If you've got a Google Wave account and would be happy to try it, please send me a wave: danny.tuppeny @ googlewave.com.

Derren Brown - How (not) to Beat a (fake) Casino

2009-10-21T19:19:00Z

I've just finished watching a recorded "Derren Brown - How to Beat a Casino". Yet again, I've been thoroughly disappointed :( It's another episode where he's moved away from the mind tricks for which he became famous, and turned to magic and camera tricks.

For those who didn't watch it, Derren "stole" £5,000 from a viewer and aimed to enter a casino and bet the whole lot on a roulette wheel, with odds of 1 in 37, to win £180,000. By calculating the speed of the wheel and ball, he was going to work out where the ball would land in time to place a bet. He had a hidden camera up his sleeve so we could watch, and the whole thing went out "live". Unfortunately Derren guessed the wrong number, and lost the £5,000. He promised he'd give the £5,000 back to the viewer.

The whole thing was a scam. It was not broadcast live.

Early on in the show, Derren pointed out people have electronic devices to calculate the speeds and guess where the ball will land. Unfortunately they're easily detected, so he would have to do it in his head.

Hold on a minute - he had a camera up his sleeve!

It's quite clear that if you can't get away with a small electronic device, you won't get away with a camera! So what really happened? Well, I don't think it's hard to figure out...

The casino was fake.

There's no way he'd get in with a camera, and the shots were so bad there's no way we would be able to tell whether it's a real casino. The film was not live. Derren pre-recorded the bet, taking multiple shots until the ball landed on the number next to his prediction. They just cut the shots in to make it look live, but it was actually all pre-recorded. He got the wrong number on purpose. If he'd got it right, he'd have to pay out £180,000 to the viewer whose money he used. By getting "the next one" he simply gives the £5,000 back and everyone thinks he's genius because he was "only one number out".

Another promising show, ruined by camera and technology trickery rather than the clever mind tricks we all like to see from Derren Brown.

Aion - How Not to Launch an MMO

2009-10-10T19:12:00Z

As is becoming tradition when a new MMO launch gets lots of hype, BootBlock and I recently started playing Aion. Aion had already been out in Korea for almost a year, so the game should be stable and the launch should be smooth. The gameplay videos looked cool - a lot like Final Fantasy XI, the first MMO I ever played (on a console!).

Although the game hasn't really lived up to the hype, and has had more bugs and missing features than NHibernate, it's been an interested change from playing World of Warcraft. Unfortunately, the launch hasn't been anywhere near as smooth as it should've been.

Note: I've tried to use the word "realm" in place of "server" to refer to a named world that you select to create your character on when playing an MMO to avoid confusion with physical (or virtual) servers.

Realm Queues

The first problem Aion has experience is massive (2 hours plus) realm queues caused by too many people trying to log onto the same realm at the same time. Queues of this length are unacceptable for a game that many will only want to play for a few hours after work/school. This may sound like NCSoft just didn't anticipate the demand for their game, but I disagree. There were over 400,000 pre-orders. NCSoft knew this. They also knew exactly how much stock they'd shipped to retailers for launch week. They'd also already launched in several countries - there's no excuse for getting the capacity so wrong.

Scaling in the Wrong Direction

If there's something to be learned from previous MMOs, it's that the population on a realm is a critical factor in how well people can play the game on that realm. Too few, and people won't be able to find groups. Too many, and the realm will be too crowded and people will have to fight over mobs.

Aion came up with a perfect solution. They call them "Channels", and it allows multiple "instances" of a given zone to split the players. While there are some issues with the implementation, this drastically simplifies the problem of managing population. The number of "Channels" can be changed per zone, on the fly, by NCSoft. This means instead of having lots of realms at launch and potentially having the population fall after the first few weeks, NCSoft could simply load a lot of channels on a small number of realms and reduce them over time as the number of players falls (along with the hardware and player caps associated with the realm).

The problem in Aion, is that it looks like their realms are restricted to a number of players (and an amount of hardware), full stop. It doesn't look like the hardware assigned to a realm can be scaled at all. This means when the queues became very long, they were seemingly unable to increase the hardware assigned to some realms (and therefore the player caps on those realms), and instead had to set up some new realms. When the population falls (which it will, after this launch!) they may end up with many underpopulated realms tying up lots of hardware. This may result in some realms being merged - a complicated and messy process.

Character/Realm Transfers

Ok, let's forgive NCSoft for not being able to anticipate demand for their game. They rolled out some new realms pretty quickly, so at least now everyone can play, right? Wrong! Since most of the people playing have created (and levelled) characters on the original (crowded) realms, they'll have to start all over again if they move realms. This means the new realms are great for new players, but not so great for existing players (specifically those that pre-ordered and had already played for over a week when the new realms arrived). Since there's currently no way to transfer your character from one realm to another, you really are stuck between starting again, or waiting many hours every time you want to play.

NCSoft do say that they're working on a character transfer feature for the coming months, but it's really too late. It's an important part of balancing the population of your MMO and should've been there from the start.

Bots, RMT and Spamming

Most MMOs have been at some point plagued by bots and spammers. RMT stands for "Real Money Trading" which refers to trading real money for in-game items. In most pay-to-play MMOs this is strictly against the rules. RMT websites take advantage of players that are happy to spend some real money in order to have an advantage in the game. They will write bots to login and play the game, killing mobs and looting money and items. They will also spam logged in players with messages asking them to visit their websites to purchase gold.

Not so long ago, this was a huge problem in World of Warcraft. However, if you login today you'll probably see very few bots or spammers. Players shouted about this a lot in the WoW forums and Blizzard (eventually) reacted. There are now very easy methods in game for reporting players as bots and spammers. When someone is reported a number of times, they will automatically be silenced and flagged for a GM to investigate. There are also restrictions stopping trial (free) accounts from speaking in public channels or sending whispers. This has had a significant impact on the RMT business and it's rare to see bots or receive spam these days.

Aion launched without any protection against this. The forums are now full of complaints about bots and spammers. NCSoft have patched in a method for blocking users that send you spam messages, but whether enough of these blocks will silence an account is unknown. If not, then it's useless. No spammer sends multiple messages to the same person from the same character. Never mind, NCSoft is on the case with some bans...

Mass Account Banning

In order to quickly reduce the number of bots and spammers in game, NCSoft have started mass banned accounts and IP ranges. It looks like they're hitting a lot of innocent people with the ban hammer as they go. Whilst I don't believe everyone crying in the forum is innocent, there are far too many of them for me to believe they're all botters/gold buyers.

The mass banning has only increased the amount of complains in the forums. The bans happened on a Friday evening, right before the weekend when most people will want to play (and presumably when they have reduced staff to resolve these incorrect bannings).

Aion had a lot of potential to take a huge chunk of the MMO market from Blizzard. Unfortunately a lot of the issues during launch will have had a detrimental impact on the game. Just like Age of Conan, a lot of players will leave after the included 30 days, never to return. Hopefully other companies working on future MMOs have been watching this launch (and others) carefully to avoid making the same mistakes.

Importing iPhone Dev Keys on a new Mac

2009-10-09T12:45:00Z

When I set my Mac Mini for iPhone development, I was told to backup a key file (.p12) because if I lost it and needed to reinstall, I wouldn't be able to deploy to my iPhone (yikes!). I backed it up as suggested, and made sure I had copies of it all over my Mac, PC and the interwebs.

With a clean install of Snow Leopard, it was time to import this keyfile and make sure everything still worked. As instructed in the docs, I double-clicked on the .p12 file and was asked for the password. I entered it and got the following message:

An error has occurred. Unable to import an item. The contents of an item cannot be retrieved.

Oh, shit.

I tried multiple times to import, and even imported other keys I'd exported before it. I'm 100% sure I was putting the password in correctly. No joy, just lots of errors. Although the key appeared in the list (after apparently failing), Xcode refused to recognise it and I'm unable to deploy to my iPhone. This was looking pretty bad...

Google to the rescue!

After a bit of searching, I came across a post from someone having the same issue, and a workaround. Importing via the terminal apparently works. I gave this a shot, and all is good. I can now deploy to my iPhone again. Thanks Dave K!

Just in case you're reading this with the same problem and the link above is now broken, here's a copy of the solution:

Work-around provided by Dave K.

You can use the 'security' command line tool:
security import priv_key.p12 -k ~/Library/Keychains/login.keychain
security import pub_key.pem -k ~/Library/Keychains/login.keychain

I'm Blogging... From a Mac!

2009-08-07T19:42:00Z

I'm shaking my head with shame. I never thought this day would come. I'm writing a blog post... from a Mac!

Those that know me will probably know I'm a massive Microsoft fanboy. I love .NET, XNA, WPF, Visual Studio, Vista and all the other things that come from Redmond. I don't like Apple.

Last year I bought an iPhone. I just couldn't help myself. I've had many Windows Mobile devices from iPaqs to XDAs and they suck. Big time. The iPhone came along and it just wiped the floor with anything out there, so I had to get one. I love it!

Months on, and I've released an Xbox game called Jungle Blocks using XNA. The whole process was pretty awesome. I got to write it in Visual Studio using C# and for very little money my game was out there being played by other people. They were paying money to play my game. That was a cool feeling!

Then my mind started to wonder... Maybe I should dabble in iPhone development? I could release iJungle Blocks! (Ok, maybe I won't call it that). In a lot of ways the App Store seems to work very similar to XNA and the Indie Games and since I'd already filled in all the forms and got an ITIN I figured I didn't have much to lose trying.

Well, except my pride. It turns out that Mr Jobs doesn't want you making money from his iPhone if you don't have his computer. You can only develop for the iPhone on a Mac (legally). This was a bummer. Macs stink. They won't do .NET and they certainly don't do Visual Studio. It was time to do some research!

I started browsing the web and speaking to a few people to find out what coding on a Mac would be. This is what I found out:

The cheapest new Intel Mac is £500
Xcode (the Mac IDE) is crap. It does not have all the features of Visual Studio 2008 :(
Objective-C is verbose and long-winded. There is no Garbage Collection!
All my friends will laugh and point if I buy a Mac

Things were looking grim, but I decided to go nuts. This week, I bought a Mac Mini. After a few hours I discovered a few more things:

Mac keyboards suck. Thankfully my Microsoft Keyboard/mouse works :)
I can't type " @ or #. They're just mapped wrong.
The Mouse on a Mac starts slow and then accelerates veryquickly. It's a pain in the ass.
Alt+Tab is Windows+Tab. Ctrl+C is Windows+C. etc. This is madness!
The Home and End keys don't work as expected. Neither do Ctrl+Left or Ctrl+Right.

I got Xcode installed and started playing around to see how bad this thing was. After a few nights of wrestling with Xcode, Objective-C, Cocoa-Touch and Mac OSX here's what I've found out:

Xcode is not crap. There's still time, but Xcode is not frustrating the hell out of me like I thought it would. It's not all that bad. Nowhere near as bad as some of the crap I've had to use (I'm looking at you, FlexBuilder). I might just get on with it!
Microsoft have Mac drivers for my Microsoft Keyboard and mouse. They not only fix the crazy mouse acceleration, it includes a proper mapping to fix " @ and #!
Even on the highest setting, the mouse still moves too slow on a Mac. The max setting on a Mac doesn't come close to the Max setting on a PC.
Cocoa-Touch is actually really nice :)
There is no nice fix for Alt+Tab. I can swap Windows/Alt keys, but then Copy/Paste becomes Alt+C/Alt+V instead of Windows+C/Windows+V.
These is seemingly no fix for Home/End/Ctrl+Left/Ctrl+Right

So far, things are not as bad as I imagined. And the Mac does have some nice stuff. But will it tempt me away from the lovely world of Visual Studio and C#? Unlikely. It's not as bad as I expected it to be, but it's just not the same experience as developing on a PC.

ASP.NET MVC HandleError Attribute, Custom Error Pages and Logging Exceptions

2009-04-17T18:24:00Z

I'm sure I don't need to tell you how bad serving a Yellow Screen of Death to your users is. Nonetheless, it seems to be pretty common practice across the web. One of the first things I do when setting up a new ASP.NET project is set up custom error pages and ensure all exceptions are logged (who wants to find out about their errors from their clients?). Since things work slightly differently in ASP.NET MVC I thought I'd dig in and find the best way to do the same thing.

The HandleError Attribute

The HandleError attribute (which appears on the default controllers in an MVC project) tells the framework that if an unhandled exception occurs in your controller that rather than showing the default Yellow Screen of Death it should instead serve up a view called Error. The controller-specific View folder will be checked first (eg. Views/Home/Error.aspx) and if it's not found, the Shared folder (Views/Home/Error.aspx) will be used.

But How Do I Log Exceptions?

You might've spotted the problem with HandleError. It just outputs a view, and doesn't let you run any code. This might be fine if you don't want users to see errors but don't really care for fixing them. Hopefully you think this isn't acceptable and you want to investigate all exceptions!

The OnException Method

The System.Web.Mvc.Controller class contains a method called OnException which is called whenever an exception occuts within an action. This does not rely on the HandleError attribute being set. If you're being a good coder and have your own base Controller class you can override this method in one place to handle/log all errors for your site. You might choose to send emails and/or detect duplicate exceptions and discard them. For now, I'm just going to write them all to a text file in my App_Data folder.

protected override void OnException(ExceptionContext filterContext)
{
	WriteLog(Settings.LogErrorFile, filterContext.Exception.ToString());
}

/// <summary>
/// Logs a message to the given log file
/// </summary>
/// <param name="logFile">The filename to log to</param>
/// <param name="text">The message to log</param>
static void WriteLog(string logFile, string text)
{
	//TODO: Format nicer
	StringBuilder message = new StringBuilder();
	message.AppendLine(DateTime.Now.ToString());
	message.AppendLine(text);
	message.AppendLine("=========================================");

	System.IO.File.AppendAllText(logFile, message.ToString());
}

This works great, but it still shows our user an unhandled exception message, even if we use the HandleError attribute. This makes the HandleError attribute look rather useless, so I've removed it. We can easily show the friendly error ourselves with the following code:

filterContext.ExceptionHandled = true;
this.View("Error").ExecuteResult(this.ControllerContext);

It's important to set ExceptionHandled to true, otherwise you'll still see the default unhandled exception message. The OnException method returns void so we must Execute the view and pass in the ControllerContext ourselves.

How Do I see my own Errors During Development?

It's a little inconvenient to open log files or keep commenting out your error handling code while developing to see exceptions and stack traces. You might remember ASP.NET has a nice web.config setting that configures custom errors. This property is exposed via MVC, so we can set up our config to show friendly errors to remote users only:

<customErrors mode="RemoteOnly" />

Then all we need to do in our OnException method is check this value and serve up the custom error view only if it returns true.

protected override void OnException(ExceptionContext filterContext)
{
	WriteLog(Settings.LogErrorFile, filterContext.Exception.ToString());

	// Output a nice error page
	if (filterContext.HttpContext.IsCustomErrorEnabled)
	{
		filterContext.ExceptionHandled = true;
		this.View("Error").ExecuteResult(this.ControllerContext);
	}
}

It's worth noting that IsCustomErrorEnabled will resolve the RemoteOnly option for you, you don't need to check where the user is coming from. Now out site serves up friendly errors to users and logs all exceptions without us losing the ability to see stack traces during development.

Using OpenID in your ASP.NET MVC Application/Blog

2009-04-14T19:23:00Z

Over the last few days I've been rewriting this blog in ASP.NET MVC. As it gets closer to a state where I can upload it, I found myself needing to implement security for the administration section (adding, editing posts, etc.). I don't want yet another username/password to remember, and I don't want to IP-restrict it because that's not very flexible (and I don't know how static my IP is!), so what are my options?

OpenID

OpenID is nothing new, it's been around since late 2005. I've been aware of what it did and how it worked, but never really played with it. I did, however, get the impression it might solve my problem. Especially having seen that you can use your Google account as an OpenID!

What's OpenID? Why is it Cool?

OpenID is a standard for authentication, allowing you to use the same identitiy/login for multiple services. It is not the same as using the same username/password at multiple websites (that's a very bad idea). Let's see an example.

I want to be able to login to my blog to edit posts. I don't want another username/password. As Google now works as an identity provider, my blog can redirect me to Google and let them authenticate me. Google will then return me to my blog saying "Yes, this is definitely Danny Tuppeny". This means I don't need any user tables, login forms, or anything else on my blog!

This might sounds complicated, but as with most things, there's a nice .NET library called dotnetopenid to hide the complexity. Let's see some code!

On the first request, dotnetopenid will return a null response. After logging in at the identity providers website, the user will be redirected back (to the same page by default, but this can be changed) with a token on the query string. This will cause dotnetopenid to return a response. The basic code looks like this:

var openId = new OpenIdRelyingParty();

if (openId.Response == null)
{
	// No response means this is the first page load
}
else
{
	// This means we're been redirected back after authentication
	if (openId.Response.Status == AuthenticationStatus.Authenticated)
		// User was logged in (as someone!)
}

On the first page load, we would usually ask the user for their OpenID Identifier/URL, however since in my case it's always going to be Google, I'm going to hard-code this as a single value.

dotnetopenid supports adding claim requests so that you can request (or even demand) specific pieces of information. In my case I only care about authenticating me, I don't need to request my name or email address. As such, I'm just going to fire a simple request off without any claim requests.

openId.CreateRequest("https://www.google.com/accounts/o8/id").RedirectToProvider();

In the else block we need to check the response. We want to make sure that the status is Authenticated and the ClaimedIdentifier matches the known identifier for my own login.

// We got a response - check it's valid
if (openId.Response.Status == AuthenticationStatus.Authenticated
	&& openId.Response.ClaimedIdentifier.ToString() == "http://google.com/blah/blah/blah")
{
	Session["Admin"] = true;
	return Redirect("/posts/edit");
}
else
	return Content("Go away, you're not me.");

The ClaimedIdentifier will be unique to each Google account. You can run the code once and examine the returned value to find out your own, and then you can check against it.

If we put all this together into a controller action, it'll look something like this:

public ActionResult Login()
{
	var openId = new OpenIdRelyingParty();

	// If we have no response, start
	if (openId.Response == null)
	{
		// Create a request and redirect the user
		openId.CreateRequest(Settings.AdminOpenIDIdentifier).RedirectToProvider();

		return null;
	}
	else
	{
		// We got a response - check it's valid and that it's me
		if (openId.Response.Status == AuthenticationStatus.Authenticated
			&& openId.Response.ClaimedIdentifier.ToString() == Settings.AdminClaimedIdentifier)
		{
			Session["Admin"] = true;
			return Redirect("/posts/edit");
		}
		else
			return Content("Go away, you're not me.");
	}
}

That's really all there is to it. Now when I hit the Login action I'll be redirected to Google's login page. After logging in, I end up back at /posts/edit on my blog with the correct session variable set. Of course, you could instead call the built-in ASP.NET authentication methods, or look up a user from your database based on their ClaimedIdentifier. There are a lot of ways you can extend this, and I'll cover using OpenID for blog comments in a future article!

IE8: Hanging with "Connecting..." when opening tabs, unable to hide Favourites bar and other bugs

2009-04-14T18:09:00Z

I'm not the only person having these problems, so I thought I'd post the solution here for all...

After hearing that IE8 will be offered via Windows Update next week, I decided to install it on my home PC running Windows Vista. I've been using it since it RTM'd at work with some major stability issues, but I put them down to my machine rather than IE. Oh, how I was wrong!

After the usual install and reboot cycle, I opened IE8. I turned off the usual trash (Accelerators, Web Slices, Compatibility View for Intranet sites, etc.) and went to hide the nasty favourites bar. Only, I couldn't. Right-clicking on the favourites bar didn't give a context menu. So I tried View » Toolbars » Favourites - the option was disabled!

I proceeded to Google, doing the usual middle-click on results to open them in new tabs. Every new tab I opened just sat with "Connecting..." in the tab title. The content was blank.

This isn't looking good...

I swiftly disabled all the non-MS addons (and Fiddler, thinking that could potentially break connections). No change.

I fired up Event Viewer and found an "Internet Explorer" event log. This would be an interesting find if the entire log wasn't blank. Great!

I opened up My Computer and navigated to my system drive. WTF - it opened in a new window. I checked folder options - it's still set to "open each folder in the same window". I don't like that :(

Rolling back to IE7 is now a serious option. This isn't what I've come to expect from Microsoft!

After a little more Googling (using Chrome, ofcourse) I found I wasn't the only one having these issues. Fixes varied from broken addons to GoogleUpdater. Nothing seemed to apply to my problem.

Then I found a magic post. Someoe had run IE as Administrator and discovered his problems disappeared. Even better, when he ran as a normal user afterwards, things still worked! Worth a shot, eh?

Well, it worked. Running as Administrator worked fine. Running again as a normal user and everything continued to work. Specifically, I could now:

Hide the IE8 favourites toolbar
Open new tabs without the "Connecting..." message/hanging
Open Windows explorer folders in the same window
Access toolbar context-menus

An interesting bug! I can't explain how it could happen, or why any errors/failures aren't written to the event log. All I can say is I'm glad it's fixed and I don't need to roll back to IE7!

Eager-Fetching of Relationships with LINQ to SQL

2009-04-13T11:34:00Z

By default, LINQ to SQL lazy-loads its relationships. This means it won't go fetching entire trees of objects when you're only using the top ones. However, this might not always be desirable. Imagine if every time you output a post for your blog, you include tagsa. You might write something like this:

foreach(var post in db.Posts)
{
	Response.Write("<h1>{0}</h1>", post.Title);
	foreach(var Tag in post.Tags)
	{
		Response.Write("<a href=\"{0}\">{1}</a>",
			Html.Encode(tag.FullUrl),
			Html.Encode(tag.Name)
		);
	}
}

This code will work fine, but if you were to examine the SQL being generated, you'd see n+1 SELECT queries. One for fetching the posts, and one for fetching the tags for each posts - individually.

What can we do about it?

LINQ to SQL allows us to specify a set of DataLoadOptions that dictate this behaviour. One of the methods of the DataLoadOptions is LoadWith<T> which allows us to say "whenever you load x, always include y". E.g.

// Create DataLoadOptions
DataLoadOptions dlo = new DataLoadOptions();

// Always fetch tags when we get posts
dlo.LoadWith<Post>(p => p.Tags);

// Set these options on the DataContext
db.LoadOptions = dlo;

If we re-run the original query, we'll now find a JOIN to the Tags table and just a single query to fetch all the data we require.

Simple! However, bear in mind that LINQ to SQL might not always generate joins. I've got a few cases where the LoadWith seems to be ignored. As soon as I figure out why, I'll be sure to update this post!

Reducing Duplicate Content with ASP.NET MVC

2009-04-09T18:54:00Z

As you're all no doubt aware, ASP.NET MVC recently went RTM. This brings the MVC-style of coding, made very popular by Ruby-on-Rails to the ASP.NET world. I've been eager to start using MVC for months, but I've been holding off until I knew the API was locked down so I don't have to change anything.

Unfortunately, like WebForms, MVC has some "issues" with regards to duplicate content, making it not all that SEO-friendly.

What do you mean, Duplicate Content?

Duplicate content is just that - the same content repeated on multiple pages/sites. This might not sound like a big deal, but it's not something search engines like. They don't want the search results to show the same content multiple times across different websites so they often penalise or hide duplicate content. Additionally, if you have two pages with the same content, your inbound links might become split between the two - reducing the pagerank passed to either.

What's this got to do with ASP.NET MVC?

Unfortunately ASP.NET MVC makes it easy to have the same content indexed multiple times. I've listed the main problems below.

Case-Sensitivity. In ASP.NET (or rather IIS and Windows), URLs are not case sensitive. That means you can write Default.asp, default.asp or even DeFalT.aSp and still get the same page. While you'll probably stick to the same case within your website, it wouldn't be hard for someone to create links to your site with different casing (e.g. they might have CAPS LOCK turned on).

Default Documents. Most websites have a default document set up to serve when a filename is not provided in the request. E.g. http://mydomain.com/ might actually serve up http://mydomain.com/default.asp, but it won't tell the browser that's what it did. It will serve it up as if the two are different URLs.

Trailing Slashes. While the above problems are general ASP.NET/IIS issues, trailing slashes are something that only really become a problem with MVC or other URL rewriting/routing. In ASP.NET if you requested http://mydomain.com/files and you had a folder named files, IIS would issue a redirect to mydomain.com/files/. However, in ASP.NET MVC the URL routing will treat trailing slashes the same as requests without. So http://mydomain.com/controller/action is exactly the same as http://mydomain.com/controller/action/ and therefore results in duplicate content.

Query Strings. Query strings can be a big problem for duplicate content. Imagine if you can add ?sort=field to the end of your page to have a table re-ordered. To a search engine this looks like another page, but the content is mostly the same. Fortunately, ASP.NET MVC doesn't really use query strings thanks to the excellent URL routing.

So, what can we do?

Lowercase URLs. We can force all requests to our application to be lowercase by catching them in BeginRequest in Global.asax and redirecting to the lowercase version if they contain any uppercase characters.

protected void Application_BeginRequest(Object sender, EventArgs e)
{
	// Get the requested URL so we can do some validation on it.
	// We exclude the query string, and add that later, so it's not included
	// in the validation
	string url = (Request.Url.Scheme + "://" + HttpContext.Current.Request.Url.Authority + HttpContext.Current.Request.Url.AbsolutePath);

	// If we've got uppercase characters, fix
	if (Regex.IsMatch(url, @"[A-Z]"))
		PermanentRedirect(url.ToLower() + HttpContext.Current.Request.Url.Query);
}

/// <summary>
/// Redirects with a 301 header to pass along any incoming
/// PageRank/link value.
/// </summary>
/// <param name="url">The URL to redirect to</param>
private void PermanentRedirect(string url)
{
	Response.Clear();
	Response.Status = "301 Moved Permanently";
	Response.AddHeader("Location", url);
	Response.End();
}

Now if anyone requests a URL with uppercase characters, they'll be redirected with a 301 redirect. This works great, but we have a problem. All URLs generated internally by MVC will continue to use Action and Controller names in Pascal case (assuming that's how your classes are named). This means every link within our site will cause two requests (the first being a redirect). To fix this, we can override the default behaviour for creating URLs. We'll create a new extension method for the RouteCollection class called MapRouteLowercase which instead of creating a Route will create an instance of a new class, called LowercaseRoute. This class will override the GetVirtualPath method to lowercase the URL before passing it back. I can't take credit for this code, I pretty much just copied it from Graham O'Neale's blog.

public class LowercaseRoute : System.Web.Routing.Route
{
	public LowercaseRoute(string url, IRouteHandler routeHandler)
		: base(url, routeHandler) { }
	public LowercaseRoute(string url, RouteValueDictionary defaults, IRouteHandler routeHandler)
		: base(url, defaults, routeHandler) { }
	public LowercaseRoute(string url, RouteValueDictionary defaults, RouteValueDictionary constraints, IRouteHandler routeHandler)
		: base(url, defaults, constraints, routeHandler) { }
	public LowercaseRoute(string url, RouteValueDictionary defaults, RouteValueDictionary constraints, RouteValueDictionary dataTokens, IRouteHandler routeHandler)
		: base(url, defaults, constraints, dataTokens, routeHandler) { }

	public override VirtualPathData GetVirtualPath(RequestContext requestContext, RouteValueDictionary values)
	{
		VirtualPathData path = base.GetVirtualPath(requestContext, values);

		if (path != null)
			path.VirtualPath = path.VirtualPath.ToLowerInvariant();

		return path;
	}
}

public static class RouteCollectionExtensions
{
	public static void MapRouteLowercase(this RouteCollection routes, string name, string url, object defaults)
	{
		routes.MapRouteLowercase(name, url, defaults, null);
	}

	public static void MapRouteLowercase(this RouteCollection routes, string name, string url, object defaults, object constraints)
	{
		if (routes == null)
			throw new ArgumentNullException("routes");

		if (url == null)
			throw new ArgumentNullException("url");

		var route = new LowercaseRoute(url, new MvcRouteHandler())
		{
			Defaults = new RouteValueDictionary(defaults),
			Constraints = new RouteValueDictionary(constraints)
		};

		if (String.IsNullOrEmpty(name))
			routes.Add(route);
		else
		routes.Add(name, route);
	}
}

You can put these classes anywhere. Because MapRouteLowercase is an extension method, you can just call it on the RouteCollection class in place of the existing MapRoute call in your Global.asax.

// Home stuff
routes.MapRouteLowercase(
	"Default",
	"{page}",
	new { controller = "Home", action = "Index", page = 1 },
	new { page = @"\d+" }
);

Default Documents. While this issue doesn't affect MVC in the same way, there's a very similar problem. In ASP.NET MVC the default routing is {controller}/{action} but it sets a default action of Index. That means on a newly-created project, both /Home/Index and /Home will serve up the same content.

To work around this, and provide some nicer URLs, I changed the routing a little so that my default actions where mapped to the root and a seperate route dealt with the homepage (which accepts pages, to allow browsing to older posts).

public static void RegisterRoutes(RouteCollection routes)
{
	routes.IgnoreRoute("{resource}.axd/{*pathInfo}");

	// Posts
	routes.MapRouteLowercase(
		"Posts",
		"posts/{url}",
		new { controller = "Post", action = "Display" }
	);

	// Tags
	routes.MapRouteLowercase(
		"Tags",
		"tags/{url}/{page}",
		new { controller = "Tag", action = "Display", page = 1 },
		new { page = @"\d+" }
	);

	// Home stuff
	routes.MapRouteLowercase(
		"Default",
		"{page}",
		new { controller = "Home", action = "Index", page = 1 },
		new { page = @"\d+" }
	);

	// Home stuff
	routes.MapRouteLowercase(
		"Home",
		"{action}",
		new { controller = "Home", action = "" }
	);

	// Catch-all for any unmatched URL
	routes.MapRouteLowercase(
		"Error Catch-All",
		"{*path}",
		new { controller = "Home", action = "NotFound" } // NotFound doesn't exist, so HandleUnknownAction will be fired
	);
}

Trailing Slashes. To avoid trailing slashes and a few other minor issues (such as people adding /1 to a URL to get page 1, which is served up without the /1) I added some additional rules to my Global.asax as below.

protected void Application_BeginRequest(Object sender, EventArgs e)
{
	// Get the requested URL so we can do some validation on it.
	// We exclude the query string, and add that later, so it's not included
	// in the validation
	string url = (Request.Url.Scheme + "://" + HttpContext.Current.Request.Url.Authority + HttpContext.Current.Request.Url.AbsolutePath);

	// If we're not a request for the root, and end with a slash, strip it off
	if (HttpContext.Current.Request.Url.AbsolutePath != "/" && HttpContext.Current.Request.Url.AbsolutePath.EndsWith("/"))
		PermanentRedirect(url.Substring(0, url.Length - 1) + HttpContext.Current.Request.Url.Query);

	// If we end with /1 we're a page 1, and don't need (shouldn't have) the page number
	if (HttpContext.Current.Request.Url.AbsolutePath.EndsWith("/1"))
		PermanentRedirect(url.Substring(0, url.Length - 2) + HttpContext.Current.Request.Url.Query);

	// If we have double-slashes, strip them out
	else if (HttpContext.Current.Request.Url.AbsolutePath.Contains("//"))
		PermanentRedirect(url.Replace("//", "/") + HttpContext.Current.Request.Url.Query);

	// If we've got uppercase characters, fix
	else if (Regex.IsMatch(url, @"[A-Z]"))
		PermanentRedirect(url.ToLower() + HttpContext.Current.Request.Url.Query);
}

This seems to stop many of the issues I came up with, however the double-slash seems to be passed through (in AbsolutePath) as a single slash here (Vista/IIS7) so doesn't work. I've left it in just in case this behaves differently on other web servers.

Is there anything else I should do?

As of February, Google, Yahoo, ASK and Microsoft Live Search support a new Canonical meta-tag. This allows you to specify on a page that this page is duplicate content and any incoming links should instead be attributed to another page. If your site has query strings or other potential for multiple requests to serve up the same content I would recommend inserting this tag to make sure the search engines choose your prefered page.

Generic Collections and Thread Safety

2009-04-06T18:39:00Z

Recently I was involved in a discussion about generic collections and thread-safety. I was under the impression the framework had thread-safe versions of the generic collections, but I was unable to locate them and resorted to Google. I was wrong. There aren't any.

I thought the subject would make an interesting post, since there seems to be a lot of bad information on the subject.

Why do we need thread-safe collections?

Imagine you're running multiple threads that take an item from a queue and perform some processing. Your code might look like this:

// Get the next request from the queue
var request = infoPackRequests.Dequeue();

// Perform some processing
SendInfoPack(request);

At first glance, you might think this code is pretty thread-safe. The first thread will get the first item, and the next thread will get the following one, right? Wrong. Sometimes you might be lucky and it will work out that way. Other times you might find your application does strange things, such as processing the same item twice.

If you take a peek at the Dequeue method, you'll find some code that looks like this:

Object removed = _array[_head];
_array[_head] = null;
_head = (_head + 1) % _array.Length;
_size--;
_version++;
return removed;

This seems fairly straightforward. It grabs the item, nulls it out from the array and decreases the size. Now consider what would happen if both threads executed the first line at the same time. They'll both return the same item. And that's not the only thing that could go wrong.

So, how do we fix it?

Fortunately, this problem has been considered, and Microsoft provided the SyncRoot property on ICollection to help us synchronise access. Usage is simple - we take a lock on SyncRoot before doing anything that's not thread-safe:

// Get the next request from the queue
var request;
lock(infoPackRequests.SyncRoot)
{
	request = infoPackRequests.Dequeue();
}

// Perform some processing
SendInfoPack(request);

Now we can be sure our Dequeue methods won't be called at the same time. Note that we don't need to (and shouldn't) include the SendInfoPack call inside the lock, because it does not touch the collection. Including it in the lock would have a negative impact on performance, because other threads would block for the entire duration of SendInfoPack on any other thread.

Problem solved! Why the long face?

Well, this all works great when you're using a Queue or ICollection. Most of us however are now using Queue<T> or ICollection<T>. These classes don't expose a SyncRoot property. The BCL Team Blog explains why this decision was taken. Developers were using "thread-safe" wrappers as if they didn't have to care about the threading issues, eg:

if (myQueue.Count > 0)
{
	DoSomething(myQueue.Dequeue());
}

The problem here is that the locks provided by the wrapper within each method/property are released between the calls to Count and Dequeue, so there's no guarantee an item still exists when the Dequeue method is called.

So what's the preferred method?

Like most things, that depends on exactly what you're doing. The reason for taking the SyncRoot property away was to encourage developers to think about the synchronisation between threads rather than just blindly using synchronised wrappers and assuming it'll all be fine. You'll want to hold a lock for the duration of any operation that could cause issues if another thread was doing something at the same time. Using the above example, you'd want to ensure nothing was removed from the queue after you checked the Count property, so you would probably have something like this:

var item = null;

lock (lockObject)
{
	if (myQueue.Count > 0)
	{
		item = myQueue.Dequeue();
	}
}

if (item != null)
	DoSomething(item);

This method makes it easier to synchronise multiple objects (imagine you were moving items to a "processed items" collection after processing) since you have a central lock object, rather than two independent SyncRoots.

Vista Disk Thrashing - Solved?

2007-06-21T08:05:00Z

I previously blogged about my crazy Vista disk activity and had a few suggestions in the comments. Judging by the number of Google referrals I'm getting for "vista disk activity" and like, I'm guessing these are very common issues.

Since posting, I've actually had very little disk thrashing, and I'm wondering if it's related to the Compatibility and Performance packs I installed around the same time.

I came across this blog saying that turning off paging and prefetch solved his problem. Additionally, the comments in my blog suggest it could be the indexing service. It's worth giving this things a try if you're still having the same problems (after installing the above packs) and see how you get on. Let us know in a comment how you get on.

Vista - Lots of Hard Disk Activity

2007-06-18T11:40:00Z

Every time I boot my Vista machine up, I hear my hard disks grinding for quite a while after the machine finishes booting and services finish starting up. I also find that if I leave my machine for a while, I'll come back to more hard disk thrashing. With noisy RAID Raptors, and a case made of mesh, it's not a particularly enjoyable sound!

Today I'd had enough, and decided to check on the web if anyone else is having this sort of behaviour. It seems many are. Lots of people are pointing at System Restore, so I fired up regedit to find it runs every 24 hours. This explains every day when I boot it up, the hard disk activity. I don't really want to turn this off, but maybe I could increase the frequency. After all, I'm not often doing anything that should upset the system!

This doesn't address the "idle" noise. I guarantee if I leave my machine for 5 minutes, Vista will begin doing something behind my back, and if I move the mouse, it'll stop within a few seconds. This isn't an accident - Vista is doing something when it thinks I'm not using the machine - but what?

Has anyone else seen this behaviour? Any ideas on what it could be, or how I could track it down? I'm more curious than annoyed - I just want to know what it is!!

VS 2008: Testing Tools to be included in Professional edition of Visual Studio 2008

2007-06-12T18:35:00Z

In case you hadn't already heard, the testing tools that were previously only available in the Team System edition of Visual Studio are included in Visual Studio 2008! This is great news for those using Nunit because they couldn't afford the Team System editions. Unlike Nunit, these tools full integrate with the IDE!

So, let's have a look. I opened up one of my existing projects, and in the context menu there's a new option for "Create Unit Tests".

If you click this option, you'll be presented with a list of your classes and their methods. You can tick those you wish to generate unit tests for.

The settings dialog allows you to set how the class and method names are generated.

Tests will automatically be generated in a new project, so when you click Ok, you'll be prompted for a name. I'm not entirely sure why this wasn't just included on the previous dialog, but it's not a problem.

Now your tests will be generated. It may take a few minutes - it took a while here, and I only had one test to generate! When finished, you'll see a new project in your solution explorer, and some config files in a new "Solution Items" folder.

In the generated tests is a property of type TestContext. I've never used anything similar in Nunit, but it looks like this allows data-driven tests, where you can have a single test run for every row in a set of data you provide. I can see this being very powerful for testing lots of different conditions without duplicating tests!

To run your tests, you can use the new Tests menu at the top of the window. You'll get build-like progress, and failing tests show similar to compile errors. Double-clicking a failed test will give details, as shown below.

While I love Nunit, I've been annoyed there's been no real (free) integration of tests into Visual Studio, so I'm made up to finally have these testing tools at my fingertips. Move over Nunit, Microsoft are taking over!

The more I use Visual Studio 2008, the more I wish I was using it at work! It's packed with new features that make coding much quicker and less error-prone!

Don't forget, there are also free versions of Visual Studio 2008 (sadly without testing tools!) for the home-coder on a budget!

The WPF Designer (Cider) part 2...

2007-06-08T21:21:00Z

After Mark Wilson-Thomas, the Program Manager on the WPF Designer team responded to my post about my WPF issues, I thought I owed it a second chance having read his comments.

Locating Controls

My first issue was with the control drop-down being removed. Mark pointed out a few alternatives like the Document Outline window and the XAML Path control. The Document Outline seems to work pretty well, though I'm still not covinced I'll get along with it. Usually when I'm heading for the old drop-down at the top of the property grid, it's because I need to select something I can't easily select in the designer (menu items, controls that are dynamically hidden/shown etc.), and I usually have the name of them in my head. That makes an alphabetical list much quicker than a tree. I know what the item is likely to be called, but not necessarily how it's embedded in a huge tree of SplitPanels. I've also noticed a few minor issues as you can see in my screenshot.

The tree is a bit flat, and not very tree-like, so it's not easy at a glance to see the hierarchy. While this might not seem like a big deal since the drop-down I love was flat, but since the items aren't sorted alphabetically now, it can be a little tricky to find your item. Though this might get easier the more I use it! The other issue is that the small preview appears squashed sometime. I'm not sure if this is a bug, or if it's by design, but the button shown in mine doesn't quite look like the actual button on the form.

The XAML tree is pretty nifty, though a little fiddly to use. You can very easily select a parent of the currently selected item, and an arrow (" /> ") list children. While this is pretty powerful, since it only deals with parents/children, it's probably more likely you'll get into the habit of using the Document Outline instead.

Adding Non-Default Event Handlers

Mark pointed me at Chuck Sterling's blog showing new intellisense that can add event handlers. I'm still not convinced this is as nice as being able to easily see a filtered list of events, it's a nice addition. This would be much more friendly if there was a quick way to filter the intellisense to just list events (though I suspect there won't be).

Finding Properties

Mark hinted at "a better way of locating the properties in the property grid before we ship" which I suspect might be live filtering of properties like in some of the Expression tools (a search box, that hides properties that don't match what you're typing). Withing seeing it, I can't say whether it'll be as natural as looking for properties based on what letter they start with, but I'll post my thoughts when if and when I see it.

Rendering Issues

As for the rending issues, I managed to get a screenshot of the kind of thing I'm seeing. This particular issue shows up when I hit "Swap Panes" to replace the XAML with the Design view. It always renders the hosted WebBrowser control, but nothing around it. When I click anywhere on the surface, everything else appears.

Here's screenshots showing what's displayed after clicking the "Swap Panes" button and the expected view (which happens when I switch to XAML then back to Designer normally).

Update:

I've posted to the Cider Forums as requested.

.NET Using Dictionaries for a Performance Boost

2007-05-18T14:27:00Z

I often see code that loops through List<T>s or uses List<T>.ForEach() to repeatedly find specific items in a list. Take the below example. Rather than hitting the database every time we need to check permissions, they're all loaded upon login and cached into the session. As the user moves around the application, the permissions are checked with code similar to the below.

/// <summary>   
/// A cached list of posts with their access levels   
/// </summary>   
List<Post> posts;   

/// <summary>   
/// Preload the posts for testing permissions   
/// </summary>   
private void preloadPosts()   
{   
	posts = DataAccess.Post.GetAll();   
}   

/// <summary>   
/// Returns the access level for a given post   
/// </summary>   
/// <param name="postID"></param>   
/// <returns></returns>   
public AccessLevel GetAccessLevel(int postID)   
{   
	// Loop through each post looking for the access level   
	foreach (Post p in posts)   
		if (p.ID == postID)   
			return p.AccessLevel;   

	// If we didn't find a post, we don't have access to it   
	return AccessLevel.NoAccess;   
}

While this isn't a big deal if the list is small, or the method isn't called much, what started off as simple code could quickly grow.

I was asked to look at the performance of one our applications, and requested the company invest in RedGate Ants Profiler (which I'd highly recommend for anyone doing .NET development - shame there isn't a cheaper Home Edition!). When I ran the profiler it became very obvious very quickly where the profile was. The problem was that our application was taking several minutes to return a single list/table of data. The database had two tables: Person and Answer. The Answer table had a PersonID, and each Person could have hundreds of Answers. There were hundreds of People.

The offending code looped through all People, and then did a .FindAll() on a List<Answer> containing every answer for every positions. This mean with 10,000 positions, each having 1,000 answers, the List<Answer> had 10,000,000 items. This meant we were calling .FindAll() 10,000 times on 10,000,000 items. That's 100,000,000,000 iterations.

Since I wasn't able to modify any of the data access code at the time, I had to work with the two lists (List<Person> = 10,000 items, List<Answer> = 10,000,000 items).

Rather than loop through the 10,000,000 items for every position, I decided to loop through them once. By adding them to a dictionary based on their PersonID, I could quickly retrieve them while looping over the people.

The new code looked like this:

public void OutputData(List<Person> people, List<Answer> answers)   
{   
	// Create a dictionary to looup answers in   
	Dictionary<int, List<Answer>> answerLookup = new Dictionary<int, List<Answer>>();   
  
	// Loop through every answer   
	foreach (Answer a in answers)   
	{   
		// If we haven't hit this Person before, create a lookup for them   
		List<Answer> myAnswers;   
		if (!answerLookup.ContainsKey(a.PersonID))   
		{   
			myAnswers = new List<Answer>();   
			answerLookup.Add(a.PersonID, myAnswers);   
		}   
		else // Else get the existing list   
			myAnswers = answerLookup[a.PersonID];   
  
		// Add this answer to our lookup   
		myAnswers.Add(a);   
	}   
  
	// Now output our data   
	foreach (Person p in people)   
	{   
		// Get a list of ONLY the answers for this position   
		List<Answer> myAnswers = answers[p.PersonID];   
		Console.WriteLine(p.Name);   
  
		// Loop through our answers   
		foreach (Answer a in myAnswers)   
			Console.WriteLine("	{0}", a.AnswerValue);   
  
	}   
}

The resulting code took seconds to run instead of minutes, and the time spent on those loops was now only a small fraction of the total time (the database calls took most if it).

So, next time you find yourself writing a .Find() call or a .ForEach() to just find a particular element, consider the performance implications, and maybe add a Dictionary to make things faster.

Loading Models in XNA with the Content Pipeline

2007-05-17T07:43:00Z

The last time I played with XNA was when it was in in beta. There was no "Content Pipeline", and loading models was so tedious, I just decided to stop using XNA until it RTM'd. Since now is that time, I thought I'd best see how things have changed...

I bought the GarageGames Orcs RTS Buildings Pack, since it looked fitting for an MMO, was pretty cheap and had .x versions already included (which I know the Content Pipeline should handle).

I fired up GSE and added the .X files to my project and clicked Build....

I was greeted to some compile errors - and I hadn't even started coding!!

Thankfully, these errors were trivial. The content pack included .x and .jpg files with the same name (orcs1.jpg and orcs1.x). The content pipeline likes to strip the extensions off for the asset names, so there was a clash. I ran through and added ".jpg" to the asset name of each jpeg (though now I'm typing, I'm thinking I could've just Excluded them from the project, since the reference in the .x file would cause them to be added!).

With them all building, it was time to see if I could add them to my world. I was starting with the ChaseCamera sample from creators.xna.com, since I knew I'd be able to move the camera around easily to see the models.

At the top of Game.cs, I added a new variable to hold my models:

List<Model> buildingModels = new List<Model>();

And in the LoadGraphicsContent() method, I loaded each model. Since the filenames weren't sequential, I just hard-coded them for now:

buildingModels.Add(content.Load<Model>("Content/OrcPack/snow/orcs1"));
buildingModels.Add(content.Load<Model>("Content/OrcPack/snow/orcs11"));
buildingModels.Add(content.Load<Model>("Content/OrcPack/snow/orcs12"));
buildingModels.Add(content.Load<Model>("Content/OrcPack/snow/orcs13"));
buildingModels.Add(content.Load<Model>("Content/OrcPack/snow/orcs14"));
buildingModels.Add(content.Load<Model>("Content/OrcPack/snow/orcs15"));
buildingModels.Add(content.Load<Model>("Content/OrcPack/snow/orcs2"));
buildingModels.Add(content.Load<Model>("Content/OrcPack/snow/orcs3"));
buildingModels.Add(content.Load<Model>("Content/OrcPack/snow/orcs4"));
buildingModels.Add(content.Load<Model>("Content/OrcPack/snow/orcs5"));

And in my Draw() method, I called the existing DrawModel() method (which came from the ChaseCamera sample, to draw the ship/ground). Since the models are in a much smaller scale than the ship, I had to scale them up. Really this should be done per-asset at compile time - I'll see about creating a custom importer for this later (since all content packs will likely have different settings, and I want my units to be metres).

DrawModel(buildingModels[0], Matrix.Identity * Matrix.CreateTranslation(new Vector3(0, 0, 0)) * Matrix.CreateScale(100));
DrawModel(buildingModels[1], Matrix.Identity * Matrix.CreateTranslation(new Vector3(300, 0, 0)) * Matrix.CreateScale(100));
DrawModel(buildingModels[2], Matrix.Identity * Matrix.CreateTranslation(new Vector3(600, 0, 0)) * Matrix.CreateScale(100));
DrawModel(buildingModels[3], Matrix.Identity * Matrix.CreateTranslation(new Vector3(600, 0, -300)) * Matrix.CreateScale(100));
DrawModel(buildingModels[4], Matrix.Identity * Matrix.CreateTranslation(new Vector3(600, 0, -600)) * Matrix.CreateScale(100));
DrawModel(buildingModels[5], Matrix.Identity * Matrix.CreateTranslation(new Vector3(300, 0, -600)) * Matrix.CreateScale(100));
DrawModel(buildingModels[6], Matrix.Identity * Matrix.CreateTranslation(new Vector3(0, 0, -600)) * Matrix.CreateScale(100));
DrawModel(buildingModels[7], Matrix.Identity * Matrix.CreateTranslation(new Vector3(0, 0, -300)) * Matrix.CreateScale(100));
DrawModel(buildingModels[8], Matrix.Identity * Matrix.CreateTranslation(new Vector3(200, 0, -200)) * Matrix.CreateScale(100));
DrawModel(buildingModels[9], Matrix.Identity * Matrix.CreateTranslation(new Vector3(400, 0, -400)) * Matrix.CreateScale(100));

I also made a slight change to the ground.x model to reference the snowy ground.jpg from the content pack. I ran the project again, and there are the models!

So, in conclusion, the Content Pipeline makes loading models trivial. The next few things to look at are heightmaps, loading models in from some sort of "level file" rather than hard-coding, and animated models that react to user input (eg. breathing/running animations).

Using iPod with Windows Media Player instead of iTunes

2007-05-12T14:27:00Z

Up until recently, I've never had an iPod. It wasn't that I didn't like them - I think they're the best looking MP3 players out there. There was one reason, and one reason only that I hadn't bought one - iTunes. On Windows, and it has to be amongst the clunkiest applications from a major software company I've ever used. I'm not sure if it's because of the desire to make it look like a Mac and custom-draw everything, or just sloppy coding. It just always seems clunky and unresponsive, and hogs machine resources.

To make things worse, Apple started telling people not to upgrade to Vista because iTunes didn't work properly on it. WTF! Apple had how long to sort this out? You can't just tell people not to upgrade their operating system because you failed at meeting the deadline for your music player! (I do see this has since been fixed).

Anyway, back to the point. What stopped me buying an iPod was iTunes. The software is nasty, and I already have my music collection in Windows Media Player, which plays nice with Windows Media Center and my Xbox 360. If Apple made the iPod work with Windows Media Player, I'd have bought one. Assuming there are other people like me, Apple are losing potential business by trying to force people on to iTunes. I found there were a few ways to get Windows Media Player to work with the iPod, but it required 3rd party plugins (and cost money), and I wasn't about to buy an iPod to find these solutions aren't stable.

As fate would have it, I received a Blue 4GB iPod nano from my auntie when she was visiting from the states. The whole idea of not wanting to buy one in case it didn't work was squashed. I had an iPod, and I was going to use it, with or without Windows Media Player support!

When I got home, I downloaded a trial of MGTEK dopisp - one of the plugins claiming to get the iPod working with Windows Media Player. There are a few other plugins to do this, but a quick Google revealed less unhappy people using this one! I installed the plugin, connected my iPod, and fired up Windows Media Player.

Error! I was greeted with a dialog telling my my iPod had never been set up (via iTunes) and couldn't be used. I had a feeling this would happen, and luckily I had a laptop running Windows XP I was about to flatten! I installed iTunes and set up the iPod, then tried again. The iPod now appeared (with the name I assigned in iTunes) in Windows Media Player as a mobile device. I grabbed a few songs and tried to sync. It worked. It worked exactly like I wanted it to. It was that easy.

That makes me wonder why Apple didn't write a similar plugin? I understand they really want people to use iTunes, but is it really worth losing iPod sales over? Forcing people to use your software is not the way to do business. Sell your iPod on what it is. Sell iTunes on what it is. If people just want one, let them have it. You're lucky I received an iPod as a present, because you'd have missed out on this sale without native Windows Media Player support.

Visual Studio 2008 Beta 2 First Impressions: WPF Applications

2007-05-04T09:12:00Z

Visual Studio 2008 Beta 2 is now available. That means all of the cool things Microsoft have been tempted us with for the last few years are available to use: Linq, Linq to SQL, Lambda Expressions, (more stable) WPF Designer!

Something I didn't realise until now, is that the "WPF Application" option hasn't replaced the old "Windows Forms Application" option. You can still create .NET 3.5-targeted, good old Windows Forms. As it turns out, it's a good job, because the WPF designer just doesn't cut it. Here's some things I noticed within 5 minutes of trying to build a simple WPF App:

The property grid doesn't have an option to show Events
The property grid doesn't have an option to sort alphabetically
There's no drop-down above the property grid showing all the objects on your forms
There are massive rendering issues in the designer on my machine (Vista)

Being the final beta and with a Go-Live licence, I was hoping this was all going to be pretty stable, but it really isn't good enough. Sure, the property grid looks like a complete rewrite in WPF, and I admire them for that, but unless it's going to ship working at least as well as the old one, is it really worth it?

The final point, about rending issues is quite a bug. Sometimes controls just don't appear on the form until I click where they should be. If I run my application and then close it (so VS gets focus back), the form is completely blank. Waving the mouse around clicking reveals all of the controls.

All in all, I'm not impressed with the WPF Designer, and I won't be building my next application in WPF!

I don't know how Microsoft choose to categories their properties, so I always use A-Z view. When I've got controls obscuring other controls, I used to use that nice drop-down. Why must I know switch to XAML just to find my control? Why must I go via XAML to see my event handlers?

Let's hope this changes before VS2008 ships!!

Danny Tuppeny

Is Google Code Readying Git Support?

(Solved) Intermittent Server."CreateObject Failed" 'ASP 0177 : 8000ffff' Error Creating .NET COM Components

The Solution

Wanted: Artist for iPhone Game

Google Buzz - What's it all About?

World of Warcraft Addon Updaters - What Do You Use and Why?

Open Sourcing Google Wave Notifier

Generic 301 Redirection Script for Google App Engine

301 vs 302 Redirects

Generic App Engine Redirect Script

App.yaml Setup

Naked Domains on App Engine

Conclusion

Full Listing

Related Reading

Google App Engine Benchmarks - db.put() Performance

db.put() Performance

db.put() Performance Consistency

Individual db.put() Performance

Batch db.put() Performance

Related Reading

New Domain - DanTup.com - Please Update Your Links!

Entity Groups, Contention and Transactions in Google App Engine

Entity Groups are not Tables!

Putting Entities in the Same Entity Group

Entity Keys Include Their Parent Entity Keys

Related Reading

Profiling Google App Engine with Appstats

Guido van Rossum's Appstats

The Appstats Dashboard

Request Statistics

Related Reading

Caching App Engine Datastore Results in Memcache

Update

Related Reading

Serving Different Ads/Content Based on a Users Location (Geo-Targeting)

Google Wave Notifier Now in 15 Languages!

Redirecting Requests from appid.appspot.com to a Custom Domain

Related Reading

1-2-3-Testing. Is This Thing On? New Blog!

From .NET to Python on App Engine: Building a Blog on Google App Engine

Related Reading

Downloading/Exporting App Engine Logs

Related Reading

Increased App Engine Quotas, for Free?

Microsoft Windows Azure vs Google App Engine: Pricing

Related Reading

Should I move my Blog to Google App Engine?

So why are you hosted on Blogger?

What about Azure?

Enter, Google App Engine

Google Wave Notifier now in Spanish, Russian, Swedish

Google Wave Notifier Update

Google Wave Notifier Update - Automatic Update Checking

Google Wave Notifier v0.3 Public Release

Balloons! - The World Just Got Smaller!

Wanted: Beta Testers for Google Wave Notifier Application

Derren Brown - How (not) to Beat a (fake) Casino

The whole thing was a scam. It was not broadcast live.

Hold on a minute - he had a camera up his sleeve!

The casino was fake.

Related Reading

Aion - How Not to Launch an MMO

Realm Queues

Scaling in the Wrong Direction

Character/Realm Transfers

Bots, RMT and Spamming

Mass Account Banning

Related Reading

Importing iPhone Dev Keys on a new Mac

Oh, shit.

Google to the rescue!

Work-around provided by Dave K.

I'm Blogging... From a Mac!

ASP.NET MVC HandleError Attribute, Custom Error Pages and Logging Exceptions

The HandleError Attribute

But How Do I Log Exceptions?

The OnException Method

How Do I see my own Errors During Development?