Danny Tuppeny

Dart Code v2 - Now with Flutter debugging and hot reload!

2017-07-11T00:00:00+00:00

It’s already been ten months since I published the first preview of Dart Code and today marks a significant milestone - version 2.0! It’s been some time in the making; this major update adds support for launching and debugging Android and iOS apps written using Flutter, including:

When a Flutter project is open your selected device/emulator will be shown in the status bar
If you have more than one connected device/emulator, clicking the device name in the status bar will allow you to switch between devices/emulators
Pressing F5 with a flutter project open will build and launch your app on the selected device/emulator
The usual debugging experience is available for Flutter apps, including breakpoints, call stacks, watch window etc.
The debugger’s Restart button (or Ctrl+Shift+F5) has been mapped to Flutter’s hot reload feature and will update the running application without needing to rebuild (this usually results in sub-second updates!)

Bram Vanbilsen, who has started an excellent set of Flutter tutorials on YouTube has created a short video showing how this integration works with a demo of hot reload:

Download

In order to get up and running you’ll need to grab a few things:

Visual Studio Code version 1.13.0 or later
v2 of Dart Code
A recent version of the Dart SDK
For Flutter support you’ll also need to set up Flutter

Finally, to avoid manual configuration you should also add the Dart and Flutter SDKs to your PATH.

Feedback

If you do try it out, I’d love your feedback. You can find me on Twitter (@DanTup and @DartCode), post issues at GitHub/Dart-Code or find me in the Gitter chat room!

Thanks

I’ve never blogged about Dart Code before so I’ve never really said thanks to all those that helped make it possible. It’s a long list! Dart Code builds on a lot of excellent work done by many people at Google (including the analyzer, which powers most of the language and editor support, the VM and debugger, the flutter tooling) and of course many people at Microsoft working on VS Code. I’ve opened a lot of issues, sent a lot of emails and asked a lot of questions in chat rooms for both projects and I’ve had great support. I’ve also had some great contributions including the bulk of the debugger work from Devon and many beta testers who helped iron out issues in the Flutter integration before I unleashed it!

Up Next - Web Support

With Flutter integration released the next major feature for Dart Code is likely to be better support for web apps, including F5 to launch and incremental compilation. Stay tuned to @DartCode on Twitter for update notifications!

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

I found a ridiculous bug in YouTube and seem unable to successfully report it :(

2017-02-03T00:00:00+00:00

Last week a colleague tweeted me telling me that the hyperlinks in the descriptions of my kids review of the Echo Dot were giving 404s. This surprised me because I’d clicked them to ensure they worked after publishing the video! I probed him and he sent a screenshot of 404s. I checked them again myself both on desktop and my mobile and everything was fine. Since this colleague has a certain sense of humour, I figured he was trying to wind me up and didn’t think much of it.

YouTube is truncating links not just for the display but the actual URL the user is taken to

The next day he showed me the issue on his mobile phone - he was right - the links had been truncated! WTF? I noticed that when he clicked YouTube links in twitter they were opening in Chrome, not the YouTube app. I tried the same thing on my mobile and was immediately able to reproduce the issue. I did some Googling and found reports of a bug in 2013 that sounded similar. I dug into this and discovered that this wasn’t the same issue - that was an issue with the inline editor truncating links when editing; my links were fine in the editor and on desktop/mobile. This is a different bug. The YouTube mobile site is truncating links not just for the display but the actual URL the user is taken to. Whaaaaa?

You might think “Who uses the mobile site anyway? Just Windows Phone users?” but the colleague that reported it to me has an Android phone that came pre-installed with the YouTube app, yet clicking YT links in Twitter opens the mobile web version. Lots of promotion of YouTube videos is done on Twitter and other social media so the number of people hitting this bug is probably not insignificant. Probably they assume the video creator is a muppet rather than a YouTube bug.

I did some more Googling and could not find a single report of this bug online! I’m surprised that a bug with links being truncated for a large number of people (if only 1% of YouTube traffic is mobile web, that’s still huge) can go unnoticed.

I decided to check it wasn’t just me so I opened up some videos from popular YouTubers I follow… Casey Neistat… Same thing.

I tried Sean Cannell… Same thing.

I tried MKBHD… Same thing.

Sara Dietschy… Same thing.

What the fuck YouTube? Where is the testing? Have we given up?.

So, I did what I always do when I find a bug. I tried to report it. Quite a few times…

In the official forums - no responses from employees nor top contributors
Via Send Feedback on the web - I use this a lot in Google apps and they all feel like black holes
Via Send Feedback in the mobile app - as above
Twitter (three times) - no response here (despite the YouTube account responding to others every day)
In /r/YouTube/ knowing employees post there; it fell off the homepage with no responses (and even downvotes)
I tweeted some of the YouTubers I followed to alert them of this issue (it’s lost revenue for those putting affiliate links here and maybe they have contacts?) though the only response I got wasn’t clear if the YouTuber really understood what I was saying.

The only workaround for this bug that I know of is to shorten all your links. I hate this because it looks scummy like you’re trying to hide something. I created my own short links for the Review Kidz videos for now that hopefully makes it clearer where they go than goo.gl but it’s not a great solution.

So, this blog post is my final attempt to get someone from YouTube/Google to see this so they might consider fixing this (IMO completely ridiculous) bug. I’d rather not have to shorten links because of this but it’s looking like that’s how it’s going to be. It’s amazing both that a product as large as YouTube can have so many broken links and apparently nobody has noticed and that there’s very little information on how to report bugs or get their attention (though honestly, this seems par for the course with Google these days; support is simply nonexistent).

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Simple Dart code to tweet using OAuth

2017-01-27T00:00:00+00:00

Writing Dart? Check out Dart Code, my Dart extension for Visual Studio Code!

Last year I blogged about the simplest C# code I could come up with to post tweets without having to jump through OpenAuth hoops. This only works for accounts you can get keys for (eg. your own) but it’s significnatly simpler than the normal OAuth flow.

Since running C# on my Raspberry Pi turned out to be significantly more effort than I expected from the “cross platform” .NET Core (I did try mono but getting SSL working to talk to Twitter was a clusterfuck of pasting commands from the internet that didn’t work) I decided to port it to one of my favourite languages, Dart! It would also be a nice task to test out Dart Code, the Dart plugin I created for Visual Studio Code (with much support from the Dart and VS Code teams) which I don’t get to do nearly enough of.

Like with the C# version it turned out to be complicated to encode strings in the way that Twitter expects. I tried all sorts of encoding methods built-in to Dart and none of them gve the expected results. The convert package published by the Dart team claimed to be RFC 3986 compliant but it seemed to encode numbers - 1 became %31 and Twitter didn’t like that. Assuming this was a bug, I raised an issue and sent a PR with tests and a fix. It took a little while but my PR was merged and a new version of the package released with the fix.

If you want the details about how to get your API keys to use this code, read the the C# version of this post which contains links to the docs at Twitter. The rest of this article is just a summary of the actual Dart code. It’s very much a direct port of the C# tidied up where Dart made it easy. If I’m honest, I think the Dart version is much less readable due to dartfmt wrapping awkwardly. I do format using this currently but I’m very much on the fence about whether it’s worthwhile sacrificing readability and the ability to use tabs for formatting (I think there are better ways of achieving the consistency they aim for).

On to the code!

As before, the constructor takes in the keys and creates a hasher. (Note: Indenting is dartfmts; two spaces for indenting but 4 spaces for wrapped lines.. I’d prefer it if the top line was unwrapped).

TwitterApi(this.consumerKey, this.consumerKeySecret, this.accessToken,
    this.accessTokenSecret) {
  var bytes = UTF8.encode("$consumerKeySecret&$accessTokenSecret");
  _sigHasher = new Hmac(sha1, bytes);
}

The tweet method is async and just builds the payload before calling a re-usable _callApi method (note: in Dart underscore prefixes automatically make things private). This will allow adding additional methods easier (as Eelco Koster did with the C# version). I set trim_user just to reduce the amount of data that comes back in the response (since I don’t use it for anything).

/// Sends a tweet with the supplied text and returns the response from the Twitter API.
Future<String> tweet(String text) {
  var data = {"status": text, "trim_user": "1"};

  return _callApi("statuses/update.json", data);
}

The _callApi method is almost identical to the C# SendRequest method so I’ll skip over it here (the full code at the bottom of the post).

I extracted some helpers for making query strings and OAuth headers (which annoyingly were subtly different; one requires quotes and one can’t have them) which made the signature and header generation methods pretty terse:

/// Generate an OAuth signature from OAuth header values.
String _generateSignature(Uri url, Map<String, String> data) {
  var sigString = _toQueryString(data);
  var fullSigData = "POST&${_encode(url.toString())}&${_encode(sigString)}";

  return BASE64.encode(_hash(fullSigData));
}

/// Generate the raw OAuth HTML header from the values (including signature).
String _generateOAuthHeader(Map<String, String> data) {
  var oauthHeaderValues = _filterMap(data, (k) => k.startsWith("oauth_"));

  return "OAuth " + _toOAuthHeader(oauthHeaderValues);
}

Next is a method for sending the request over to Twitter. I originally missed the http.close from the end of this which resulted in a 10-15 second hang when my program finished (the Dart VM didn’t exit until the connection timed out). I don’t really like that you need to add that (it’s easily forgotten and a pain to debug/understand) but I don’t know of a good fix.

This is the most unreadable part of this class IMO after running through dartfmt (I would put breaks/indent before .then’s and unwrap the contentType header).

Edit: Slightly more readable after switching to await from .then (thanks Benjamin Campbell!).

/// Send HTTP Request and return the response.
Future<String> _sendRequest(
    Uri fullUrl, String oAuthHeader, String body) async {
  final http = new HttpClient();
  final request = await http.postUrl(fullUrl);
  request.headers
    ..contentType = new ContentType("application", "x-www-form-urlencoded",
        charset: "utf-8")
    ..add("Authorization", oAuthHeader);
  request.write(body);
  final response = await request.close().whenComplete(http.close);
  return response.transform(UTF8.decoder).join("");
}

The class is used in the same way as the C# version and the response from tweet awaited:

var twitter = new TwitterApi(liveConsumerKey, liveConsumerKeySecret,
    liveAccessToken, liveAccessTokenSecret);

var resp = await twitter.tweet("YOUR TWEET HERE");

Since my need is a very simple script where I receive the output my email, my error handling is very lame:

if (resp.startsWith("{\"errors\"")) {
  print("Failed to send tweet:");
  print("");
  print(tweetToSend);
  print("");
  print(resp);
} else {
  // Success!
}

If using this more seriously, you’d want to return a nice class from the tweet method instead.

Here’s the full class; you’ll need to add convert and crypto to your pubspec.yaml and run pub get to get this working.

import 'dart:async';
import 'dart:convert';
import 'dart:io';
import 'package:convert/convert.dart';
import 'package:crypto/crypto.dart';

const String twitterApiBaseUrl = "https://api.twitter.com/1.1/";

class TwitterApi {
  final String consumerKey, consumerKeySecret, accessToken, accessTokenSecret;
  Hmac _sigHasher;
  final DateTime _epochUtc = new DateTime(1970, 1, 1);

  // TwitterApi class adapted from DanTup:
  // https://blog.dantup.com/2017/01/simplest-dart-code-to-post-a-tweet-using-oauth/

  TwitterApi(this.consumerKey, this.consumerKeySecret, this.accessToken,
      this.accessTokenSecret) {
    var bytes = UTF8.encode("$consumerKeySecret&$accessTokenSecret");
    _sigHasher = new Hmac(sha1, bytes);
  }

  /// Sends a tweet with the supplied text and returns the response from the Twitter API.
  Future<String> tweet(String text) {
    var data = {"status": text, "trim_user": "1"};

    return _callApi("statuses/update.json", data);
  }

  Future<String> _callApi(String url, Map<String, String> data) {
    var fullUrl = Uri.parse(twitterApiBaseUrl + url);

    // Timestamps are in seconds since 1/1/1970.
    var timestamp = new DateTime.now().toUtc().difference(_epochUtc).inSeconds;

    // Add all the OAuth headers we'll need to use when constructing the hash.
    data["oauth_consumer_key"] = consumerKey;
    data["oauth_signature_method"] = "HMAC-SHA1";
    data["oauth_timestamp"] = timestamp.toString();
    data["oauth_nonce"] = "a"; // Required, but Twitter doesn't appear to use it
    data["oauth_token"] = accessToken;
    data["oauth_version"] = "1.0";

    // Generate the OAuth signature and add it to our payload.
    data["oauth_signature"] = _generateSignature(fullUrl, data);

    // Build the OAuth HTTP Header from the data.
    var oAuthHeader = _generateOAuthHeader(data);

    // Build the form data (exclude OAuth stuff that's already in the header).
    var formData = _filterMap(data, (k) => !k.startsWith("oauth_"));

    return _sendRequest(fullUrl, oAuthHeader, _toQueryString(formData));
  }

  /// Generate an OAuth signature from OAuth header values.
  String _generateSignature(Uri url, Map<String, String> data) {
    var sigString = _toQueryString(data);
    var fullSigData = "POST&${_encode(url.toString())}&${_encode(sigString)}";

    return BASE64.encode(_hash(fullSigData));
  }

  /// Generate the raw OAuth HTML header from the values (including signature).
  String _generateOAuthHeader(Map<String, String> data) {
    var oauthHeaderValues = _filterMap(data, (k) => k.startsWith("oauth_"));

    return "OAuth " + _toOAuthHeader(oauthHeaderValues);
  }

  /// Send HTTP Request and return the response.
  Future<String> _sendRequest(
      Uri fullUrl, String oAuthHeader, String body) async {
    final http = new HttpClient();
    final request = await http.postUrl(fullUrl);
    request.headers
      ..contentType = new ContentType("application", "x-www-form-urlencoded",
          charset: "utf-8")
      ..add("Authorization", oAuthHeader);
    request.write(body);
    final response = await request.close().whenComplete(http.close);
    return response.transform(UTF8.decoder).join("");
  }

  Map<String, String> _filterMap(
      Map<String, String> map, bool test(String key)) {
    return new Map.fromIterable(map.keys.where(test), value: (k) => map[k]);
  }

  String _toQueryString(Map<String, String> data) {
    var items = data.keys.map((k) => "$k=${_encode(data[k])}").toList();
    items.sort();

    return items.join("&");
  }

  String _toOAuthHeader(Map<String, String> data) {
    var items = data.keys.map((k) => "$k=\"${_encode(data[k])}\"").toList();
    items.sort();

    return items.join(", ");
  }

  List<int> _hash(String data) => _sigHasher.convert(data.codeUnits).bytes;

  String _encode(String data) => percent.encode(data.codeUnits);
}

You’re free to do as you please with this code. If you improve anything significantly, do leave a comment!

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Making more money with Amazon: Splitting visitors between Amazon US/UK based on their location

2017-01-14T00:00:00+00:00

Recently I was chatting with a friend in the UK Amazon Associates programme. He hadn’t realised that all of the US traffic he sends to affiliate links at amazon.co.uk didn’t count for anything as the affiliate programmes are completely separate. Most products can’t be shipped (or are expensive to) between Amazon regions so he’s missing out on a lot of earnings by not sending US visitors to amazon.com. I provided him with a little JavaScript and some instructions to rememdy this but, since he’s not a developer and is using a CMS it wasn’t exactly a simple solution having to hand-edit code to include both UK and US product identifiers (which are rarely the same) in his links for my script to work.

This gave me an idea for a new service; enter us-uk-redirector.com (aka amzredir.com)!

Here you can paste in URLs for products on amazon.com and amazon.co.uk and you’ll be given back a URL that will redirect UK visitors to amazon.co.uk and others to amazon.com. There’s nothing to install on your site and there’s no messing in markup - just paste a hyperlink in the way that you normally would. There are also separate fields for your affiliate IDs which means you can paste in any old Amazon link without having to ensure your affiliate tag is in there. They’re also persisted to local storage so next time you come back they’re pre-filled!

In addition to pasting URLs you can enter ASIN numbers or search terms directly (did I mention you can paste in search results URLs too? it’ll parse them!). A banner is shown beneath each box explaining exactly what will happen to visitors for this region.

The URLs will automatically combine similar info to keep them short, so if the ASIN or search terms are the same, or if your affiliate tags are the same for UK and US (except for the -20 and -21 suffix) they will be combined and handled during the redirect.

It will even shout at you if you enter invalid URLs or affiliate tags to ensure you’re not messing up.

The URLs are simple to build yourself too, the format is described on the site.

If you’re an Amazon affiliate check it out, I’d love feedback. It’ll work for non-affiliates too if you’d still like to send visitors to the correct site (or if you’re only an affiliate in one region, it’ll still work).

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Visiting a site that uses Disqus comments when not logged in sends the URL to Facebook

2017-01-06T00:00:00+00:00

(Update: As a result of this post Disqus fixed this issue on 9th Jan! See here)

Recently I was making some tweaks to my blog and reviewing the page load time. I noticed that when I loaded an article the Facebook SDK was being loaded!

This was a bit strange as I didn’t recall ever having added any Facebook widgets to my blog. Not only is this somewhat wasteful and harming my load time, it also bugged me that visitors were having these URLs sent to Facebook. I’ve always thought it strange that Facebook seems to know everywhere I’ve been and it seems my blog is adding to this without me even knowing!

So, I checked why this file was being loaded and was surprised to see Disqus as the initiator. I do have Disqus comments on my blog but that doesn’t seem like a good reason to have the Facebook SDK loaded immediately! I tweeted Disqus though unsurprisingly haven’t recieved a response.

Fast forward a week and I was reading Troy Hunt’s I just permanently removed all ad network code from my blog. Troy cited tracking as one of the reasons for removing ads (I strongly support this decision; running unknown JavaScript from ad networks on an HTTPS site always seemed a bit weird to me). In a response on twitter I noticed someone asked him about still having Disqus, which reminded me about what I’d seen on my blog and wondered if the same thing happened there that I saw on my blog.

Well, it turns out it does. Here’s Troy’s blog loading the Facebook SDK and clearly sending the URL for the post I’m reading to Facebook in the referer header (it’s URL-encoded in the querystring of the Disqus comments frame URL).

Now, you might say that this isn’t really the URL of the page, it’s going via Disqus and Facebook won’t read it. I’d counter that since Facebook are very interested in tracking where you visit and Disqus is so common that it is in their best interest to understand this URL and extract the real article URL from the querystring.

I did a little more digging and discovered that this only happens if you’re not logged in to Disqus. It also happens even if Do-Not-Track is enabled in your browser. It’s almost certainly as a result of the “Sign in with Facebook” functionality however I don’t believe it has to, not should, be this way. If I implemented “Sign in with Facebook” on my site directly then I’ve chosen to pull the Facebook SDK in and probably understand the implications. Adding Disqus comments doesn’t really scream out that the Facebook SDK will be loaded for all “anonymous” visitors even before they’ve scrolled anywhere near the Disqus comments.

I’m certain Disqus could fix this, not only resulting in better load times but also better privacy for users. Yes, logging in with Facebook might become slightly slower as a result but this doesn’t seem like a compelling enough reason to keep it as-is to me.

I don’t think Disqus will change this as a result of my tweet so I figured I’d try and raise some awareness and maybe that would help encourage them.

Yes, my blog is still using Disqus today. If they don’t fix this then I will probably investigate moving away. I don’t know what good alternatives exist (or how easily I can move all existing comments - which have great value on some posts) so it will take some time if it happens.

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Using Alexa (Amazon Echo) to Preheat my Renault ZOE and check Battery Status

2016-12-02T00:00:00+00:00

Yesterday my 2nd Gen Amazon Echo Dot arrived (check out my kids review of the Echo Dot here) and I thought it’d be fun to try and hook it up to my Renault ZOE so that I could check battery status and preheat it without having to load the clunky app.

Alexa Skills can be hosted on AWS Lambda and there’s an easy-to-follow tutorial in the dev docs. The Renault ZOE doesn’t have an official API but last year I’d used Fiddler to sniff the mobile app and been able to emulate that. Since then their app has just become a webview wrapper so I was a little worried I wouldn’t be able to do this - however, SPAs to the rescue… the app is written in Angular and calls the backend in a really simple JSON API (way better than the previous XML abomination!).

I had a quick Google and found that Terence Eden had already pasted all of the request/response info online which saved me a little effort trawling through. I whipped up a quick node script that calls the API to login and request battery status / preheating. It’s relatively simple, not much code at all (though could do with a little refactoring).

"use strict";

let https = require("https");

let token, vin, zoeUsername, zoePassword, loginFailureCallback;

function sendRequest(action, requestData, successCallback, failureCallback) {
	const options = {
		hostname: "www.services.renault-ze.com",
		port: 443,
		path: "/api" + action,
		method: requestData ? "POST" : "GET",
		headers: {
			"Content-Type": "application/json",
			"Authorization": (token ? "Bearer " + token : "")
		}
	};

	const req = https.request(options, resp => {
		if (resp.statusCode < 200 || resp.statusCode > 300) {
			console.log(`Failed to send request ${action} (${resp.statusCode}: ${resp.statusMessage})`);
			if (failureCallback)
				failureCallback();
			return;
		}

		console.log(`Successful request ${action} (${resp.statusCode}: ${resp.statusMessage})`);
		let respData = "";

		resp.on("data", c => {
			//console.log("<== " + c.toString());
			respData += c.toString();
		});
		resp.on("end", () => {
			if (successCallback)
				successCallback(respData && respData.length ? JSON.parse(respData) : null);
		});
	});
	if (requestData && JSON.stringify(requestData) !== '{}')
		req.write(JSON.stringify(requestData));
	req.end();
}

function login(successCallback) {
	sendRequest("/user/login", {
		username: zoeUsername,
		password: zoePassword
	}, loginResponse => {
		token = loginResponse.token;
		vin = loginResponse.user.vehicle_details.VIN;
		successCallback();
	}, loginFailureCallback);
}

exports.setLogin = (username, password, failureCallback) => {
	zoeUsername = username;
	zoePassword = password;
	loginFailureCallback = failureCallback;
}

exports.getBatteryStatus = (successCallback, failureCallback) => {
	login(() => sendRequest("/vehicle/" + vin + "/battery", null, successCallback, failureCallback));
}

exports.sendPreheatCommand = (successCallback, failureCallback) => {
	login(() => sendRequest("/vehicle/" + vin + "/air-conditioning", {}, successCallback, failureCallback));
}

Next was wiring this up to work the Alexa API. I created a separate JS file that require'd the one above and just handled two simple intents (preheat and battery status). If you ask for help or launch without an intent it just tells you what the options are. Most requests return a card too, so you can see them inside the Alexa app.

Since we have two cars (two ZOEs!), I actually set this up to respond to two different app IDs and connect to the correct account (and rejected any other apps, so if you jokesters find the ID you can’t burn my battery ;)). The usernames/passwords come from environment variables I would set up in Lambda to make it easier to share code without removing them.

"use strict";

const dannyAlexaApp = "amzn1.ask.skill.xxxxx";
const heatherAlexaApp = "amzn1.ask.skill.yyyyy";

let car = require("./car");

function buildResponse(output, card, shouldEndSession) {
	return {
		version: "1.0",
		response: {
			outputSpeech: {
				type: "PlainText",
				text: output,
			},
			card,
			shouldEndSession
		}
	};
}

// Helper to build the text response from battery information.
function buildBatteryStatus(battery) {
	let response = `You have ${battery.charge_level}% battery which will get you approximately ${Math.round(battery.remaining_range * 0.621371)} miles. `;

	if (battery.plugged)
		response += "The car is plugged in";
	else
		response += "The car is not plugged in";

	if (battery.charging)
		response += " and charging";

	return response + ".";
}

exports.handler = (event, context) => {
	// Helper to return a response with a card.		
	const sendResponse = (title, text) => {
		context.succeed(buildResponse(text, {
			"type": "Simple",
			"title": title,
			"content": text
		}));
	};
	
	try {
		console.log(`event.session.application.applicationId=${event.session.application.applicationId}`);

		// Shared callbacks.
		const exitCallback = () => context.succeed(buildResponse("Goodbye!"));
		const helpCallback = () => context.succeed(buildResponse("What would you like to do? You can preheat the car or ask for battery status.", null, false));
		const loginFailureCallback = () => sendResponse("Authorisation Failure", "Unable to login to Renault Z.E. Services, please check your login credentials.");

		// Set login based on the Alexa app ID.
		//   We have two cars, so two activation names ("my car" and "Heather's car").
		//   If you're not either of these apps, you're not allowed to control our cars!
		if (event.session.application.applicationId === dannyAlexaApp) {
			car.setLogin(process.env.ZOE_USER_DANNY, process.env.ZOE_PASS_DANNY, loginFailureCallback);
		} else if (event.session.application.applicationId === heatherAlexaApp) {
			car.setLogin(process.env.ZOE_USER_HEATHER, process.env.ZOE_PASS_HEATHER, loginFailureCallback);
		} else {
			sendResponse("Invalid Application ID", "You are not allowed to use this service.");
			return;
		}

		// Handle launches without intents by just asking what to do.		
		if (event.request.type === "LaunchRequest") {
			helpCallback();
		} else if (event.request.type === "IntentRequest") {
			// Handle different intents by sending commands to the API and providing callbacks.
			switch (event.request.intent.name) {
				case "PreheatIntent":
					car.sendPreheatCommand(
						response => sendResponse("Car Preheat", "The car will begin preheating shortly."),
						() => sendResponse("Car Preheat", "Unable to begin preheating. Have you already done this recently?")
					);
					break;
				case "GetBatteryStatusIntent":
					car.getBatteryStatus(
						battery => sendResponse("Car Battery Status", buildBatteryStatus(battery)),
						() => sendResponse("Car Battery Status", "Unable to get car battery status, please check your login details.")
					);
					break;
				case "AMAZON.HelpIntent":
					helpCallback();
					break;
				case "AMAZON.StopIntent":
				case "AMAZON.CancelIntent":
					exitCallback();
					break;
			}
		} else if (event.request.type === "SessionEndedRequest") {
			exitCallback();
		}
	} catch (err) {
		console.error(err.message);
		sendResponse("Error Occurred", "An error occurred. Fire the programmer! " + err.message);
	}
};

I created two apps in Alexa (both using the same Lambda function) and set up the intent schema:

{
  "intents": [
    { "intent": "PreheatIntent" },
    { "intent": "GetBatteryStatusIntent" },
    { "intent": "AMAZON.HelpIntent" },
    { "intent": "AMAZON.StopIntent" }
  ]
}

… and some utterances:

PreheatIntent pre heat
PreheatIntent preheat
PreheatIntent heat
PreheatIntent heater
PreheatIntent turn the heater on
PreheatIntent warm up
PreheatIntent heat up
PreheatIntent preheat for me
PreheatIntent start the heaters
PreheatIntent turn the heater on
PreheatIntent turn on the heater
PreheatIntent turn the heat on
PreheatIntent turn on the heat
PreheatIntent warm up
PreheatIntent make it toasty for me
PreheatIntent it's cold outside
PreheatIntent heat my seat up
GetBatteryStatusIntent battery status
GetBatteryStatusIntent status
GetBatteryStatusIntent battery
GetBatteryStatusIntent level
GetBatteryStatusIntent battery level
GetBatteryStatusIntent how much range
GetBatteryStatusIntent range
GetBatteryStatusIntent what is your battery status
GetBatteryStatusIntent tell me your status
GetBatteryStatusIntent are you charged
GetBatteryStatusIntent how do your batteries look
GetBatteryStatusIntent what is your battery level
GetBatteryStatusIntent how are your batteries
GetBatteryStatusIntent is there any charge left
GetBatteryStatusIntent how much charge is left
GetBatteryStatusIntent how much range is left
GetBatteryStatusIntent what is your range
GetBatteryStatusIntent how many miles can we drive
GetBatteryStatusIntent what is the range left in the battery
GetBatteryStatusIntent how much range is left in the battery
GetBatteryStatusIntent how far can we drive

Finally, I created a little dummy script that lets me test the functions without having to go through the Echo for a faster dev cycle. I didn’t go as far as mocking the HTTP requests but I did modify the code temporarily to ensure the error handling worked properly (detecting invalid username/password, or if you try to pre-heat too often).

"use strict";

let zoe = require('./lambda/index');

let batteryStatusEvent = {
	"session": {
		"application": {
			"applicationId": "amzn1.ask.skill.zzz"
		}
	},
	"version": "1.0",
	"request": {
		"type": "IntentRequest",
		"intent": { "name": "GetBatteryStatusIntent" }
	}
};

let preheatEvent = {
	"session": {
		"application": {
			"applicationId": "amzn1.ask.skill.zzz"
		}
	},
	"version": "1.0",
	"request": {
		"type": "IntentRequest",
		"intent": { "name": "PreheatIntent" }
	}
};

let context = {
	succeed: function (resp) {
		console.log("\n\nResult:\n" + JSON.stringify(resp, null, 4));
	}
};

zoe.handler(batteryStatusEvent, context);
zoe.handler(preheatEvent, context);

After some successfully tests, I zipped the files up and uploaded them to Lambda to replace the favourite-color sample I’d been testing with. The final result is in the video at the top of this post!

You’re free to do as you please with this code. If you improve anything significantly, do leave a comment! If you’ve done anything similar with your Echo Dot, do leave a comment!

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Simplest C# code to post a tweet using OAuth

2016-07-07T00:00:00+00:00

Edit: I’ve posted a Dart port of this code here.

For a few years I’ve been meaning to automate the posting of tweets to a couple of twitter accounts I run. I’ve always been too ~~lazy~~busy but this week I had some time and decided to get it done. For many reasons (including that my parents taught me not to take binary dependencies from strangers on the internet ;)) I decided not to use a 5,000 line-of-code third party library but just instead write the simplest code for what I needed.

While Googling for other peoples implementations of “simplest code to post tweets” to see what I was getting myself into (I didn’t actually find any!) I stumbled across a page on the Twitter site about Single-user OAuth. This allows you to generate tokens for authenticating without having to go through the normal OAuth ballache. Obviously you can only do this for your own account but it massively simplifies things.

By using a single access token, you don’t need to implement the entire OAuth token acquisition dance. Instead, you can pick up from the point where you are working with an access token to make signed requests for Twitter resources.

Most of the work is fairly straight forward but it also requires an OAuth signature in the OAuth header. This was a bit of a pain in the ass… You have to build a big string of all the data in a specific order with specific encoding and then hash it with the secret keys you got. I implemented this (or so I thought) and sent it off to Twitter only go get a BAD AUTHENTICATION DATA response with zero information on what exactly was wrong. Great :(

After much keyboard-bashing I ragequit for the night. Stupid Twitter. After some much-required sleep I realised that on the Creating a signature page, Twitter actually gave an example including the secret keys and timestamp. This is a perfect test case for testing out hashing code! It even gives the values at each step of the algorithm to help track down where you’re going wrong. My issues turned out to mostly be encoding - I’d tried using WebUtility.UrlEncode and Uri.EscapeUriString but neither encoded spaces and pluses the way that Twitter excepted. It turned out that Uri.EscapeDataString does encode exactly as Twitter requires.

In total, my class turned out to be around 80 lines of code without comments. It’s provided here in its entirety for easy copy/pasting (no, there’s no NuGet package.. didn’t anyone tell you that you shouldn’t trust binary dependencies from strangers on the internet?).

/// <summary>
/// Simple class for sending tweets to Twitter using Single-user OAuth.
/// https://dev.twitter.com/oauth/overview/single-user
/// 
/// Get your access keys by creating an app at apps.twitter.com then visiting the
/// "Keys and Access Tokens" section for your app. They can be found under the
/// "Your Access Token" heading.
/// </summary>
class TwitterApi
{
	const string TwitterApiBaseUrl = "https://api.twitter.com/1.1/";
	readonly string consumerKey, consumerKeySecret, accessToken, accessTokenSecret;
	readonly HMACSHA1 sigHasher;
	readonly DateTime epochUtc = new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc);

	/// <summary>
	/// Creates an object for sending tweets to Twitter using Single-user OAuth.
	/// 
	/// Get your access keys by creating an app at apps.twitter.com then visiting the
	/// "Keys and Access Tokens" section for your app. They can be found under the
	/// "Your Access Token" heading.
	/// </summary>
	public TwitterApi(string consumerKey, string consumerKeySecret, string accessToken, string accessTokenSecret)
	{
		this.consumerKey = consumerKey;
		this.consumerKeySecret = consumerKeySecret;
		this.accessToken = accessToken;
		this.accessTokenSecret = accessTokenSecret;

		sigHasher = new HMACSHA1(new ASCIIEncoding().GetBytes(string.Format("{0}&{1}", consumerKeySecret, accessTokenSecret)));
	}

	/// <summary>
	/// Sends a tweet with the supplied text and returns the response from the Twitter API.
	/// </summary>
	public Task<string> Tweet(string text)
	{
		var data = new Dictionary<string, string> {
			{ "status", text },
			{ "trim_user", "1" }
		};

		return SendRequest("statuses/update.json", data);
	}

	Task<string> SendRequest(string url, Dictionary<string, string> data)
	{
		var fullUrl = TwitterApiBaseUrl + url;

		// Timestamps are in seconds since 1/1/1970.
		var timestamp = (int)((DateTime.UtcNow - epochUtc).TotalSeconds);

		// Add all the OAuth headers we'll need to use when constructing the hash.
		data.Add("oauth_consumer_key", consumerKey);
		data.Add("oauth_signature_method", "HMAC-SHA1");
		data.Add("oauth_timestamp", timestamp.ToString());
		data.Add("oauth_nonce", "a"); // Required, but Twitter doesn't appear to use it, so "a" will do.
		data.Add("oauth_token", accessToken);
		data.Add("oauth_version", "1.0");

		// Generate the OAuth signature and add it to our payload.
		data.Add("oauth_signature", GenerateSignature(fullUrl, data));

		// Build the OAuth HTTP Header from the data.
		string oAuthHeader = GenerateOAuthHeader(data);

		// Build the form data (exclude OAuth stuff that's already in the header).
		var formData = new FormUrlEncodedContent(data.Where(kvp => !kvp.Key.StartsWith("oauth_")));

		return SendRequest(fullUrl, oAuthHeader, formData);
	}

	/// <summary>
	/// Generate an OAuth signature from OAuth header values.
	/// </summary>
	string GenerateSignature(string url, Dictionary<string, string> data)
	{
		var sigString = string.Join(
			"&",
			data
				.Union(data)
				.Select(kvp => string.Format("{0}={1}", Uri.EscapeDataString(kvp.Key), Uri.EscapeDataString(kvp.Value)))
				.OrderBy(s => s)
		);

		var fullSigData = string.Format(
			"{0}&{1}&{2}",
			"POST",
			Uri.EscapeDataString(url),
			Uri.EscapeDataString(sigString.ToString())
		);

		return Convert.ToBase64String(sigHasher.ComputeHash(new ASCIIEncoding().GetBytes(fullSigData.ToString())));
	}

	/// <summary>
	/// Generate the raw OAuth HTML header from the values (including signature).
	/// </summary>
	string GenerateOAuthHeader(Dictionary<string, string> data)
	{
		return "OAuth " + string.Join(
			", ",
			data
				.Where(kvp => kvp.Key.StartsWith("oauth_"))
				.Select(kvp => string.Format("{0}=\"{1}\"", Uri.EscapeDataString(kvp.Key), Uri.EscapeDataString(kvp.Value)))
				.OrderBy(s => s)
		);
	}

	/// <summary>
	/// Send HTTP Request and return the response.
	/// </summary>
	async Task<string> SendRequest(string fullUrl, string oAuthHeader, FormUrlEncodedContent formData)
	{
		using (var http = new HttpClient())
		{
			http.DefaultRequestHeaders.Add("Authorization", oAuthHeader);

			var httpResp = await http.PostAsync(fullUrl, formData);
			var respBody = await httpResp.Content.ReadAsStringAsync();

			return respBody;
		}
	}
}

And to use it:

var twitter = new TwitterApi(ConsumerKey, ConsumerKeySecret, AccessToken, AccessTokenSecret);
var response = await twitter.Tweet("This is my first automated tweet!");
Console.WriteLine(response);

You’re free to do as you please with this code. If you improve anything significantly, do leave a comment!

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

HTML5 Pong in 20min with C#/Bridge.NET

2016-06-25T00:00:00+00:00

I recorded a short (20min) video of creating a simple Pong game in HTML5 using C#/Bridge.NET. It’s not the best Pong game in the world but it is from scratch with no libraries other than Bridge. Maybe useful if you’re thinking of picking up Bridge or building a simple Canvas game.

All source code for this project can be found on GitHub.

Edit: Geoffrey McGill from the Bridge team copied this code to deck.net so you can run it live and play around with tweaking the code! http://deck.net/59eed4385a7f4caee8a0232a63cf72f7

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

DaChip8JS - My C# Chip-8 Interpreter running in the browser

2016-06-12T00:00:00+00:00

Yesterday I blogged about building DaChip8, my C# Chip-8 interpreter (I recommend reading that post before this one; the changes detailed here will make more sense if you have an idea of the original code written). It works pretty well but running on the desktop is sooo 2010. It’s 2016, it’s Browser or Bust.

The problem with the browser is JavaScript. I know lots of you love it, but I don’t. Luckily we have many options for compiling other languages down to JavaScript and the one I happen to have been using recently at work is Bridge.NET. I thought it’d be fun to run my code through it and see what’s involved in making it run in the browser. It turned out to be much less effort than I expected and the main chip implementation is identical code to the C# version!

Live Demo

Before we go on, here it is. Running in the browser. It works for me in Chrome and Edge but not in IE (no ArrayBuffer support on XmlHttpRequest to fetch the ROM, and possibly more). It even works on Chrome for Android (with touch)! YMMV.

The ROM loaded below is Breakout (Brix hack) [David Winter, 1997] from the Chip-8 Pack.

You can use the cursor keys to move the paddle on a desktop or tap on the left/right half of the gamescreen on a mobile device.

Video

In case the demo doesn’t work in your browser, here’s a little video (taken well before it was complete, but my internet sucks too much to upload another one!):

Code Changes

Obviously since rendering on the web is very different to rendering in WinForms I had to make some changes. I also had to work with C# 5 for Bridge (for now). The changes are all external to the Chip8 class (which has hooks to call out for rendering the buffer to screen or playing a beep to keep it platform-agnostic). Below is a summary of the changes I had to make.

C# 5

Currently Bridge.NET only supports C# 5 so I had to make a couple of changes for it to compile.

Expression-bodied methods had to become normal methods:

// C# 6
void AddXToI(OpCodeData data) => I += V[data.X];

// C# 5
void AddXToI(OpCodeData data)
{
	I += V[data.X];
}

String interpolation had to be replaced with string.Format calls:

// C# 6
Debug.WriteLine($"Drawing {data.N}-line sprite from {I} at {startX}, {startY}"));

// C# 5
Debug.WriteLine(string.Format("Drawing {0}-line sprite from {1} at {2}, {3}", data.N, I, startX, startY));

Key and Touch Mappings

I had to tweak the key mappings (which previously used the .NET Framework’s Keys enum) to my own KeyCode enum which contains the keys required (I added cursor keys here to make the demo ROM better which was missing from the desktop version).

Game Loop Changes

In the C# version my game loop hooked the Idle event of the WinForm and used a timer to tell when it was time to call Tick and Tick60Hz. The Bridge version turned out to be much simpler using setInterval:

static void StartGameLoop()
{
	Window.SetInterval(chip8.Tick, targetElapsedTime);
	Window.SetInterval(chip8.Tick60Hz, targetElapsedTime60Hz);
}

Actually, it turned out to be slightly more complicated than that. setInterval has a minimum resolution of 4ms, so I had to tweak this slightly when I realised the game loop wasn’t running as fast as expected (which meant a slower game - Chip-8 games don’t account for their run speed).

// HTML5 has minimum resolution of 4ms 
// http://developer.mozilla.org/en/DOM/window.setTimeout#Minimum_delay_and_timeout_nesting
static readonly int minimumSetIntervalResolution = 4;

static void StartGameLoop()
{
	Window.SetInterval(Tick, minimumSetIntervalResolution);
	Window.SetInterval(chip8.Tick60Hz, targetElapsedTime60Hz);
}

static void Tick()
{
	var numTicksToExecute = minimumSetIntervalResolution / targetElapsedTime;
	for (var i = 0; i < numTicksToExecute; i++)
		chip8.Tick();
}

ROM Loading

In the C# version I just used File.ReadAllBytes to read the ROM from the disk. Obviously I can’t do that in the browser so I used XmlHttpRequest to fetch the ROM. I had to refactor the code a little to allow starting the game when that completed (the C# version loaded synchronously).

[Ready]
public static void OnReady()
{
	// ...

	// Kick off async loading of ROM.
	BeginLoadRom(ROM);
}

static void BeginLoadRom(string rom)
{
	var req = new XMLHttpRequest();
	req.ResponseType = XMLHttpRequestResponseType.ArrayBuffer;
	req.Open("GET", rom);
	req.OnLoad = e => EndLoadRom(GetResponseAsByteArray(req));
	req.Send();
}

// Convert the response to a byte[] as that's what our existing C# expects.
static byte[] GetResponseAsByteArray(XMLHttpRequest req)
{
	return new Uint8Array(req.Response as ArrayBuffer).As<byte[]>();
}

// After loading the ROM, only then is it valid to begin the game loop.
static void EndLoadRom(byte[] data)
{
	chip8.LoadProgram(data);

	StartGameLoop();
}

Rendering Changes

The constructor of my Chip8 class takes an Action<bool[,]> which gets called whenever the screen needs to be rendered. This made it easy to change the rendering code without making changes to Chip8. I used HTML5 canvas to render to:

<canvas id="screen" width="64" height="32" style="min-width: 640px; min-height: 320px;"></canvas>

Note the width/height on a canvas relate to the coordinates being used for drawing and not the actual size in the document. These settings make the canvas ten times bigger than the resolution of the Chip-8.

I did a bit of Googling to see what’s the fastest way to render pixels to a canvas and found a solution that involves creating ImageData for the pixel then rendering that at each location:

// Set up canvas rendering.
screen = Document.GetElementById<HTMLCanvasElement>("screen");
screenContext = screen.GetContext(CanvasTypes.CanvasContext2DType.CanvasRenderingContext2D);
lightPixel = screenContext.CreateImageData(1, 1);
lightPixel.Data[1] = 0x64; // Set green part (#006400)
lightPixel.Data[3] = 255; // Alpha

static void Draw(bool[,] buffer)
{
	var width = buffer.GetLength(0);
	var height = buffer.GetLength(1);

	// For performance, we only draw lit pixels so we need to clear the screen first.
	screenContext.ClearRect(0, 0, width, height);

	// Render each lit pixel.
	for (var x = 0; x < width; x++)
		for (var y = 0; y < height; y++)
			if (buffer[x, y])
				screenContext.PutImageData(lightPixel, x, y);
}

Sound Changes

In the WinForms version I used Console.Beep. As with the drawing, I pass an action into the Chip8 constructor for this (an Action<int> this time, withthe int being the duration). The simplest way to play a beep in JS seemed to be with an HTML5 Audio element with a data-url for the beep. I didn’t have control of the duration here, but I figured it’d work fine for now.

// Set up audio.
// http://stackoverflow.com/a/23395136/25124
beep = new HTMLAudioElement("data:audio/wav;base64,//uQRAAAAWM (trim a big long string)");

static void Beep(int milliseconds)
{
	beep.Play();
}

Unfortunately I came across an issue when testing this on mobile. You can’t play sounds that weren’t initiated by the user! As a workaround, I made the sound play when you tap to start the emulator.

Touch Support

Since a large portion of traffic to this blog comes from mobile (and I thought having an embedded demo would be pretty cool), I figured I should ensure it works there. I added some touch handlers to the canvas that just emulated pressing the cursor keys based on the side of the canvas that was touched:

// Set up touch events for canvas so we can play on mobile.
screen.OnTouchStart += SetTouchDown;
screen.OnTouchEnd += SetTouchUp;

static void SetTouchDown(TouchEvent<HTMLCanvasElement> e)
{
	if (e.Touches[0].ClientX < 320)
		chip8.KeyDown(keyMapping[KeyCode.LeftCursor]);
	else
		chip8.KeyDown(keyMapping[KeyCode.RightCursor]);
	e.PreventDefault();
}

static void SetTouchUp(TouchEvent<HTMLCanvasElement> e)
{
	chip8.KeyUp(keyMapping[KeyCode.LeftCursor]);
	chip8.KeyUp(keyMapping[KeyCode.RightCursor]);
	e.PreventDefault();
}

And that was pretty much it!

All source code for this project can be found on GitHub.

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Building a Chip-8 Interpreter in C#

2016-06-11T00:00:00+00:00

Edit: Using Bridge.NET, this interpreter can now run in your browser! Click here to see a live playable demo!.

A few weeks ago after watching a few episodes of Ferris Makes Emulators where Jake Taylor is live-streaming building an N64 emulator in Rust I decided it might be a fun project to try and build my own. Since I’ve never built an emulator before (and my knowledge of assembly is fairly weak) I thought the NES would be a good choice and although I want to learn some new languages I thought I’d start with C# and once I understand how to actually make an emulator I can build in something like Rust.

The current code for DaNES is on GitHub but to cut a long story short, I’m stuck on the rendering. While taking a little break from it I thought it might be wiser to try something simpler like Chip-8. Maybe coming back to the NES with a fresher pair of eyes will make things easier!

So, the new project is DaChip8 (also on GitHub). It’s working and seems to run all the ROMs I’ve tested fine :) Here’s some on how it works for any other newbies that want to give it a shot. Some of the code is a bit messy as I was “learning on the job” but the main implementation of the chip is pretty clean (I think).

Here’s a video of it in action:

Chip-8 isn’t a physical device like a NES so what we’re building is really an interpreter or VM. Because of this, multiple hardware platforms can(/did) support Chip-8 and games/apps don’t need to be re-written for different hardware. Unfortunately there doesn’t seem to be a defined clock speed for Chip 8 and games were not written to adapt so games will run faster on faster chips. We’ll have to artificially run ours slower because hardware today is so much faster (such as by controlling the rate at which we call Tick).

I implemented my chip as a class in C#. All registers are contained within it and it exposes two methods, Tick and Tick60Hz which are controlled externally. Tick is intended to be called at the clock speed (which I’m considering somewhere between 500Hz-1Mhz) and Tick60Hz needs to be called 60 times per second (this is because there are two registers for Delay and Sound which must decrement at this rate).

Here’s a list of the fields in my Chip8 class and what they are:

// Constants for screen size
const int ScreenWidth = 64, ScreenHeight = 32;

// A buffer containing a bool for each pixel on the screen (Chip-8 was monochrome)
bool[,] buffer = new bool[ScreenWidth, ScreenHeight];

// Actions that the chip should invoke (draw will be called from inside Tick60Hz, beep whenever we need to beep).
// These are passed in on the constuctor since they'll generally by implemented in some platform-specific way.
Action<bool[,]> draw;
Action<int> beep;

// Chip-8 has 16x 8-bit registers (referred to as V) which we'll store as byte[16].
// As with most things in this file, we'll use hex to refer to things, so these 
// are V[0x0] to V[0xF]. The last register, V[F] is also used for special purposes
// such as a carry and collision flag.
byte[] V = new byte[16];

// There are supposed to be two timers (Sound, Delay) but the easiest way for me to implement a beep
// was with Console.Beep(frequency, duration) so rather than have a Sound timer that counts down
// I just immediately start playing for the given duration. This might not be perfect (you can't
// cancel an in-progress sound) but I suspect in reality it works for all games.
byte Delay;

// The address register (referred to as I) is 16-bit and is used store memory addresses such as
// when rendering sprites.
ushort I;

// The program counter (where we're currently executing instructions from) is also 16-bit and
// This starts at the location 0x200 (512) because the first 512 bytes were used by the interpreter.
// We use 0x0 to 0x200 to store things like the built-in font data.
ushort PC = 0x200;

// The Chip-8 stack is only used to store program counter locations when jumping to subroutines.
// We use an array that can store 16 addresses and a single byte to index into it. Whenever jumping to
// a subroutine we will push the program counter (PC) to the stack (Stack[SP]) and increment SP
// (the stack pointer). When we need to return, we will decrement SP and then read the address to return
// to from Stack[SP].
byte SP;
ushort[] Stack = new ushort[16];

// The original Chip-8 systems had 4K (0x1000) of addressable memory. 
byte[] RAM = new byte[0x1000];

// To enable fast lookups of functions to execute for each opcode, we use a dictionary.
// Normally we use the first nibble (4 bits) of the two-byte instruction to decide what to do,
// however there's a bunch of random instructions all lumped into the 0xF000 nibble so we have
// an additional dictionary to look them up based on the second byte.
Dictionary<byte, Action<OpCodeData>> opCodes;
Dictionary<byte, Action<OpCodeData>> opCodesMisc;

// One of the instructions is to generate a random number, so we'll need this.
Random rnd = new Random();

// We also need to keep track of which keys are currently held down. Chip-8 had a 16-key keyboard
// much like a numeric pad with A-F around the edges:
//    1   2   3   C
//    4   5   6   D
//    7   8   9   E
//    A   0   B   F
// We store these in a HashSet to allow easy checking of whether a key is down.
HashSet<byte> pressedKeys = new HashSet<byte>();

For building and testing, I used ROMs from the Chip-8 Pack. As far as I can tell these are all games written by hobbyists and freely distributable/usable.

For loading a ROM, we simply read the bytes from the file and write them into RAM starting at 0x200 (as explained above):

public void LoadProgram(byte[] data) =>
    Array.Copy(data, 0, RAM, 0x200, data.Length);

The way opcodes are interpreted is generally based on the first nibble (4 bits). There’s a great table on Wikipedia which I won’t duplicate here, but here are a few examples:

1NNN    Jumps to address NNN.
5XY0    Skips the next instruction if VX equals VY.

Where you see N, X or Y in these instructions, those are placeholders. Other values (from 0 to F) are the bits being matched. The placeholder values also fall into the same pattern, so we can easily assign names to each part of the instruction:

X = second nibble
Y = third nibble
N = final (fourth) nibble
NN = second byte
NNN = last three nibbles

To make the opcode methods have the same signature, I wrapped this data up in a struct. Now every opcode instruction function just takes an OpCodeData which looks like this:

struct OpCodeData
{
	public ushort OpCode;
	public ushort NNN;
	public byte NN, X, Y, N;
}

Now we know how opcodes are laid out, we can build our dictionary based on the first nibble (and in the case of the misc instructions, the second byte):

opCodes = new Dictionary<byte, Action<OpCodeData>>
{
	{ 0x0, ClearOrReturn }, // There are two instructions starting with a 0 nibble, didn't seem worth its own dictionary
	{ 0x1, Jump },
	{ 0x2, CallSubroutine },
	{ 0x3, SkipIfXEqual },
	{ 0x4, SkipIfXNotEqual },
	{ 0x5, SkipIfXEqualY },
	{ 0x6, SetX },
	{ 0x7, AddX },
	{ 0x8, Arithmetic },
	{ 0x9, SkipIfXNotEqualY },
	{ 0xA, SetI },
	{ 0xB, JumpWithOffset },
	{ 0xC, Rnd },
	{ 0xD, DrawSprite },
	{ 0xE, SkipOnKey },
	{ 0xF, Misc }, // This will do a lookup from the second dictionary
};

opCodesMisc = new Dictionary<byte, Action<OpCodeData>>
{
	{ 0x07, SetXToDelay },
	{ 0x0A, WaitForKey },
	{ 0x15, SetDelay },
	{ 0x18, SetSound },
	{ 0x1E, AddXToI },
	{ 0x29, SetIForChar },
	{ 0x33, BinaryCodedDecimal },
	{ 0x55, SaveX },
	{ 0x65, LoadX },
};

Next we need to implement our Tick function which will read a two-byte opcode from RAM at the location the Program Counter (PC) points to then look up the instruction using the first nibble. When implementing this I found there were some instructions in the misc section that we don’t care about so I skipped over them (rather than crashing).

public void Tick()
{
	// Read the two bytes of OpCode. This is big-endian which means
	// the most significant byte comes first. We shift it 8 bytes to
	// the left then bitwise-or it with the next byte to get the full
	// 16-bit value.
	var opCode = (ushort)(RAM[PC++] << 8 | RAM[PC++]);

	// Write the PC and the OpCode we read for debugging.
	Debug.WriteLine((PC - 2).ToString("X4") + ": " + opCode.ToString("X4"));

	// Split data into the possible formats the instruction might need.
	// https://en.wikipedia.org/wiki/CHIP-8#Opcode_table
	var op = new OpCodeData()
	{
		// We use bitwise-and with a mask to extract specific nibbles.
		OpCode = opCode,
		NNN = (ushort)(opCode & 0x0FFF),
		NN = (byte)(opCode & 0x00FF),
		N = (byte)(opCode & 0x000F),
		X = (byte)((opCode & 0x0F00) >> 8), // Where don't use the lower nibbles, bitshift right to get just the raw value
		Y = (byte)((opCode & 0x00F0) >> 4), // Eg. we want 0x4 not 0x40
	};

	// Loop up the OpCode using the first nibble and execute.
	opCodes[(byte)(opCode >> 12)](op);
}

// Misc has its own dictionary because it's full of random stuff.
void Misc(OpCodeData data)
{
	if (opCodesMisc.ContainsKey(data.NN))
		opCodesMisc[data.NN](data);
}

Ok, so now we have a function we can tick at 500Hz and it’ll decode an OpCode and call the correct function. Next we need to implement all of those functions!

I won’t go through all of them here on the blog but I’ll cover the first five from my code. The rest you can find on GitHub.

The first instruction (where the first nibble is 0x0) is actually two completely different instructions, one to clear the screen and one to return from a subroutine. There’s no point having another dictionary for these two so we just use a simple if.

/// <summary>
/// Handles 0x0... which either clears the screen or returns from a subroutine.
/// </summary>
void ClearOrReturn(OpCodeData data)
{
	if (data.NN == 0xE0)
	{
		// Our screen buffer is just an array of bools, so set them all to false when
		// instructed to clear the screen.
		for (var x = 0; x < ScreenWidth; x++)
			for (var y = 0; y < ScreenHeight; y++)
				buffer[x, y] = false;
	}
	// Otherwise we're returning from a subroutine. Here we need to pop the address of the stack
	// and set the program counter to it (we'll cover Pop/Push implementations below).
	else if (data.NN == 0xEE)
		PC = Pop();
}

There are two simple jump instructions, 0x1000 and 0xB000. One jumps directly to the address from the lower three nibbles (NNN) and the other does the same but adds on the value of the first register (V[0]).

/// <summary>
/// Jumps to location nnn (not a subroutine, so old PC is not pushed to the stack).
/// </summary>
void Jump(OpCodeData data) =>
	PC = data.NNN;

/// <summary>
/// Jumps to location nnn + v[0] (not a subroutine, so old PC is not pushed to the stack).
/// </summary>
void JumpWithOffset(OpCodeData data) =>
	PC = (ushort)(data.NNN + V[0]);

There’s a third jump instruction which is a “go to subroutine”. This does the same as above but we need to be able to return back to where we were when it finishes executing (it will end with the return instruction covered above). In order to preserve the return address we push the Program Counter (PC) onto the stack before jumping.

/// <summary>
/// Jumps to subroutine nnn (unlike Jump, this pushes the previous PC to the stack to allow return).
/// </summary>
void CallSubroutine(OpCodeData data)
{
	Push(PC);
	PC = data.NNN;
}

The next OpCode from my code is one that skips the next instruction if V[x] is equal to nn (both x and nn are taken from the OpCodeData we decoded). This is how the code branches based on conditions (eg. the next instruction might be a go to subroutine).

/// <summary>
/// Skips the next instruction (two bytes) if V[x] == nn.
/// </summary>
void SkipIfXEqual(OpCodeData data)
{
	if (V[data.X] == data.NN)
		PC += 2;
}

Something that was much easier to implement than I expected was the stack. It only holds 16-bit addresses so the implementations of Pop and Push are incredibly simple:

/// <summary>
/// Pushes a 16-bit value onto the stack, incrementing the SP.
/// </summary>
void Push(ushort value) =>
	Stack[SP++] = value;

/// <summary>
/// Retrieves a 16-bit value from the stack, decrementing the SP.
/// </summary>
ushort Pop() =>
	Stack[--SP];

Once all instructions are implemented, we need to implement Tick60Hz. All this does is decrement the Delay counter and cause a redraw. The draw function is an Action<byte[,]> supplied in the constructor so it’s up to the consuming code to get that on the screen.

public void Tick60Hz()
{
	if (Delay > 0)
		Delay--;

	draw(buffer);
}

Finally, we need to something to Tick the chip and draw to the screen. I used a simple WinForms app with a PictureBox which calls Tick and Tick60Hz at the appropriate times and implements the Draw and Beep actions that are passed into the chip like this:

void Draw(bool[,] buffer)
{
	for (var x = 0; x < screen.Width; x++)
		for (var y = 0; y < screen.Height; y++)
			screen.SetPixel(x, y, buffer[x, y] ? Color.DarkGreen : Color.Black);
}

void Beep(int milliseconds) =>
	Console.Beep(500, milliseconds);

That’s pretty much all there is to it! In addition to the other instructions, there are a few things I didn’t cover here (such as the built-in font, which has sprites for the characters from 0x0 to 0xF) but you can find all the code on GitHub (you’ll need to download the ROMs as mention in the README to run).

If you’re thinking of building an emulator but unsure where to start, Chip-8 is a great first project. Pretty much everything I’ve done in this one has been the same for the NES emulator I started (with the exception of rendering) so it’s all useful practice.

Edit: Using Bridge.NET, this interpreter can now run in your browser! Click here to see a live playable demo!.

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Running Go on a Chromebook in Developer Mode (without installing Linux)

2016-05-29T00:00:00+00:00

For the last few years the computer I use most at home has been a Dell Chromebook 11. It’s great for browsing the web but not so great at being a developer machine. Although you can install Linux on it if you wish, that comes with an awful lot of baggage. To avoid using my desktop so much (it’s hard to lock myself away upstairs for hours with a family) I often find myself tryingto get new things running on the Chromebook that I can play around with from the sofa.

Previously it was the Dart VM and today it is Go. I actually tried Rust and it hasn’t worked out (yet) so Go was a fallback :D

Note: I’ve never written a line of Go in my life until about 30 minutes ago. This post might not be the best way of doing things and the instructions may become broken in future with Go or ChromeOS changes.

Setting up Go actually turned out to be pretty simple. Easier than Dart in fact!

When I did this for Dart, I ended up editing .bash_profile in vi (or using echo) and it was a bit of a clusterfuck. Subsequently I’ve installed Text and created a symlink at ~/Downloads/bash_profile to ~/.bash_profile so it can be edited in Text (ChromeOS file manager can only see files inside ~/Downloads and not hidden files, hence the missing dot).

Enable Developer Mode

Instructions on enabling developer mode can be found here. Every time you boot your device in this mode you will get a scary splash screen and have to press Ctrl+D to avoid resetting your device. WARNING: Enabling developer mode will delete all of your locally stored data (though since it’s a Chromebook, this is really only your Downloads folder).

Download and Extract Go

Download the Linux version of Go to your Downloads folder. I picked the 64 bit AMD version for my Dell Chromebook 11; some Chromebooks might require the 32 bit or ARM version.

Next, open a terminal window by pressing Ctrl+Alt+T then typing shell. Then extract Go somewhere suitable. I put it at ~/Coding/go/ (the archived already has everything inside a folder named go).

mkdir ~/Coding
tar -xzf ~/Downloads/go1.6.2.linux-amd64.tar.gz -C ~/Coding/

Set Execute Permissions for the File System

In order to execute anyting from this drive you’ll need to remount it as executable. You may have already done this if you’ve set other things up (such as the Dart VM). Without this, you’ll get Permission denied trying to execute anything.

echo "sudo mount -i -o remount,exec /home/chronos/user/" >> ~/.bash_profile

Note: In order for go run to work you will also need to remount /tmp as executable. go run compiles into a temp folder then executes from there. There may be security implications in doing this!

echo "sudo mount -i -o remount,exec /tmp/" >> ~/.bash_profile

Set Environment Variables

The easiet way to do this is with Text and a symlink to ~/.bash_profile as mentioned before. Otherwise you can try using vi (or echo)!

export GOROOT=~/Coding/go # This is where you extracted Go
export GOPATH=~/Downloads/go # This is where you will keep your Go projects
PATH=$PATH:$GOROOT/bin:$GOPATH/bin # This adds both Go and your "installed" projects to PATH

Restart and Test

Type exit or close the terminal window then re-open with Ctrl+Alt+T and type shell. Type go version and you should see go version go1.6.2 linux/amd64 or similar.

Hello, World!

Now we have a working Go, we should build Hello World!

There seems to be a convention that you should name your packages after the repo you’ll be hosting them in. They’ll need to hang off $GOPATH/src, eg:

mkdir $GOPATH/src/github.com/{username}/hello -p

Next create a hello.go file in that directory using your favourite editor with a simple Hello World:

package main

import "fmt"

func main() {
	fmt.Printf("Hello, World!\n")
}

Now we want to build and execute it. If we use go install instead of go build it will copy the executable into $GOPATH/bin which we added to our PATH earlier. We can either pass the package path (relative to $GOPATH) to go install or change to the directory and omit it:

cd $GOPATH/src/github.com/{username}/hello
go install
hello

If you chose to remount /tmp as executable then you can also use go run {file}:

cd $GOPATH/src/github.com/{username}/hello
go run hello.go

If you’ve done everything correctly, this will output Hello, World! :)

If you’re interested in a Chromebook; don’t forget to take a look at Chromebook Comparison Chart! :)

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Easy JavaScript and CSS bundling and minification on GitHub Pages without build steps

2016-05-08T00:00:00+00:00

Recently we were tidying up the design a little on chromebookchart.com and trying to make it work better on mobile. While setting up Google Analytics and Google Webmaster Tools I somehow found myself at Google PageSpeed and trying to improve our score there and in Google Chrome’s Audit tab.

Most of the issues resolved around external files. We had quite a few of them (thanks to Google MDL) and the caching and gzip wasn’t working on some (there’s something ironic about a Google project serving up uncompressed files to Chrome). To fix this I decided we should just download the Google Material Design Lite files locally (they don’t change, they’re versioned). CloudFlare lets me control the caching (and enables gzip) on these better than we were getting from Google. But, this meant more files being served from our domain and not bundling them together seemed a bit silly.

The site is hosted free using GitHub Pages (run through CloudFlare to get free HTTPS). We have a small C# app which rips data from the public spreadsheet and spits it out into Jekyll-formatted files for display but otherwis it’s entirely static. CloudFlare is already minifying the content (though in this case, this third party code is already pre-minified) so I just wanted the simplest possible way to bundle all the scripts together in one file that didn’t not require us to run any scripts whenever we changed a file (sometimes we tweak things directly through the GitHub web app on Chromebooks where it’s more coplicated to run scripts and push to GitHub).

Turned out, I already had the solution on this blog. My Google Analytics script is included using Jekyll includes like this:

{% include analytics.htm %}

So I thought I’d give it a try with JavaScript. I created a file named bundle.js inside my s/js folder and added the following:

{% include material.min.js %}
{% include jquery-1.11.1.min.js %}
{% include jquery.tablesorter.min.js %}
{% include jquery.tablesorter.widgets.js %}

I pushed to GitHub and navigated to /s/js/bundle.js in my browser. It came through completely untransformed. Doh.

I did some digging around to try and understand why this wouldn’t work and discovered (or rather re-remembered) that you need YAML front-matter for a page to be transformed in Jekyll. So I tried again:

---
---
{% include material.min.js %}
{% include jquery-1.11.1.min.js %}
{% include jquery.tablesorter.min.js %}
{% include jquery.tablesorter.widgets.js %}

After much refreshing, nothing changed. I was about to give up when I noticed an email from GitHub:

The page build failed with the following error:

A file was included in s/js/bundle.js that is a symlink or does not exist in your _includes directory. For more information, see > https://help.github.com/articles/page-build-failed-file-is-a-symlink.

Whoops! Includes normally live in _includes so I need to use include_relative:

---
---
{% include_relative material.min.js %}
{% include_relative jquery-1.11.1.min.js %}
{% include_relative jquery.tablesorter.min.js %}
{% include_relative jquery.tablesorter.widgets.js %}

After pushing this, everything came through as expected. Now all these files are combined into one file and CloudFlare is minifying them. One request to the same domain as the website is much better than four requests that include an additional uncommon domain.

Via CloudFlare, this bundle is set to cache in browsers for one month. This makes the site nice and fast but will also impact the ability to make changes. For this reason our own JavaScript (/js/main.js) is neither included in this bundle nor in the /s/ folder (anything in /s/ gets the 1-month-cache header).

There’s still lots of room for improvement (like the CSS) but it’s a good start and chromebookchart.com seems to load very quickly already!

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Have Software Developers Given Up?

2016-04-26T00:00:00+00:00

Edit: I’ve been finding so many software failures lately, I created @SoftwareFailed on Twitter to tweet and retweet the best failures!

Note: I’m a software developer. I create bugs. I once switched a production SQL database to Simple recovery mode and Truncated an important table causing a ton of work for my colleagues. The content of this post is aimed as much at me and the company I work for as the companies listed here. I think our industry has a real quality problem. I don’t entirely know what the solution is.

Over the last few years it feels like the quality of software and services across the industry is falling rather than climbing. Everything is always beta (both in name and quality). Things are shipped when marketing wants them to rather than when they’re ready because “we can easily patch them”. End users have basically become testers, but it’s ok, because this is Agile. We’ve started coding to expect failure and somehow with it decided that failure is normal and expected and we don’t need to put so much effort into avoiding it. Supporting millions of customers is complicated so we don’t bother. Why waste time reading bug reports from users when you can just send them into an endless maze of help links with no contact information?

I never used to be this grumpy. The last few years I’ve seen so many ridiculous errors in software and on websites that I just can’t help but feel a little embarassed about what we (as software devs) are unleashing on the world. I know we’re a young, inexperienced industry and that there aren’t enough skilled devs to go around but lately it feels like we’re really not even trying.

Here’s a collection of some screenshots I’ve taken just in the last month showing what I mean. Is it just me? Am I really unlucky? Or does this happen to everyone and it’s just me that likes to put effort into being vocal and annoyed by it?

It’s not unusual to see data that’s unescaped or badly encoded, but it’s not often you see things over-encoded. NPM package names appear to be HTML encoded twice! Let’s hope the first one wasn’t as it was saved into the database :/

While investigating an issue for a friend, I found a page on the ASUS site where the title shown in Google perfectly matched our problem. I clicked through and the page was blank, except for this form asking if it solved my problem.

IKEA emailed me reminding me I hadn’t completed a survey that I had. I figured I hadn’t clicked save. I clicked to complete it again and was told I’d already filled it in.

I got an email today from Coursera with an unreplaced replacement token in the subject.

Our company anti-virus for the second time in less than a month blocked access to most of the internet. Twitter was full of people having the same issue. I find this truly astonishing. Do they not install updates on a machine and give them a quick test before pushing out to the world? Large sites like the BBC and Google were blocked!

Not sure what to make of this message when shutting my Raspberry Pi down!

Tried to visit the HTC Vive website and got a MongoDB connection error spewed to the screen. Reported it on Twitter and quickly got a witty response; but it remained broken until the next day.

I tried setting up Ubuntu MATE on my Pi. I wrote their (Pi-specific) image to my SD card and this happened on first boot. Something wasn’t tested; whether it’s the image or some dependency being pulled from the web at first boot, I’m not sure.

I was trying to download IIS Log Parser today but the download didn’t seem to work. I opened the dev tools to find “jQuery not defined” errors all over the place. Persisted across refreshes.

I tried to move my gas and electric to NPower. They sent me a welcome email and when I clicked the button in it I got a big scary SSL warning from my browser. It turned out to be a wildcard certificate (*.npower.com) but the emails links point to https://npower.com/. I reported this to at least 5 different people there, yet nobody seemed to take it seriously (or even understand it, as shown here). I certainly wouldn’t want to show my new customers browser warnings that say attackers may be trying to steal their information!

One security issue on your site is bad enough; but come on, the live chat has a different certificate error!

Yesterday I tried to checkout with Paypal and after logging in I got this message. It’s bad enough to get an error during payment for something, but this message is particularly useless. There’s no information to help me. No support phone number, no indication of whether I was charged or what I should do next.

Many years ago, Microsoft took a lot of flak for podcasts only working on Windows Phone in the US. Fast forward a few years and it seems Google want in. I see no good reason for this; they’re free audio files. All of the other Google Play Music services work in the UK, so really, what’s the reason for this?

Podomatic helpfully prefix your iTunes URL with “http://”. This happens even if you already have “https://” (which, by the way, is the correct protocol for an iTunes link - HTTP gets redirected automatically). The only solution is to manually edit the URL to be HTTP before saving, even though that’s wrong.

This one is amazing. Someone signed up my email address to a service using the name “Pro_Hacking”. I reported it because I was concerned that maybe their service was not very secure and someone might end up with an account tied to my email. It’s hard to believe the support person was even reading the emails the number of times I explained “Pro_Hacking” is not my name, to get a response starting “Hello Pro-Hacking”!

The move of our gas and electric to NPower didn’t work out. It turns out that for some “mysterious technical reason” they are unable to take over supplies from the company that (physically) supply our gas. So we arranged to move to SWALEC. Weeks on, we’ve still had no code mailed to us and are therefore unable to log in to their website. The contact link on the only page we can access goes to an error page.

This website accused me of using an ad blocker and wouldn’t let me read this article. Quite clearly from the screenshot, I am not using an AdBlocker. I completely appreciate the need to monetise content (which is why I’ve been putting off installing an Ad Blocker) but if sites are going to add code like this it should fail-safe. Accusing me of stealing from your children when I’m innocent is just going to drive me away!

I tried setting up VNC on my Pi. It’s behind a firewall so I entered a short password. I was told it was too short, so I added a few characters. I was told it was truncated to EIGHT characters.

When installing a Visual Studio update, it told me the Visual Studio Improvement Program was optional. I’m pretty sure I opted-out at install time; however this time it was both ticked and disabled (yet still sporting the “Optional” tag).

Immediately after installing Visual Studio 2015 Update 2 (which claimed to fix a bunch of stability issues), I launched Visual Studio and it just crashed. I don’t believe I have any non-Microsoft extensions installed.

Every few weeks without me doing anything, my Moto 360 Android Wear watch will just randomly start draining it’s battery until it’s flat/rebooted. I reported this to Google once and was told “this is expected behaviour after updates”. I hadn’t had any updates, but even so, that would be stupid.

While preparing this post I went looking for a command line tool to optimise the images in this post. I came across this post and every time I tried to scroll down the page, it immediately jumped back up to the top. After about 40 seconds a full-page ad appeared, which appears to have been the cause. sigh.

That’s a lot of fail for just one month, and I’m sure there’s a lot more I didn’t have screenshots of. I can’t help but feel that as an industry we’re just not doing our best for our users. Even companies that have previously been known for exceptional quality and testing seem to have gone down the shitter. I’m no stranger to commercial pressures causing things to be shipped before they’re done, but surely there’s room for improvement?

Or maybe it’s the end users fault? Maybe we don’t complain enough when things are bad, so companies have little motivation to improve?

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Removing [action 'action 17' suspended] rsyslog Spam on Raspberry Pi (Raspian Jessie)

2016-04-23T00:00:00+00:00

As a bit of a Linux noob, I’ve been keeping an eye on the logs for my Raspberry Pi to ensure I haven’t set anything up wrong. I noticed that quite frequently my syslog contains this sort of junk:

Apr  2 01:23:51 raspberrypi rsyslogd-2007: action 'action 17' suspended, next retry is Sat Apr  2 01:24:21 2016 [try http://www.rsyslog.com/e/2007 ]
Apr  2 01:29:41 raspberrypi rsyslogd-2007: action 'action 17' suspended, next retry is Sat Apr  2 01:30:11 2016 [try http://www.rsyslog.com/e/2007 ]
Apr  2 01:31:35 raspberrypi rsyslogd-2007: action 'action 17' suspended, next retry is Sat Apr  2 01:32:05 2016 [try http://www.rsyslog.com/e/2007 ]
Apr  2 01:35:52 raspberrypi rsyslogd-2007: action 'action 17' suspended, next retry is Sat Apr  2 01:36:22 2016 [try http://www.rsyslog.com/e/2007 ]
Apr  2 01:39:06 raspberrypi rsyslogd-2007: action 'action 17' suspended, next retry is Sat Apr  2 01:40:06 2016 [try http://www.rsyslog.com/e/2007 ]
Apr  2 01:44:39 raspberrypi rsyslogd-2007: action 'action 17' suspended, next retry is Sat Apr  2 01:45:39 2016 [try http://www.rsyslog.com/e/2007 ]
Apr  2 01:53:08 raspberrypi rsyslogd-2007: action 'action 17' suspended, next retry is Sat Apr  2 01:54:08 2016 [try http://www.rsyslog.com/e/2007 ]
Apr  2 01:59:41 raspberrypi rsyslogd-2007: action 'action 17' suspended, next retry is Sat Apr  2 02:00:41 2016 [try http://www.rsyslog.com/e/2007 ]

As well as being annoying when scanning through logs, it also doesn’t really help the life of the SD card on a Pi if things are churning data out needlessly. It turns out to be caused by rsyslog trying to write to /dev/xconsole and for whatever reason, failing. The rsyslog.conf file in Raspbian Jessie contains this at the end:

# The named pipe /dev/xconsole is for the `xconsole' utility.  To use it,
# you must invoke `xconsole' with the `-file' option:
# 
#    $ xconsole -file /dev/xconsole [...]
#
# NOTE: adjust the list below, or you'll go crazy if you have a reasonably
#      busy site..
#
daemon.*;mail.*;\
    news.err;\
    *.=debug;*.=info;\
    *.=notice;*.=warn   |/dev/xconsole

The fix is simple enough - just remove those last lines. I’ve been trying to script all of the setup of my Pi so I can easily rebuild it if (when) I trash things or to try things out on a new SD card.

After much Googling, I settled on this command to do it for me:

sudo sed -i '/# The named pipe \/dev\/xconsole/,$d' /etc/rsyslog.conf

This says match all lines starting with the one that matches # The named pipe /dev/console and ending with the last line ($) and delete them. If you have any additional lines at the end of your file you’ll need to tweak this, but that seems unlikely (most things are likely to write into /etc/rsyslog.d/. I don’t know if you need to restart rsyslog after this; I rebooted just for fun.

Hope this is helpful. If you have any problems, leave a comment. Bear in mind I’m a Linux noob and what’s written above might not be the best way to achieve this and I take no responsibility if anything breaks :)

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Setting up Automatic Updates on Raspberry Pi (Raspian Jessie)

2016-04-09T00:00:00+00:00

Logging into machines and installing security updates periodically isn’t fun but for internet-exposed devices it’s important. Any device that’s on your home network has the possibility of being a stepping stone for attackers if it can be easily breached.

The first thing to do before setting up automatic updates is to ensure your Raspberry Pi can send email. You’ll want to know when updates are being installed (or if they fail). My previous blog post covers how to do this with Postfix.

As always, start off by making sure your apt list and existing packages are up-to-date:

# Update the package list, update all packages and remove any packages that are no longer required
sudo apt-get update -y && sudo apt-get dist-upgrade -y && sudo apt-get autoremove -y

Next we need to install the unattended-upgrades package and to ensure it sends emails the apt-listchanges package. apt-listchanges also requires a mailx program so if you don’t already have one you can grab bsd-mailx:

sudo apt-get install unattended-upgrades apt-listchanges bsd-mailx -y

Next we should configure where the updates are allowed to come from.If you choose to stick with Stable then when the next version of Raspbian goes stable (Stretch) it’ll automatically update. I’ve decided to stick with Jessie for now. This config lives in /etc/apt/apt.conf.d/50unattended-upgrades and you can use this script to uncomment the line for Jessie.

sudo sed -i 's/^\/\/      "o=Raspbian,n=jessie"/      "o=Raspbian,n=jessie"/g' /etc/apt/apt.conf.d/50unattended-upgrades

Next we want to instruct the updater to send emails. Again, this is already in the config file so it’s just a case of uncommenting it (you may wish to tweak the user, but I’ve already set root mail to be forwarded on to my user).

sudo sed -i 's/^\/\/Unattended-Upgrade::Mail "root";/Unattended-Upgrade::Mail "root";/g' /etc/apt/apt.conf.d/50unattended-upgrades

By default your Pi won’t be rebooted if required, so if you want it to (and want to set the time) you can do that like this:

sudo sed -i 's/^\/\/Unattended-Upgrade::Automatic-Reboot "false";/Unattended-Upgrade::Automatic-Reboot "true";/g' /etc/apt/apt.conf.d/50unattended-upgrades
sudo sed -i 's/^\/\/Unattended-Upgrade::Automatic-Reboot-Time "02:00";/Unattended-Upgrade::Automatic-Reboot-Time "02:00";/g' /etc/apt/apt.conf.d/50unattended-upgrades

And if if you want unused packages to be removed (like when you run apt-get autoremove:

sudo sed -i 's/^\/\/Unattended-Upgrade::Remove-Unused-Dependencies "false";/Unattended-Upgrade::Remove-Unused-Dependencies "true";/g' /etc/apt/apt.conf.d/50unattended-upgrades

Next we must create the /etc/apt/apt.conf.d/20auto-upgrades file to instruct the updater what to do:

# You could also create this file by running "dpkg-reconfigure -plow unattended-upgrades"
sudo tee /etc/apt/apt.conf.d/20auto-upgrades > /dev/null <<EOF
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";
EOF

And that’s all there is to it! Every day your Pi will now check for updates and you’ll receive an email like this if there were:

If you want to chek it’s working, you can check the log file tomorrow:

cat /var/log/unattended-upgrades/unattended-upgrades.log

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Setting up a Raspberry Pi (Raspian Jessie) to send Email

2016-04-08T00:00:00+00:00

There are a bunch of things on Linux that send emails by default (for example, the output of Cron jobs). Being a Linux noob I wanted to ensure I received these emails (what if something goes bad because of my noobness?) and also wanted to ensure my Raspberry Pi can send emails to me from scripts if required in the future.

There are a lot of tutorials online about how to do this most of which require hard-coding your email account password on your Pi. I’m not ready for this sort of commitment (even if it’s an App-Specific password) and knowing a little about SMTP I was fairly sure it wasn’t required if I was only deliverying mail to one place.

So, the things we need to do are:

Set up all mail to be passed on to the same local account (in my case danny)
Set up that accounts mail to forward to an external address (a real email address, mine being hosted by Google)
Install and configure an MTA that can actually connect to the external SMTP server and deliver the email

All of these seemed pretty straight forward, until I actually came to try… It turns out that if you’re deliverying email to Google and your ISP supports IPv6 (mine does) and you don’t want to disable IPv6 (I do not) then you will find yourself receiving errors like this:

Our system has detected that this message does not meet IPv6 sending guidelines regarding PTR records and authentication. Please review https://support.google.com/mail/?p=ipv6_authentication_error for more information.

For some reason Google have added some extra restrictions on being able to talk to their SMTP servers over IPv6, some of which you likely can’t solve (because the IP address is owned by your ISP and not you). It’s not just me that thinks this sucks.

So, we have an additional requirement:

Our MTA must be able to deliver only over IPv4, despite IPv6 being set up and configured

I set about testing a whole bunch of MTAs and I failed at gtting almost all of them working (mostly because of this IPv6 issue). Possibly this is because of lack of experience for Linux but whatever the reason, I needed something I could make work! Eventually I got PostFix working (or at least, I thought I did… keep reading!).

The first thing I do when setting up a new Raspberry Pi is ensure everything is up-to-date. If your Raspbian image is quite old there might be many updates for you to install. This might take a little while:

# Update the package list, update all packages and remove any packages that are no longer required
sudo apt-get update -y && sudo apt-get dist-upgrade -y && sudo apt-get autoremove -y

Next we need to install and configure Postfix. When you install this it will automatically open a config utility (I don’t think it should, because of -y but it does…). Just select No Config and OK since we’re going to write the entire config file ourselves.

sudo apt-get install postfix -y

Now to write the Postfix config file. We need to set a couple of things; descriptions are inline in comments:

sudo tee /etc/postfix/main.cf > /dev/null <<EOF

# Where to read account aliases, used to map all emails onto one account
# and then on to a real email address
alias_maps = hash:/etc/aliases

# This sets the hostname, which will be used for outgoing email
myhostname = pi.dantup.com

# This is the mailserver to connect to deliver email
# NOTE: This must be the MX server for the account you wish to deliver email to
# or an open relay (but you hopefully won't find one of them). In my case, this
# is Google's first MX server (which can be found by doing an MX lookup on my domain).
relayhost = aspmx.l.google.com

# Which interfaces to listen on. We don't want anyone connected to our Pi to send email,
# so we set this to the local loopback interface only.
inet_interfaces = loopback-only

# This one is important for the reasons mentioned above. This means only IPv4 will be used,
# avoiding the IPv6 restrictions Google have in place.
inet_protocols = ipv4

EOF

Now we need to create the aliases file referenced above. We simply map root mail on to our main user (danny) and then that user on to the required email address.

# Set the aliases
echo "root: danny" | sudo tee -a /etc/aliases
echo "danny: danny+pi@notreallymydomain.com" | sudo tee -a /etc/aliases

# Rebuild alias db and restart Postfix
sudo newaliases
sudo /etc/init.d/postfix start

And that’s that. Simples! Next we need to send a test email to ensure everything is working. We send this to a local account (not a full email address) to ensure we’re testing the whole thing.

echo -e "FROM: root\nTO: root\nSubject: Test email from the Raspberry Pi\n\nThis is a test email sent from your Raspberry Pi" | sendmail -t
echo -e "\e[33mTest email sent. Make sure it turns up :)\e[0m"

If you’ve done everything right and Google (or your mailhost) don’t suspect you of being a spammer (which is possible if you’re connecting from a domestic IP address, but so far I’ve had no issues) you should have an email in your mailbox. For extra profit, add the sender to your address book and add a nice profile picture :)

For extra points (and to reduce the chance of being labelled a spammer), you probably want to set up SPF records for your home IP address. If this changes a lot, you could use a dynamic DNS service. For example, my SPF record looks like this:

v=spf1 include:_spf.google.com a:home.dantup.com ~all

This will allow Google and the IP that home.dantup.com resolves to to send mail and it be considered an SPF pass.

Not so fast…

At this point I thought everything was dandy… Until I rebooted! If you’re using Raspbian Jessie (but not Raspbian Jessie Lite) you’ll find that after a reboot that your email is broken. This reason for this is that Raspbian Jessie defaults to a Fast Boot mode where it does not wait for the network during booting. Postfixes creates a chroot jail and copies /etc/resolv.conf into it at startup, but in fast boot mode this is too early and it ends up blank. This means Postfix is unable to resolve DNS and no emails get sent!

It took me several days to track down what was happening here but the fix is as simple as running sudo raspi-config and selecting Slow Boot, or running the following command:

sudo raspi-config nonint do_wait_for_network Slow

This will create a script that runs at startup which blocks until DHCP completes before allowing other services (like Postfix) to start up. I posted my frustration with this default on GitHub and was informed that neither way was great, because Slow Boot would wait for a DHCP timeout (which could be quite long) if a Pi booted and wasn’t connected to the network.

It’s worth noting that in Raspbian Jessie Lite the defaut is the Slow Boot (yay!) and therefore you don’t have to worry about this step.

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Installing Lighttpd, PHP 7 and LetsEncrypt on a Raspberry Pi (Raspbian Jessie Lite)

2016-03-04T00:00:00+00:00

I have a friend who wanted to serve some simple PHP scripts over HTTPS from a Raspberry Pi. He’d seen some benchmarks showing that Apache was a bit of a hog so was trying to use Nginx but was having trouble because none of these apps are in the Raspbian repos.

I’m a total noob when it comes to Linux (I got my first Pi a week or so ago), but it didn’t seem like this should be a complicated setup, so I decided to see if I could quickly get it working. I have plenty of SD cards lying around and my Pi wasn’t doing anything important so there’s nothing to lose.

Goals

The main goal for me was to take a new install of Raspbian (I’m using Raspbian Jessie Lite, but standard Raspbian via NOOBS should be fine too) and render a PHP “Hello World” over HTTPS with a real cert trusted by browsers with minimal effort, while:

Preferring NGINX or Lighttpd to Apache
Preferring PHP 7 to anything older
Getting free TLS certs from LetsEncrypt
Scheduling renewal of TLS certs via cron

Replacements

If you’re going to use any of these scripts, there are two things littered throughout that you must replace with your own values:

/var/www/html is the webroot. I’m using the standard location Lighttpd (and Apache!) use out-of-the-box.
pi.dantup.com is the public hostname for the site/cert. This is mine; replace it with your own!

Complications

The LetsEncrypt client is not available in the Raspbian Jessie repositories (nor debian Jessie)
PHP 7 is not available in the Raspbian Jessie repositories (nor Debian Jessie or Jessie Backports)

Getting Raspbian

Step 1 is to get a working Raspbian install. I chose to use Raspbian Jessie Lite from here because I tend to run my Pis headless. These instructions should all be the same for standard Raspbian (inc. via NOOBS).

Security Considerations

If you’re going to expose your Pi (or any other computer) to the web, you should consider the security. Someone breaching a device on your network could give them access to other devices inside your network! Some things you might wish to consider doing to your Pi if you’re going to expose it (especially if opening SSH):

Create your own user and ditch the Pi user (at a minimum, change the password to something strong!)
Disable root login over SSH
Change the SSH port
Set up unattended security upgrades

I’ve been working on a script that does this (and more) for me when I set up a new Pi. I’m hoping to finish/tidy/blog it over the coming weeks.

Setting up Raspbian

First thing I do when setting up Raspbian is expand the filesystem (you won’t need to do this if using NOOBS, but will if you’ve written a Raspbian .img) and ensure all packages are up-to-date.

# Expand the filesystem and reboot
sudo raspi-config --expand-rootfs
sudo reboot

# Update lists, perform upgrades, remove orphaned packages
sudo apt-get update -y && sudo apt-get dist-upgrade -y && sudo apt-get autoremove -y

Switch to root shell

Pretty much everything here requires root, so it’s easiest to switch to a root shell to avoid prefixing everything with sudo.

sudo -i

Installing Lighttpd

This one is pretty painless. Install via apt and replace the default page with our own Hello World.

apt install lighttpd -y
rm /var/www/html/index.lighttpd.html
echo '<h1>Hello, World!</h1>' > /var/www/html/index.html

At this point you should be able to visit http://raspberrypi/ (assuming you haven’t changed the hostname) and see your Hello World message. It’s a good start, but it’s not HTTPS!

Installing packages from Debian

Because neither PHP 7 nor the LetsEncrypt client are in the Raspbian repos, we need to fetch them from the Debian ones. Before apt will download things from there, we need to trust them. You should probably confirm these keys (such as by skipping this step, letting the next step fail, and then getting the keys from the error message) rather than copy/paste them from some random guys blog (which wasn’t served over HTTPS) though!

The error you’ll get prior to trusting these keys looks like this:

W: GPG error: http://http.debian.net jessie-backports InRelease: The following
signatures couldn't be verified because the public key is not available:
    NO_PUBKEY 8B48AD6246925553 NO_PUBKEY 7638D0442B90D010

I don’t know why there are two. Did I mention I’m a Linux noob?

gpg --keyserver pgpkeys.mit.edu --recv-key  8B48AD6246925553      
gpg -a --export 8B48AD6246925553 | apt-key add -
gpg --keyserver pgpkeys.mit.edu --recv-key 7638D0442B90D010      
gpg -a --export 7638D0442B90D010 | apt-key add -

Install the LetsEncrypt Client

The LetsEncrypt client is available from Jessie Backports. We’ll need to add the source to be able to install it from there. My script removes the source at the end (and re-updates the packages; I don’t know if that’s required) since we only want to borrow these packages and not use this for anything more.

echo "deb http://httpredir.debian.org/debian jessie-backports main contrib non-free" > /etc/apt/sources.list.d/debian-jessie-backports.list
apt-get update -y
apt install letsencrypt -t jessie-backports -y
rm /etc/apt/sources.list.d/debian-jessie-backports.list
apt-get update -y

Requesting TLS Certs

In order for the LetsEncrypt client to work, it needs to be able to connect to your Pi using the hostname you want the cert for (this is to verify you actually own this domain, or at least, control the web server it points at). This may mean setting up DNS records to point the domain at your Pi and/or forwarding ports from a router (you’ll need port 80 for this step, but may as well also map 443 while you’re at it to save coming back to it later).

Because there’s no automatic module to set up Lighttpd we need to use the --webroot method (this means the client will write files into your webroot and LetsEncrypt will connect back and read them). Pass each webroot with -w and the domain with -d. You can add multiple sets of these as required.

Remember to replace the webroot and domain with your own!

letsencrypt certonly --webroot -w /var/www/html -d pi.dantup.com

Setting up TLS in Lighttpd

Lighttpd expects certs to be combined, so we need to concatonate them before we can configure it. Remember to replace your domain in the path (note: this is in /etc/letsencrypt and not your webroot!).

pushd /etc/letsencrypt/live/pi.dantup.com/
cat privkey.pem cert.pem > combined.pem
popd

Next we need to add TLS config for Lighttpd. We need to point at the cert we just combined as well as the full chain certificate that will be served up to the client browser. We also disable SSLv2 and SSLv3 for security.

Again, be sure to replace your domain in the certificate paths.

tee /etc/lighttpd/conf-enabled/letsencrypt.conf > /dev/null <<EOF
$SERVER["socket"] == ":443" {
        ssl.engine = "enable"
        ssl.pemfile = "/etc/letsencrypt/live/pi.dantup.com/combined.pem"
        ssl.ca-file =  "/etc/letsencrypt/live/pi.dantup.com/fullchain.pem"
        ssl.cipher-list = "ECDHE-RSA-AES256-SHA384:AES256-SHA256:HIGH:!MD5:!aNULL:!EDH:!AESGCM"
        ssl.honor-cipher-order = "enable"
        ssl.use-sslv2 = "disable"
        ssl.use-sslv3 = "disable"
}
EOF
/etc/init.d/lighttpd force-reload

At this point, you should now be able to visit https://raspberrypi/ (assuming you haven’t changed the hostname) and see your Hello World message again. You’ll get a warning about the cert name not matching (since you’re not accessing it via the real domain), but that’s expected. Accessing it via the real domain may work depending on your setup. Hurrah! Encryption!

Installing PHP 7

Like the LetsEncrypt client, PHP 7 is not available in the Raspbian repos. Nor is it available in Debian Jessie Backports. However it is available in Stretch (the next version after Jessie),so we can do a similar thing and get it from there.

echo "deb http://httpredir.debian.org/debian stretch main contrib non-free" | tee /etc/apt/sources.list.d/debian-stretch.list
apt-get update -y
apt install php7.0 php7.0-fpm -t stretch -y
rm /etc/apt/sources.list.d/debian-stretch.list
apt-get update -y

Next we need to enable fastcgi and tell Lighttpd where to find PHP.

tee /etc/lighttpd/conf-enabled/php.conf > /dev/null <<EOF
fastcgi.server += (".php" => ((
        "socket" => "/var/run/php/php7.0-fpm.sock"
)))
EOF
lighttpd-enable-mod fastcgi
/etc/init.d/lighttpd force-reload

Finally, let’s replace our boring old Hello World with a nice PHP one!

rm /var/www/html/index.html
echo '<?php echo "<h1>Hello, World (from PHP)!</h1>"; ?> ' > /var/www/html/index.php

If you’ve done all this right, https://raspberrypi/ (assuming you haven’t changed the hostname) will now greet you from PHP! Almost done…

Automatic TLS Cert Renewal

Not renewing your certificates would be pretty embarassing, so we should make that automatic. We’ll do this via Cron. Cron sends emails with the output of commands, so if you’ve set up emails to be forwarded to a real mailbox (this will be covered in my Pi setup script which I hope to blog in the coming weeks) you’ll get a note each time this runs. We’ll set it to run weekly since by default it only renews certs that’ll expire in the next 30 days, so monthly could cause them to be missed.

Edit: Mike Scalora posted a better version of this that handles multiple domains neatly in the comments :)

tee /etc/cron.weekly/letsencrypt > /dev/null <<EOF
# Renew cert
letsencrypt renew

# Rebuild the cert
pushd /etc/letsencrypt/live/pi.dantup.com/
cat privkey.pem cert.pem > combined.pem
popd

# Reload
/etc/init.d/lighttpd force-reload
EOF
chmod +x /etc/cron.weekly/letsencrypt

That’s It!

I believe that’s everything! If you hit problems please post in the comments and I’ll try to fix them up. Please note that I’m a Linux noob, some (or all) of this might not be done in the best possible way. Again, I’ll make tweaks if people send corrections/improvements :-)

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

My Week in a Tesla Model S P85

2015-12-15T00:00:00+00:00

9 months ago my wife and I took delivery of two Renault ZOEs. We hadn’t had EVs before but after test driving a ZOE and being offered a great deal on two we found it hard to turn down! Although the range of the ZOE is only 70-100 miles (depending on weather, etc.) we’ve had no problems using the ZOE for a summer holiday racking up over a thousand miles in a week, including two 270 mile trips to/from the holiday park. Neither of us would willingly go back to combustion cars.

In August, Chargemaster Plc (who made the chargepoint we have at home and also run the POLAR charging network) launched a new scheme called “POLAR Plus” which rewarded customers for charging on their network with points which could be used to “bid” for a weeks use of one of their fleet of electric vehicles (including a Tesla Model S and a BMW i8!). The number of points needed to loan the Tesla Model S was only 70 (which is just 7 qualifying charge sessions). I was sceptical but they assured me all was as it seemed. The scheme costs £7.85 per month but the first six months are free.

Fast forward to November and it happened, my week in Chargemaster’s Tesla Model S P85 was confirmed, 8th - 14th December! At this time I’m still in my free 6-month trial so I’ve paid £0 to Chargemaster! :)

I picked up the car from Chargemaster’s head office in Luton on Tuesday 8th Dec. At more than twice the price of the most expensive car I’ve owned I was expecting it to be great but it still seriously exceeded my expectations. I wanted one from the minute I sat inside it! Evie (an apt name) from Chargemaster ran me through some paperwork and the basics of the Model S and we were soon on our way home. It should be a 3-4hr journey back to where we live in Cheshire but with terrible traffic and a stop at Warrington to try out the Supercharger it took approximately 5hrs driving (mostly in the dark and rain) to get home.

Exterior

I’d already seen pictures/videos of the Model S but it looked much better than I expected up-close. If you’re used to spending this sort of money on a car you may be used to having cars that look this nice but sadly, I am not! It looked stunning and it was pretty huge (especially compared to the ZOE which is quite tiny).

Performance

This Model S was a P85. This is the rear-wheel-drive “performance” version (the all-wheel-drive performance version is the P85D). It’s not available in this configuration anymore but I believe the 0-60 time is something like 4.1 seconds. To me, this qualifies as “stupidly fast”. For various reasons (public roads, very wet week, it’s not my car and being somewhat scared of the performance I did experience) I didn’t get to experience how fast this thing would accelerate foot-to-the-floor but it was quite clear from just a little bit of “fun” I had that the power in this car is utterly ridiculous. I’ve never driven a RWD car before and feeling the back end twitching in the wet is scary. If I somehow ended up with the money to buy one of these I think I’d opt for the “D” version just to avoid that danger! :D

Interior

Inside, the Model S feels expensive. With a few exceptions (the plastic on the front of the arm rest rattled a little and felt a bit cheap) everything feels very premium and well-made. There’s nice stitched leather around the dash and the leather seats are very comfortable. As there’s no handbrake/gear-selector between the front seats, everything is very open and spacious. The back easily accommodates adults with decent legroom and was big enough that the kids couldn’t reach to kick the back of the seat. Bonus!

Software/Touchscreen

The thing that is always a huge disappointment in cars I’ve owned is the software. Every time I get a new car I think “software must’ve moved on, it’ll be good now”, but no. The Model S bucks this trend and has excellent software on an enormous 17” touch screen front-centre. Rather than hardware dials/knobs that can’t be changed (and can break) the Model S does almost everything in software from this same touchscreen. My wife initially said the touchscreen was “too big” but I think she was coming around to it by the end of the week. It has Google Maps-based navigation, the reversing camera, calendar, Bluetooth/music/radio, consumption graphs and everything else you can think of all together in one place. All the settings you might like to tweak (regen, creep, suspension/ride height, sunroof, charge timer, automatic lights etc.) are all controlled here. Whereas many cars feel like a Frankenstein of disparate systems “integrated” together, everything here feels like it was built as one.

The excellent software extends to the display behind the steering wheel which shows a small version of the navigation map (when navigating) so you can keep your eyes directly ahead.

Boot/Frunk Storage

Although it’s not very deep, the boot is rather large and easily fit our pushchair, coats and plenty of shopping. There’s also a really decent amount of storage under the bonnet (“frunk”/”front trunk”) that could easily fit all of our shopping in it and would be less likely to roll around than in the boot.

Regen Braking

When I picked the car up the regen braking was set to “Low”. I switched it to “Standard” as I was quite used to regen braking in the ZOE and liked minimising the use of the brake pedal. I was surprised at just how aggressive “Standard” was and found myself having to put my foot back on the gas after lifting off many times while I got used to it. Once I got the hang of it, it was absolutely brilliant - I previously thought I didn’t use the brake pedal much in the ZOE but this was just miles better. One-pedal-driving is great!

On one of the days it was around 5 degrees Celsius. I noticed the regen braking wasn’t anywhere near as effective (it caught me by surprise) and I noticed the speedo showed a dotted line in the regen area which indicates that it can’t regen/charge at the normal rate. It was a little weird that the cold weather was changing the driving experience so much but after just a little driving/braking it seemed to have warmed up enough for this to go away and things were back to normal.

Gadgets

The Model S comes with a bunch of nice things/gadgets, many of which aren’t unique to the Model S (or even EVs) but they still added to the experience.

With the exception of the rear view mirror all adjustments are electric (seats, wing mirrors and even steering wheel adjustment) and saved against your driver profile. If someone else drives the car, selecting your profile afterwards will move everything back to how you had it.

The wing mirrors automatically angle downwards when you shift into reverse to help you see white lines or parking spaces on the road (and return when you shift out of reverse).

The reversing camera is far higher definition than the car in our ZOE and shows on the huge screen making it really easy to reverse-park (which is super useful because forwards parking is tricky when you don’t really know how big the car is and are unable to see the front corners!).

Front seats are heated. I’ve never cared much for these in the past but with the cold weather they were really nice! Something small but I liked, was that when you turned them on they defaulted to the highest setting and reduced as you continue to tap the button. This matches how you’d use them pretty well (it’s cold, put them on full then reduce over time) so you don’t have to keep tapping to increase the heat when you turn them on. Unfortunately there was no heated steering wheel to go with it and the wheel was often quite cold so I found myself driving around all toasty and warm but with cold hands!

The key for the car is a miniature Model S. Pressing the front opens the frunk, the back opens the boot and the middle opens the car. It’s a nice touch and looks pretty cool.

When I was returning the car I entered the business park and the car showed a message along the lines of “Increasing ride height based on location”. I don’t know how this had been set up, but due to speed bumps the ride height was increased!

Negatives

There really aren’t many things we didn’t like about the Model S, it really is an excellent car. There are a couple of niggles or things I think could be improved though:

Chargeflap seemed to stick a lot; took lots of locking/unlocking before it would open (even the button on the screen didn’t seem to work). On the way back to Luton I had to give up trying to top it up at the services because of this (though unsurprisingly, it appeared to be fine when we arrived at Chargemaster HQ!).
The plastic at the front of the arm rest rattled a little which made it feel cheap. I don’t know if they all do this or it was just this car.
Can’t pre-heat the car from the fob (the ZOE can do this), so if you’re going out unexpectedly you might have a cold car. It’s possible the app allows you to do this but sadly we didn’t have access to that (it requires a username/password we didn’t have).
Can’t set a pre-heat timer within the car (again, ZOE does this), only enable “smart preconditioning” (which doesn’t help if your schedule isn’t fixed or you’re loaning a car that has been used by others). Again, the app might allow this but I was unable to check.
At one point I was driving along and the sunroof dumped a load of water on my passenger! I don’t think the sunroof was leaking and it hadn’t been opened in days but I suspect water had collected somewhere inside (it had been opened when it was a little wet a few days earlier) and a hill/bend caused it to come dripping out.
Because the cold weather affects the regen braking, it’d be nice if there was an option for the car to make up for it with friction braking so that the braking/one-pedal-driving experience doesn’t vary so much as the weather changes or battery warms up. It’s a surprise to lift your foot off the accelerator and the car not slow down as much as it normally does.
The price… Before I picked the car up I thought it was overpriced. I don’t think this anymore! However even if it’s worth the money it’s still way out of my price range so of course I wish it was cheaper! =D
I was a little paranoid about the key not being directly attached to my keyring but just in a pouch which it looked like it could easily have slipped out. It didn’t come out for me, but I wouldn’t be surprised if people have lost their “keys” because of this.
The Satnav doesn’t show the speed limit for the current road. I’m used to having this in TomTom in the ZOE and kept looking for it. I know the Autopilot version shows this (read from the road signs; maybe only in US?) but there’s no reason it can’t use a database too (presumably it has one as part of the Google mapping for travel times?)

Other Opinions

Everybody that I took out in the Model S seemed to love it. Even the people that previously weren’t entirely sold on current-gen EVs (for various reasons) like my parents loved it. My mum (who doesn’t really care about cars) text me saying “it’s ace” and I think if my dad could possibly make the numbers work out he would place his order today. We definitely convinced a bunch more people this week that EVs can be great and that it’s undoubtedly not a fad that’s going to disappear.

Summary

It was a great experience driving the Model S for a week and I’m really grateful to Chargemaster for making it happen. It’s expected that the Model 3 may be released before we plan to return our ZOEs at the end of their PCP term and may only cost £30,000 so I’ll be keeping a close eye on that and what government incentives we have at the time. Chargemaster told me they’ve ordered their Model X and it will be added to the EV Experience scheme when it arrives (expected late Q2/early Q3 2016).

If I ever find myself with £80,000 to spend on a new car, I know what will be top of the shopping list. In the meantime, I guess I’m stuck with the ZOE (except for if/when I’m able to borrow more cars from Chargemaster!).

Going Back to the ZOE

Today I got to do my commute back in the ZOE. It was an interesting change! The ZOE seating position is much higher (despite the Model S also having batteries along the bottom) and the car feels like it has a much higher centre of gravity. The reversing camera is a tiny low-definition mess. The weak regen braking caught me out a few times and I found myself having to hit the brakes while I got used to it. Although the ZOE felt fast compared to my previous petrol car (a 1.6 petrol Megane) it feels really slow compared the Model S and the accelerator is (unsurprisingly) just nowhere near as responsive.

All that said, once I got used to driving the ZOE again towards the end of my commute, I was reminded how fun it was. Driving the really Model S hasn’t changed that and I still arrived at work with a big grin on my face!

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Simple Windows utility to act as default browser and launch different browsers based on domain

2015-09-20T00:00:00+00:00

I recently saw a tweet about having different default browsers on Windows for different urls. This seemed like an interesting idea - I use a few systems that work better in one browser than another. I retweeted and saw a few responses from others that seemed interested in the idea.

It seemed like a pretty easy thing to do - register as a browser and when a url comes in, just bounce it over to the correct “real” browser based on some config. So I thought I’d knock something together…

It didn’t take long, nor much code. There are about 30 lines of code to create (or delete) registry values to register with Windows as a browser, about 60 lines of code to parse a poor version of an INI file, and around 50 lines of plumbing to pull them together and look up the URLs.

Code is up on GitHub (DanTup/BrowserSelector), and a binary release if you just want to run it.

Config looks like this:

; Default browser is first in list
[browsers]
chrome = C:\Program Files (x86)\Google\Chrome\Application\chrome.exe
ie = iexplore.exe
	
; Url preferences.
; Only * is treated as a special character (wildcard).
; Matches are domain-only. Protocols and paths are ignored.
; Use "*.blah.com" for subdomains, not "*blah.com" as that would also match "abcblah.com".
[urls]
microsoft.com = ie
*.microsoft.com = ie

It’s a first version, so I’m sure it has bugs and can be improved. I can’t make any promises to evolve it further (it’s hard to find coding time since kids came along!) but since it’s pretty trivial maybe it won’t need much work to improve and fix bugs.

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Response from CYC

2015-07-27T00:00:00+00:00

I recently blogged about poor experiences using the CYC network (mostly GMEV chargers) and sent the post to CYC. Today CYC responded.

Hi Dan,

Thanks for your email. I’ll work down your blog post point by point if that’s ok?

First of all, the problems you’ve have had with the App. We’re a little hamstrung here as our App was built by a third party who still retain most of the administrational rights for it so all changes need to go through them and be made on their side of things. When you first reported the email problem this was immediately sent to them to correct. I will also make them aware of the App security issues you have flagged up but I assure you that both CYC and our developers are fully PCI Compliant.

The mistakenly taken £1 pre-auths – I use the word mistakenly because I think ‘fraudulently’ is quite harsh – thank you for flagging this is up us. As we told you at the time, this was a problem in the way the app had been set up with Paypoint and when someone registered their card an error in the code meant that Paypoint counted it as a transaction and not a pre-auth. As soon as you flagged this up we reported it to the developers and after a fair bit of troubleshooting the problem was identified and rectified. CYC then retroactively went through and refunded everyone who had ever signed up their £1 – Including me, so thank you.

You will be pleased, I’m sure, to know that we are currently building a new App in-house that will hopefully solve a large amount of these issues and improve communications between the app and post. We (or I at least) are happy to admit that the App isn’t great atm. It’s sticky, it lags, it’s clunky. Previous management were keen to be first to market with the app and so a lot of mistakes were made that will hopefully be corrected in the new version. The wireframes for the new app have recently been signed off and so we hope it will be ready by late September, however this is dependent on how many cups of coffee and cans of energy drink we can pour down Joe.

RFID Cards attached to wrong accounts – This has happened on the odd occasion unfortunately but has been rectified as soon as it’s been flagged up. CYC Applications are processed by a member of the team here and, as is always the way, human error can occur. As much as I’d like to delete my colleagues, I do enjoy their company on occasion and so I have asked them to be far more vigilant. As far as charging drivers for usage that isn’t their own, please rest assured that any charges levied in error will be removed and ANY money taken in error will ALWAYS be refunded in full as soon as it’s flagged up. You seem to be under the impression that we’re out to stiff drivers and take their money – I can categorically say that this is not the case. We’re actually quite nice people.

RFID Cards. You hate RFID cards, I hate RFID cards, everyone hates RFID cards. Unfortunately though for funding to be issued for a charger, it NEEDS to have a unique access method so its usage and EV uptake can be monitored. There is currently no way around it – Although we are looking into alternatives. As such, we offer RFID cards priced at £20 per annum. Transport for Greater Manchester do subsidise RFID Cards for drivers within their remit, this was done as a way of encouraging EV uptake in the region and the cards are priced at £10 ONE OFF. As you can imagine, a lot of people would rather pay £10 one off than £20 per annum and so we get drivers from outside of the region trying to register (Holland being the furthest so far) If we get an application from outside of the region but close to Manchester (such as yourself) we ask TfGM about the application. As we explained to you at the time, they confirmed that you fell outside of their area.

We did say at the time that you could apply for a CYC Access Card if you wanted and we explained to you why we could not give you one for free in the same way that other networks do. The income generated by Access Cards (actually very little when you take away the VAT, cost of the card, postage and admin time) is directly used to help fund the CYC Helpdesk and our Out of Hours support – Something which we DO NOT wish to withdraw. Sorry to jump to another point, but it seems to lead on nicely.

The problem you had this weekend with a trapped cable. You are correct, the main CYC office is closed at weekends. At present we operate 08.00-18.00 Mon-Fri. Outside of these hours we operate an emergency support number. This is a member of the CYC team with a mobile phone and laptop who, no matter what time of night or day, is available to aid stranded drivers. If you call the office number you will hear a message stating that the office is closed but if you have a trapped cable please call a number. Had you of called that number, the member of staff on call this weekend would have been more than happy to stop what they were doing and release your cable.

Just so we’re clear, the staff member on Out of Hours only has limited access to systems and is only there for releasing trapped cables. However we never knowingly leave a driver stranded. There have been many, many occasions where we’ve started sessions, reset posts, directed to nearby chargers, helped non-CYC members get a charge etc… The reason the phone does not simply direct to the team member is in an attempt divert what we’d deem ‘non-emergencies’ to business hours, for example, a call at 5am because a driver couldn’t remember their Apple ID.

Incorrect charger locations and blank numbers – This is hugely disappointing. When a CP is commissioned the installing engineer takes a lat/lon reading from the post and send it to us. If this is incorrect, we’re in trouble from the get go. As you can imagine we don’t know where each of our chargers are, we’re based in Brighton and I haven’t even set foot in Manchester for years, so we rely on correct information from others. The same goes for parking information. The CYC site showed what we were given. As for the stickers, I am looking into why these haven’t been applied at the moment. I assume they were and some hilarious child removed them. Clearly we need to rethink how we number our Charge Points if they can be removed so easily.

Regarding the failed sessions at various Charge Points, if your starttransation message was reaching our servers, there would be a record of it. If it hasn’t reached us, we don’t know about it. By the sounds of it, it looks like you’ve had several occasions when, for whatever reason, the starttransation has not made it to our server. We’re looking into why this is but for the time being I have removed the chargers at Hulme Street, Bury Market and Media City from the App. Looking that the records from last week there were a number of successful charge sessions, started by the App, at the Trafford Centre and I know this is a popular location so these have not been removed at present. Again, we hope that the new version of the app will solve a lot of these issues but it’s worth mentioning that last week the CYC estate clocked up 762 Charging Sessions started with the App. 43 of which were in Manchester.

Lastly, I’d like to thank you for reporting these problems to us and I hope the fact that I’ve given such a thorough and frank reply will go some way to convincing you that we do actually care.

If you have any more questions or comments I would be more than pleased to help.

Many thanks

So CYC are now certainly aware of the experience that I (and maybe others) have had. It’s great that CYC took the time to respond in this detail :)

It’s interesting that they’re sure that if the “charge start” request hits their server, it will be logged. If the issue we’re seeing is not connectivity, then maybe it’s fixable in software (I assumed it would need better mobile antennas in the chargers). I’m hoping to proxy some failing “start charge” requests next time we go to Manchester (if I can find a way to do such a think on Android without a portable machine running Fiddler!) in the hope of maybe helping understand this unreliability.

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

GMEV and CYC - The UKs Awful Recharging Network

2015-07-25T00:00:00+00:00

EDIT: CYC provided a response to this post. It can be found here.

A little over three months ago, my wife and I took delivery of our new cars - Renault ZOEs. These are pure electric cars with 22kWh batteries that will do in the region of 90 miles on a full charge. We love our EVs and wouldn’t change them for anything, but we’ve had terrible experiences with the CYC network. Since we tell friends and family of all the positive things about EVs, it’s only fair we also share details of the bad things to ensure we’re not misleading anyone about what it’s like to own them.

This is a long post; but that’s because there have been so many issues! :(

I signed up with CYC back in April. I signed up on the website, which allowed me to use an email address with a + in it (this is a perfectly valid character to use in an email address). When I came to install the app on my phone, I discovered that the app does not allow you to login with email addresses that contain plusses! This is ridiculous for two reasons; 1) plusses are valid and 2) there is no value in validating an email address prior to attempting to login. I reported this to CYC, asking them to either fix their app or update my email address, but was left unable to use their app until this was done. A week or so later, they replied to say they’d updated my email address (no word on them fixing the issue), and I was finally able to login.

App security issues

While waiting for CYC to update my email address (above), I made some attempts to login to their app by trying to guess what the issue might be. I guessed maybe they were failing to encode the email address properly when sending it to their server (such that the + may be interpreted as a space, as is possible with url decoding). I entered the url-encoded version of a plus, and the app gave me back an error message with a stack trace from their server. This means there are certain characters in email addresses that pass the (bad) validation yet cause server errors. This makes me suspicious that there is a security exploit hiding in here. I have reported this to CYC and had no response about it. This really makes me question how safe my card details are on their system.

Fraudulent £1 charges from CYC

While waiting for CYC to update my email address (above), we wanted to try using one of their chargers. Since my wife has an EV, she signed for a CYC account too. When we tried to use their charger in Birkenhead at the ferry terminal we found that she needed to add card details (despite the charger being free). She did so, and a few days later we discovered that the £1 pre-auth the app claimed to make was actually a real £1 charge. Whilst this is not a significant amount of money, it is unacceptable (and fraudulent) to take money from someones account like this. We reported this to CYC who acknowledged the issue, blamed PayPoint(!) and refunded it. I posted on the SpeakEV forums in case others had had the same charges and despite several people quickly saying they had, it was a month before CYC finally acknowledged the issue and said they had refunded everyone affected.

RFID cards attached to the wrong accounts

One of the responses in the thread I made about the £1 charges (above) had a link to this thread which talks of several people having had RFID cards linked to their accounts that they did not own. This suggests there is some flaw in the process of CYC sending RFID cards that might mean someone gets a card on your account, and you pay for their charging!

Unreliable network (Birkenhead Ferry Terminal)

This is something that will be come up several times in this post! When trying to start a charge the CYC servers failed to connect to the charge point. This left us looking at a “Trying to start charge…” screen in the app until it timed out. We tried this over and over. It took 20 minutes of faffing before we eventually got the charger to start. Some people said this could be the mobile connection but we had a full “H” data connection and were able to access other services fine (including full use of the CYC app, until we tried to start the charge).

Confusion over whether fees are annual or one-off

The GMEV Website says the fee for their card is £10 one-off (this is subsidised). The CYC website says the fee for their card is £20 annually. It seems strange that one would be one off and one annual, so I emailed both GMEV and CYC asking about this. GMEV (eventually, after 6 weeks) told me theirs is a one-off fee, while CYC didn’t actually answer the question. This makes me suspect CYC might be charging GMEV cardholders annually, despite not being what GMEV are advertising :(

GMEV cards not available to people who live near Manchester

Although CYC never answered my question about annual/one-off fee, they did tell me that I can’t have a GMEV card because I don’t live inside Manchester. This is despite living near Manchester (Cheshire) and visiting often. This seems a little backwards given that people in Manchester with EVs probably have home chargers and probably need the GMEV network less. Those that live around Manchester are probably those who would benefit most from GMEV chargers.

Incorrect charger location (Hulme Street)

Once I finally had access to my CYC account via the app, we visited Manchester. We found a Hulme Street charger on their app and drove to the marked location on the map. It wasn’t there. Turned out that the location on their map was completely wrong, it was marked several streets away from where it had actually be installed! We headed to Hulme Street (by name) and eventually found the charger.

No charger number (Hulme Street)

In addition to using the RFID card or App, there is an option to pay by phone. To use this, you need the number of the charger which is (supposed to be) printed on the side of the charger. It is often not. The Hulme Street charger had big blank spaces where the numbers should be, making it impossible to start a charge by phone unless you can find the charger number (eg. via the app).

Unreliable parking fee info (Hulme Street)

The Hulme Street charger was marked as “free parking while charging” on the CYC website. When we got there we discovered that this wasn’t the case, and it was pay-and-display. The pay-and-display sign clearly mentioned the EV charger so it wasn’t just for non-charging vehicles.

Unreliable network - no charge (Hulme Street)

After getting the kids out of the car and the pushchair unfolded, we tried for 30 minutes to get the Hulme Street charger to start a charge. It failed every time. Clearly the CYC network had no ability to connect to this charger. After 30 minutes, we gave up and left Hulme Street to find another charger. Because we had such little charge left we were concerned about getting home. Rather than driving around Manchester using up our remaining juice and having the same experience, we headed to the Trafford Centre (where we knew there were quite a few chargers) getting there with just 7 miles remaining charge!

Unreliable network (Trafford Centre)

Once we arrived at the Trafford Centre, we spent another 20 minutes trying to start a charge. At one point, both my wife and I were trying to start charges from our own apps/logins on different sockets in the hope one would work. We did eventually get charging, but only after much frustration.

No charger number (Trafford Centre)

Like Hulme Street, two ot the three charge posts at Trafford Centre have no numbers on them. It looks like they were written on with a marker pen which, unsurprisingly, has washed away in the rain. The only way we were able to start a charge was by guessing the numbers based on the one unit that them still visible (though the numbers go backwards, which confuses things further).

Unreliable network (Trafford Centre… again)

A few weeks later we were were at the Traffod Centre again. There were no other EVs so we had a choice of three posts (6 sockets) to charge. We once again spent around 30 minutes trying to start a charge. We tried all six sockets before we eventually got a charge to start, all because of the CYC servers failing to connect to the chargepoints.

Unreliable network (MediaCityUK)

We were staying at MediaCityUK the weekend we visited the Traffod Centre. Their multistory car park has two GMEV/CYC chargers. Again, we had to try both sockets on each post before we eventually got one to start a charge. This process was repeated the following morning when we topped up.

CYC have no visibility of how unreliable their chargers are

After this latest trip, I contacted CYC on twitter (again) about this unreliability. They told me they couldn’t see any failed charge attempts at either Trafford Centre or MediaCityUK. They believe that their system does show them, however it is quite clear that it does not (we tried over 10 times to start charges at the Trafford Centre). So clearly there is no visibility of failed charges (there should be; since it’s their servers failing to talk to the chargepoint). Wow.

Incorrect charger location (Bury Market)

Today we headed to Bury Market. The CYC website shows a charger at the market (Hilton Road). Where it’s marked, is actually a building. I went into this building and asked and was informed that the charger was actually in the market car park, some drive away. There is no way without help you’d be able to find this charger from the location on the CYC map (there’s a main road in between).

Unreliable network (Bury Market)

Because of issues using the apps, we decided to try and use “pay by phone” today at Bury Market. I called and gave the charger number to the automated phone system and was told the charge had started. It took several minutes for the charger to receive the message to allow charging to start.

Unreliable charger (Bury Market)

Even after the charger said it was ready to start the charge, it took 10 minutes of plugging and unplugging before the charge finally locked the cable and allowed the car to start charging.

Unreliable network (Bury Market)

When we returned to the car, we tried to stop the charge. We called the number again and it told us to wait while it stopped the charge. The charger did not get the signal… The automated phone system kept saying “Please wait…” for some time before it realised it had failed (our cable was still locked in the charge). It told use to press star if the cable had not unlocked. We did. It then said it was trying to restart the charger to unlock the cable. Unsurprisingly, that failed too. It told us if the cable was still not unlocked, we must call the helpline.

We called the helpline… IT WAS CLOSED ON A SATURDAY.

EDIT: Turns out it there is a number in the message if you keep listening to call for stuck cables. I missed this because it says “Thanks for calling CYC, our offices are now closed. Please leave a message after the tone. (pause, noise). If you have a stuck cable, call….”. So this was my fault, but the way the message is recorded really didn’t help.

We are now stood, 50 miles from home, with our several-hundred-pound cable locked into a charger on what seems to be the worlds-most-unreliable charging network. Fuck.

I thought I would call and try to stop the charge session again. Maybe this time it would work? I called the number. However, the system thinks I’ve already finished, so didn’t offer to stop my charge. It did offer to stop a session if I provided the last 4 digits of a card number used to start a session. However since the GEMV charger is free, there was no number I could give it that would match this session. In any case, I don’t think that would’ve worked as the system thought the session had finished.

Fuck. I really cannot afford to leave this expensive cable in Bury!

Then I had an idea - what happens if I start a new charge session, then end that? Maybe by fluke it will work? This time, I decided to try the app. Trying to start a session actually caused the cable to unlock, so I grabbed it.

Phew!

If this hadn’t had worked; I would’ve been pushing CYC to cover the cost of a replacement charge cable. To me, it is unacceptable to have a charger that can have a cable locked in, with no available helpline during a Saturday.

Summary

Our experience of the CYC network (mostly GMEV, but even non-GMEV chargers) has been poor. No other network has given us any issues like this. CYC keep telling me that an RFID card will solve these issues (no surprise - “give us money, that will make it better”), however I’m reluctant to pay £40/year (for the two of us) to find out if that’s true. We would happily pay £20 one-off for two GMEV cards, but alas, we don’t qualify, despite living just far enough from Manchester to need a charge if we visit.

Others keep telling me CYC is fine, and these problems are not CYCs fault. However, these chargers are connected to their network. If CYC can’t monitor the reliability, nobody can. The fact that they don’t even have any visibility of failed charge attempts speaks volumes about how much they care about making their network reliable.

I’d love to hear from others that use CYC (especially GMEV) chargers (in the comments, Google+, SpeakEV or Reddit). A few of these issues you could consider to just be hiccups but when we’ve had the same issues across 10 difference chargers across as many visits, it seems like a far more serious underlying problem.

This doesn’t change our opinions of EVs in any way, but we will definitely avoid CYC chargers wherever we can, and be sure to always have a fallback. Poor charging infrastructure is one of the big hurdles to EV adoption, and CYC are making this worse, not better by giving people a false sense of security about being able to get a charge.

EDIT: CYC provided a response to this post. It can be found here.

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

How to Get Email Notifications of Xbox Live Games with Gold and Deals with Gold

2014-12-30T00:00:00+00:00

I recently picked up an Xbox One console (Gamertag is DanTup) and have been downloading the free Games with Gold titles and picked up a few of the Deals with Gold offers.

I looked around for some way of getting notifications of new deals/free games so that I wouldn’t miss anything if I was busy or didn’t play Xbox for a while, but couldn’t find anything. I did, however, discover that Major Nelson posted about every deal/free game (and this month, also the Countdown to 2015 deals). He also has RSS feeds for all marketplace posts or all posts tagged ‘deal’.

I did a bit of searching online, and found a couple of free RSS-to-Email services. Some of them looked a bit naff and some wouldn’t let me sign up with a plus symbol in my address (urgh!). I found two that looked reasonable and didn’t have poor email validation but since I didn’t know how reliable they’d be, I decided to sign up for both and see how they went.

The two services I signed up with are:

Blogtrottr - Free with ads; or paid plans are available.
Feedrabbit - Free with no ads, but restricted to 10 RSS feeds and 20 emails/day max; or paid plan available.

I was pleasantly surprised that both services have worked perfectly well since I signed up at the start of December, which is around 17 emails so far (they’re currently arriving daily, due to the Countdown to 2015 deals). The emails tend to arrive within an hour of each other; and according to the timestamps; actually arrive before Major Nelson even posts them! (Timezones FTW).

As mentioned above; Blogtrottr does include ads; but other than that, there’s actually very little difference in the two emails:

Since the emails always arrive together and it’s such little effort to archive them; I’m going to keep both services active (reducing the chances I might miss one if a service disappears overnight). Although you might prefer Feedrabbit’s ad-free emails, it could be argued that Blogtrottr’s ad-sponsored version might be more sustainable. Since both have been completely reliable, I can’t really offer any additional information that might swing your choice one way or the other.

If you know of any other services that work well; or have any comments on these two; feel free to post in the comments!

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Using Inbox by Gmail with IMAP or from a Google Apps or non-Gmail email account

2014-11-06T00:00:00+00:00

If you’re looking for IMAP support (lots of search terms suggest this post is getting a lot of traffic for this) then you can just use the same IMAP details you do for Gmail (see here). Inbox is just another frontend onto your Gmail account, you can even switch between the two as you like!

Google recently launched Inbox by Gmail. Like all new Google Products, if you’re a Google Apps user (whether it be a legacy free account, or as a paying business), you’ve been excluded with no indication from Google for how long.

After much faffing about; I was able to set Inbox up to work with my Google Apps account; including:

Emails sent to my Google Apps address show up in Inbox
Replies sent from Inbox show my Google Apps address as the sender
Replies sent from Inbox show up in the Sent folder in my Google Apps account

In theory; this should work with both Google Apps accounts and any other email account that you can access via IMAP/POP and SMTP.

You will need

A Gmail account that has been invited to Inbox
An email account you would like to use with Inbox that is not a Gmail account that you have IMAP/POP and SMTP access to

Step 1: Forward your email to the Gmail account

If you’re using a Google Apps account, this is done in Settings -> Forwarding and POP/IMAP. If you’re using a non-Google account, this may vary. If forwarding from a Google Apps account, your Gmail account will be sent an email to click to confirm the address.

With this done; new email sent to you will turn up in Inbox.

Step 2: Set up your outgoing email address in Gmail

Login to your Gmail account, and in Settings -> Accounts and Import you’ll need to set up a “Send mail as” address for your primary email address.

Be sure to tick “Send as Alias”; as this will stop Inbox from including yourself in the To field of all replies.

It is important that you choose the option to send mail through your own SMTP server. If using a Google Apps account; you’ll want to use:

Server: smtp.gmail.com, Port: 587
Username: Your full Google Apps email address
Password: Your full Google Apps email password; or if you have two-factor auth on, an Application-Specific password

This allows your Gmail account to send email via your primary accounts SMTP server.

Step 3: Mark your new outgoing address as default

This will ensure any new emails you create in Inbox (with the Compose option) will come from your primary email address.

Step 4: Ensure replies are sent from the email address that received them

This will ensure when you reply to an email in Inbox, the senders address is set to the same outgoing address that it was addressed to (and in the case of your primary email account, that means the email is sent via the configured SMTP server, which is what causes it to show up in the Sent folder of your Google Apps account).

Step 5: Clear your Inbox by Gmail app data

This step might be important. I didn’t do this the first time I tried this; and Inbox didn’t honour my outgoing email address settings (I believe it had cached the fact that I didn’t have any alternative outgoing email addresses). After clearing the data; Inbox correctly let me pick the outgoing email address, and defaulted using the settings I’d set above.

Enjoy!

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Have the Angular Team lost their marbles?

2014-10-29T00:00:00+00:00

I’ve posted before that I’m not a big fan of frameworks like Angular. Trying to re-invent programming in declarative HTML tags/attributes/{{ curlies }} will likely always feel lacking; and every framework having its own unique way of doing this means a ton of effort needs to be ~~wasted~~invested in updating tooling to support/understand it.

So, with my pre-existing bias out of the way; I have to say that recent information about Angular’s future only makes me hold my head in my hands even more…

Not just one new concept to add to tooling; but three!

It wasn’t enough that tooling needed to be updated to support Angular; it seems that the Angular Team now want tooling to support:

Angular v1 syntax
Angular v2 syntax (this appears to be significantly different)
AtScript

Different, but no better

In this article there’s a comparison of code for Angular v1 and Angular v2; that looks like this:

Angular v1

<div ng-controller="SantaTodoController">
	<input type="text" ng-model="newTodoTitle">
	<button ng-click="addTodo()">+</button>
	<tab-container>
		<tab-pane title="Tobias">
			<div ng-repeat="todo in todosOf('tobias')">
				<input type="checkbox" ng-model="todo.done">
				{{todo.title}}
				<button ng-click="deleteTodo(todo)">X</button>
			</div>
		<tab-pane>
	</tab-container>
</div>

Angular v2

(Note: I’ve tweaked the HTML slightly to what I believe it should be; the article seemed to have some issues).

<div>
	<input type="text" [value]="newTodoTitle">
	<button (click)="addTodo()">+</button>
	<tab-container>
		<tab-pane title="Good kids">
			<div [ng-repeat|todo]="todosOf('good')">
				<input type="checkbox" [checked]="todo.done">
				{{todo.title}}
				<button (click)="deleteTodo(todo)">X</button>
			</div>
		<tab-pane>
	</tab-container>
</div>

There are some significant differences in how Angular concepts are marked up here; but are they any better? Really? Previously things were embedded in strings in attributes, which was bad enough. But in Angular v2, we’re now using [brackets], (parens) and pipes in attribute names too. It’s different, but I really don’t think it is better.

Not ready until 2016

Angular is aiming for a release by the end of 2015 - but early 2016 seems more realistic given the drastic changes that are planned

http://jaxenter.com/angular-2-0-112094.html

This is another big WTF. The changes are so significant that they can’t be progressively released. This makes the v1 -> v2 migration seem like it’s going to be an incredible amount of work.

Angular 1.3 EOL after just 18-24 months

According to Brad Green of Angular, Angular 1.3 will continue to receive bugfix and security patch support for 18-24 months after the release of version 2.0.

http://jaxenter.com/angular-2-0-112094.html

This whole thing is starting to sound like Python 2.7 vs 3.0. Python 3.0 was released in 2008, and the change was so big and breaking, that Python 2.7 end-of-life is currently slated for 2020. That’s a 12 year cross-over. The Angular Team expect to give just 18-24 months. I don’t know which I think is more insane; supporting an old version of your product for 12 years, or giving just 18-24 months for people to migrate; both of them seem bat-shit-crazy to me.

Conclusion: Don’t pick Angular

Where I work; we’re currently evaluating a bunch of new technologies/frameworks/etc. to see which might be suitable for use on vNext of our products. Amongst those being prototyped are Dart, React, Angular and a few others. With all of this info on Angular; I’m struggling to see how we could possibly pick it. Our current codebase has parts that are over 10 years old; and we hope our new codebase will last this long too. It seems that if we start writing Angular today; we’ll be forced to rewrite the frontend in three to four years at latest (and with the way apps are going, the frontend is likely to be a large codebase). This doesn’t sound very attractive.

Maybe I’ve misunderstood something; but I’m really struggling to see a world in which this all makes good sense.

We need frameworks that are stable and supported long-term; not that are constantly inventing new concepts and being rewritten with breaking changes every 5 minutes. Of everyone, Google should know how hard it is to maintain large web apps :-/

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Running the Dart VM on a Chromebook in Developer Mode (without installing Linux)

2014-09-16T00:00:00+00:00

Writing Dart? Check out Dart Code, my Dart extension for Visual Studio Code!

I recently picked up a Chromebook (a Dell Chromebook 11) and hoped to do a a little bit of hacking on Dart with it. Out-of-the-box the Chromebook can’t run native Dart code (as command line, nor in the browser), so you’re limited to web-based apps using Chrome Dev Editor (which is rather slow at compiling Dart in its JavaScript form).

In order to get around this, many people install Linux (usually Ubuntu via something like Crouton) but I didn’t want to do this because I don’t really like Ubuntu, and it seemed like quite an overhead just to run the (fairly light) Dart VM!

So, when my (replacement!) Chromebook (my 3rd in two weeks!) turned up today; I thought I’d see if the Dart VM would work in Chrome OS’s Developer Mode without having to resort to Linux.

It turned out to actually be pretty simple; and it might be even simpler to someone that understands Linux better than I do! The steps I followed are below. If anyone can improve these; please do post in the comments so I can update them. The editing of the bash_profile is a bit clunky (mainly because I don’t know vi; and I didn’t want to use another Chrome extension just to edit the file!) and I don’t really understand why I need to remount.

Enable developer mode
- WARNING: This will delete all of your locally stored data!
- Every time you boot your device in this mode, you will get a scary splash screen and have to press Ctrl+D not to reset your device
Download the Dart SDK to your Downloads folder
- I picked the 64bit version for my Dell Chromebook 11; some Chromebooks might require the 32 bit version
Unzip the SDK to your downloads folder
- I put this at ~/Downloads/dart-sdk/
Switch to the terminal window by pressing Ctrl+Alt+=>
- (=> being the forward button where F2 would normally be)
Run chmod +x ~/Downloads/dart-sdk/bin/dart
- This will mark dart as executable
Run chmod +x ~/Downloads/dart-sdk/bin/pub
- This will mark pub as executable
Run echo "sudo mount -i -o remount,exec /home/chronos/user/" >> ~/.bash_profile
- I’m not a Linux guy; but this seems to be required to allow you to execute anything on this drive
- Without this, you’ll get Permission denied trying to execute anything
Run echo "PATH=\$PATH:~/Downloads/dart-sdk/bin/" >> ~/.bash_profile
- This adds the Dart bin folder to your path so you can just type dart without the full path
Reboot your Chromebook
- This isn’t strictly required, you could just re-run .bash_profile, but it’s a good way to ensure it works after a reboot

Now when you want to run dart, you can do it from the normal browser-based shell!

Press Ctrl + Alt + T to open a the ChromeOS Terminal
Type shell to switch to the proper shell
Type dart or pub to execute dart/pub

Here’s a screenshot showing it works on my Chromebook; I’m using Chrome Dev Editor for the Dart editing, and executing the tests in the shell. You can run pub serve this way and hit up http://localhost:8080/ in the browser for web apps (which I suspect will compile much quicker than the Chrome Dev Editor version).

If you’re interested in a Chromebook; don’t forget to take a look at Chromebook Comparison Chart! :)

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Chromebook Comparison Chart

2014-08-30T00:00:00+00:00

I’ve recently been looking for a new Chromebook. My Nexus 7 tablet just doesn’t cut it for a lot of things, and I’d like to not have to boot my desktop for some of these (relatively simply) tasks.

I’m aware there are some new Chromebooks being released that are fanless and more powerful and have better battery life than previous versions (such as i3 and Tegra processors) so I started looking online.

I was surprised (and disappointed) that there aren’t any good places that compare hardware in all of the Chromebooks, despite the hardware being the only thing that’s really different with Chromebooks! I’m a bit picky, and definitely want my Chromebook to have a higher-than-1366-768 screen resolution and at least 4GB RAM and it was hard to just find a list of the devices that met these requirements.

I started compiling a table of all the Chromebooks I could find, and kinda got carried away. I thought it was worth posting it online so that others can sort/filter/compare Chromebook models more easily!

Google Chromebook Comparison Chart

If you find any mistakes or anything missing; please let me know! You can contact me in various ways from the buttons across the top of this blog.

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

You have ruined HTML

2014-08-25T00:00:00+00:00

Not content with ruining JavaScript, it seems you’re set on ruining HTML too.

If you ask people what they’re building their client-side web apps in today; chances are they’ll tell you some hipster JavaScript framework that could be found listed on the TodoMVC website. The most popular ones tend to be Angular, Ember, Knockout, Backbone and more recently, Polymer.

Most of these frameworks have this “great” feature that lets you “easily” reference data in your views using binding expressions. They look something like this:

<!-- Knockout -->
<p>First name: <input data-bind="value: firstName" /></p>
<p>Last name: <input data-bind="value: lastName" /></p>
<h2>Hello, <span data-bind="text: fullName"> </span>!</h2>

<!-- Angular -->
<ul class="phones">
	<li ng-repeat="phone in phones | filter:query | orderBy:orderProp">
		<span>{{phone.name}}</span>
		<p>{{phone.snippet}}</p>
	</li>
</ul>

<!-- Ember -->
<div>
	<label>Name:</label>
	{{input type="text" value=name placeholder="Enter your name"}}
</div>
<div class="text">
	<h1>My name is {{name}} and I want to learn Ember!</h1>
</div>

Although the syntax varies slightly; they’re all doing similar things. They give the ability to embed some expression into your HTML that is “magically processed” by the framework. This means no manual DOM manipulation; you can write nicely formed (ish) HTML inside these expressions and the framework will do all the mundane tasks like creating nodes, wiring up event handlers and updating the DOM when your model changes. This all sounds like a nice convenience, it’s not hard to see why everyone is jumping on the bandwagon and developing this way.

However; turns out that it’s not all as advertised; there are some significant flaws in this approach that often go unnoticed until you’re trying to do something complicated.

Where I work, we’re currently building some prototypes in many of these new technologies to help guide our decision on our technologies for new versions of our products. It seems that each dev using one of these fancy frameworks is struggling and we spend a lot of time debugging issues or scouring StackOverflow for how to do (what we’d consider) basic tasks.

Some of the frustrations include…

No help from your tools

Because these binding expressions aren’t real HTML constructs, you’ll get no help from your tooling with them. No highlighting; no code-completion; no tooltips; no static analysis or type-checking. As far as your HTML editor is concerned, these expressions are just strings.

Simple tags are never simple

You’ll start off binding properties like {{ name }}, but you’ll soon find your expressions growing into complicated messes. You’ll find the built-in functionality doesn’t quite do what you need and you’ll start having to bend the framework with crap like this, writing a JavaScript function that returns a function simply to apply a filter with a dynamic name:

<tr
	ng-show="banners"
	ng-repeat="banner in banners.data"
>
	<td ng-repeat="col in banner">
		{{ col | dynamicFilter : banners.cols[$index].formatter }}
	</td>
</tr>
	
<script>
	myApp.filter('dynamicFilter', function ($filter) {
		return function (value, filterName) {
			if (Formatters[filterName])
				return Formatters[filterName](value);
			else
				return value;
		};
	});
</script>

Nothing ever works first time

As your binding expressions get more complicated, you’ll find that they rarely work first time. You’ll end up playing the shotgun-debugging game of tweaking your expression and reloading to see if you got it right (this will really suck if you have a long cycle to get back into the correct position for testing; such as having to manually log in to your app). For example, in Knockout, observables are functions, so when you want values you often need to add parens to the property. However sometimes your properties are not observable, so sometimes you don’t.

<!-- Which ones need parens()? -->
<input data-bind="value: filterData.DateCriteria().DateFrom" type="text" id="DateFrom" />

Error messages rarely exist and are frequently useless

When you get your binding expressions wrong; you’ll frequently see nothing. If you’re lucky, you might get a JavaScript error. If you do, it’ll probably say something like e is not a function and give you a stack trace that’s a hundred frames deep inside your framework and no part of it points at your HTML binding expression.

Debugging is near impossible

You can’t put a breakpoint on a {{ stupid binding expression }} because it’s not JavaScript (or is it?). You’ll often end up pulling them out into “computed properties” or some equivalent, but now your JavaScript is filling up with extension functions on objects that make no sense outside of framework-specific view-funkiness. All because your framework has a half-arsed parsing of JavaScript expressions for bindings.

What’s the alternative?

Simple. Do your programming in a programming language. Don’t try to embed it in some crazy will-never-be-a-standard binding expression invented by a fly-by-night JavaScript framework.

You probably don’t want to go back to completely manipulating the DOM by hand, but there are options like React (without JSX, since JSX sucks for many of the same reasons above) which avoids DOM manipulation (instead favouring the idea of just describing the “current state”) and keeps all your code as code. This way, your existing tools, linters, coverage etc. will all work fine. Want to extract a function to reuse it? No problem! Want to refactor->rename? No problem (well, ish… as well as this works in JS!). Want to use TypeScript and have type-checking in your expressions? No problem (well, ish… when TypeScript+React play nicer together ;)).

At work, one of our devs is building a prototype being built in React, and I’m also building one in Dart (in a React-inspired, code-generating elements way, while I quietly hope Dart TagTree evolves into a usable library). Neither of these prototypes are hitting the same frustrations as those that encourage {{ silly binding expressions }} in HTML.

HTML is a declarative markup language. Let’s leave it as such, and stop trying to crowbar non-standard binding expressions into it.

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Jawbone Up24 vs Fitbit Flex

2014-08-17T00:00:00+00:00

I previously blogged about my negative experience with Jawbone’s Up24, which resulted in exchanging the Jawbone Up24 for a Fitbit Flex. I’ve had the Flex for around 6 weeks now (my wife has had hers for about 4); so I thought it was worth posting a summary of my thoughts on the two devices compared.

Jawbone Up24 Pros

“Intelligent” Alarm: You provide a window in which you’d like to be woken up, and the Up/Up24 will aim to wake you up when you’re in the lightest sleep during that window (which is intended to make you feel less tired upon waking).
Idle Alert: Set a maximum time for which you’d like to be idle; and the Up/Up24 will vibrate to remind you to get up and move.
The mobile app is colourful and well designed.
Food logging in the app is reasonably good.

Jawbone Up24 Cons

Battery drain is absolutely poor; my Nexus 4 went from lasting 2 entire days to far less than a day. This is ultimately what caused me to return it; I’m not living with having to charge my phone more than once a day.
The mobile app is very slow/sluggish (the Nexus 4 isn’t the fastest Android phone on the market; but it’s also far from slowest).
Food logging had some glaring bugs (drinks often logged as “meals”, no reasonable way to save your own meals and easily reuse them later).
Band is rather uncomfy; it’s bulky on the bottom of your wrist (bad if you’re a programmer like me, wrists frequently on the desk) and the size isn’t adjustable for different size wrists, you have to pick the “closest fit” from the available sizes.
Requires smartphone app; no ability to log or view data from a computer.
More expensive than the Fitbit Flex!

Fitbit Flex Pros

Band is much comfier; and all devices come with two sizes of adjustable bands, so you don’t need to “pick the closest” at time or purchase.
Flex comes with a USB dongle that allows syncing without a computer.
Decent web app to allow viewing/logging of data directly from your computer.
Food logging prioritises items you create/favourite in the autocomplete list.
Battery lasts around a week (usually… see Cons).
Cheaper than the Up24!

Fitbit Flex Cons

Periodically, the Flex will completely die with no warning/notifications. Battery can be 80%, and then device just goes dead. It’s possible it just “crashes”, but when charging, it always starts at 1 light. We’ve had 4 Flexes between us, and all 4 have done it (though the most recent two only once or twice in almost 6 weeks).
Doesn’t log steps at all accurately when pushing a pushchair (I read both bands were supposed to be quite good at this; but we’re seeing probably half the steps we’d expect).
Sleep logging seems way off for me (I’m a bit of an insomniac). In “standard” mode, it says I sleep 7-8 hours when I know for a fact I didn’t. In “sensitive” mode, it logs 2-4 hours each night, in a pattern that doesn’t match the sleep I believe I get. I guess this is just down to the way it has to work (assuming still is asleep, even though when trying to fall asleep, we’re usually still!).
Support is a bit hit-and-miss. I’ve been round in circles with them trying to get Facebook/Twitter weekly summaries working, and I was asked to “re-link” both accounts 4 times with their shotgun-debugging. I also asked why the “After Dinner” option is missing from Android; and have had 4 responses so far, and I’m still not sure whether it will ever be fixed (they say it’s not a bug, but expected behaviour, but they’ve failed to explain how this could be expected).
Failing (and expensive-to-replace) Bands: Although we haven’t had any issues ourselves in the 6 weeks, I’ve seen lots of complaints of bands breaking. This wouldn’t be such an annoyance if the bands weren’t so pricey. Though I’ve friends that have bought cheap replacement Flex bands and been happy with them.

Conclusion

So, in summary; I would definitely recommend the Fitbit Flex. I would definitely not recommend the Jawbone Up24.

That said; given the success of these sorts of bands, I think we’ll see an increase in the number of options for fitness bands over the coming 12 months, including better features (such as heart rate monitoring, GPS tracking of activities). With Apple and Google both building platforms for these devices to sit on top of (iOS Health app, Google Fit) I think we’ll also end up with a more consistent view of our data, regardless of what band we choose.

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Why I returned my Jawbone Up24 after just one week

2014-06-29T00:00:00+00:00

Last week I spent some time in and out of A&E with chest pains. It’s still not completely clear what the cause was; but the doctors seem to believe it’s not a problem with my heart. However, several ECG’s have shown an (intermittent!) unusual heart rhythm, so I’m currently awaiting for a 24hr ECG to understand what’s going on (maybe my heart rhythm is timed with IEEE floating point?).

Whether this issue turns out to be related to my heart or not; it’s been a bit of a wake up call. I eat a lot of junk food, do no real exercise, sit at a computer all day every day, and barely eat 5 fruit&veg per month! I decided it’s time to start being a bit more health-conscious, and I hoped an activity tracker might motivate me a little more to get active.

I did some comparing of the Fitbit Flex and the Jawbone Up24 and decided to buy the Up24 for two features the Fitbit Flex didn’t have:

“Intelligent” Alarm: Wakes you up within a defined window when you’re in a light sleep. Apparently being woken up from a deep sleep makes you feel worse.
Idle Alert: The wristband vibrates if you don’t move for a defined amount of time (eg. an hour). Since it’s easy to sit for a long time while programming, a reminder to get up and move seems like a good idea.

I had the Up24 for almost a week, and I was generally impressed. The app was relatively polished (if a bit sluggish), and the push notifications at various intervals were quite handy. The food tracking worked fairly well, but had some annoying quirks (especially around managing your own “library”). However, there were two fairly serious flaws that ultimately resulted in me returning the device today.

The constant syncing of the band drains my Nexus 4 battery quickly. I used to get 2 days on a charge; but with the Up24 I was lucky to get 24 hours. Almost every day I woke up to a completely dead-and-turned-off phone, even sometimes if I’d charged it the night before.
Every time the band synced with my phone, it would disconnected all other Bluetooth devices. This means almost every phone call to my wife on the way home from work in my car got disconnected (the call stays active, in my pocket, until the sync finishes).

Despite not selling them in-store, Currys told me this was the third one they’d had back recently; so I’m guessing others haven’t been happy too.

I don’t know whether either of these issues are Jawbone’s fault; it may turn out that the Fitbit Flex does the same (we’ll find out in the next few days!), though I did notice that on the Fitbit website it says that all-day-sync is disabled on my device by default due to battery drain issues. The difference here, is that the sync can be disabled. The Jawbone Up24 doesn’t have any ability to disable the sync (or reduce the frequency).

I do hope that these issues get ironed out (with software updates, or future versions of the bands/phones) and that Fitbit add the missing features I liked about the Up24. I think the idea of having a small piece of technology with us at all times to help us stay fit and healthy is a great one; and I think there’s huge scope for improvement in this area.

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

DartVS - Dart Support for Visual Studio

2014-06-03T00:00:00+00:00

Note: DartVS is not being maintained, however Dart Code, my Dart extension for Visual Studio Code is and is much more complete.

Yesterday I published my fourth Visual Studio extension! :)

Recently I’ve been playing around with Dart, in the hope it might one day be a viable alternative to JavaScript. Something that always put me off trying it out in the past was a lack of Visual Studio support (I don’t want to use another IDE, I’m already using VS for ASP.NET, etc.). Since it doesn’t look like Google nor Microsoft will ever add this, I decided to give it a shot myself - I mean, how hard can it be? ;-)

Showing errors/warnings/hints in the Error List works, as does syntax highlighting (mostly). Next up is intellisense; but that’s likely to take a little longer as it’ll involve actually parsing some code!

If you’re using Dart and have Visual Studio; please try it out and let me know how you get on!

Implemented

Use DartAnalyzer from the SDK when saving any .dart file and report errors/warnings/hints to the Visual Studio error list
Clicking errors navigates to the correct place in code
Syntax Highlighting

Not Implemented

Use Batch Mode of DartAnalyzer for better performance
Intellisense
Debugging

Installation

Install from the Visual Studio Gallery
Download and unzip the Dart SDK
Set the DART_SDK environment variable to point at the SDK root

Feedback

Please send your feedback/issues/feature requests! :-)

GitHub Issues: DartVS/issues
Twitter: @DanTup
Email: danny+dartvs@tuppeny.com

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

(Web) Development Sucks, and It's Not Getting Any Better

2014-05-11T00:00:00+00:00

That’s quite a claim! I’m sure many will disagree with me, but since this is my blog you’re going to get my rant! :)

The Problem

Over the past few months, I’ve been thinking more and more about the general frustrations around web programming in 2014. For example:

There are more programming languages than there have ever been; yet it feels like there is no choice
We have more experience writing code than we ever have; yet we still struggle to express our requirements in code
We have higher level languages than we’ve ever had before; yet it feels like we’re writing more mundane boiler-plate plumbing
We have better browser support for standards; yet it’s more work than ever to support all of our users

It feels to me that as an industry, we’re treading water. Every week there’s something new and shiny, but little that progresses the significant things that could make an enormous difference to our jobs.

No Choice; JavaScript is a Monopoly

It’s no secret that I’m not a big fan of JavaScript. It’s a hacked together mess with silly quirks and no typing. But the problem here is not my opinion of JavaScript, but one of choice…

On the server, I can write my code in pretty much whatever programming language I want: C#, VB.NET, VBScript, Python, Perl, Erlang, Haskell, Clojure, Go, C… and the list goes on. Most of those even work across different operating systems. This means I can pick the language/runtime that will give me the best productivity/stability/maintainability based on the task I’m doing. If I want quick prototyping, I can use something dynamic. If I’m writing something complex that is critical is correct, I can use something more formal.

In the browser, I’m restricted to one language: JavaScript. This feels somewhat ironic given the open nature of the web! There’s no choice here. If I’m writing a huge app with millions of lines of code and I want something that can be safely type-checked and refactored, it’s tough. Your option is JavaScript. If you don’t like it, you can limit the client-side code you write, but that will be at the expense of user experience.

Want to share code between client and server? Now you’re even more screwed, because you’re writing JavaScript on the server too ;)

“AH, BUT…!” I hear you shout. And you’re right, there is another way! …

Tools That Convert to JavaScript are a HACK

Every month there’s a new tool that will let you write something that isn’t JavaScript and convert it to JavaScript. These range from simple ideas like TypeScript and new languages like Dart to huge programs to convert .NET and even native code. Well, they’re all a hack. Hack hack hackety hack. These tools are not being built because they are a good idea, they’re being built because we’re forced into using JavaScript and most people don’t like it. As a result, they’re usually full of edge cases, varying browser support, inconsistent performance and other strangeness.

We Cannot Express our Requirements in Code

Even if you throw away all this client side nonsense for a minute, most mainstream programming languages still fail to allow us to express really common rules at compile time and force us to throw errors at runtime. There are lots of ideas out there about how we can fix it, but where’s the progress? Why is Microsoft’s Code Contracts still an unstable research project? Why hasn’t it had an update for 8 months? Why don’t more languages have dependent types? Why are there not more static analysis tools of this type? Why are even new programming languages being designed today with nullable references you can’t opt-out of?

Our High-Level Languages are not High-Level Enough

With most languages today, it still feels like I’m writing a bunch of plumbing code that I shouldn’t need to. I can easily write a few hundred lines of code across several classes to do something, yet describe the functionality in just a few short sentences. Why do I still need so much third party code to perform basic functions? For example, look at this nodejs module, exit. All it does is exit a node application in the way one would expect, after flushing standard streams like STDOUT. It’s a 41 line node module. A THIRD PARTY MODULE AND FORTY-ONE LINES TO EXIT CLEANLY!!

How can we ever talk about declarative programming when we can’t even abstract the basics away?

Browser/Device Support is Harder than Ever

Browsers implement standards better than they ever have; yet making your app work in all of your users browsers is more difficult today than ever before. Why? There are more browser rendering engines than there have ever been. Even within a single browser rendering engine, there are more versions out there today than there have ever been. IE having huge market share might have had its own issues, but was it really worse than battling with differences between IE7, IE8, IE9, IE10, IE11, IE Metro (likely multiple versions), Chrome, Firefox, Safari, Opera, Chrome for Android, Samsung’s Browser, etc.? And now let’s add in screen size and pixel density! People are trying to write media queries that use size to adjust to the screen, but it’s futile when small devices have hi-res screens and there are no clear boundaries. We need to base things on physical size, not pixel density, yet support for detecting this is poor. These things have become so complicated that services exist just to let you see/execute your apps across all of these devices because nobody can realistically have physical access to all of these things!

How can we Improve Things?

I think it all comes down to a few things:

I should be able to pick my programming language
My programming language should be flexible enough for me to express things I know up-front and get compiler assistance instead of runtime errors
I should not have to write much code that is not specific to my application/domain
My code should give the exact same results everywhere

Stating requirements is easy; pulling them off less so! Note: I’m just an average developer! I’ve no experience with anything as complicated as what I’m proposing here. Don’t interpret this as me suggesting this is easy!

Here’s what I think we should be investing time and effort into:

A new intermediate format/VM for running code in the browser to allow multiple languages
- It doesn’t matter what this is based on; JVM, MSIL, NaCl, something else, as long as it’s something up to the job (with solid sensible behaviour, not JavaScript)
- It must support being targeted by at least a handful of mainstream languages (this will ensure suitability for new languages too)
- Both Microsoft and Google agree to implement it (I’m certain other browser vendors would follow if both of these did, but I doubt that Google or MS would follow others)
All browsers auto-update by default
- No off-by-default, even for enterprise
- Browser vendors drop support for old versions quickly
Good static analysis tools for mainstream languages to allow us to move more runtime crashes to compile errors
- Microsoft should get Code Contracts or something similar back on the roadmap
- If we had a good intermediate format (as mentioned above), these tools may be able to be generic and work for multiple languages
New programming languages; It’d be very hard to try and retro-fit some of the things we need to existing languages
- Designed to be runnable both on the server and within the browser (see above), including compile-time enforcement of API sets (eg. no access to file system in client libraries)
- No nulls!
- Contracts, Dependent Types or some other way to express our requirements better to cut down on runtime errors
- Strong typing
- Ability to enforce purity and exhaustiveness/totality on functions

You may argue some of these are not necessary, but I think the end goal needs to include all of these, even if the path to get there is a progressive one.

The Chances of Things Getting Better are Slim

That’s a pretty pessimistic view, but sadly I think it’s where we are. The problem is; the entire “plan” above pretty much hinges on support and collaboration between two of the biggest software companies in the world: Microsoft and Google. Both of these companies have a serious chunk of the browser market and it’s a near-impossible task to ever challenge that. With their rocky relationship I don’t see a world where either company will add the others technology to their browser (eg. IE getting NaCl or even Dart support), nor anyone coming up with a standard intermediate format they would both agree to support. Browser plugins are not a realistic option even today and their support is only likely to shrink in the future.

Conclusion

If you’re writing code on the server, you’ve got it great. You can pick and choose your language and over time, you’ll get more of the programming language features you need, or new languages to provide them.

If you’re writing code for the browser, then unless you love JavaScript, your life is going to continue to be frustrating for many many years to come!

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Is F# Ready for Production?

2014-04-19T00:00:00+00:00

It took me some time to get my head around F#. I had no experience with functional programming and failed twice to pick it up. I persevered; and third time lucky, it started to make sense!

I became a huge fan! Being part of the .NET Framework made it feel fairly familiar and more likely we could start to introduce it at work (we’re primarily .NET devs). Options instead of nulls; immutability, pattern matching, DUs, loads of stuff that looked like it would solve problems we have every day (especially the nulls - we have a > 200k LoC inherited legacy codebase that started life in .NET 1.1 to fight with).

I gave a talk to colleagues who were (inexplicably!) unconvinced, so I gave another one a little later and soon after one of my colleagues arranged working through F# Koans as a team; and the wheels seemed to be moving.

Since we’re busy as ever; F# sort of fell to one side. We were all busy working meeting deadlines; and as a team we weren’t confident enough in F# to pull it into production. The team were even less convinced about F# after I declared PowerShell to be an annoying crock of buggy shit that gets no responses to bug reports, when I’d been so enthusiastic about it months earlier. They expected I’d be announcing F# to be a big failure soon too (oh, the irony of this blog post!) :-/

Fast forward a few months; and I’m charging along again. I got the latest F# up and running on the work build server. Production are up to .NET 4.5.1. I set up an internal NuGet feed and extracted some simple libraries from our main products solution into NuGet packages for easy replacement. Everything is ready to start getting small F# libraries into the main product, allowing the team to get more familiar with writing and reviewing F#! :)

So, what’s the problem? Get to it already!

Well; once we take the dive; it’s hard to go back. This means I’ve been thinking more about it this week than previously; and I’m now starting to wonder if F# is really ready. There are a few little things that quite bug me…

IDE support even in Visual Studio 2013 is absolutely poor. Basic things like document formatting and refactoring are missing. The community have worked hard on improving this somewhat; but I hit serious bugs (edit: this was fixed within 2 days; kudos!) in their extension within minutes. We spend a lot of time in our IDEs; we need first class support; and F# is miles off.
Options are great; nulls are bad. But because F# runs on the CLR; nulls are not gone. This means we now just have two different types of null; the bad one, and the good one. I’m on the fence about whether this is really an improvement. Even if you have the luxury of being F# for all of your own code; the BCL is riddled with null returning/expecting methods.
F# was completely missed from Roslyn. I understand the reasons for this; but with this new “open” Microsoft; there’s still no word on whether this will happen. If it doesn’t; it means F# will miss out on a new generation of tools being built on Roslyn (for example, Code Diagnostics/Fix support).
There’s little info on what’s coming in the next version of F#; yet we know quite a lot of language features coming in the new versions of VB and C#. It feels like C# and VB are charging forwards at a much faster rate than F#. Do we want to get off the fast train and board the slow train?
Hiring developers is already very hard. Do we want to limit the pool of available candidates significantly (and take the increased salaries that come with the harder-to-find skills)?

Scott Hanselman was right; programming is hard, but apparently not just the writing of code!

I’m sure use of F# will continue to grow at our company. Both myself and the other team leader are both big fans. However I think the journey will be much slower and more cautious than I’d originally hoped.

Don’t take this article as a bashing of F#; I’m still a huge fan (both of F# and general FP ideas), but I think it may be some time before it’s the obvious choice to make.

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Free Software and Services for Open Source Projects

2014-04-15T00:00:00+00:00

Yesterday I blogged a list of some of my favourite open source projects. I thought it was also worth blogging some of my favourite services/software with free offerings for open source projects that make it much easier or cheaper for open source projects to exist, collaborate and build great software.

I’ve tried to limit the list to things I have first-hand experience of or seem popular amongst people I follow on social networks; but there’s a chance there are omissions. Ping me on Twitter if you think there’s something I should really add.

Source Code Hosting

GitHub - Git source code hosting; issue tracker and web page hosting. By far the most popular/active for open source projects right now.
Codeplex - Git, TFS and Mercurial source code hosting, issue tracker, wiki. Needs a bit of love, to be honest!
BitBucket - Git and Mercurial source code hosting. BitBucket has free private repos too (so isn’t just free for open source). I used to use this quite a lot until GitHub got so popular that it was hard to ignore!
Google Code - Git, Mercurial and Subversion. Doesn’t seem to be much here these days; mostly just dead projects. Even lots of Google’s own OSS is hosted on GitHub!
Kiln/FogBugz - Whilst this isn’t strictly for open source; Kiln and FogBugz are free for two users and supports “community users” for public access. I couldn’t not include this; it’s my favourite DVCS hosting and includes a great Code Review system.

Web App Hosting

Azure - Host 10 web apps for free; though you’ll need to pay for custom domains.
AppHarbor - A .NET PaaS (Platform as a Service) provider. When first launched, this service made Azure look incredibly clunky and complicated for simple web apps, and used to host the website for G+ Notifier for free under their OSS program.

Hosted Continuous Integration / Deployment

Travis CI - This seems to be getting more and more popular for GitHub projects, possibly down to really nice integration that allows running tests and reporting pass/fail status in pull requests automatically!
BuildHive - Powered by Jenkins, also with easy integration with GitHub.
CodeBetter CI - Powered by Team City.
Bamboo - From Atlassian; also available for download.

Testing

Sauce Labs - Run your web and mobile app tests across hundreds of real browsers and platforms.
BrowserStack JavaScript Testing - An HTTP-based API that can be used to open any URL in any combination of browser/OS on BrowserStack to execute JavaScript tests.

Other Development Software / Web Apps

TeamCity - Continuous integration server (also available hosted as CodeBetter CI).
Reflector / ANTS Profilers - Code and memory profilers and decompiler for .NET.
Moqups - Online tool for mocking up UIs quickly.
myBalsamiq Mockups - Online tool for mocking up UIs quickly.
JIRA / Bamboo / Confluence - Issue tracker, continious integration and team collaboration software. Also available hosted.
RhodeCode - A locally-installed DVCS and code review system.

Misc

GoDaddy SSL Certificates - I’m not sure there’s much for love for these guys, but they are claiming to give free SSL certs for OSS.
Crowdin - Localisation platform to help manage your translations.
BugSense - Analytics software for catching and collating errors in mobile apps.

After posting this; I was tweeted a link to ossperks.com, a similar (but more-complete) list of free stuff for Open Source projects. If you’re running an OSS project, Irecommend checking it out!

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

My Favourite Open Source Software

2014-04-14T00:00:00+00:00

(You may also be interested in my post Free Software and Services for Open Source Projects.)

In a recent blog post, Scott Hanselman suggested:

We don’t sing contributor’s praises for their hard work and success while their software works, instead we wait until a single line (albeit one of the more important lines) fails to live up to expectations.

I don’t completely agree with this; I often see people praising open source software (and I didn’t see anyone really pointing fingers over Heartbleed), but I do think the general point that open source software is taken for granted and under-appreciated is valid.

This got me thinking about some of the open source software I use; some as part of my job delivering commercial applications and some at home when I’m just playing around. I thought it’d be good to blog a list of my favourites; maybe it’ll help others discover them. Much of it is pretty well known (some even from MS/Google), but I thought I’d include all my favourite stuff for completeness.

I may have missed some; but there’s a lot of great stuff out there. If I remember anything cool Ive missed, I’ll try and add it in.

If you have a blog (or even Google+ ;-)) I invite you to do the same!

(In alphabetical order; with no grouping, and often their own marketing speak!)

7-Zip - A file archiver with a high compression ratio. There’s a portable version I use to avoid having extra stuff installed, but still allowing the higher compression of .7z when required.
AngularJS - A toolset for building the framework most suited to your application development. I’m currently playing around with this prototyping some ideas for a possible vNext of a big project at work; I think the future of web apps is (sadly ;)) in client-side JavaScript with API backends.
Bootstrap - A popular front-end framework for developing responsive, mobile first projects on the web. This is great for getting something looking decent very quickly, especially for non-designers (programmers!).
Chosen - A jQuery plugin that makes long, unwieldy select boxes much more user-friendly. These look really nice; especially compared against the default selects in some of the latest browser versions!
ELMAH - An application-wide error logging facility.
F# - A functional-first programming language that interops relatively nicely with .NET. This could be the key to getting current-.NET devs interested more in functional ideas.
FAKE - A cross platform build automation system (F# Make). Haven’t used this seriously yet; but looking for someone to replace all the built steps on our build server with a single script that can be run on dev machines to give a like-for-like build.
Glimpse - Inspects web requests as they happen, providing insights and tooling that reduce debugging time.
Jasmine - A nice behavior-driven development framework for testing JavaScript code.
Jekyll - A blog-aware, static-site generator (powering this blog for free, via GitHub Pages!)
Karma - JavaScript test runner that executes tests in multiple browsers and gives instant feedback (now with Visual Studio integration ;)).
Knockout - A JavaScript library that helps create rich, responsive display and editor user interfaces using two-way data-binding. This helps de-couple JavaScript from the DOM, making things much simpler!
Mercurial - A distributed source control management tool. This is much easier IMO to use than Git; though sadly I don’t think I can convicne the masses ;)
Node.js - A platform built on Chrome’s V8 JavaScript runtime for building fast, scalable network applications. Great package manager, and really useful for non-browser based JS execution.
NuGet - A package manager for the Microsoft development platform including .NET. This is basically the first place I look for any third party library these days.
Reactive Extensions - A library for composing async/event-based programs using observable sequences and LINQ-style query operators.
remark - A simple, in-browser, Markdown-driven slideshow tool targeted at people who know their way around HTML and CSS (I blogged more about this yesterday).
Selenium - Automates browsers to aid testing of web applications. If we’re going to start writing more client-side JavaScript for our web apps; we need to get better at testing them!
SignalR - A library for ASP.NET developers that makes it incredibly simple to add real-time web functionality to applications.
StructureMap - A dependency injection / inversion of control tool for .NET.
TickSpec - A lightweight F# BDD framework.
TypeScript - A typed superset of JavaScript that compiles to plain JavaScript.
Web API - A framework that makes it easy to build HTTP services that reach a broad range of clients, including browsers and mobile devices.
xUnit - A community-focused unit testing tool for the .NET Framework.

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Markdown-based Presentations with Remark

2014-04-13T00:00:00+00:00

After giving a talk at work to colleagues recently using slides I’d created in Google Docs; I thought I’d had a great idea! What if I could edit my slides as text (something like Markdown) and bypass the nasty WYSIWYG editor?!

Well; as is usually the case when you have a great idea; it’s already been done. I played around with a few I found online and one that was really neat. Not only does it support markdown; but it also has extensions so you can add CSS classes to content to lay things out just like a web developer is used to! As it’s web-based, you can even put your own JavaScript in to mess with things further!

The project is called remark, you can see a slideshow created using their script on their website. The source text can be seen here.

In addition to being able to knock together a slideshow in text; there are some other cool features (you can try these out on their website slideshow):

Press ‘c’ to clone the window, so you can show a separate copy on an external screen (the clones are kept in sync with an API)
Press ‘p’ to open presenter mode for the active window, which shows current/next slide and speaker notes (and unlike Google Docs; the slide preview is not tiny!)
Press ‘f’ to make the slideshow full screen (you can set the aspect ratio in the code)

I often use other peoples software; but I don’t have enough time to contribute to many. However, I liked this one so much, I had a stab at fixing a niggle I found; and it’s now been merged into the main repo :)

Slide

Presenter Mode

So, it all looks pretty slick. One thing I’m certain of; if this hadn’t existed and I’d started my own; it wouldn’t have looked this neat! If you’re a web developer; give it a go. I’m certain you’ll prefer it to PowerPoint and Google Docs! :)

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Cross-browser JavaScript Testing with Karma and Visual Studio

2014-03-09T00:00:00+00:00

Yesterday I published my third Visual Studio extension! :)

It’s a fairly generic extension, with reads static XML files that contain test results into the Visual Studio Test Explorer window. While this might sound a bit strange, its goal is to integrate with third party tools that can output results as XML and update them realtime. For example, Google’s Karma test runner.

Here’s a video showing it all in action:

Installation and Setup

Install my Karma Test Adapter from Visual Studio Gallery
Install/download nodejs (I grabbed the exe from here and added it to my PATH)
Install/download npm (I grabbed the zip from here and put it in the same folder as node.exe)
Install the karma-cli node module globally
- npm install -g karma-cli

Usage

Create/navigate to the folder that contains/will contain your tests
Create a package.json to hold your node dependencies
- PowerShell: "{}" | Out-File -Encoding ascii package.json
- or CMD: echo '{}' > package.json
Install karma and some dependencies
1. npm install karma --save-dev
2. npm install karma-xml-reporter --save-dev
3. npm install karma-jasmine --save-dev
4. npm install karma-phantomjs-launcher --save-dev (required only if you want to use PhantomJS)
5. npm install karma-ie-launcher --save-dev (required only if you want Karma to be able to launch IE)
6. npm install karma-chrome-launcher --save-dev (required only if you want Karma to be able to launch Chrome)
7. npm install karma-firefox-launcher --save-dev (required only if you want Karma to be able to launch Firefox)
Create a karma config file in the folder with your tests
1. karma init
2. answer the questions, for example:
  1. Test framework: jasmine
  2. RequireJS: no
  3. Browsers:
    - Chrome
    - IE
    - PhantomJS
  4. JavaScript and Test files:
    - app\**.js
    - tests\**.js
  5. Exclusions: (none)
  6. Watch for changes: yes
Open up the generated karma.config.js in your text editor and add ‘xml’ as a reporter
- reporters: ['progress'] becomes reporters: ['progress', 'xml']
Open Visual Studio and create a project that lives in the folder that contains your tests/karma config file
- I usually choose File -> Open Web Site and choose the FileSystem option, pointing at the folder
Create a simple test we can use to verify everything works. The filename must match the filter you gave to karma init earlier (eg. MyTestSpecs.js)
- describe("a test", function() { it("passes", function() { expect(true).toBe(true); }); });
From the command line, start karma
- karma start
- note: This will fire up the browsers that you configured; these must be left open to execute tests
In Visual Studio, toggle the “Run Tests on Build” option on in the Test Explorer Window
Right-click -> Refresh on the project, to ensure Visual Studio spots the test-results.testxml file
As you make changes to your js files and press save, Karma will now automatically execute your tests and the results will magically update in Visual Studio!
Bonus: You can open up other browsers and point them at the url that’s loaded in the existing browsers and they will automatically be used for tests. Eg. enter the URL into your smartphone browser, and now it will show up in the test results too!

Feedback

Please send your feedback/issues/feature requests! :-)

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

A Jasmine Test Adapter for Visual Studio

2014-03-02T00:00:00+00:00

Today I published my second Visual Studio extension! :)

Jasmine Test Adapter for Visual Studio

It’s similar to the Lua Adapter I created recently, but uses node.exe instead of lua.exe and includes an internal copy of the Jasmine JavaScript testing framework. I’ve started playing around with AngularJS, and since it seems to be written with unit testing in mind, I somehow ended up at Jasmine and decided it needed an easy way to be run within Visual Studio!

If you try it out, please do send feedback!

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Some things I learned while building my Visual Studio Test Adapter

2014-02-16T00:00:00+00:00

Here’s a couple of things I learned while creating a Visual Studio test adapter that included a custom ITestContainerDiscoverer:

There is almost no documentation on creating Visual Studio test adapters (and even less on ITestContainerDiscoverer and friends)
You need to install the Visual Sudio SDK to get the required assemblies for creating test container discoverers (but not for just test adapters)
You cannot create a test adapter without a test container discoverer if you want to scan non-DLL/EXE files
Although you can deploy test adapters via NuGet; this does not work for test container discoverers; they must be registered as real Visual Studio extensions
To have Visual Studio load your test container discoverer, you must add an asset with type “UnitTestExtension” to your VSIX; an option which is not in the list (nor, seemingly, documented)
Just because you’re in the middle of responding to a request from Visual Studio to get a list of tests; doesn’t mean it won’t ask you AGAIN AT THE SAME TIME on another thread :/
When building Visual Studio extensions, the entire VSIX output folder is copied into the expirimental instance, not just the files you’d get via the VSIX. This means references to other projects might work fine on your machine, but not on somebody elses!
The “Run tests after build” function actually means “when test container change”, so even for non-building projects (eg. lua tests), it can work when you save files :)

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

A Lua Test Adapter and Framework for Visual Studio

2014-02-16T00:00:00+00:00

Today I published my first Visual Studio extension! :)

Lua Test Adapter and Framework

It was created because I’d recently been working on a World of Warcraft Addon, RareShare. As the addon currently has no UI (mostly just interacts with the World of Warcraft API), I decided to create a set of automated tests for it (as any self-respecting software developer would ;)). I quickly became annoyed by reading the output of lua.exe and then looking up test line numbers, so decided to investigate how hard it was to create a Visual Studio test adapter!

It actually turned out to be slightly more complicated than expected; so I’ll leave that for the next post. However, I did build it in a way that keeps the Lua-specific stuff out of the main assembly; such that it should be quite simple to add other types of external tests (eg. Python, Perl, JavaScript, whatever). I can’t promise I’ll get around to any of them; but I might take pull requests ;-)

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Easily Calling Windows APIs from PowerShell

2013-10-26T00:00:00+00:00

After my previous post of some snippets from my PowerShell profile; I received another email from the person that prompted me to write that post, asking some questions about calling Windows APIs from PowerShell. It turns out that this isn’t so straight-forward, despite PowerShell’s pitch as a system automation language!

The confusion came from various varying samples online of how to do this in PowerShell; and they all used blocks of C#, which many PowerShell users aren’t particularly familiar with.

Although I couldn’t find a way to do this without the C# wrapper, I did think it was worthwhile extracting some parts out of the C# code to avoid having to manipulate so much C# code in a quoted string inside PowerShell to add new Windows API methods. Unfortunately the argument types still need to be in C#, because I don’t know how to map them from the C++ examples easily.

It’s very basic; and can probably be improved by anyone that has some knowledge of calling Windows APIs; but hopefully it’s a little simpler if you’re less comfortable with C#.

In PowerShell profile:

# Helper functions for building the class
$script:nativeMethods = @();
function Register-NativeMethod([string]$dll, [string]$methodSignature)
{
    $script:nativeMethods += [PSCustomObject]@{ Dll = $dll; Signature = $methodSignature; }
}
function Add-NativeMethods()
{
    $nativeMethodsCode = $script:nativeMethods | % { "
        [DllImport(`"$($_.Dll)`")]
        public static extern $($_.Signature);
    " }

    Add-Type @"
        using System;
        using System.Runtime.InteropServices;
        public static class NativeMethods {
            $nativeMethodsCode
        }
"@
}


# Add methods here
Register-NativeMethod "user32.dll" "bool SetForegroundWindow(IntPtr hWnd)"
Register-NativeMethod "user32.dll" "bool ShowWindowAsync(IntPtr hWnd, int nCmdShow)"

# This builds the class and registers them (you can only do this one-per-session, as the type cannot be unloaded?)
Add-NativeMethods

And to use them:

# (the Out-Null is just to throw away the return value)
[NativeMethods]::SetForegroundWindow((Get-Process -name notepad).MainWindowHandle) | Out-Null
[NativeMethods]::ShowWindowAsync((Get-Process -name notepad).MainWindowHandle, 2) | Out-Null

Hopefully this helps a little. If anyone can suggest improvements (being able to paste method signatures directly from the documentation would be good), I’ll update the post.

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Useful PowerShell Profile Snippets

2013-10-23T00:00:00+00:00

Yesterday I recevied an email about some code snippets in my PowerShell profile that I’d mentioned on StackOverflow. While trying to retrieve them; I noticed that I hadn’t put them on to my newly-installed Windows 8.1 machine; so I thought it was worth sharing them here while I was copying them from my work machine anyway.

I don’t use PowerShell for scripting much now; I’ve been using FSI (type safety FTW). However, I do still think PowerShell is an excellent shell, and the ability to connect remotely to servers is incredibly useful for executing our deployment/IIS setup scripts/etc.

Adding Colour to Hostnames in PowerShell Remoting Windows

With some help from StackOverflow, I have some functions to help me connect to frequently-used servers, highlighting the hostname depending on whether the server is local/staging/live (green/yellow/red). They’re all prefixed with R- so I can type R- and then tab through the different servers.

# Connect to servers
function R-LocalQAServer    { R-_Connect Green  "LocalQAServer"       $null          "D:\Applications\MyApps" }
function R-BuildServer2     { R-_Connect Green  "buildserver2"        $null          "D:\CI"               }
function R-StageWebserver   { R-_Connect Yellow "stageweby.mycompany" "mycompany\me" "D:\Applications\MyApps" }
function R-ProductionServer { R-_Connect Red    "live.mycompany"      "mycompany\me" "D:\Applications\MyApps" }

# Shared function to handle connecting to servers with specified colour and a specific starting directory
function R-_Connect([ConsoleColor]$color, [string]$hostname, [string]$user, [string]$root)
{
	If ($user) { $session = New-PSSession $hostname -Credential $user }
	Else       { $session = New-PSSession $hostname }

	# Set up MYAPP:\ Drive
	Invoke-Command -Session $session -ScriptBlock {
		New-PSDrive -Name MYAPP -PSProvider FileSystem -Root $using:root | Out-Null
		CD MYAPP:\
	}

	# Pass the hostname+color in so we know what was connected to, so we can overwrite the prompt in colour
	Invoke-Command -Session $session -ScriptBlock { $connectedHost = $using:hostname }
	Invoke-Command -Session $session -ScriptBlock { $promptColor =   $using:color }

	# Overwrite the prompt function to colour the server name
	Invoke-Command -Session $session -ScriptBlock ([ScriptBlock]::Create("function prompt { $(Get-Content function:\RemotePrompt) }"))

	# Connect the local/remote sessions
	Enter-PSSession -Session $session
}

# Helper function that is transferred to the remote server to rewrite the prompt with colour
function RemotePrompt {
	# write computer name with color
	Write-Host "[${env:computername} (${env:username})]: " -Fore $promptColor -NoNew

	# generate regular prompt you would be showing
	$defaultPrompt = "PS $($executionContext.SessionState.Path.CurrentLocation)$('>' * ($nestedPromptLevel + 1)) "

	# generate backspaces to cover [computername]: pre-prompt printed by powershell
	If (!$connectedHost) { $connectedHost = $env:Computername }

	$backspaces = "`b" * ($connectedHost.Length + 4)

	# compute how much extra, if any, needs to be cleaned up at the end
	$remainingChars = [Math]::Max(($connectedHost.Length + 4) - $defaultPrompt.Length, 0)
	$tail = (" " * $remainingChars) + ("`b" * $remainingChars)

	"${backspaces}${defaultPrompt}${tail}"
}

Launch Kiln/BitBucket/Whatever from Shell

The first thing Iusually do after pushing changes to Kiln, is open up the web app and raise code reviews. So; I added a command “Kiln” which just launches Kiln at the correct repo page for where I am. The Kiln extension can do this (“hg kiln”), but we’ve had bad experiences with the Kiln extensions not working on recent versions of Mercurial (due to PyWin32 dependencies) so most of us have them all disabled (except KilnAuth).

# Launch Kiln for current repo
Function Kiln
{
	if (Test-Path ".\.hg\hgrc")
	{
		$repoUrl = (Select-String "default = (.*)" -Path ".\.hg\hgrc" -AllMatches).Matches.Groups[1].Value
		Start $repoUrl
	}
	else
		{ Write-Warning "Not in a repo!" }
}

Launch Visual Studio for Current Project

Because I switch between repos a lot (different versions of our product), it’s a pain to change directory in PoSh and also navigate to the new solution in Visual Studio. So I simply close Visual Studio and type “VS” when in the correct directory for the new branch to launch Visual Studio for any sln file in the current folder. This could be extended to recurse down (or up), but I found it most convenient just working in the immediate folder.

# Launch VS for sln(s) in current folder
Function VS
{
	ii *.sln
}

Launch Windows Explorer for current folder

Why type “start .” when you can just type “e” to start explorer in the current folder?

# Launch explorer in current folder
Function e { ii . }

Launch default browser for current folder via web server

Sometimes you have a static html file you want to load up in a browser to hack on and test, but some browsers (like Chrome) won’t let you pull in scripts for file:/// paths. This function (which requires Python in your PATH) launches the builtin Python web server for the current folder and fires up your browser. If you pass an optional filename

Serve mytest.htm

then it’ll load that fie in the browser.

Function Serve ($page)
{
	Start-Process python -ArgumentList "-m SimpleHTTPServer 8000"
	Start "http://localhost:8000/$page"	
}

Run Mercurial Commands with Less Typing!

And while we’re at it; why have to type “hg” all the time?

# Mercurial helpers
Function st { hg st $args }
Function vd { hg vd $args }
Function ci { hg ci $args }
Function clone { hg clone $args }
Function hist { hg hist -l 5 $args }

Add Visual Studio / .NET Tools to PATH

I like the Visual Studio Developer Command Prompt because it has lots of useful stuff in PATH; but trying to get this into PoSh sucks. You can’t call the old .bat files because they won’t update PowerShells PATH. Re-implementing them is insane, so it’s easier to just hard-code the important few, and add/update as required!

# Add some useful stuff to PATH
$env:Path += ";C:\Program Files (x86)\Microsoft SDKs\F#\3.1\Framework\v4.0\; C:\windows\Microsoft.NET\Framework\v4.0.30319; C:\windows\Microsoft.NET\Framework\v3.5; C:\Program Files (x86)\Windows Kits\8.1\bin\x86;"

Clean all folders with MSBuild

# Reclaim a ton of disk space from branches that have been built but aren't actively needed
# It's hard-coded with my code folder and MYAPP:\ so I can call it from anywhere, but
# it can be made more generic and simplified with James Tryand's walk function further down
Function Clean-All
{
	pushd MYAPP:\
	Get-ChildItem -Directory | % {
		pushd $_
		Get-Item *.sln | % {
			Write-Host ("    Cleaning {0}" -f $_.ToString().Replace("C:\Work\Source\MYAPP\", "")) -F Yellow
			&'msbuild' /t:Clean /nologo /v:m $_
		}
		popd
	}
	popd
	Write-Host ""
}

Create a PSDrive for Code Folder

To make it easy to always get back to my root code folder, I’ve created a PDrive, and change into it at launch.

# Set up MYAPP:\ to point at code checkout folder
New-PSDrive -Name MYAPP -PSProvider FileSystem -Root C:\Work\Source\MYAPP | Out-Null
CD MYAPP:\

I think that’s all the useful stuff incurrently in my profile that might be handy to others. Do post your own useful snippets in the comments!

Highlights from the Comments

Execute command for all child directories

Function Walk-ChildDirectory
{
    Param(
        [Parameter(Mandatory=$true,ValueFromPipeline=$true)][ScriptBlock]$Task
    )
    ls -Directory | %{
        pushd $_
        & $Task
        popd
    }
}
Set-Alias walk Walk-ChildDirectory

# eg.:
# walk { git gc } # Git GC (WTF is this?)
# walk { msbuild /t:clean } # Clean all projects to reclaim disk space!
# walk { walk { walk { pwd } } } # Can be nested!

Ensure stuff doesn’t drop off the history list

$MaximumHistoryCount = 1024

Handy functions to handle strings more nicely

function Quote-String { "$args" }
function Quote-List { $args }
Set-Alias qs Quote-String
Set-Alias ql Quote-List

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Functional Programming Challenge; Words with Indexes

2013-09-20T00:00:00+00:00

A colleague has also been starting to learn F#, and today gave me a problem he’d been trying to solve in F# in a “functional style”. He wanted a function that took a string input, and returned a list of the words, tupled with the starting index.

I spent the whole of lunch staring at my screen, trying to wrap my head around List.fold, list.foldBack and other functions; but no joy.

While driving home, it occurred to me that I could do away with that nonsense, and just have a recursive function that passes all the values down, top-to-bottom!

Here’s my latest attempt… The results are correct, which is an improvement over my original attempt!

let splitOn f x = (Seq.takeWhile f x, Seq.skipWhile f x)
let isSpace c = c = ' '
let notSpace c = c <> ' '
let string (s : seq<char>) = new String(Seq.toArray s)

/// Get a list of all words in a string tupled with the starting index
let getWords input =

    let rec getWordsRec index results input =
        match input with
        | [] -> results
        | _ ->
            let white, rest = splitOn isSpace input
            let word, rest = splitOn notSpace rest
            getWordsRec
                (index + Seq.length white + Seq.length word)
                ((index + Seq.length white, string word) :: results)
                (Seq.toList rest)

    getWordsRec 0 [] (Seq.toList input) |> List.rev



let expected = [(0, "Hello"); (6, "test"); (12, "World")]
printfn "Result is %A" ((getWords "Hello test  World" = expected))

Can anyone come up with something more elegant?

Update

I managed to simplify/flatten it a little, though I’m not convinced it’s any more readable!

let string (s : seq<char>) = new String(Seq.toArray s)

let getWords input =
    input
    |> Seq.zip (Seq.initInfinite id) // Zip the list with indexes
    |> Seq.scan (fun (i1, x1) (i2, x2) -> // Scan; replacing  index with the previous index when not a space
        match x1 with
        | ' ' -> (i2, x2)
        | _ -> (i1, x2)
    ) (0, ' ')
    |> Seq.filter (fun (_, x) -> x <> ' ') // Strip the spaces
    |> Seq.groupBy (fun (i, _) -> i) // Group by the word start index
    |> Seq.map (fun (i, x) -> (i, x |> Seq.map snd |> string)) // Strip redundant indexes; convert to string

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Opt-in Nulls; an F# Feature Worth Switching For?

2013-09-14T00:00:00+00:00

Recently I’ve been learning a little F#. This is actually the third time I’ve tried to pick it up… The first two times I just couldn’t get my head around it. This time, I think it’s making sense! Whether or not I ever get to use it commercially/in production, I think it’s been a valuable excercise. Just the basics of F# I’ve picked up so far have already start influencing how I’m writing C# in a good way.

There are a lot of things in F# that would save me a lot of time and/or drastically improve quality. One such feature is the handling of “nulls” using the Option type.

The Option type is similar to the Nullable<T> type used nullable with value types in .NET, but it can be applied to reference types too. Because of this, types are (by default) not nullable in F# ( generally …).

Take the following code that prints out a customers name:

type Customer = { Name : string }

let printCustomerName customer =
    printfn "Customer name is %s" customer.Name

In C#, this code would need to have a null check before accessing the Name property to avoid a possible NullReferenceException. In F#, because the argument is a Customer and not a Customer option, this check is not needed. The F# compiler will ensure that only Customer types can be passed as an argument to this function (note: if you’re consuming this code from another .NET language, all bets are off!).

I recently gave a talk to my colleagues about some of the interesting F# features I’d come across so far; the Option type being one of them. After the talk, one of my colleagues was debugging a NullReferenceException and I (jokingly) said that every time I hear someone debugging a NullReferenceException or see a null check, I’m going to exclaim “You wouldn’t have that in F#!”.

Well; it’s been only a week. Already; the number of times this thought has popped into my head is way above what I thought it would be. The number of null checks going into our code on a daily basis and the number of NullReferenceExceptions raised by our (sadly, very legacy) codebase is staggering. I don’t know why we put up for this crap for so long! Runtime NullReferenceExceptions are embarassing, and thousands of null checks just add to code noise. Why can’t we have our cake and eat it?

This is just one of many F# features that I geniunely believe could have a big enough impact on your code (both quality, and readability) that it should be a factor when picking your programming language. It has access to the same BCL that C# does. It has access to the same third party libraries and NuGet packages. It can even interact with your existing C# code. Unfortunately, there are still some significant reasons not to switch; the tooling is nowhere near as good as with C# (even only considering stock VS functionality), and StackOverflow has only a fraction of the answers you’ll want in F# vs C#. Both of these will improve in time and I think the idea of using F# for much of your .NET coding will become even more compelling.

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Migrating my blog from Google App Engine to GitHub Pages and Jekyll

2013-09-08T00:00:00+00:00

I’ve been trying to migrate my blog from Google App Engine to something a little more manageable (and not tied to GAE infrastructure) for some time. However, all of my attempts to rewrite the blog in ASP.NET have failed due to newer versions of “things” coming out, making me start over before I get to the end. I’ve come to the conclusion that coding my own blog is getting in the way of me blogging.

However; yesterday I came across Jekyll, which runs on GitHub Pages. This combination solves a lot of my blog requirements without me having to do anything!

It’s free to host on GitHub pages
All of my existing posts can stay in HTML
My new posts can be written in Markdown! (finally! this one is completely Markdown :))
It’s not tied to GitHub, or even Jekyll. I can take the static HTML version of my site and put it anywhere (or replace it with a dynamic one in future!)
Code formatting support

Hopefully; if my blog is low-maintenance, I might actually get back to blogging a little more! Sadly, there are a few little drawbacks:

Jekyll doesn’t support tag/category listing/index pages out-of-the-box, and GitHub has plugins disabled. I came up with a workaround for this (as you can see, I have tags in the sidebar that link), but it relies on creating a stub page for each tag; but that doesn’t seem too bad compared to having to run Jekyll locally and just publish the build output! I might blog the details of this once everything is fully working.
All of my posts (including those that have .html extensions) end up with trailing slashes in Jekyll. I can’t explain this, but they get 301 redirects due to the folder names, so I don’t think it’s a huge problem.

I’ve still got some tidying up to do, but maybe now that my blog posts can just be text files in a repo, I’ll blog a little more!

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Fixing Duplicate Contacts in Android People app from Google Contacts and Google+

2013-06-11T00:00:00+00:00

Now that I've swapped my Windows Phone for a Nexus 4, I've discovered what a mess my Google Contacts are in once synced to Android and enabling Google+ to supplement contact data. It didn't matter too much on the Nexus 7 tablet, but on the phone it's more frequently used, so having the same contacts appear three times is a problem!

Note: The People app has a "join" feature that lets you merge contacts on the device; however this is stored on the device and not synced anywhere. With multiple devices (and the possibility of buying new devices/factory resets/etc.), this isn't an ideal solution, so I wanted to fix the problem at the source.

I did all the usual stuff (merging duplicates in the Google Contacts web app), but still found some contacts where I had 2, 3 or even 4 copies once they made it to Android! I spent many hours going through making tiny changes to contacts to figure out what was causing the duplicates, and I've listed all the things that bit me below.

"Email Contacts" in Google+ Circles
I found a number of circles that had "email" contacts in. When Google+ first launched, I just put people suggested by Google into circles. These often turned out to be non-Gmail email addresses; and the person subsequently joined Google+, making two contacts with the same name/email. The fix is easy; just remove all email contacts from circles.
People with multiple Google+ Profiles
Because Google are pushing us into upgrading all Google accounts into Google+ Profiles (eg. you can't share pictures with Hangouts without this; despite it working in Google Talk), many people have two accounts (especially if they use Google Apps for work). To fix this; I just uncircled all but what appeared to be the "main" account for anybody with multiple.
Google Contact records with different names to Google+
This one is pretty stupid. I had my parents named "Mum" and "Dad" in Gooogle Contacts, and some contacts as Mike and Matt that were Michael and Matthew in Gooogle+. This seemed to confuse the Google+ sync, and showed them as separate contacts in the People app. I couldn't find a fix for this; so mum and dad now have their full names in People :(
Google+ syncs stale data!
This one bit me the hardest. Every time I thought I'd fixed the issue, dupes would reappear a few minutes later. I believe the problem was that the Google+ app cached circle data (so all my changes to circles for the above where stale) and then syncs it into People. To fix this, after each change, I did "Clear Data" on the Google+ app (in addition to the Contacts and Contacts Storage).
Multiple profile URLs against a Google Contact
This is similar to a contact having multiple Google+ accounts. I found that when I'd added some contacts to circles, it had copied their profile URL into the contact record (under URLs, with a type of Profile). When I'd merged contacts, I ended up with multiple of these against the same record. This also caused the contact to appear twice! Fixing this was easy; I just deleted all profile URLs against all contacts, unless it was the only piece of information tying them to their Google+ profile (in most cases, I had the email address, so this was not needed).

After all these changes, I've cleared data/caches and forced syncs several times no both devices, and haven't (yet) seen any contacts duplicated again :-)

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Farewell Windows Phone (You can't say I didn't try!). Hello Android!

2013-06-11T00:00:00+00:00

After the craziness of a previous Windows Phone blog entry ("Why I'm Close to Giving Up on Windows Phone 7, as a User and a Developer"), I thought it was worth posting an update, now that my "relationship" with Windows Phone has come to an end. Recently I placed an order for a Google/LG Nexus 4 Android phone...

You can't say I didn't give Microsoft a fair shot. You can read the full history in this post; and I continued to use Windows Phone for more than another two years since then. Windows Phone wasn't the great MS phone it should've been (and that I was expecting it to be). It's time to move on. I know a lot of naysayers in the comments of the original post will be saying "Told you so", and you were right. But I had to give it a try :)

Anyone that follows me on Twitter or Google+ probably knows this has been a long time coming. I've been a big fan of Android since swapping my iPad for a Nexus 7. In many ways, Windows Phone feels to me now, how my iPad felt to me then; kinda restricted and "doesn't do a lot". I expect the change to Android for my phone will bring the same sort of advantages and increased *ahem* "productivity" (where productivity means, email, social, gaming and other non-productive stuff) that changing my tablet gave.

To be constructive, here's some of the reasons:

Reasons Android seems better to me than Windows Phone

Not enough investment from Microsoft in Windows Phone.
Only a few major updates since launch, and they added nonsense stuff like different sized icons and made the browser less crappy.
Not enough investment from third parties in Windows Phone.
eg. Google just launched Hangouts on Android, iOS, Web, but no Windows Phone; no Google+ on Windows Phone. It's not just Google; other big apps are still missing.
My annoying Bluetooth issue still exists in Windows Phone.
Even on the Lumia 900 I was Kindly sent by Microsoft (they thought my issue was isolated to HTC, not Windows Phone) it's still bust, and there's no way I can help debug :o( It's early days, but I haven't yet had the issue with Android, so it seems more likely WP than my Car (the problem also didn't occur on iPhone).
Microsoft Web apps are clunky.
Google's web apps (Gmail, GDocs/Drive, Calendar) all work really well (even if they're not very pretty), and sync nicely with the Android apps. The last time I tried the equiv online apps from Microsoft (Hotmail/Outlook, online Office) they were very fat and clunky. Being able to do things on the web when I'm at a PC is a big deal; some things are just tiresome on a mobile device. I want to be able to switch devices seamlessly.
Google Hangouts!
It's only just launched, but I think Hangouts have a bright future. It's like iMessage, but works on iOS, Android and the web! It's a real bummer that Google didn't include Windows Phone support, but I'm really liking this a lot. The way mobile/desktop work together is very nice; such as not appearing "online" from a mobile device, which cuts down on random "chat" messages. Being able to switch device between PC, Tablet, Mobile is killer.
Bluetooth Audio Streaming Quality is terrible on Windows Phone
The audio quality of BT streaming on my Windows Phones was terrible. This wasn't so bad for phone calls; but made it unusable for music. I thought it might've been HTC-specific, but had the same reason with the Lumia. My Nexus has no such issue; and I'm able to happily play music from Google Music without any quality issues :-)
Faster Access to Updates
With a Nexus device, I know I'll get OS updates fairly quickly. Microsoft are still bending over for carriers and letting them block updates. No excuse for this.

There are many other niggles I had with Windows Phone; I find myself rarely using it for anything other than calls/texts because it feels limited and slow. Whenever I go to the Windows Phone marketplace, I'm amazed at how basic it is; how little information/options/etc. (and it's so slow!). It almost feels unfinished :(

Additionally, I'm quite a big user of Google services (eg.; Gmail/Calendar/Docs), so having native apps is nice (for example; the Gmail app has nice support for Gmail-native features, like switching From addresses and the new automatic categories).

The auto-backup of photos with Google+ works really well too; and ironically, the day after receiving my Nexus 4, my daughter spilled juice over my wife's Windows Phone and completely fried it, and we've been unable to recover the photos from it. I know Windows Phone can auto-backup too (wife's fault for not enabling it, or mine for not telling her, I guess!), but I believe it won't do full-res versions of the photo, which is kinda crap.

That said; I must say there are things I'm going to miss...

Things I'll miss about Windows Phone

SMS messages read to me in my car.
I love that Windows Phone has built-in functionality to read text messages to me via Bluetooth in my car. Replying has never worked for me (it can never tell what I'm saying, and just rudely hangs up after a few tries), but just having them read has been very useful. Looks like there are third party apps that do this on Android, but there's always that additional confidence you have in built-in functionality :(
The dedicated camera button.
This is a big one for me, because I have a 1yr old daughter and a week old son. Being able to launch the camera really quickly is killer.
Update: Since receiving my Nexus 4, I discovered there was a pre-configured Camera widget on the lock screen; I just tap the power button, then swipe left, and it launches the camera. Although it's two actions, it's about the same (or less) time than the WP long-press on the camera button, so it's actually not been the issue I thought it would :-)
The people hub.
Having my phone know that this email address, this twitter user, this facebook user etc. are all the same person is killer. Android + iOS have work to do here. Facebook doesn't sync contacts (despite showing the option too), nor does Twitter. My People app just shows Google contacts :(
Update: Since receiving my Nexus 4, I discovered that Android does support this; but Facebook and Twitter just don't use it (though there are third party apps for Facebook at least). So it's not as bad as I thought, especially as I had Twitter contacts hidden on WP because there are so many people I don't really "know".
Live Tiles.
Not for anything useful, but little touches like the Pictures tile showing a random photo I've taken (usually of my daughter pulling a cheeky face); really makes me smile :-) There are some photo widgets on Android, but none of them really behave in a similar way that I find them worth having.

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Fix for Google Music showing prices in USD and US-only services for non-US users

2013-06-09T00:00:00+00:00

Many people gained access to Google Music before it launched in their country by first accessing it from the US (sometimes via a VPN, EC2 instance, etc.). When Google Music launched "for real" in their countries, many (like me), found that that service believed they were in the US, and offered US-only services (such as All Access), and showed US-only offers, and prices in USD.

After many months of exchanging emails with Google Play support and trying all sorts of things, I was ultimately (and repeatedly) told they were aware of the issue, but did not yet have a fix :(

Recently, I was able to solve this issue, with the help from some info posted to an XDA-Developers forum by a poster named Movisman. Although the instructions didn't work for me as posted, it triggered a thought that lead to the fix. It actually turned out to be quite a simple fix:

Purchase a non-free track, using a card based on your home country as payment. Do not use Google Play credit!

All of my previous purchases had used the free credit that I got when I purchased my Nexus 7. As soon as I purchased a track using a UK card, the options for All Access vanished and all the recommendation prices changed from USD to GBP!

Since posting this fix in the forum, a number of other people have repeated this success. I'm posting it here in the hope it'll turn up in Google searches to help others. I've also notified Google Play support of how we fixed it (and provided a link to the forum) in the hope they can more quickly give a fix to anybody else reporting it.

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

PowerShell function to launch Kiln/BitBucket/Google Code/etc. for current Mercurial repo from command line

2013-04-22T00:00:00+00:00

A small, but useful, PowerShell function that I have in my PowerShell profile that reads the default repo path from .hg\hgrc and launches it in the default browser. This means after I've done hg push I can just ype kiln to quickly get to the repo page to raise code reviews, etc.

# Launch Kiln for current repo
Function Kiln
{
	if (Test-Path ".\.hg\hgrc")
	{
		$repoUrl = (Select-String "default = (.*)" -Path ".\.hg\hgrc" -AllMatches).Matches.Groups[1].Value
		Start $repoUrl
	}
	else
		{ Write-Warning "Not in a repo!" }
}

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Thoughts on TypeScript, Dart, and the future of browser-based coding

2012-10-13T00:00:00+00:00

Had a bit of a discussion on Twitter with Kevin Darty about TypeScript. It started off with this tweet:

Please RT if you actually know JavaScript and don't need a crutch like CoffeeScript, Dart or TypeScript.
— Kevin Darty (@kdarty) October 12, 2012

I'm still a bit puzzled by all the hate for TypeScript. There seem to be a lot of NaySayers, but I've still yet to see a single example of a codebase that couldn't benefit from TypeScript. Since it's a superset of JS, you don't have to do anything to start using it except rename your .js files to .ts, and then you can add TypeScript only where you think it will add value.

It's like having a magic switch that helps you find errors and refactor. How can it possibly be a bad thing?

I don't buy the "it'll stop people learning proper JavaScript" argument. I don't know a huge amount of Assembly or IL. Sure, there might be situations were knowing more of them would be useful, but is it really worth the time investment when I could instead be improving my C#/other skills that I use every single day?

Does it matter if I can't type javascript without looking it up to hook up subclass prototypes in JavaScript if TypeScript can do it for me? No. Does it help if I understand the generated code? Sure. Is it a requirement? No. All that matters is I understand how it behaves and any limitations of it. The same way I don't need to understand 100% of the IL generated when I write C#.

I'd actually argue that TypeScript will actually improve most peoples ability to write this kind of JavaScript. Think of the workflow... Chances are, you'll store both .ts and .js files in source control (that's how I'd probably do it, so code reviews can see the actual code going to the browser, and we don't have to worry about devs seeing different code to what's being deployed by the CI server). That means when you're viewing your diffs prior to committing, you'll review both your changes and the generated changes from TypeScript. I'd say this will teach you how these things work in JS. You'll become more aware of how sub-classes work, the behaviour of "this" in event handlers, etc.

I'm still generally curious; I'd love to see someone present a codebase that can't benefit from TypeScript and prove me wrong. I genuinely can't think of how anyone could write JavaScript to run in the browser that wouldn't benefit from type-checking (whether it by their own objects, or access to the DOM) in any way. If you're a NaySayer, then take this challenge! :-)

There was also some discussion about Dart following on from +Kevin Darty's tweet, like:

@cdhowie what about how MSFT Devs everywhere ripped Google a new one for Dart but are embracing TypeScript with open arms. Hypocritical?
— Kevin Darty (@kdarty) October 13, 2012

I can't speak for all MS devs, but I thought it worth sharing my opinion... I certainly LOL'd at the Hell World JavaScript created from Dart, but that was kinda funny (moreso with recent talk of Tree Shaking being a huge advantage of Dart). In general, however, I think Dart is a good idea, but possibly this effort could be better spent doing things slightly differently (more on that later).

I think Dart is a very ambitious project. I really hope it catches on and we see support in all browsers. However, I just can't see it happening :O( Dart and TypeScript solve some of the same issues but coming from the exact opposite sides.

TypeScript is designed to be compatible with JavaScript so you can use all of the existing JS libraries out there, and slowly/progressively add TypeScript to your code.

Dart goes further, and also provides a runtime, which brings huge performance benefits (where it can be executed natively). It compiles to JavaScript for compatibility (a necessity to ever gain traction). It tries to do so much more than TypeScript, however the trade-off is that it doesn't integrate with existing JS libraries.

In an ideal world, the decision to use something like TypeScript vs Dart should be based on things like:

Whether a Dart runtime is available to the majority of your users
Whether you need to interact with a lot of existing JS libraries
Whether you need to interact with a lot of your own JS (legacy projects)
Whether you can invest the time required to learn Dart (there's a lot less to learn with TypeScript)

However, I can't help but think that the decision is actually going to be based on one thing only:

Are you using Visual Studio or an Open Source IDE?

I just can't see TypeScript support ever being as good in non-VS IDEs. Nor can I see Dart support in VS ever being as good as it will elsewhere.

I actually think we could do much better than Dart or TypeScript. Scott Hanselman described JavaScript as the "Assembly Language for the Web". I think this is the first problem to address. We need something that runs in all browsers that addresses the issues with JavaScript, but it should be more like an intermediate language that other languages/tools can compile to (like we currently do with JavaScript, but less shit). That way, the choice of language can be decoupled from the browser support. If you want to write C#, VB.NET, Dart, Java, etc., it shouldn't matter. As long as they can all be compiled to a standard "IL", they should be deployable to a browser.

I don't know whether any of the existing technologies are good candidates for this (could we compile C#/Java/etc. into Dart? NaCl?), but I think it would be the best solution for all. People could use the IDE and languages they feel most comfortable/productive with.

In the meantime, compiling to JavaScript is the only thing we can do for compatibility. But in the background, I wish the standards bodies and browser vendors could get their heads together and agree on a new way of executing code in the browser in a way that they can all offer the experiences they wish to their devs. It doesn't matter whether it's based on something existing or new; the important thing is that it's not tied to the web development language and that the browser vendors and the companies with huge influences in development get behind it. If we can develop desktop apps and server-side code in the language of our choice; why can't we do the same for the browser? This would allow huge amounts of code reuse across our applications that we don't currently get.

Chrome already has NaCl, and will presumably have native Dart support soon. I wouldn't be surprised if IE ended up with native TypeScript support. This is going backwards, not forwards.

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Fixing adb "Device not found" with Nexus 7/Android devices in Recovery Mode

2012-10-11T00:00:00+00:00

I'm a huge fan of the Nexus 7, but one of the things that annoyed me from day one was the lack of landscape support on the homescreen. I almost always use the tablet in landscape mode, so when switching to an app I didn't already have in the open/recent apps list, I would have to hit the Home button; which flipped everything around temporarily. There was already a user-set orientation lock, and we knew Android could handle it (from other devices, and those with rooted devices changing build properties), so it always seemed like a really random restriction.

This was announced as fixed in Android 4.1.2, which began rolling out only a few days ago. As of this evening, my device still showed no update, so after finding some instructions, I decided to load the update myself.

These instructions do *not* require you to root your device or unlock the bootloader. If your device is unlocked/rooted, I have no idea what affect that might have on this process. If you're unsure about this, don't do it. I accept no liability if you mess things up. I have only limited knowledge of Android! The steps seemed fairly straight forward, which is the only reason I was attempting them!

My first problem was that I was unable to boot into recovery mode using the instructions. When I selected Recovery Mode from the boot menu, my device just booted normally. I didn't get the Android with the red exclamation mark. This was easily fixed by using adb (installed as part of the Android SDK) and running:

adb reboot recovery

This rebooted the device to the screen showing an Android with a red exclamation mark, and pressing Power + Up allowed me to select the "Update from ADB" option.

The problem was, once in this mode, Windows failed to recognise the decide/load the drivers that were previously loaded (before the reboot), which meant adb would no longer be able to connect the device:

adb: device not found

After a lot of searching online, I figured out that the problem was that the Hardware ID of my Nexus 7 when in recovery mode was not listed in "android_winusb.inf" file from the Google Android Windows USB Drivers. The ID I saw in device manager against the Nexus was USB\VID_18D1&PID_D001, which didn't turn a lot up in Google!

I decided to try adding this to the inf file manually (C:\Program Files (x86)\Android\android-sdk\extras\google\usb_driver\android_winusb.inf). As I'm running on a 64bit machine, I added the following to the [Google.NTamd64] section (note: this applies to Intel 64 bit machines, not just AMDs!). If you're using 32bit, you'd want to add it to the [Google.NTx86] section:

[Google.NTamd64]

; !-- snip other devices... --!

; Google Nexus 7
; !-- snip existing device IDs --!
; This is the HardwareID shown in Device Manager for the "Nexus"
;  device when in Recovery mode
%CompositeAdbInterface%     = USB_Install, USB\VID_18D1&PID_D001

After making this change, I uninstalled the device drivers, then reinstalled them using this inf file. I once again rebooted into Recovery mode, and this time, Windows correctly displayed the device as "Android ADB Interface", and I was then able to sideload the Android 4.1.2 update (downloaded directly from Google) by typing:

adb sideload 03a4eaf95f73.signed-nakasi-JZO54K-from-JRO03D.03a4eaf9.zip

ADB transmitted the file to the Nexus 7, then the Nexus spat out the progress as it verified the image and updated the device. A reboot later, and my Nexus 7 was running Android 4.1.2!

This post was served up via my RSS feed. Please visit the original article to read/post comments. If you found this article interesting, why not follow @DanTup on Twitter for more? :)

Danny Tuppeny

Dart Code v2 - Now with Flutter debugging and hot reload!

Download

Feedback

Thanks

Up Next - Web Support

I found a ridiculous bug in YouTube and seem unable to successfully report it :(

Simple Dart code to tweet using OAuth

Making more money with Amazon: Splitting visitors between Amazon US/UK based on their location

Visiting a site that uses Disqus comments when not logged in sends the URL to Facebook

Using Alexa (Amazon Echo) to Preheat my Renault ZOE and check Battery Status

Simplest C# code to post a tweet using OAuth

HTML5 Pong in 20min with C#/Bridge.NET

DaChip8JS - My C# Chip-8 Interpreter running in the browser

Live Demo

Video

Code Changes

C# 5

Key and Touch Mappings

Game Loop Changes

ROM Loading

Rendering Changes

Sound Changes

Touch Support

Building a Chip-8 Interpreter in C#

Running Go on a Chromebook in Developer Mode (without installing Linux)

Enable Developer Mode

Download and Extract Go

Set Execute Permissions for the File System

Set Environment Variables

Restart and Test

Hello, World!

Easy JavaScript and CSS bundling and minification on GitHub Pages without build steps

Have Software Developers Given Up?

Removing [action 'action 17' suspended] rsyslog Spam on Raspberry Pi (Raspian Jessie)

Setting up Automatic Updates on Raspberry Pi (Raspian Jessie)

Setting up a Raspberry Pi (Raspian Jessie) to send Email

Not so fast…

Installing Lighttpd, PHP 7 and LetsEncrypt on a Raspberry Pi (Raspbian Jessie Lite)

Goals

Replacements

Complications

Getting Raspbian

Security Considerations

Setting up Raspbian

Switch to root shell

Installing Lighttpd

Installing packages from Debian

Install the LetsEncrypt Client

Requesting TLS Certs

Setting up TLS in Lighttpd

Installing PHP 7

Automatic TLS Cert Renewal

That’s It!

My Week in a Tesla Model S P85

Exterior

Performance

Interior

Software/Touchscreen

Boot/Frunk Storage

Regen Braking

Gadgets

Negatives

Other Opinions

Summary

Going Back to the ZOE

Simple Windows utility to act as default browser and launch different browsers based on domain

Response from CYC

GMEV and CYC - The UKs Awful Recharging Network

Sign up on the website, can’t login to app…

App security issues

Fraudulent £1 charges from CYC

RFID cards attached to the wrong accounts

Unreliable network (Birkenhead Ferry Terminal)

Confusion over whether fees are annual or one-off

GMEV cards not available to people who live near Manchester

Incorrect charger location (Hulme Street)

No charger number (Hulme Street)

Unreliable parking fee info (Hulme Street)

Unreliable network - no charge (Hulme Street)