tag:blog.dantup.com,2009:/2022-06-20T11:05:44+00:00Danny TuppenyDanny Tuppenyhttps://blog.dantup.com/tag:blog.dantup.com,2021-04-25:/2021/04/cardano-producer-using-kubernetes/2021-04-25T00:00:00+00:002021-04-25T00:00:00+00:00Setting up a Cardano Producer node using Kubernetes/microk8s
<p>I previously blogged about <a href="/2021/03/cardano-relays-using-kubernetes/">setting up</a>, <a href="/2021/03/monitoring-cardano-relays-on-kubernetes-with-grafana-and-prometheus/">monitoring</a> and <a href="/2021/04/using-kubernetes-configmaps-for-cardano-node-topology-config/">using ConfigMaps</a> for a Cardano relay using Kubernetes. This post describes the changes to also run a Producer node, which connects to the relay and uses Kubernetes secrets for its keys.</p>
<p>Posts in this series:</p>
<ul>
<li><a href="/2021/03/cardano-relays-using-kubernetes/">Step 1: Setting up Cardano Relays using Kubernetes/microk8s</a></li>
<li><a href="/2021/03/monitoring-cardano-relays-on-kubernetes-with-grafana-and-prometheus/">Step 2: Monitoring Cardano Relays on Kubernetes with Grafana and Prometheus</a></li>
<li><a href="/2021/04/using-kubernetes-configmaps-for-cardano-node-topology-config/">Step 3: Using Kubernetes ConfigMaps for Cardano Node Topology Config</a></li>
<li>Step 4: Setting up a Cardano Producer node using Kubernetes/microk8s</li>
</ul>
<blockquote>
<p>If you find this post useful or are looking for somewhere to delegate while setting up your own pool, <a href="https://cardano.dantup.com/">check out my pool [CODER]</a>! š</p>
</blockquote>
<h2 id="creating-and-registering-a-pool-and-cold-keys">Creating and Registering a Pool and Cold Keys</h2>
<p>This post assumes you have already set up your pool and create the required keys and focuses only on setting up the Producer node in Kubernetes. If you do not already have your pool set up you should follow the <a href="https://docs.cardano.org/en/latest/getting-started/stake-pool-operators/index.html">official Cardano docs</a>.</p>
<p>Be sure to use an offline machine for the creation of the cold keys and ensure you keep secure backups. If you have a significant pledge I would consider using a <a href="https://cryptowalletschart.com/">Crypto Hardware wallet</a> to secure it (and for the rewards address) instead of the wallet on the cold machine (ensure you have good security/backups of the hardware seed phrase!). <a href="https://adalite.medium.com/cardano-stake-pool-owners-hw-support-6d9278dba0ba">This AdaLite post</a> has some good instructions for how to include a hardware wallet as a pool owner. Doing this will reduce the loss if your cold machine was lost/compromised to just the deposit and future rewards rather than also existing unclaimed rewards and the pledge.</p>
<h2 id="creating-kubernetes-secrets">Creating Kubernetes Secrets</h2>
<p>After following the pool creation instructions above, you should have three files from your cold environment that your Producer node will need:</p>
<ul>
<li>kes.skey</li>
<li>vrf.skey</li>
<li>node.cert</li>
</ul>
<p>No other files from the cold machine should be used on the producer (although they <em>should</em> be safely backed up from the cold machine).</p>
<p>These three files will be stored in Kubernetes as <a href="https://kubernetes.io/docs/concepts/configuration/secret/">Secrets</a>. The default is for Secrets to be stored base64 encoded in Kubernetes. This is <em>not</em> encryption - it is simply to support binary files. For simplicity this post will stick with base64 but you should consider <a href="https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/#configuration-and-determining-whether-encryption-at-rest-is-already-enabled">enabling encryption for Secrets</a> and ensure your config files are secure and cannot be read by anything that does not need them.</p>
<p>If these three files are compromised, it does not give up control of your pool, your deposit or your pledge. However it would allow someone else to run a producer for your pool (which could result in short-lived forks that negatively impact the network) or calculate which slots your pool will produce (making a DoS attack a little easier).</p>
<p><strong>DO NOT use online web pages for base64 encoding Kubernetes secrets!</strong></p>
<p>To base64 encode the contents of a file, you can use the <code class="language-plaintext highlighter-rouge">base64</code> command:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ echo 'this is a test' > testfile.txt
$ base64 -i testfile.txt
dGhpcyBpcyBhIHRlc3QK
</code></pre></div></div>
<p>You can verify the contents using <code class="language-plaintext highlighter-rouge">base64 -d</code></p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ echo "dGhpcyBpcyBhIHRlc3QK" | base64 -d
this is a test
</code></pre></div></div>
<p>These values can then be added to a new Kubernetes config file along with the rest of your config using the names weād like to use as filenames when mounting these into the pod as the keys:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Secret</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">mainnet-producer-keys</span>
<span class="na">data</span><span class="pi">:</span>
<span class="s">kes.skey</span><span class="pi">:</span> <span class="s">ewogXXXXXXXXXXXXXXXXXXXXXXXX==</span> <span class="c1"># replace with base64 encoded version of this file</span>
<span class="s">vrf.skey</span><span class="pi">:</span> <span class="s">ICJ0XXXXXXXXXXXXXXXXXXXXXXX==</span> <span class="c1"># replace with base64 encoded version of this file</span>
<span class="s">node.cert</span><span class="pi">:</span> <span class="s">eXBXXXXXXXXXXXXXXXXXXXXXXXX==</span> <span class="c1"># replace with base64 encoded version of this file</span>
</code></pre></div></div>
<h2 id="node-volume">Node Volume</h2>
<p>Extend the <a href="https://blog.dantup.com/2021/03/cardano-relays-using-kubernetes/#node-volumes">node volumes</a> config created for the relays to include a similar definition for the producer. The producer will need its own data folder, though to speed up the initial sync you may wish to copy the <code class="language-plaintext highlighter-rouge">db</code> folder in from an existing relay. Youāll also want to copy the <code class="language-plaintext highlighter-rouge">configuration</code> folder since that will likely be the same for each node <a href="/2021/04/using-kubernetes-configmaps-for-cardano-node-topology-config/">now weāre using ConfigMaps for topology</a>.</p>
<p>If youāre using a local node folder for storage, you should again set <code class="language-plaintext highlighter-rouge">nodeAffinity</code> to ensure this pod always runs on the same node (bear in mind this means the pod wonāt run if this node is unavailable, but redundant storage is outside of the scope of this post).</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">PersistentVolume</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">cardano-mainnet-producer-data-pv-1</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">capacity</span><span class="pi">:</span>
<span class="na">storage</span><span class="pi">:</span> <span class="s">25Gi</span>
<span class="na">accessModes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">ReadWriteOnce</span>
<span class="na">storageClassName</span><span class="pi">:</span> <span class="s">local-storage-cardano-mainnet-producer</span>
<span class="c1"># This volume is on the host named "mario"</span>
<span class="na">local</span><span class="pi">:</span>
<span class="na">path</span><span class="pi">:</span> <span class="s">/home/danny/cardano/producer-mainnet/data</span>
<span class="na">nodeAffinity</span><span class="pi">:</span>
<span class="na">required</span><span class="pi">:</span>
<span class="na">nodeSelectorTerms</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">matchExpressions</span><span class="pi">:</span>
<span class="err"> </span> <span class="c1"># Restrict this volume to a specific Kubernetes node by hostname</span>
<span class="pi">-</span> <span class="na">key</span><span class="pi">:</span> <span class="s">kubernetes.io/hostname</span>
<span class="na">operator</span><span class="pi">:</span> <span class="s">In</span>
<span class="na">values</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">mario</span> <span class="c1"># hostname for this volume</span>
<span class="nn">---</span>
<span class="na">apiVersion</span><span class="pi">:</span> <span class="s">storage.k8s.io/v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">StorageClass</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">local-storage-cardano-mainnet-producer</span>
<span class="na">provisioner</span><span class="pi">:</span> <span class="s">kubernetes.io/no-provisioner</span>
<span class="c1"># https://kubernetes.io/docs/concepts/storage/storage-classes/#local</span>
<span class="na">volumeBindingMode</span><span class="pi">:</span> <span class="s">WaitForFirstConsumer</span>
</code></pre></div></div>
<h2 id="topology">Topology</h2>
<p>The <a href="/2021/04/using-kubernetes-configmaps-for-cardano-node-topology-config/">previously created topology ConfigMap file</a> needs updating to include config for the producer, and also to point the relay to the producer.</p>
<p>Iāve used the hostnames <code class="language-plaintext highlighter-rouge">cardano-mainnet-relay-service.default.svc.cluster.local</code> and <code class="language-plaintext highlighter-rouge">cardano-mainnet-producer-service.default.svc.cluster.local</code> in my config files, which Kubernetes DNS automatically resolves to the IP addresses of the services created for the relays/producer respectively. This avoids needing to hardcode specific IP addresses when first setting this up.</p>
<p>Iāve omitted my other peers and used just the default IOHK hostname to keep the example shorter.</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">ConfigMap</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">mainnet-producer-topology</span>
<span class="na">data</span><span class="pi">:</span>
<span class="s">topology.json</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">{</span>
<span class="s">"Producers": [</span>
<span class="s">{</span>
<span class="s">"addr": "cardano-mainnet-relay-service.default.svc.cluster.local",</span>
<span class="s">"port": 30801,</span>
<span class="s">"valency": 1</span>
<span class="s">}</span>
<span class="s">]</span>
<span class="s">}</span>
<span class="s">---</span>
<span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">ConfigMap</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">mainnet-relay-topology</span>
<span class="na">data</span><span class="pi">:</span>
<span class="s">topology.json</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">{</span>
<span class="s">"Producers": [</span>
<span class="s">{</span>
<span class="s">"addr": "cardano-mainnet-producer-service.default.svc.cluster.local",</span>
<span class="s">"port": 30800,</span>
<span class="s">"valency": 1</span>
<span class="s">},</span>
<span class="s">{</span>
<span class="s">"addr": "relays-new.cardano-mainnet.iohk.io",</span>
<span class="s">"port": 3001,</span>
<span class="s">"valency": 2</span>
<span class="s">}</span>
<span class="s">]</span>
<span class="s">}</span>
</code></pre></div></div>
<h2 id="statefulsetpod-definition">StatefulSet/Pod Definition</h2>
<p>The configuration for the producer is mostly the same as for the relay, with a few differences:</p>
<ul>
<li>Names and volumes are updated to reference āproducerā instead of ārelayā</li>
<li>A new volume is used to mount the secrets (keys) into the pod</li>
<li>Additional arguments are passed to the node containing paths to the keys</li>
<li>The exposed service does not bind a <code class="language-plaintext highlighter-rouge">NodePort</code> because incoming connections are only from inside the cluster (the relays)</li>
</ul>
<p>My full config looks like this:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">apps/v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">StatefulSet</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">cardano-mainnet-producer-deployment</span>
<span class="na">labels</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">cardano-mainnet-producer-deployment</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">serviceName</span><span class="pi">:</span> <span class="s">cardano-mainnet-producer</span>
<span class="na">replicas</span><span class="pi">:</span> <span class="m">1</span>
<span class="na">selector</span><span class="pi">:</span>
<span class="na">matchLabels</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">cardano-mainnet-node</span>
<span class="na">cardano-mainnet-node-type</span><span class="pi">:</span> <span class="s">producer</span>
<span class="na">template</span><span class="pi">:</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">labels</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">cardano-mainnet-node</span>
<span class="na">cardano-mainnet-node-type</span><span class="pi">:</span> <span class="s">producer</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">containers</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">cardano-mainnet-producer</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">inputoutput/cardano-node</span>
<span class="na">imagePullPolicy</span><span class="pi">:</span> <span class="s">Always</span>
<span class="err"> </span> <span class="c1">####### Additional arguments for keys</span>
<span class="na">args</span><span class="pi">:</span> <span class="pi">[</span><span class="s2">"</span><span class="s">run"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">--config"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">/data/configuration/mainnet-config.json"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">--topology"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">/topology/topology.json"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">--database-path"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">/data/db"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">--socket-path"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">/data/node.socket"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">--port"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">4000"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">--shelley-kes-key"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">/keys/kes.skey"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">--shelley-vrf-key"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">/keys/vrf.skey"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">--shelley-operational-certificate"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">/keys/node.cert"</span><span class="pi">]</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">containerPort</span><span class="pi">:</span> <span class="m">12798</span>
<span class="pi">-</span> <span class="na">containerPort</span><span class="pi">:</span> <span class="m">4000</span>
<span class="na">volumeMounts</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">data</span>
<span class="na">mountPath</span><span class="pi">:</span> <span class="s">/data</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">topology</span>
<span class="na">mountPath</span><span class="pi">:</span> <span class="s">/topology</span>
<span class="err"> </span> <span class="c1">####### Additional volume mount for keys</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">keys</span>
<span class="na">mountPath</span><span class="pi">:</span> <span class="s">/keys</span>
<span class="na">readOnly</span><span class="pi">:</span> <span class="no">true</span>
<span class="na">volumes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">topology</span>
<span class="na">configMap</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">mainnet-producer-topology</span>
<span class="err"> </span><span class="c1">####### Additional volume for keys</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">keys</span>
<span class="na">secret</span><span class="pi">:</span>
<span class="na">secretName</span><span class="pi">:</span> <span class="s">mainnet-producer-keys</span>
<span class="na">defaultMode</span><span class="pi">:</span> <span class="m">0400</span>
<span class="err"> </span><span class="na">volumeClaimTemplates</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">data</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">accessModes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">ReadWriteOnce</span>
<span class="na">storageClassName</span><span class="pi">:</span> <span class="s">local-storage-cardano-mainnet-producer</span>
<span class="na">resources</span><span class="pi">:</span>
<span class="na">requests</span><span class="pi">:</span>
<span class="na">storage</span><span class="pi">:</span> <span class="s">25Gi</span>
<span class="nn">---</span>
<span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Service</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">cardano-mainnet-producer-service</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">NodePort</span>
<span class="na">selector</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">cardano-mainnet-node</span>
<span class="na">cardano-mainnet-node-type</span><span class="pi">:</span> <span class="s">producer</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">port</span><span class="pi">:</span> <span class="m">30800</span>
<span class="na">targetPort</span><span class="pi">:</span> <span class="m">4000</span>
</code></pre></div></div>
<p>With all of these files <code class="language-plaintext highlighter-rouge">kubectl apply -f</code>ād, the producer node should start up and being syncing. Since itās tagged with <code class="language-plaintext highlighter-rouge">cardano-mainnet-node</code>, it should automatically be picked up by the <a href="/2021/03/monitoring-cardano-relays-on-kubernetes-with-grafana-and-prometheus/">Prometheus config we previously set up</a> and show in Grafana when itās fully running.</p>
<p>I created a bash alias that allows me to quickly print the last few blocks from each node so I can quickly verify theyāre in-sync and the latency of them syncing blocks:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">alias </span>print-block-sync<span class="o">=</span><span class="s1">'kubectl logs pod/cardano-mainnet-relay-deployment-0 | grep extended | tail -n 5 | sed "s/cardano-:cardano.node.ChainDB:Notice/Relay/" && \
kubectl logs pod/cardano-mainnet-producer-deployment-0 | grep extended | tail -n 5 | sed "s/cardano-:cardano.node.ChainDB:Notice/Producer/"'</span>
</code></pre></div></div>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>print-block-sync
<span class="o">[</span>Relay 11:52:36.22] new tip: 92cb0868c150fa16602082b0aec3bc8d57a2edc5bbd5b3fe9ed7df401342e62b at slot 27785265
<span class="o">[</span>Relay 11:52:47.38] new tip: af5dedd7f5d0fd77581e14c7b50cb5c49005b801ff6e99522ed6b18f41f60230 at slot 27785276
<span class="o">[</span>Relay 11:52:54.37] new tip: d37c64c0dfae9f0dc465ed787719ed54e15e1145476cc296f64835cb6b6ef608 at slot 27785283
<span class="o">[</span>Relay 11:53:12.17] new tip: 3c5a5f0ab9843fb71d2bb83b53ec957e4b82b405c5b11a141e567d1f7ccae6cd at slot 27785301
<span class="o">[</span>Relay 11:53:18.25] new tip: 87b1b0b9a97e236476e9cefe8e7a926b66046a2a93f5aa3d96d5591f42a9e009 at slot 27785307
<span class="o">[</span>Producer 11:52:36.23] new tip: 92cb0868c150fa16602082b0aec3bc8d57a2edc5bbd5b3fe9ed7df401342e62b at slot 27785265
<span class="o">[</span>Producer 11:52:47.39] new tip: af5dedd7f5d0fd77581e14c7b50cb5c49005b801ff6e99522ed6b18f41f60230 at slot 27785276
<span class="o">[</span>Producer 11:52:54.38] new tip: d37c64c0dfae9f0dc465ed787719ed54e15e1145476cc296f64835cb6b6ef608 at slot 27785283
<span class="o">[</span>Producer 11:53:12.18] new tip: 3c5a5f0ab9843fb71d2bb83b53ec957e4b82b405c5b11a141e567d1f7ccae6cd at slot 27785301
<span class="o">[</span>Producer 11:53:18.26] new tip: 87b1b0b9a97e236476e9cefe8e7a926b66046a2a93f5aa3d96d5591f42a9e009 at slot 27785307
</code></pre></div></div>
<p>I also configured some additional panels and alerts in Grafana to easily monitor the remaining KES periods (itās import to periodically create new KES keys and copy them - along with an updated <code class="language-plaintext highlighter-rouge">node.cert</code> into the secrets file and re-apply them!).</p>
<p><a href="/post_images/cardano_k8s/grafana_with_kes.png">
<img src="/post_images/cardano_k8s/grafana_with_kes.png" alt="Cardano metrics on Grafana" width="910" />
</a></p>
<blockquote>
<p>If you find this post useful or are looking for somewhere to delegate while setting up your own pool, <a href="https://cardano.dantup.com/">check out my pool [CODER]</a>! š</p>
</blockquote>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2021/04/cardano-producer-using-kubernetes/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2021-04-18:/2021/04/using-kubernetes-configmaps-for-cardano-node-topology-config/2021-04-18T00:00:00+00:002021-04-18T00:00:00+00:00Using Kubernetes ConfigMaps for Cardano Node Topology Config
<p>I previously blogged about <a href="/2021/03/cardano-relays-using-kubernetes/">setting up</a> and <a href="/2021/03/monitoring-cardano-relays-on-kubernetes-with-grafana-and-prometheus/">monitoring</a> a Cardano relay using Kubernetes. Since Iām storing files on the hosts filesystem using a local <code class="language-plaintext highlighter-rouge">PersistentVolume</code> and was unable map a single configuration folder into all nodes (I found no explanation of this online, but having multiple pods try to mount a <code class="language-plaintext highlighter-rouge">ReadOnlyMany</code> volume seems to result in all but the first getting stuck pending), Iāve ended up with a copy of the configuration files for each node. I decided to move at least the topology config into a <code class="language-plaintext highlighter-rouge">ConfigMap</code> since it often changes (when I add/update peer relays from other SPOs) to avoid having to keep synchronising it between nodes.</p>
<p>Posts in this series:</p>
<ul>
<li><a href="/2021/03/cardano-relays-using-kubernetes/">Step 1: Setting up Cardano Relays using Kubernetes/microk8s</a></li>
<li><a href="/2021/03/monitoring-cardano-relays-on-kubernetes-with-grafana-and-prometheus/">Step 2: Monitoring Cardano Relays on Kubernetes with Grafana and Prometheus</a></li>
<li>Step 3: Using Kubernetes ConfigMaps for Cardano Node Topology Config</li>
<li><a href="/2021/04/cardano-producer-using-kubernetes/">Step 4: Setting up a Cardano Producer node using Kubernetes/microk8s</a></li>
</ul>
<blockquote>
<p>If you find this post useful or are looking for somewhere to delegate while setting up your own pool, <a href="https://cardano.dantup.com/">check out my pool [CODER]</a>! š</p>
</blockquote>
<h2 id="creating-a-configmap">Creating a ConfigMap</h2>
<p>Like the rest of the config, our <code class="language-plaintext highlighter-rouge">ConfigMap</code> will live in a <code class="language-plaintext highlighter-rouge">.yaml</code> file. Each <code class="language-plaintext highlighter-rouge">ConfigMap</code> needs a unique name and can contain multiple files.</p>
<p>Iāve omitted my peers and used just the default IOHK hostname to keep the example shorter.</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">ConfigMap</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">mainnet-relay-topology</span>
<span class="na">data</span><span class="pi">:</span>
<span class="s">topology.json</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">{</span>
<span class="s">"Producers": [</span>
<span class="s">{</span>
<span class="s">"addr": "relays-new.cardano-mainnet.iohk.io",</span>
<span class="s">"port": 3001,</span>
<span class="s">"valency": 2</span>
<span class="s">}</span>
<span class="s">]</span>
<span class="s">}</span>
</code></pre></div></div>
<p>Next we need to add the <code class="language-plaintext highlighter-rouge">ConfigMap</code>s to the <code class="language-plaintext highlighter-rouge">volume</code> and <code class="language-plaintext highlighter-rouge">volumeMounts</code> sections of the relay <code class="language-plaintext highlighter-rouge">StatefulSet</code> configs. My relays config now looks like this:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">apps/v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">StatefulSet</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">cardano-mainnet-relay-deployment</span>
<span class="na">labels</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">cardano-mainnet-relay-deployment</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">serviceName</span><span class="pi">:</span> <span class="s">cardano-mainnet-relay</span>
<span class="na">replicas</span><span class="pi">:</span> <span class="m">1</span>
<span class="na">selector</span><span class="pi">:</span>
<span class="na">matchLabels</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">cardano-mainnet-node</span>
<span class="na">cardano-mainnet-node-type</span><span class="pi">:</span> <span class="s">relay</span>
<span class="na">template</span><span class="pi">:</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">labels</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">cardano-mainnet-node</span>
<span class="na">cardano-mainnet-node-type</span><span class="pi">:</span> <span class="s">relay</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">containers</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">cardano-mainnet-relay</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">inputoutput/cardano-node</span>
<span class="na">imagePullPolicy</span><span class="pi">:</span> <span class="s">Always</span>
<span class="c1">############ The path '/topology/topology.json' here was updated</span>
<span class="na">args</span><span class="pi">:</span> <span class="pi">[</span><span class="s2">"</span><span class="s">run"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">--config"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">/data/configuration/mainnet-config.json"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">--topology"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">/topology/topology.json"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">--database-path"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">/data/db"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">--socket-path"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">/data/node.socket"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">--port"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">4000"</span><span class="pi">]</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">containerPort</span><span class="pi">:</span> <span class="m">12798</span>
<span class="pi">-</span> <span class="na">containerPort</span><span class="pi">:</span> <span class="m">4000</span>
<span class="na">volumeMounts</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">data</span>
<span class="na">mountPath</span><span class="pi">:</span> <span class="s">/data</span>
<span class="c1">############ NEWLY ADDED (START)</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">topology</span>
<span class="na">mountPath</span><span class="pi">:</span> <span class="s">/topology</span>
<span class="na">volumes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">topology</span>
<span class="na">configMap</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">mainnet-relay-topology</span>
<span class="c1">############ NEWLY ADDED (END)</span>
<span class="na">volumeClaimTemplates</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">data</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">accessModes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">ReadWriteOnce</span>
<span class="na">storageClassName</span><span class="pi">:</span> <span class="s">local-storage-cardano-mainnet-relay</span>
<span class="na">resources</span><span class="pi">:</span>
<span class="na">requests</span><span class="pi">:</span>
<span class="na">storage</span><span class="pi">:</span> <span class="s">25Gi</span>
</code></pre></div></div>
<p>And thatās all there was to it. I <code class="language-plaintext highlighter-rouge">microk8s.kubectl apply -f</code>ād the files, deleted the old topology files from disk and restarted the nodes. Checking the logs with <code class="language-plaintext highlighter-rouge">microk8s.kubectl logs pod/cardano-mainnet-relay-deployment-0</code> I saw the node connecting out and after a few minutes of loading the databases, appear back on <a href="/2021/03/monitoring-cardano-relays-on-kubernetes-with-grafana-and-prometheus/">Grafana</a> as processing transactions.</p>
<blockquote>
<p>If you find this post useful or are looking for somewhere to delegate while setting up your own pool, <a href="https://cardano.dantup.com/">check out my pool [CODER]</a>! š</p>
</blockquote>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2021/04/using-kubernetes-configmaps-for-cardano-node-topology-config/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2021-03-28:/2021/03/monitoring-cardano-relays-on-kubernetes-with-grafana-and-prometheus/2021-03-28T00:00:00+00:002021-03-28T00:00:00+00:00Monitoring Cardano Relays on Kubernetes with Grafana and Prometheus
<p>I <a href="/2021/03/cardano-relays-using-kubernetes/">previously blogged</a> about setting up a Cardano relay using Kubernetes. The IOHK cardano-node container image has built-in support for exposing Prometheus metrics and microk8s has addons for Grafana/Prometheus, so it made sense to use them together.</p>
<p>Posts in this series:</p>
<ul>
<li><a href="/2021/03/cardano-relays-using-kubernetes/">Step 1: Setting up Cardano Relays using Kubernetes/microk8s</a></li>
<li>Step 2: Monitoring Cardano Relays on Kubernetes with Grafana and Prometheus</li>
<li><a href="/2021/04/using-kubernetes-configmaps-for-cardano-node-topology-config/">Step 3: Using Kubernetes ConfigMaps for Cardano Node Topology Config</a></li>
<li><a href="/2021/04/cardano-producer-using-kubernetes/">Step 4: Setting up a Cardano Producer node using Kubernetes/microk8s</a></li>
</ul>
<blockquote>
<p>If you find this post useful or are looking for somewhere to delegate while setting up your own pool, <a href="https://cardano.dantup.com/">check out my pool [CODER]</a>! š</p>
</blockquote>
<h2 id="enabling-grafana-and-prometheus">Enabling Grafana and Prometheus</h2>
<p>microk8s comes with a number of <a href="https://microk8s.io/docs/addons">addons</a> to set up some common services. I installed <code class="language-plaintext highlighter-rouge">dashboard</code> (to help me poke around while learning), <code class="language-plaintext highlighter-rouge">dns</code> (to allow pods and services to discover each other using hostnames) and <code class="language-plaintext highlighter-rouge">prometheus</code> (to scrape metrics from both the Kubernetes nodes and the Cardano pods).</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>microk8s <span class="nb">enable </span>dns
microk8s <span class="nb">enable </span>dashboard
microk8s <span class="nb">enable </span>prometheus
</code></pre></div></div>
<p>By default, these services are only exposed within the cluster. In order to be able to access Grafana without having to SSH in and forward a port, I bound it to a <a href="https://kubernetes.io/docs/concepts/services-networking/service/#nodeport"><code class="language-plaintext highlighter-rouge">NodePort</code></a>.</p>
<p>I created a file <code class="language-plaintext highlighter-rouge">grafana-nodeport.yaml</code> and applied it with <code class="language-plaintext highlighter-rouge">microk8s.kubectl apply -f grafana-nodeport.yaml</code>.</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Service</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">grafana-nodeport</span>
<span class="na">namespace</span><span class="pi">:</span> <span class="s">monitoring</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">NodePort</span>
<span class="na">selector</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">grafana</span> <span class="c1"># The name of the grafana app, pre-configured by the microk8s addons</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">port</span><span class="pi">:</span> <span class="m">3000</span> <span class="c1"># The port grafana is listening on, pre-configured by the microk8s addon</span>
<span class="na">nodePort</span><span class="pi">:</span> <span class="m">30500</span> <span class="c1"># The port to expose on the host machine</span>
</code></pre></div></div>
<p>With this done, I hit <code class="language-plaintext highlighter-rouge">http://k8snode1:30500/</code> in the browser on my local machine and was greeted by a Grafana login page. The default username/password was <code class="language-plaintext highlighter-rouge">admin</code>/<code class="language-plaintext highlighter-rouge">admin</code> (youāre prompted to change the passed upon login) and once logged in I found many preconfigured dashboards for Kubernetes and Prometheus, including <strong>Nodes</strong> which gave a good overview of the nodes resources:</p>
<p><a href="/post_images/cardano_k8s/grafana_nodes.png">
<img src="/post_images/cardano_k8s/grafana_nodes.png" alt="Kubernetes metrics on Grafana" width="910" />
</a></p>
<h2 id="updating-the-cardano-node-configuration-files">Updating the Cardano Node Configuration Files</h2>
<p>There were a few changes I had to change in the <a href="/2021/03/cardano-relays-using-kubernetes/#node-configuration-files">previously-downloaded config files</a> to get the cardano-node metrics into Grafana.</p>
<p>In <code class="language-plaintext highlighter-rouge">mainnet-config.json</code> I changed the IP address for <code class="language-plaintext highlighter-rouge">hasPrometheus</code> from <code class="language-plaintext highlighter-rouge">127.0.0.1</code> to <code class="language-plaintext highlighter-rouge">0.0.0.0</code>:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="dl">"</span><span class="s2">hasPrometheus</span><span class="dl">"</span><span class="p">:</span> <span class="p">[</span>
<span class="dl">"</span><span class="s2">0.0.0.0</span><span class="dl">"</span><span class="p">,</span>
<span class="mi">12798</span>
<span class="p">],</span>
</code></pre></div></div>
<p>Without this, I was unable to connect to <code class="language-plaintext highlighter-rouge">/metrics</code> from outside the pod/container - a requirement for the Prometheus metrics to be scraped. Doing this allows anyone with access to the container to access these metrics, although in the case of Kubernetes, this only includes other things inside the cluster (unless you expose the port further) so is fine for me.</p>
<p>Additionally, I enabled <code class="language-plaintext highlighter-rouge">TraceBlockFetchDecisions</code>:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="dl">"</span><span class="s2">TraceBlockFetchDecisions</span><span class="dl">"</span><span class="p">:</span> <span class="kc">true</span><span class="p">,</span>
</code></pre></div></div>
<p>Without this, the <code class="language-plaintext highlighter-rouge">BlockFetchDecision_peers_connectedPeers</code> metric didnāt seem to show up in <code class="language-plaintext highlighter-rouge">/metrics</code>, which I wanted to show on Grafana as a check that the nodes were connecting to other peers correctly (although this only includes outgoing peers - I donāt believe thereās currently a way to get inbound peers listed in <code class="language-plaintext highlighter-rouge">/metrics</code>).</p>
<h2 id="setting-up-the-servicemonitor">Setting up the ServiceMonitor</h2>
<p>Finally, we need to ensure that the Prometheus instance weāre running is scraping the metrics from the relay. We can do this with a ServiceMonitor. The addons we installed are already running <a href="https://github.com/prometheus-operator">prometheus-operator</a> (in the <code class="language-plaintext highlighter-rouge">monitoring</code> namespace) which use ServiceMonitor configurations to know what to scrape. This makes it as simple as defining a ServiceMonitor that points at our service and with a scrape interval.</p>
<p>In order to avoid end up eventually having two ServiceMonitors (one for relays, one for producer) I created a service that includes all of my Cardano nodes (whether relay or producer):</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Service</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">cardano-mainnet-node-prometheus-service</span>
<span class="na">labels</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">cardano-mainnet-node-prometheus</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">selector</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">cardano-mainnet-node</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">metrics</span>
<span class="na">port</span><span class="pi">:</span> <span class="m">30101</span>
<span class="na">targetPort</span><span class="pi">:</span> <span class="m">12798</span>
</code></pre></div></div>
<p>I then created a ServiceMonitor to monitor this āserviceā (all pods in the service will be scraped):</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">monitoring.coreos.com/v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">ServiceMonitor</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">cardano-mainnet-node-monitor</span>
<span class="na">namespace</span><span class="pi">:</span> <span class="s">monitoring</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">endpoints</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">interval</span><span class="pi">:</span> <span class="s">30s</span>
<span class="na">port</span><span class="pi">:</span> <span class="s">metrics</span>
<span class="na">path</span><span class="pi">:</span> <span class="s">/metrics</span>
<span class="na">namespaceSelector</span><span class="pi">:</span>
<span class="na">matchNames</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">default</span>
<span class="na">selector</span><span class="pi">:</span>
<span class="na">matchLabels</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">cardano-mainnet-node-prometheus</span>
</code></pre></div></div>
<p>A few <code class="language-plaintext highlighter-rouge">microk8s.kubectl apply -f</code>ās and a few minutes later, I was able to access metrics from the relay in a new Grafana dashboard:</p>
<p><a href="/post_images/cardano_k8s/grafana_cardano.png">
<img src="/post_images/cardano_k8s/grafana_cardano.png" alt="Cardano metrics on Grafana" width="910" />
</a></p>
<p>Each metric includes the pod it was scraped from, so you can use `` in the Legend field to show nice pod names for each set of data.</p>
<blockquote>
<p>If you find this post useful or are looking for somewhere to delegate while setting up your own pool, <a href="https://cardano.dantup.com/">check out my pool [CODER]</a>! š</p>
</blockquote>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2021/03/monitoring-cardano-relays-on-kubernetes-with-grafana-and-prometheus/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2021-03-27:/2021/03/cardano-relays-using-kubernetes/2021-03-27T00:00:00+00:002021-03-27T00:00:00+00:00Setting up Cardano Relays using Kubernetes/microk8s
<p>I recently started a <a href="https://cardano.dantup.com/">Cardano Stake Pool, <strong>[CODER]</strong></a>, hosted using Kubernetes. This is the first of several posts on how I set it up. The code/config in this post will be taken from my <em>mainnet</em> configuration, though I would recommend starting with <em>testnet</em> when starting out.</p>
<p>Posts in this series:</p>
<ul>
<li>Step 1: Setting up Cardano Relays using Kubernetes/microk8s</li>
<li><a href="/2021/03/monitoring-cardano-relays-on-kubernetes-with-grafana-and-prometheus/">Step 2: Monitoring Cardano Relays on Kubernetes with Grafana and Prometheus</a></li>
<li><a href="/2021/04/using-kubernetes-configmaps-for-cardano-node-topology-config/">Step 3: Using Kubernetes ConfigMaps for Cardano Node Topology Config</a></li>
<li><a href="/2021/04/cardano-producer-using-kubernetes/">Step 4: Setting up a Cardano Producer node using Kubernetes/microk8s</a></li>
</ul>
<h2 id="goals">Goals</h2>
<p>My goals were:</p>
<ul>
<li>Use Kubernetes
<ul>
<li>Part of the reason for this project was to have a real project to learn more about Kubernetes</li>
<li>Iād like to manage the nodes without needing to set up individual machines or manage individual containers via Docker</li>
<li>Iād like to be able to migrate/rebuild the pool (eg. from Intel NUCs to the cloud) later with as little effort as possible</li>
</ul>
</li>
<li>Use Grafana/Prometheus to monitor the nodes without having to learn too much about them or set them up myself (specifically via the microk8s addons and ServiceMonitors)</li>
<li>Secure the pool pledge and rewards with <a href="https://cryptowalletschart.com/">a crypto hardware wallet</a> so if any node was compromised or destroyed, none of these are lost</li>
</ul>
<p>This is my first time using Kubernetes so whatās described here may not be the optimal solution. If you do have suggestions/improvements, please do comment below!</p>
<blockquote>
<p>If you find this post useful or are looking for somewhere to delegate while setting up your own pool, <a href="https://cardano.dantup.com/">check out my pool [CODER]</a>! š</p>
</blockquote>
<h2 id="installing-kubernetes-microk8s">Installing Kubernetes (microk8s)</h2>
<p>The first step is setting up Kubernetes. I decided to go with <a href="https://microk8s.io/">microk8s</a> by Canonical since it seemed to have as good reviews as any and I figured being from Canonical there would be fewer potential compatibility issues with Ubuntu Server. It also included addons for Prometheus/Grafana which looked like it may simplify setting those up. Installing it turned out to be rather trivial, as the Ubuntu Server installation wizard offered it as an option in the final step (which I believe just installs <a href="https://snapcraft.io/microk8s">the Snap package</a>).</p>
<h2 id="node-volumes">Node Volumes</h2>
<p>By default, changes made to a containers file system will be lost when the container is terminated/restarted. Since it takes many hours to sync the Cardano blockchain, I knew Iād need some way to persist its <code class="language-plaintext highlighter-rouge">db</code> folder. For simplicity, I decided (for now) to just map some folders from the host machine into the containers. While I only have one Kubernetes node right now, this might not always be the case so the volumes need to be configured to be specific to this node (and if I decide to balance the relays across nodes, I would manually copy the data and then update the <code class="language-plaintext highlighter-rouge">nodeAffinity</code> section).</p>
<p>Again, I created a <code class="language-plaintext highlighter-rouge">.yml</code> file to hold the config and applied it with <code class="language-plaintext highlighter-rouge">microk8s.kubectl apply -f ...yml</code>:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">PersistentVolume</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="c1"># To provide volumes for multiple nodes, this section is</span>
<span class="c1"># duplicated and each one has a unique name</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">cardano-mainnet-relay-data-pv-1</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">capacity</span><span class="pi">:</span>
<span class="na">storage</span><span class="pi">:</span> <span class="s">25Gi</span>
<span class="na">accessModes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">ReadWriteOnce</span> <span class="c1"># Only allow one container to use this volume</span>
<span class="na">persistentVolumeReclaimPolicy</span><span class="pi">:</span> <span class="s">Delete</span>
<span class="na">storageClassName</span><span class="pi">:</span> <span class="s">local-storage-mainnet-relay</span>
<span class="c1"># Set the path to the folder on the node that will be mounted for this volume</span>
<span class="c1"># This volume is on the host named "k8snode1"</span>
<span class="na">local</span><span class="pi">:</span>
<span class="na">path</span><span class="pi">:</span> <span class="s">/home/danny/cardano/relay-mainnet-1/data</span>
<span class="na">nodeAffinity</span><span class="pi">:</span>
<span class="na">required</span><span class="pi">:</span>
<span class="na">nodeSelectorTerms</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">matchExpressions</span><span class="pi">:</span>
<span class="c1"># Restrict this volume to a specific Kubernetes node by hostname</span>
<span class="pi">-</span> <span class="na">key</span><span class="pi">:</span> <span class="s">kubernetes.io/hostname</span>
<span class="na">operator</span><span class="pi">:</span> <span class="s">In</span>
<span class="na">values</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">k8snode1</span> <span class="c1"># hostname for this volume</span>
<span class="nn">---</span>
<span class="na">apiVersion</span><span class="pi">:</span> <span class="s">storage.k8s.io/v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">StorageClass</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">local-storage-mainnet-relay</span>
<span class="na">provisioner</span><span class="pi">:</span> <span class="s">kubernetes.io/no-provisioner</span>
<span class="c1"># Docs say that local bindings should be set to WaitForFirstConsumer.</span>
<span class="c1"># https://kubernetes.io/docs/concepts/storage/storage-classes/#local</span>
<span class="na">volumeBindingMode</span><span class="pi">:</span> <span class="s">WaitForFirstConsumer</span>
</code></pre></div></div>
<h2 id="node-configuration-files">Node Configuration Files</h2>
<p>To run a node <a href="https://docs.cardano.org/projects/cardano-node/en/latest/stake-pool-operations/getConfigFiles_AND_Connect.html">weāll need some config files</a>. For mainnet they can be downloaded like this:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>wget https://hydra.iohk.io/job/Cardano/cardano-node/cardano-deployment/latest-finished/download/1/mainnet-config.json
wget https://hydra.iohk.io/job/Cardano/cardano-node/cardano-deployment/latest-finished/download/1/mainnet-byron-genesis.json
wget https://hydra.iohk.io/job/Cardano/cardano-node/cardano-deployment/latest-finished/download/1/mainnet-shelley-genesis.json
wget https://hydra.iohk.io/job/Cardano/cardano-node/cardano-deployment/latest-finished/download/1/mainnet-topology.json
</code></pre></div></div>
<p>I renamed the <code class="language-plaintext highlighter-rouge">topology.json</code> file to <code class="language-plaintext highlighter-rouge">relay-topology.json</code> since weāll eventually also have a <code class="language-plaintext highlighter-rouge">producer-topology.json</code> that will be different, and I want to be able to collect/back up the files in one place.</p>
<h2 id="statefulsetpod-definition">StatefulSet/Pod Definition</h2>
<p>StatefulSets in Kubernetes are like Deployments but with sticky identities. Rather than pod names being suffixed with some random key that changes upon destroy/recreate, the pods in a StatefulSet will be numbered, with the identities reused if a pod is destroyed/recreated. This helps if you need other services to know identities of each pod but also simplifies using the pod names in <code class="language-plaintext highlighter-rouge">kubectl</code> commands without having to keep looking them up if youāre constantly recreating them š</p>
<p>Again, I created a <code class="language-plaintext highlighter-rouge">.yml</code> file and <code class="language-plaintext highlighter-rouge">microk8s.kubectl apply -f</code>ād it.</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">apps/v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">StatefulSet</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">cardano-mainnet-relay-deployment</span>
<span class="na">labels</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">cardano-mainnet-relay-deployment</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">serviceName</span><span class="pi">:</span> <span class="s">cardano-mainnet-relay</span>
<span class="c1"># Control the number of relays here (you'll need enough volumes to cover them!)</span>
<span class="na">replicas</span><span class="pi">:</span> <span class="m">1</span>
<span class="na">selector</span><span class="pi">:</span>
<span class="na">matchLabels</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">cardano-mainnet-node</span>
<span class="na">cardano-mainnet-node-type</span><span class="pi">:</span> <span class="s">relay</span>
<span class="na">template</span><span class="pi">:</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">labels</span><span class="pi">:</span>
<span class="c1"># I added two labels to make it easy to select either "all nodes" or</span>
<span class="c1"># "just relays" or just producer node"</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">cardano-mainnet-node</span>
<span class="na">cardano-mainnet-node-type</span><span class="pi">:</span> <span class="s">relay</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">containers</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">cardano-mainnet-relay</span>
<span class="c1"># I'm using the official IOHK/cardano-node image to avoid needing to build anything</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">inputoutput/cardano-node</span>
<span class="c1"># Expose both the cardano-node port and the /metrics endpoint port.</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">containerPort</span><span class="pi">:</span> <span class="m">12798</span>
<span class="pi">-</span> <span class="na">containerPort</span><span class="pi">:</span> <span class="m">4000</span>
<span class="c1"># Mount the data volume at /data (see the Volume Claim below)</span>
<span class="na">volumeMounts</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">data</span>
<span class="na">mountPath</span><span class="pi">:</span> <span class="s">/data</span>
<span class="c1"># My configuration lives inside the mounted /data folder, and that's also where</span>
<span class="c1"># the db data should be written</span>
<span class="na">args</span><span class="pi">:</span> <span class="pi">[</span><span class="s2">"</span><span class="s">run"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">--config"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">/data/configuration/mainnet-config.json"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">--topology"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">/data/configuration/relay-topology.json"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">--database-path"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">/data/db"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">--socket-path"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">/data/node.socket"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">--port"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">4000"</span><span class="pi">]</span>
<span class="na">volumeClaimTemplates</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">data</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">accessModes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">ReadWriteOnce</span>
<span class="c1"># This storageClassName is used on the previously defined volume that</span>
<span class="c1"># provides storage on the host machine</span>
<span class="na">storageClassName</span><span class="pi">:</span> <span class="s">local-storage-mainnet-relay</span>
<span class="na">resources</span><span class="pi">:</span>
<span class="na">requests</span><span class="pi">:</span>
<span class="na">storage</span><span class="pi">:</span> <span class="s">25Gi</span>
</code></pre></div></div>
<h2 id="exposing-the-relay-with-nodeport">Exposing the Relay with NodePort</h2>
<p>Next, we need to ensure the node is accessible to the world. We need inbound peers to be able to get mempool transactions (which weāll ultimately need to include in any blocks produced by our producer). This uses a <code class="language-plaintext highlighter-rouge">NodePort</code>, just like the Grafana service above. One unresolved niggle I have is that this exposes only a single port and balances across multiple relays. This means right now I can only provide other peers a single relay hostname/port even though in reality I may be running multiple (if anyone has a solution to this, Iād love to know!).</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Service</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">cardano-mainnet-relay-service</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">NodePort</span>
<span class="na">selector</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">cardano-mainnet-node</span>
<span class="na">cardano-mainnet-node-type</span><span class="pi">:</span> <span class="s">relay</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="c1"># Export port 30801 pointing at port 4000 on the pod(s)</span>
<span class="pi">-</span> <span class="na">port</span><span class="pi">:</span> <span class="m">30801</span>
<span class="na">nodePort</span><span class="pi">:</span> <span class="m">30801</span>
<span class="na">targetPort</span><span class="pi">:</span> <span class="m">4000</span>
</code></pre></div></div>
<p>With this applied, I shared my relay details with a few friendly SPOs and used the log files to verify I had incoming connections and transactions. Step one, complete!</p>
<blockquote>
<p>If you find this post useful or are looking for somewhere to delegate while setting up your own pool, <a href="https://cardano.dantup.com/">check out my pool [CODER]</a>! š</p>
</blockquote>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2021/03/cardano-relays-using-kubernetes/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2020-06-20:/2020/06/review-of-seeed-grove-components/2020-06-20T00:00:00+00:002020-06-20T00:00:00+00:00Review of Seeed Grove Components
<p>I was recently sent some <a href="https://www.seeedstudio.com/category/Grove-c-1003.html">Grove components</a> by Seeed to review:</p>
<ul>
<li><a href="https://www.seeedstudio.com/Grove-Shield-for-Arduino-Nano-p-4112.html">Grove Shield for Arduino Nano</a></li>
<li><a href="https://www.seeedstudio.com/Grove-Shield-for-micro-bit-v2-0.html">Grove Shield for micro:bit v2.0</a></li>
<li><a href="https://www.seeedstudio.com/Grove-16x2-LCD-White-on-Blue.html">Grove - 16x2 LCD (White on Blue)</a></li>
<li><a href="https://www.seeedstudio.com/Grove-WS2813-RGB-LED-Strip-Waterproof-30-LED-m-1m-p-3124.html">Grove - WS2813 RGB LED Strip Waterproof - 30 LED/m - 1m</a></li>
<li><a href="https://www.seeedstudio.com/Grove-Universal-4-Pin-20cm-Unbuckled-Cable-5-PCs-Pack-p-749.html">Grove - Universal 4 Pin 20cm Unbuckled Cable (5 PCs Pack)</a></li>
<li><a href="https://www.seeedstudio.com/Grove-Yellow-LED-Button-p-3101.html">Grove - Yellow LED Button</a></li>
</ul>
<p>These components normally total around $30 USD but were provided to me for free to review. I intend for my review to be honest and unbiased (and I am not part of any affiliate/commission scheme).</p>
<p>Grove modules use an (open source) standardised connector and there are shields for many types of boards (Raspberry Pi, Arduino, micro:bit, NodeMCU, Beaglebone, etc). Coincidentally, I was aware of (and interested in) Grove prior to being contacted by Seeed. Although I like tinkering with Pi and Arduino (thereās something fun about software interacting with hardware you donāt get from just software dev), I always found wiring up components on a breadboard (and soldering) a drag. Anything that makes this plug-and-play so I can focus on the idea/programming definitely gets my interest. Since my son also has a micro:bit and thereās a micro:bit shield, I could also pretend that this was totally for him and not at all for me :-)</p>
<h2 id="shipping-from-seeedstudio">Shipping from SeeedStudio</h2>
<p>It took 10 days from placing the order to my items arriving. I was expecting it to take longer (the website says the products were coming from a China warehouse) but itās possible my shipping selection (the cheapest) was upgraded.</p>
<h2 id="packaging">Packaging</h2>
<p>The items arrived well protected in a box padded with bubble wrap. Each item was also in its own packaging (most in resealable bags). Each component also came with a Grove cable (I hadnāt realised this, so had requested a set of them that it turned out I didnāt need).</p>
<div class="filmstrip">
<img src="/post_images/grove/package_1_t.jpg" />
<img src="/post_images/grove/package_2_t.jpg" />
<img src="/post_images/grove/package_3_t.jpg" />
</div>
<h2 id="led-button">LED Button</h2>
<p>Since these components were āfor my sonā we started off with micro:bit. The LED button seemed like the simplest component to try out, so we plugged the micro:bit into the shield, the LED button into the shield (P0/P14 port) and then used <a href="https://makecode.microbit.org/">MakeCode for micro:bit</a> (running on a <a href="https://chromebookcompare.com/">Chromebook</a>) to make a simple program that would toggle the LED when the button was pressed. It wasnāt obvious to me which pin was connected to the LED and which to the button (even after examining the board) so we had to guess (which we did correctly - P0 was the LED and P14 was the button).</p>
<p>No surprises, this worked as expected. The button feels solid with a nice click as you press it. The button/LED sticks out from the board enough that it should be easy to mount inside a 3D-printed case (or similar) with just the button accessible, though youād need a little space around it to account for the board and the connector. The LED is bright enough that you could easily see it illuminated in our reasonably bright room.</p>
<p>I later connected the button to an Arduino Nano via the Nano shield and again it functioned exactly as youād expect.</p>
<div class="filmstrip">
<img src="/post_images/grove/button_1_t.jpg" />
<img src="/post_images/grove/button_2_t.jpg" />
<img src="/post_images/grove/button_3_t.jpg" />
</div>
<h2 id="led-light-strip">LED Light Strip</h2>
<p>Next up, more LEDs! Having not used a light strip before, I expected this to be less straight-foward (since weāre controlling many LEDs and colours from a single pin). I was aware that MakeCode had support for āextensionsā but hadnāt checked in advance if there was an extension for this. Unaware of what the protocol was (turns out itās a pretty standard RGB WS2813 - mentioned in the product name!) I saw a NeoPixels extension so decided to give that a go in the hope it was compatible.</p>
<p>We made a simple program to rotate some colours along the strip and to my surprise it just worked as expected out of the box (yay for standards!). These LEDs are also bright enough that they were clear in our fairly bright room. They updated quickly, and all the colours we tried came through clearly. The light strip is inside a plastic surround and is very flexible. Itās advertised as waterproof though we didnāhavenāt tested this out yet.</p>
<p>Again, I later connected this to my Arduino Nano and using the Adafruit_NeoPixel library I was quickly able to control the LEDs. The only niggle was that some colours were incorrect until I passed NEO_BRG + NEO_KHZ800 when creating the light strip which looks like the frequency of updates and the order of the RGB values (which actually seems like the opposite order of the colours to the micro:bit extension).</p>
<div class="filmstrip">
<img src="/post_images/grove/led_1_t.jpg" />
<img src="/post_images/grove/led_2_t.jpg" />
<img src="/post_images/grove/led_3_t.jpg" />
</div>
<h2 id="16x2-lcd-display">16x2 LCD Display</h2>
<p>The LCD Display is where we hit our first issue. There are some LCD display extensions for micro:bit, but after much testing weāve been (so far) unable to make any of them work (if you can, please let me know!). The <a href="http://wiki.seeedstudio.com/Grove-16x2_LCD_Series/">Seeed wiki</a> says the I2C address of the display is 0x3E (62) but we tried enumerating all addresses from 1 to 150 but were unable to get the display to show anything.</p>
<p>To check the screen wasnāt faulty, I connected it to the Arduino and using the rgc_lcd library and with two lines of code the screen was showing a message (I actually forgot to set the address, so the screen mustāve been using an address that the library used by default).</p>
<p>The screen is clear and bright, and also sticks out from the board enough that it should be easy to mount inside something like a 3D-printed box and look good. Itās very slightly inset from housing but Seeed have some screens without a housing like this if youād prefer the screen right up against your own housing (<a href="https://www.seeedstudio.com/Grove-OLED-Display-0-96.html">like this one</a>).</p>
<div class="filmstrip">
<img src="/post_images/grove/lcd_1_t.jpg" />
<img src="/post_images/grove/lcd_2_t.jpg" />
<img src="/post_images/grove/lcd_3_t.jpg" />
</div>
<h2 id="build-quality">Build Quality</h2>
<p>All of the items received feel high quality and well made. My Arduino boards/components are a mix of super-cheap from eBay (which often turn up in plastic bags with broken pins and badly aligned components) and some slightly more expensive from Amazon sellers (which are much better and often packaged/protected individually). The Seeed components feel on-par with the better quality items Iāve bought.</p>
<h2 id="value-for-money">Value for Money</h2>
<p>I think most of the components I received are reasonable value for money for the quality. The LCD screen and LED strips are $5.95 and the combined LED/button is $2.45. Iāve certainly bought components cheaper, but they have been of significantly worse quality (and didnāt have Grove connectors). The micro:bit shield was the only item that seemed a little expensive for what it was at $9.90 (versus $2.60 for an Arduino Nano shield for example).</p>
<h2 id="would-i-buy-them">Would I Buy Them?</h2>
<p>If I hadnāt been sent these components for free, would I have paid for them? For the listed prices - and now knowing what Iād be getting - I think I would. The cost wouldāve been around Ā£21 GBP (plus shipping) which I think is fair given the quality of the components, the fun the kids have already had using them in MakeCode and how easily I can use them myself in Arduino without having to mess around on breadboards. Connecting things on breadboards might not be particualrly difficult, but it can take time and is easy to wire things up incorrect (and diagnosing issues with wiring can often take time).</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2020/06/review-of-seeed-grove-components/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2019-11-17:/2019/11/easily-compiling-dart-to-native-executables-for-windows-linux-macos-with-github-actions/2019-11-17T00:00:00+00:002019-11-17T00:00:00+00:00Compiling Dart to Native Executables for Windows, Linux and macOS with GitHub Actions
<h1 id="compiling-dart-to-native-executables-for-windows-linux-and-macos-using-github-actions">Compiling Dart to Native Executables for Windows, Linux and macOS using GitHub Actions</h1>
<p><a href="https://medium.com/dartlang/dart2native-a76c815e6baf">Dart v2.6</a> includes <a href="https://dart.dev/tools/dart2native">dart2native</a> - a new tool for compiling Dart scripts to native executables for Windows, Linux and macOS. Currently thereās no cross-compilation support (see <a href="https://github.com/dart-lang/sdk/issues/28617">dart-lang/sdk#28617</a>) so each binary needs to be compiled from the correct OS.</p>
<p>Cloud services like GitHub Actions make it very easy to run code on different OSes so I created a sample repo that compiles a simple Dart script to native executables for each platform and upload them as artifacts for easy download:</p>
<p><a href="https://github.com/DanTup/dart-native-executables">github.com/DanTup/dart-native-executables</a></p>
<p>You can try this out with your own scripts by forking the repo and modifying the script or copying the relevant parts out of the <a href="https://github.com/DanTup/dart-native-executables/blob/master/.github/workflows/main.yml">GitHub workflow</a>.</p>
<h2 id="the-github-workflow">The GitHub Workflow</h2>
<p>The workflow is relatively straight forward. It sets up a matrix to run on all platforms and sets an <code class="language-plaintext highlighter-rouge">output-name</code> variable for the executable.</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">jobs</span><span class="pi">:</span>
<span class="na">build</span><span class="pi">:</span>
<span class="na">runs-on</span><span class="pi">:</span> <span class="s">${{ matrix.os }}</span>
<span class="na">strategy</span><span class="pi">:</span>
<span class="na">matrix</span><span class="pi">:</span>
<span class="na">os</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">ubuntu-latest</span><span class="pi">,</span> <span class="nv">windows-latest</span><span class="pi">,</span> <span class="nv">macOS-latest</span><span class="pi">]</span>
<span class="na">include</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">os</span><span class="pi">:</span> <span class="s">ubuntu-latest</span>
<span class="na">output-name</span><span class="pi">:</span> <span class="s">hello-linux</span>
<span class="pi">-</span> <span class="na">os</span><span class="pi">:</span> <span class="s">macOS-latest</span>
<span class="na">output-name</span><span class="pi">:</span> <span class="s">hello-mac</span>
<span class="pi">-</span> <span class="na">os</span><span class="pi">:</span> <span class="s">windows-latest</span>
<span class="na">output-name</span><span class="pi">:</span> <span class="s">hello-windows.exe</span>
</code></pre></div></div>
<p>Dart is downloaded using <a href="https://github.com/DanTup/gh-actions/blob/master/src/setup-dart/main.ts">a simple GitHub action I created</a>:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="pi">-</span> <span class="na">uses</span><span class="pi">:</span> <span class="s">DanTup/gh-actions/setup-dart@master</span>
<span class="c1"># Defaults to stable channel, but can be overriden. </span>
<span class="c1"># with:</span>
<span class="c1"># channel: dev</span>
</code></pre></div></div>
<p>Next, <code class="language-plaintext highlighter-rouge">dart2native</code> is invoked with the platform-specific output name:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="pi">-</span> <span class="na">run</span><span class="pi">:</span> <span class="s">dart2native bin/main.dart -v -o build/${{ matrix.output-name }}</span>
</code></pre></div></div>
<p>And finally, the executable is uploaded as a GitHub artifact:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="pi">-</span> <span class="na">uses</span><span class="pi">:</span> <span class="s">actions/upload-artifact@v1</span>
<span class="na">with</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">native-executables</span>
<span class="na">path</span><span class="pi">:</span> <span class="s">build</span>
</code></pre></div></div>
<p>After each commit, GitHub will run this action on each platform:</p>
<p><a href="/post_images/dart2native_1_run.png">
<img src="/post_images/dart2native_1_run.png" alt="GitHub Actions run" width="472" />
</a></p>
<p>The executables will be attached to the Action results in the artifacts list:</p>
<p><a href="/post_images/dart2native_2_artifacts.png">
<img src="/post_images/dart2native_2_artifacts.png" alt="GitHub Actions artifacts" width="389" />
</a></p>
<p><a href="/post_images/dart2native_3_zip_contents.png">
<img src="/post_images/dart2native_3_zip_contents.png" alt="Compiled executables in ZIP file" width="534" />
</a></p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2019/11/easily-compiling-dart-to-native-executables-for-windows-linux-macos-with-github-actions/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2017-07-11:/2017/07/dart-code-v2-now-with-flutter-debugging-and-hot-reload/2017-07-11T00:00:00+00:002017-07-11T00:00:00+00:00Dart Code v2 - Now with Flutter debugging and hot reload!
<p><a href="https://github.com/Dart-Code/Dart-Code"><img style="position: absolute; top: 0; right: 0; border: 0;" src="https://camo.githubusercontent.com/652c5b9acfaddf3a9c326fa6bde407b87f7be0f4/68747470733a2f2f73332e616d617a6f6e6177732e636f6d2f6769746875622f726962626f6e732f666f726b6d655f72696768745f6f72616e67655f6666373630302e706e67" alt="Fork me on GitHub" data-canonical-src="https://s3.amazonaws.com/github/ribbons/forkme_right_orange_ff7600.png" /></a></p>
<p>Itās already been ten months since I published the first preview of <a href="https://marketplace.visualstudio.com/items?itemName=Dart-Code.dart-code">Dart Code</a> and today marks a significant milestone - version 2.0! Itās been some time in the making; this major update adds support for launching and debugging Android and iOS apps written using <a href="https://flutter.io/">Flutter</a>, including:</p>
<ul>
<li>When a Flutter project is open your selected device/emulator will be shown in the status bar</li>
<li>If you have more than one connected device/emulator, clicking the device name in the status bar will allow you to switch between devices/emulators</li>
<li>Pressing <code class="language-plaintext highlighter-rouge">F5</code> with a flutter project open will build and launch your app on the selected device/emulator</li>
<li>The usual debugging experience is available for Flutter apps, including breakpoints, call stacks, watch window etc.</li>
<li>The debuggerās <code class="language-plaintext highlighter-rouge">Restart</code> button (or <code class="language-plaintext highlighter-rouge">Ctrl+Shift+F5</code>) has been mapped to Flutterās <code class="language-plaintext highlighter-rouge">hot reload</code> feature and will update the running application without needing to rebuild (this usually results in sub-second updates!)</li>
</ul>
<p><a href="https://twitter.com/BramVanbilsen">Bram Vanbilsen</a>, who has started <a href="https://www.youtube.com/playlist?list=PLxU9Ryxq6p58PsNmJL70J4_7UzfSqf35n">an excellent set of Flutter tutorials on YouTube</a> has created a short video showing how this integration works with a demo of hot reload:</p>
<div align="center">
<iframe width="450" height="253" src="https://www.youtube.com/embed/hhP1tE-IHos?VQ=HD1080" frameborder="0" allowfullscreen=""></iframe>
</div>
<h2 id="download">Download</h2>
<p>In order to get up and running youāll need to grab a few things:</p>
<ul>
<li><a href="https://code.visualstudio.com/">Visual Studio Code</a> version 1.13.0 or later</li>
<li><a href="https://marketplace.visualstudio.com/items?itemName=Dart-Code.dart-code">v2 of Dart Code</a></li>
<li>A recent version of the <a href="https://www.dartlang.org/install">Dart SDK</a></li>
<li>For Flutter support youāll also need to <a href="https://flutter.io/setup/">set up Flutter</a></li>
</ul>
<p>Finally, to avoid manual configuration you should also add the Dart and Flutter SDKs to your <code class="language-plaintext highlighter-rouge">PATH</code>.</p>
<h2 id="feedback">Feedback</h2>
<p>If you do try it out, Iād love your feedback. You can find me on Twitter (<a href="https://twitter.com/DanTup">@DanTup</a> and <a href="https://twitter.com/DartCode">@DartCode</a>), post issues at <a href="https://github.com/Dart-Code/Dart-Code/issues">GitHub/Dart-Code</a> or find me in the <a href="https://gitter.im/dart-code/Dart-Code">Gitter chat room</a>!</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2017/07/dart-code-v2-now-with-flutter-debugging-and-hot-reload/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2017-02-03:/2017/02/i-found-a-ridiculous-bug-in-youtube-and-seem-unable-to-successfully-report-it/2017-02-03T00:00:00+00:002017-02-03T00:00:00+00:00I found a ridiculous bug in YouTube and seem unable to successfully report it :(
<p>Last week a colleague tweeted me telling me that the hyperlinks in the descriptions of <a href="https://www.youtube.com/watch?v=GKI7aKY_hxY">my kids review of the Echo Dot</a> were giving 404s. This surprised me because Iād clicked them to ensure they worked after publishing the video! I probed him and he sent a screenshot of 404s. I checked them again myself both on desktop and my mobile and everything was fine. Since this colleague has a certain sense of humour, I figured he was trying to wind me up and didnāt think much of it.</p>
<blockquote class="bigquote right">
YouTube is truncating links not just for the display but the actual URL the user is taken to
</blockquote>
<p>The next day he showed me the issue on his mobile phone - he was right - the links had been <strong>truncated</strong>! WTF? I noticed that when he clicked YouTube links in twitter they were opening in Chrome, not the YouTube app. I tried the same thing on my mobile and was immediately able to reproduce the issue. I did some Googling and found reports of <a href="https://productforums.google.com/forum/#!topic/youtube/uDB4NY6kEvc">a bug in 2013</a> that sounded similar. I dug into this and discovered that this wasnāt the same issue - that was an issue with the inline editor truncating links when editing; my links were fine in the editor and on desktop/mobile. This is a different bug. The YouTube mobile site is truncating links not just for the display but the <em>actual URL the user is taken to</em>. Whaaaaa?</p>
<p>You might think āWho uses the mobile site anyway? Just Windows Phone users?ā but the colleague that reported it to me has an Android phone that came pre-installed with the YouTube app, yet clicking YT links in Twitter opens the mobile web version. Lots of promotion of YouTube videos is done on Twitter and other social media so the number of people hitting this bug is probably not insignificant. Probably they assume the video creator is a muppet rather than a YouTube bug.</p>
<p>I did some more Googling and could not find a single report of this bug online! Iām surprised that a bug with links being truncated for a large number of people (if only 1% of YouTube traffic is mobile web, thatās still huge) can go unnoticed.</p>
<p>I decided to check it wasnāt just me so I opened up some videos from popular YouTubers I followā¦ Casey Neistatā¦ Same thing.</p>
<p class="bdr"><img src="/post_images/yt_bug_casey.jpg" alt="404 from Casey Neistat's YouTube video" /></p>
<p>I tried Sean Cannellā¦ Same thing.</p>
<p class="bdr"><img src="/post_images/yt_bug_sean.jpg" alt="404 from Sean Cannell's YouTube video" /></p>
<p>I tried MKBHDā¦ Same thing.</p>
<p class="bdr"><img src="/post_images/yt_bug_mkbhd.jpg" alt="404 from MKBHD's YouTube video" /></p>
<p>Sara Dietschyā¦ Same thing.</p>
<p class="bdr"><img src="/post_images/yt_bug_sara.jpg" alt="404 from Sara's YouTube video" /></p>
<p>WTF YouTube? Where is the testing? <a href="https://blog.dantup.com/2016/04/have-software-developers-given-up/">Have we given up?</a>.</p>
<p>So, I did what I always do when I find a bug. I tried to report it. Quite a few timesā¦</p>
<ul>
<li><a href="https://productforums.google.com/forum/#!msg/youtube/BlU8eV2AQQo/zYR16f6XDAAJ">In the official forums</a> - no responses from employees nor top contributors</li>
<li>Via Send Feedback on the web - I use this a lot in Google apps and they all feel like black holes</li>
<li>Via Send Feedback in the mobile app - as above</li>
<li><a href="https://twitter.com/DanTup/status/826488712557101058">Twitter</a> (<a href="https://twitter.com/DanTup/status/826500598249857024">three</a> <a href="https://twitter.com/DanTup/status/826907619323412481">times</a>) - no response here (despite the YouTube account responding to others every day)</li>
<li><a href="https://www.reddit.com/r/youtube/comments/5ra0gg/bug_mobile_web_version_of_youtube_truncates_links/">In /r/YouTube/</a> knowing employees post there; it fell off the homepage with no responses (and even downvotes)</li>
<li>I tweeted <a href="https://twitter.com/DanTup/status/826490526157434881">some</a> <a href="https://twitter.com/DanTup/status/827438719036694528">of</a> <a href="https://twitter.com/DanTup/status/827437746100711424">the</a> <a href="https://twitter.com/DanTup/status/827438073814335488">YouTubers</a> I followed to alert them of this issue (itās lost revenue for those putting affiliate links here and maybe they have contacts?) though the only response I got <a href="https://twitter.com/seancannell/status/826495961937555456">wasnāt clear if the YouTuber really understood</a> what I was saying.</li>
</ul>
<p>The only workaround for this bug that I know of is to shorten all your links. I hate this because it looks scummy like youāre trying to hide something. I created my own short links for the <a href="https://www.youtube.com/channel/UCJxVQCWS4O_c38fPp5stDxw">Review Kidz videos</a> for now that hopefully makes it clearer where they go than goo.gl but itās not a great solution.</p>
<p>So, this blog post is my final attempt to get someone from YouTube/Google to see this so they might consider fixing this (IMO completely ridiculous) bug. Iād rather not have to shorten links because of this but itās looking like thatās how itās going to be. Itās amazing both that a product as large as YouTube can have so many broken links and apparently nobody has noticed and that thereās very little information on how to report bugs or get their attention (though honestly, this seems par for the course with Google these days; support is simply nonexistent).</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2017/02/i-found-a-ridiculous-bug-in-youtube-and-seem-unable-to-successfully-report-it/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2017-01-27:/2017/01/simplest-dart-code-to-post-a-tweet-using-oauth/2017-01-27T00:00:00+00:002017-01-27T00:00:00+00:00Simple Dart code to tweet using OAuth
<blockquote>
<p>Writing Dart? Check out <a href="https://marketplace.visualstudio.com/items?itemName=Dart-Code.dart-code">Dart Code, my Dart extension for Visual Studio Code!</a></p>
</blockquote>
<p>Last year <a href="/2016/07/simplest-csharp-code-to-post-a-tweet-using-oauth/">I blogged about the simplest C# code I could come up with to post tweets</a> without having to jump through OpenAuth hoops. This only works for accounts you can get keys for (eg. your own) but itās significnatly simpler than the normal OAuth flow.</p>
<p>Since running C# on my Raspberry Pi turned out to be <a href="https://github.com/dotnet/cli/issues/2556">significantly more effort</a> than I expected from the ācross platformā .NET Core (I did try mono but getting SSL working to talk to Twitter was a clusterfuck of pasting commands from the internet that didnāt work) I decided to port it to one of my favourite languages, Dart! It would also be a nice task to test out <a href="https://marketplace.visualstudio.com/items?itemName=Dart-Code.dart-code">Dart Code</a>, the Dart plugin I created for Visual Studio Code (with much support from the Dart and VS Code teams) which I donāt get to do nearly enough of.</p>
<p>Like with <a href="/2016/07/simplest-csharp-code-to-post-a-tweet-using-oauth/">the C# version</a> it turned out to be complicated to encode strings in the way that Twitter expects. I tried all sorts of encoding methods built-in to Dart and none of them gve the expected results. The <a href="https://pub.dartlang.org/packages/convert">convert package</a> published by the Dart team claimed to be RFC 3986 compliant but it seemed to encode numbers - <code class="language-plaintext highlighter-rouge">1</code> became <code class="language-plaintext highlighter-rouge">%31</code> and Twitter didnāt like that. Assuming this was a bug, I <a href="https://github.com/dart-lang/convert/issues/3">raised an issue</a> and <a href="https://github.com/dart-lang/convert/pull/4">sent a PR with tests and a fix</a>. It took a little while but my PR was merged and a new version of the package released with the fix.</p>
<p>If you want the details about how to get your API keys to use this code, read the <a href="/2016/07/simplest-csharp-code-to-post-a-tweet-using-oauth/">the C# version of this post</a> which contains links to the docs at Twitter. The rest of this article is just a summary of the actual Dart code. Itās very much a direct port of the C# tidied up where Dart made it easy. If Iām honest, I think the Dart version is much less readable due to <code class="language-plaintext highlighter-rouge">dartfmt</code> wrapping awkwardly. I do format using this currently but Iām very much on the fence about whether itās worthwhile sacrificing readability and the ability to use tabs for formatting (I think there are better ways of achieving the consistency they aim for).</p>
<p>On to the code!</p>
<p>As before, the constructor takes in the keys and creates a hasher. <em>(Note: Indenting is <code class="language-plaintext highlighter-rouge">dartfmts</code>; two spaces for indenting but 4 spaces for wrapped lines.. Iād prefer it if the top line was unwrapped)</em>.</p>
<div class="language-dart highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">TwitterApi</span><span class="o">(</span><span class="k">this</span><span class="o">.</span><span class="na">consumerKey</span><span class="o">,</span> <span class="k">this</span><span class="o">.</span><span class="na">consumerKeySecret</span><span class="o">,</span> <span class="k">this</span><span class="o">.</span><span class="na">accessToken</span><span class="o">,</span>
<span class="k">this</span><span class="o">.</span><span class="na">accessTokenSecret</span><span class="o">)</span> <span class="o">{</span>
<span class="kd">var</span> <span class="n">bytes</span> <span class="o">=</span> <span class="n">UTF8</span><span class="o">.</span><span class="na">encode</span><span class="o">(</span><span class="s">"</span><span class="si">$consumerKeySecret</span><span class="s">&</span><span class="si">$accessTokenSecret</span><span class="s">"</span><span class="o">);</span>
<span class="n">_sigHasher</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Hmac</span><span class="o">(</span><span class="n">sha1</span><span class="o">,</span> <span class="n">bytes</span><span class="o">);</span>
<span class="o">}</span>
</code></pre></div></div>
<p>The tweet method is async and just builds the payload before calling a re-usable <code class="language-plaintext highlighter-rouge">_callApi</code> method (note: in Dart underscore prefixes automatically make things private). This will allow adding additional methods easier (as <a href="https://blog.dantup.com/2016/07/simplest-csharp-code-to-post-a-tweet-using-oauth/#comment-3112597725">Eelco Koster did with the C# version</a>). I set <code class="language-plaintext highlighter-rouge">trim_user</code> just to reduce the amount of data that comes back in the response (since I donāt use it for anything).</p>
<div class="language-dart highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">/// Sends a tweet with the supplied text and returns the response from the Twitter API.</span>
<span class="n">Future</span><span class="o"><</span><span class="kt">String</span><span class="o">></span> <span class="n">tweet</span><span class="o">(</span><span class="kt">String</span> <span class="n">text</span><span class="o">)</span> <span class="o">{</span>
<span class="kd">var</span> <span class="n">data</span> <span class="o">=</span> <span class="o">{</span><span class="s">"status"</span><span class="o">:</span> <span class="n">text</span><span class="o">,</span> <span class="s">"trim_user"</span><span class="o">:</span> <span class="s">"1"</span><span class="o">};</span>
<span class="k">return</span> <span class="n">_callApi</span><span class="o">(</span><span class="s">"statuses/update.json"</span><span class="o">,</span> <span class="n">data</span><span class="o">);</span>
<span class="o">}</span>
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">_callApi</code> method is almost identical to the C# <code class="language-plaintext highlighter-rouge">SendRequest</code> method so Iāll skip over it here (the full code at the bottom of the post).</p>
<p>I extracted some helpers for making query strings and OAuth headers (which annoyingly were subtly different; one requires quotes and one canāt have them) which made the signature and header generation methods pretty terse:</p>
<div class="language-dart highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">/// Generate an OAuth signature from OAuth header values.</span>
<span class="kt">String</span> <span class="nf">_generateSignature</span><span class="p">(</span><span class="n">Uri</span> <span class="n">url</span><span class="o">,</span> <span class="n">Map</span><span class="o"><</span><span class="kt">String</span><span class="o">,</span> <span class="kt">String</span><span class="o">></span> <span class="n">data</span><span class="o">)</span> <span class="o">{</span>
<span class="kd">var</span> <span class="n">sigString</span> <span class="o">=</span> <span class="n">_toQueryString</span><span class="o">(</span><span class="n">data</span><span class="o">);</span>
<span class="kd">var</span> <span class="n">fullSigData</span> <span class="o">=</span> <span class="s">"POST&</span><span class="si">${_encode(url.toString())}</span><span class="s">&</span><span class="si">${_encode(sigString)}</span><span class="s">"</span><span class="o">;</span>
<span class="k">return</span> <span class="n">BASE64</span><span class="o">.</span><span class="na">encode</span><span class="o">(</span><span class="n">_hash</span><span class="o">(</span><span class="n">fullSigData</span><span class="o">));</span>
<span class="o">}</span>
<span class="c1">/// Generate the raw OAuth HTML header from the values (including signature).</span>
<span class="kt">String</span> <span class="nf">_generateOAuthHeader</span><span class="p">(</span><span class="n">Map</span><span class="o"><</span><span class="kt">String</span><span class="o">,</span> <span class="kt">String</span><span class="o">></span> <span class="n">data</span><span class="o">)</span> <span class="o">{</span>
<span class="kd">var</span> <span class="n">oauthHeaderValues</span> <span class="o">=</span> <span class="n">_filterMap</span><span class="o">(</span><span class="n">data</span><span class="o">,</span> <span class="o">(</span><span class="n">k</span><span class="o">)</span> <span class="o">=></span> <span class="n">k</span><span class="o">.</span><span class="na">startsWith</span><span class="o">(</span><span class="s">"oauth_"</span><span class="o">));</span>
<span class="k">return</span> <span class="s">"OAuth "</span> <span class="o">+</span> <span class="n">_toOAuthHeader</span><span class="o">(</span><span class="n">oauthHeaderValues</span><span class="o">);</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Next is a method for sending the request over to Twitter. I originally missed the <code class="language-plaintext highlighter-rouge">http.close</code> from the end of this which resulted in a 10-15 second hang when my program finished (the Dart VM didnāt exit until the connection timed out). I donāt really like that you need to add that (itās easily forgotten and a pain to debug/understand) but I donāt know of a good fix.</p>
<p>This is the most unreadable part of this class IMO after running through <code class="language-plaintext highlighter-rouge">dartfmt</code> <em>(I would put breaks/indent before <code class="language-plaintext highlighter-rouge">.then</code>ās and unwrap the <code class="language-plaintext highlighter-rouge">contentType</code> header)</em>.</p>
<p><strong>Edit:</strong> Slightly more readable after switching to <code class="language-plaintext highlighter-rouge">await</code> from <code class="language-plaintext highlighter-rouge">.then</code> (thanks <a href="#comment-3123212558">Benjamin Campbell!</a>).</p>
<div class="language-dart highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">/// Send HTTP Request and return the response.</span>
<span class="n">Future</span><span class="o"><</span><span class="kt">String</span><span class="o">></span> <span class="n">_sendRequest</span><span class="o">(</span>
<span class="n">Uri</span> <span class="n">fullUrl</span><span class="o">,</span> <span class="kt">String</span> <span class="n">oAuthHeader</span><span class="o">,</span> <span class="kt">String</span> <span class="n">body</span><span class="o">)</span> <span class="n">async</span> <span class="o">{</span>
<span class="kd">final</span> <span class="n">http</span> <span class="o">=</span> <span class="k">new</span> <span class="n">HttpClient</span><span class="o">();</span>
<span class="kd">final</span> <span class="n">request</span> <span class="o">=</span> <span class="n">await</span> <span class="n">http</span><span class="o">.</span><span class="na">postUrl</span><span class="o">(</span><span class="n">fullUrl</span><span class="o">);</span>
<span class="n">request</span><span class="o">.</span><span class="na">headers</span>
<span class="o">..</span><span class="na">contentType</span> <span class="o">=</span> <span class="k">new</span> <span class="n">ContentType</span><span class="o">(</span><span class="s">"application"</span><span class="o">,</span> <span class="s">"x-www-form-urlencoded"</span><span class="o">,</span>
<span class="nl">charset:</span> <span class="s">"utf-8"</span><span class="o">)</span>
<span class="o">..</span><span class="na">add</span><span class="o">(</span><span class="s">"Authorization"</span><span class="o">,</span> <span class="n">oAuthHeader</span><span class="o">);</span>
<span class="n">request</span><span class="o">.</span><span class="na">write</span><span class="o">(</span><span class="n">body</span><span class="o">);</span>
<span class="kd">final</span> <span class="n">response</span> <span class="o">=</span> <span class="n">await</span> <span class="n">request</span><span class="o">.</span><span class="na">close</span><span class="o">().</span><span class="na">whenComplete</span><span class="o">(</span><span class="n">http</span><span class="o">.</span><span class="na">close</span><span class="o">);</span>
<span class="k">return</span> <span class="n">response</span><span class="o">.</span><span class="na">transform</span><span class="o">(</span><span class="n">UTF8</span><span class="o">.</span><span class="na">decoder</span><span class="o">).</span><span class="na">join</span><span class="o">(</span><span class="s">""</span><span class="o">);</span>
<span class="o">}</span>
</code></pre></div></div>
<p>The class is used in the same way as the C# version and the response from <code class="language-plaintext highlighter-rouge">tweet</code> awaited:</p>
<div class="language-dart highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">var</span> <span class="n">twitter</span> <span class="o">=</span> <span class="k">new</span> <span class="n">TwitterApi</span><span class="o">(</span><span class="n">liveConsumerKey</span><span class="o">,</span> <span class="n">liveConsumerKeySecret</span><span class="o">,</span>
<span class="n">liveAccessToken</span><span class="o">,</span> <span class="n">liveAccessTokenSecret</span><span class="o">);</span>
<span class="kd">var</span> <span class="n">resp</span> <span class="o">=</span> <span class="n">await</span> <span class="n">twitter</span><span class="o">.</span><span class="na">tweet</span><span class="o">(</span><span class="s">"YOUR TWEET HERE"</span><span class="o">);</span>
</code></pre></div></div>
<p>Since my need is a very simple script where I receive the output my email, my error handling is very lame:</p>
<div class="language-dart highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">if</span> <span class="o">(</span><span class="n">resp</span><span class="o">.</span><span class="na">startsWith</span><span class="o">(</span><span class="s">"{</span><span class="se">\"</span><span class="s">errors</span><span class="se">\"</span><span class="s">"</span><span class="o">))</span> <span class="o">{</span>
<span class="n">print</span><span class="o">(</span><span class="s">"Failed to send tweet:"</span><span class="o">);</span>
<span class="n">print</span><span class="o">(</span><span class="s">""</span><span class="o">);</span>
<span class="n">print</span><span class="o">(</span><span class="n">tweetToSend</span><span class="o">);</span>
<span class="n">print</span><span class="o">(</span><span class="s">""</span><span class="o">);</span>
<span class="n">print</span><span class="o">(</span><span class="n">resp</span><span class="o">);</span>
<span class="o">}</span> <span class="k">else</span> <span class="o">{</span>
<span class="c1">// Success!</span>
<span class="o">}</span>
</code></pre></div></div>
<p>If using this more seriously, youād want to return a nice class from the <code class="language-plaintext highlighter-rouge">tweet</code> method instead.</p>
<p>Hereās the full class; youāll need to add <code class="language-plaintext highlighter-rouge">convert</code> and <code class="language-plaintext highlighter-rouge">crypto</code> to your <code class="language-plaintext highlighter-rouge">pubspec.yaml</code> and run <code class="language-plaintext highlighter-rouge">pub get</code> to get this working.</p>
<div class="language-dart highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="s">'dart:async'</span><span class="o">;</span>
<span class="kn">import</span> <span class="s">'dart:convert'</span><span class="o">;</span>
<span class="kn">import</span> <span class="s">'dart:io'</span><span class="o">;</span>
<span class="kn">import</span> <span class="s">'package:convert/convert.dart'</span><span class="o">;</span>
<span class="kn">import</span> <span class="s">'package:crypto/crypto.dart'</span><span class="o">;</span>
<span class="kd">const</span> <span class="kt">String</span> <span class="n">twitterApiBaseUrl</span> <span class="o">=</span> <span class="s">"https://api.twitter.com/1.1/"</span><span class="o">;</span>
<span class="kd">class</span> <span class="nc">TwitterApi</span> <span class="o">{</span>
<span class="kd">final</span> <span class="kt">String</span> <span class="n">consumerKey</span><span class="o">,</span> <span class="n">consumerKeySecret</span><span class="o">,</span> <span class="n">accessToken</span><span class="o">,</span> <span class="n">accessTokenSecret</span><span class="o">;</span>
<span class="n">Hmac</span> <span class="n">_sigHasher</span><span class="o">;</span>
<span class="kd">final</span> <span class="n">DateTime</span> <span class="n">_epochUtc</span> <span class="o">=</span> <span class="k">new</span> <span class="n">DateTime</span><span class="o">(</span><span class="mi">1970</span><span class="o">,</span> <span class="mi">1</span><span class="o">,</span> <span class="mi">1</span><span class="o">);</span>
<span class="c1">// TwitterApi class adapted from DanTup:</span>
<span class="c1">// https://blog.dantup.com/2017/01/simplest-dart-code-to-post-a-tweet-using-oauth/</span>
<span class="n">TwitterApi</span><span class="o">(</span><span class="k">this</span><span class="o">.</span><span class="na">consumerKey</span><span class="o">,</span> <span class="k">this</span><span class="o">.</span><span class="na">consumerKeySecret</span><span class="o">,</span> <span class="k">this</span><span class="o">.</span><span class="na">accessToken</span><span class="o">,</span>
<span class="k">this</span><span class="o">.</span><span class="na">accessTokenSecret</span><span class="o">)</span> <span class="o">{</span>
<span class="kd">var</span> <span class="n">bytes</span> <span class="o">=</span> <span class="n">UTF8</span><span class="o">.</span><span class="na">encode</span><span class="o">(</span><span class="s">"</span><span class="si">$consumerKeySecret</span><span class="s">&</span><span class="si">$accessTokenSecret</span><span class="s">"</span><span class="o">);</span>
<span class="n">_sigHasher</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Hmac</span><span class="o">(</span><span class="n">sha1</span><span class="o">,</span> <span class="n">bytes</span><span class="o">);</span>
<span class="o">}</span>
<span class="c1">/// Sends a tweet with the supplied text and returns the response from the Twitter API.</span>
<span class="n">Future</span><span class="o"><</span><span class="kt">String</span><span class="o">></span> <span class="n">tweet</span><span class="o">(</span><span class="kt">String</span> <span class="n">text</span><span class="o">)</span> <span class="o">{</span>
<span class="kd">var</span> <span class="n">data</span> <span class="o">=</span> <span class="o">{</span><span class="s">"status"</span><span class="o">:</span> <span class="n">text</span><span class="o">,</span> <span class="s">"trim_user"</span><span class="o">:</span> <span class="s">"1"</span><span class="o">};</span>
<span class="k">return</span> <span class="n">_callApi</span><span class="o">(</span><span class="s">"statuses/update.json"</span><span class="o">,</span> <span class="n">data</span><span class="o">);</span>
<span class="o">}</span>
<span class="n">Future</span><span class="o"><</span><span class="kt">String</span><span class="o">></span> <span class="n">_callApi</span><span class="o">(</span><span class="kt">String</span> <span class="n">url</span><span class="o">,</span> <span class="n">Map</span><span class="o"><</span><span class="kt">String</span><span class="o">,</span> <span class="kt">String</span><span class="o">></span> <span class="n">data</span><span class="o">)</span> <span class="o">{</span>
<span class="kd">var</span> <span class="n">fullUrl</span> <span class="o">=</span> <span class="n">Uri</span><span class="o">.</span><span class="na">parse</span><span class="o">(</span><span class="n">twitterApiBaseUrl</span> <span class="o">+</span> <span class="n">url</span><span class="o">);</span>
<span class="c1">// Timestamps are in seconds since 1/1/1970.</span>
<span class="kd">var</span> <span class="n">timestamp</span> <span class="o">=</span> <span class="k">new</span> <span class="n">DateTime</span><span class="o">.</span><span class="na">now</span><span class="o">().</span><span class="na">toUtc</span><span class="o">().</span><span class="na">difference</span><span class="o">(</span><span class="n">_epochUtc</span><span class="o">).</span><span class="na">inSeconds</span><span class="o">;</span>
<span class="c1">// Add all the OAuth headers we'll need to use when constructing the hash.</span>
<span class="n">data</span><span class="o">[</span><span class="s">"oauth_consumer_key"</span><span class="o">]</span> <span class="o">=</span> <span class="n">consumerKey</span><span class="o">;</span>
<span class="n">data</span><span class="o">[</span><span class="s">"oauth_signature_method"</span><span class="o">]</span> <span class="o">=</span> <span class="s">"HMAC-SHA1"</span><span class="o">;</span>
<span class="n">data</span><span class="o">[</span><span class="s">"oauth_timestamp"</span><span class="o">]</span> <span class="o">=</span> <span class="n">timestamp</span><span class="o">.</span><span class="na">toString</span><span class="o">();</span>
<span class="n">data</span><span class="o">[</span><span class="s">"oauth_nonce"</span><span class="o">]</span> <span class="o">=</span> <span class="s">"a"</span><span class="o">;</span> <span class="c1">// Required, but Twitter doesn't appear to use it</span>
<span class="n">data</span><span class="o">[</span><span class="s">"oauth_token"</span><span class="o">]</span> <span class="o">=</span> <span class="n">accessToken</span><span class="o">;</span>
<span class="n">data</span><span class="o">[</span><span class="s">"oauth_version"</span><span class="o">]</span> <span class="o">=</span> <span class="s">"1.0"</span><span class="o">;</span>
<span class="c1">// Generate the OAuth signature and add it to our payload.</span>
<span class="n">data</span><span class="o">[</span><span class="s">"oauth_signature"</span><span class="o">]</span> <span class="o">=</span> <span class="n">_generateSignature</span><span class="o">(</span><span class="n">fullUrl</span><span class="o">,</span> <span class="n">data</span><span class="o">);</span>
<span class="c1">// Build the OAuth HTTP Header from the data.</span>
<span class="kd">var</span> <span class="n">oAuthHeader</span> <span class="o">=</span> <span class="n">_generateOAuthHeader</span><span class="o">(</span><span class="n">data</span><span class="o">);</span>
<span class="c1">// Build the form data (exclude OAuth stuff that's already in the header).</span>
<span class="kd">var</span> <span class="n">formData</span> <span class="o">=</span> <span class="n">_filterMap</span><span class="o">(</span><span class="n">data</span><span class="o">,</span> <span class="o">(</span><span class="n">k</span><span class="o">)</span> <span class="o">=></span> <span class="o">!</span><span class="n">k</span><span class="o">.</span><span class="na">startsWith</span><span class="o">(</span><span class="s">"oauth_"</span><span class="o">));</span>
<span class="k">return</span> <span class="n">_sendRequest</span><span class="o">(</span><span class="n">fullUrl</span><span class="o">,</span> <span class="n">oAuthHeader</span><span class="o">,</span> <span class="n">_toQueryString</span><span class="o">(</span><span class="n">formData</span><span class="o">));</span>
<span class="o">}</span>
<span class="c1">/// Generate an OAuth signature from OAuth header values.</span>
<span class="kt">String</span> <span class="n">_generateSignature</span><span class="o">(</span><span class="n">Uri</span> <span class="n">url</span><span class="o">,</span> <span class="n">Map</span><span class="o"><</span><span class="kt">String</span><span class="o">,</span> <span class="kt">String</span><span class="o">></span> <span class="n">data</span><span class="o">)</span> <span class="o">{</span>
<span class="kd">var</span> <span class="n">sigString</span> <span class="o">=</span> <span class="n">_toQueryString</span><span class="o">(</span><span class="n">data</span><span class="o">);</span>
<span class="kd">var</span> <span class="n">fullSigData</span> <span class="o">=</span> <span class="s">"POST&</span><span class="si">${_encode(url.toString())}</span><span class="s">&</span><span class="si">${_encode(sigString)}</span><span class="s">"</span><span class="o">;</span>
<span class="k">return</span> <span class="n">BASE64</span><span class="o">.</span><span class="na">encode</span><span class="o">(</span><span class="n">_hash</span><span class="o">(</span><span class="n">fullSigData</span><span class="o">));</span>
<span class="o">}</span>
<span class="c1">/// Generate the raw OAuth HTML header from the values (including signature).</span>
<span class="kt">String</span> <span class="n">_generateOAuthHeader</span><span class="o">(</span><span class="n">Map</span><span class="o"><</span><span class="kt">String</span><span class="o">,</span> <span class="kt">String</span><span class="o">></span> <span class="n">data</span><span class="o">)</span> <span class="o">{</span>
<span class="kd">var</span> <span class="n">oauthHeaderValues</span> <span class="o">=</span> <span class="n">_filterMap</span><span class="o">(</span><span class="n">data</span><span class="o">,</span> <span class="o">(</span><span class="n">k</span><span class="o">)</span> <span class="o">=></span> <span class="n">k</span><span class="o">.</span><span class="na">startsWith</span><span class="o">(</span><span class="s">"oauth_"</span><span class="o">));</span>
<span class="k">return</span> <span class="s">"OAuth "</span> <span class="o">+</span> <span class="n">_toOAuthHeader</span><span class="o">(</span><span class="n">oauthHeaderValues</span><span class="o">);</span>
<span class="o">}</span>
<span class="c1">/// Send HTTP Request and return the response.</span>
<span class="n">Future</span><span class="o"><</span><span class="kt">String</span><span class="o">></span> <span class="n">_sendRequest</span><span class="o">(</span>
<span class="n">Uri</span> <span class="n">fullUrl</span><span class="o">,</span> <span class="kt">String</span> <span class="n">oAuthHeader</span><span class="o">,</span> <span class="kt">String</span> <span class="n">body</span><span class="o">)</span> <span class="n">async</span> <span class="o">{</span>
<span class="kd">final</span> <span class="n">http</span> <span class="o">=</span> <span class="k">new</span> <span class="n">HttpClient</span><span class="o">();</span>
<span class="kd">final</span> <span class="n">request</span> <span class="o">=</span> <span class="n">await</span> <span class="n">http</span><span class="o">.</span><span class="na">postUrl</span><span class="o">(</span><span class="n">fullUrl</span><span class="o">);</span>
<span class="n">request</span><span class="o">.</span><span class="na">headers</span>
<span class="o">..</span><span class="na">contentType</span> <span class="o">=</span> <span class="k">new</span> <span class="n">ContentType</span><span class="o">(</span><span class="s">"application"</span><span class="o">,</span> <span class="s">"x-www-form-urlencoded"</span><span class="o">,</span>
<span class="nl">charset:</span> <span class="s">"utf-8"</span><span class="o">)</span>
<span class="o">..</span><span class="na">add</span><span class="o">(</span><span class="s">"Authorization"</span><span class="o">,</span> <span class="n">oAuthHeader</span><span class="o">);</span>
<span class="n">request</span><span class="o">.</span><span class="na">write</span><span class="o">(</span><span class="n">body</span><span class="o">);</span>
<span class="kd">final</span> <span class="n">response</span> <span class="o">=</span> <span class="n">await</span> <span class="n">request</span><span class="o">.</span><span class="na">close</span><span class="o">().</span><span class="na">whenComplete</span><span class="o">(</span><span class="n">http</span><span class="o">.</span><span class="na">close</span><span class="o">);</span>
<span class="k">return</span> <span class="n">response</span><span class="o">.</span><span class="na">transform</span><span class="o">(</span><span class="n">UTF8</span><span class="o">.</span><span class="na">decoder</span><span class="o">).</span><span class="na">join</span><span class="o">(</span><span class="s">""</span><span class="o">);</span>
<span class="o">}</span>
<span class="n">Map</span><span class="o"><</span><span class="kt">String</span><span class="o">,</span> <span class="kt">String</span><span class="o">></span> <span class="n">_filterMap</span><span class="o">(</span>
<span class="n">Map</span><span class="o"><</span><span class="kt">String</span><span class="o">,</span> <span class="kt">String</span><span class="o">></span> <span class="n">map</span><span class="o">,</span> <span class="kt">bool</span> <span class="n">test</span><span class="o">(</span><span class="kt">String</span> <span class="n">key</span><span class="o">))</span> <span class="o">{</span>
<span class="k">return</span> <span class="k">new</span> <span class="n">Map</span><span class="o">.</span><span class="na">fromIterable</span><span class="o">(</span><span class="n">map</span><span class="o">.</span><span class="na">keys</span><span class="o">.</span><span class="na">where</span><span class="o">(</span><span class="n">test</span><span class="o">),</span> <span class="nl">value:</span> <span class="o">(</span><span class="n">k</span><span class="o">)</span> <span class="o">=></span> <span class="n">map</span><span class="o">[</span><span class="n">k</span><span class="o">]);</span>
<span class="o">}</span>
<span class="kt">String</span> <span class="n">_toQueryString</span><span class="o">(</span><span class="n">Map</span><span class="o"><</span><span class="kt">String</span><span class="o">,</span> <span class="kt">String</span><span class="o">></span> <span class="n">data</span><span class="o">)</span> <span class="o">{</span>
<span class="kd">var</span> <span class="n">items</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="na">keys</span><span class="o">.</span><span class="na">map</span><span class="o">((</span><span class="n">k</span><span class="o">)</span> <span class="o">=></span> <span class="s">"</span><span class="si">$k</span><span class="s">=</span><span class="si">${_encode(data[k])}</span><span class="s">"</span><span class="o">).</span><span class="na">toList</span><span class="o">();</span>
<span class="n">items</span><span class="o">.</span><span class="na">sort</span><span class="o">();</span>
<span class="k">return</span> <span class="n">items</span><span class="o">.</span><span class="na">join</span><span class="o">(</span><span class="s">"&"</span><span class="o">);</span>
<span class="o">}</span>
<span class="kt">String</span> <span class="n">_toOAuthHeader</span><span class="o">(</span><span class="n">Map</span><span class="o"><</span><span class="kt">String</span><span class="o">,</span> <span class="kt">String</span><span class="o">></span> <span class="n">data</span><span class="o">)</span> <span class="o">{</span>
<span class="kd">var</span> <span class="n">items</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="na">keys</span><span class="o">.</span><span class="na">map</span><span class="o">((</span><span class="n">k</span><span class="o">)</span> <span class="o">=></span> <span class="s">"</span><span class="si">$k</span><span class="s">=</span><span class="se">\"</span><span class="si">${_encode(data[k])}</span><span class="se">\"</span><span class="s">"</span><span class="o">).</span><span class="na">toList</span><span class="o">();</span>
<span class="n">items</span><span class="o">.</span><span class="na">sort</span><span class="o">();</span>
<span class="k">return</span> <span class="n">items</span><span class="o">.</span><span class="na">join</span><span class="o">(</span><span class="s">", "</span><span class="o">);</span>
<span class="o">}</span>
<span class="n">List</span><span class="o"><</span><span class="kt">int</span><span class="o">></span> <span class="n">_hash</span><span class="o">(</span><span class="kt">String</span> <span class="n">data</span><span class="o">)</span> <span class="o">=></span> <span class="n">_sigHasher</span><span class="o">.</span><span class="na">convert</span><span class="o">(</span><span class="n">data</span><span class="o">.</span><span class="na">codeUnits</span><span class="o">).</span><span class="na">bytes</span><span class="o">;</span>
<span class="kt">String</span> <span class="n">_encode</span><span class="o">(</span><span class="kt">String</span> <span class="n">data</span><span class="o">)</span> <span class="o">=></span> <span class="n">percent</span><span class="o">.</span><span class="na">encode</span><span class="o">(</span><span class="n">data</span><span class="o">.</span><span class="na">codeUnits</span><span class="o">);</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Youāre free to do as you please with this code. If you improve anything significantly, do leave a comment!</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2017/01/simplest-dart-code-to-post-a-tweet-using-oauth/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2017-01-06:/2017/01/visiting-a-site-that-uses-disqus-comments-when-not-logged-in-sends-the-url-to-facebook/2017-01-06T00:00:00+00:002017-01-06T00:00:00+00:00Visiting a site that uses Disqus comments when not logged in sends the URL to Facebook
<p><em>(<strong>Update</strong>: As a result of this post Disqus fixed this issue on 9th Jan! <a href="/2017/01/visiting-a-site-that-uses-disqus-comments-when-not-logged-in-sends-the-url-to-facebook/#comment-3091263180">See here</a>)</em></p>
<p>Recently I was making some tweaks to my blog and reviewing the page load time. I noticed that when I loaded an article the Facebook SDK was being loaded!</p>
<p><a href="/post_images/facebook_sdk_blog.png"><img src="/post_images/facebook_sdk_blog.png" alt="Facebook SDK being loaded into my blog" /></a></p>
<p>This was a bit strange as I didnāt recall ever having added any Facebook widgets to my blog. Not only is this somewhat wasteful and harming my load time, it also bugged me that visitors were having these URLs sent to Facebook. Iāve always thought it strange that Facebook seems to know <em>everywhere</em> Iāve been and it seems my blog is adding to this without me even knowing!</p>
<p>So, I checked why this file was being loaded and was surprised to see Disqus as the initiator. I do have Disqus comments on my blog but that doesnāt seem like a good reason to have the Facebook SDK loaded immediately! I <a href="https://twitter.com/DanTup/status/814902576449814528">tweeted Disqus</a> though unsurprisingly havenāt recieved a response.</p>
<p>Fast forward a week and I was reading Troy Huntās <a href="https://www.troyhunt.com/i-just-permanently-removed-all-ad-network-code-from-my-blog/">I just permanently removed all ad network code from my blog</a>. Troy cited tracking as one of the reasons for removing ads (I strongly support this decision; running unknown JavaScript from ad networks on an HTTPS site always seemed a bit weird to me). In a response on twitter I noticed someone asked him about still having Disqus, which reminded me about what Iād seen on my blog and wondered if the same thing happened there that I saw on my blog.</p>
<p>Well, it turns out it does. Hereās Troyās blog loading the Facebook SDK and clearly sending the URL for the post Iām reading to Facebook in the referer header (itās URL-encoded in the querystring of the Disqus comments frame URL).</p>
<p><a href="/post_images/troy_fb_tracking.png"><img src="/post_images/troy_fb_tracking.png" alt="Facebook SDK being loaded into my blog" /></a></p>
<p>Now, you might say that this isnāt really the URL of the page, itās going via Disqus and Facebook wonāt read it. Iād counter that since Facebook are very interested in tracking where you visit and Disqus is so common that it is in their best interest to understand this URL and extract the real article URL from the querystring.</p>
<p>I did a little more digging and discovered that this only happens if youāre not logged in to Disqus. It also happens even if <code class="language-plaintext highlighter-rouge">Do-Not-Track</code> is enabled in your browser. Itās almost certainly as a result of the āSign in with Facebookā functionality however I donāt believe it has to be (nor should be) this way. If I implemented āSign in with Facebookā on my site directly then Iāve chosen to pull the Facebook SDK in and probably understand the implications. Adding Disqus comments doesnāt really scream out that the Facebook SDK will be loaded for all āanonymousā visitors even before theyāve scrolled anywhere near the Disqus comments.</p>
<p>Iām certain Disqus could fix this, not only resulting in better load times but also better privacy for users. Yes, logging in with Facebook might become slightly slower as a result but this doesnāt seem like a compelling enough reason to keep it as-is to me.</p>
<p>I donāt think Disqus will change this as a result of my tweet so I figured Iād try and raise some awareness and maybe that would help encourage them.</p>
<p>Yes, my blog is still using Disqus today. If they donāt fix this then I will probably investigate moving away. I donāt know what good alternatives exist (or how easily I can move all existing comments - which have great value on some posts) so it will take some time if it happens.</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2017/01/visiting-a-site-that-uses-disqus-comments-when-not-logged-in-sends-the-url-to-facebook/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2016-12-02:/2016/12/using-alexa-to-preheat-the-car-and-check-battery-status/2016-12-02T00:00:00+00:002016-12-02T00:00:00+00:00Using Alexa (Amazon Echo) to Preheat my Renault ZOE and check Battery Status
<p>Yesterday my 2nd Gen Amazon Echo Dot arrived (check out <a href="https://www.youtube.com/watch?v=GKI7aKY_hxY">my kids review of the Echo Dot here</a>) and I thought itād be fun to try and hook it up to my <a href="https://www.renault.co.uk/vehicles/new-vehicles/zoe-250.html">Renault ZOE</a> so that I could check battery status and preheat it without having to load the clunky app.</p>
<iframe width="450" height="253" src="https://www.youtube.com/embed/CyCv6NzrtOI?VQ=HD720" frameborder="0" allowfullscreen=""></iframe>
<p>Alexa Skills can be hosted on AWS Lambda and thereās an <a href="https://developer.amazon.com/public/solutions/alexa/alexa-skills-kit/docs/developing-an-alexa-skill-as-a-lambda-function">easy-to-follow tutorial in the dev docs</a>. The Renault ZOE doesnāt have an official API but last year Iād used Fiddler to sniff the mobile app and been able to emulate that. Since then their app has just become a webview wrapper so I was a little worried I wouldnāt be able to do this - however, SPAs to the rescueā¦ the app is written in Angular and calls the backend in a really simple JSON API (way better than the previous XML abomination!).</p>
<p>I had a quick Google and found that <a href="https://shkspr.mobi/blog/2016/10/reverse-engineering-the-renault-zoe-api/">Terence Eden had already pasted all of the request/response info online</a> which saved me a little effort trawling through. I whipped up a quick node script that calls the API to login and request battery status / preheating. Itās relatively simple, not much code at all (though could do with a little refactoring).</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="dl">"</span><span class="s2">use strict</span><span class="dl">"</span><span class="p">;</span>
<span class="kd">let</span> <span class="nx">https</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">"</span><span class="s2">https</span><span class="dl">"</span><span class="p">);</span>
<span class="kd">let</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">vin</span><span class="p">,</span> <span class="nx">zoeUsername</span><span class="p">,</span> <span class="nx">zoePassword</span><span class="p">,</span> <span class="nx">loginFailureCallback</span><span class="p">;</span>
<span class="kd">function</span> <span class="nx">sendRequest</span><span class="p">(</span><span class="nx">action</span><span class="p">,</span> <span class="nx">requestData</span><span class="p">,</span> <span class="nx">successCallback</span><span class="p">,</span> <span class="nx">failureCallback</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">options</span> <span class="o">=</span> <span class="p">{</span>
<span class="na">hostname</span><span class="p">:</span> <span class="dl">"</span><span class="s2">www.services.renault-ze.com</span><span class="dl">"</span><span class="p">,</span>
<span class="na">port</span><span class="p">:</span> <span class="mi">443</span><span class="p">,</span>
<span class="na">path</span><span class="p">:</span> <span class="dl">"</span><span class="s2">/api</span><span class="dl">"</span> <span class="o">+</span> <span class="nx">action</span><span class="p">,</span>
<span class="na">method</span><span class="p">:</span> <span class="nx">requestData</span> <span class="p">?</span> <span class="dl">"</span><span class="s2">POST</span><span class="dl">"</span> <span class="p">:</span> <span class="dl">"</span><span class="s2">GET</span><span class="dl">"</span><span class="p">,</span>
<span class="na">headers</span><span class="p">:</span> <span class="p">{</span>
<span class="dl">"</span><span class="s2">Content-Type</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">application/json</span><span class="dl">"</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">Authorization</span><span class="dl">"</span><span class="p">:</span> <span class="p">(</span><span class="nx">token</span> <span class="p">?</span> <span class="dl">"</span><span class="s2">Bearer </span><span class="dl">"</span> <span class="o">+</span> <span class="nx">token</span> <span class="p">:</span> <span class="dl">""</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="kd">const</span> <span class="nx">req</span> <span class="o">=</span> <span class="nx">https</span><span class="p">.</span><span class="nx">request</span><span class="p">(</span><span class="nx">options</span><span class="p">,</span> <span class="nx">resp</span> <span class="o">=></span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">resp</span><span class="p">.</span><span class="nx">statusCode</span> <span class="o"><</span> <span class="mi">200</span> <span class="o">||</span> <span class="nx">resp</span><span class="p">.</span><span class="nx">statusCode</span> <span class="o">></span> <span class="mi">300</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">`Failed to send request </span><span class="p">${</span><span class="nx">action</span><span class="p">}</span><span class="s2"> (</span><span class="p">${</span><span class="nx">resp</span><span class="p">.</span><span class="nx">statusCode</span><span class="p">}</span><span class="s2">: </span><span class="p">${</span><span class="nx">resp</span><span class="p">.</span><span class="nx">statusMessage</span><span class="p">}</span><span class="s2">)`</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">failureCallback</span><span class="p">)</span>
<span class="nx">failureCallback</span><span class="p">();</span>
<span class="k">return</span><span class="p">;</span>
<span class="p">}</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">`Successful request </span><span class="p">${</span><span class="nx">action</span><span class="p">}</span><span class="s2"> (</span><span class="p">${</span><span class="nx">resp</span><span class="p">.</span><span class="nx">statusCode</span><span class="p">}</span><span class="s2">: </span><span class="p">${</span><span class="nx">resp</span><span class="p">.</span><span class="nx">statusMessage</span><span class="p">}</span><span class="s2">)`</span><span class="p">);</span>
<span class="kd">let</span> <span class="nx">respData</span> <span class="o">=</span> <span class="dl">""</span><span class="p">;</span>
<span class="nx">resp</span><span class="p">.</span><span class="nx">on</span><span class="p">(</span><span class="dl">"</span><span class="s2">data</span><span class="dl">"</span><span class="p">,</span> <span class="nx">c</span> <span class="o">=></span> <span class="p">{</span>
<span class="c1">//console.log("<== " + c.toString());</span>
<span class="nx">respData</span> <span class="o">+=</span> <span class="nx">c</span><span class="p">.</span><span class="nx">toString</span><span class="p">();</span>
<span class="p">});</span>
<span class="nx">resp</span><span class="p">.</span><span class="nx">on</span><span class="p">(</span><span class="dl">"</span><span class="s2">end</span><span class="dl">"</span><span class="p">,</span> <span class="p">()</span> <span class="o">=></span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">successCallback</span><span class="p">)</span>
<span class="nx">successCallback</span><span class="p">(</span><span class="nx">respData</span> <span class="o">&&</span> <span class="nx">respData</span><span class="p">.</span><span class="nx">length</span> <span class="p">?</span> <span class="nx">JSON</span><span class="p">.</span><span class="nx">parse</span><span class="p">(</span><span class="nx">respData</span><span class="p">)</span> <span class="p">:</span> <span class="kc">null</span><span class="p">);</span>
<span class="p">});</span>
<span class="p">});</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">requestData</span> <span class="o">&&</span> <span class="nx">JSON</span><span class="p">.</span><span class="nx">stringify</span><span class="p">(</span><span class="nx">requestData</span><span class="p">)</span> <span class="o">!==</span> <span class="dl">'</span><span class="s1">{}</span><span class="dl">'</span><span class="p">)</span>
<span class="nx">req</span><span class="p">.</span><span class="nx">write</span><span class="p">(</span><span class="nx">JSON</span><span class="p">.</span><span class="nx">stringify</span><span class="p">(</span><span class="nx">requestData</span><span class="p">));</span>
<span class="nx">req</span><span class="p">.</span><span class="nx">end</span><span class="p">();</span>
<span class="p">}</span>
<span class="kd">function</span> <span class="nx">login</span><span class="p">(</span><span class="nx">successCallback</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">sendRequest</span><span class="p">(</span><span class="dl">"</span><span class="s2">/user/login</span><span class="dl">"</span><span class="p">,</span> <span class="p">{</span>
<span class="na">username</span><span class="p">:</span> <span class="nx">zoeUsername</span><span class="p">,</span>
<span class="na">password</span><span class="p">:</span> <span class="nx">zoePassword</span>
<span class="p">},</span> <span class="nx">loginResponse</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">token</span> <span class="o">=</span> <span class="nx">loginResponse</span><span class="p">.</span><span class="nx">token</span><span class="p">;</span>
<span class="nx">vin</span> <span class="o">=</span> <span class="nx">loginResponse</span><span class="p">.</span><span class="nx">user</span><span class="p">.</span><span class="nx">vehicle_details</span><span class="p">.</span><span class="nx">VIN</span><span class="p">;</span>
<span class="nx">successCallback</span><span class="p">();</span>
<span class="p">},</span> <span class="nx">loginFailureCallback</span><span class="p">);</span>
<span class="p">}</span>
<span class="nx">exports</span><span class="p">.</span><span class="nx">setLogin</span> <span class="o">=</span> <span class="p">(</span><span class="nx">username</span><span class="p">,</span> <span class="nx">password</span><span class="p">,</span> <span class="nx">failureCallback</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">zoeUsername</span> <span class="o">=</span> <span class="nx">username</span><span class="p">;</span>
<span class="nx">zoePassword</span> <span class="o">=</span> <span class="nx">password</span><span class="p">;</span>
<span class="nx">loginFailureCallback</span> <span class="o">=</span> <span class="nx">failureCallback</span><span class="p">;</span>
<span class="p">}</span>
<span class="nx">exports</span><span class="p">.</span><span class="nx">getBatteryStatus</span> <span class="o">=</span> <span class="p">(</span><span class="nx">successCallback</span><span class="p">,</span> <span class="nx">failureCallback</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">login</span><span class="p">(()</span> <span class="o">=></span> <span class="nx">sendRequest</span><span class="p">(</span><span class="dl">"</span><span class="s2">/vehicle/</span><span class="dl">"</span> <span class="o">+</span> <span class="nx">vin</span> <span class="o">+</span> <span class="dl">"</span><span class="s2">/battery</span><span class="dl">"</span><span class="p">,</span> <span class="kc">null</span><span class="p">,</span> <span class="nx">successCallback</span><span class="p">,</span> <span class="nx">failureCallback</span><span class="p">));</span>
<span class="p">}</span>
<span class="nx">exports</span><span class="p">.</span><span class="nx">sendPreheatCommand</span> <span class="o">=</span> <span class="p">(</span><span class="nx">successCallback</span><span class="p">,</span> <span class="nx">failureCallback</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">login</span><span class="p">(()</span> <span class="o">=></span> <span class="nx">sendRequest</span><span class="p">(</span><span class="dl">"</span><span class="s2">/vehicle/</span><span class="dl">"</span> <span class="o">+</span> <span class="nx">vin</span> <span class="o">+</span> <span class="dl">"</span><span class="s2">/air-conditioning</span><span class="dl">"</span><span class="p">,</span> <span class="p">{},</span> <span class="nx">successCallback</span><span class="p">,</span> <span class="nx">failureCallback</span><span class="p">));</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Next was wiring this up to work the Alexa API. I created a separate JS file that <code class="language-plaintext highlighter-rouge">require'd</code> the one above and just handled two simple intents (<code class="language-plaintext highlighter-rouge">preheat</code> and <code class="language-plaintext highlighter-rouge">battery status</code>). If you ask for help or launch without an intent it just tells you what the options are. Most requests return a card too, so you can see them inside the Alexa app.</p>
<p>Since we have two cars (two ZOEs!), I actually set this up to respond to two different app IDs and connect to the correct account (and rejected any other apps, so if you jokesters find the ID you canāt burn my battery ;)). The usernames/passwords come from environment variables I would set up in Lambda to make it easier to share code without removing them.</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="dl">"</span><span class="s2">use strict</span><span class="dl">"</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">dannyAlexaApp</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">amzn1.ask.skill.xxxxx</span><span class="dl">"</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">heatherAlexaApp</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">amzn1.ask.skill.yyyyy</span><span class="dl">"</span><span class="p">;</span>
<span class="kd">let</span> <span class="nx">car</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">"</span><span class="s2">./car</span><span class="dl">"</span><span class="p">);</span>
<span class="kd">function</span> <span class="nx">buildResponse</span><span class="p">(</span><span class="nx">output</span><span class="p">,</span> <span class="nx">card</span><span class="p">,</span> <span class="nx">shouldEndSession</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="p">{</span>
<span class="na">version</span><span class="p">:</span> <span class="dl">"</span><span class="s2">1.0</span><span class="dl">"</span><span class="p">,</span>
<span class="na">response</span><span class="p">:</span> <span class="p">{</span>
<span class="na">outputSpeech</span><span class="p">:</span> <span class="p">{</span>
<span class="na">type</span><span class="p">:</span> <span class="dl">"</span><span class="s2">PlainText</span><span class="dl">"</span><span class="p">,</span>
<span class="na">text</span><span class="p">:</span> <span class="nx">output</span><span class="p">,</span>
<span class="p">},</span>
<span class="nx">card</span><span class="p">,</span>
<span class="nx">shouldEndSession</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="p">}</span>
<span class="c1">// Helper to build the text response from battery information.</span>
<span class="kd">function</span> <span class="nx">buildBatteryStatus</span><span class="p">(</span><span class="nx">battery</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">let</span> <span class="nx">response</span> <span class="o">=</span> <span class="s2">`You have </span><span class="p">${</span><span class="nx">battery</span><span class="p">.</span><span class="nx">charge_level</span><span class="p">}</span><span class="s2">% battery which will get you approximately </span><span class="p">${</span><span class="nb">Math</span><span class="p">.</span><span class="nx">round</span><span class="p">(</span><span class="nx">battery</span><span class="p">.</span><span class="nx">remaining_range</span> <span class="o">*</span> <span class="mf">0.621371</span><span class="p">)}</span><span class="s2"> miles. `</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">battery</span><span class="p">.</span><span class="nx">plugged</span><span class="p">)</span>
<span class="nx">response</span> <span class="o">+=</span> <span class="dl">"</span><span class="s2">The car is plugged in</span><span class="dl">"</span><span class="p">;</span>
<span class="k">else</span>
<span class="nx">response</span> <span class="o">+=</span> <span class="dl">"</span><span class="s2">The car is not plugged in</span><span class="dl">"</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">battery</span><span class="p">.</span><span class="nx">charging</span><span class="p">)</span>
<span class="nx">response</span> <span class="o">+=</span> <span class="dl">"</span><span class="s2"> and charging</span><span class="dl">"</span><span class="p">;</span>
<span class="k">return</span> <span class="nx">response</span> <span class="o">+</span> <span class="dl">"</span><span class="s2">.</span><span class="dl">"</span><span class="p">;</span>
<span class="p">}</span>
<span class="nx">exports</span><span class="p">.</span><span class="nx">handler</span> <span class="o">=</span> <span class="p">(</span><span class="nx">event</span><span class="p">,</span> <span class="nx">context</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="c1">// Helper to return a response with a card. </span>
<span class="kd">const</span> <span class="nx">sendResponse</span> <span class="o">=</span> <span class="p">(</span><span class="nx">title</span><span class="p">,</span> <span class="nx">text</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">context</span><span class="p">.</span><span class="nx">succeed</span><span class="p">(</span><span class="nx">buildResponse</span><span class="p">(</span><span class="nx">text</span><span class="p">,</span> <span class="p">{</span>
<span class="dl">"</span><span class="s2">type</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">Simple</span><span class="dl">"</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">title</span><span class="dl">"</span><span class="p">:</span> <span class="nx">title</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">content</span><span class="dl">"</span><span class="p">:</span> <span class="nx">text</span>
<span class="p">}));</span>
<span class="p">};</span>
<span class="k">try</span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">`event.session.application.applicationId=</span><span class="p">${</span><span class="nx">event</span><span class="p">.</span><span class="nx">session</span><span class="p">.</span><span class="nx">application</span><span class="p">.</span><span class="nx">applicationId</span><span class="p">}</span><span class="s2">`</span><span class="p">);</span>
<span class="c1">// Shared callbacks.</span>
<span class="kd">const</span> <span class="nx">exitCallback</span> <span class="o">=</span> <span class="p">()</span> <span class="o">=></span> <span class="nx">context</span><span class="p">.</span><span class="nx">succeed</span><span class="p">(</span><span class="nx">buildResponse</span><span class="p">(</span><span class="dl">"</span><span class="s2">Goodbye!</span><span class="dl">"</span><span class="p">));</span>
<span class="kd">const</span> <span class="nx">helpCallback</span> <span class="o">=</span> <span class="p">()</span> <span class="o">=></span> <span class="nx">context</span><span class="p">.</span><span class="nx">succeed</span><span class="p">(</span><span class="nx">buildResponse</span><span class="p">(</span><span class="dl">"</span><span class="s2">What would you like to do? You can preheat the car or ask for battery status.</span><span class="dl">"</span><span class="p">,</span> <span class="kc">null</span><span class="p">,</span> <span class="kc">false</span><span class="p">));</span>
<span class="kd">const</span> <span class="nx">loginFailureCallback</span> <span class="o">=</span> <span class="p">()</span> <span class="o">=></span> <span class="nx">sendResponse</span><span class="p">(</span><span class="dl">"</span><span class="s2">Authorisation Failure</span><span class="dl">"</span><span class="p">,</span> <span class="dl">"</span><span class="s2">Unable to login to Renault Z.E. Services, please check your login credentials.</span><span class="dl">"</span><span class="p">);</span>
<span class="c1">// Set login based on the Alexa app ID.</span>
<span class="c1">// We have two cars, so two activation names ("my car" and "Heather's car").</span>
<span class="c1">// If you're not either of these apps, you're not allowed to control our cars!</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">event</span><span class="p">.</span><span class="nx">session</span><span class="p">.</span><span class="nx">application</span><span class="p">.</span><span class="nx">applicationId</span> <span class="o">===</span> <span class="nx">dannyAlexaApp</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">car</span><span class="p">.</span><span class="nx">setLogin</span><span class="p">(</span><span class="nx">process</span><span class="p">.</span><span class="nx">env</span><span class="p">.</span><span class="nx">ZOE_USER_DANNY</span><span class="p">,</span> <span class="nx">process</span><span class="p">.</span><span class="nx">env</span><span class="p">.</span><span class="nx">ZOE_PASS_DANNY</span><span class="p">,</span> <span class="nx">loginFailureCallback</span><span class="p">);</span>
<span class="p">}</span> <span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nx">event</span><span class="p">.</span><span class="nx">session</span><span class="p">.</span><span class="nx">application</span><span class="p">.</span><span class="nx">applicationId</span> <span class="o">===</span> <span class="nx">heatherAlexaApp</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">car</span><span class="p">.</span><span class="nx">setLogin</span><span class="p">(</span><span class="nx">process</span><span class="p">.</span><span class="nx">env</span><span class="p">.</span><span class="nx">ZOE_USER_HEATHER</span><span class="p">,</span> <span class="nx">process</span><span class="p">.</span><span class="nx">env</span><span class="p">.</span><span class="nx">ZOE_PASS_HEATHER</span><span class="p">,</span> <span class="nx">loginFailureCallback</span><span class="p">);</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="nx">sendResponse</span><span class="p">(</span><span class="dl">"</span><span class="s2">Invalid Application ID</span><span class="dl">"</span><span class="p">,</span> <span class="dl">"</span><span class="s2">You are not allowed to use this service.</span><span class="dl">"</span><span class="p">);</span>
<span class="k">return</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">// Handle launches without intents by just asking what to do. </span>
<span class="k">if</span> <span class="p">(</span><span class="nx">event</span><span class="p">.</span><span class="nx">request</span><span class="p">.</span><span class="nx">type</span> <span class="o">===</span> <span class="dl">"</span><span class="s2">LaunchRequest</span><span class="dl">"</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">helpCallback</span><span class="p">();</span>
<span class="p">}</span> <span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nx">event</span><span class="p">.</span><span class="nx">request</span><span class="p">.</span><span class="nx">type</span> <span class="o">===</span> <span class="dl">"</span><span class="s2">IntentRequest</span><span class="dl">"</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// Handle different intents by sending commands to the API and providing callbacks.</span>
<span class="k">switch</span> <span class="p">(</span><span class="nx">event</span><span class="p">.</span><span class="nx">request</span><span class="p">.</span><span class="nx">intent</span><span class="p">.</span><span class="nx">name</span><span class="p">)</span> <span class="p">{</span>
<span class="k">case</span> <span class="dl">"</span><span class="s2">PreheatIntent</span><span class="dl">"</span><span class="p">:</span>
<span class="nx">car</span><span class="p">.</span><span class="nx">sendPreheatCommand</span><span class="p">(</span>
<span class="nx">response</span> <span class="o">=></span> <span class="nx">sendResponse</span><span class="p">(</span><span class="dl">"</span><span class="s2">Car Preheat</span><span class="dl">"</span><span class="p">,</span> <span class="dl">"</span><span class="s2">The car will begin preheating shortly.</span><span class="dl">"</span><span class="p">),</span>
<span class="p">()</span> <span class="o">=></span> <span class="nx">sendResponse</span><span class="p">(</span><span class="dl">"</span><span class="s2">Car Preheat</span><span class="dl">"</span><span class="p">,</span> <span class="dl">"</span><span class="s2">Unable to begin preheating. Have you already done this recently?</span><span class="dl">"</span><span class="p">)</span>
<span class="p">);</span>
<span class="k">break</span><span class="p">;</span>
<span class="k">case</span> <span class="dl">"</span><span class="s2">GetBatteryStatusIntent</span><span class="dl">"</span><span class="p">:</span>
<span class="nx">car</span><span class="p">.</span><span class="nx">getBatteryStatus</span><span class="p">(</span>
<span class="nx">battery</span> <span class="o">=></span> <span class="nx">sendResponse</span><span class="p">(</span><span class="dl">"</span><span class="s2">Car Battery Status</span><span class="dl">"</span><span class="p">,</span> <span class="nx">buildBatteryStatus</span><span class="p">(</span><span class="nx">battery</span><span class="p">)),</span>
<span class="p">()</span> <span class="o">=></span> <span class="nx">sendResponse</span><span class="p">(</span><span class="dl">"</span><span class="s2">Car Battery Status</span><span class="dl">"</span><span class="p">,</span> <span class="dl">"</span><span class="s2">Unable to get car battery status, please check your login details.</span><span class="dl">"</span><span class="p">)</span>
<span class="p">);</span>
<span class="k">break</span><span class="p">;</span>
<span class="k">case</span> <span class="dl">"</span><span class="s2">AMAZON.HelpIntent</span><span class="dl">"</span><span class="p">:</span>
<span class="nx">helpCallback</span><span class="p">();</span>
<span class="k">break</span><span class="p">;</span>
<span class="k">case</span> <span class="dl">"</span><span class="s2">AMAZON.StopIntent</span><span class="dl">"</span><span class="p">:</span>
<span class="k">case</span> <span class="dl">"</span><span class="s2">AMAZON.CancelIntent</span><span class="dl">"</span><span class="p">:</span>
<span class="nx">exitCallback</span><span class="p">();</span>
<span class="k">break</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span> <span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nx">event</span><span class="p">.</span><span class="nx">request</span><span class="p">.</span><span class="nx">type</span> <span class="o">===</span> <span class="dl">"</span><span class="s2">SessionEndedRequest</span><span class="dl">"</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">exitCallback</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">}</span> <span class="k">catch</span> <span class="p">(</span><span class="nx">err</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">error</span><span class="p">(</span><span class="nx">err</span><span class="p">.</span><span class="nx">message</span><span class="p">);</span>
<span class="nx">sendResponse</span><span class="p">(</span><span class="dl">"</span><span class="s2">Error Occurred</span><span class="dl">"</span><span class="p">,</span> <span class="dl">"</span><span class="s2">An error occurred. Fire the programmer! </span><span class="dl">"</span> <span class="o">+</span> <span class="nx">err</span><span class="p">.</span><span class="nx">message</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">};</span>
</code></pre></div></div>
<p>I created two apps in Alexa (both using the same Lambda function) and set up the intent schema:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span>
<span class="dl">"</span><span class="s2">intents</span><span class="dl">"</span><span class="p">:</span> <span class="p">[</span>
<span class="p">{</span> <span class="dl">"</span><span class="s2">intent</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">PreheatIntent</span><span class="dl">"</span> <span class="p">},</span>
<span class="p">{</span> <span class="dl">"</span><span class="s2">intent</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">GetBatteryStatusIntent</span><span class="dl">"</span> <span class="p">},</span>
<span class="p">{</span> <span class="dl">"</span><span class="s2">intent</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">AMAZON.HelpIntent</span><span class="dl">"</span> <span class="p">},</span>
<span class="p">{</span> <span class="dl">"</span><span class="s2">intent</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">AMAZON.StopIntent</span><span class="dl">"</span> <span class="p">}</span>
<span class="p">]</span>
<span class="p">}</span>
</code></pre></div></div>
<p>ā¦ and some utterances:</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>PreheatIntent pre heat
PreheatIntent preheat
PreheatIntent heat
PreheatIntent heater
PreheatIntent turn the heater on
PreheatIntent warm up
PreheatIntent heat up
PreheatIntent preheat for me
PreheatIntent start the heaters
PreheatIntent turn the heater on
PreheatIntent turn on the heater
PreheatIntent turn the heat on
PreheatIntent turn on the heat
PreheatIntent warm up
PreheatIntent make it toasty for me
PreheatIntent it's cold outside
PreheatIntent heat my seat up
GetBatteryStatusIntent battery status
GetBatteryStatusIntent status
GetBatteryStatusIntent battery
GetBatteryStatusIntent level
GetBatteryStatusIntent battery level
GetBatteryStatusIntent how much range
GetBatteryStatusIntent range
GetBatteryStatusIntent what is your battery status
GetBatteryStatusIntent tell me your status
GetBatteryStatusIntent are you charged
GetBatteryStatusIntent how do your batteries look
GetBatteryStatusIntent what is your battery level
GetBatteryStatusIntent how are your batteries
GetBatteryStatusIntent is there any charge left
GetBatteryStatusIntent how much charge is left
GetBatteryStatusIntent how much range is left
GetBatteryStatusIntent what is your range
GetBatteryStatusIntent how many miles can we drive
GetBatteryStatusIntent what is the range left in the battery
GetBatteryStatusIntent how much range is left in the battery
GetBatteryStatusIntent how far can we drive
</code></pre></div></div>
<p>Finally, I created a little dummy script that lets me test the functions without having to go through the Echo for a faster dev cycle. I didnāt go as far as mocking the HTTP requests but I did modify the code temporarily to ensure the error handling worked properly (detecting invalid username/password, or if you try to pre-heat too often).</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="dl">"</span><span class="s2">use strict</span><span class="dl">"</span><span class="p">;</span>
<span class="kd">let</span> <span class="nx">zoe</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">./lambda/index</span><span class="dl">'</span><span class="p">);</span>
<span class="kd">let</span> <span class="nx">batteryStatusEvent</span> <span class="o">=</span> <span class="p">{</span>
<span class="dl">"</span><span class="s2">session</span><span class="dl">"</span><span class="p">:</span> <span class="p">{</span>
<span class="dl">"</span><span class="s2">application</span><span class="dl">"</span><span class="p">:</span> <span class="p">{</span>
<span class="dl">"</span><span class="s2">applicationId</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">amzn1.ask.skill.zzz</span><span class="dl">"</span>
<span class="p">}</span>
<span class="p">},</span>
<span class="dl">"</span><span class="s2">version</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">1.0</span><span class="dl">"</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">request</span><span class="dl">"</span><span class="p">:</span> <span class="p">{</span>
<span class="dl">"</span><span class="s2">type</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">IntentRequest</span><span class="dl">"</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">intent</span><span class="dl">"</span><span class="p">:</span> <span class="p">{</span> <span class="dl">"</span><span class="s2">name</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">GetBatteryStatusIntent</span><span class="dl">"</span> <span class="p">}</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="kd">let</span> <span class="nx">preheatEvent</span> <span class="o">=</span> <span class="p">{</span>
<span class="dl">"</span><span class="s2">session</span><span class="dl">"</span><span class="p">:</span> <span class="p">{</span>
<span class="dl">"</span><span class="s2">application</span><span class="dl">"</span><span class="p">:</span> <span class="p">{</span>
<span class="dl">"</span><span class="s2">applicationId</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">amzn1.ask.skill.zzz</span><span class="dl">"</span>
<span class="p">}</span>
<span class="p">},</span>
<span class="dl">"</span><span class="s2">version</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">1.0</span><span class="dl">"</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">request</span><span class="dl">"</span><span class="p">:</span> <span class="p">{</span>
<span class="dl">"</span><span class="s2">type</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">IntentRequest</span><span class="dl">"</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">intent</span><span class="dl">"</span><span class="p">:</span> <span class="p">{</span> <span class="dl">"</span><span class="s2">name</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">PreheatIntent</span><span class="dl">"</span> <span class="p">}</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="kd">let</span> <span class="nx">context</span> <span class="o">=</span> <span class="p">{</span>
<span class="na">succeed</span><span class="p">:</span> <span class="kd">function</span> <span class="p">(</span><span class="nx">resp</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="dl">"</span><span class="se">\n\n</span><span class="s2">Result:</span><span class="se">\n</span><span class="dl">"</span> <span class="o">+</span> <span class="nx">JSON</span><span class="p">.</span><span class="nx">stringify</span><span class="p">(</span><span class="nx">resp</span><span class="p">,</span> <span class="kc">null</span><span class="p">,</span> <span class="mi">4</span><span class="p">));</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="nx">zoe</span><span class="p">.</span><span class="nx">handler</span><span class="p">(</span><span class="nx">batteryStatusEvent</span><span class="p">,</span> <span class="nx">context</span><span class="p">);</span>
<span class="nx">zoe</span><span class="p">.</span><span class="nx">handler</span><span class="p">(</span><span class="nx">preheatEvent</span><span class="p">,</span> <span class="nx">context</span><span class="p">);</span>
</code></pre></div></div>
<p>After some successfully tests, I zipped the files up and uploaded them to Lambda to replace the favourite-color sample Iād been testing with. The final result is in the video at the top of this post!</p>
<p>Youāre free to do as you please with this code. If you improve anything significantly, do leave a comment! If youāve done anything similar with your Echo Dot, do leave a comment!</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2016/12/using-alexa-to-preheat-the-car-and-check-battery-status/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2016-07-07:/2016/07/simplest-csharp-code-to-post-a-tweet-using-oauth/2016-07-07T00:00:00+00:002016-07-07T00:00:00+00:00Simplest C# code to post a tweet using OAuth
<blockquote>
<p><strong>Edit</strong>: Iāve <a href="/2017/01/simplest-dart-code-to-post-a-tweet-using-oauth/">posted a Dart port of this code here</a>.</p>
</blockquote>
<p>For a few years Iāve been meaning to automate the posting of tweets to a couple of twitter accounts I run. Iāve always been too <s>lazy</s>busy but this week I had some time and decided to get it done. For many reasons (including that my parents taught me not to take binary dependencies from strangers on the internet ;)) I decided not to use a 5,000 line-of-code third party library but just instead write the simplest code for what I needed.</p>
<p>While Googling for other peoples implementations of āsimplest code to post tweetsā to see what I was getting myself into (I didnāt actually find any!) I stumbled across a page on the Twitter site about <a href="https://dev.twitter.com/oauth/overview/single-user">Single-user OAuth</a>. This allows you to generate tokens for authenticating without having to go through the normal OAuth ballache. Obviously you can only do this for your own account but it <em>massively</em> simplifies things.</p>
<blockquote>
<p>By using a single access token, you donāt need to implement the entire OAuth token acquisition dance. Instead, you can pick up from the point where you are working with an access token to make signed requests for Twitter resources.</p>
</blockquote>
<p>Most of the work is fairly straight forward but it also requires an OAuth signature in the OAuth header. This was a bit of a pain in the assā¦ You have to build a big string of all the data in a specific order with specific encoding and then hash it with the secret keys you got. I implemented this (or so I thought) and sent it off to Twitter only go get a <strong>BAD AUTHENTICATION DATA</strong> response with zero information on what exactly was wrong. Great :(</p>
<p>After much keyboard-bashing I ragequit for the night. Stupid Twitter. After some much-required sleep I realised that on the <a href="https://dev.twitter.com/oauth/overview/creating-signatures">Creating a signature page</a>, Twitter actually gave an example <em>including the secret keys and timestamp</em>. This is a perfect test case for testing out hashing code! It even gives the values at each step of the algorithm to help track down where youāre going wrong. My issues turned out to mostly be encoding - Iād tried using <code class="language-plaintext highlighter-rouge">WebUtility.UrlEncode</code> and <code class="language-plaintext highlighter-rouge">Uri.EscapeUriString</code> but neither encoded spaces and pluses the way that Twitter excepted. It turned out that <code class="language-plaintext highlighter-rouge">Uri.EscapeDataString</code> does encode exactly as Twitter requires.</p>
<p>In total, my class turned out to be around 80 lines of code without comments. Itās provided here in its entirety for easy copy/pasting (no, thereās no NuGet package.. didnāt anyone tell you that you shouldnāt trust binary dependencies from strangers on the internet?).</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">/// <summary></span>
<span class="c1">/// Simple class for sending tweets to Twitter using Single-user OAuth.</span>
<span class="c1">/// https://dev.twitter.com/oauth/overview/single-user</span>
<span class="c1">/// </span>
<span class="c1">/// Get your access keys by creating an app at apps.twitter.com then visiting the</span>
<span class="c1">/// "Keys and Access Tokens" section for your app. They can be found under the</span>
<span class="c1">/// "Your Access Token" heading.</span>
<span class="c1">/// </summary></span>
<span class="k">class</span> <span class="nc">TwitterApi</span>
<span class="p">{</span>
<span class="k">const</span> <span class="kt">string</span> <span class="n">TwitterApiBaseUrl</span> <span class="p">=</span> <span class="s">"https://api.twitter.com/1.1/"</span><span class="p">;</span>
<span class="k">readonly</span> <span class="kt">string</span> <span class="n">consumerKey</span><span class="p">,</span> <span class="n">consumerKeySecret</span><span class="p">,</span> <span class="n">accessToken</span><span class="p">,</span> <span class="n">accessTokenSecret</span><span class="p">;</span>
<span class="k">readonly</span> <span class="n">HMACSHA1</span> <span class="n">sigHasher</span><span class="p">;</span>
<span class="k">readonly</span> <span class="n">DateTime</span> <span class="n">epochUtc</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">DateTime</span><span class="p">(</span><span class="m">1970</span><span class="p">,</span> <span class="m">1</span><span class="p">,</span> <span class="m">1</span><span class="p">,</span> <span class="m">0</span><span class="p">,</span> <span class="m">0</span><span class="p">,</span> <span class="m">0</span><span class="p">,</span> <span class="n">DateTimeKind</span><span class="p">.</span><span class="n">Utc</span><span class="p">);</span>
<span class="c1">/// <summary></span>
<span class="c1">/// Creates an object for sending tweets to Twitter using Single-user OAuth.</span>
<span class="c1">/// </span>
<span class="c1">/// Get your access keys by creating an app at apps.twitter.com then visiting the</span>
<span class="c1">/// "Keys and Access Tokens" section for your app. They can be found under the</span>
<span class="c1">/// "Your Access Token" heading.</span>
<span class="c1">/// </summary></span>
<span class="k">public</span> <span class="nf">TwitterApi</span><span class="p">(</span><span class="kt">string</span> <span class="n">consumerKey</span><span class="p">,</span> <span class="kt">string</span> <span class="n">consumerKeySecret</span><span class="p">,</span> <span class="kt">string</span> <span class="n">accessToken</span><span class="p">,</span> <span class="kt">string</span> <span class="n">accessTokenSecret</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">this</span><span class="p">.</span><span class="n">consumerKey</span> <span class="p">=</span> <span class="n">consumerKey</span><span class="p">;</span>
<span class="k">this</span><span class="p">.</span><span class="n">consumerKeySecret</span> <span class="p">=</span> <span class="n">consumerKeySecret</span><span class="p">;</span>
<span class="k">this</span><span class="p">.</span><span class="n">accessToken</span> <span class="p">=</span> <span class="n">accessToken</span><span class="p">;</span>
<span class="k">this</span><span class="p">.</span><span class="n">accessTokenSecret</span> <span class="p">=</span> <span class="n">accessTokenSecret</span><span class="p">;</span>
<span class="n">sigHasher</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">HMACSHA1</span><span class="p">(</span><span class="k">new</span> <span class="nf">ASCIIEncoding</span><span class="p">().</span><span class="nf">GetBytes</span><span class="p">(</span><span class="kt">string</span><span class="p">.</span><span class="nf">Format</span><span class="p">(</span><span class="s">"{0}&{1}"</span><span class="p">,</span> <span class="n">consumerKeySecret</span><span class="p">,</span> <span class="n">accessTokenSecret</span><span class="p">)));</span>
<span class="p">}</span>
<span class="c1">/// <summary></span>
<span class="c1">/// Sends a tweet with the supplied text and returns the response from the Twitter API.</span>
<span class="c1">/// </summary></span>
<span class="k">public</span> <span class="n">Task</span><span class="p"><</span><span class="kt">string</span><span class="p">></span> <span class="nf">Tweet</span><span class="p">(</span><span class="kt">string</span> <span class="n">text</span><span class="p">)</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">data</span> <span class="p">=</span> <span class="k">new</span> <span class="n">Dictionary</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">string</span><span class="p">></span> <span class="p">{</span>
<span class="p">{</span> <span class="s">"status"</span><span class="p">,</span> <span class="n">text</span> <span class="p">},</span>
<span class="p">{</span> <span class="s">"trim_user"</span><span class="p">,</span> <span class="s">"1"</span> <span class="p">}</span>
<span class="p">};</span>
<span class="k">return</span> <span class="nf">SendRequest</span><span class="p">(</span><span class="s">"statuses/update.json"</span><span class="p">,</span> <span class="n">data</span><span class="p">);</span>
<span class="p">}</span>
<span class="n">Task</span><span class="p"><</span><span class="kt">string</span><span class="p">></span> <span class="nf">SendRequest</span><span class="p">(</span><span class="kt">string</span> <span class="n">url</span><span class="p">,</span> <span class="n">Dictionary</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">string</span><span class="p">></span> <span class="n">data</span><span class="p">)</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">fullUrl</span> <span class="p">=</span> <span class="n">TwitterApiBaseUrl</span> <span class="p">+</span> <span class="n">url</span><span class="p">;</span>
<span class="c1">// Timestamps are in seconds since 1/1/1970.</span>
<span class="kt">var</span> <span class="n">timestamp</span> <span class="p">=</span> <span class="p">(</span><span class="kt">int</span><span class="p">)((</span><span class="n">DateTime</span><span class="p">.</span><span class="n">UtcNow</span> <span class="p">-</span> <span class="n">epochUtc</span><span class="p">).</span><span class="n">TotalSeconds</span><span class="p">);</span>
<span class="c1">// Add all the OAuth headers we'll need to use when constructing the hash.</span>
<span class="n">data</span><span class="p">.</span><span class="nf">Add</span><span class="p">(</span><span class="s">"oauth_consumer_key"</span><span class="p">,</span> <span class="n">consumerKey</span><span class="p">);</span>
<span class="n">data</span><span class="p">.</span><span class="nf">Add</span><span class="p">(</span><span class="s">"oauth_signature_method"</span><span class="p">,</span> <span class="s">"HMAC-SHA1"</span><span class="p">);</span>
<span class="n">data</span><span class="p">.</span><span class="nf">Add</span><span class="p">(</span><span class="s">"oauth_timestamp"</span><span class="p">,</span> <span class="n">timestamp</span><span class="p">.</span><span class="nf">ToString</span><span class="p">());</span>
<span class="n">data</span><span class="p">.</span><span class="nf">Add</span><span class="p">(</span><span class="s">"oauth_nonce"</span><span class="p">,</span> <span class="s">"a"</span><span class="p">);</span> <span class="c1">// Required, but Twitter doesn't appear to use it, so "a" will do.</span>
<span class="n">data</span><span class="p">.</span><span class="nf">Add</span><span class="p">(</span><span class="s">"oauth_token"</span><span class="p">,</span> <span class="n">accessToken</span><span class="p">);</span>
<span class="n">data</span><span class="p">.</span><span class="nf">Add</span><span class="p">(</span><span class="s">"oauth_version"</span><span class="p">,</span> <span class="s">"1.0"</span><span class="p">);</span>
<span class="c1">// Generate the OAuth signature and add it to our payload.</span>
<span class="n">data</span><span class="p">.</span><span class="nf">Add</span><span class="p">(</span><span class="s">"oauth_signature"</span><span class="p">,</span> <span class="nf">GenerateSignature</span><span class="p">(</span><span class="n">fullUrl</span><span class="p">,</span> <span class="n">data</span><span class="p">));</span>
<span class="c1">// Build the OAuth HTTP Header from the data.</span>
<span class="kt">string</span> <span class="n">oAuthHeader</span> <span class="p">=</span> <span class="nf">GenerateOAuthHeader</span><span class="p">(</span><span class="n">data</span><span class="p">);</span>
<span class="c1">// Build the form data (exclude OAuth stuff that's already in the header).</span>
<span class="kt">var</span> <span class="n">formData</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">FormUrlEncodedContent</span><span class="p">(</span><span class="n">data</span><span class="p">.</span><span class="nf">Where</span><span class="p">(</span><span class="n">kvp</span> <span class="p">=></span> <span class="p">!</span><span class="n">kvp</span><span class="p">.</span><span class="n">Key</span><span class="p">.</span><span class="nf">StartsWith</span><span class="p">(</span><span class="s">"oauth_"</span><span class="p">)));</span>
<span class="k">return</span> <span class="nf">SendRequest</span><span class="p">(</span><span class="n">fullUrl</span><span class="p">,</span> <span class="n">oAuthHeader</span><span class="p">,</span> <span class="n">formData</span><span class="p">);</span>
<span class="p">}</span>
<span class="c1">/// <summary></span>
<span class="c1">/// Generate an OAuth signature from OAuth header values.</span>
<span class="c1">/// </summary></span>
<span class="kt">string</span> <span class="nf">GenerateSignature</span><span class="p">(</span><span class="kt">string</span> <span class="n">url</span><span class="p">,</span> <span class="n">Dictionary</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">string</span><span class="p">></span> <span class="n">data</span><span class="p">)</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">sigString</span> <span class="p">=</span> <span class="kt">string</span><span class="p">.</span><span class="nf">Join</span><span class="p">(</span>
<span class="s">"&"</span><span class="p">,</span>
<span class="n">data</span>
<span class="p">.</span><span class="nf">Union</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
<span class="p">.</span><span class="nf">Select</span><span class="p">(</span><span class="n">kvp</span> <span class="p">=></span> <span class="kt">string</span><span class="p">.</span><span class="nf">Format</span><span class="p">(</span><span class="s">"{0}={1}"</span><span class="p">,</span> <span class="n">Uri</span><span class="p">.</span><span class="nf">EscapeDataString</span><span class="p">(</span><span class="n">kvp</span><span class="p">.</span><span class="n">Key</span><span class="p">),</span> <span class="n">Uri</span><span class="p">.</span><span class="nf">EscapeDataString</span><span class="p">(</span><span class="n">kvp</span><span class="p">.</span><span class="n">Value</span><span class="p">)))</span>
<span class="p">.</span><span class="nf">OrderBy</span><span class="p">(</span><span class="n">s</span> <span class="p">=></span> <span class="n">s</span><span class="p">)</span>
<span class="p">);</span>
<span class="kt">var</span> <span class="n">fullSigData</span> <span class="p">=</span> <span class="kt">string</span><span class="p">.</span><span class="nf">Format</span><span class="p">(</span>
<span class="s">"{0}&{1}&{2}"</span><span class="p">,</span>
<span class="s">"POST"</span><span class="p">,</span>
<span class="n">Uri</span><span class="p">.</span><span class="nf">EscapeDataString</span><span class="p">(</span><span class="n">url</span><span class="p">),</span>
<span class="n">Uri</span><span class="p">.</span><span class="nf">EscapeDataString</span><span class="p">(</span><span class="n">sigString</span><span class="p">.</span><span class="nf">ToString</span><span class="p">())</span>
<span class="p">);</span>
<span class="k">return</span> <span class="n">Convert</span><span class="p">.</span><span class="nf">ToBase64String</span><span class="p">(</span><span class="n">sigHasher</span><span class="p">.</span><span class="nf">ComputeHash</span><span class="p">(</span><span class="k">new</span> <span class="nf">ASCIIEncoding</span><span class="p">().</span><span class="nf">GetBytes</span><span class="p">(</span><span class="n">fullSigData</span><span class="p">.</span><span class="nf">ToString</span><span class="p">())));</span>
<span class="p">}</span>
<span class="c1">/// <summary></span>
<span class="c1">/// Generate the raw OAuth HTML header from the values (including signature).</span>
<span class="c1">/// </summary></span>
<span class="kt">string</span> <span class="nf">GenerateOAuthHeader</span><span class="p">(</span><span class="n">Dictionary</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">string</span><span class="p">></span> <span class="n">data</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="s">"OAuth "</span> <span class="p">+</span> <span class="kt">string</span><span class="p">.</span><span class="nf">Join</span><span class="p">(</span>
<span class="s">", "</span><span class="p">,</span>
<span class="n">data</span>
<span class="p">.</span><span class="nf">Where</span><span class="p">(</span><span class="n">kvp</span> <span class="p">=></span> <span class="n">kvp</span><span class="p">.</span><span class="n">Key</span><span class="p">.</span><span class="nf">StartsWith</span><span class="p">(</span><span class="s">"oauth_"</span><span class="p">))</span>
<span class="p">.</span><span class="nf">Select</span><span class="p">(</span><span class="n">kvp</span> <span class="p">=></span> <span class="kt">string</span><span class="p">.</span><span class="nf">Format</span><span class="p">(</span><span class="s">"{0}=\"{1}\""</span><span class="p">,</span> <span class="n">Uri</span><span class="p">.</span><span class="nf">EscapeDataString</span><span class="p">(</span><span class="n">kvp</span><span class="p">.</span><span class="n">Key</span><span class="p">),</span> <span class="n">Uri</span><span class="p">.</span><span class="nf">EscapeDataString</span><span class="p">(</span><span class="n">kvp</span><span class="p">.</span><span class="n">Value</span><span class="p">)))</span>
<span class="p">.</span><span class="nf">OrderBy</span><span class="p">(</span><span class="n">s</span> <span class="p">=></span> <span class="n">s</span><span class="p">)</span>
<span class="p">);</span>
<span class="p">}</span>
<span class="c1">/// <summary></span>
<span class="c1">/// Send HTTP Request and return the response.</span>
<span class="c1">/// </summary></span>
<span class="k">async</span> <span class="n">Task</span><span class="p"><</span><span class="kt">string</span><span class="p">></span> <span class="nf">SendRequest</span><span class="p">(</span><span class="kt">string</span> <span class="n">fullUrl</span><span class="p">,</span> <span class="kt">string</span> <span class="n">oAuthHeader</span><span class="p">,</span> <span class="n">FormUrlEncodedContent</span> <span class="n">formData</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">using</span> <span class="p">(</span><span class="kt">var</span> <span class="n">http</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">HttpClient</span><span class="p">())</span>
<span class="p">{</span>
<span class="n">http</span><span class="p">.</span><span class="n">DefaultRequestHeaders</span><span class="p">.</span><span class="nf">Add</span><span class="p">(</span><span class="s">"Authorization"</span><span class="p">,</span> <span class="n">oAuthHeader</span><span class="p">);</span>
<span class="kt">var</span> <span class="n">httpResp</span> <span class="p">=</span> <span class="k">await</span> <span class="n">http</span><span class="p">.</span><span class="nf">PostAsync</span><span class="p">(</span><span class="n">fullUrl</span><span class="p">,</span> <span class="n">formData</span><span class="p">);</span>
<span class="kt">var</span> <span class="n">respBody</span> <span class="p">=</span> <span class="k">await</span> <span class="n">httpResp</span><span class="p">.</span><span class="n">Content</span><span class="p">.</span><span class="nf">ReadAsStringAsync</span><span class="p">();</span>
<span class="k">return</span> <span class="n">respBody</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>And to use it:</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">var</span> <span class="n">twitter</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">TwitterApi</span><span class="p">(</span><span class="n">ConsumerKey</span><span class="p">,</span> <span class="n">ConsumerKeySecret</span><span class="p">,</span> <span class="n">AccessToken</span><span class="p">,</span> <span class="n">AccessTokenSecret</span><span class="p">);</span>
<span class="kt">var</span> <span class="n">response</span> <span class="p">=</span> <span class="k">await</span> <span class="n">twitter</span><span class="p">.</span><span class="nf">Tweet</span><span class="p">(</span><span class="s">"This is my first automated tweet!"</span><span class="p">);</span>
<span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="n">response</span><span class="p">);</span>
</code></pre></div></div>
<p>Youāre free to do as you please with this code. If you improve anything significantly, do leave a comment!</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2016/07/simplest-csharp-code-to-post-a-tweet-using-oauth/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2016-06-25:/2016/06/html5-pong-in-20min-with-csharp-bridgedotnet/2016-06-25T00:00:00+00:002016-06-25T00:00:00+00:00HTML5 Pong in 20min with C#/Bridge.NET
<p><a href="https://github.com/DanTup/bridge-pong">
<img style="position: absolute; top: 0; right: 0; border: 0;" src="https://camo.githubusercontent.com/a6677b08c955af8400f44c6298f40e7d19cc5b2d/68747470733a2f2f73332e616d617a6f6e6177732e636f6d2f6769746875622f726962626f6e732f666f726b6d655f72696768745f677261795f3664366436642e706e67" alt="Fork me on GitHub" data-canonical-src="https://s3.amazonaws.com/github/ribbons/forkme_right_gray_6d6d6d.png" />
</a></p>
<p>I recorded a short (20min) video of creating a simple Pong game in HTML5 using C#/<a href="http://bridge.net/">Bridge.NET</a>. Itās not the best Pong game in the world but it is from scratch with no libraries other than Bridge. Maybe useful if youāre thinking of picking up Bridge or building a simple Canvas game.</p>
<iframe width="450" height="253" src="https://www.youtube.com/embed/5IVHFX_Q0wA?VQ=HD720" frameborder="0" allowfullscreen=""></iframe>
<p>All source code for this project can be found <a href="https://github.com/DanTup/bridge-pong">on GitHub</a>.</p>
<blockquote>
<p><strong>Edit</strong>: Geoffrey McGill from the Bridge team copied this code to deck.net so you can run it live and play around with tweaking the code! <a href="http://deck.net/59eed4385a7f4caee8a0232a63cf72f7">http://deck.net/59eed4385a7f4caee8a0232a63cf72f7</a></p>
</blockquote>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2016/06/html5-pong-in-20min-with-csharp-bridgedotnet/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2016-06-12:/2016/06/dachip8js-my-csharp-chip8-interpreter-running-in-the-browser/2016-06-12T00:00:00+00:002016-06-12T00:00:00+00:00DaChip8JS - My C# Chip-8 Interpreter running in the browser
<p><a href="https://github.com/DanTup/DaChip8JS">
<img style="position: absolute; top: 0; right: 0; border: 0;" src="https://camo.githubusercontent.com/a6677b08c955af8400f44c6298f40e7d19cc5b2d/68747470733a2f2f73332e616d617a6f6e6177732e636f6d2f6769746875622f726962626f6e732f666f726b6d655f72696768745f677261795f3664366436642e706e67" alt="Fork me on GitHub" data-canonical-src="https://s3.amazonaws.com/github/ribbons/forkme_right_gray_6d6d6d.png" />
</a></p>
<p>Yesterday I <a href="/2016/06/building-a-chip-8-interpreter-in-csharp/">blogged about building DaChip8, my C# Chip-8 interpreter</a> (I recommend reading that post before this one; the changes detailed here will make more sense if you have an idea of the original code written). It works pretty well but running on the desktop is sooo 2010. Itās 2016, itās Browser or Bust.</p>
<p>The problem with the browser is JavaScript. I know lots of you love it, but I donāt. Luckily we have many options for compiling other languages down to JavaScript and the one I happen to have been using recently at work is <a href="http://bridge.net/">Bridge.NET</a>. I thought itād be fun to run my code through it and see whatās involved in making it run in the browser. It turned out to be much less effort than I expected and <a href="https://github.com/DanTup/DaChip8JS/blob/master/DaChip8JS/Chip8.cs">the main chip implementation</a> is identical code to <a href="https://github.com/DanTup/DaChip8/blob/master/DaChip8/Chip8.cs">the C# version</a>!</p>
<h2 id="live-demo">Live Demo</h2>
<p>Before we go on, here it is. Running in the browser. It works for me in Chrome and Edge but not in IE (no ArrayBuffer support on XmlHttpRequest to fetch the ROM, and possibly more). <strong>It even works on Chrome for Android</strong> (with touch)! <em>YMMV</em>.</p>
<p>The ROM loaded below is <em>Breakout (Brix hack) [David Winter, 1997]</em> from the <a href="http://www.chip8.com/?page=109">Chip-8 Pack</a>.</p>
<p>You can use the cursor keys to move the paddle on a desktop or tap on the left/right half of the gamescreen on a mobile device.</p>
<script src="/post_scripts/DaChip8JS.js"></script>
<div style="position: relative; width: 640px; max-width: 100%; margin: auto;">
<button id="start-dachip8js-game" style="position: absolute; width: 100%; height: 100%; background-color: #000; font-size: 30px; color: #fff;">Click to Play</button>
<canvas id="screen" width="64" height="32" style="width: 640px; image-rendering: pixelated; background-color: #000; cursor: none;"></canvas>
</div>
<div id="debug" style="display: none;"></div>
<h2 id="video">Video</h2>
<p>In case the demo doesnāt work in your browser, hereās a little video (taken well before it was complete, but my internet sucks too much to upload another one!):</p>
<iframe width="450" height="253" src="https://www.youtube.com/embed/tDgoDhCumPg?VQ=HD720" frameborder="0" allowfullscreen=""></iframe>
<h2 id="code-changes">Code Changes</h2>
<p>Obviously since rendering on the web is very different to rendering in WinForms I had to make some changes. I also had to work with C# 5 for Bridge (for now). The changes are all external to the Chip8 class (which has hooks to call out for rendering the buffer to screen or playing a beep to keep it platform-agnostic). Below is a summary of the changes I had to make.</p>
<h3 id="c-5">C# 5</h3>
<p>Currently Bridge.NET only supports C# 5 so I had to make a couple of changes for it to compile.</p>
<p>Expression-bodied methods had to become normal methods:</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// C# 6</span>
<span class="k">void</span> <span class="nf">AddXToI</span><span class="p">(</span><span class="n">OpCodeData</span> <span class="n">data</span><span class="p">)</span> <span class="p">=></span> <span class="n">I</span> <span class="p">+=</span> <span class="n">V</span><span class="p">[</span><span class="n">data</span><span class="p">.</span><span class="n">X</span><span class="p">];</span>
<span class="c1">// C# 5</span>
<span class="k">void</span> <span class="nf">AddXToI</span><span class="p">(</span><span class="n">OpCodeData</span> <span class="n">data</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">I</span> <span class="p">+=</span> <span class="n">V</span><span class="p">[</span><span class="n">data</span><span class="p">.</span><span class="n">X</span><span class="p">];</span>
<span class="p">}</span>
</code></pre></div></div>
<p>String interpolation had to be replaced with <code class="language-plaintext highlighter-rouge">string.Format</code> calls:</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// C# 6</span>
<span class="n">Debug</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">$"Drawing </span><span class="p">{</span><span class="n">data</span><span class="p">.</span><span class="n">N</span><span class="p">}</span><span class="s">-line sprite from </span><span class="p">{</span><span class="n">I</span><span class="p">}</span><span class="s"> at </span><span class="p">{</span><span class="n">startX</span><span class="p">}</span><span class="s">, </span><span class="p">{</span><span class="n">startY</span><span class="p">}</span><span class="s">"</span><span class="p">));</span>
<span class="c1">// C# 5</span>
<span class="n">Debug</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="kt">string</span><span class="p">.</span><span class="nf">Format</span><span class="p">(</span><span class="s">"Drawing {0}-line sprite from {1} at {2}, {3}"</span><span class="p">,</span> <span class="n">data</span><span class="p">.</span><span class="n">N</span><span class="p">,</span> <span class="n">I</span><span class="p">,</span> <span class="n">startX</span><span class="p">,</span> <span class="n">startY</span><span class="p">));</span>
</code></pre></div></div>
<h3 id="key-and-touch-mappings">Key and Touch Mappings</h3>
<p>I had to tweak the key mappings (which previously used the .NET Frameworkās <code class="language-plaintext highlighter-rouge">Keys</code> enum) to <a href="https://github.com/DanTup/DaChip8JS/blob/master/DaChip8JS/KeyCode.cs">my own KeyCode enum</a> which contains the keys required (I added cursor keys here to make the demo ROM better which was missing from the desktop version).</p>
<h3 id="game-loop-changes">Game Loop Changes</h3>
<p>In the C# version my game loop hooked the <code class="language-plaintext highlighter-rouge">Idle</code> event of the WinForm and used a timer to tell when it was time to call <code class="language-plaintext highlighter-rouge">Tick</code> and <code class="language-plaintext highlighter-rouge">Tick60Hz</code>. The Bridge version turned out to be much simpler using <code class="language-plaintext highlighter-rouge">setInterval</code>:</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">static</span> <span class="k">void</span> <span class="nf">StartGameLoop</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">Window</span><span class="p">.</span><span class="nf">SetInterval</span><span class="p">(</span><span class="n">chip8</span><span class="p">.</span><span class="n">Tick</span><span class="p">,</span> <span class="n">targetElapsedTime</span><span class="p">);</span>
<span class="n">Window</span><span class="p">.</span><span class="nf">SetInterval</span><span class="p">(</span><span class="n">chip8</span><span class="p">.</span><span class="n">Tick60Hz</span><span class="p">,</span> <span class="n">targetElapsedTime60Hz</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Actually, it turned out to be slightly more complicated than that. <code class="language-plaintext highlighter-rouge">setInterval</code> has a minimum resolution of 4ms, so I had to tweak this slightly when I realised the game loop wasnāt running as fast as expected (which meant a slower game - Chip-8 games donāt account for their run speed).</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// HTML5 has minimum resolution of 4ms </span>
<span class="c1">// http://developer.mozilla.org/en/DOM/window.setTimeout#Minimum_delay_and_timeout_nesting</span>
<span class="k">static</span> <span class="k">readonly</span> <span class="kt">int</span> <span class="n">minimumSetIntervalResolution</span> <span class="p">=</span> <span class="m">4</span><span class="p">;</span>
<span class="k">static</span> <span class="k">void</span> <span class="nf">StartGameLoop</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">Window</span><span class="p">.</span><span class="nf">SetInterval</span><span class="p">(</span><span class="n">Tick</span><span class="p">,</span> <span class="n">minimumSetIntervalResolution</span><span class="p">);</span>
<span class="n">Window</span><span class="p">.</span><span class="nf">SetInterval</span><span class="p">(</span><span class="n">chip8</span><span class="p">.</span><span class="n">Tick60Hz</span><span class="p">,</span> <span class="n">targetElapsedTime60Hz</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">static</span> <span class="k">void</span> <span class="nf">Tick</span><span class="p">()</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">numTicksToExecute</span> <span class="p">=</span> <span class="n">minimumSetIntervalResolution</span> <span class="p">/</span> <span class="n">targetElapsedTime</span><span class="p">;</span>
<span class="k">for</span> <span class="p">(</span><span class="kt">var</span> <span class="n">i</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span> <span class="n">i</span> <span class="p"><</span> <span class="n">numTicksToExecute</span><span class="p">;</span> <span class="n">i</span><span class="p">++)</span>
<span class="n">chip8</span><span class="p">.</span><span class="nf">Tick</span><span class="p">();</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="rom-loading">ROM Loading</h3>
<p>In the C# version I just used <code class="language-plaintext highlighter-rouge">File.ReadAllBytes</code> to read the ROM from the disk. Obviously I canāt do that in the browser so I used <code class="language-plaintext highlighter-rouge">XmlHttpRequest</code> to fetch the ROM. I had to refactor the code a little to allow starting the game when that completed (the C# version loaded synchronously).</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span><span class="n">Ready</span><span class="p">]</span>
<span class="k">public</span> <span class="k">static</span> <span class="k">void</span> <span class="nf">OnReady</span><span class="p">()</span>
<span class="p">{</span>
<span class="c1">// ...</span>
<span class="c1">// Kick off async loading of ROM.</span>
<span class="nf">BeginLoadRom</span><span class="p">(</span><span class="n">ROM</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">static</span> <span class="k">void</span> <span class="nf">BeginLoadRom</span><span class="p">(</span><span class="kt">string</span> <span class="n">rom</span><span class="p">)</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">req</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">XMLHttpRequest</span><span class="p">();</span>
<span class="n">req</span><span class="p">.</span><span class="n">ResponseType</span> <span class="p">=</span> <span class="n">XMLHttpRequestResponseType</span><span class="p">.</span><span class="n">ArrayBuffer</span><span class="p">;</span>
<span class="n">req</span><span class="p">.</span><span class="nf">Open</span><span class="p">(</span><span class="s">"GET"</span><span class="p">,</span> <span class="n">rom</span><span class="p">);</span>
<span class="n">req</span><span class="p">.</span><span class="n">OnLoad</span> <span class="p">=</span> <span class="n">e</span> <span class="p">=></span> <span class="nf">EndLoadRom</span><span class="p">(</span><span class="nf">GetResponseAsByteArray</span><span class="p">(</span><span class="n">req</span><span class="p">));</span>
<span class="n">req</span><span class="p">.</span><span class="nf">Send</span><span class="p">();</span>
<span class="p">}</span>
<span class="c1">// Convert the response to a byte[] as that's what our existing C# expects.</span>
<span class="k">static</span> <span class="kt">byte</span><span class="p">[]</span> <span class="nf">GetResponseAsByteArray</span><span class="p">(</span><span class="n">XMLHttpRequest</span> <span class="n">req</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="k">new</span> <span class="nf">Uint8Array</span><span class="p">(</span><span class="n">req</span><span class="p">.</span><span class="n">Response</span> <span class="k">as</span> <span class="n">ArrayBuffer</span><span class="p">).</span><span class="n">As</span><span class="p"><</span><span class="kt">byte</span><span class="p">[</span><span class="k">]></span><span class="p">();</span>
<span class="p">}</span>
<span class="c1">// After loading the ROM, only then is it valid to begin the game loop.</span>
<span class="k">static</span> <span class="k">void</span> <span class="nf">EndLoadRom</span><span class="p">(</span><span class="kt">byte</span><span class="p">[]</span> <span class="n">data</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">chip8</span><span class="p">.</span><span class="nf">LoadProgram</span><span class="p">(</span><span class="n">data</span><span class="p">);</span>
<span class="nf">StartGameLoop</span><span class="p">();</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="rendering-changes">Rendering Changes</h3>
<p>The constructor of my Chip8 class takes an <code class="language-plaintext highlighter-rouge">Action<bool[,]></code> which gets called whenever the screen needs to be rendered. This made it easy to change the rendering code without making changes to Chip8. I used HTML5 canvas to render to:</p>
<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt"><canvas</span> <span class="na">id=</span><span class="s">"screen"</span> <span class="na">width=</span><span class="s">"64"</span> <span class="na">height=</span><span class="s">"32"</span> <span class="na">style=</span><span class="s">"min-width: 640px; min-height: 320px;"</span><span class="nt">></canvas></span>
</code></pre></div></div>
<p>Note the <code class="language-plaintext highlighter-rouge">width</code>/<code class="language-plaintext highlighter-rouge">height</code> on a canvas relate to the coordinates being used for drawing and not the actual size in the document. These settings make the canvas ten times bigger than the resolution of the Chip-8.</p>
<p>I did a bit of Googling to see whatās the fastest way to render pixels to a canvas and found a solution that involves creating <code class="language-plaintext highlighter-rouge">ImageData</code> for the pixel then rendering that at each location:</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// Set up canvas rendering.</span>
<span class="n">screen</span> <span class="p">=</span> <span class="n">Document</span><span class="p">.</span><span class="n">GetElementById</span><span class="p"><</span><span class="n">HTMLCanvasElement</span><span class="p">>(</span><span class="s">"screen"</span><span class="p">);</span>
<span class="n">screenContext</span> <span class="p">=</span> <span class="n">screen</span><span class="p">.</span><span class="nf">GetContext</span><span class="p">(</span><span class="n">CanvasTypes</span><span class="p">.</span><span class="n">CanvasContext2DType</span><span class="p">.</span><span class="n">CanvasRenderingContext2D</span><span class="p">);</span>
<span class="n">lightPixel</span> <span class="p">=</span> <span class="n">screenContext</span><span class="p">.</span><span class="nf">CreateImageData</span><span class="p">(</span><span class="m">1</span><span class="p">,</span> <span class="m">1</span><span class="p">);</span>
<span class="n">lightPixel</span><span class="p">.</span><span class="n">Data</span><span class="p">[</span><span class="m">1</span><span class="p">]</span> <span class="p">=</span> <span class="m">0x64</span><span class="p">;</span> <span class="c1">// Set green part (#006400)</span>
<span class="n">lightPixel</span><span class="p">.</span><span class="n">Data</span><span class="p">[</span><span class="m">3</span><span class="p">]</span> <span class="p">=</span> <span class="m">255</span><span class="p">;</span> <span class="c1">// Alpha</span>
<span class="k">static</span> <span class="k">void</span> <span class="nf">Draw</span><span class="p">(</span><span class="kt">bool</span><span class="p">[,]</span> <span class="n">buffer</span><span class="p">)</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">width</span> <span class="p">=</span> <span class="n">buffer</span><span class="p">.</span><span class="nf">GetLength</span><span class="p">(</span><span class="m">0</span><span class="p">);</span>
<span class="kt">var</span> <span class="n">height</span> <span class="p">=</span> <span class="n">buffer</span><span class="p">.</span><span class="nf">GetLength</span><span class="p">(</span><span class="m">1</span><span class="p">);</span>
<span class="c1">// For performance, we only draw lit pixels so we need to clear the screen first.</span>
<span class="n">screenContext</span><span class="p">.</span><span class="nf">ClearRect</span><span class="p">(</span><span class="m">0</span><span class="p">,</span> <span class="m">0</span><span class="p">,</span> <span class="n">width</span><span class="p">,</span> <span class="n">height</span><span class="p">);</span>
<span class="c1">// Render each lit pixel.</span>
<span class="k">for</span> <span class="p">(</span><span class="kt">var</span> <span class="n">x</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span> <span class="n">x</span> <span class="p"><</span> <span class="n">width</span><span class="p">;</span> <span class="n">x</span><span class="p">++)</span>
<span class="k">for</span> <span class="p">(</span><span class="kt">var</span> <span class="n">y</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span> <span class="n">y</span> <span class="p"><</span> <span class="n">height</span><span class="p">;</span> <span class="n">y</span><span class="p">++)</span>
<span class="k">if</span> <span class="p">(</span><span class="n">buffer</span><span class="p">[</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">])</span>
<span class="n">screenContext</span><span class="p">.</span><span class="nf">PutImageData</span><span class="p">(</span><span class="n">lightPixel</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="sound-changes">Sound Changes</h3>
<p>In the WinForms version I used <code class="language-plaintext highlighter-rouge">Console.Beep</code>. As with the drawing, I pass an action into the Chip8 constructor for this (an <code class="language-plaintext highlighter-rouge">Action<int></code> this time, withthe int being the duration). The simplest way to play a beep in JS seemed to be with an HTML5 Audio element with a data-url for the beep. I didnāt have control of the duration here, but I figured itād work fine for now.</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// Set up audio.</span>
<span class="c1">// http://stackoverflow.com/a/23395136/25124</span>
<span class="n">beep</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">HTMLAudioElement</span><span class="p">(</span><span class="s">"data:audio/wav;base64,//uQRAAAAWM (trim a big long string)"</span><span class="p">);</span>
<span class="k">static</span> <span class="k">void</span> <span class="nf">Beep</span><span class="p">(</span><span class="kt">int</span> <span class="n">milliseconds</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">beep</span><span class="p">.</span><span class="nf">Play</span><span class="p">();</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Unfortunately I came across an issue when testing this on mobile. You canāt play sounds that werenāt initiated by the user! As a workaround, I made the sound play when you tap to start the emulator.</p>
<h3 id="touch-support">Touch Support</h3>
<p>Since a large portion of traffic to this blog comes from mobile (and I thought having an embedded demo would be pretty cool), I figured I should ensure it works there. I added some touch handlers to the canvas that just emulated pressing the cursor keys based on the side of the canvas that was touched:</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// Set up touch events for canvas so we can play on mobile.</span>
<span class="n">screen</span><span class="p">.</span><span class="n">OnTouchStart</span> <span class="p">+=</span> <span class="n">SetTouchDown</span><span class="p">;</span>
<span class="n">screen</span><span class="p">.</span><span class="n">OnTouchEnd</span> <span class="p">+=</span> <span class="n">SetTouchUp</span><span class="p">;</span>
<span class="k">static</span> <span class="k">void</span> <span class="nf">SetTouchDown</span><span class="p">(</span><span class="n">TouchEvent</span><span class="p"><</span><span class="n">HTMLCanvasElement</span><span class="p">></span> <span class="n">e</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="n">e</span><span class="p">.</span><span class="n">Touches</span><span class="p">[</span><span class="m">0</span><span class="p">].</span><span class="n">ClientX</span> <span class="p"><</span> <span class="m">320</span><span class="p">)</span>
<span class="n">chip8</span><span class="p">.</span><span class="nf">KeyDown</span><span class="p">(</span><span class="n">keyMapping</span><span class="p">[</span><span class="n">KeyCode</span><span class="p">.</span><span class="n">LeftCursor</span><span class="p">]);</span>
<span class="k">else</span>
<span class="n">chip8</span><span class="p">.</span><span class="nf">KeyDown</span><span class="p">(</span><span class="n">keyMapping</span><span class="p">[</span><span class="n">KeyCode</span><span class="p">.</span><span class="n">RightCursor</span><span class="p">]);</span>
<span class="n">e</span><span class="p">.</span><span class="nf">PreventDefault</span><span class="p">();</span>
<span class="p">}</span>
<span class="k">static</span> <span class="k">void</span> <span class="nf">SetTouchUp</span><span class="p">(</span><span class="n">TouchEvent</span><span class="p"><</span><span class="n">HTMLCanvasElement</span><span class="p">></span> <span class="n">e</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">chip8</span><span class="p">.</span><span class="nf">KeyUp</span><span class="p">(</span><span class="n">keyMapping</span><span class="p">[</span><span class="n">KeyCode</span><span class="p">.</span><span class="n">LeftCursor</span><span class="p">]);</span>
<span class="n">chip8</span><span class="p">.</span><span class="nf">KeyUp</span><span class="p">(</span><span class="n">keyMapping</span><span class="p">[</span><span class="n">KeyCode</span><span class="p">.</span><span class="n">RightCursor</span><span class="p">]);</span>
<span class="n">e</span><span class="p">.</span><span class="nf">PreventDefault</span><span class="p">();</span>
<span class="p">}</span>
</code></pre></div></div>
<p>And that was pretty much it!</p>
<p>All source code for this project can be found <a href="https://github.com/DanTup/DaChip8JS">on GitHub</a>.</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2016/06/dachip8js-my-csharp-chip8-interpreter-running-in-the-browser/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2016-06-11:/2016/06/building-a-chip-8-interpreter-in-csharp/2016-06-11T00:00:00+00:002016-06-11T00:00:00+00:00Building a Chip-8 Interpreter in C#
<p><a href="https://github.com/DanTup/DaChip8">
<img style="position: absolute; top: 0; right: 0; border: 0;" src="https://camo.githubusercontent.com/a6677b08c955af8400f44c6298f40e7d19cc5b2d/68747470733a2f2f73332e616d617a6f6e6177732e636f6d2f6769746875622f726962626f6e732f666f726b6d655f72696768745f677261795f3664366436642e706e67" alt="Fork me on GitHub" data-canonical-src="https://s3.amazonaws.com/github/ribbons/forkme_right_gray_6d6d6d.png" />
</a></p>
<blockquote>
<p><strong>Edit</strong>: Using Bridge.NET, this interpreter can now run in your browser! <a href="/2016/06/dachip8js-my-csharp-chip8-interpreter-running-in-the-browser/">Click here to see a live playable demo!</a>.</p>
</blockquote>
<p>A few weeks ago after watching a few episodes of <a href="https://www.youtube.com/playlist?list=PL-sXmdrqqYYcznDg4xwAJWQgNL2gRray2">Ferris Makes Emulators</a> where Jake Taylor is live-streaming building an N64 emulator in Rust I decided it might be a fun project to try and build my own. Since Iāve never built an emulator before (and my knowledge of assembly is fairly weak) I thought the NES would be a good choice and although I want to learn some new languages I thought Iād start with C# and once I understand how to actually make an emulator I can build in something like Rust.</p>
<p>The current code for <a href="https://github.com/DanTup/DaNES">DaNES is on GitHub</a> but to cut a long story short, Iām stuck on the rendering. While taking a little break from it I thought it might be wiser to try something simpler like Chip-8. Maybe coming back to the NES with a fresher pair of eyes will make things easier!</p>
<p>So, the new project is DaChip8 (<a href="https://github.com/DanTup/DaChip8">also on GitHub</a>). Itās working and seems to run all the ROMs Iāve tested fine :) Hereās some on how it works for any other newbies that want to give it a shot. Some of the code is a bit messy as I was ālearning on the jobā but the main implementation of the chip is pretty clean (I think).</p>
<p>Hereās a video of it in action:</p>
<iframe width="450" height="253" src="https://www.youtube.com/embed/O4Jti7J3moY?VQ=HD720" frameborder="0" allowfullscreen=""></iframe>
<p>Chip-8 isnāt a physical device like a NES so what weāre building is really an interpreter or VM. Because of this, multiple hardware platforms can(/did) support Chip-8 and games/apps donāt need to be re-written for different hardware. Unfortunately there doesnāt seem to be a defined clock speed for Chip 8 and games were not written to adapt so games will run faster on faster chips. Weāll have to artificially run ours slower because hardware today is so much faster (such as by controlling the rate at which we call <code class="language-plaintext highlighter-rouge">Tick</code>).</p>
<p>I implemented my chip as a class in C#. All registers are contained within it and it exposes two methods, <code class="language-plaintext highlighter-rouge">Tick</code> and <code class="language-plaintext highlighter-rouge">Tick60Hz</code> which are controlled externally. <code class="language-plaintext highlighter-rouge">Tick</code> is intended to be called at the clock speed (which Iām considering somewhere between 500Hz-1Mhz) and <code class="language-plaintext highlighter-rouge">Tick60Hz</code> needs to be called 60 times per second (this is because there are two registers for Delay and Sound which must decrement at this rate).</p>
<p>Hereās a list of the fields in my <a href="https://github.com/DanTup/DaChip8/blob/master/DaChip8/Chip8.cs">Chip8 class</a> and what they are:</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// Constants for screen size</span>
<span class="k">const</span> <span class="kt">int</span> <span class="n">ScreenWidth</span> <span class="p">=</span> <span class="m">64</span><span class="p">,</span> <span class="n">ScreenHeight</span> <span class="p">=</span> <span class="m">32</span><span class="p">;</span>
<span class="c1">// A buffer containing a bool for each pixel on the screen (Chip-8 was monochrome)</span>
<span class="kt">bool</span><span class="p">[,]</span> <span class="n">buffer</span> <span class="p">=</span> <span class="k">new</span> <span class="kt">bool</span><span class="p">[</span><span class="n">ScreenWidth</span><span class="p">,</span> <span class="n">ScreenHeight</span><span class="p">];</span>
<span class="c1">// Actions that the chip should invoke (draw will be called from inside Tick60Hz, beep whenever we need to beep).</span>
<span class="c1">// These are passed in on the constuctor since they'll generally by implemented in some platform-specific way.</span>
<span class="n">Action</span><span class="p"><</span><span class="kt">bool</span><span class="p">[,</span><span class="k">]></span> <span class="n">draw</span><span class="p">;</span>
<span class="n">Action</span><span class="p"><</span><span class="kt">int</span><span class="p">></span> <span class="n">beep</span><span class="p">;</span>
<span class="c1">// Chip-8 has 16x 8-bit registers (referred to as V) which we'll store as byte[16].</span>
<span class="c1">// As with most things in this file, we'll use hex to refer to things, so these </span>
<span class="c1">// are V[0x0] to V[0xF]. The last register, V[F] is also used for special purposes</span>
<span class="c1">// such as a carry and collision flag.</span>
<span class="kt">byte</span><span class="p">[]</span> <span class="n">V</span> <span class="p">=</span> <span class="k">new</span> <span class="kt">byte</span><span class="p">[</span><span class="m">16</span><span class="p">];</span>
<span class="c1">// There are supposed to be two timers (Sound, Delay) but the easiest way for me to implement a beep</span>
<span class="c1">// was with Console.Beep(frequency, duration) so rather than have a Sound timer that counts down</span>
<span class="c1">// I just immediately start playing for the given duration. This might not be perfect (you can't</span>
<span class="c1">// cancel an in-progress sound) but I suspect in reality it works for all games.</span>
<span class="kt">byte</span> <span class="n">Delay</span><span class="p">;</span>
<span class="c1">// The address register (referred to as I) is 16-bit and is used store memory addresses such as</span>
<span class="c1">// when rendering sprites.</span>
<span class="kt">ushort</span> <span class="n">I</span><span class="p">;</span>
<span class="c1">// The program counter (where we're currently executing instructions from) is also 16-bit and</span>
<span class="c1">// This starts at the location 0x200 (512) because the first 512 bytes were used by the interpreter.</span>
<span class="c1">// We use 0x0 to 0x200 to store things like the built-in font data.</span>
<span class="kt">ushort</span> <span class="n">PC</span> <span class="p">=</span> <span class="m">0x200</span><span class="p">;</span>
<span class="c1">// The Chip-8 stack is only used to store program counter locations when jumping to subroutines.</span>
<span class="c1">// We use an array that can store 16 addresses and a single byte to index into it. Whenever jumping to</span>
<span class="c1">// a subroutine we will push the program counter (PC) to the stack (Stack[SP]) and increment SP</span>
<span class="c1">// (the stack pointer). When we need to return, we will decrement SP and then read the address to return</span>
<span class="c1">// to from Stack[SP].</span>
<span class="kt">byte</span> <span class="n">SP</span><span class="p">;</span>
<span class="kt">ushort</span><span class="p">[]</span> <span class="n">Stack</span> <span class="p">=</span> <span class="k">new</span> <span class="kt">ushort</span><span class="p">[</span><span class="m">16</span><span class="p">];</span>
<span class="c1">// The original Chip-8 systems had 4K (0x1000) of addressable memory. </span>
<span class="kt">byte</span><span class="p">[]</span> <span class="n">RAM</span> <span class="p">=</span> <span class="k">new</span> <span class="kt">byte</span><span class="p">[</span><span class="m">0x1000</span><span class="p">];</span>
<span class="c1">// To enable fast lookups of functions to execute for each opcode, we use a dictionary.</span>
<span class="c1">// Normally we use the first nibble (4 bits) of the two-byte instruction to decide what to do,</span>
<span class="c1">// however there's a bunch of random instructions all lumped into the 0xF000 nibble so we have</span>
<span class="c1">// an additional dictionary to look them up based on the second byte.</span>
<span class="n">Dictionary</span><span class="p"><</span><span class="kt">byte</span><span class="p">,</span> <span class="n">Action</span><span class="p"><</span><span class="n">OpCodeData</span><span class="p">>></span> <span class="n">opCodes</span><span class="p">;</span>
<span class="n">Dictionary</span><span class="p"><</span><span class="kt">byte</span><span class="p">,</span> <span class="n">Action</span><span class="p"><</span><span class="n">OpCodeData</span><span class="p">>></span> <span class="n">opCodesMisc</span><span class="p">;</span>
<span class="c1">// One of the instructions is to generate a random number, so we'll need this.</span>
<span class="n">Random</span> <span class="n">rnd</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">Random</span><span class="p">();</span>
<span class="c1">// We also need to keep track of which keys are currently held down. Chip-8 had a 16-key keyboard</span>
<span class="c1">// much like a numeric pad with A-F around the edges:</span>
<span class="c1">// 1 2 3 C</span>
<span class="c1">// 4 5 6 D</span>
<span class="c1">// 7 8 9 E</span>
<span class="c1">// A 0 B F</span>
<span class="c1">// We store these in a HashSet to allow easy checking of whether a key is down.</span>
<span class="n">HashSet</span><span class="p"><</span><span class="kt">byte</span><span class="p">></span> <span class="n">pressedKeys</span> <span class="p">=</span> <span class="k">new</span> <span class="n">HashSet</span><span class="p"><</span><span class="kt">byte</span><span class="p">>();</span>
</code></pre></div></div>
<p>For building and testing, I used ROMs from the <a href="http://www.chip8.com/?page=109">Chip-8 Pack</a>. As far as I can tell these are all games written by hobbyists and freely distributable/usable.</p>
<p>For loading a ROM, we simply read the bytes from the file and write them into RAM starting at <code class="language-plaintext highlighter-rouge">0x200</code> (as explained above):</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">void</span> <span class="nf">LoadProgram</span><span class="p">(</span><span class="kt">byte</span><span class="p">[]</span> <span class="n">data</span><span class="p">)</span> <span class="p">=></span>
<span class="n">Array</span><span class="p">.</span><span class="nf">Copy</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="m">0</span><span class="p">,</span> <span class="n">RAM</span><span class="p">,</span> <span class="m">0x200</span><span class="p">,</span> <span class="n">data</span><span class="p">.</span><span class="n">Length</span><span class="p">);</span>
</code></pre></div></div>
<p>The way opcodes are interpreted is generally based on the first nibble (4 bits). Thereās a <a href="https://en.wikipedia.org/wiki/CHIP-8#Opcode_table">great table on Wikipedia</a> which I wonāt duplicate here, but here are a few examples:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>1NNN Jumps to address NNN.
5XY0 Skips the next instruction if VX equals VY.
</code></pre></div></div>
<p>Where you see <code class="language-plaintext highlighter-rouge">N</code>, <code class="language-plaintext highlighter-rouge">X</code> or <code class="language-plaintext highlighter-rouge">Y</code> in these instructions, those are placeholders. Other values (from <code class="language-plaintext highlighter-rouge">0</code> to <code class="language-plaintext highlighter-rouge">F</code>) are the bits being matched. The placeholder values also fall into the same pattern, so we can easily assign names to each part of the instruction:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>X = second nibble
Y = third nibble
N = final (fourth) nibble
NN = second byte
NNN = last three nibbles
</code></pre></div></div>
<p>To make the opcode methods have the same signature, I wrapped this data up in a struct. Now every opcode instruction function just takes an <code class="language-plaintext highlighter-rouge">OpCodeData</code> which looks like this:</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">struct</span> <span class="nc">OpCodeData</span>
<span class="p">{</span>
<span class="k">public</span> <span class="kt">ushort</span> <span class="n">OpCode</span><span class="p">;</span>
<span class="k">public</span> <span class="kt">ushort</span> <span class="n">NNN</span><span class="p">;</span>
<span class="k">public</span> <span class="kt">byte</span> <span class="n">NN</span><span class="p">,</span> <span class="n">X</span><span class="p">,</span> <span class="n">Y</span><span class="p">,</span> <span class="n">N</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Now we know how opcodes are laid out, we can build our dictionary based on the first nibble (and in the case of the misc instructions, the second byte):</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">opCodes</span> <span class="p">=</span> <span class="k">new</span> <span class="n">Dictionary</span><span class="p"><</span><span class="kt">byte</span><span class="p">,</span> <span class="n">Action</span><span class="p"><</span><span class="n">OpCodeData</span><span class="p">>></span>
<span class="p">{</span>
<span class="p">{</span> <span class="m">0x0</span><span class="p">,</span> <span class="n">ClearOrReturn</span> <span class="p">},</span> <span class="c1">// There are two instructions starting with a 0 nibble, didn't seem worth its own dictionary</span>
<span class="p">{</span> <span class="m">0x1</span><span class="p">,</span> <span class="n">Jump</span> <span class="p">},</span>
<span class="p">{</span> <span class="m">0x2</span><span class="p">,</span> <span class="n">CallSubroutine</span> <span class="p">},</span>
<span class="p">{</span> <span class="m">0x3</span><span class="p">,</span> <span class="n">SkipIfXEqual</span> <span class="p">},</span>
<span class="p">{</span> <span class="m">0x4</span><span class="p">,</span> <span class="n">SkipIfXNotEqual</span> <span class="p">},</span>
<span class="p">{</span> <span class="m">0x5</span><span class="p">,</span> <span class="n">SkipIfXEqualY</span> <span class="p">},</span>
<span class="p">{</span> <span class="m">0x6</span><span class="p">,</span> <span class="n">SetX</span> <span class="p">},</span>
<span class="p">{</span> <span class="m">0x7</span><span class="p">,</span> <span class="n">AddX</span> <span class="p">},</span>
<span class="p">{</span> <span class="m">0x8</span><span class="p">,</span> <span class="n">Arithmetic</span> <span class="p">},</span>
<span class="p">{</span> <span class="m">0x9</span><span class="p">,</span> <span class="n">SkipIfXNotEqualY</span> <span class="p">},</span>
<span class="p">{</span> <span class="m">0xA</span><span class="p">,</span> <span class="n">SetI</span> <span class="p">},</span>
<span class="p">{</span> <span class="m">0xB</span><span class="p">,</span> <span class="n">JumpWithOffset</span> <span class="p">},</span>
<span class="p">{</span> <span class="m">0xC</span><span class="p">,</span> <span class="n">Rnd</span> <span class="p">},</span>
<span class="p">{</span> <span class="m">0xD</span><span class="p">,</span> <span class="n">DrawSprite</span> <span class="p">},</span>
<span class="p">{</span> <span class="m">0xE</span><span class="p">,</span> <span class="n">SkipOnKey</span> <span class="p">},</span>
<span class="p">{</span> <span class="m">0xF</span><span class="p">,</span> <span class="n">Misc</span> <span class="p">},</span> <span class="c1">// This will do a lookup from the second dictionary</span>
<span class="p">};</span>
<span class="n">opCodesMisc</span> <span class="p">=</span> <span class="k">new</span> <span class="n">Dictionary</span><span class="p"><</span><span class="kt">byte</span><span class="p">,</span> <span class="n">Action</span><span class="p"><</span><span class="n">OpCodeData</span><span class="p">>></span>
<span class="p">{</span>
<span class="p">{</span> <span class="m">0x07</span><span class="p">,</span> <span class="n">SetXToDelay</span> <span class="p">},</span>
<span class="p">{</span> <span class="m">0x0A</span><span class="p">,</span> <span class="n">WaitForKey</span> <span class="p">},</span>
<span class="p">{</span> <span class="m">0x15</span><span class="p">,</span> <span class="n">SetDelay</span> <span class="p">},</span>
<span class="p">{</span> <span class="m">0x18</span><span class="p">,</span> <span class="n">SetSound</span> <span class="p">},</span>
<span class="p">{</span> <span class="m">0x1E</span><span class="p">,</span> <span class="n">AddXToI</span> <span class="p">},</span>
<span class="p">{</span> <span class="m">0x29</span><span class="p">,</span> <span class="n">SetIForChar</span> <span class="p">},</span>
<span class="p">{</span> <span class="m">0x33</span><span class="p">,</span> <span class="n">BinaryCodedDecimal</span> <span class="p">},</span>
<span class="p">{</span> <span class="m">0x55</span><span class="p">,</span> <span class="n">SaveX</span> <span class="p">},</span>
<span class="p">{</span> <span class="m">0x65</span><span class="p">,</span> <span class="n">LoadX</span> <span class="p">},</span>
<span class="p">};</span>
</code></pre></div></div>
<p>Next we need to implement our <code class="language-plaintext highlighter-rouge">Tick</code> function which will read a two-byte opcode from RAM at the location the Program Counter (<code class="language-plaintext highlighter-rouge">PC</code>) points to then look up the instruction using the first nibble. When implementing this I found there were some instructions in the misc section that we donāt care about so I skipped over them (rather than crashing).</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">void</span> <span class="nf">Tick</span><span class="p">()</span>
<span class="p">{</span>
<span class="c1">// Read the two bytes of OpCode. This is big-endian which means</span>
<span class="c1">// the most significant byte comes first. We shift it 8 bytes to</span>
<span class="c1">// the left then bitwise-or it with the next byte to get the full</span>
<span class="c1">// 16-bit value.</span>
<span class="kt">var</span> <span class="n">opCode</span> <span class="p">=</span> <span class="p">(</span><span class="kt">ushort</span><span class="p">)(</span><span class="n">RAM</span><span class="p">[</span><span class="n">PC</span><span class="p">++]</span> <span class="p"><<</span> <span class="m">8</span> <span class="p">|</span> <span class="n">RAM</span><span class="p">[</span><span class="n">PC</span><span class="p">++]);</span>
<span class="c1">// Write the PC and the OpCode we read for debugging.</span>
<span class="n">Debug</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">((</span><span class="n">PC</span> <span class="p">-</span> <span class="m">2</span><span class="p">).</span><span class="nf">ToString</span><span class="p">(</span><span class="s">"X4"</span><span class="p">)</span> <span class="p">+</span> <span class="s">": "</span> <span class="p">+</span> <span class="n">opCode</span><span class="p">.</span><span class="nf">ToString</span><span class="p">(</span><span class="s">"X4"</span><span class="p">));</span>
<span class="c1">// Split data into the possible formats the instruction might need.</span>
<span class="c1">// https://en.wikipedia.org/wiki/CHIP-8#Opcode_table</span>
<span class="kt">var</span> <span class="n">op</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">OpCodeData</span><span class="p">()</span>
<span class="p">{</span>
<span class="c1">// We use bitwise-and with a mask to extract specific nibbles.</span>
<span class="n">OpCode</span> <span class="p">=</span> <span class="n">opCode</span><span class="p">,</span>
<span class="n">NNN</span> <span class="p">=</span> <span class="p">(</span><span class="kt">ushort</span><span class="p">)(</span><span class="n">opCode</span> <span class="p">&</span> <span class="m">0x0FFF</span><span class="p">),</span>
<span class="n">NN</span> <span class="p">=</span> <span class="p">(</span><span class="kt">byte</span><span class="p">)(</span><span class="n">opCode</span> <span class="p">&</span> <span class="m">0x00FF</span><span class="p">),</span>
<span class="n">N</span> <span class="p">=</span> <span class="p">(</span><span class="kt">byte</span><span class="p">)(</span><span class="n">opCode</span> <span class="p">&</span> <span class="m">0x000F</span><span class="p">),</span>
<span class="n">X</span> <span class="p">=</span> <span class="p">(</span><span class="kt">byte</span><span class="p">)((</span><span class="n">opCode</span> <span class="p">&</span> <span class="m">0x0F00</span><span class="p">)</span> <span class="p">>></span> <span class="m">8</span><span class="p">),</span> <span class="c1">// Where don't use the lower nibbles, bitshift right to get just the raw value</span>
<span class="n">Y</span> <span class="p">=</span> <span class="p">(</span><span class="kt">byte</span><span class="p">)((</span><span class="n">opCode</span> <span class="p">&</span> <span class="m">0x00F0</span><span class="p">)</span> <span class="p">>></span> <span class="m">4</span><span class="p">),</span> <span class="c1">// Eg. we want 0x4 not 0x40</span>
<span class="p">};</span>
<span class="c1">// Loop up the OpCode using the first nibble and execute.</span>
<span class="n">opCodes</span><span class="p">[(</span><span class="kt">byte</span><span class="p">)(</span><span class="n">opCode</span> <span class="p">>></span> <span class="m">12</span><span class="p">)](</span><span class="n">op</span><span class="p">);</span>
<span class="p">}</span>
<span class="c1">// Misc has its own dictionary because it's full of random stuff.</span>
<span class="k">void</span> <span class="nf">Misc</span><span class="p">(</span><span class="n">OpCodeData</span> <span class="n">data</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="n">opCodesMisc</span><span class="p">.</span><span class="nf">ContainsKey</span><span class="p">(</span><span class="n">data</span><span class="p">.</span><span class="n">NN</span><span class="p">))</span>
<span class="n">opCodesMisc</span><span class="p">[</span><span class="n">data</span><span class="p">.</span><span class="n">NN</span><span class="p">](</span><span class="n">data</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Ok, so now we have a function we can tick at 500Hz and itāll decode an OpCode and call the correct function. Next we need to implement all of those functions!</p>
<p>I wonāt go through all of them here on the blog but Iāll cover the first five from my code. The rest you can find <a href="https://github.com/DanTup/DaChip8/blob/master/DaChip8/Chip8.cs#L168">on GitHub</a>.</p>
<p>The first instruction (where the first nibble is <code class="language-plaintext highlighter-rouge">0x0</code>) is actually two completely different instructions, one to clear the screen and one to return from a subroutine. Thereās no point having another dictionary for these two so we just use a simple if.</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">/// <summary></span>
<span class="c1">/// Handles 0x0... which either clears the screen or returns from a subroutine.</span>
<span class="c1">/// </summary></span>
<span class="k">void</span> <span class="nf">ClearOrReturn</span><span class="p">(</span><span class="n">OpCodeData</span> <span class="n">data</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="n">data</span><span class="p">.</span><span class="n">NN</span> <span class="p">==</span> <span class="m">0xE0</span><span class="p">)</span>
<span class="p">{</span>
<span class="c1">// Our screen buffer is just an array of bools, so set them all to false when</span>
<span class="c1">// instructed to clear the screen.</span>
<span class="k">for</span> <span class="p">(</span><span class="kt">var</span> <span class="n">x</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span> <span class="n">x</span> <span class="p"><</span> <span class="n">ScreenWidth</span><span class="p">;</span> <span class="n">x</span><span class="p">++)</span>
<span class="k">for</span> <span class="p">(</span><span class="kt">var</span> <span class="n">y</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span> <span class="n">y</span> <span class="p"><</span> <span class="n">ScreenHeight</span><span class="p">;</span> <span class="n">y</span><span class="p">++)</span>
<span class="n">buffer</span><span class="p">[</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">]</span> <span class="p">=</span> <span class="k">false</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">// Otherwise we're returning from a subroutine. Here we need to pop the address of the stack</span>
<span class="c1">// and set the program counter to it (we'll cover Pop/Push implementations below).</span>
<span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="n">data</span><span class="p">.</span><span class="n">NN</span> <span class="p">==</span> <span class="m">0xEE</span><span class="p">)</span>
<span class="n">PC</span> <span class="p">=</span> <span class="nf">Pop</span><span class="p">();</span>
<span class="p">}</span>
</code></pre></div></div>
<p>There are two simple jump instructions, <code class="language-plaintext highlighter-rouge">0x1000</code> and <code class="language-plaintext highlighter-rouge">0xB000</code>. One jumps directly to the address from the lower three nibbles (<code class="language-plaintext highlighter-rouge">NNN</code>) and the other does the same but adds on the value of the first register (<code class="language-plaintext highlighter-rouge">V[0]</code>).</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">/// <summary></span>
<span class="c1">/// Jumps to location nnn (not a subroutine, so old PC is not pushed to the stack).</span>
<span class="c1">/// </summary></span>
<span class="k">void</span> <span class="nf">Jump</span><span class="p">(</span><span class="n">OpCodeData</span> <span class="n">data</span><span class="p">)</span> <span class="p">=></span>
<span class="n">PC</span> <span class="p">=</span> <span class="n">data</span><span class="p">.</span><span class="n">NNN</span><span class="p">;</span>
<span class="c1">/// <summary></span>
<span class="c1">/// Jumps to location nnn + v[0] (not a subroutine, so old PC is not pushed to the stack).</span>
<span class="c1">/// </summary></span>
<span class="k">void</span> <span class="nf">JumpWithOffset</span><span class="p">(</span><span class="n">OpCodeData</span> <span class="n">data</span><span class="p">)</span> <span class="p">=></span>
<span class="n">PC</span> <span class="p">=</span> <span class="p">(</span><span class="kt">ushort</span><span class="p">)(</span><span class="n">data</span><span class="p">.</span><span class="n">NNN</span> <span class="p">+</span> <span class="n">V</span><span class="p">[</span><span class="m">0</span><span class="p">]);</span>
</code></pre></div></div>
<p>Thereās a third jump instruction which is a āgo to subroutineā. This does the same as above but we need to be able to return back to where we were when it finishes executing (it will end with the return instruction covered above). In order to preserve the return address we push the Program Counter (<code class="language-plaintext highlighter-rouge">PC</code>) onto the stack before jumping.</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">/// <summary></span>
<span class="c1">/// Jumps to subroutine nnn (unlike Jump, this pushes the previous PC to the stack to allow return).</span>
<span class="c1">/// </summary></span>
<span class="k">void</span> <span class="nf">CallSubroutine</span><span class="p">(</span><span class="n">OpCodeData</span> <span class="n">data</span><span class="p">)</span>
<span class="p">{</span>
<span class="nf">Push</span><span class="p">(</span><span class="n">PC</span><span class="p">);</span>
<span class="n">PC</span> <span class="p">=</span> <span class="n">data</span><span class="p">.</span><span class="n">NNN</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The next OpCode from my code is one that skips the next instruction if <code class="language-plaintext highlighter-rouge">V[x]</code> is equal to <code class="language-plaintext highlighter-rouge">nn</code> (both <code class="language-plaintext highlighter-rouge">x</code> and <code class="language-plaintext highlighter-rouge">nn</code> are taken from the <code class="language-plaintext highlighter-rouge">OpCodeData</code> we decoded). This is how the code branches based on conditions (eg. the next instruction might be a go to subroutine).</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">/// <summary></span>
<span class="c1">/// Skips the next instruction (two bytes) if V[x] == nn.</span>
<span class="c1">/// </summary></span>
<span class="k">void</span> <span class="nf">SkipIfXEqual</span><span class="p">(</span><span class="n">OpCodeData</span> <span class="n">data</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="n">V</span><span class="p">[</span><span class="n">data</span><span class="p">.</span><span class="n">X</span><span class="p">]</span> <span class="p">==</span> <span class="n">data</span><span class="p">.</span><span class="n">NN</span><span class="p">)</span>
<span class="n">PC</span> <span class="p">+=</span> <span class="m">2</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Something that was much easier to implement than I expected was the stack. It only holds 16-bit addresses so the implementations of <code class="language-plaintext highlighter-rouge">Pop</code> and <code class="language-plaintext highlighter-rouge">Push</code> are incredibly simple:</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">/// <summary></span>
<span class="c1">/// Pushes a 16-bit value onto the stack, incrementing the SP.</span>
<span class="c1">/// </summary></span>
<span class="k">void</span> <span class="nf">Push</span><span class="p">(</span><span class="kt">ushort</span> <span class="k">value</span><span class="p">)</span> <span class="p">=></span>
<span class="n">Stack</span><span class="p">[</span><span class="n">SP</span><span class="p">++]</span> <span class="p">=</span> <span class="k">value</span><span class="p">;</span>
<span class="c1">/// <summary></span>
<span class="c1">/// Retrieves a 16-bit value from the stack, decrementing the SP.</span>
<span class="c1">/// </summary></span>
<span class="kt">ushort</span> <span class="nf">Pop</span><span class="p">()</span> <span class="p">=></span>
<span class="n">Stack</span><span class="p">[--</span><span class="n">SP</span><span class="p">];</span>
</code></pre></div></div>
<p>Once all instructions are implemented, we need to implement <code class="language-plaintext highlighter-rouge">Tick60Hz</code>. All this does is decrement the <code class="language-plaintext highlighter-rouge">Delay</code> counter and cause a redraw. The draw function is an <code class="language-plaintext highlighter-rouge">Action<byte[,]></code> supplied in the constructor so itās up to the consuming code to get that on the screen.</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">void</span> <span class="nf">Tick60Hz</span><span class="p">()</span>
<span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="n">Delay</span> <span class="p">></span> <span class="m">0</span><span class="p">)</span>
<span class="n">Delay</span><span class="p">--;</span>
<span class="nf">draw</span><span class="p">(</span><span class="n">buffer</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Finally, we need to something to <code class="language-plaintext highlighter-rouge">Tick</code> the chip and draw to the screen. I used a simple WinForms app with a <code class="language-plaintext highlighter-rouge">PictureBox</code> which calls <code class="language-plaintext highlighter-rouge">Tick</code> and <code class="language-plaintext highlighter-rouge">Tick60Hz</code> at the appropriate times and implements the <code class="language-plaintext highlighter-rouge">Draw</code> and <code class="language-plaintext highlighter-rouge">Beep</code> actions that are passed into the chip like this:</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">void</span> <span class="nf">Draw</span><span class="p">(</span><span class="kt">bool</span><span class="p">[,]</span> <span class="n">buffer</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">for</span> <span class="p">(</span><span class="kt">var</span> <span class="n">x</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span> <span class="n">x</span> <span class="p"><</span> <span class="n">screen</span><span class="p">.</span><span class="n">Width</span><span class="p">;</span> <span class="n">x</span><span class="p">++)</span>
<span class="k">for</span> <span class="p">(</span><span class="kt">var</span> <span class="n">y</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span> <span class="n">y</span> <span class="p"><</span> <span class="n">screen</span><span class="p">.</span><span class="n">Height</span><span class="p">;</span> <span class="n">y</span><span class="p">++)</span>
<span class="n">screen</span><span class="p">.</span><span class="nf">SetPixel</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">buffer</span><span class="p">[</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">]</span> <span class="p">?</span> <span class="n">Color</span><span class="p">.</span><span class="n">DarkGreen</span> <span class="p">:</span> <span class="n">Color</span><span class="p">.</span><span class="n">Black</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">void</span> <span class="nf">Beep</span><span class="p">(</span><span class="kt">int</span> <span class="n">milliseconds</span><span class="p">)</span> <span class="p">=></span>
<span class="n">Console</span><span class="p">.</span><span class="nf">Beep</span><span class="p">(</span><span class="m">500</span><span class="p">,</span> <span class="n">milliseconds</span><span class="p">);</span>
</code></pre></div></div>
<p>Thatās pretty much all there is to it! In addition to the other instructions, there are a few things I didnāt cover here (such as the built-in font, which has sprites for the characters from <code class="language-plaintext highlighter-rouge">0x0</code> to <code class="language-plaintext highlighter-rouge">0xF</code>) but you can find <a href="https://github.com/DanTup/DaChip8/">all the code on GitHub</a> (youāll need to download the ROMs as mention in the README to run).</p>
<p>If youāre thinking of building an emulator but unsure where to start, Chip-8 is a great first project. Pretty much everything Iāve done in this one has been the same for the NES emulator I started (with the exception of rendering) so itās all useful practice.</p>
<blockquote>
<p><strong>Edit</strong>: Using Bridge.NET, this interpreter can now run in your browser! <a href="/2016/06/dachip8js-my-csharp-chip8-interpreter-running-in-the-browser/">Click here to see a live playable demo!</a>.</p>
</blockquote>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2016/06/building-a-chip-8-interpreter-in-csharp/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2016-05-29:/2016/05/go-on-a-chromebook-without-linux/2016-05-29T00:00:00+00:002016-05-29T00:00:00+00:00Running Go on a Chromebook in Developer Mode (without installing Linux)
<p>For the last few years the computer I use most at home has been a <a href="https://chromebookcompare.com/dell-chromebook-11/">Dell Chromebook 11</a>. Itās great for browsing the web but not so great at being a developer machine. Although you can install Linux on it if you wish, that comes with an awful lot of baggage. To avoid using my desktop so much (itās hard to lock myself away upstairs for hours with a family) I often find myself tryingto get new things running on the Chromebook that I can play around with from the sofa.</p>
<p><a href="/2014/09/dart-vm-on-a-chromebook-without-linux/">Previously it was the Dart VM</a> and today it is Go. I actually tried Rust and it hasnāt worked out (yet) so Go was a fallback :D</p>
<p>Note: Iāve never written a line of Go in my life until about 30 minutes ago. This post might not be the best way of doing things and the instructions may become broken in future with Go or ChromeOS changes.</p>
<p>Setting up Go actually turned out to be pretty simple. Easier than Dart in fact!</p>
<p>When I did this for Dart, I ended up editing <code class="language-plaintext highlighter-rouge">.bash_profile</code> in <code class="language-plaintext highlighter-rouge">vi</code> (or using <code class="language-plaintext highlighter-rouge">echo</code>) and it was a bit of a mess. Subsequently Iāve installed <a href="https://chrome.google.com/webstore/detail/text/mmfbcljfglbokpmkimbfghdkjmjhdgbg?hl=en">Text</a> and created a symlink at <code class="language-plaintext highlighter-rouge">~/Downloads/bash_profile</code> to <code class="language-plaintext highlighter-rouge">~/.bash_profile</code> so it can be edited in Text (ChromeOS file manager can only see files inside <code class="language-plaintext highlighter-rouge">~/Downloads</code> and not hidden files, hence the missing dot).</p>
<h2 id="enable-developer-mode">Enable Developer Mode</h2>
<p>Instructions on enabling developer mode <a href="https://sites.google.com/site/chromeoswikisite/home/what-s-new-in-dev-and-beta/developer-mode">can be found here</a>. Every time you boot your device in this mode you will get a <a href="https://sites.google.com/site/chromeoswikisite/home/what-s-new-in-dev-and-beta/developer-mode">scary splash screen</a> and have to press <code class="language-plaintext highlighter-rouge">Ctrl+D</code> to avoid resetting your device. <strong>WARNING</strong>: Enabling developer mode will delete all of your locally stored data (though since itās a <a href="https://www.chromebookchart.com/">Chromebook</a>, this is really only your Downloads folder).</p>
<h2 id="download-and-extract-go">Download and Extract Go</h2>
<p>Download the <a href="https://golang.org/dl/">Linux version of Go</a> to your Downloads folder. I picked the 64 bit AMD version for my <a href="https://chromebookcompare.com/dell-chromebook-11/">Dell Chromebook 11</a>; some Chromebooks might require the 32 bit or ARM version.</p>
<p>Next, open a terminal window by pressing <code class="language-plaintext highlighter-rouge">Ctrl+Alt+T</code> then typing <code class="language-plaintext highlighter-rouge">shell</code>. Then extract Go somewhere suitable. I put it at <code class="language-plaintext highlighter-rouge">~/Coding/go/</code> (the archived already has everything inside a folder named <code class="language-plaintext highlighter-rouge">go</code>).</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">mkdir</span> ~/Coding
<span class="nb">tar</span> <span class="nt">-xzf</span> ~/Downloads/go1.6.2.linux-amd64.tar.gz <span class="nt">-C</span> ~/Coding/
</code></pre></div></div>
<h2 id="set-execute-permissions-for-the-file-system">Set Execute Permissions for the File System</h2>
<p>In order to execute anyting from this drive youāll need to remount it as executable. You may have already done this if youāve set other things up (such as the <a href="/2014/09/dart-vm-on-a-chromebook-without-linux/">Dart VM</a>). Without this, youāll get <code class="language-plaintext highlighter-rouge">Permission denied</code> trying to execute anything.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">echo</span> <span class="s2">"sudo mount -i -o remount,exec /home/chronos/user/"</span> <span class="o">>></span> ~/.bash_profile
</code></pre></div></div>
<p>Note: In order for <code class="language-plaintext highlighter-rouge">go run</code> to work you will also need to remount <code class="language-plaintext highlighter-rouge">/tmp</code> as executable. <code class="language-plaintext highlighter-rouge">go run</code> compiles into a temp folder then executes from there. There may be security implications in doing this!</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">echo</span> <span class="s2">"sudo mount -i -o remount,exec /tmp/"</span> <span class="o">>></span> ~/.bash_profile
</code></pre></div></div>
<h2 id="set-environment-variables">Set Environment Variables</h2>
<p>The easiet way to do this is with <a href="https://chrome.google.com/webstore/detail/text/mmfbcljfglbokpmkimbfghdkjmjhdgbg?hl=en">Text</a> and a symlink to <code class="language-plaintext highlighter-rouge">~/.bash_profile</code> as mentioned before. Otherwise you can try using <code class="language-plaintext highlighter-rouge">vi</code> (or <code class="language-plaintext highlighter-rouge">echo</code>)!</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">export </span><span class="nv">GOROOT</span><span class="o">=</span>~/Coding/go <span class="c"># This is where you extracted Go</span>
<span class="nb">export </span><span class="nv">GOPATH</span><span class="o">=</span>~/Downloads/go <span class="c"># This is where you will keep your Go projects</span>
<span class="nv">PATH</span><span class="o">=</span><span class="nv">$PATH</span>:<span class="nv">$GOROOT</span>/bin:<span class="nv">$GOPATH</span>/bin <span class="c"># This adds both Go and your "installed" projects to PATH</span>
</code></pre></div></div>
<h2 id="restart-and-test">Restart and Test</h2>
<p>Type <code class="language-plaintext highlighter-rouge">exit</code> or close the terminal window then re-open with <code class="language-plaintext highlighter-rouge">Ctrl+Alt+T</code> and type <code class="language-plaintext highlighter-rouge">shell</code>. Type <code class="language-plaintext highlighter-rouge">go version</code> and you should see <code class="language-plaintext highlighter-rouge">go version go1.6.2 linux/amd64</code> or similar.</p>
<h2 id="hello-world">Hello, World!</h2>
<p>Now we have a working Go, we should build <a href="https://golang.org/doc/install#testing">Hello World</a>!</p>
<p>There seems to be a convention that you should name your packages after the repo youāll be hosting them in. Theyāll need to hang off <code class="language-plaintext highlighter-rouge">$GOPATH/src</code>, eg:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">mkdir</span> <span class="nv">$GOPATH</span>/src/github.com/<span class="o">{</span>username<span class="o">}</span>/hello <span class="nt">-p</span>
</code></pre></div></div>
<p>Next create a <code class="language-plaintext highlighter-rouge">hello.go</code> file in that directory using your favourite editor with a simple Hello World:</p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">package</span> <span class="n">main</span>
<span class="k">import</span> <span class="s">"fmt"</span>
<span class="k">func</span> <span class="n">main</span><span class="p">()</span> <span class="p">{</span>
<span class="n">fmt</span><span class="o">.</span><span class="n">Printf</span><span class="p">(</span><span class="s">"Hello, World!</span><span class="se">\n</span><span class="s">"</span><span class="p">)</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Now we want to build and execute it. If we use <code class="language-plaintext highlighter-rouge">go install</code> instead of <code class="language-plaintext highlighter-rouge">go build</code> it will copy the executable into <code class="language-plaintext highlighter-rouge">$GOPATH/bin</code> which we added to our <code class="language-plaintext highlighter-rouge">PATH</code> earlier. We can either pass the package path (relative to <code class="language-plaintext highlighter-rouge">$GOPATH</code>) to <code class="language-plaintext highlighter-rouge">go install</code> or change to the directory and omit it:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">cd</span> <span class="nv">$GOPATH</span>/src/github.com/<span class="o">{</span>username<span class="o">}</span>/hello
go <span class="nb">install
</span>hello
</code></pre></div></div>
<p>If you chose to remount <code class="language-plaintext highlighter-rouge">/tmp</code> as executable then you can also use <code class="language-plaintext highlighter-rouge">go run {file}</code>:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">cd</span> <span class="nv">$GOPATH</span>/src/github.com/<span class="o">{</span>username<span class="o">}</span>/hello
go run hello.go
</code></pre></div></div>
<p>If youāve done everything correctly, this will output <code class="language-plaintext highlighter-rouge">Hello, World!</code> :)</p>
<p>If youāre interested in a Chromebook; donāt forget to take a look at <a href="https://chromebookcompare.com/">Chromebook Comparison Chart</a>! :)</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2016/05/go-on-a-chromebook-without-linux/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2016-05-08:/2016/05/easy-javascript-and-css-bundling-on-github-pages-without-build-steps/2016-05-08T00:00:00+00:002016-05-08T00:00:00+00:00Easy JavaScript and CSS bundling and minification on GitHub Pages without build steps
<p>Recently we were tidying up the design a little on <a href="https://chromebookcompare.com/">chromebookcompare.com</a> and trying to make it work better on mobile. While setting up Google Analytics and Google Webmaster Tools I somehow found myself at Google PageSpeed and trying to improve our score there and in <a href="https://developer.chrome.com/devtools#audits">Google Chromeās Audit tab</a>.</p>
<p>Most of the issues resolved around external files. We had quite a few of them (thanks to Google MDL) and the caching and <a href="https://github.com/google/material-design-lite/issues/4319">gzip wasnāt working</a> on some (thereās something ironic about a Google project serving up uncompressed files to Chrome). To fix this I decided we should just download the Google Material Design Lite files locally (they donāt change, theyāre versioned). CloudFlare lets me control the caching (and enables gzip) on these better than we were getting from Google. But, this meant more files being served from our domain and not bundling them together seemed a bit silly.</p>
<p>The site is hosted free using <a href="https://pages.github.com/">GitHub Pages</a> (run through <a href="https://www.cloudflare.com/">CloudFlare</a> to get free HTTPS). We have a small C# app which rips data from the <a href="https://docs.google.com/spreadsheets/d/1JvRtWh8ixWeYoOGu7oZ3fpw53eKfzsTWKp_XlWKofNs/edit">public spreadsheet</a> and spits it out into Jekyll-formatted files for display but otherwis itās entirely static. <a href="https://support.cloudflare.com/hc/en-us/articles/200168196-How-do-I-minify-HTML-CSS-and-JavaScript-to-optimize-my-site-">CloudFlare is already minifying</a> the content (though in this case, this third party code is already pre-minified) so I just wanted the simplest possible way to bundle all the scripts together in one file that didnāt not require us to run any scripts whenever we changed a file (sometimes we tweak things directly through the GitHub web app on Chromebooks where itās more coplicated to run scripts and push to GitHub).</p>
<p>Turned out, I already had the solution on this blog. My Google Analytics script is included using Jekyll includes like this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{% include analytics.htm %}
</code></pre></div></div>
<p>So I thought Iād give it a try with JavaScript. I created a file named <code class="language-plaintext highlighter-rouge">bundle.js</code> inside my <code class="language-plaintext highlighter-rouge">s/js</code> folder and added the following:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{% include material.min.js %}
{% include jquery-1.11.1.min.js %}
{% include jquery.tablesorter.min.js %}
{% include jquery.tablesorter.widgets.js %}
</code></pre></div></div>
<p>I pushed to GitHub and navigated to <code class="language-plaintext highlighter-rouge">/s/js/bundle.js</code> in my browser. It came through completely untransformed. Doh.</p>
<p>I did some digging around to try and understand why this wouldnāt work and discovered (or rather re-remembered) that you need <a href="https://jekyllrb.com/docs/frontmatter/">YAML front-matter</a> for a page to be transformed in Jekyll. So I tried again:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>---
---
{% include material.min.js %}
{% include jquery-1.11.1.min.js %}
{% include jquery.tablesorter.min.js %}
{% include jquery.tablesorter.widgets.js %}
</code></pre></div></div>
<p>After much refreshing, nothing changed. I was about to give up when I noticed an email from GitHub:</p>
<blockquote>
<p>The page build failed with the following error:</p>
<p>A file was included in <code class="language-plaintext highlighter-rouge">s/js/bundle.js</code> that is a symlink or does not exist in your <code class="language-plaintext highlighter-rouge">_includes</code> directory. For more information, see > https://help.github.com/articles/page-build-failed-file-is-a-symlink.</p>
</blockquote>
<p>Whoops! Includes normally live in <code class="language-plaintext highlighter-rouge">_includes</code> so I need to use <code class="language-plaintext highlighter-rouge">include_relative</code>:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>---
---
{% include_relative material.min.js %}
{% include_relative jquery-1.11.1.min.js %}
{% include_relative jquery.tablesorter.min.js %}
{% include_relative jquery.tablesorter.widgets.js %}
</code></pre></div></div>
<p>After pushing this, everything came through as expected. Now all these files are combined into one file and CloudFlare is minifying them. One request to the same domain as the website is much better than four requests that include an additional uncommon domain.</p>
<p>Via CloudFlare, this bundle is set to cache in browsers for one month. This makes the site nice and fast but will also impact the ability to make changes. For this reason our own JavaScript (<code class="language-plaintext highlighter-rouge">/js/main.js</code>) is neither included in this bundle nor in the <code class="language-plaintext highlighter-rouge">/s/</code> folder (anything in <code class="language-plaintext highlighter-rouge">/s/</code> gets the 1-month-cache header).</p>
<p>Thereās still lots of room for improvement (like the CSS) but itās a good start and <a href="https://chromebookcompare.com/">chromebookcompare.com</a> seems to load very quickly already!</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2016/05/easy-javascript-and-css-bundling-on-github-pages-without-build-steps/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2016-04-26:/2016/04/have-software-developers-given-up/2016-04-26T00:00:00+00:002016-04-26T00:00:00+00:00Have Software Developers Given Up?
<style type="text/css">
.post-content img {
margin: 0 10px 10px 30px;
border: solid 1px #ddd;
max-width: 90%;
}
</style>
<p><strong>Edit</strong>: Iāve been finding so many software failures lately, I created <a href="https://twitter.com/SoftwareFailed">@SoftwareFailed</a> on Twitter to tweet and retweet the best failures!</p>
<p>Note: Iām a software developer. I create bugs. I once switched a production SQL database to Simple recovery mode and Truncated an important table causing a ton of work for my colleagues. The content of this post is aimed as much at me and the company I work for as the companies listed here. I think our industry has a real quality problem. I donāt entirely know what the solution is.</p>
<p>Over the last few years it feels like the quality of software and services across the industry is falling rather than climbing. Everything is always beta (both in name and quality). Things are shipped when marketing wants them to rather than when theyāre ready because āwe can easily patch themā. End users have basically become testers, but itās ok, because this is Agile. Weāve started coding to expect failure and somehow with it decided that failure is normal and expected and we donāt need to put so much effort into avoiding it. Supporting millions of customers is complicated so we donāt bother. Why waste time reading bug reports from users when you can just send them into an endless maze of help links with no contact information?</p>
<p>I never used to be this grumpy. The last few years Iāve seen so many ridiculous errors in software and on websites that I just canāt help but feel a little embarassed about what we (as software devs) are unleashing on the world. I know weāre a young, inexperienced industry and that there arenāt enough skilled devs to go around but lately it feels like weāre really not even trying.</p>
<p>Hereās a collection of some screenshots Iāve taken <strong>just in the last month</strong> showing what I mean. Is it just me? Am I really unlucky? Or does this happen to everyone and itās just me that likes to put effort into being vocal and annoyed by it?</p>
<p>Itās not unusual to see data thatās unescaped or badly encoded, but itās not often you see things over-encoded. NPM package names appear to be HTML encoded twice! Letās hope the first one wasnāt as it was saved into the database :/</p>
<p><img src="/post_images/fails/npm_encoding_fail.jpg" alt="If in doubt, encode it again!" /></p>
<p>While investigating an issue for a friend, I found a page on the ASUS site where the title shown in Google perfectly matched our problem. I clicked through and the page was blank, except for this form asking if it solved my problem.</p>
<p><img src="/post_images/fails/asus_missing_content.jpg" alt="Did this blank page solved you problem?" /></p>
<p>IKEA emailed me reminding me I hadnāt completed a survey that I had. I figured I hadnāt clicked save. I clicked to complete it again and was told Iād already filled it in.</p>
<p><img src="/post_images/fails/bad_reminder.jpg" alt="Accusation of not completing a survey that was completed the day it was sent!" /></p>
<p>I got an email today from Coursera with an unreplaced replacement token in the subject.</p>
<p><img src="/post_images/fails/coursera.jpg" alt="Unreplaced token in email subject" /></p>
<p>Our company anti-virus for the <em>second time in less than a month</em> blocked access to most of the internet. Twitter was full of people having the same issue. I find this truly astonishing. Do they not install updates on a machine and give them a quick test before pushing out to the world? Large sites like the BBC and Google were blocked!</p>
<p><img src="/post_images/fails/eset_killed_internet.jpg" alt="ESET antivirus killed half of the internet AGAIN" /></p>
<p>Not sure what to make of this message when shutting my Raspberry Pi down!</p>
<p><img src="/post_images/fails/failed_or_succeeded.jpg" alt="Succeeded to fail!" /></p>
<p>Tried to visit the HTC Vive website and got a MongoDB connection error spewed to the screen. Reported it on Twitter and quickly got a witty response; but it remained broken until the next day.</p>
<p><img src="/post_images/fails/htc_vive.jpg" alt="New product website down over the weekend, technical details on display" /></p>
<p>I tried setting up Ubuntu MATE on my Pi. I wrote their (Pi-specific) image to my SD card and this happened on first boot. Something wasnāt tested; whether itās the image or some dependency being pulled from the web at first boot, Iām not sure.</p>
<p><img src="/post_images/fails/mate_install_fail.jpg" alt="Clean operating system image fails to start up" /></p>
<p>I was trying to download IIS Log Parser today but the download didnāt seem to work. I opened the dev tools to find ājQuery not definedā errors all over the place. Persisted across refreshes.</p>
<p><img src="/post_images/fails/microsoft_jquery_fail.jpg" alt="Microsoft website non-functional apparently due to missing jQuery" /></p>
<p>I tried to move my gas and electric to NPower. They sent me a welcome email and when I clicked the button in it I got a big scary SSL warning from my browser. It turned out to be a wildcard certificate (*.npower.com) but the emails links point to https://npower.com/. I reported this to at least 5 different people there, yet nobody seemed to take it seriously (or even understand it, as shown here). I certainly wouldnāt want to show my new customers browser warnings that say attackers may be trying to steal their information!</p>
<p><img src="/post_images/fails/npower_ssl_issue.jpg" alt="Security is overrated" /></p>
<p>One security issue on your site is bad enough; but come on, the live chat has a different certificate error!</p>
<p><img src="/post_images/fails/npower_chat_ssl.jpg" alt="If we don't have working SSL on our website, why would we have it on the Live Chat?" /></p>
<p>Yesterday I tried to checkout with Paypal and after logging in I got this message. Itās bad enough to get an error during payment for something, but this message is particularly useless. Thereās no information to help me. No support phone number, no indication of whether I was charged or what I should do next.</p>
<p><img src="/post_images/fails/paypal.jpg" alt="Did you just charge my card or not?" /></p>
<p>Many years ago, Microsoft took a lot of flak for podcasts only working on Windows Phone in the US. Fast forward a few years and it seems Google want in. I see no good reason for this; theyāre free audio files. All of the other Google Play Music services work in the UK, so really, whatās the reason for this?</p>
<p><img src="/post_images/fails/podcasts_country.jpg" alt="Audio is country-specific..." /></p>
<p>Podomatic helpfully prefix your iTunes URL with āhttp://ā. This happens even if you already have āhttps://ā (which, by the way, is the correct protocol for an iTunes link - HTTP gets redirected automatically). The only solution is to manually edit the URL to be HTTP before saving, even though thatās wrong.</p>
<p><img src="/post_images/fails/podomatic_forced_http.jpg" alt="Are you sure you didn't mean HTTP?" /></p>
<p>This one is amazing. Someone signed up my email address to a service using the name āPro_Hackingā. I reported it because I was concerned that maybe their service was not very secure and someone might end up with an account tied to my email. Itās hard to believe the support person was even reading the emails the number of times I explained āPro_Hackingā is not my name, to get a response starting āHello Pro-Hackingā!</p>
<p><img src="/post_images/fails/server_pro_hacking.jpg" alt="Hello, Pro_Hacking!" /></p>
<p>The move of our gas and electric to NPower didnāt work out. It turns out that for some āmysterious technical reasonā they are unable to take over supplies from the company that (physically) supply our gas. So we arranged to move to SWALEC. Weeks on, weāve still had no code mailed to us and are therefore unable to log in to their website. The contact link on the only page we can access goes to an error page.</p>
<p><img src="/post_images/fails/swalec_contact_fail.jpg" alt="If you're frustrated we didn't do our job, here's an error page!" /></p>
<p>This website accused me of using an ad blocker and wouldnāt let me read this article. Quite clearly from the screenshot, I am not using an AdBlocker. I completely appreciate the need to monetise content (which is why Iāve been putting off installing an Ad Blocker) but if sites are going to add code like this it should fail-safe. Accusing me of stealing from your children when Iām innocent is just going to drive me away!</p>
<p><img src="/post_images/fails/v3_adblock.jpg" alt="Website refuses to let me read articles because it claims I'm using an AdBlocker (if I am, it's not very effective)" /></p>
<p>I tried setting up VNC on my Pi. Itās behind a firewall so I entered a short password. I was told it was too short, so I added a few characters. I was told it was truncated to EIGHT characters.</p>
<p><img src="/post_images/fails/vnc_passwords.jpg" alt="Short passwords are not secure. Long passwords are too secure" /></p>
<p>When installing a Visual Studio update, it told me the Visual Studio Improvement Program was optional. Iām pretty sure I opted-out at install time; however this time it was both ticked and disabled (yet still sporting the āOptionalā tag).</p>
<p><img src="/post_images/fails/vs_optional_but_not.jpg" alt="I think we have different definitions of the word Optional" /></p>
<p>Immediately after installing Visual Studio 2015 Update 2 (which claimed to fix a bunch of stability issues), I launched Visual Studio and it just crashed. I donāt believe I have any non-Microsoft extensions installed.</p>
<p><img src="/post_images/fails/vs_update_2.jpg" alt="Update installed! *crash*" /></p>
<p>Every few weeks without me doing anything, my Moto 360 Android Wear watch will just randomly start draining itās battery until itās flat/rebooted. I reported this to Google once and was told āthis is expected behaviour after updatesā. I hadnāt had any updates, but even so, that would be stupid.</p>
<p><img src="/post_images/fails/wear_drain.jpg" alt="???" /></p>
<p>While preparing this post I went looking for a command line tool to optimise the images in this post. I came across <a href="http://www.creativebloq.com/design/image-compression-tools-1132865">this post</a> and every time I tried to scroll down the page, it immediately jumped back up to the top. After about 40 seconds a full-page ad appeared, which appears to have been the cause. <em>sigh</em>.</p>
<p>Thatās a lot of fail for just one month, and Iām sure thereās a lot more I didnāt have screenshots of. I canāt help but feel that as an industry weāre just not doing our best for our users. Even companies that have previously been known for exceptional quality and testing seem to have gone down the shitter. Iām no stranger to commercial pressures causing things to be shipped before theyāre done, but surely thereās room for improvement?</p>
<p>Or maybe itās the end users fault? Maybe we donāt complain enough when things are bad, so companies have little motivation to improve?</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2016/04/have-software-developers-given-up/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2016-04-23:/2016/04/removing-rsyslog-spam-on-raspberry-pi-raspbian-jessie/2016-04-23T00:00:00+00:002016-04-23T00:00:00+00:00Removing [action 'action 17' suspended] rsyslog Spam on Raspberry Pi (Raspian Jessie)
<p>As a bit of a Linux noob, Iāve been keeping an eye on the logs for my Raspberry Pi to ensure I havenāt set anything up wrong. I noticed that quite frequently my syslog contains this sort of junk:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Apr 2 01:23:51 raspberrypi rsyslogd-2007: action <span class="s1">'action 17'</span> suspended, next retry is Sat Apr 2 01:24:21 2016 <span class="o">[</span>try http://www.rsyslog.com/e/2007 <span class="o">]</span>
Apr 2 01:29:41 raspberrypi rsyslogd-2007: action <span class="s1">'action 17'</span> suspended, next retry is Sat Apr 2 01:30:11 2016 <span class="o">[</span>try http://www.rsyslog.com/e/2007 <span class="o">]</span>
Apr 2 01:31:35 raspberrypi rsyslogd-2007: action <span class="s1">'action 17'</span> suspended, next retry is Sat Apr 2 01:32:05 2016 <span class="o">[</span>try http://www.rsyslog.com/e/2007 <span class="o">]</span>
Apr 2 01:35:52 raspberrypi rsyslogd-2007: action <span class="s1">'action 17'</span> suspended, next retry is Sat Apr 2 01:36:22 2016 <span class="o">[</span>try http://www.rsyslog.com/e/2007 <span class="o">]</span>
Apr 2 01:39:06 raspberrypi rsyslogd-2007: action <span class="s1">'action 17'</span> suspended, next retry is Sat Apr 2 01:40:06 2016 <span class="o">[</span>try http://www.rsyslog.com/e/2007 <span class="o">]</span>
Apr 2 01:44:39 raspberrypi rsyslogd-2007: action <span class="s1">'action 17'</span> suspended, next retry is Sat Apr 2 01:45:39 2016 <span class="o">[</span>try http://www.rsyslog.com/e/2007 <span class="o">]</span>
Apr 2 01:53:08 raspberrypi rsyslogd-2007: action <span class="s1">'action 17'</span> suspended, next retry is Sat Apr 2 01:54:08 2016 <span class="o">[</span>try http://www.rsyslog.com/e/2007 <span class="o">]</span>
Apr 2 01:59:41 raspberrypi rsyslogd-2007: action <span class="s1">'action 17'</span> suspended, next retry is Sat Apr 2 02:00:41 2016 <span class="o">[</span>try http://www.rsyslog.com/e/2007 <span class="o">]</span>
</code></pre></div></div>
<p>As well as being annoying when scanning through logs, it also doesnāt really help the life of the SD card on a Pi if things are churning data out needlessly. It turns out to be caused by rsyslog trying to write to /dev/xconsole and for whatever reason, failing. The <code class="language-plaintext highlighter-rouge">rsyslog.conf</code> file in Raspbian Jessie contains this at the end:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># The named pipe /dev/xconsole is for the `xconsole' utility. To use it,</span>
<span class="c"># you must invoke `xconsole' with the `-file' option:</span>
<span class="c"># </span>
<span class="c"># $ xconsole -file /dev/xconsole [...]</span>
<span class="c">#</span>
<span class="c"># NOTE: adjust the list below, or you'll go crazy if you have a reasonably</span>
<span class="c"># busy site..</span>
<span class="c">#</span>
daemon.<span class="k">*</span><span class="p">;</span>mail.<span class="k">*</span><span class="p">;</span><span class="se">\</span>
news.err<span class="p">;</span><span class="se">\</span>
<span class="k">*</span>.<span class="o">=</span>debug<span class="p">;</span><span class="k">*</span>.<span class="o">=</span>info<span class="p">;</span><span class="se">\</span>
<span class="k">*</span>.<span class="o">=</span>notice<span class="p">;</span><span class="k">*</span>.<span class="o">=</span>warn |/dev/xconsole
</code></pre></div></div>
<p>The fix is simple enough - just remove those last lines. Iāve been trying to script all of the setup of my Pi so I can easily rebuild it if (when) I trash things or to try things out on a new SD card.</p>
<p>After much Googling, I settled on this command to do it for me:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo sed</span> <span class="nt">-i</span> <span class="s1">'/# The named pipe \/dev\/xconsole/,$d'</span> /etc/rsyslog.conf
</code></pre></div></div>
<p>This says match all lines starting with the one that matches <code class="language-plaintext highlighter-rouge"># The named pipe /dev/console</code> and ending with the last line (<code class="language-plaintext highlighter-rouge">$</code>) and delete them. If you have any additional lines at the end of your file youāll need to tweak this, but that seems unlikely (most things are likely to write into <code class="language-plaintext highlighter-rouge">/etc/rsyslog.d/</code>. I donāt know if you need to restart rsyslog after this; I rebooted just for fun.</p>
<p>Hope this is helpful. If you have any problems, leave a comment. Bear in mind Iām a Linux noob and whatās written above might not be the best way to achieve this and I take no responsibility if anything breaks :)</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2016/04/removing-rsyslog-spam-on-raspberry-pi-raspbian-jessie/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2016-04-09:/2016/04/setting-up-automatic-updates-on-raspberry-pi-raspbian-jessie/2016-04-09T00:00:00+00:002016-04-09T00:00:00+00:00Setting up Automatic Updates on Raspberry Pi (Raspian Jessie)
<p>Logging into machines and installing security updates periodically isnāt fun but for internet-exposed devices itās important. Any device thatās on your home network has the possibility of being a stepping stone for attackers if it can be easily breached.</p>
<p>The first thing to do before setting up automatic updates is to ensure your Raspberry Pi can send email. Youāll want to know when updates are being installed (or if they fail). My <a href="/2016/04/setting-up-raspberry-pi-raspbian-jessie-to-send-email/">previous blog post</a> covers how to do this with Postfix.</p>
<p>As always, start off by making sure your apt list and existing packages are up-to-date:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># Update the package list, update all packages and remove any packages that are no longer required</span>
<span class="nb">sudo </span>apt-get update <span class="nt">-y</span> <span class="o">&&</span> <span class="nb">sudo </span>apt-get dist-upgrade <span class="nt">-y</span> <span class="o">&&</span> <span class="nb">sudo </span>apt-get autoremove <span class="nt">-y</span></code></pre></figure>
<p>Next we need to install the <code class="language-plaintext highlighter-rouge">unattended-upgrades</code> package and to ensure it sends emails the <code class="language-plaintext highlighter-rouge">apt-listchanges</code> package. <code class="language-plaintext highlighter-rouge">apt-listchanges</code> also requires a <code class="language-plaintext highlighter-rouge">mailx</code> program so if you donāt already have one you can grab <code class="language-plaintext highlighter-rouge">bsd-mailx</code>:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">sudo </span>apt-get <span class="nb">install </span>unattended-upgrades apt-listchanges bsd-mailx <span class="nt">-y</span></code></pre></figure>
<p>Next we should configure where the updates are allowed to come from.If you choose to stick with Stable then when the next version of Raspbian goes stable (Stretch) itāll automatically update. Iāve decided to stick with Jessie for now. This config lives in <code class="language-plaintext highlighter-rouge">/etc/apt/apt.conf.d/50unattended-upgrades</code> and you can use this script to uncomment the line for Jessie.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">sudo sed</span> <span class="nt">-i</span> <span class="s1">'s/^\/\/ "o=Raspbian,n=jessie"/ "o=Raspbian,n=jessie"/g'</span> /etc/apt/apt.conf.d/50unattended-upgrades</code></pre></figure>
<p>Next we want to instruct the updater to send emails. Again, this is already in the config file so itās just a case of uncommenting it (you may wish to tweak the user, but Iāve <a href="/2016/04/setting-up-raspberry-pi-raspbian-jessie-to-send-email/">already set</a> <code class="language-plaintext highlighter-rouge">root</code> mail to be forwarded on to my user).</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">sudo sed</span> <span class="nt">-i</span> <span class="s1">'s/^\/\/Unattended-Upgrade::Mail "root";/Unattended-Upgrade::Mail "root";/g'</span> /etc/apt/apt.conf.d/50unattended-upgrades</code></pre></figure>
<p>By default your Pi wonāt be rebooted if required, so if you want it to (and want to set the time) you can do that like this:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">sudo sed</span> <span class="nt">-i</span> <span class="s1">'s/^\/\/Unattended-Upgrade::Automatic-Reboot "false";/Unattended-Upgrade::Automatic-Reboot "true";/g'</span> /etc/apt/apt.conf.d/50unattended-upgrades
<span class="nb">sudo sed</span> <span class="nt">-i</span> <span class="s1">'s/^\/\/Unattended-Upgrade::Automatic-Reboot-Time "02:00";/Unattended-Upgrade::Automatic-Reboot-Time "02:00";/g'</span> /etc/apt/apt.conf.d/50unattended-upgrades</code></pre></figure>
<p>And if if you want unused packages to be removed (like when you run <code class="language-plaintext highlighter-rouge">apt-get autoremove</code>:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">sudo sed</span> <span class="nt">-i</span> <span class="s1">'s/^\/\/Unattended-Upgrade::Remove-Unused-Dependencies "false";/Unattended-Upgrade::Remove-Unused-Dependencies "true";/g'</span> /etc/apt/apt.conf.d/50unattended-upgrades</code></pre></figure>
<p>Next we must create the <code class="language-plaintext highlighter-rouge">/etc/apt/apt.conf.d/20auto-upgrades</code> file to instruct the updater what to do:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># You could also create this file by running "dpkg-reconfigure -plow unattended-upgrades"</span>
<span class="nb">sudo tee</span> /etc/apt/apt.conf.d/20auto-upgrades <span class="o">></span> /dev/null <span class="o"><<</span><span class="no">EOF</span><span class="sh">
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";
EOF</span></code></pre></figure>
<p>And thatās all there is to it! Every day your Pi will now check for updates and youāll receive an email like this if there were:</p>
<p><img src="/post_images/pi_updates.jpg" alt="Updates email from Raspberry Pi" /></p>
<p>If you want to chek itās working, you can check the log file tomorrow:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">cat</span> /var/log/unattended-upgrades/unattended-upgrades.log</code></pre></figure>
<p>Hope this is helpful. If you have any problems, leave a comment. Bear in mind Iām a Linux noob and whatās written above might not be the best way to achieve this and I take no responsibility if anything breaks :)</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2016/04/setting-up-automatic-updates-on-raspberry-pi-raspbian-jessie/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2016-04-08:/2016/04/setting-up-raspberry-pi-raspbian-jessie-to-send-email/2016-04-08T00:00:00+00:002016-04-08T00:00:00+00:00Setting up a Raspberry Pi (Raspian Jessie) to send Email
<p>There are a bunch of things on Linux that send emails by default (for example, the output of Cron jobs). Being a Linux noob I wanted to ensure I received these emails (what if something goes bad because of my noobness?) and also wanted to ensure my Raspberry Pi can send emails to me from scripts if required in the future.</p>
<p>There are a lot of tutorials online about how to do this most of which <strong>require hard-coding your email account password on your Pi</strong>. Iām not ready for this sort of commitment (even if itās an App-Specific password) and knowing a little about SMTP I was fairly sure it wasnāt required if I was only deliverying mail to one place.</p>
<p>So, the things we need to do are:</p>
<ol>
<li>Set up all mail to be passed on to the same local account (in my case <code class="language-plaintext highlighter-rouge">danny</code>)</li>
<li>Set up that accounts mail to forward to an external address (a real email address, mine being hosted by Google)</li>
<li>Install and configure an MTA that can actually connect to the external SMTP server and deliver the email</li>
</ol>
<p>All of these seemed pretty straight forward, until I actually came to tryā¦ It turns out that if youāre deliverying email to Google and your ISP supports IPv6 (mine does) and you donāt want to disable IPv6 (I do not) then you will find yourself receiving errors like this:</p>
<blockquote>
<p>Our system has detected that this message does not meet IPv6 sending guidelines regarding PTR records and authentication. Please review https://support.google.com/mail/?p=ipv6_authentication_error for more information.</p>
</blockquote>
<p>For some reason Google have added some extra restrictions on being able to talk to their SMTP servers over IPv6, some of which you likely canāt solve (because the IP address is owned by your ISP and not you). <a href="http://tanguy.ortolo.eu/blog/article109/google-ipv6-smtp-restrictions">Itās not just me that thinks this sucks</a>.</p>
<p>So, we have an additional requirement:</p>
<ul>
<li>Our MTA must be able to deliver only over IPv4, despite IPv6 being set up and configured</li>
</ul>
<p>I set about testing a whole bunch of MTAs and I failed at gtting almost all of them working (mostly because of this IPv6 issue). Possibly this is because of lack of experience for Linux but whatever the reason, I needed something I could make work! Eventually I got PostFix working (or at least, I thought I didā¦ keep reading!).</p>
<p>The first thing I do when setting up a new Raspberry Pi is ensure everything is up-to-date. If your Raspbian image is quite old there might be many updates for you to install. This might take a little while:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># Update the package list, update all packages and remove any packages that are no longer required</span>
<span class="nb">sudo </span>apt-get update <span class="nt">-y</span> <span class="o">&&</span> <span class="nb">sudo </span>apt-get dist-upgrade <span class="nt">-y</span> <span class="o">&&</span> <span class="nb">sudo </span>apt-get autoremove <span class="nt">-y</span></code></pre></figure>
<p>Next we need to install and configure Postfix. When you install this it will automatically open a config utility (I donāt think it should, because of <code class="language-plaintext highlighter-rouge">-y</code> but it doesā¦). <strong>Just select <code class="language-plaintext highlighter-rouge">No Config</code> and <code class="language-plaintext highlighter-rouge">OK</code></strong> since weāre going to write the entire config file ourselves.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">sudo </span>apt-get <span class="nb">install </span>postfix <span class="nt">-y</span></code></pre></figure>
<p>Now to write the Postfix config file. We need to set a couple of things; descriptions are inline in comments:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">sudo tee</span> /etc/postfix/main.cf <span class="o">></span> /dev/null <span class="o"><<</span><span class="no">EOF</span><span class="sh">
# Where to read account aliases, used to map all emails onto one account
# and then on to a real email address
alias_maps = hash:/etc/aliases
# This sets the hostname, which will be used for outgoing email
myhostname = pi.dantup.com
# This is the mailserver to connect to deliver email
# NOTE: This must be the MX server for the account you wish to deliver email to
# or an open relay (but you hopefully won't find one of them). In my case, this
# is Google's first MX server (which can be found by doing an MX lookup on my domain).
relayhost = aspmx.l.google.com
# Which interfaces to listen on. We don't want anyone connected to our Pi to send email,
# so we set this to the local loopback interface only.
inet_interfaces = loopback-only
# This one is important for the reasons mentioned above. This means only IPv4 will be used,
# avoiding the IPv6 restrictions Google have in place.
inet_protocols = ipv4
EOF</span></code></pre></figure>
<p>Now we need to create the aliases file referenced above. We simply map <code class="language-plaintext highlighter-rouge">root</code> mail on to our main user (<code class="language-plaintext highlighter-rouge">danny</code>) and then that user on to the required email address.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># Set the aliases</span>
<span class="nb">echo</span> <span class="s2">"root: danny"</span> | <span class="nb">sudo tee</span> <span class="nt">-a</span> /etc/aliases
<span class="nb">echo</span> <span class="s2">"danny: danny+pi@notreallymydomain.com"</span> | <span class="nb">sudo tee</span> <span class="nt">-a</span> /etc/aliases
<span class="c"># Rebuild alias db and restart Postfix</span>
<span class="nb">sudo </span>newaliases
<span class="nb">sudo</span> /etc/init.d/postfix start</code></pre></figure>
<p>And thatās that. Simples! Next we need to send a test email to ensure everything is working. We send this to a local account (not a full email address) to ensure weāre testing the whole thing.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">echo</span> <span class="nt">-e</span> <span class="s2">"FROM: root</span><span class="se">\n</span><span class="s2">TO: root</span><span class="se">\n</span><span class="s2">Subject: Test email from the Raspberry Pi</span><span class="se">\n\n</span><span class="s2">This is a test email sent from your Raspberry Pi"</span> | sendmail <span class="nt">-t</span>
<span class="nb">echo</span> <span class="nt">-e</span> <span class="s2">"</span><span class="se">\e</span><span class="s2">[33mTest email sent. Make sure it turns up :)</span><span class="se">\e</span><span class="s2">[0m"</span></code></pre></figure>
<p>If youāve done everything right and Google (or your mailhost) donāt suspect you of being a spammer (which is possible if youāre connecting from a domestic IP address, but so far Iāve had no issues) you should have an email in your mailbox. For extra profit, add the sender to your address book and add a nice profile picture :)</p>
<p><img src="/post_images/pi_email.png" alt="Email from Raspberry Pi" /></p>
<p>For extra points (and to reduce the chance of being labelled a spammer), you probably want to set up SPF records for your home IP address. If this changes a lot, you could use a dynamic DNS service. For example, my SPF record looks like this:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">v</span><span class="o">=</span>spf1 include:_spf.google.com a:home.dantup.com ~all</code></pre></figure>
<p>This will allow Google and the IP that home.dantup.com resolves to to send mail and it be considered an SPF pass.</p>
<h2 id="not-so-fast">Not so fastā¦</h2>
<p>At this point I thought everything was dandyā¦ Until I rebooted! If youāre using Raspbian Jessie (but not Raspbian Jessie Lite) youāll find that after a reboot that your email is broken. This reason for this is that Raspbian Jessie defaults to a <code class="language-plaintext highlighter-rouge">Fast Boot mode</code> where it does not wait for the network during booting. Postfixes creates a <code class="language-plaintext highlighter-rouge">chroot jail</code> and copies <code class="language-plaintext highlighter-rouge">/etc/resolv.conf</code> into it at startup, but in fast boot mode this is too early and it ends up blank. This means Postfix is unable to resolve DNS and no emails get sent!</p>
<p>It took me several days to track down what was happening here but the fix is as simple as running <code class="language-plaintext highlighter-rouge">sudo raspi-config</code> and selecting <code class="language-plaintext highlighter-rouge">Slow Boot</code>, or running the following command:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">sudo </span>raspi-config nonint do_wait_for_network Slow</code></pre></figure>
<p>This will create a script that runs at startup which blocks until DHCP completes before allowing other services (like Postfix) to start up. I <a href="https://github.com/RPi-Distro/repo/issues/24">posted my frustration with this default on GitHub</a> and was informed that neither way was great, because <code class="language-plaintext highlighter-rouge">Slow Boot</code> would wait for a DHCP timeout (which could be quite long) if a Pi booted and wasnāt connected to the network.</p>
<p>Itās worth noting that in Raspbian Jessie Lite the defaut is the <code class="language-plaintext highlighter-rouge">Slow Boot</code> (yay!) and therefore you donāt have to worry about this step.</p>
<p>Hope this is helpful. If you have any problems, leave a comment. Bear in mind Iām a Linux noob and whatās written above might not be the best way to achieve this and I take no responsibility if anything breaks :)</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2016/04/setting-up-raspberry-pi-raspbian-jessie-to-send-email/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2016-03-04:/2016/03/installing-lighttpd-php7-and-letsencrypt-on-raspberry-pi-raspbian-jessie-lite/2016-03-04T00:00:00+00:002016-03-04T00:00:00+00:00Installing Lighttpd, PHP 7 and LetsEncrypt on a Raspberry Pi (Raspbian Jessie Lite)
<p>I have a friend who wanted to serve some simple PHP scripts over HTTPS from a Raspberry Pi. Heād seen some benchmarks showing that Apache was a bit of a hog so was trying to use Nginx but was having trouble because none of these apps are in the Raspbian repos.</p>
<p>Iām a total noob when it comes to Linux (I got my first Pi a week or so ago), but it didnāt seem like this should be a complicated setup, so I decided to see if I could quickly get it working. I have plenty of SD cards lying around and my Pi wasnāt doing anything important so thereās nothing to lose.</p>
<h2 id="goals">Goals</h2>
<p>The main goal for me was to take a new install of Raspbian (Iām using Raspbian Jessie Lite, but standard Raspbian via NOOBS should be fine too) and render a PHP āHello Worldā over HTTPS with a real cert trusted by browsers with minimal effort, while:</p>
<ul>
<li>Preferring NGINX or Lighttpd to Apache</li>
<li>Preferring PHP 7 to anything older</li>
<li>Getting free TLS certs from <a href="https://letsencrypt.org/">LetsEncrypt</a></li>
<li>Scheduling renewal of TLS certs via cron</li>
</ul>
<h2 id="replacements">Replacements</h2>
<p>If youāre going to use any of these scripts, there are two things littered throughout that you must replace with your own values:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">/var/www/html</code> is the webroot. Iām using the standard location Lighttpd (and Apache!) use out-of-the-box.</li>
<li><code class="language-plaintext highlighter-rouge">pi.dantup.com</code> is the public hostname for the site/cert. This is mine; replace it with your own!</li>
</ul>
<h2 id="complications">Complications</h2>
<ul>
<li>The LetsEncrypt client is not available in the Raspbian Jessie repositories (nor debian Jessie)</li>
<li>PHP 7 is not available in the Raspbian Jessie repositories (nor Debian Jessie or Jessie Backports)</li>
</ul>
<h2 id="getting-raspbian">Getting Raspbian</h2>
<p>Step 1 is to get a working Raspbian install. I chose to use <a href="https://www.raspberrypi.org/downloads/raspbian/">Raspbian Jessie Lite from here</a> because I tend to run my Pis headless. These instructions should all be the same for standard Raspbian (inc. via NOOBS).</p>
<h2 id="security-considerations">Security Considerations</h2>
<p>If youāre going to expose your Pi (or any other computer) to the web, you should consider the security. Someone breaching a device on your network could give them access to other devices inside your network! Some things you might wish to consider doing to your Pi if youāre going to expose it (especially if opening SSH):</p>
<ul>
<li>Create your own user and ditch the <code class="language-plaintext highlighter-rouge">Pi</code> user (at a minimum, change the password to something strong!)</li>
<li>Disable root login over SSH</li>
<li>Change the SSH port</li>
<li>Set up <a href="https://wiki.debian.org/UnattendedUpgrades">unattended security upgrades</a></li>
</ul>
<p>Iāve been working on a script that does this (and more) for me when I set up a new Pi. Iām hoping to finish/tidy/blog it over the coming weeks.</p>
<h2 id="setting-up-raspbian">Setting up Raspbian</h2>
<p>First thing I do when setting up Raspbian is expand the filesystem (you wonāt need to do this if using NOOBS, but will if youāve written a Raspbian <code class="language-plaintext highlighter-rouge">.img</code>) and ensure all packages are up-to-date.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># Expand the filesystem and reboot</span>
<span class="nb">sudo </span>raspi-config <span class="nt">--expand-rootfs</span>
<span class="nb">sudo </span>reboot
<span class="c"># Update lists, perform upgrades, remove orphaned packages</span>
<span class="nb">sudo </span>apt-get update <span class="nt">-y</span> <span class="o">&&</span> <span class="nb">sudo </span>apt-get dist-upgrade <span class="nt">-y</span> <span class="o">&&</span> <span class="nb">sudo </span>apt-get autoremove <span class="nt">-y</span></code></pre></figure>
<h2 id="switch-to-root-shell">Switch to root shell</h2>
<p>Pretty much everything here requires root, so itās easiest to switch to a root shell to avoid prefixing everything with <code class="language-plaintext highlighter-rouge">sudo</code>.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">sudo</span> <span class="nt">-i</span></code></pre></figure>
<h2 id="installing-lighttpd">Installing Lighttpd</h2>
<p>This one is pretty painless. Install via <code class="language-plaintext highlighter-rouge">apt</code> and replace the default page with our own Hello World.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">apt <span class="nb">install </span>lighttpd <span class="nt">-y</span>
<span class="nb">rm</span> /var/www/html/index.lighttpd.html
<span class="nb">echo</span> <span class="s1">'<h1>Hello, World!</h1>'</span> <span class="o">></span> /var/www/html/index.html</code></pre></figure>
<p>At this point you should be able to visit http://raspberrypi/ (assuming you havenāt changed the hostname) and see your Hello World message. Itās a good start, but itās not HTTPS!</p>
<h2 id="installing-packages-from-debian">Installing packages from Debian</h2>
<p>Because neither PHP 7 nor the LetsEncrypt client are in the Raspbian repos, we need to fetch them from the Debian ones. Before <code class="language-plaintext highlighter-rouge">apt</code> will download things from there, we need to trust them. You should probably confirm these keys (such as by skipping this step, letting the next step fail, and then getting the keys from the error message) rather than copy/paste them from some random guys blog (which wasnāt served over HTTPS) though!</p>
<p>The error youāll get prior to trusting these keys looks like this:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">W: GPG error: http://http.debian.net jessie-backports InRelease: The following
signatures couldn<span class="s1">'t be verified because the public key is not available:
NO_PUBKEY 8B48AD6246925553 NO_PUBKEY 7638D0442B90D010</span></code></pre></figure>
<p>I donāt know why there are two. Did I mention Iām a Linux noob?</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">gpg <span class="nt">--keyserver</span> pgpkeys.mit.edu <span class="nt">--recv-key</span> 8B48AD6246925553
gpg <span class="nt">-a</span> <span class="nt">--export</span> 8B48AD6246925553 | apt-key add -
gpg <span class="nt">--keyserver</span> pgpkeys.mit.edu <span class="nt">--recv-key</span> 7638D0442B90D010
gpg <span class="nt">-a</span> <span class="nt">--export</span> 7638D0442B90D010 | apt-key add -</code></pre></figure>
<h2 id="install-the-letsencrypt-client">Install the LetsEncrypt Client</h2>
<p>The LetsEncrypt client is available from Jessie Backports. Weāll need to add the source to be able to install it from there. My script removes the source at the end (and re-updates the packages; I donāt know if thatās required) since we only want to borrow these packages and not use this for anything more.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">echo</span> <span class="s2">"deb http://httpredir.debian.org/debian jessie-backports main contrib non-free"</span> <span class="o">></span> /etc/apt/sources.list.d/debian-jessie-backports.list
apt-get update <span class="nt">-y</span>
apt <span class="nb">install </span>letsencrypt <span class="nt">-t</span> jessie-backports <span class="nt">-y</span>
<span class="nb">rm</span> /etc/apt/sources.list.d/debian-jessie-backports.list
apt-get update <span class="nt">-y</span></code></pre></figure>
<h2 id="requesting-tls-certs">Requesting TLS Certs</h2>
<p>In order for the LetsEncrypt client to work, it needs to be able to connect to your Pi using the hostname you want the cert for (this is to verify you actually own this domain, or at least, control the web server it points at). This may mean setting up DNS records to point the domain at your Pi and/or forwarding ports from a router (youāll need port 80 for this step, but may as well also map 443 while youāre at it to save coming back to it later).</p>
<p>Because thereās no automatic module to set up <code class="language-plaintext highlighter-rouge">Lighttpd</code> we need to use the <code class="language-plaintext highlighter-rouge">--webroot</code> method (this means the client will write files into your webroot and LetsEncrypt will connect back and read them). Pass each webroot with <code class="language-plaintext highlighter-rouge">-w</code> and the domain with <code class="language-plaintext highlighter-rouge">-d</code>. You can add multiple sets of these as required.</p>
<p>Remember to replace the webroot and domain with your own!</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">letsencrypt certonly <span class="nt">--webroot</span> <span class="nt">-w</span> /var/www/html <span class="nt">-d</span> pi.dantup.com</code></pre></figure>
<h2 id="setting-up-tls-in-lighttpd">Setting up TLS in Lighttpd</h2>
<p>Lighttpd expects certs to be combined, so we need to concatonate them before we can configure it. Remember to replace your domain in the path (note: this is in /etc/letsencrypt and not your webroot!).</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">pushd</span> /etc/letsencrypt/live/pi.dantup.com/
<span class="nb">cat </span>privkey.pem cert.pem <span class="o">></span> combined.pem
<span class="nb">popd</span></code></pre></figure>
<p>Next we need to add TLS config for Lighttpd. We need to point at the cert we just combined as well as the full chain certificate that will be served up to the client browser. We also disable SSLv2 and SSLv3 for security.</p>
<p>Again, be sure to replace your domain in the certificate paths.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">tee</span> /etc/lighttpd/conf-enabled/letsencrypt.conf <span class="o">></span> /dev/null <span class="o"><<</span><span class="no">EOF</span><span class="sh">
</span><span class="nv">$SERVER</span><span class="sh">["socket"] == ":443" {
ssl.engine = "enable"
ssl.pemfile = "/etc/letsencrypt/live/pi.dantup.com/combined.pem"
ssl.ca-file = "/etc/letsencrypt/live/pi.dantup.com/fullchain.pem"
ssl.cipher-list = "ECDHE-RSA-AES256-SHA384:AES256-SHA256:HIGH:!MD5:!aNULL:!EDH:!AESGCM"
ssl.honor-cipher-order = "enable"
ssl.use-sslv2 = "disable"
ssl.use-sslv3 = "disable"
}
</span><span class="no">EOF
</span>/etc/init.d/lighttpd force-reload</code></pre></figure>
<p>At this point, you should now be able to visit https://raspberrypi/ (assuming you havenāt changed the hostname) and see your Hello World message again. Youāll get a warning about the cert name not matching (since youāre not accessing it via the real domain), but thatās expected. Accessing it via the real domain may work depending on your setup. Hurrah! Encryption!</p>
<h2 id="installing-php-7">Installing PHP 7</h2>
<p>Like the LetsEncrypt client, PHP 7 is not available in the Raspbian repos. Nor is it available in Debian Jessie Backports. However it is available in Stretch (the next version after Jessie),so we can do a similar thing and get it from there.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">echo</span> <span class="s2">"deb http://httpredir.debian.org/debian stretch main contrib non-free"</span> | <span class="nb">tee</span> /etc/apt/sources.list.d/debian-stretch.list
apt-get update <span class="nt">-y</span>
apt <span class="nb">install </span>php7.0 php7.0-fpm <span class="nt">-t</span> stretch <span class="nt">-y</span>
<span class="nb">rm</span> /etc/apt/sources.list.d/debian-stretch.list
apt-get update <span class="nt">-y</span></code></pre></figure>
<p>Next we need to enable <code class="language-plaintext highlighter-rouge">fastcgi</code> and tell <code class="language-plaintext highlighter-rouge">Lighttpd</code> where to find PHP.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">tee</span> /etc/lighttpd/conf-enabled/php.conf <span class="o">></span> /dev/null <span class="o"><<</span><span class="no">EOF</span><span class="sh">
fastcgi.server += (".php" => ((
"socket" => "/var/run/php/php7.0-fpm.sock"
)))
</span><span class="no">EOF
</span>lighttpd-enable-mod fastcgi
/etc/init.d/lighttpd force-reload</code></pre></figure>
<p>Finally, letās replace our boring old Hello World with a nice PHP one!</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">rm</span> /var/www/html/index.html
<span class="nb">echo</span> <span class="s1">'<?php echo "<h1>Hello, World (from PHP)!</h1>"; ?> '</span> <span class="o">></span> /var/www/html/index.php</code></pre></figure>
<p>If youāve done all this right, https://raspberrypi/ (assuming you havenāt changed the hostname) will now greet you from PHP! Almost doneā¦</p>
<h2 id="automatic-tls-cert-renewal">Automatic TLS Cert Renewal</h2>
<p>Not renewing your certificates would be pretty embarassing, so we should make that automatic. Weāll do this via Cron. Cron sends emails with the output of commands, so if youāve set up emails to be forwarded to a real mailbox (this will be covered in my Pi setup script which I hope to blog in the coming weeks) youāll get a note each time this runs. Weāll set it to run weekly since by default it only renews certs thatāll expire in the next 30 days, so monthly could cause them to be missed.</p>
<p><strong>Edit</strong>: Mike Scalora <a href="#comment-2671739539">posted</a> a <a href="https://gist.github.com/mscalora/94f384d1311f66ac09ea6d31d77a102e">better version</a> of this that handles multiple domains neatly in the comments :)</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">tee</span> /etc/cron.weekly/letsencrypt <span class="o">></span> /dev/null <span class="o"><<</span><span class="no">EOF</span><span class="sh">
# Renew cert
letsencrypt renew
# Rebuild the cert
pushd /etc/letsencrypt/live/pi.dantup.com/
cat privkey.pem cert.pem > combined.pem
popd
# Reload
/etc/init.d/lighttpd force-reload
</span><span class="no">EOF
</span><span class="nb">chmod</span> +x /etc/cron.weekly/letsencrypt</code></pre></figure>
<h2 id="thats-it">Thatās It!</h2>
<p>I believe thatās everything! If you hit problems please post in the comments and Iāll try to fix them up. Please note that Iām a Linux noob, some (or all) of this might not be done in the best possible way. Again, Iāll make tweaks if people send corrections/improvements :-)</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2016/03/installing-lighttpd-php7-and-letsencrypt-on-raspberry-pi-raspbian-jessie-lite/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2015-12-15:/2015/12/my-week-in-a-tesla-model-s-p85/2015-12-15T00:00:00+00:002015-12-15T00:00:00+00:00My Week in a Tesla Model S P85
<p>9 months ago my wife and I took delivery of two Renault ZOEs. We hadnāt had EVs before but after test driving a ZOE and being offered a great deal on two we found it hard to turn down! Although the range of the ZOE is only 70-100 miles (depending on weather, etc.) weāve had no problems using the ZOE for a summer holiday racking up over a thousand miles in a week, including two 270 mile trips to/from the holiday park. Neither of us would willingly go back to combustion cars.</p>
<p>In August, Chargemaster Plc (who made the chargepoint we have at home and also run the POLAR charging network) <a href="https://www.chargemasterplc.com/index.php/blog/polar-plus/">launched a new scheme called āPOLAR Plusā</a> which rewarded customers for charging on their network with points <a href="http://polar-network.com/experience">which could be used to ābidā</a> for a weeks use of one of their fleet of electric vehicles (including a Tesla Model S and a BMW i8!). The number of points needed to loan the Tesla Model S was only 70 (which is just 7 qualifying charge sessions). I was sceptical but <a href="https://twitter.com/ChargemasterPlc/status/631766409538736128">they assured me</a> all was as it seemed. The scheme costs Ā£7.85 per month but the first six months are free.</p>
<p>Fast forward to November and it happened, my week in Chargemasterās Tesla Model S P85 was confirmed, 8th - 14th December! At this time Iām still in my free 6-month trial so Iāve paid Ā£0 to Chargemaster! :)</p>
<div class="filmstrip">
<img src="/post_images/tesla/me_1_t.jpg" />
<img src="/post_images/tesla/me_2_t.jpg" />
<img src="/post_images/tesla/gadget_3_t.jpg" />
</div>
<p>I picked up the car from Chargemasterās head office in Luton on Tuesday 8th Dec. At more than twice the price of the most expensive car Iāve owned I was expecting it to be great but it still seriously exceeded my expectations. I wanted one from the minute I sat inside it! Evie (an apt name) from Chargemaster ran me through some paperwork and the basics of the Model S and we were soon on our way home. It should be a 3-4hr journey back to where we live in Cheshire but with terrible traffic and a stop at Warrington to try out the Supercharger it took approximately 5hrs driving (mostly in the dark and rain) to get home.</p>
<h2 id="exterior">Exterior</h2>
<p>Iād already seen pictures/videos of the Model S but it looked much better than I expected up-close. If youāre used to spending this sort of money on a car you may be used to having cars that look this nice but sadly, I am not! It looked stunning and it was pretty huge (especially compared to the ZOE which is quite tiny).</p>
<div class="filmstrip">
<img src="/post_images/tesla/ext_1_t.jpg" />
<img src="/post_images/tesla/ext_2_t.jpg" />
<img src="/post_images/tesla/ext_3_t.jpg" />
</div>
<h2 id="performance">Performance</h2>
<p>This Model S was a P85. This is the rear-wheel-drive āperformanceā version (the all-wheel-drive performance version is the P85D). Itās not available in this configuration anymore but I believe the 0-60 time is something like 4.1 seconds. To me, this qualifies as āstupidly fastā. For various reasons (public roads, very wet week, itās not my car and being somewhat scared of the performance I did experience) I didnāt get to experience how fast this thing would accelerate foot-to-the-floor but it was quite clear from just a little bit of āfunā I had that the power in this car is utterly ridiculous. Iāve never driven a RWD car before and feeling the back end twitching in the wet is scary. If I somehow ended up with the money to buy one of these I think Iād opt for the āDā version just to avoid that danger! :D</p>
<h2 id="interior">Interior</h2>
<p>Inside, the Model S feels expensive. With a few exceptions (the plastic on the front of the arm rest rattled a little and felt a bit cheap) everything feels very premium and well-made. Thereās nice stitched leather around the dash and the leather seats are very comfortable. As thereās no handbrake/gear-selector between the front seats, everything is very open and spacious. The back easily accommodates adults with decent legroom and was big enough that the kids couldnāt reach to kick the back of the seat. Bonus!</p>
<div class="filmstrip">
<img src="/post_images/tesla/int_1_t.jpg" />
<img src="/post_images/tesla/int_2_t.jpg" />
<img src="/post_images/tesla/int_3_t.jpg" />
</div>
<h2 id="softwaretouchscreen">Software/Touchscreen</h2>
<p>The thing that is always a huge disappointment in cars Iāve owned is the software. Every time I get a new car I think āsoftware mustāve moved on, itāll be good nowā, but no. The Model S bucks this trend and has excellent software on an enormous 17ā touch screen front-centre. Rather than hardware dials/knobs that canāt be changed (and can break) the Model S does almost everything in software from this same touchscreen. My wife initially said the touchscreen was ātoo bigā but I think she was coming around to it by the end of the week. It has Google Maps-based navigation, the reversing camera, calendar, Bluetooth/music/radio, consumption graphs and everything else you can think of all together in one place. All the settings you might like to tweak (regen, creep, suspension/ride height, sunroof, charge timer, automatic lights etc.) are all controlled here. Whereas many cars feel like a Frankenstein of disparate systems āintegratedā together, everything here feels like it was built as one.</p>
<p>The excellent software extends to the display behind the steering wheel which shows a small version of the navigation map (when navigating) so you can keep your eyes directly ahead.</p>
<h2 id="bootfrunk-storage">Boot/Frunk Storage</h2>
<p>Although itās not very deep, the boot is rather large and easily fit our pushchair, coats and plenty of shopping. Thereās also a really decent amount of storage under the bonnet (āfrunkā/āfront trunkā) that could easily fit all of our shopping in it and would be less likely to roll around than in the boot.</p>
<div class="filmstrip">
<img src="/post_images/tesla/storage_1_t.jpg" />
<img src="/post_images/tesla/storage_2_t.jpg" />
<img src="/post_images/tesla/storage_3_t.jpg" />
</div>
<h2 id="regen-braking">Regen Braking</h2>
<p>When I picked the car up the regen braking was set to āLowā. I switched it to āStandardā as I was quite used to regen braking in the ZOE and liked minimising the use of the brake pedal. I was surprised at just how aggressive āStandardā was and found myself having to put my foot back on the gas after lifting off many times while I got used to it. Once I got the hang of it, it was absolutely brilliant - I previously thought I didnāt use the brake pedal much in the ZOE but this was just miles better. One-pedal-driving is great!</p>
<p>On one of the days it was around 5 degrees Celsius. I noticed the regen braking wasnāt anywhere near as effective (it caught me by surprise) and I noticed the speedo showed a dotted line in the regen area which indicates that it canāt regen/charge at the normal rate. It was a little weird that the cold weather was changing the driving experience so much but after just a little driving/braking it seemed to have warmed up enough for this to go away and things were back to normal.</p>
<h2 id="gadgets">Gadgets</h2>
<p>The Model S comes with a bunch of nice things/gadgets, many of which arenāt unique to the Model S (or even EVs) but they still added to the experience.</p>
<p>With the exception of the rear view mirror all adjustments are electric (seats, wing mirrors and even steering wheel adjustment) and saved against your driver profile. If someone else drives the car, selecting your profile afterwards will move everything back to how you had it.</p>
<p>The wing mirrors automatically angle downwards when you shift into reverse to help you see white lines or parking spaces on the road (and return when you shift out of reverse).</p>
<p>The reversing camera is far higher definition than the car in our ZOE and shows on the huge screen making it really easy to reverse-park (which is super useful because forwards parking is tricky when you donāt really know how big the car is and are unable to see the front corners!).</p>
<p>Front seats are heated. Iāve never cared much for these in the past but with the cold weather they were <em>really</em> nice! Something small but I liked, was that when you turned them on they defaulted to the highest setting and reduced as you continue to tap the button. This matches how youād use them pretty well (itās cold, put them on full then reduce over time) so you donāt have to keep tapping to increase the heat when you turn them on. Unfortunately there was no heated steering wheel to go with it and the wheel was often quite cold so I found myself driving around all toasty and warm but with cold hands!</p>
<p>The key for the car is a miniature Model S. Pressing the front opens the frunk, the back opens the boot and the middle opens the car. Itās a nice touch and looks pretty cool.</p>
<p>When I was returning the car I entered the business park and the car showed a message along the lines of āIncreasing ride height based on locationā. I donāt know how this had been set up, but due to speed bumps the ride height was increased!</p>
<div class="filmstrip">
<img src="/post_images/tesla/gadget_1_t.jpg" />
<img src="/post_images/tesla/gadget_2_t.jpg" />
<img src="/post_images/tesla/gadget_3_t.jpg" />
</div>
<h2 id="negatives">Negatives</h2>
<p>There really arenāt many things we didnāt like about the Model S, it really is an excellent car. There are a couple of niggles or things I think could be improved though:</p>
<ul>
<li>Chargeflap seemed to stick a lot; took lots of locking/unlocking before it would open (even the button on the screen didnāt seem to work). On the way back to Luton I had to give up trying to top it up at the services because of this (though unsurprisingly, it appeared to be fine when we arrived at Chargemaster HQ!).</li>
<li>The plastic at the front of the arm rest rattled a little which made it feel cheap. I donāt know if they all do this or it was just this car.</li>
<li>Canāt pre-heat the car from the fob (the ZOE can do this), so if youāre going out unexpectedly you might have a cold car. Itās possible the app allows you to do this but sadly we didnāt have access to that (it requires a username/password we didnāt have).</li>
<li>Canāt set a pre-heat timer within the car (again, ZOE does this), only enable āsmart preconditioningā (which doesnāt help if your schedule isnāt fixed or youāre loaning a car that has been used by others). Again, the app might allow this but I was unable to check.</li>
<li>At one point I was driving along and the sunroof dumped a load of water on my passenger! I donāt think the sunroof was leaking and it hadnāt been opened in days but I suspect water had collected somewhere inside (it had been opened when it was a little wet a few days earlier) and a hill/bend caused it to come dripping out.</li>
<li>Because the cold weather affects the regen braking, itād be nice if there was an option for the car to make up for it with friction braking so that the braking/one-pedal-driving experience doesnāt vary so much as the weather changes or battery warms up. Itās a surprise to lift your foot off the accelerator and the car not slow down as much as it normally does.</li>
<li>The priceā¦ Before I picked the car up I thought it was overpriced. I donāt think this anymore! However even if itās worth the money itās still way out of my price range so of course I wish it was cheaper! =D</li>
<li>I was a little paranoid about the key not being directly attached to my keyring but just in a pouch which it looked like it could easily have slipped out. It didnāt come out for me, but I wouldnāt be surprised if people have lost their ākeysā because of this.</li>
<li>The Satnav doesnāt show the speed limit for the current road. Iām used to having this in TomTom in the ZOE and kept looking for it. I know the Autopilot version shows this (read from the road signs; maybe only in US?) but thereās no reason it canāt use a database too (presumably it has one as part of the Google mapping for travel times?)</li>
</ul>
<h2 id="other-opinions">Other Opinions</h2>
<p>Everybody that I took out in the Model S seemed to love it. Even the people that previously werenāt entirely sold on current-gen EVs (for various reasons) like my parents loved it. My mum (who doesnāt really care about cars) text me saying āitās aceā and I think if my dad could possibly make the numbers work out he would place his order today. We definitely convinced a bunch more people this week that EVs can be great and that itās undoubtedly not a fad thatās going to disappear.</p>
<h2 id="summary">Summary</h2>
<p>It was a great experience driving the Model S for a week and Iām really grateful to Chargemaster for making it happen. Itās expected that the Model 3 may be released before we plan to return our ZOEs at the end of their PCP term and may only cost Ā£30,000 so Iāll be keeping a close eye on that and what government incentives we have at the time. Chargemaster told me theyāve ordered their Model X and it will be added to the EV Experience scheme when it arrives (expected late Q2/early Q3 2016).</p>
<p>If I ever find myself with Ā£80,000 to spend on a new car, I know what will be top of the shopping list. In the meantime, I guess Iām stuck with the ZOE (except for if/when Iām able to borrow more cars from Chargemaster!).</p>
<h2 id="going-back-to-the-zoe">Going Back to the ZOE</h2>
<p>Today I got to do my commute back in the ZOE. It was an interesting change! The ZOE seating position is much higher (despite the Model S also having batteries along the bottom) and the car feels like it has a much higher centre of gravity. The reversing camera is a tiny low-definition mess. The weak regen braking caught me out a few times and I found myself having to hit the brakes while I got used to it. Although the ZOE felt fast compared to my previous petrol car (a 1.6 petrol Megane) it feels <em>really</em> slow compared the Model S and the accelerator is (unsurprisingly) just nowhere near as responsive.</p>
<p>All that said, once I got used to driving the ZOE again towards the end of my commute, I was reminded how fun it was. Driving the really Model S hasnāt changed that and I still arrived at work with a big grin on my face!</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2015/12/my-week-in-a-tesla-model-s-p85/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2015-09-20:/2015/09/simple-windows-browser-selector/2015-09-20T00:00:00+00:002015-09-20T00:00:00+00:00Simple Windows utility to act as default browser and launch different browsers based on domain
<p><a href="https://github.com/DanTup/BrowserSelector">
<img style="position: absolute; top: 0; right: 0; border: 0;" src="https://camo.githubusercontent.com/a6677b08c955af8400f44c6298f40e7d19cc5b2d/68747470733a2f2f73332e616d617a6f6e6177732e636f6d2f6769746875622f726962626f6e732f666f726b6d655f72696768745f677261795f3664366436642e706e67" alt="Fork me on GitHub" data-canonical-src="https://s3.amazonaws.com/github/ribbons/forkme_right_gray_6d6d6d.png" />
</a></p>
<p>I recently saw <a href="https://twitter.com/ckindel/status/644970792095080448">a tweet</a> about having different default browsers on Windows for different urls. This seemed like an interesting idea - I use a few systems that work better in one browser than another. I retweeted and saw a few responses from others that seemed interested in the idea.</p>
<p>It seemed like a pretty easy thing to do - register as a browser and when a url comes in, just bounce it over to the correct ārealā browser based on some config. So I thought Iād knock something togetherā¦</p>
<p>It didnāt take long, nor much code. There are about 30 lines of code to create (or delete) registry values to register with Windows as a browser, about 60 lines of code to parse a poor version of an INI file, and around 50 lines of plumbing to pull them together and look up the URLs.</p>
<p>Code is up on <a href="https://github.com/DanTup/BrowserSelector">GitHub (DanTup/BrowserSelector)</a>, and a <a href="https://github.com/DanTup/BrowserSelector/releases">binary release</a> if you just want to run it.</p>
<p>Config looks like this:</p>
<figure class="highlight"><pre><code class="language-ini" data-lang="ini"><span class="c">; Default browser is first in list
</span><span class="nn">[browsers]</span>
<span class="py">chrome</span> <span class="p">=</span> <span class="s">C:</span><span class="se">\P</span><span class="s">rogram Files (x86)</span><span class="se">\G</span><span class="s">oogle</span><span class="se">\C</span><span class="s">hrome</span><span class="se">\A</span><span class="s">pplication</span><span class="se">\c</span><span class="s">hrome.exe</span>
<span class="py">ie</span> <span class="p">=</span> <span class="s">iexplore.exe</span>
<span class="c">; Url preferences.
; Only * is treated as a special character (wildcard).
; Matches are domain-only. Protocols and paths are ignored.
; Use "*.blah.com" for subdomains, not "*blah.com" as that would also match "abcblah.com".
</span><span class="nn">[urls]</span>
<span class="py">microsoft.com</span> <span class="p">=</span> <span class="s">ie</span>
<span class="err">*</span><span class="py">.microsoft.com</span> <span class="p">=</span> <span class="s">ie</span></code></pre></figure>
<p>Itās a first version, so Iām sure it has bugs and can be improved. I canāt make any promises to evolve it further (itās hard to find coding time since kids came along!) but since itās pretty trivial maybe it wonāt need much work to improve and fix bugs.</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2015/09/simple-windows-browser-selector/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2015-07-27:/2015/07/response-from-cyc/2015-07-27T00:00:00+00:002015-07-27T00:00:00+00:00Response from CYC
<p>I <a href="/2015/07/gmev-and-cyc-the-uks-awful-recharging-network/">recently blogged about poor experiences using the CYC network</a> (mostly GMEV chargers) and sent the post to CYC. Today CYC responded.</p>
<blockquote>
<p>Hi Dan,</p>
<p>Thanks for your email. Iāll work down your blog post point by point if thatās ok?</p>
<p>First of all, the problems youāve have had with the App. Weāre a little hamstrung here as our App was built by a third party who still retain most of the administrational rights for it so all changes need to go through them and be made on their side of things. When you first reported the email problem this was immediately sent to them to correct. I will also make them aware of the App security issues you have flagged up but I assure you that both CYC and our developers are fully PCI Compliant.</p>
<p>The mistakenly taken Ā£1 pre-auths ā I use the word mistakenly because I think āfraudulentlyā is quite harsh ā thank you for flagging this is up us. As we told you at the time, this was a problem in the way the app had been set up with Paypoint and when someone registered their card an error in the code meant that Paypoint counted it as a transaction and not a pre-auth. As soon as you flagged this up we reported it to the developers and after a fair bit of troubleshooting the problem was identified and rectified. CYC then retroactively went through and refunded everyone who had ever signed up their Ā£1 ā Including me, so thank you.</p>
<p>You will be pleased, Iām sure, to know that we are currently building a new App in-house that will hopefully solve a large amount of these issues and improve communications between the app and post. We (or I at least) are happy to admit that the App isnāt great atm. Itās sticky, it lags, itās clunky. Previous management were keen to be first to market with the app and so a lot of mistakes were made that will hopefully be corrected in the new version. The wireframes for the new app have recently been signed off and so we hope it will be ready by late September, however this is dependent on how many cups of coffee and cans of energy drink we can pour down Joe.</p>
<p>RFID Cards attached to wrong accounts ā This has happened on the odd occasion unfortunately but has been rectified as soon as itās been flagged up. CYC Applications are processed by a member of the team here and, as is always the way, human error can occur. As much as Iād like to delete my colleagues, I do enjoy their company on occasion and so I have asked them to be far more vigilant. As far as charging drivers for usage that isnāt their own, please rest assured that any charges levied in error will be removed and ANY money taken in error will ALWAYS be refunded in full as soon as itās flagged up. You seem to be under the impression that weāre out to stiff drivers and take their money ā I can categorically say that this is not the case. Weāre actually quite nice people.</p>
<p>RFID Cards. You hate RFID cards, I hate RFID cards, everyone hates RFID cards. Unfortunately though for funding to be issued for a charger, it NEEDS to have a unique access method so its usage and EV uptake can be monitored. There is currently no way around it ā Although we are looking into alternatives. As such, we offer RFID cards priced at Ā£20 per annum. Transport for Greater Manchester do subsidise RFID Cards for drivers within their remit, this was done as a way of encouraging EV uptake in the region and the cards are priced at Ā£10 ONE OFF. As you can imagine, a lot of people would rather pay Ā£10 one off than Ā£20 per annum and so we get drivers from outside of the region trying to register (Holland being the furthest so far) If we get an application from outside of the region but close to Manchester (such as yourself) we ask TfGM about the application. As we explained to you at the time, they confirmed that you fell outside of their area.</p>
<p>We did say at the time that you could apply for a CYC Access Card if you wanted and we explained to you why we could not give you one for free in the same way that other networks do. The income generated by Access Cards (actually very little when you take away the VAT, cost of the card, postage and admin time) is directly used to help fund the CYC Helpdesk and our Out of Hours support ā Something which we DO NOT wish to withdraw. Sorry to jump to another point, but it seems to lead on nicely.</p>
<p>The problem you had this weekend with a trapped cable. You are correct, the main CYC office is closed at weekends. At present we operate 08.00-18.00 Mon-Fri. Outside of these hours we operate an emergency support number. This is a member of the CYC team with a mobile phone and laptop who, no matter what time of night or day, is available to aid stranded drivers. If you call the office number you will hear a message stating that the office is closed but if you have a trapped cable please call a number. Had you of called that number, the member of staff on call this weekend would have been more than happy to stop what they were doing and release your cable.</p>
<p>Just so weāre clear, the staff member on Out of Hours only has limited access to systems and is only there for releasing trapped cables. However we never knowingly leave a driver stranded. There have been many, many occasions where weāve started sessions, reset posts, directed to nearby chargers, helped non-CYC members get a charge etcā¦ The reason the phone does not simply direct to the team member is in an attempt divert what weād deem ānon-emergenciesā to business hours, for example, a call at 5am because a driver couldnāt remember their Apple ID.</p>
<p>Incorrect charger locations and blank numbers ā This is hugely disappointing. When a CP is commissioned the installing engineer takes a lat/lon reading from the post and send it to us. If this is incorrect, weāre in trouble from the get go. As you can imagine we donāt know where each of our chargers are, weāre based in Brighton and I havenāt even set foot in Manchester for years, so we rely on correct information from others. The same goes for parking information. The CYC site showed what we were given. As for the stickers, I am looking into why these havenāt been applied at the moment. I assume they were and some hilarious child removed them. Clearly we need to rethink how we number our Charge Points if they can be removed so easily.</p>
<p>Regarding the failed sessions at various Charge Points, if your starttransation message was reaching our servers, there would be a record of it. If it hasnāt reached us, we donāt know about it. By the sounds of it, it looks like youāve had several occasions when, for whatever reason, the starttransation has not made it to our server. Weāre looking into why this is but for the time being I have removed the chargers at Hulme Street, Bury Market and Media City from the App. Looking that the records from last week there were a number of successful charge sessions, started by the App, at the Trafford Centre and I know this is a popular location so these have not been removed at present. Again, we hope that the new version of the app will solve a lot of these issues but itās worth mentioning that last week the CYC estate clocked up 762 Charging Sessions started with the App. 43 of which were in Manchester.</p>
<p>Lastly, Iād like to thank you for reporting these problems to us and I hope the fact that Iāve given such a thorough and frank reply will go some way to convincing you that we do actually care.</p>
<p>If you have any more questions or comments I would be more than pleased to help.</p>
<p>Many thanks</p>
</blockquote>
<p>So CYC are now certainly aware of the experience that I (and maybe others) have had. Itās great that CYC took the time to respond in this detail :)</p>
<p>Itās interesting that theyāre sure that if the ācharge startā request hits their server, it will be logged. If the issue weāre seeing is not connectivity, then maybe itās fixable in software (I assumed it would need better mobile antennas in the chargers). Iām hoping to proxy some failing āstart chargeā requests next time we go to Manchester (if I can find a way to do such a think on Android without a portable machine running Fiddler!) in the hope of maybe helping understand this unreliability.</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2015/07/response-from-cyc/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2015-07-25:/2015/07/gmev-and-cyc-the-uks-awful-recharging-network/2015-07-25T00:00:00+00:002015-07-25T00:00:00+00:00GMEV and CYC - The UKs Awful Recharging Network
<p><strong>EDIT</strong>: CYC provided a response to this post. It can be found <a href="/2015/07/response-from-cyc/">here</a>.</p>
<p>A little over three months ago, my wife and I took delivery of our new cars - Renault ZOEs. These are pure electric cars with 22kWh batteries that will do in the region of 90 miles on a full charge. We love our EVs and wouldnāt change them for anything, but weāve had terrible experiences with the CYC network. Since we tell friends and family of all the positive things about EVs, itās only fair we also share details of the bad things to ensure weāre not misleading anyone about what itās like to own them.</p>
<p>This is a long post; but thatās because there have been so many issues! :(</p>
<h2 id="sign-up-on-the-website-cant-login-to-app">Sign up on the website, canāt login to appā¦</h2>
<p>I signed up with <a href="http://chargeyourcar.org.uk/">CYC</a> back in April. I signed up on the website, which allowed me to use an email address with a + in it (this is a perfectly valid character to use in an email address). When I came to install the app on my phone, I discovered that the app does not allow you to login with email addresses that contain plusses! This is ridiculous for two reasons; 1) plusses are valid and 2) there is no value in validating an email address prior to attempting to login. I reported this to CYC, asking them to either fix their app or update my email address, but was left unable to use their app until this was done. A week or so later, they replied to say theyād updated my email address (no word on them fixing the issue), and I was finally able to login.</p>
<h2 id="app-security-issues">App security issues</h2>
<p>While waiting for CYC to update my email address (above), I made some attempts to login to their app by trying to guess what the issue might be. I guessed maybe they were failing to encode the email address properly when sending it to their server (such that the + may be interpreted as a space, as is possible with url decoding). I entered the url-encoded version of a plus, and the app gave me back an error message <em>with a stack trace</em> from their server. This means there are certain characters in email addresses that <em>pass the (bad) validation</em> yet cause server errors. This makes me suspicious that there is a security exploit hiding in here. I have reported this to CYC and had no response about it. This really makes me question how safe my card details are on their system.</p>
<h2 id="fraudulent-1-charges-from-cyc">Fraudulent Ā£1 charges from CYC</h2>
<p>While waiting for CYC to update my email address (above), we wanted to try using one of their chargers. Since my wife has an EV, she signed for a CYC account too. When we tried to use their charger in Birkenhead at the ferry terminal we found that she needed to add card details (despite the charger being free). She did so, and a few days later we discovered that the Ā£1 pre-auth the app claimed to make was actually a real Ā£1 charge. Whilst this is not a significant amount of money, it is unacceptable (and fraudulent) to take money from someones account like this. We reported this to CYC who acknowledged the issue, blamed PayPoint(!) and refunded it. I posted <a href="https://speakev.com/threads/oops-cyc-charging-1-at-signup-supposed-to-be-pre-auth.8780/">on the SpeakEV forums</a> in case others had had the same charges and despite several people quickly saying they had, it was a month before CYC finally acknowledged the issue and said they had refunded everyone affected.</p>
<h2 id="rfid-cards-attached-to-the-wrong-accounts">RFID cards attached to the wrong accounts</h2>
<p>One of the responses in the thread I made about the Ā£1 charges (above) had a link to <a href="https://speakev.com/threads/cyc-free-charge-anywhere-is-over-for-one-person.4976/">this thread</a> which talks of several people having had RFID cards linked to their accounts that they did not own. This suggests there is some flaw in the process of CYC sending RFID cards that might mean someone gets a card on your account, and you pay for their charging!</p>
<h2 id="unreliable-network-birkenhead-ferry-terminal">Unreliable network (Birkenhead Ferry Terminal)</h2>
<p>This is something that will be come up several times in this post! When trying to start a charge the CYC servers failed to connect to the charge point. This left us looking at a āTrying to start chargeā¦ā screen in the app until it timed out. We tried this over and over. It took 20 minutes of faffing before we eventually got the charger to start. Some people said this could be the mobile connection but we had a full āHā data connection and were able to access other services fine (including full use of the CYC app, until we tried to start the charge).</p>
<h2 id="confusion-over-whether-fees-are-annual-or-one-off">Confusion over whether fees are annual or one-off</h2>
<p>The <a href="http://ev.tfgm.com/">GMEV Website</a> says the fee for their card is Ā£10 one-off (this is subsidised). The CYC website says the fee for their card is Ā£20 annually. It seems strange that one would be one off and one annual, so I emailed both GMEV and CYC asking about this. GMEV (eventually, after 6 weeks) told me theirs is a one-off fee, while CYC didnāt actually answer the question. This makes me suspect CYC might be charging GMEV cardholders annually, despite not being what GMEV are advertising :(</p>
<h2 id="gmev-cards-not-available-to-people-who-live-near-manchester">GMEV cards not available to people who live near Manchester</h2>
<p>Although CYC never answered my question about annual/one-off fee, they did tell me that I canāt have a GMEV card because I donāt live inside Manchester. This is despite living near Manchester (Cheshire) and visiting often. This seems a little backwards given that people in Manchester with EVs probably have home chargers and probably need the GMEV network less. Those that live around Manchester are probably those who would benefit most from GMEV chargers.</p>
<h2 id="incorrect-charger-location-hulme-street">Incorrect charger location (Hulme Street)</h2>
<p>Once I finally had access to my CYC account via the app, we visited Manchester. We found a Hulme Street charger on their app and drove to the marked location on the map. It wasnāt there. Turned out that the location on their map was completely wrong, it was marked several streets away from where it had actually be installed! We headed to Hulme Street (by name) and eventually found the charger.</p>
<h2 id="no-charger-number-hulme-street">No charger number (Hulme Street)</h2>
<p>In addition to using the RFID card or App, there is an option to pay by phone. To use this, you need the number of the charger which is (supposed to be) printed on the side of the charger. It is often not. The Hulme Street charger had big blank spaces where the numbers should be, making it impossible to start a charge by phone unless you can find the charger number (eg. via the app).</p>
<h2 id="unreliable-parking-fee-info-hulme-street">Unreliable parking fee info (Hulme Street)</h2>
<p>The Hulme Street charger was marked as āfree parking while chargingā on the CYC website. When we got there we discovered that this wasnāt the case, and it was pay-and-display. The pay-and-display sign clearly mentioned the EV charger so it wasnāt just for non-charging vehicles.</p>
<h2 id="unreliable-network---no-charge-hulme-street">Unreliable network - no charge (Hulme Street)</h2>
<p>After getting the kids out of the car and the pushchair unfolded, we tried for 30 minutes to get the Hulme Street charger to start a charge. It failed every time. Clearly the CYC network had no ability to connect to this charger. After 30 minutes, we gave up and left Hulme Street to find another charger. Because we had such little charge left we were concerned about getting home. Rather than driving around Manchester using up our remaining juice and having the same experience, we headed to the Trafford Centre (where we knew there were quite a few chargers) getting there with just 7 miles remaining charge!</p>
<h2 id="unreliable-network-trafford-centre">Unreliable network (Trafford Centre)</h2>
<p>Once we arrived at the Trafford Centre, we spent another 20 minutes trying to start a charge. At one point, both my wife and I were trying to start charges from our own apps/logins on different sockets in the hope one would work. We did eventually get charging, but only after much frustration.</p>
<h2 id="no-charger-number-trafford-centre">No charger number (Trafford Centre)</h2>
<p>Like Hulme Street, two ot the three charge posts at Trafford Centre have no numbers on them. It looks like they were written on with a marker pen which, unsurprisingly, has washed away in the rain. The only way we were able to start a charge was by guessing the numbers based on the one unit that them still visible (though the numbers go backwards, which confuses things further).</p>
<h2 id="unreliable-network-trafford-centre-again">Unreliable network (Trafford Centreā¦ again)</h2>
<p>A few weeks later we were were at the Traffod Centre again. There were no other EVs so we had a choice of three posts (6 sockets) to charge. We once again spent around 30 minutes trying to start a charge. We tried <em>all six</em> sockets before we eventually got a charge to start, all because of the CYC servers failing to connect to the chargepoints.</p>
<h2 id="unreliable-network-mediacityuk">Unreliable network (MediaCityUK)</h2>
<p>We were staying at MediaCityUK the weekend we visited the Traffod Centre. Their multistory car park has two GMEV/CYC chargers. Again, we had to try both sockets on each post before we eventually got one to start a charge. This process was repeated the following morning when we topped up.</p>
<h2 id="cyc-have-no-visibility-of-how-unreliable-their-chargers-are">CYC have no visibility of how unreliable their chargers are</h2>
<p>After this latest trip, I contacted CYC on twitter (again) about this unreliability. They told me they couldnāt see <em>any</em> failed charge attempts at either <a href="https://twitter.com/ChargeYourCar/status/623873837701144576">Trafford Centre</a> or <a href="https://twitter.com/ChargeYourCar/status/623873583933227008">MediaCityUK</a>. They believe that their system does show them, however it is quite clear that it does not (we tried over 10 times to start charges at the Trafford Centre). So clearly there is no visibility of failed charges (there should be; since itās their servers failing to talk to the chargepoint). Wow.</p>
<h2 id="incorrect-charger-location-bury-market">Incorrect charger location (Bury Market)</h2>
<p>Today we headed to Bury Market. The CYC website shows a charger at the market (Hilton Road). Where itās marked, is actually a building. I went into this building and asked and was informed that the charger was actually in the market car park, some drive away. There is no way without help youād be able to find this charger from the location on the CYC map (thereās a main road in between).</p>
<h2 id="unreliable-network-bury-market">Unreliable network (Bury Market)</h2>
<p>Because of issues using the apps, we decided to try and use āpay by phoneā today at Bury Market. I called and gave the charger number to the automated phone system and was told the charge had started. It took several minutes for the charger to receive the message to allow charging to start.</p>
<h2 id="unreliable-charger-bury-market">Unreliable charger (Bury Market)</h2>
<p>Even after the charger said it was ready to start the charge, it took 10 minutes of plugging and unplugging before the charge finally locked the cable and allowed the car to start charging.</p>
<h2 id="unreliable-network-bury-market-1">Unreliable network (Bury Market)</h2>
<p>When we returned to the car, we tried to stop the charge. We called the number again and it told us to wait while it stopped the charge. The charger did not get the signalā¦ The automated phone system kept saying āPlease waitā¦ā for some time before it realised it had failed (our cable was still locked in the charge). It told use to press star if the cable had not unlocked. We did. It then said it was trying to restart the charger to unlock the cable. Unsurprisingly, that failed too. It told us if the cable was still not unlocked, we must call the helpline.</p>
<p>We called the helplineā¦ <em>IT WAS CLOSED ON A SATURDAY</em>.</p>
<p><strong>EDIT:</strong> Turns out it there is a number in the message if you keep listening to call for stuck cables. I missed this because it says āThanks for calling CYC, our offices are now closed. Please leave a message after the tone. (pause, noise). If you have a stuck cable, callā¦.ā. So this was my fault, but the way the message is recorded really didnāt help.</p>
<p>We are now stood, 50 miles from home, with our several-hundred-pound cable locked into a charger on what seems to be the worlds-most-unreliable charging network.</p>
<p>I thought I would call and try to stop the charge session again. Maybe this time it would work? I called the number. However, the system thinks Iāve already finished, so didnāt offer to stop my charge. It did offer to stop a session if I provided the last 4 digits of a card number used to start a session. However since the GEMV charger is free, there was no number I could give it that would match this session. In any case, I donāt think that wouldāve worked as the system thought the session had finished.</p>
<p>Crap. I really cannot afford to leave this expensive cable in Bury!</p>
<p>Then I had an idea - what happens if I start a new charge session, then end that? Maybe by fluke it will work? This time, I decided to try the app. Trying to start a session actually caused the cable to unlock, so I grabbed it.</p>
<p>Phew!</p>
<p>If this hadnāt had worked; I wouldāve been pushing CYC to cover the cost of a replacement charge cable. To me, it is unacceptable to have a charger that can have a cable locked in, with no available helpline during a Saturday.</p>
<h2 id="summary">Summary</h2>
<p>Our experience of the CYC network (mostly GMEV, but even non-GMEV chargers) has been poor. No other network has given us any issues like this. CYC keep telling me that an RFID card will solve these issues (no surprise - āgive us money, that will make it betterā), however Iām reluctant to pay Ā£40/year (for the two of us) to find out if thatās true. We would happily pay Ā£20 one-off for two GMEV cards, but alas, we donāt qualify, despite living just far enough from Manchester to need a charge if we visit.</p>
<p>Others keep telling me CYC is fine, and these problems are not CYCs fault. However, these chargers are connected to their network. If CYC canāt monitor the reliability, nobody can. The fact that they donāt even have any visibility of failed charge attempts speaks volumes about how much they care about making their network reliable.</p>
<p>Iād love to hear from others that use CYC (especially GMEV) chargers (in the comments, <a href="https://speakev.com/threads/gmev-and-cyc-the-uks-awful-recharging-network.10259/">SpeakEV</a> or <a href="https://www.reddit.com/r/electricvehicles/comments/3ekxdu/gmev_and_cyc_the_uks_awful_recharging_network/">Reddit</a>). A few of these issues you could consider to just be hiccups but when weāve had the same issues across 10 difference chargers across as many visits, it seems like a far more serious underlying problem.</p>
<p>This doesnāt change our opinions of EVs in any way, but we will definitely avoid CYC chargers wherever we can, and be sure to always have a fallback. Poor charging infrastructure is one of the big hurdles to EV adoption, and CYC are making this worse, not better by giving people a false sense of security about being able to get a charge.</p>
<p><strong>EDIT</strong>: CYC provided a response to this post. It can be found <a href="/2015/07/response-from-cyc/">here</a>.</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2015/07/gmev-and-cyc-the-uks-awful-recharging-network/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2014-12-30:/2014/12/getting-email-notifications-of-xbox-live-games-with-gold-and-deals-with-gold/2014-12-30T00:00:00+00:002014-12-30T00:00:00+00:00How to Get Email Notifications of Xbox Live Games with Gold and Deals with Gold
<p>I recently picked up an Xbox One console (Gamertag is <a href="https://account.xbox.com/Profile?Gamertag=DanTup">DanTup</a>) and have been downloading the free <a href="http://www.xbox.com/live/games-with-gold">Games with Gold</a> titles and picked up a few of the <a href="http://www.xbox.com/live/deals-with-gold">Deals with Gold</a> offers.</p>
<p>I looked around for some way of getting notifications of new deals/free games so that I wouldnāt miss anything if I was busy or didnāt play Xbox for a while, but couldnāt find anything. I did, however, discover that <a href="http://majornelson.com/">Major Nelson</a> posted about every deal/free game (and this month, also the Countdown to 2015 deals). He also has RSS feeds for <a href="http://feeds.feedburner.com/xboxlivemarketplace">all marketplace posts</a> or <a href="http://majornelson.com/category/deal/feed/">all posts tagged ādealā</a>.</p>
<p>I did a bit of searching online, and found a couple of free RSS-to-Email services. Some of them looked a bit naff and some wouldnāt let me sign up with a plus symbol in my address (urgh!). I found two that looked reasonable and didnāt have poor email validation but since I didnāt know how reliable theyād be, I decided to sign up for both and see how they went.</p>
<p>The two services I signed up with are:</p>
<ul>
<li><strong><a href="https://blogtrottr.com/">Blogtrottr</a></strong> - Free with ads; or paid plans are available.</li>
<li><strong><a href="https://feedrabbit.com/">Feedrabbit</a></strong> - Free with no ads, but restricted to 10 RSS feeds and 20 emails/day max; or paid plan available.</li>
</ul>
<p>I was pleasantly surprised that both services have worked perfectly well since I signed up at the start of December, which is around 17 emails so far (theyāre currently arriving daily, due to the Countdown to 2015 deals). The emails tend to arrive within an hour of each other; and according to the timestamps; actually arrive before Major Nelson even posts them! (Timezones FTW).</p>
<p>As mentioned above; Blogtrottr does include ads; but other than that, thereās actually <em>very</em> little difference in the two emails:</p>
<p><a href="/post_images/major_nelson_rss_email.png"><img src="/post_images/major_nelson_rss_email.png" alt="Comparison of emails from Blogtrottr and Feedrabbit" /></a></p>
<p>Since the emails always arrive together and itās such little effort to archive them; Iām going to keep both services active (reducing the chances I might miss one if a service disappears overnight). Although you might prefer Feedrabbitās ad-free emails, it could be argued that Blogtrottrās ad-sponsored version might be more sustainable. Since both have been completely reliable, I canāt really offer any additional information that might swing your choice one way or the other.</p>
<p>If you know of any other services that work well; or have any comments on these two; feel free to post in the comments!</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2014/12/getting-email-notifications-of-xbox-live-games-with-gold-and-deals-with-gold/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2014-11-06:/2014/11/using-inbox-by-gmail-from-a-google-apps-or-non-gmail-email-account/2014-11-06T00:00:00+00:002014-11-06T00:00:00+00:00Using Inbox by Gmail with IMAP or from a Google Apps or non-Gmail email account
<blockquote>
<p><strong>If youāre looking for IMAP support</strong> (lots of search terms suggest this post is getting a lot of traffic for this) then you can just use the same IMAP details you do for Gmail (<a href="https://support.google.com/mail/answer/7126229?hl=en">see here</a>). Inbox is just another frontend onto your Gmail account, you can even switch between the two as you like!</p>
</blockquote>
<p>Google recently launched <a href="http://www.google.com/inbox">Inbox by Gmail</a>. Like all new Google Products, if youāre a Google Apps user (whether it be a legacy free account, or as a paying business), youāve been excluded with no indication from Google for how long.</p>
<p>After much faffing about; I was able to set Inbox up to work with my Google Apps account; including:</p>
<ul>
<li>Emails sent to my Google Apps address show up in Inbox</li>
<li>Replies sent from Inbox show my Google Apps address as the sender</li>
<li>Replies sent from Inbox show up in the Sent folder in my <em>Google Apps</em> account</li>
</ul>
<p>In theory; this should work with both Google Apps accounts and any other email account that you can access via IMAP/POP and SMTP.</p>
<h2 id="you-will-need">You will need</h2>
<ol>
<li>A Gmail account that has been invited to Inbox</li>
<li>An email account you would like to use with Inbox that is not a Gmail account that you have IMAP/POP and SMTP access to</li>
</ol>
<h2 id="step-1-forward-your-email-to-the-gmail-account">Step 1: Forward your email to the Gmail account</h2>
<p>If youāre using a Google Apps account, this is done in Settings -> Forwarding and POP/IMAP. If youāre using a non-Google account, this may vary. If forwarding from a Google Apps account, your Gmail account will be sent an email to click to confirm the address.</p>
<p><img src="/post_images/inbox-1-forward.png" alt="Forward your email to the Gmail account" /></p>
<p>With this done; new email sent to you will turn up in Inbox.</p>
<h2 id="step-2-set-up-your-outgoing-email-address-in-gmail">Step 2: Set up your outgoing email address in Gmail</h2>
<p>Login to your Gmail account, and in Settings -> Accounts and Import youāll need to set up a āSend mail asā address for your primary email address.</p>
<p>Be sure to tick āSend as Aliasā; as this will stop Inbox from including yourself in the To field of all replies.</p>
<p>It is important that you choose the option to send mail through your own SMTP server. If using a Google Apps account; youāll want to use:</p>
<ul>
<li>Server: smtp.gmail.com, Port: 587</li>
<li>Username: Your full Google Apps email address</li>
<li>Password: Your full Google Apps email password; or if you have two-factor auth on, an Application-Specific password</li>
</ul>
<p><img src="/post_images/inbox-2-send-mail-as.png" alt="Set up your outgoing email address in Gmail" /></p>
<p>This allows your Gmail account to send email via your primary accounts SMTP server.</p>
<h2 id="step-3-mark-your-new-outgoing-address-as-default">Step 3: Mark your new outgoing address as default</h2>
<p><img src="/post_images/inbox-3-default-email-address.png" alt="Mark your new outgoing address as default" /></p>
<p>This will ensure any new emails you create in Inbox (with the Compose option) will come from your primary email address.</p>
<h2 id="step-4-ensure-replies-are-sent-from-the-email-address-that-received-them">Step 4: Ensure replies are sent from the email address that received them</h2>
<p><img src="/post_images/inbox-4-reply-from-same.png" alt="Ensure replies are sent from the email address that received them" /></p>
<p>This will ensure when you reply to an email in Inbox, the senders address is set to the same outgoing address that it was addressed to (and in the case of your primary email account, that means the email is sent via the configured SMTP server, which is what causes it to show up in the Sent folder of your Google Apps account).</p>
<h2 id="step-5-clear-your-inbox-by-gmail-app-data">Step 5: Clear your Inbox by Gmail app data</h2>
<p>This step might be important. I didnāt do this the first time I tried this; and Inbox didnāt honour my outgoing email address settings (I believe it had cached the fact that I didnāt have any alternative outgoing email addresses). After clearing the data; Inbox correctly let me pick the outgoing email address, and defaulted using the settings Iād set above.</p>
<p><img src="/post_images/inbox-5-clear-app.png" alt="Clear your Inbox by Gmail app data" /></p>
<p>Enjoy!</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2014/11/using-inbox-by-gmail-from-a-google-apps-or-non-gmail-email-account/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2014-10-29:/2014/10/have-the-angular-team-lost-their-marbles/2014-10-29T00:00:00+00:002014-10-29T00:00:00+00:00Have the Angular Team lost their marbles?
<p>Iāve <a href="/2014/08/you-have-ruined-html/">posted before</a> that Iām not a big fan of frameworks like Angular. Trying to re-invent programming in declarative HTML tags/attributes/{{ curlies }} will likely <a href="http://gbracha.blogspot.co.uk/2014/09/a-domain-of-shadows.html">always feel lacking</a>; and every framework having its own unique way of doing this means a ton of effort needs to be <del>wasted</del><ins>invested</ins> in updating tooling to support/understand it.</p>
<p>So, with my pre-existing bias out of the way; I have to say that recent information about Angularās future only makes me hold my head in my hands even moreā¦</p>
<h2 id="not-just-one-new-concept-to-add-to-tooling-but-three">Not just one new concept to add to tooling; but three!</h2>
<p>It wasnāt enough that tooling needed to be updated to support Angular; it seems that the Angular Team now want tooling to support:</p>
<ul>
<li>Angular v1 syntax</li>
<li>Angular v2 syntax (this appears to be <a href="http://jaxenter.com/angular-2-0-112094.html">significantly different</a>)</li>
<li><a href="https://docs.google.com/document/d/11YUzC-1d0V1-Q3V0fQ7KSit97HnZoKVygDxpWzEYW0U/mobilebasic?pli=1&viewopt=127">AtScript</a></li>
</ul>
<h2 id="different-but-no-better">Different, but no better</h2>
<p>In <a href="http://jaxenter.com/angular-2-0-112094.html">this article</a> thereās a comparison of code for Angular v1 and Angular v2; that looks like this:</p>
<h3 id="angular-v1">Angular v1</h3>
<figure class="highlight"><pre><code class="language-html" data-lang="html"><span class="nt"><div</span> <span class="na">ng-controller=</span><span class="s">"SantaTodoController"</span><span class="nt">></span>
<span class="nt"><input</span> <span class="na">type=</span><span class="s">"text"</span> <span class="na">ng-model=</span><span class="s">"newTodoTitle"</span><span class="nt">></span>
<span class="nt"><button</span> <span class="na">ng-click=</span><span class="s">"addTodo()"</span><span class="nt">></span>+<span class="nt"></button></span>
<span class="nt"><tab-container></span>
<span class="nt"><tab-pane</span> <span class="na">title=</span><span class="s">"Tobias"</span><span class="nt">></span>
<span class="nt"><div</span> <span class="na">ng-repeat=</span><span class="s">"todo in todosOf('tobias')"</span><span class="nt">></span>
<span class="nt"><input</span> <span class="na">type=</span><span class="s">"checkbox"</span> <span class="na">ng-model=</span><span class="s">"todo.done"</span><span class="nt">></span>
{{todo.title}}
<span class="nt"><button</span> <span class="na">ng-click=</span><span class="s">"deleteTodo(todo)"</span><span class="nt">></span>X<span class="nt"></button></span>
<span class="nt"></div></span>
<span class="nt"><tab-pane></span>
<span class="nt"></tab-container></span>
<span class="nt"></div></span></code></pre></figure>
<h3 id="angular-v2">Angular v2</h3>
<p>(Note: Iāve tweaked the HTML slightly to what I believe it should be; the article seemed to have some issues).</p>
<figure class="highlight"><pre><code class="language-html" data-lang="html"><span class="nt"><div></span>
<span class="nt"><input</span> <span class="na">type=</span><span class="s">"text"</span> <span class="na">[value]=</span><span class="s">"newTodoTitle"</span><span class="nt">></span>
<span class="nt"><button</span> <span class="na">(click)=</span><span class="s">"addTodo()"</span><span class="nt">></span>+<span class="nt"></button></span>
<span class="nt"><tab-container></span>
<span class="nt"><tab-pane</span> <span class="na">title=</span><span class="s">"Good kids"</span><span class="nt">></span>
<span class="nt"><div</span> <span class="err">[</span><span class="na">ng-repeat</span><span class="err">|</span><span class="na">todo]=</span><span class="s">"todosOf('good')"</span><span class="nt">></span>
<span class="nt"><input</span> <span class="na">type=</span><span class="s">"checkbox"</span> <span class="na">[checked]=</span><span class="s">"todo.done"</span><span class="nt">></span>
{{todo.title}}
<span class="nt"><button</span> <span class="na">(click)=</span><span class="s">"deleteTodo(todo)"</span><span class="nt">></span>X<span class="nt"></button></span>
<span class="nt"></div></span>
<span class="nt"><tab-pane></span>
<span class="nt"></tab-container></span>
<span class="nt"></div></span></code></pre></figure>
<p>There are some significant differences in how Angular concepts are marked up here; but are they any better? Really? Previously things were embedded in strings in attributes, which was bad enough. But in Angular v2, weāre now using [brackets], (parens) and pipes in attribute names too. Itās different, but I really donāt think it is better.</p>
<h2 id="not-ready-until-2016">Not ready until 2016</h2>
<blockquote>
<p>Angular is aiming for a release by the end of 2015 - but early 2016 seems more realistic given the drastic changes that are planned</p>
<ul>
<li>http://jaxenter.com/angular-2-0-112094.html</li>
</ul>
</blockquote>
<p>This is another big WTF. The changes are so significant that they canāt be progressively released. This makes the v1 -> v2 migration seem like itās going to be an incredible amount of work.</p>
<h2 id="angular-13-eol-after-just-18-24-months">Angular 1.3 EOL after just 18-24 months</h2>
<blockquote>
<p>According to Brad Green of Angular, Angular 1.3 will continue to receive bugfix and security patch support for 18-24 months after the release of version 2.0.</p>
<ul>
<li>http://jaxenter.com/angular-2-0-112094.html</li>
</ul>
</blockquote>
<p>This whole thing is starting to sound like Python 2.7 vs 3.0. Python 3.0 was released in 2008, and the change was so big and breaking, that Python 2.7 end-of-life is currently slated for <strong>2020</strong>. Thatās a 12 year cross-over. The Angular Team expect to give just 18-24 months. I donāt know which I think is more insane; supporting an old version of your product for 12 years, or giving just 18-24 months for people to migrate; both of them seem bat-shit-crazy to me.</p>
<h2 id="conclusion-dont-pick-angular">Conclusion: Donāt pick Angular</h2>
<p>Where I work; weāre currently evaluating a bunch of new technologies/frameworks/etc. to see which might be suitable for use on vNext of our products. Amongst those being prototyped are Dart, React, Angular and a few others. With all of this info on Angular; Iām struggling to see how we could possibly pick it. Our current codebase has parts that are over 10 years old; and we hope our new codebase will last this long too. It seems that if we start writing Angular today; weāll be forced to rewrite the frontend in three to four years at latest (and with the way apps are going, the frontend is likely to be a large codebase). This doesnāt sound very attractive.</p>
<p>Maybe Iāve misunderstood something; but Iām really struggling to see a world in which this all makes good sense.</p>
<p>We need frameworks that are stable and supported long-term; not that are constantly inventing new concepts and being rewritten with breaking changes every 5 minutes. Of everyone, Google should know how hard it is to maintain large web apps :-/</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2014/10/have-the-angular-team-lost-their-marbles/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2014-09-16:/2014/09/dart-vm-on-a-chromebook-without-linux/2014-09-16T00:00:00+00:002014-09-16T00:00:00+00:00Running the Dart VM on a Chromebook in Developer Mode (without installing Linux)
<blockquote>
<p>Writing Dart? Check out <a href="https://marketplace.visualstudio.com/items?itemName=Dart-Code.dart-code">Dart Code, my Dart extension for Visual Studio Code!</a></p>
</blockquote>
<p>I recently picked up a <a href="https://chromebookcompare.com/">Chromebook</a> (a Dell Chromebook 11) and hoped to do a a little bit of hacking on Dart with it. Out-of-the-box the Chromebook canāt run native Dart code (as command line, nor in the browser), so youāre limited to web-based apps using Chrome Dev Editor (which is rather slow at compiling Dart in its JavaScript form).</p>
<p>In order to get around this, many people install Linux (usually Ubuntu via something like Crouton) but I didnāt want to do this because I donāt really like Ubuntu, and it seemed like quite an overhead just to run the (fairly light) Dart VM!</p>
<p>So, when my (replacement!) Chromebook (my 3rd in two weeks!) turned up today; I thought Iād see if the Dart VM would work in Chrome OSās <code class="language-plaintext highlighter-rouge">Developer Mode</code> without having to resort to Linux.</p>
<p>It turned out to actually be pretty simple; and it might be even simpler to someone that understands Linux better than I do! The steps I followed are below. If anyone can improve these; please do post in the comments so I can update them. The editing of the <code class="language-plaintext highlighter-rouge">bash_profile</code> is a bit clunky (mainly because I donāt know <code class="language-plaintext highlighter-rouge">vi</code>; and I didnāt want to use another Chrome extension just to edit the file!) and I donāt really understand why I need to remount.</p>
<ol>
<li><strong><a href="https://sites.google.com/site/chromeoswikisite/home/what-s-new-in-dev-and-beta/developer-mode">Enable developer mode</a></strong>
<ul>
<li><strong>WARNING</strong>: This will delete all of your locally stored data!</li>
<li>Every time you boot your device in this mode, you will get a <a href="https://sites.google.com/site/chromeoswikisite/home/what-s-new-in-dev-and-beta/developer-mode">scary splash screen</a> and have to press Ctrl+D not to reset your device</li>
</ul>
</li>
<li><strong>Download the <a href="https://www.dartlang.org/tools/download.html#a_la_carte">Dart SDK</a> to your Downloads folder</strong>
<ul>
<li>I picked the 64bit version for my <a href="https://chromebookcompare.com/">Dell Chromebook 11</a>; some Chromebooks might require the 32 bit version</li>
</ul>
</li>
<li><strong>Unzip the SDK to your downloads folder</strong>
<ul>
<li>I put this at <code class="language-plaintext highlighter-rouge">~/Downloads/dart-sdk/</code></li>
</ul>
</li>
<li><strong>Switch to the terminal window by pressing <code class="language-plaintext highlighter-rouge">Ctrl+Alt+=></code></strong>
<ul>
<li>(<code class="language-plaintext highlighter-rouge">=></code> being the forward button where F2 would normally be)</li>
</ul>
</li>
<li><strong>Run <code class="language-plaintext highlighter-rouge">chmod +x ~/Downloads/dart-sdk/bin/dart</code></strong>
<ul>
<li>This will mark <code class="language-plaintext highlighter-rouge">dart</code> as executable</li>
</ul>
</li>
<li><strong>Run <code class="language-plaintext highlighter-rouge">chmod +x ~/Downloads/dart-sdk/bin/pub</code></strong>
<ul>
<li>This will mark <code class="language-plaintext highlighter-rouge">pub</code> as executable</li>
</ul>
</li>
<li><strong>Run <code class="language-plaintext highlighter-rouge">echo "sudo mount -i -o remount,exec /home/chronos/user/" >> ~/.bash_profile</code></strong>
<ul>
<li>Iām not a Linux guy; but this seems to be required to allow you to execute anything on this drive</li>
<li>Without this, youāll get <code class="language-plaintext highlighter-rouge">Permission denied</code> trying to execute anything</li>
</ul>
</li>
<li><strong>Run <code class="language-plaintext highlighter-rouge">echo "PATH=\$PATH:~/Downloads/dart-sdk/bin/" >> ~/.bash_profile</code></strong>
<ul>
<li>This adds the Dart <code class="language-plaintext highlighter-rouge">bin</code> folder to your path so you can just type <code class="language-plaintext highlighter-rouge">dart</code> without the full path</li>
</ul>
</li>
<li><strong>Reboot your Chromebook</strong>
<ul>
<li>This isnāt strictly required, you could just re-run <code class="language-plaintext highlighter-rouge">.bash_profile</code>, but itās a good way to ensure it works after a reboot</li>
</ul>
</li>
</ol>
<p>Now when you want to run dart, you can do it from the normal browser-based shell!</p>
<ol>
<li>Press <code class="language-plaintext highlighter-rouge">Ctrl + Alt + T</code> to open a the ChromeOS Terminal</li>
<li>Type <code class="language-plaintext highlighter-rouge">shell</code> to switch to the proper shell</li>
<li>Type <code class="language-plaintext highlighter-rouge">dart</code> or <code class="language-plaintext highlighter-rouge">pub</code> to execute dart/pub</li>
</ol>
<p>Hereās a screenshot showing it works on my <a href="https://www.chromebookchart.com/">Chromebook</a>; Iām using <a href="https://chrome.google.com/webstore/detail/chrome-dev-editor-develop/pnoffddplpippgcfjdhbmhkofpnaalpg?hl=en">Chrome Dev Editor</a> for the Dart editing, and executing the tests in the shell. You can run <code class="language-plaintext highlighter-rouge">pub serve</code> this way and hit up <code class="language-plaintext highlighter-rouge">http://localhost:8080/</code> in the browser for web apps (which I suspect will compile much quicker than the Chrome Dev Editor version).</p>
<p><img src="/post_images/dart_chromebook.png" alt="Dart VM on Chromebook in Developer Mode" /></p>
<p>If youāre interested in a Chromebook; donāt forget to take a look at <a href="https://chromebookcompare.com/">Chromebook Comparison Chart</a>! :)</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2014/09/dart-vm-on-a-chromebook-without-linux/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2014-08-30:/2014/08/chromebook-comparison-chart/2014-08-30T00:00:00+00:002014-08-30T00:00:00+00:00Chromebook Comparison Chart
<p>Iāve recently been <a href="https://www.chromebookchart.com/">looking for a new Chromebook</a>. My Nexus 7 tablet just doesnāt cut it for a lot of things, and Iād like to not have to boot my desktop for some of these (relatively simply) tasks.</p>
<p>Iām aware there are some new Chromebooks being released that are fanless and more powerful and have better battery life than previous versions (such as i3 and Tegra processors) so I started looking online.</p>
<p>I was surprised (and disappointed) that there arenāt any good places that compare hardware in all of the Chromebooks, despite the hardware being the only thing thatās really different with Chromebooks! Iām a bit picky, and definitely want my Chromebook to have a higher-than-1366-768 screen resolution and at least 4GB RAM and it was hard to just find a list of the devices that met these requirements.</p>
<p>I started compiling a table of all the Chromebooks I could find, and kinda got carried away. I thought it was worth posting it online so that others can <a href="https://www.chromebookchart.com/">sort/filter/compare Chromebook models</a> more easily!</p>
<div style="text-align: center; font-size: 25px;">
<a href="https://chromebookcompare.com/">
Google Chromebook Comparison Chart<br />
<img src="http://www.google.com/chrome/assets/common/images/devices/chromebook-family-homepage-promo.jpg" />
</a>
</div>
<p>If you find any mistakes or anything missing; please let me know! You can contact me in various ways from the buttons across the top of this blog.</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2014/08/chromebook-comparison-chart/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2014-08-25:/2014/08/you-have-ruined-html/2014-08-25T00:00:00+00:002014-08-25T00:00:00+00:00You have ruined HTML
<p>Not content with <a href="http://codeofrob.com/entries/you-have-ruined-javascript.html">ruining JavaScript</a>, it seems youāre set on ruining HTML too.</p>
<p>If you ask people what theyāre building their client-side web apps in today; chances are theyāll tell you some hipster JavaScript framework that could be found listed on the <a href="http://todomvc.com/">TodoMVC</a> website. The most popular ones tend to be Angular, Ember, Knockout, Backbone and more recently, Polymer.</p>
<p>Most of these frameworks have this āgreatā feature that lets you āeasilyā reference data in your views using binding expressions. They look something like this:</p>
<figure class="highlight"><pre><code class="language-html" data-lang="html"><span class="c"><!-- Knockout --></span>
<span class="nt"><p></span>First name: <span class="nt"><input</span> <span class="na">data-bind=</span><span class="s">"value: firstName"</span> <span class="nt">/></p></span>
<span class="nt"><p></span>Last name: <span class="nt"><input</span> <span class="na">data-bind=</span><span class="s">"value: lastName"</span> <span class="nt">/></p></span>
<span class="nt"><h2></span>Hello, <span class="nt"><span</span> <span class="na">data-bind=</span><span class="s">"text: fullName"</span><span class="nt">></span> <span class="nt"></span></span>!<span class="nt"></h2></span>
<span class="c"><!-- Angular --></span>
<span class="nt"><ul</span> <span class="na">class=</span><span class="s">"phones"</span><span class="nt">></span>
<span class="nt"><li</span> <span class="na">ng-repeat=</span><span class="s">"phone in phones | filter:query | orderBy:orderProp"</span><span class="nt">></span>
<span class="nt"><span></span>{{phone.name}}<span class="nt"></span></span>
<span class="nt"><p></span>{{phone.snippet}}<span class="nt"></p></span>
<span class="nt"></li></span>
<span class="nt"></ul></span>
<span class="c"><!-- Ember --></span>
<span class="nt"><div></span>
<span class="nt"><label></span>Name:<span class="nt"></label></span>
{{input type="text" value=name placeholder="Enter your name"}}
<span class="nt"></div></span>
<span class="nt"><div</span> <span class="na">class=</span><span class="s">"text"</span><span class="nt">></span>
<span class="nt"><h1></span>My name is {{name}} and I want to learn Ember!<span class="nt"></h1></span>
<span class="nt"></div></span></code></pre></figure>
<p>Although the syntax varies slightly; theyāre all doing similar things. They give the ability to embed some expression into your HTML that is āmagically processedā by the framework. This means no manual DOM manipulation; you can write nicely formed (ish) HTML inside these expressions and the framework will do all the mundane tasks like creating nodes, wiring up event handlers and updating the DOM when your model changes. This all sounds like a nice convenience, itās not hard to see why everyone is jumping on the bandwagon and developing this way.</p>
<p>However; turns out that itās not all as advertised; there are some significant flaws in this approach that often go unnoticed until youāre trying to do something complicated.</p>
<p>Where I work, weāre currently building some prototypes in many of these new technologies to help guide our decision on our technologies for new versions of our products. It seems that each dev using one of these fancy frameworks is struggling and we spend a lot of time debugging issues or scouring StackOverflow for how to do (what weād consider) basic tasks.</p>
<p>Some of the frustrations includeā¦</p>
<h3 id="no-help-from-your-tools">No help from your tools</h3>
<p>Because these binding expressions arenāt real HTML constructs, youāll get no help from your tooling with them. No highlighting; no code-completion; no tooltips; no static analysis or type-checking. As far as your HTML editor is concerned, these expressions are just strings.</p>
<h3 id="simple-tags-are-never-simple">Simple tags are never simple</h3>
<p>Youāll start off binding properties like {{ name }}, but youāll soon find your expressions growing into complicated messes. Youāll find the built-in functionality doesnāt quite do what you need and youāll start having to bend the framework with <a href="http://stackoverflow.com/a/21492524">crap like this</a>, writing a JavaScript function that returns a function simply to apply a filter with a dynamic name:</p>
<figure class="highlight"><pre><code class="language-html" data-lang="html"><span class="nt"><tr</span>
<span class="na">ng-show=</span><span class="s">"banners"</span>
<span class="na">ng-repeat=</span><span class="s">"banner in banners.data"</span>
<span class="nt">></span>
<span class="nt"><td</span> <span class="na">ng-repeat=</span><span class="s">"col in banner"</span><span class="nt">></span>
{{ col | dynamicFilter : banners.cols[$index].formatter }}
<span class="nt"></td></span>
<span class="nt"></tr></span>
<span class="nt"><script></span>
<span class="nx">myApp</span><span class="p">.</span><span class="nx">filter</span><span class="p">(</span><span class="dl">'</span><span class="s1">dynamicFilter</span><span class="dl">'</span><span class="p">,</span> <span class="kd">function</span> <span class="p">(</span><span class="nx">$filter</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="kd">function</span> <span class="p">(</span><span class="nx">value</span><span class="p">,</span> <span class="nx">filterName</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">Formatters</span><span class="p">[</span><span class="nx">filterName</span><span class="p">])</span>
<span class="k">return</span> <span class="nx">Formatters</span><span class="p">[</span><span class="nx">filterName</span><span class="p">](</span><span class="nx">value</span><span class="p">);</span>
<span class="k">else</span>
<span class="k">return</span> <span class="nx">value</span><span class="p">;</span>
<span class="p">};</span>
<span class="p">});</span>
<span class="nt"></script></span></code></pre></figure>
<h3 id="nothing-ever-works-first-time">Nothing ever works first time</h3>
<p>As your binding expressions get more complicated, youāll find that they rarely work first time. Youāll end up playing the shotgun-debugging game of tweaking your expression and reloading to see if you got it right (this will really suck if you have a long cycle to get back into the correct position for testing; such as having to manually log in to your app). For example, in Knockout, observables are functions, so when you want values you often need to add parens to the property. However sometimes your properties are not observable, so sometimes you donāt.</p>
<figure class="highlight"><pre><code class="language-html" data-lang="html"><span class="c"><!-- Which ones need parens()? --></span>
<span class="nt"><input</span> <span class="na">data-bind=</span><span class="s">"value: filterData.DateCriteria().DateFrom"</span> <span class="na">type=</span><span class="s">"text"</span> <span class="na">id=</span><span class="s">"DateFrom"</span> <span class="nt">/></span></code></pre></figure>
<h3 id="error-messages-rarely-exist-and-are-frequently-useless">Error messages rarely exist and are frequently useless</h3>
<p>When you get your binding expressions wrong; youāll frequently see <em>nothing</em>. If youāre lucky, you might get a JavaScript error. If you do, itāll probably say something like <code class="language-plaintext highlighter-rouge">e is not a function</code> and give you a stack trace thatās a hundred frames deep inside your framework and no part of it points at your HTML binding expression.</p>
<h3 id="debugging-is-near-impossible">Debugging is near impossible</h3>
<p>You canāt put a breakpoint on a {{ stupid binding expression }} because itās not JavaScript (or is it?). Youāll often end up pulling them out into ācomputed propertiesā or some equivalent, but now your JavaScript is filling up with extension functions on objects that make no sense outside of framework-specific view-funkiness. All because your framework has a half-arsed parsing of JavaScript expressions for bindings.</p>
<h2 id="whats-the-alternative">Whatās the alternative?</h2>
<p>Simple. <em>Do your programming in a programming language</em>. Donāt try to embed it in some crazy will-never-be-a-standard binding expression invented by a fly-by-night JavaScript framework.</p>
<p>You probably donāt want to go back to completely manipulating the DOM by hand, but there are options like <a href="http://facebook.github.io/react/">React</a> (without JSX, since JSX sucks for many of the same reasons above) which avoids DOM manipulation (instead favouring the idea of just describing the ācurrent stateā) and keeps all your code as code. This way, your existing tools, linters, coverage etc. will all work fine. Want to extract a function to reuse it? No problem! Want to refactor->rename? No problem (well, ishā¦ as well as this works in JS!). Want to use TypeScript and have type-checking in your expressions? No problem (well, ishā¦ when TypeScript+React play nicer together ;)).</p>
<p>At work, one of our devs is building a prototype being built in React, and Iām also building one in Dart (in a React-inspired, code-generating elements way, while I quietly hope <a href="https://github.com/google/dart-tagtree">Dart TagTree</a> evolves into a usable library). Neither of these prototypes are hitting the same frustrations as those that encourage {{ silly binding expressions }} in HTML.</p>
<p><strong>HTML is a declarative markup language. Letās leave it as such, and stop trying to crowbar non-standard binding expressions into it.</strong></p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2014/08/you-have-ruined-html/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2014-08-17:/2014/08/jawbone-up24-vs-fitbit-flex/2014-08-17T00:00:00+00:002014-08-17T00:00:00+00:00Jawbone Up24 vs Fitbit Flex
<p>I previously blogged about <a href="/2014/06/why-i-returned-my-jawbone-up24-after-just-one-week/">my negative experience</a> with Jawboneās Up24, which resulted in exchanging the <a href="http://prodct.info/up24">Jawbone Up24</a> for a <a href="http://prodct.info/flex">Fitbit Flex</a>. Iāve had the Flex for around 6 weeks now (my wife has had hers for about 4); so I thought it was worth posting a summary of my thoughts on the two devices compared.</p>
<h2 id="jawbone-up24-pros">Jawbone Up24 Pros</h2>
<ul>
<li>āIntelligentā Alarm: You provide a window in which youād like to be woken up, and the Up/Up24 will aim to wake you up when youāre in the lightest sleep during that window (which is intended to make you feel less tired upon waking).</li>
<li>Idle Alert: Set a maximum time for which youād like to be idle; and the Up/Up24 will vibrate to remind you to get up and move.</li>
<li>The mobile app is colourful and well designed.</li>
<li>Food logging in the app is <em>reasonably</em> good.</li>
</ul>
<h2 id="jawbone-up24-cons">Jawbone Up24 Cons</h2>
<ul>
<li>Battery drain is absolutely poor; my Nexus 4 went from lasting 2 entire days to far less than a day. This is ultimately what caused me to return it; Iām not living with having to charge my phone more than once a day.</li>
<li>The mobile app is very slow/sluggish (the Nexus 4 isnāt the fastest Android phone on the market; but itās also <em>far</em> from slowest).</li>
<li>Food logging had some glaring bugs (drinks often logged as āmealsā, no reasonable way to save your own meals and easily reuse them later).</li>
<li>Band is rather uncomfy; itās bulky on the bottom of your wrist (bad if youāre a programmer like me, wrists frequently on the desk) and the size isnāt adjustable for different size wrists, you have to pick the āclosest fitā from the available sizes.</li>
<li>Requires smartphone app; no ability to log or view data from a computer.</li>
<li>More expensive than the Fitbit Flex!</li>
</ul>
<h2 id="fitbit-flex-pros">Fitbit Flex Pros</h2>
<ul>
<li>Band is much comfier; and all devices come with two sizes of adjustable bands, so you donāt need to āpick the closestā at time or purchase.</li>
<li>Flex comes with a USB dongle that allows syncing without a computer.</li>
<li>Decent web app to allow viewing/logging of data directly from your computer.</li>
<li>Food logging prioritises items you create/favourite in the autocomplete list.</li>
<li>Battery lasts around a week (usuallyā¦ see Cons).</li>
<li>Cheaper than the Up24!</li>
</ul>
<h2 id="fitbit-flex-cons">Fitbit Flex Cons</h2>
<ul>
<li>Periodically, the Flex will completely die with no warning/notifications. Battery can be 80%, and then device just goes dead. Itās possible it just ācrashesā, but when charging, it always starts at 1 light. Weāve had 4 Flexes between us, and all 4 have done it (though the most recent two only once or twice in almost 6 weeks).</li>
<li>Doesnāt log steps at all accurately when pushing a pushchair (I read both bands were supposed to be quite good at this; but weāre seeing probably half the steps weād expect).</li>
<li>Sleep logging seems way off for me (Iām a bit of an insomniac). In āstandardā mode, it says I sleep 7-8 hours when I know for a fact I didnāt. In āsensitiveā mode, it logs 2-4 hours each night, in a pattern that doesnāt match the sleep I believe I get. I guess this is just down to the way it has to work (assuming still is asleep, even though when trying to fall asleep, weāre usually still!).</li>
<li>Support is a bit hit-and-miss. Iāve been round in circles with them trying to get Facebook/Twitter weekly summaries working, and I was asked to āre-linkā both accounts 4 times with their shotgun-debugging. I also asked why the āAfter Dinnerā option is missing from Android; and have had 4 responses so far, and Iām still not sure whether it will ever be fixed (they say itās not a bug, but expected behaviour, but theyāve failed to explain how this could be expected).</li>
<li>Failing (and expensive-to-replace) Bands: Although we havenāt had any issues ourselves in the 6 weeks, Iāve seen lots of complaints of bands breaking. This wouldnāt be such an annoyance if the bands werenāt so pricey. Though Iāve friends that have bought <a href="http://prodct.info/flex-bands">cheap replacement Flex bands</a> and been happy with them.</li>
</ul>
<h2 id="conclusion">Conclusion</h2>
<p>So, in summary; I would definitely recommend the <a href="http://prodct.info/flex">Fitbit Flex</a>. I would definitely <em>not</em> recommend the <a href="http://prodct.info/up24">Jawbone Up24</a>.</p>
<p>That said; given the success of these sorts of bands, I think weāll see an increase in the number of options for fitness bands over the coming 12 months, including better features (such as heart rate monitoring, GPS tracking of activities). With Apple and Google both building platforms for these devices to sit on top of (iOS Health app, Google Fit) I think weāll also end up with a more consistent view of our data, regardless of what band we choose.</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2014/08/jawbone-up24-vs-fitbit-flex/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2014-06-29:/2014/06/why-i-returned-my-jawbone-up24-after-just-one-week/2014-06-29T00:00:00+00:002014-06-29T00:00:00+00:00Why I returned my Jawbone Up24 after just one week
<p>Last week I spent some time in and out of A&E with chest pains. Itās still not completely clear what the cause was; but the doctors seem to believe itās not a problem with my heart. However, several ECGās have shown an (intermittent!) unusual heart rhythm, so Iām currently awaiting for a 24hr ECG to understand whatās going on (maybe my heart rhythm is timed with IEEE floating point?).</p>
<p>Whether this issue turns out to be related to my heart or not; itās been a bit of a wake up call. I eat a lot of junk food, do no real exercise, sit at a computer all day every day, and barely eat 5 fruit&veg per month! I decided itās time to start being a bit more health-conscious, and I hoped an activity tracker might motivate me a little more to get active.</p>
<p>I did some comparing of the <a href="http://prodct.info/flex">Fitbit Flex</a> and the <a href="http://prodct.info/up24">Jawbone Up24</a> and decided to buy the Up24 for two features the Fitbit Flex didnāt have:</p>
<ol>
<li>āIntelligentā Alarm: Wakes you up within a defined window when youāre in a light sleep. Apparently being woken up from a deep sleep makes you feel worse.</li>
<li>Idle Alert: The wristband vibrates if you donāt move for a defined amount of time (eg. an hour). Since itās easy to sit for a long time while programming, a reminder to get up and move seems like a good idea.</li>
</ol>
<p>I had the Up24 for almost a week, and I was generally impressed. The app was relatively polished (if a bit sluggish), and the push notifications at various intervals were quite handy. The food tracking worked fairly well, but had some annoying quirks (especially around managing your own ālibraryā). However, there were two fairly serious flaws that ultimately resulted in me returning the device today.</p>
<ol>
<li>The constant syncing of the band drains my Nexus 4 battery quickly. I used to get 2 days on a charge; but with the Up24 I was lucky to get 24 hours. Almost every day I woke up to a completely dead-and-turned-off phone, even sometimes if Iād charged it the night before.</li>
<li>Every time the band synced with my phone, it would disconnected all other Bluetooth devices. This means almost every phone call to my wife on the way home from work in my car got disconnected (the call stays active, in my pocket, until the sync finishes).</li>
</ol>
<p>Despite not selling them in-store, Currys told me this was the third one theyād had back recently; so Iām guessing others havenāt been happy too.</p>
<p>I donāt know whether either of these issues are Jawboneās fault; it may turn out that the <a href="http://prodct.info/flex">Fitbit Flex</a> does the same (weāll find out in the next few days!), though I did notice that on the Fitbit website it says that all-day-sync is disabled on my device by default due to battery drain issues. The difference here, is that the sync can be disabled. The <a href="http://prodct.info/up24">Jawbone Up24</a> doesnāt have any ability to disable the sync (or reduce the frequency).</p>
<p>I do hope that these issues get ironed out (with software updates, or future versions of the bands/phones) and that Fitbit add the missing features I liked about the Up24. I think the idea of having a small piece of technology with us at all times to help us stay fit and healthy is a great one; and I think thereās huge scope for improvement in this area.</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2014/06/why-i-returned-my-jawbone-up24-after-just-one-week/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2014-06-03:/2014/06/dartvs-dart-support-for-visual-studio/2014-06-03T00:00:00+00:002014-06-03T00:00:00+00:00DartVS - Dart Support for Visual Studio
<blockquote>
<p><strong>Note:</strong> DartVS is not being maintained, however <a href="https://marketplace.visualstudio.com/items?itemName=Dart-Code.dart-code">Dart Code, my Dart extension for Visual Studio Code</a> is and is much more complete.</p>
</blockquote>
<p>Yesterday I published my <em>fourth</em> Visual Studio extension! :)</p>
<p>Recently Iāve been playing around with Dart, in the hope it might one day be a viable alternative to JavaScript. Something that always put me off trying it out in the past was a lack of Visual Studio support (I donāt want to use another IDE, Iām already using VS for ASP.NET, etc.). Since it doesnāt look like Google nor Microsoft will ever add this, I decided to give it a shot myself - I mean, how hard can it be? ;-)</p>
<p>Showing errors/warnings/hints in the Error List works, as does syntax highlighting (mostly). Next up is intellisense; but thatās likely to take a little longer as itāll involve actually parsing some code!</p>
<p>If youāre using Dart and have Visual Studio; please <a href="http://visualstudiogallery.msdn.microsoft.com/69112f14-62d0-40fb-9ccc-03e3534e7121">try it out</a> and let me know how you get on!</p>
<h2 id="implemented">Implemented</h2>
<ul>
<li>Use DartAnalyzer from the SDK when saving any .dart file and report errors/warnings/hints to the Visual Studio error list</li>
<li>Clicking errors navigates to the correct place in code</li>
<li>Syntax Highlighting</li>
</ul>
<h2 id="not-implemented">Not Implemented</h2>
<ul>
<li>Use Batch Mode of DartAnalyzer for better performance</li>
<li>Intellisense</li>
<li>Debugging</li>
</ul>
<h2 id="installation">Installation</h2>
<ul>
<li>Install from the <a href="http://visualstudiogallery.msdn.microsoft.com/69112f14-62d0-40fb-9ccc-03e3534e7121">Visual Studio Gallery</a></li>
<li>Download and unzip the <a href="https://www.dartlang.org/tools/sdk/">Dart SDK</a></li>
<li>Set the DART_SDK environment variable to point at the SDK root</li>
</ul>
<h1 id="feedback">Feedback</h1>
<p>Please send your feedback/issues/feature requests! :-)</p>
<ul>
<li>GitHub Issues: <a href="https://github.com/DanTup/DartVS/issues">DartVS/issues</a></li>
<li>Twitter: <a href="https://twitter.com/DanTup">@DanTup</a></li>
<li>Email: <a href="mailto:danny+dartvs@tuppeny.com">danny+dartvs@tuppeny.com</a></li>
</ul>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2014/06/dartvs-dart-support-for-visual-studio/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2014-05-11:/2014/05/web-development-sucks-and-its-not-getting-any-better/2014-05-11T00:00:00+00:002014-05-11T00:00:00+00:00(Web) Development Sucks, and It's Not Getting Any Better
<p>Thatās quite a claim! Iām sure many will disagree with me, but since this is my blog youāre going to get my rant! :)</p>
<h2 id="the-problem">The Problem</h2>
<p>Over the past few months, Iāve been thinking more and more about the general frustrations around web programming in 2014. For example:</p>
<ul>
<li>There are more programming languages than there have ever been; yet it feels like there is no choice</li>
<li>We have more experience writing code than we ever have; yet we still struggle to express our requirements in code</li>
<li>We have higher level languages than weāve ever had before; yet it feels like weāre writing more mundane boiler-plate plumbing</li>
<li>We have better browser support for standards; yet itās more work than ever to support all of our users</li>
</ul>
<p>It feels to me that as an industry, weāre treading water. Every week thereās something new and shiny, but little that progresses the significant things that could make an enormous difference to our jobs.</p>
<h3 id="no-choice-javascript-is-a-monopoly">No Choice; JavaScript is a Monopoly</h3>
<p>Itās no secret that Iām not a big fan of JavaScript. Itās a hacked together mess with silly quirks and no typing. But the problem here is not my opinion of JavaScript, but one of choiceā¦</p>
<p>On the server, I can write my code in pretty much whatever programming language I want: C#, VB.NET, VBScript, Python, Perl, Erlang, Haskell, Clojure, Go, Cā¦ and the list goes on. Most of those even work across <em>different operating systems</em>. This means I can pick the language/runtime that will give me the best productivity/stability/maintainability based on the task Iām doing. If I want quick prototyping, I can use something dynamic. If Iām writing something complex that is critical is correct, I can use something more formal.</p>
<p>In the browser, Iām restricted to one language: JavaScript. This feels somewhat ironic given the open nature of the web! Thereās no choice here. If Iām writing a huge app with millions of lines of code and I want something that can be safely type-checked and refactored, itās tough. Your option is JavaScript. If you donāt like it, you can limit the client-side code you write, but that will be at the expense of user experience.</p>
<p>Want to share code between client and server? Now youāre even more screwed, because youāre writing JavaScript on the server too ;)</p>
<p>āAH, BUTā¦!ā I hear you shout. And youāre right, there is another way! ā¦</p>
<h3 id="tools-that-convert-to-javascript-are-a-hack">Tools That Convert to JavaScript are a HACK</h3>
<p>Every month thereās a new tool that will let you write something that isnāt JavaScript and convert it to JavaScript. These range from simple ideas like TypeScript and new languages like Dart to huge programs to convert .NET and even native code. Well, theyāre all a hack. Hack hack hackety hack. These tools are not being built because they are a good idea, theyāre being built because weāre forced into using JavaScript and most people donāt like it. As a result, theyāre usually full of edge cases, varying browser support, inconsistent performance and other strangeness.</p>
<h3 id="we-cannot-express-our-requirements-in-code">We Cannot Express our Requirements in Code</h3>
<p>Even if you throw away all this client side nonsense for a minute, most mainstream programming languages still fail to allow us to express really common rules at compile time and force us to throw errors at runtime. There are lots of ideas out there about how we can fix it, but whereās the progress? Why is Microsoftās Code Contracts still an unstable research project? Why hasnāt it had an update for 8 months? Why donāt more languages have dependent types? Why are there not more static analysis tools of this type? Why are even new programming languages being designed today with nullable references you canāt opt-out of?</p>
<h3 id="our-high-level-languages-are-not-high-level-enough">Our High-Level Languages are not High-Level Enough</h3>
<p>With most languages today, it still feels like Iām writing a bunch of plumbing code that I shouldnāt need to. I can easily write a few hundred lines of code across several classes to do something, yet describe the functionality in just a few short sentences. Why do I still need so much third party code to perform basic functions? For example, look at <a href="https://www.npmjs.org/package/exit">this nodejs module, exit</a>. All it does is exit a node application in the way one would expect, after flushing standard streams like STDOUT. Itās a 41 line node module. A THIRD PARTY MODULE AND FORTY-ONE LINES TO EXIT CLEANLY!!</p>
<p>How can we ever talk about declarative programming when we canāt even abstract the basics away?</p>
<h3 id="browserdevice-support-is-harder-than-ever">Browser/Device Support is Harder than Ever</h3>
<p>Browsers implement standards better than they ever have; yet making your app work in all of your users browsers is more difficult today than ever before. Why? There are more browser rendering engines than there have ever been. Even within a single browser rendering engine, there are more versions out there today than there have ever been. IE having huge market share might have had its own issues, but was it really worse than battling with differences between IE7, IE8, IE9, IE10, IE11, IE Metro (likely multiple versions), Chrome, Firefox, Safari, Opera, Chrome for Android, Samsungās Browser, etc.? And now letās add in screen size and pixel density! People are trying to write media queries that use size to adjust to the screen, but itās futile when small devices have hi-res screens and there are no clear boundaries. We need to base things on physical size, not pixel density, yet support for detecting this is poor. These things have become so complicated that services exist just to let you see/execute your apps across all of these devices because nobody can realistically have physical access to all of these things!</p>
<h2 id="how-can-we-improve-things">How can we Improve Things?</h2>
<p>I think it all comes down to a few things:</p>
<ul>
<li>I should be able to pick my programming language</li>
<li>My programming language should be flexible enough for me to express things I know up-front and get compiler assistance instead of runtime errors</li>
<li>I should not have to write much code that is not <em>specific</em> to my application/domain</li>
<li>My code should give the exact same results everywhere</li>
</ul>
<p>Stating requirements is easy; pulling them off less so! Note: Iām just an average developer! Iāve no experience with anything as complicated as what Iām proposing here. Donāt interpret this as me suggesting this is easy!</p>
<p>Hereās what I think we should be investing time and effort into:</p>
<ul>
<li>A new intermediate format/VM for running code in the browser to allow multiple languages
<ul>
<li>It doesnāt matter what this is based on; JVM, MSIL, NaCl, something else, as long as itās something up to the job (with solid sensible behaviour, <a href="https://www.destroyallsoftware.com/talks/wat">not JavaScript</a>)</li>
<li>It must support being targeted by at least a handful of mainstream languages (this will ensure suitability for new languages too)</li>
<li>Both Microsoft and Google agree to implement it (Iām certain other browser vendors would follow if both of these did, but I doubt that Google or MS would follow others)</li>
</ul>
</li>
<li>All browsers auto-update by default
<ul>
<li>No off-by-default, even for enterprise</li>
<li>Browser vendors drop support for old versions quickly</li>
</ul>
</li>
<li>Good static analysis tools for mainstream languages to allow us to move more runtime crashes to compile errors
<ul>
<li>Microsoft should get Code Contracts or something similar back on the roadmap</li>
<li>If we had a good intermediate format (as mentioned above), these tools may be able to be generic and work for multiple languages</li>
</ul>
</li>
<li>New programming languages; Itād be very hard to try and retro-fit some of the things we need to existing languages
<ul>
<li>Designed to be runnable both on the server and within the browser (see above), including compile-time enforcement of API sets (eg. no access to file system in client libraries)</li>
<li>No nulls!</li>
<li>Contracts, Dependent Types or some other way to express our requirements better to cut down on runtime errors</li>
<li>Strong typing</li>
<li>Ability to enforce purity and exhaustiveness/totality on functions</li>
</ul>
</li>
</ul>
<p>You may argue some of these are not <em>necessary</em>, but I think the end goal needs to include all of these, even if the path to get there is a progressive one.</p>
<h3 id="the-chances-of-things-getting-better-are-slim">The Chances of Things Getting Better are Slim</h3>
<p>Thatās a pretty pessimistic view, but sadly I think itās where we are. The problem is; the entire āplanā above pretty much hinges on support and collaboration between two of the biggest software companies in the world: Microsoft and Google. Both of these companies have a serious chunk of the browser market and itās a near-impossible task to ever challenge that. With their rocky relationship I donāt see a world where either company will add the others technology to their browser (eg. IE getting NaCl or even Dart support), nor anyone coming up with a standard intermediate format they would both agree to support. Browser plugins are not a realistic option even today and their support is only likely to shrink in the future.</p>
<h2 id="conclusion">Conclusion</h2>
<p>If youāre writing code on the server, youāve got it great. You can pick and choose your language and over time, youāll get more of the programming language features you need, or new languages to provide them.</p>
<p>If youāre writing code for the browser, then unless you love JavaScript, your life is going to continue to be frustrating for many many years to come!</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2014/05/web-development-sucks-and-its-not-getting-any-better/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2014-04-19:/2014/04/is-fsharp-ready-for-production/2014-04-19T00:00:00+00:002014-04-19T00:00:00+00:00Is F# Ready for Production?
<p>It took me some time to get my head around F#. I had no experience with functional programming and failed twice to pick it up. I persevered; and third time lucky, it started to make sense!</p>
<p>I became a huge fan! Being part of the .NET Framework made it feel fairly familiar and more likely we could start to introduce it at work (weāre primarily .NET devs). Options instead of nulls; immutability, pattern matching, DUs, loads of stuff that looked like it would solve problems we have every day (especially the nulls - we have a > 200k LoC inherited legacy codebase that started life in .NET 1.1 to fight with).</p>
<p>I gave a talk to colleagues who were (inexplicably!) unconvinced, so I gave another one a little later and soon after one of my colleagues arranged working through <a href="https://github.com/ChrisMarinos/FSharpKoans">F# Koans</a> as a team; and the wheels seemed to be moving.</p>
<p>Since weāre busy as ever; F# sort of fell to one side. We were all busy working meeting deadlines; and as a team we werenāt confident enough in F# to pull it into production. The team were even less convinced about F# after I declared PowerShell to be an annoying crock of <a href="http://connect.microsoft.com/PowerShell/feedback/details/778371/invoke-webrequest-getelementsbytagname-is-incredibly-slow-on-some-machines">buggy</a> <a href="http://connect.microsoft.com/PowerShell/feedback/details/776801/weekly-tasks-created-via-powershell-using-a-different-user-immediately-fail-with-error-0x41306">junk</a> that gets <a href="https://twitter.com/jsnover/status/419214625691795456">no responses to bug reports</a>, when Iād been so enthusiastic about it months earlier. They expected Iād be announcing F# to be a big failure soon too (oh, the irony of this blog post!) :-/</p>
<p>Fast forward a few months; and Iām charging along again. I got the latest F# up and running on the work build server. Production are up to .NET 4.5.1. I set up an internal NuGet feed and extracted some simple libraries from our main products solution into NuGet packages for easy replacement. Everything is ready to start getting small F# libraries into the main product, allowing the team to get more familiar with writing and reviewing F#! :)</p>
<h2 id="so-whats-the-problem-get-to-it-already">So, whatās the problem? Get to it already!</h2>
<p>Well; once we take the dive; itās hard to go back. This means Iāve been thinking more about it this week than previously; and Iām now starting to wonder if F# is really ready. There are a few little things that quite bug meā¦</p>
<ol>
<li>IDE support even in Visual Studio 2013 is absolutely poor. Basic things like document formatting and refactoring are missing. The community have worked hard on improving this somewhat; but I hit <a href="https://github.com/fsprojects/VisualFSharpPowerTools/issues/352">serious bugs</a> (<em>edit: this was fixed within 2 days; kudos!</em>) in their extension within minutes. We spend a <em>lot</em> of time in our IDEs; we need first class support; and F# is miles off.</li>
<li>Options are great; nulls are bad. But because F# runs on the CLR; nulls are not gone. This means we now just have two different types of null; the bad one, and the good one. Iām on the fence about whether this is really an improvement. Even if you have the luxury of being F# for <em>all</em> of your own code; the BCL is riddled with null returning/expecting methods.</li>
<li>F# was completely missed from Roslyn. I understand the reasons for this; but with this new āopenā Microsoft; thereās still no word on whether this will happen. If it doesnāt; it means F# will miss out on a new generation of tools being built on Roslyn (for example, Code Diagnostics/Fix support).</li>
<li>Thereās little info on whatās coming in the next version of F#; yet we know quite a lot of language features coming in the new versions of VB and C#. It feels like C# and VB are charging forwards at a much faster rate than F#. Do we want to get off the fast train and board the slow train?</li>
<li>Hiring developers is already very hard. Do we want to limit the pool of available candidates significantly (and take the increased salaries that come with the harder-to-find skills)?</li>
</ol>
<p>Scott Hanselman was right; <a href="https://twitter.com/shanselman/status/457444398268030977">programming is hard</a>, but apparently not just the writing of code!</p>
<p>Iām sure use of F# will continue to grow at our company. Both myself and the other team leader are both big fans. However I think the journey will be <em>much</em> slower and more cautious than Iād originally hoped.</p>
<p>Donāt take this article as a bashing of F#; Iām still a huge fan (both of F# and general FP ideas), but I think it may be some time before itās the <em>obvious</em> choice to make.</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2014/04/is-fsharp-ready-for-production/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2014-04-15:/2014/04/free-software-and-services-for-open-source-projects/2014-04-15T00:00:00+00:002014-04-15T00:00:00+00:00Free Software and Services for Open Source Projects
<p>Yesterday <a href="/2014/04/my-favourite-open-source-software/">I blogged a list of some of my favourite open source projects</a>. I thought it was also worth blogging some of my favourite services/software with free offerings for open source projects that make it much easier or cheaper for open source projects to exist, collaborate and build great software.</p>
<p>Iāve tried to limit the list to things I have first-hand experience of or seem popular amongst people I follow on social networks; but thereās a chance there are omissions. <a href="https://twitter.com/dantup">Ping me on Twitter</a> if you think thereās something I should really add.</p>
<h2 id="source-code-hosting">Source Code Hosting</h2>
<ul>
<li><a href="https://github.com/">GitHub</a> - Git source code hosting; issue tracker and web page hosting. By far the most popular/active for open source projects right now.</li>
<li><a href="https://www.codeplex.com/">Codeplex</a> - Git, TFS and Mercurial source code hosting, issue tracker, wiki. Needs a bit of love, to be honest!</li>
<li><a href="https://bitbucket.org/">BitBucket</a> - Git and Mercurial source code hosting. BitBucket has free private repos too (so isnāt just free for open source). I used to use this quite a lot until GitHub got so popular that it was hard to ignore!</li>
<li><a href="https://code.google.com/">Google Code</a> - Git, Mercurial and Subversion. Doesnāt seem to be much here these days; mostly just dead projects. Even lots of Googleās own OSS is hosted on GitHub!</li>
<li><a href="https://www.fogcreek.com/kiln/pricing/">Kiln</a>/<a href="https://www.fogcreek.com/fogbugz/pricing/ondemand.html">FogBugz</a> - Whilst this isnāt strictly for open source; Kiln and FogBugz are free for two users and supports ācommunity usersā for public access. I couldnāt not include this; itās my favourite DVCS hosting and includes a great Code Review system.</li>
</ul>
<h2 id="web-app-hosting">Web App Hosting</h2>
<ul>
<li><a href="http://azure.microsoft.com/en-gb/develop/net/aspnet/">Azure</a> - Host 10 web apps for free; though youāll need to pay for custom domains.</li>
<li><a href="https://appharbor.com/page/community-application-program">AppHarbor</a> - A .NET PaaS (Platform as a Service) provider. When first launched, this service made Azure look incredibly clunky and complicated for simple web apps, and used to host the website for <a href="http://gplusnotifier.dantup.com/">G+ Notifier</a> for free under their OSS program.</li>
</ul>
<h2 id="hosted-continuous-integration--deployment">Hosted Continuous Integration / Deployment</h2>
<ul>
<li><a href="https://travis-ci.com/">Travis CI</a> - This seems to be getting more and more popular for GitHub projects, possibly down to really nice integration that allows running tests and reporting pass/fail status in pull requests <em>automatically</em>!</li>
<li><a href="https://buildhive.cloudbees.com/">BuildHive</a> - Powered by Jenkins, also with easy integration with GitHub.</li>
<li><a href="http://codebetter.com/codebetter-ci/">CodeBetter CI</a> - Powered by Team City.</li>
<li><a href="https://www.atlassian.com/opensource/overview">Bamboo</a> - From Atlassian; also available for download.</li>
</ul>
<h2 id="testing">Testing</h2>
<ul>
<li><a href="https://saucelabs.com/opensauce">Sauce Labs</a> - Run your web and mobile app tests across hundreds of real browsers and platforms.</li>
<li><a href="http://www.browserstack.com/javascript-testing-api">BrowserStack JavaScript Testing</a> - An HTTP-based API that can be used to open any URL in any combination of browser/OS on BrowserStack to execute JavaScript tests.</li>
</ul>
<h2 id="other-development-software--web-apps">Other Development Software / Web Apps</h2>
<ul>
<li><a href="http://www.jetbrains.com/teamcity/buy/choose_edition.jsp?license=OPEN_SOURCE">TeamCity</a> - Continuous integration server (also available hosted as CodeBetter CI).</li>
<li><a href="http://reflectorblog.red-gate.com/2013/07/open-source/">Reflector / ANTS Profilers</a> - Code and memory profilers and decompiler for .NET.</li>
<li><a href="http://community.moqups.com/knowledgebase/articles/188147-faq">Moqups</a> - Online tool for mocking up UIs quickly.</li>
<li><a href="https://balsamiq.com/free/">myBalsamiq Mockups</a> - Online tool for mocking up UIs quickly.</li>
<li><a href="https://www.atlassian.com/opensource/overview">JIRA / Bamboo / Confluence</a> - Issue tracker, continious integration and team collaboration software. Also available hosted.</li>
<li><a href="https://rhodecode.com/pricing">RhodeCode</a> - A locally-installed DVCS and code review system.</li>
</ul>
<h2 id="misc">Misc</h2>
<ul>
<li><a href="http://uk.godaddy.com/ssl/ssl-open-source.aspx">GoDaddy SSL Certificates</a> - Iām not sure thereās much for love for these guys, but they are claiming to give free SSL certs for OSS.</li>
<li><a href="https://crowdin.net/page/open-source-project-setup-request">Crowdin</a> - Localisation platform to help manage your translations.</li>
<li><a href="https://www.bugsense.com/">BugSense</a> - Analytics software for catching and collating errors in mobile apps.</li>
</ul>
<p>After posting this; I was tweeted a link to <a href="http://ossperks.com/">ossperks.com</a>, a similar (but more-complete) list of free stuff for Open Source projects. If youāre running an OSS project, Irecommend checking it out!</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2014/04/free-software-and-services-for-open-source-projects/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2014-04-14:/2014/04/my-favourite-open-source-software/2014-04-14T00:00:00+00:002014-04-14T00:00:00+00:00My Favourite Open Source Software
<p>(You may also be interested in my post <a href="/2014/04/free-software-and-services-for-open-source-projects/">Free Software and Services for Open Source Projects</a>.)</p>
<p>In a <a href="http://www.hanselman.com/blog/OpenSourceIsAThanklessJobWeDoItAnyway.aspx">recent blog post</a>, Scott Hanselman suggested:</p>
<blockquote>
<p>We donāt sing contributorās praises for their hard work and success while their software works, instead we wait until a single line (albeit one of the more important lines) fails to live up to expectations.</p>
</blockquote>
<p>I donāt completely agree with this; I often see people praising open source software (and I didnāt see anyone really pointing fingers over Heartbleed), but I do think the general point that open source software is taken for granted and under-appreciated is valid.</p>
<p>This got me thinking about some of the open source software I use; some as part of my job delivering commercial applications and some at home when Iām just playing around. I thought itād be good to blog a list of my favourites; maybe itāll help others discover them. Much of it is pretty well known (some even from MS/Google), but I thought Iād include all my favourite stuff for completeness.</p>
<p>I may have missed some; but thereās a lot of great stuff out there. If I remember anything cool Ive missed, Iāll try and add it in.</p>
<p><strong>If you have a blog (or even Google+ ;-)) I invite you to do the same!</strong></p>
<p>(In alphabetical order; with no grouping, and often their own marketing speak!)</p>
<ul>
<li><a href="http://www.7-zip.org/">7-Zip</a> - A file archiver with a high compression ratio. Thereās a portable version I use to avoid having extra stuff installed, but still allowing the higher compression of .7z when required.</li>
<li><a href="http://angularjs.org/">AngularJS</a> - A toolset for building the framework most suited to your application development. Iām currently playing around with this prototyping some ideas for a possible vNext of a big project at work; I think the future of web apps is (sadly ;)) in client-side JavaScript with API backends.</li>
<li><a href="http://getbootstrap.com/">Bootstrap</a> - A popular front-end framework for developing responsive, mobile first projects on the web. This is great for getting something looking decent very quickly, especially for non-designers (programmers!).</li>
<li><a href="http://harvesthq.github.io/chosen/">Chosen</a> - A jQuery plugin that makes long, unwieldy select boxes much more user-friendly. These look really nice; especially compared against the default selects in some of the latest browser versions!</li>
<li><a href="https://code.google.com/p/elmah/">ELMAH</a> - An application-wide error logging facility.</li>
<li><a href="http://fsharp.org/">F#</a> - A functional-first programming language that interops relatively nicely with .NET. This could be the key to getting current-.NET devs interested more in functional ideas.</li>
<li><a href="https://github.com/fsharp/FAKE">FAKE</a> - A cross platform build automation system (F# Make). Havenāt used this seriously yet; but looking for someone to replace all the built steps on our build server with a single script that can be run on dev machines to give a like-for-like build.</li>
<li><a href="http://getglimpse.com/">Glimpse</a> - Inspects web requests as they happen, providing insights and tooling that reduce debugging time.</li>
<li><a href="http://jasmine.github.io/">Jasmine</a> - A nice behavior-driven development framework for testing JavaScript code.</li>
<li><a href="http://jekyllrb.com/">Jekyll</a> - A blog-aware, static-site generator (powering this blog for free, via GitHub Pages!)</li>
<li><a href="http://karma-runner.github.io/">Karma</a> - JavaScript test runner that executes tests in multiple browsers and gives instant feedback (now with <a href="/2014/03/cross-browser-javascript-testing-with-karma-and-visual-studio/">Visual Studio integration ;)</a>).</li>
<li><a href="http://knockoutjs.com/">Knockout</a> - A JavaScript library that helps create rich, responsive display and editor user interfaces using two-way data-binding. This helps de-couple JavaScript from the DOM, making things <em>much</em> simpler!</li>
<li><a href="http://mercurial.selenic.com/">Mercurial</a> - A distributed source control management tool. This is <em>much</em> easier IMO to use than Git; though sadly I donāt think I can convicne the masses ;)</li>
<li><a href="http://nodejs.org/">Node.js</a> - A platform built on Chromeās V8 JavaScript runtime for building fast, scalable network applications. Great package manager, and really useful for non-browser based JS execution.</li>
<li><a href="http://www.nuget.org/">NuGet</a> - A package manager for the Microsoft development platform including .NET. This is basically the first place I look for <em>any</em> third party library these days.</li>
<li><a href="https://rx.codeplex.com/">Reactive Extensions</a> - A library for composing async/event-based programs using observable sequences and LINQ-style query operators.</li>
<li><a href="http://remarkjs.com">remark</a> - A simple, in-browser, Markdown-driven slideshow tool targeted at people who know their way around HTML and CSS (<a href="/2014/04/markdown-based-presentations-with-remark/">I blogged more about this yesterday</a>).</li>
<li><a href="http://docs.seleniumhq.org/">Selenium</a> - Automates browsers to aid testing of web applications. If weāre going to start writing more client-side JavaScript for our web apps; we <em>need</em> to get better at testing them!</li>
<li><a href="https://github.com/SignalR/SignalR">SignalR</a> - A library for ASP.NET developers that makes it incredibly simple to add real-time web functionality to applications.</li>
<li><a href="http://docs.structuremap.net/">StructureMap</a> - A dependency injection / inversion of control tool for .NET.</li>
<li><a href="https://tickspec.codeplex.com/">TickSpec</a> - A lightweight F# BDD framework.</li>
<li><a href="http://www.typescriptlang.org/">TypeScript</a> - A typed superset of JavaScript that compiles to plain JavaScript.</li>
<li><a href="https://aspnetwebstack.codeplex.com/">Web API</a> - A framework that makes it easy to build HTTP services that reach a broad range of clients, including browsers and mobile devices.</li>
<li><a href="https://github.com/xunit/xunit">xUnit</a> - A community-focused unit testing tool for the .NET Framework.</li>
</ul>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2014/04/my-favourite-open-source-software/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2014-04-13:/2014/04/markdown-based-presentations-with-remark/2014-04-13T00:00:00+00:002014-04-13T00:00:00+00:00Markdown-based Presentations with Remark
<p>After giving a talk at work to colleagues recently using slides Iād created in Google Docs; I thought Iād had a great idea! What if I could edit my slides as text (something like Markdown) and bypass the nasty WYSIWYG editor?!</p>
<p>Well; as is usually the case when you have a great idea; itās already been done. I played around with a few I found online and one that was <em>really</em> neat. Not only does it support markdown; but it also has extensions so you can add CSS classes to content to lay things out just like a web developer is used to! As itās web-based, you can even put your own JavaScript in to mess with things further!</p>
<p>The project is called <a href="http://remarkjs.com/">remark</a>, you can see a slideshow created using their script on <a href="http://remarkjs.com/">their website</a>. The source text can be <a href="https://github.com/gnab/remark/blob/gh-pages/index.html#L123">seen here</a>.</p>
<p>In addition to being able to knock together a slideshow in text; there are some other cool features (you can try these out on <a href="http://remarkjs.com/">their website slideshow</a>):</p>
<ul>
<li>Press ācā to clone the window, so you can show a separate copy on an external screen (the clones are kept in sync with an API)</li>
<li>Press āpā to open presenter mode for the active window, which shows current/next slide and speaker notes (and unlike Google Docs; the slide preview is <em>not tiny</em>!)</li>
<li>Press āfā to make the slideshow full screen (you can set the aspect ratio in the code)</li>
</ul>
<p>I often use other peoples software; but I donāt have enough time to contribute to many. However, I liked this one so much, I had a stab at fixing a niggle I found; and itās <a href="https://github.com/gnab/remark/pull/106">now been merged into the main repo</a> :)</p>
<h1 id="slide">Slide</h1>
<p><img src="/post_images/remark_slide_thumb.png" alt="remark slide" /></p>
<h1 id="presenter-mode">Presenter Mode</h1>
<p><img src="/post_images/remark_presenter_thumb.png" alt="remark presenter mode" /></p>
<p>So, it all looks pretty slick. One thing Iām certain of; if this hadnāt existed and Iād started my own; it wouldnāt have looked this neat! If youāre a web developer; give it a go. Iām certain youāll prefer it to PowerPoint and Google Docs! :)</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2014/04/markdown-based-presentations-with-remark/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2014-03-09:/2014/03/cross-browser-javascript-testing-with-karma-and-visual-studio/2014-03-09T00:00:00+00:002014-03-09T00:00:00+00:00Cross-browser JavaScript Testing with Karma and Visual Studio
<p><a href="https://github.com/DanTup/TestAdapters">
<img style="position: absolute; top: 0; right: 0; border: 0;" src="https://camo.githubusercontent.com/a6677b08c955af8400f44c6298f40e7d19cc5b2d/68747470733a2f2f73332e616d617a6f6e6177732e636f6d2f6769746875622f726962626f6e732f666f726b6d655f72696768745f677261795f3664366436642e706e67" alt="Fork me on GitHub" data-canonical-src="https://s3.amazonaws.com/github/ribbons/forkme_right_gray_6d6d6d.png" />
</a></p>
<p>Yesterday I published my <em>third</em> Visual Studio extension! :)</p>
<p>Itās a fairly generic extension, with reads static XML files that contain test results into the Visual Studio Test Explorer window. While this might sound a bit strange, its goal is to integrate with third party tools that can output results as XML and update them realtime. For example, Googleās Karma test runner.</p>
<p>Hereās a video showing it all in action:</p>
<iframe width="450" height="253" src="https://www.youtube.com/embed/-28j6Ek8D7w?VQ=HD720" frameborder="0" allowfullscreen=""></iframe>
<h1 id="installation-and-setup">Installation and Setup</h1>
<ol>
<li>Install my Karma Test Adapter from <a href="http://visualstudiogallery.msdn.microsoft.com/bfe6feb7-7ec4-4e8e-9d90-cf6ea2cd2169">Visual Studio Gallery</a></li>
<li>Install/download nodejs (I grabbed the exe <a href="http://nodejs.org/dist/">from here</a> and added it to my PATH)</li>
<li>Install/download npm (I grabbed the zip <a href="http://nodejs.org/dist/npm/">from here</a> and put it in the same folder as node.exe)</li>
<li>Install the karma-cli node module globally
<ul>
<li><code class="language-plaintext highlighter-rouge">npm install -g karma-cli</code></li>
</ul>
</li>
</ol>
<h1 id="usage">Usage</h1>
<ol>
<li>Create/navigate to the folder that contains/will contain your tests</li>
<li>Create a package.json to hold your node dependencies
<ul>
<li>PowerShell: <code class="language-plaintext highlighter-rouge">"{}" | Out-File -Encoding ascii package.json</code></li>
<li>or CMD: <code class="language-plaintext highlighter-rouge">echo '{}' > package.json</code></li>
</ul>
</li>
<li>Install karma and some dependencies
<ol>
<li><code class="language-plaintext highlighter-rouge">npm install karma --save-dev</code></li>
<li><code class="language-plaintext highlighter-rouge">npm install karma-xml-reporter --save-dev</code></li>
<li><code class="language-plaintext highlighter-rouge">npm install karma-jasmine --save-dev</code></li>
<li><code class="language-plaintext highlighter-rouge">npm install karma-phantomjs-launcher --save-dev</code> <em>(required only if you want to use PhantomJS)</em></li>
<li><code class="language-plaintext highlighter-rouge">npm install karma-ie-launcher --save-dev</code> <em>(required only if you want Karma to be able to launch IE)</em></li>
<li><code class="language-plaintext highlighter-rouge">npm install karma-chrome-launcher --save-dev</code> <em>(required only if you want Karma to be able to launch Chrome)</em></li>
<li><code class="language-plaintext highlighter-rouge">npm install karma-firefox-launcher --save-dev</code> <em>(required only if you want Karma to be able to launch Firefox)</em></li>
</ol>
</li>
<li>Create a karma config file in the folder with your tests
<ol>
<li><code class="language-plaintext highlighter-rouge">karma init</code></li>
<li><em>answer the questions, for example:</em>
<ol>
<li>Test framework: jasmine</li>
<li>RequireJS: no</li>
<li>Browsers:
<ul>
<li>Chrome</li>
<li>IE</li>
<li>PhantomJS</li>
</ul>
</li>
<li>JavaScript and Test files:
<ul>
<li>app\**.js</li>
<li>tests\**.js</li>
</ul>
</li>
<li>Exclusions: <em>(none)</em></li>
<li>Watch for changes: yes</li>
</ol>
</li>
</ol>
</li>
<li>Open up the generated karma.config.js in your text editor and add āxmlā as a reporter
<ul>
<li><code class="language-plaintext highlighter-rouge">reporters: ['progress']</code> becomes <code class="language-plaintext highlighter-rouge">reporters: ['progress', 'xml']</code></li>
</ul>
</li>
<li>Open Visual Studio and create a project that lives in the folder that contains your tests/karma config file
<ul>
<li>I usually choose File -> Open Web Site and choose the FileSystem option, pointing at the folder</li>
</ul>
</li>
<li>Create a simple test we can use to verify everything works. The filename must match the filter you gave to <code class="language-plaintext highlighter-rouge">karma init</code> earlier (eg. <code class="language-plaintext highlighter-rouge">MyTestSpecs.js</code>)
<ul>
<li><code class="language-plaintext highlighter-rouge">describe("a test", function() {
it("passes", function() {
expect(true).toBe(true);
});
});</code></li>
</ul>
</li>
<li>From the command line, start karma
<ul>
<li><code class="language-plaintext highlighter-rouge">karma start</code></li>
<li>note: This will fire up the browsers that you configured; these must be left open to execute tests</li>
</ul>
</li>
<li>In Visual Studio, toggle the āRun Tests on Buildā option <em>on</em> in the Test Explorer Window</li>
<li>Right-click -> Refresh on the project, to ensure Visual Studio spots the test-results.testxml file</li>
<li>As you make changes to your js files and press save, Karma will now automatically execute your tests and the results will magically update in Visual Studio!</li>
<li>Bonus: You can open up other browsers and point them at the url thatās loaded in the existing browsers and they will automatically be used for tests. Eg. enter the URL into your smartphone browser, and now it will show up in the test results too!</li>
</ol>
<h1 id="feedback">Feedback</h1>
<p>Please send your feedback/issues/feature requests! :-)</p>
<ul>
<li>GitHub Issues: <a href="https://github.com/DanTup/TestAdapters/issues">TestAdapters/issues</a></li>
<li>Twitter: <a href="https://twitter.com/DanTup">@DanTup</a></li>
<li>Email: <a href="mailto:danny+testadapters@tuppeny.com">danny+testadapters@tuppeny.com</a></li>
</ul>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2014/03/cross-browser-javascript-testing-with-karma-and-visual-studio/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2014-03-02:/2014/03/a-jasmine-test-adapter-for-visual-studio/2014-03-02T00:00:00+00:002014-03-02T00:00:00+00:00A Jasmine Test Adapter for Visual Studio
<p><a href="https://github.com/DanTup/TestAdapters">
<img style="position: absolute; top: 0; right: 0; border: 0;" src="https://camo.githubusercontent.com/a6677b08c955af8400f44c6298f40e7d19cc5b2d/68747470733a2f2f73332e616d617a6f6e6177732e636f6d2f6769746875622f726962626f6e732f666f726b6d655f72696768745f677261795f3664366436642e706e67" alt="Fork me on GitHub" data-canonical-src="https://s3.amazonaws.com/github/ribbons/forkme_right_gray_6d6d6d.png" />
</a></p>
<p>Today I published my <em>second</em> Visual Studio extension! :)</p>
<p><a href="http://visualstudiogallery.msdn.microsoft.com/102979e0-61ba-4c6f-a18c-ca64cc7bd2c6">Jasmine Test Adapter for Visual Studio</a></p>
<iframe width="450" height="253" src="https://www.youtube.com/embed/Gc4xLjUxxOY?VQ=HD720" frameborder="0" allowfullscreen=""></iframe>
<p>Itās similar to the <a href="/2014/02/a-lua-test-adapter-and-framework-for-visual-studio/">Lua Adapter</a> I created recently, but uses node.exe instead of lua.exe and includes an internal copy of the Jasmine JavaScript testing framework. Iāve started playing around with AngularJS, and since it seems to be written with unit testing in mind, I somehow ended up at Jasmine and decided it needed an easy way to be run within Visual Studio!</p>
<p><img src="https://github.com/DanTup/TestAdapters/raw/master/DanTup.TestAdapters.Jasmine.Vsix/Screenshot.png" alt="Screenshot of Jasmine Test Adapter in action" /></p>
<p>If you try it out, please do send feedback!</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2014/03/a-jasmine-test-adapter-for-visual-studio/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2014-02-16:/2014/02/some-things-i-learned-while-building-my-visual-studio-test-adapter/2014-02-16T00:00:00+00:002014-02-16T00:00:00+00:00Some things I learned while building my Visual Studio Test Adapter
<p>Hereās a couple of things I learned while creating a Visual Studio test adapter that included a custom ITestContainerDiscoverer:</p>
<ul>
<li>There is almost no documentation on creating Visual Studio test adapters (and even less on ITestContainerDiscoverer and friends)</li>
<li>You need to install the Visual Sudio SDK to get the required assemblies for creating test container discoverers (but not for just test adapters)</li>
<li>You cannot create a test adapter without a test container discoverer if you want to scan non-DLL/EXE files</li>
<li>Although you can deploy test adapters via NuGet; this does not work for test container discoverers; they must be registered as real Visual Studio extensions</li>
<li>To have Visual Studio load your test container discoverer, you must add an asset with type āUnitTestExtensionā to your VSIX; an option which is not in the list (nor, seemingly, documented)</li>
<li>Just because youāre in the middle of responding to a request from Visual Studio to get a list of tests; doesnāt mean it wonāt ask you AGAIN AT THE SAME TIME on another thread :/</li>
<li>When building Visual Studio extensions, the entire VSIX output folder is copied into the expirimental instance, not just the files youād get via the VSIX. This means references to other projects might work fine on your machine, but not on somebody elses!</li>
<li>The āRun tests after buildā function actually means āwhen test container changeā, so even for non-building projects (eg. lua tests), it can work when you save files :)</li>
</ul>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2014/02/some-things-i-learned-while-building-my-visual-studio-test-adapter/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2014-02-16:/2014/02/a-lua-test-adapter-and-framework-for-visual-studio/2014-02-16T00:00:00+00:002014-02-16T00:00:00+00:00A Lua Test Adapter and Framework for Visual Studio
<p><a href="https://github.com/DanTup/TestAdapters">
<img style="position: absolute; top: 0; right: 0; border: 0;" src="https://camo.githubusercontent.com/a6677b08c955af8400f44c6298f40e7d19cc5b2d/68747470733a2f2f73332e616d617a6f6e6177732e636f6d2f6769746875622f726962626f6e732f666f726b6d655f72696768745f677261795f3664366436642e706e67" alt="Fork me on GitHub" data-canonical-src="https://s3.amazonaws.com/github/ribbons/forkme_right_gray_6d6d6d.png" />
</a></p>
<p>Today I published my first Visual Studio extension! :)</p>
<p><a href="http://visualstudiogallery.msdn.microsoft.com/8a046271-217f-48b6-8293-2b8447081695">Lua Test Adapter and Framework</a></p>
<iframe width="450" height="253" src="https://www.youtube.com/embed/fW3L3LTMdBw?VQ=HD720" frameborder="0" allowfullscreen=""></iframe>
<p>It was created because Iād recently been <a href="https://github.com/DanTup/RareShare">working on a World of Warcraft Addon, RareShare</a>. As the addon currently has no UI (mostly just interacts with the World of Warcraft API), I decided to create a set of automated tests for it (as any self-respecting software developer would ;)). I quickly became annoyed by reading the output of lua.exe and then looking up test line numbers, so decided to investigate how hard it was to create a Visual Studio test adapter!</p>
<p>It actually turned out to be slightly more complicated than expected; so Iāll leave that for the next post. However, I did build it in a way that keeps the Lua-specific stuff out of the main assembly; such that it should be quite simple to add other types of external tests (eg. Python, Perl, JavaScript, whatever). I canāt promise Iāll get around to any of them; but I might take pull requests ;-)</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2014/02/a-lua-test-adapter-and-framework-for-visual-studio/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2013-10-26:/2013/10/easily-calling-windows-apis-from-powershell/2013-10-26T00:00:00+00:002013-10-26T00:00:00+00:00Easily Calling Windows APIs from PowerShell
<p>After my <a href="/2013/10/useful-powershell-profile-snippets/">previous post of some snippets from my PowerShell profile</a>; I received another email from the person that prompted me to write that post, asking some questions about calling Windows APIs from PowerShell. It turns out that this isnāt so straight-forward, despite PowerShellās pitch as a system automation language!</p>
<p>The confusion came from various varying samples online of how to do this in PowerShell; and they all used blocks of C#, which many PowerShell users arenāt particularly familiar with.</p>
<p>Although I couldnāt find a way to do this without the C# wrapper, I did think it was worthwhile extracting some parts out of the C# code to avoid having to manipulate so much C# code in a quoted string inside PowerShell to add new Windows API methods. Unfortunately the argument types still need to be in C#, because I donāt know how to map them from the C++ examples easily.</p>
<p>Itās very basic; and can probably be improved by anyone that has some knowledge of calling Windows APIs; but hopefully itās a little simpler if youāre less comfortable with C#.</p>
<p>In PowerShell profile:</p>
<figure class="highlight"><pre><code class="language-powershell" data-lang="powershell"><span class="c"># Helper functions for building the class</span><span class="w">
</span><span class="nv">$</span><span class="nn">script</span><span class="p">:</span><span class="nv">nativeMethods</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">@();</span><span class="w">
</span><span class="kr">function</span><span class="w"> </span><span class="nf">Register-NativeMethod</span><span class="p">([</span><span class="n">string</span><span class="p">]</span><span class="nv">$dll</span><span class="p">,</span><span class="w"> </span><span class="p">[</span><span class="n">string</span><span class="p">]</span><span class="nv">$methodSignature</span><span class="p">)</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nv">$</span><span class="nn">script</span><span class="p">:</span><span class="nv">nativeMethods</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="p">[</span><span class="n">PSCustomObject</span><span class="p">]@{</span><span class="w"> </span><span class="nx">Dll</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$dll</span><span class="p">;</span><span class="w"> </span><span class="nx">Signature</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$methodSignature</span><span class="p">;</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">function</span><span class="w"> </span><span class="nf">Add-NativeMethods</span><span class="p">()</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nv">$nativeMethodsCode</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$</span><span class="nn">script</span><span class="p">:</span><span class="nv">nativeMethods</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">%</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="s2">"
[DllImport(</span><span class="se">`"</span><span class="si">$(</span><span class="bp">$_</span><span class="o">.</span><span class="nf">Dll</span><span class="si">)</span><span class="se">`"</span><span class="s2">)]
public static extern </span><span class="si">$(</span><span class="bp">$_</span><span class="o">.</span><span class="nf">Signature</span><span class="si">)</span><span class="s2">;
"</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="n">Add-Type</span><span class="w"> </span><span class="sh">@"
using System;
using System.Runtime.InteropServices;
public static class NativeMethods {
</span><span class="nv">$nativeMethodsCode</span><span class="sh">
}
"@</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="c"># Add methods here</span><span class="w">
</span><span class="n">Register-NativeMethod</span><span class="w"> </span><span class="s2">"user32.dll"</span><span class="w"> </span><span class="s2">"bool SetForegroundWindow(IntPtr hWnd)"</span><span class="w">
</span><span class="n">Register-NativeMethod</span><span class="w"> </span><span class="s2">"user32.dll"</span><span class="w"> </span><span class="s2">"bool ShowWindowAsync(IntPtr hWnd, int nCmdShow)"</span><span class="w">
</span><span class="c"># This builds the class and registers them (you can only do this one-per-session, as the type cannot be unloaded?)</span><span class="w">
</span><span class="n">Add-NativeMethods</span></code></pre></figure>
<p>And to use them:</p>
<figure class="highlight"><pre><code class="language-powershell" data-lang="powershell"><span class="c"># (the Out-Null is just to throw away the return value)</span><span class="w">
</span><span class="p">[</span><span class="n">NativeMethods</span><span class="p">]::</span><span class="n">SetForegroundWindow</span><span class="p">((</span><span class="n">Get-Process</span><span class="w"> </span><span class="nt">-name</span><span class="w"> </span><span class="nx">notepad</span><span class="p">)</span><span class="o">.</span><span class="nf">MainWindowHandle</span><span class="p">)</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Out-Null</span><span class="w">
</span><span class="p">[</span><span class="n">NativeMethods</span><span class="p">]::</span><span class="nx">ShowWindowAsync</span><span class="p">((</span><span class="n">Get-Process</span><span class="w"> </span><span class="nt">-name</span><span class="w"> </span><span class="nx">notepad</span><span class="p">)</span><span class="o">.</span><span class="nf">MainWindowHandle</span><span class="p">,</span><span class="w"> </span><span class="mi">2</span><span class="p">)</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Out-Null</span></code></pre></figure>
<p>Hopefully this helps a little. If anyone can suggest improvements (being able to paste method signatures directly from <a href="http://msdn.microsoft.com/en-us/library/windows/desktop/ms633539.aspx">the documentation</a> would be good), Iāll update the post.</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2013/10/easily-calling-windows-apis-from-powershell/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2013-10-23:/2013/10/useful-powershell-profile-snippets/2013-10-23T00:00:00+00:002013-10-23T00:00:00+00:00Useful PowerShell Profile Snippets
<p>Yesterday I recevied an email about some code snippets in my PowerShell profile that Iād mentioned on StackOverflow. While trying to retrieve them; I noticed that I hadnāt put them on to my newly-installed Windows 8.1 machine; so I thought it was worth sharing them here while I was copying them from my work machine anyway.</p>
<p>I donāt use PowerShell for scripting much now; Iāve been using FSI (type safety FTW). However, I do still think PowerShell is an excellent shell, and the ability to connect remotely to servers is incredibly useful for executing our deployment/IIS setup scripts/etc.</p>
<h2 id="adding-colour-to-hostnames-in-powershell-remoting-windows">Adding Colour to Hostnames in PowerShell Remoting Windows</h2>
<p>With some help from <a href="http://stackoverflow.com/q/13689797">StackOverflow</a>, I have some functions to help me connect to frequently-used servers, highlighting the hostname depending on whether the server is local/staging/live (green/yellow/red). Theyāre all prefixed with R- so I can type R- and then tab through the different servers.</p>
<figure class="highlight"><pre><code class="language-powershell" data-lang="powershell"><span class="c"># Connect to servers</span><span class="w">
</span><span class="kr">function</span><span class="w"> </span><span class="nf">R-LocalQAServer</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">R-_Connect</span><span class="w"> </span><span class="nx">Green</span><span class="w"> </span><span class="s2">"LocalQAServer"</span><span class="w"> </span><span class="bp">$null</span><span class="w"> </span><span class="s2">"D:\Applications\MyApps"</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="kr">function</span><span class="w"> </span><span class="nf">R-BuildServer2</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">R-_Connect</span><span class="w"> </span><span class="nx">Green</span><span class="w"> </span><span class="s2">"buildserver2"</span><span class="w"> </span><span class="bp">$null</span><span class="w"> </span><span class="s2">"D:\CI"</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="kr">function</span><span class="w"> </span><span class="nf">R-StageWebserver</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">R-_Connect</span><span class="w"> </span><span class="nx">Yellow</span><span class="w"> </span><span class="s2">"stageweby.mycompany"</span><span class="w"> </span><span class="s2">"mycompany\me"</span><span class="w"> </span><span class="s2">"D:\Applications\MyApps"</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="kr">function</span><span class="w"> </span><span class="nf">R-ProductionServer</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">R-_Connect</span><span class="w"> </span><span class="nx">Red</span><span class="w"> </span><span class="s2">"live.mycompany"</span><span class="w"> </span><span class="s2">"mycompany\me"</span><span class="w"> </span><span class="s2">"D:\Applications\MyApps"</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="c"># Shared function to handle connecting to servers with specified colour and a specific starting directory</span><span class="w">
</span><span class="kr">function</span><span class="w"> </span><span class="nf">R-_Connect</span><span class="p">([</span><span class="n">ConsoleColor</span><span class="p">]</span><span class="nv">$color</span><span class="p">,</span><span class="w"> </span><span class="p">[</span><span class="n">string</span><span class="p">]</span><span class="nv">$hostname</span><span class="p">,</span><span class="w"> </span><span class="p">[</span><span class="n">string</span><span class="p">]</span><span class="nv">$user</span><span class="p">,</span><span class="w"> </span><span class="p">[</span><span class="n">string</span><span class="p">]</span><span class="nv">$root</span><span class="p">)</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="nv">$user</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nv">$session</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">New-PSSession</span><span class="w"> </span><span class="nv">$hostname</span><span class="w"> </span><span class="nt">-Credential</span><span class="w"> </span><span class="nv">$user</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="kr">Else</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nv">$session</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">New-PSSession</span><span class="w"> </span><span class="nv">$hostname</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="c"># Set up MYAPP:\ Drive</span><span class="w">
</span><span class="n">Invoke-Command</span><span class="w"> </span><span class="nt">-Session</span><span class="w"> </span><span class="nv">$session</span><span class="w"> </span><span class="nt">-ScriptBlock</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="n">New-PSDrive</span><span class="w"> </span><span class="nt">-Name</span><span class="w"> </span><span class="nx">MYAPP</span><span class="w"> </span><span class="nt">-PSProvider</span><span class="w"> </span><span class="nx">FileSystem</span><span class="w"> </span><span class="nt">-Root</span><span class="w"> </span><span class="nv">$</span><span class="nn">using</span><span class="p">:</span><span class="nv">root</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Out-Null</span><span class="w">
</span><span class="nx">CD</span><span class="w"> </span><span class="nx">MYAPP:\</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="c"># Pass the hostname+color in so we know what was connected to, so we can overwrite the prompt in colour</span><span class="w">
</span><span class="n">Invoke-Command</span><span class="w"> </span><span class="nt">-Session</span><span class="w"> </span><span class="nv">$session</span><span class="w"> </span><span class="nt">-ScriptBlock</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nv">$connectedHost</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$</span><span class="nn">using</span><span class="p">:</span><span class="nv">hostname</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="n">Invoke-Command</span><span class="w"> </span><span class="nt">-Session</span><span class="w"> </span><span class="nv">$session</span><span class="w"> </span><span class="nt">-ScriptBlock</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nv">$promptColor</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$</span><span class="nn">using</span><span class="p">:</span><span class="nv">color</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="c"># Overwrite the prompt function to colour the server name</span><span class="w">
</span><span class="n">Invoke-Command</span><span class="w"> </span><span class="nt">-Session</span><span class="w"> </span><span class="nv">$session</span><span class="w"> </span><span class="nt">-ScriptBlock</span><span class="w"> </span><span class="p">([</span><span class="n">ScriptBlock</span><span class="p">]::</span><span class="n">Create</span><span class="p">(</span><span class="s2">"function prompt { </span><span class="si">$(</span><span class="n">Get-Content</span><span class="w"> </span><span class="nx">function:\RemotePrompt</span><span class="p">)</span><span class="w"> </span><span class="p">}</span><span class="s2">"))
# Connect the local/remote sessions
Enter-PSSession -Session </span><span class="nv">$session</span><span class="s2">
}
# Helper function that is transferred to the remote server to rewrite the prompt with colour
function RemotePrompt {
# write computer name with color
Write-Host "</span><span class="p">[</span><span class="nv">${env:computername}</span><span class="w"> </span><span class="p">(</span><span class="nv">${env:username}</span><span class="p">)]:</span><span class="w"> </span><span class="s2">" -Fore </span><span class="nv">$promptColor</span><span class="s2"> -NoNew
# generate regular prompt you would be showing
</span><span class="nv">$defaultPrompt</span><span class="s2"> = "</span><span class="n">PS</span><span class="w"> </span><span class="err">$</span><span class="p">(</span><span class="nv">$executionContext</span><span class="o">.</span><span class="nf">SessionState</span><span class="o">.</span><span class="nf">Path</span><span class="o">.</span><span class="nf">CurrentLocation</span><span class="si">)$(</span><span class="s1">'>'</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="p">(</span><span class="nv">$nestedPromptLevel</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="mi">1</span><span class="si">)</span><span class="s2">) "</span><span class="w">
</span><span class="c"># generate backspaces to cover [computername]: pre-prompt printed by powershell</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="o">!</span><span class="nv">$connectedHost</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nv">$connectedHost</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$</span><span class="nn">env</span><span class="p">:</span><span class="nv">Computername</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="nv">$backspaces</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"</span><span class="se">`b</span><span class="s2">"</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="p">(</span><span class="nv">$connectedHost</span><span class="o">.</span><span class="nf">Length</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="mi">4</span><span class="p">)</span><span class="w">
</span><span class="c"># compute how much extra, if any, needs to be cleaned up at the end</span><span class="w">
</span><span class="nv">$remainingChars</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">[</span><span class="n">Math</span><span class="p">]::</span><span class="n">Max</span><span class="p">((</span><span class="nv">$connectedHost</span><span class="o">.</span><span class="nf">Length</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="mi">4</span><span class="p">)</span><span class="w"> </span><span class="o">-</span><span class="w"> </span><span class="nv">$defaultPrompt</span><span class="o">.</span><span class="nf">Length</span><span class="p">,</span><span class="w"> </span><span class="mi">0</span><span class="p">)</span><span class="w">
</span><span class="nv">$tail</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="s2">" "</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="nv">$remainingChars</span><span class="p">)</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="p">(</span><span class="s2">"</span><span class="se">`b</span><span class="s2">"</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="nv">$remainingChars</span><span class="p">)</span><span class="w">
</span><span class="s2">"</span><span class="nv">${backspaces}${defaultPrompt}${tail}</span><span class="s2">"</span><span class="w">
</span><span class="p">}</span></code></pre></figure>
<h2 id="launch-kilnbitbucketwhatever-from-shell">Launch Kiln/BitBucket/Whatever from Shell</h2>
<p>The first thing Iusually do after pushing changes to Kiln, is open up the web app and raise code reviews. So; I added a command āKilnā which just launches Kiln at the correct repo page for where I am. The Kiln extension can do this (āhg kilnā), but weāve had bad experiences with the Kiln extensions not working on recent versions of Mercurial (due to PyWin32 dependencies) so most of us have them all disabled (except KilnAuth).</p>
<figure class="highlight"><pre><code class="language-powershell" data-lang="powershell"><span class="c"># Launch Kiln for current repo</span><span class="w">
</span><span class="kr">Function</span><span class="w"> </span><span class="nf">Kiln</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="kr">if</span><span class="w"> </span><span class="p">(</span><span class="n">Test-Path</span><span class="w"> </span><span class="s2">".\.hg\hgrc"</span><span class="p">)</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nv">$repoUrl</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="n">Select-String</span><span class="w"> </span><span class="s2">"default = (.*)"</span><span class="w"> </span><span class="nt">-Path</span><span class="w"> </span><span class="s2">".\.hg\hgrc"</span><span class="w"> </span><span class="nt">-AllMatches</span><span class="p">)</span><span class="o">.</span><span class="nf">Matches</span><span class="o">.</span><span class="n">Groups</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="nf">Value</span><span class="w">
</span><span class="n">Start</span><span class="w"> </span><span class="nv">$repoUrl</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">else</span><span class="w">
</span><span class="p">{</span><span class="w"> </span><span class="n">Write-Warning</span><span class="w"> </span><span class="s2">"Not in a repo!"</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="p">}</span></code></pre></figure>
<h2 id="launch-visual-studio-for-current-project">Launch Visual Studio for Current Project</h2>
<p>Because I switch between repos a lot (different versions of our product), itās a pain to change directory in PoSh and also navigate to the new solution in Visual Studio. So I simply close Visual Studio and type āVSā when in the correct directory for the new branch to launch Visual Studio for any sln file in the current folder. This could be extended to recurse down (or up), but I found it most convenient just working in the immediate folder.</p>
<figure class="highlight"><pre><code class="language-powershell" data-lang="powershell"><span class="c"># Launch VS for sln(s) in current folder</span><span class="w">
</span><span class="kr">Function</span><span class="w"> </span><span class="nf">VS</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="n">ii</span><span class="w"> </span><span class="o">*.</span><span class="nf">sln</span><span class="w">
</span><span class="p">}</span></code></pre></figure>
<h2 id="launch-windows-explorer-for-current-folder">Launch Windows Explorer for current folder</h2>
<p>Why type āstart .ā when you can just type āeā to start explorer in the current folder?</p>
<figure class="highlight"><pre><code class="language-powershell" data-lang="powershell"><span class="c"># Launch explorer in current folder</span><span class="w">
</span><span class="kr">Function</span><span class="w"> </span><span class="n">e</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">ii</span><span class="w"> </span><span class="o">.</span><span class="w"> </span><span class="p">}</span></code></pre></figure>
<h2 id="launch-default-browser-for-current-folder-via-web-server">Launch default browser for current folder via web server</h2>
<p>Sometimes you have a static html file you want to load up in a browser to hack on and test, but some browsers (like Chrome) wonāt let you pull in scripts for file:/// paths. This function (which requires Python in your PATH) launches the builtin Python web server for the current folder and fires up your browser. If you pass an optional filename</p>
<figure class="highlight"><pre><code class="language-powershell" data-lang="powershell"><span class="n">Serve</span><span class="w"> </span><span class="nx">mytest.htm</span></code></pre></figure>
<p>then itāll load that fie in the browser.</p>
<figure class="highlight"><pre><code class="language-powershell" data-lang="powershell"><span class="kr">Function</span><span class="w"> </span><span class="nf">Serve</span><span class="w"> </span><span class="p">(</span><span class="nv">$page</span><span class="p">)</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="n">Start-Process</span><span class="w"> </span><span class="nx">python</span><span class="w"> </span><span class="nt">-ArgumentList</span><span class="w"> </span><span class="s2">"-m SimpleHTTPServer 8000"</span><span class="w">
</span><span class="n">Start</span><span class="w"> </span><span class="s2">"http://localhost:8000/</span><span class="nv">$page</span><span class="s2">"</span><span class="w">
</span><span class="p">}</span></code></pre></figure>
<h2 id="run-mercurial-commands-with-less-typing">Run Mercurial Commands with Less Typing!</h2>
<p>And while weāre at it; why have to type āhgā all the time?</p>
<figure class="highlight"><pre><code class="language-powershell" data-lang="powershell"><span class="c"># Mercurial helpers</span><span class="w">
</span><span class="kr">Function</span><span class="w"> </span><span class="nf">st</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">hg</span><span class="w"> </span><span class="nx">st</span><span class="w"> </span><span class="bp">$args</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="kr">Function</span><span class="w"> </span><span class="nf">vd</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">hg</span><span class="w"> </span><span class="nx">vd</span><span class="w"> </span><span class="bp">$args</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="kr">Function</span><span class="w"> </span><span class="nf">ci</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">hg</span><span class="w"> </span><span class="nx">ci</span><span class="w"> </span><span class="bp">$args</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="kr">Function</span><span class="w"> </span><span class="nf">clone</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">hg</span><span class="w"> </span><span class="nx">clone</span><span class="w"> </span><span class="bp">$args</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="kr">Function</span><span class="w"> </span><span class="nf">hist</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">hg</span><span class="w"> </span><span class="nx">hist</span><span class="w"> </span><span class="nt">-l</span><span class="w"> </span><span class="nx">5</span><span class="w"> </span><span class="bp">$args</span><span class="w"> </span><span class="p">}</span></code></pre></figure>
<h2 id="add-visual-studio--net-tools-to-path">Add Visual Studio / .NET Tools to PATH</h2>
<p>I like the Visual Studio Developer Command Prompt because it has lots of useful stuff in PATH; but trying to get this into PoSh sucks. You canāt call the old .bat files because they wonāt update PowerShells PATH. Re-implementing them is insane, so itās easier to just hard-code the important few, and add/update as required!</p>
<figure class="highlight"><pre><code class="language-powershell" data-lang="powershell"><span class="c"># Add some useful stuff to PATH</span><span class="w">
</span><span class="nv">$</span><span class="nn">env</span><span class="p">:</span><span class="nv">Path</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="s2">";C:\Program Files (x86)\Microsoft SDKs\F#\3.1\Framework\v4.0\; C:\windows\Microsoft.NET\Framework\v4.0.30319; C:\windows\Microsoft.NET\Framework\v3.5; C:\Program Files (x86)\Windows Kits\8.1\bin\x86;"</span></code></pre></figure>
<h2 id="clean-all-folders-with-msbuild">Clean all folders with MSBuild</h2>
<figure class="highlight"><pre><code class="language-powershell" data-lang="powershell"><span class="c"># Reclaim a ton of disk space from branches that have been built but aren't actively needed</span><span class="w">
</span><span class="c"># It's hard-coded with my code folder and MYAPP:\ so I can call it from anywhere, but</span><span class="w">
</span><span class="c"># it can be made more generic and simplified with James Tryand's walk function further down</span><span class="w">
</span><span class="kr">Function</span><span class="w"> </span><span class="nf">Clean-All</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="n">pushd</span><span class="w"> </span><span class="nx">MYAPP:\</span><span class="w">
</span><span class="n">Get-ChildItem</span><span class="w"> </span><span class="nt">-Directory</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">%</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="n">pushd</span><span class="w"> </span><span class="bp">$_</span><span class="w">
</span><span class="n">Get-Item</span><span class="w"> </span><span class="o">*.</span><span class="nf">sln</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">%</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="n">Write-Host</span><span class="w"> </span><span class="p">(</span><span class="s2">" Cleaning {0}"</span><span class="w"> </span><span class="nt">-f</span><span class="w"> </span><span class="bp">$_</span><span class="o">.</span><span class="nf">ToString</span><span class="p">()</span><span class="o">.</span><span class="nf">Replace</span><span class="p">(</span><span class="s2">"C:\Work\Source\MYAPP\"</span><span class="p">,</span><span class="w"> </span><span class="s2">""</span><span class="p">))</span><span class="w"> </span><span class="nt">-F</span><span class="w"> </span><span class="n">Yellow</span><span class="w">
</span><span class="o">&</span><span class="s1">'msbuild'</span><span class="w"> </span><span class="nx">/t:Clean</span><span class="w"> </span><span class="nx">/nologo</span><span class="w"> </span><span class="nx">/v:m</span><span class="w"> </span><span class="bp">$_</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="n">popd</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="n">popd</span><span class="w">
</span><span class="nx">Write-Host</span><span class="w"> </span><span class="s2">""</span><span class="w">
</span><span class="p">}</span></code></pre></figure>
<h2 id="create-a-psdrive-for-code-folder">Create a PSDrive for Code Folder</h2>
<p>To make it easy to always get back to my root code folder, Iāve created a PDrive, and change into it at launch.</p>
<figure class="highlight"><pre><code class="language-powershell" data-lang="powershell"><span class="c"># Set up MYAPP:\ to point at code checkout folder</span><span class="w">
</span><span class="n">New-PSDrive</span><span class="w"> </span><span class="nt">-Name</span><span class="w"> </span><span class="nx">MYAPP</span><span class="w"> </span><span class="nt">-PSProvider</span><span class="w"> </span><span class="nx">FileSystem</span><span class="w"> </span><span class="nt">-Root</span><span class="w"> </span><span class="nx">C:\Work\Source\MYAPP</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Out-Null</span><span class="w">
</span><span class="nx">CD</span><span class="w"> </span><span class="nx">MYAPP:\</span></code></pre></figure>
<p>I think thatās all the useful stuff incurrently in my profile that might be handy to others. Do post your own useful snippets in the comments!</p>
<h2 id="highlights-from-the-comments">Highlights from the Comments</h2>
<h3 id="execute-command-for-all-child-directories">Execute command for all child directories</h3>
<figure class="highlight"><pre><code class="language-powershell" data-lang="powershell"><span class="kr">Function</span><span class="w"> </span><span class="nf">Walk-ChildDirectory</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="kr">Param</span><span class="p">(</span><span class="w">
</span><span class="p">[</span><span class="n">Parameter</span><span class="p">(</span><span class="n">Mandatory</span><span class="o">=</span><span class="bp">$true</span><span class="p">,</span><span class="n">ValueFromPipeline</span><span class="o">=</span><span class="bp">$true</span><span class="p">)][</span><span class="n">ScriptBlock</span><span class="p">]</span><span class="nv">$Task</span><span class="w">
</span><span class="p">)</span><span class="w">
</span><span class="n">ls</span><span class="w"> </span><span class="nt">-Directory</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">%</span><span class="p">{</span><span class="w">
</span><span class="n">pushd</span><span class="w"> </span><span class="bp">$_</span><span class="w">
</span><span class="o">&</span><span class="w"> </span><span class="nv">$Task</span><span class="w">
</span><span class="n">popd</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="n">Set-Alias</span><span class="w"> </span><span class="nx">walk</span><span class="w"> </span><span class="nx">Walk-ChildDirectory</span><span class="w">
</span><span class="c"># eg.:</span><span class="w">
</span><span class="c"># walk { git gc } # Git GC (WTF is this?)</span><span class="w">
</span><span class="c"># walk { msbuild /t:clean } # Clean all projects to reclaim disk space!</span><span class="w">
</span><span class="c"># walk { walk { walk { pwd } } } # Can be nested!</span></code></pre></figure>
<h3 id="ensure-stuff-doesnt-drop-off-the-history-list">Ensure stuff doesnāt drop off the history list</h3>
<figure class="highlight"><pre><code class="language-powershell" data-lang="powershell"><span class="nv">$MaximumHistoryCount</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">1024</span></code></pre></figure>
<h3 id="handy-functions-to-handle-strings-more-nicely">Handy functions to handle strings more nicely</h3>
<figure class="highlight"><pre><code class="language-powershell" data-lang="powershell"><span class="kr">function</span><span class="w"> </span><span class="nf">Quote-String</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="s2">"</span><span class="bp">$args</span><span class="s2">"</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="kr">function</span><span class="w"> </span><span class="nf">Quote-List</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="bp">$args</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="n">Set-Alias</span><span class="w"> </span><span class="nx">qs</span><span class="w"> </span><span class="nx">Quote-String</span><span class="w">
</span><span class="n">Set-Alias</span><span class="w"> </span><span class="nx">ql</span><span class="w"> </span><span class="nx">Quote-List</span></code></pre></figure>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2013/10/useful-powershell-profile-snippets/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2013-09-20:/2013/09/functional-programming-challenge-words-with-indexes/2013-09-20T00:00:00+00:002013-09-20T00:00:00+00:00Functional Programming Challenge; Words with Indexes
<p>A colleague has also been starting to learn F#, and today gave me a problem heād been trying to solve in F# in a āfunctional styleā. He wanted a function that took a string input, and returned a list of the words, tupled with the starting index.</p>
<p>I spent the whole of lunch staring at my screen, trying to wrap my head around List.fold, list.foldBack and other functions; but no joy.</p>
<p>While driving home, it occurred to me that I could do away with that nonsense, and just have a recursive function that passes all the values down, top-to-bottom!</p>
<p>Hereās my latest attemptā¦ The results are correct, which is an improvement over my original attempt!</p>
<figure class="highlight"><pre><code class="language-fsharp" data-lang="fsharp"><span class="k">let</span> <span class="n">splitOn</span> <span class="n">f</span> <span class="n">x</span> <span class="p">=</span> <span class="p">(</span><span class="nn">Seq</span><span class="p">.</span><span class="n">takeWhile</span> <span class="n">f</span> <span class="n">x</span><span class="p">,</span> <span class="nn">Seq</span><span class="p">.</span><span class="n">skipWhile</span> <span class="n">f</span> <span class="n">x</span><span class="p">)</span>
<span class="k">let</span> <span class="n">isSpace</span> <span class="n">c</span> <span class="p">=</span> <span class="n">c</span> <span class="p">=</span> <span class="k">'</span> <span class="k">'</span>
<span class="k">let</span> <span class="n">notSpace</span> <span class="n">c</span> <span class="p">=</span> <span class="n">c</span> <span class="p"><></span> <span class="k">'</span> <span class="k">'</span>
<span class="k">let</span> <span class="kt">string</span> <span class="p">(</span><span class="n">s</span> <span class="p">:</span> <span class="n">seq</span><span class="p"><</span><span class="kt">char</span><span class="o">>)</span> <span class="p">=</span> <span class="k">new</span> <span class="nc">String</span><span class="p">(</span><span class="nn">Seq</span><span class="p">.</span><span class="n">toArray</span> <span class="n">s</span><span class="p">)</span>
<span class="c1">/// Get a list of all words in a string tupled with the starting index</span>
<span class="k">let</span> <span class="n">getWords</span> <span class="n">input</span> <span class="p">=</span>
<span class="k">let</span> <span class="k">rec</span> <span class="n">getWordsRec</span> <span class="n">index</span> <span class="n">results</span> <span class="n">input</span> <span class="p">=</span>
<span class="k">match</span> <span class="n">input</span> <span class="k">with</span>
<span class="p">|</span> <span class="bp">[]</span> <span class="p">-></span> <span class="n">results</span>
<span class="p">|</span> <span class="p">_</span> <span class="p">-></span>
<span class="k">let</span> <span class="n">white</span><span class="p">,</span> <span class="n">rest</span> <span class="p">=</span> <span class="n">splitOn</span> <span class="n">isSpace</span> <span class="n">input</span>
<span class="k">let</span> <span class="n">word</span><span class="p">,</span> <span class="n">rest</span> <span class="p">=</span> <span class="n">splitOn</span> <span class="n">notSpace</span> <span class="n">rest</span>
<span class="n">getWordsRec</span>
<span class="p">(</span><span class="n">index</span> <span class="o">+</span> <span class="nn">Seq</span><span class="p">.</span><span class="n">length</span> <span class="n">white</span> <span class="o">+</span> <span class="nn">Seq</span><span class="p">.</span><span class="n">length</span> <span class="n">word</span><span class="p">)</span>
<span class="o">((</span><span class="n">index</span> <span class="o">+</span> <span class="nn">Seq</span><span class="p">.</span><span class="n">length</span> <span class="n">white</span><span class="p">,</span> <span class="kt">string</span> <span class="n">word</span><span class="p">)</span> <span class="p">::</span> <span class="n">results</span><span class="p">)</span>
<span class="p">(</span><span class="nn">Seq</span><span class="p">.</span><span class="n">toList</span> <span class="n">rest</span><span class="p">)</span>
<span class="n">getWordsRec</span> <span class="mi">0</span> <span class="bp">[]</span> <span class="p">(</span><span class="nn">Seq</span><span class="p">.</span><span class="n">toList</span> <span class="n">input</span><span class="p">)</span> <span class="p">|></span> <span class="nn">List</span><span class="p">.</span><span class="n">rev</span>
<span class="k">let</span> <span class="n">expected</span> <span class="p">=</span> <span class="o">[(</span><span class="mi">0</span><span class="p">,</span> <span class="s2">"Hello"</span><span class="o">);</span> <span class="p">(</span><span class="mi">6</span><span class="p">,</span> <span class="s2">"test"</span><span class="o">);</span> <span class="p">(</span><span class="mi">12</span><span class="p">,</span> <span class="s2">"World"</span><span class="o">)]</span>
<span class="n">printfn</span> <span class="s2">"Result is %A"</span> <span class="o">((</span><span class="n">getWords</span> <span class="s2">"Hello test World"</span> <span class="p">=</span> <span class="n">expected</span><span class="o">))</span></code></pre></figure>
<p>Can anyone come up with something more elegant?</p>
<h2 id="update">Update</h2>
<p>I managed to simplify/flatten it a little, though Iām not convinced itās any more readable!</p>
<figure class="highlight"><pre><code class="language-fsharp" data-lang="fsharp"><span class="k">let</span> <span class="kt">string</span> <span class="p">(</span><span class="n">s</span> <span class="p">:</span> <span class="n">seq</span><span class="p"><</span><span class="kt">char</span><span class="o">>)</span> <span class="p">=</span> <span class="k">new</span> <span class="nc">String</span><span class="p">(</span><span class="nn">Seq</span><span class="p">.</span><span class="n">toArray</span> <span class="n">s</span><span class="p">)</span>
<span class="k">let</span> <span class="n">getWords</span> <span class="n">input</span> <span class="p">=</span>
<span class="n">input</span>
<span class="p">|></span> <span class="nn">Seq</span><span class="p">.</span><span class="n">zip</span> <span class="p">(</span><span class="nn">Seq</span><span class="p">.</span><span class="n">initInfinite</span> <span class="n">id</span><span class="p">)</span> <span class="c1">// Zip the list with indexes</span>
<span class="p">|></span> <span class="nn">Seq</span><span class="p">.</span><span class="n">scan</span> <span class="p">(</span><span class="k">fun</span> <span class="p">(</span><span class="n">i1</span><span class="p">,</span> <span class="n">x1</span><span class="p">)</span> <span class="p">(</span><span class="n">i2</span><span class="p">,</span> <span class="n">x2</span><span class="p">)</span> <span class="p">-></span> <span class="c1">// Scan; replacing index with the previous index when not a space</span>
<span class="k">match</span> <span class="n">x1</span> <span class="k">with</span>
<span class="p">|</span> <span class="k">'</span> <span class="k">'</span> <span class="p">-></span> <span class="p">(</span><span class="n">i2</span><span class="p">,</span> <span class="n">x2</span><span class="p">)</span>
<span class="p">|</span> <span class="p">_</span> <span class="p">-></span> <span class="p">(</span><span class="n">i1</span><span class="p">,</span> <span class="n">x2</span><span class="p">)</span>
<span class="p">)</span> <span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="k">'</span> <span class="k">'</span><span class="p">)</span>
<span class="p">|></span> <span class="nn">Seq</span><span class="p">.</span><span class="n">filter</span> <span class="p">(</span><span class="k">fun</span> <span class="o">(_,</span> <span class="n">x</span><span class="p">)</span> <span class="p">-></span> <span class="n">x</span> <span class="p"><></span> <span class="k">'</span> <span class="k">'</span><span class="p">)</span> <span class="c1">// Strip the spaces</span>
<span class="p">|></span> <span class="nn">Seq</span><span class="p">.</span><span class="n">groupBy</span> <span class="p">(</span><span class="k">fun</span> <span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="o">_)</span> <span class="p">-></span> <span class="n">i</span><span class="p">)</span> <span class="c1">// Group by the word start index</span>
<span class="p">|></span> <span class="nn">Seq</span><span class="p">.</span><span class="n">map</span> <span class="p">(</span><span class="k">fun</span> <span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="n">x</span><span class="p">)</span> <span class="p">-></span> <span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="n">x</span> <span class="p">|></span> <span class="nn">Seq</span><span class="p">.</span><span class="n">map</span> <span class="n">snd</span> <span class="p">|></span> <span class="kt">string</span><span class="o">))</span> <span class="c1">// Strip redundant indexes; convert to string</span></code></pre></figure>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2013/09/functional-programming-challenge-words-with-indexes/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2013-09-14:/2013/09/opt-in-nulls-an-fsharp-feature-worth-switching-for/2013-09-14T00:00:00+00:002013-09-14T00:00:00+00:00Opt-in Nulls; an F# Feature Worth Switching For?
<p>Recently Iāve been learning a little F#. This is actually the third time Iāve tried to pick it upā¦ The first two times I just couldnāt get my head around it. This time, I think itās making sense! Whether or not I ever get to use it commercially/in production, I think itās been a valuable excercise. Just the basics of F# Iāve picked up so far have already start influencing how Iām writing C# in a good way.</p>
<p>There are a <em>lot</em> of things in F# that would save me a lot of time and/or drastically improve quality. One such feature is the handling of ānullsā using the <code class="language-plaintext highlighter-rouge">Option</code> type.</p>
<p>The <code class="language-plaintext highlighter-rouge">Option</code> type is similar to the <code class="language-plaintext highlighter-rouge">Nullable<T></code> type used nullable with value types in .NET, but it can be applied to reference types too. Because of this, types are (by default) not nullable in F# ( <em>generally</em> ā¦).</p>
<p>Take the following code that prints out a customers name:</p>
<figure class="highlight"><pre><code class="language-fsharp" data-lang="fsharp"><span class="k">type</span> <span class="nc">Customer</span> <span class="p">=</span> <span class="p">{</span> <span class="nc">Name</span> <span class="p">:</span> <span class="kt">string</span> <span class="p">}</span>
<span class="k">let</span> <span class="n">printCustomerName</span> <span class="n">customer</span> <span class="p">=</span>
<span class="n">printfn</span> <span class="s2">"Customer name is %s"</span> <span class="n">customer</span><span class="p">.</span><span class="nc">Name</span></code></pre></figure>
<p>In C#, this code would need to have a null check before accessing the <code class="language-plaintext highlighter-rouge">Name</code> property to avoid a possible <code class="language-plaintext highlighter-rouge">NullReferenceException</code>. In F#, because the argument is a <code class="language-plaintext highlighter-rouge">Customer</code> and not a <code class="language-plaintext highlighter-rouge">Customer option</code>, this check is not needed. The F# compiler will ensure that only <code class="language-plaintext highlighter-rouge">Customer</code> types can be passed as an argument to this function (note: if youāre consuming this code from another .NET language, all bets are off!).</p>
<p>I recently gave a talk to my colleagues about some of the interesting F# features Iād come across so far; the <code class="language-plaintext highlighter-rouge">Option</code> type being one of them. After the talk, one of my colleagues was debugging a <code class="language-plaintext highlighter-rouge">NullReferenceException</code> and I (jokingly) said that every time I hear someone debugging a <code class="language-plaintext highlighter-rouge">NullReferenceException</code> or see a null check, Iām going to exclaim āYou wouldnāt have that in F#!ā.</p>
<p>Well; itās been only a week. Already; the number of times this thought has popped into my head is <em>way</em> above what I thought it would be. The number of null checks going into our code <em>on a daily basis</em> and the number of <code class="language-plaintext highlighter-rouge">NullReferenceException</code>s raised by our (sadly, very legacy) codebase is staggering. I donāt know why we put up for this crap for so long! Runtime <code class="language-plaintext highlighter-rouge">NullReferenceException</code>s are embarassing, and thousands of null checks just add to code noise. Why canāt we have our cake and eat it?</p>
<p>This is just one of many F# features that I geniunely believe could have a big enough impact on your code (both quality, and readability) that it should be a factor when picking your programming language. It has access to the same BCL that C# does. It has access to the same third party libraries and NuGet packages. It can even interact with your existing C# code. Unfortunately, there are still some significant reasons not to switch; the tooling is nowhere near as good as with C# (even only considering stock VS functionality), and StackOverflow has only a fraction of the answers youāll want in F# vs C#. Both of these will improve in time and I think the idea of using F# for much of your .NET coding will become even more compelling.</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2013/09/opt-in-nulls-an-fsharp-feature-worth-switching-for/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2013-09-08:/2013/09/migrating-blog-to-github-pages-and-jekyll/2013-09-08T00:00:00+00:002013-09-08T00:00:00+00:00Migrating my blog from Google App Engine to GitHub Pages and Jekyll
<p>Iāve been trying to migrate my blog from Google App Engine to something a little more manageable (and not tied to GAE infrastructure) for some time. However, all of my attempts to rewrite the blog in ASP.NET have failed due to newer versions of āthingsā coming out, making me start over before I get to the end. Iāve come to the conclusion that coding my own blog is getting in the way of me blogging.</p>
<p>However; yesterday I came across Jekyll, which runs on GitHub Pages. This combination solves a lot of my blog requirements without me having to do anything!</p>
<ul>
<li>Itās free to host on GitHub pages</li>
<li>All of my existing posts can stay in HTML</li>
<li>My new posts can be written in Markdown! (finally! this one is completely Markdown :))</li>
<li>Itās not tied to GitHub, or even Jekyll. I can take the static HTML version of my site and put it anywhere (or replace it with a dynamic one in future!)</li>
<li>Code formatting support</li>
</ul>
<p>Hopefully; if my blog is low-maintenance, I might actually get back to blogging a little more! Sadly, there are a few little drawbacks:</p>
<ul>
<li>Jekyll doesnāt support tag/category listing/index pages out-of-the-box, and GitHub has plugins disabled. I came up with a workaround for this (as you can see, I have tags in the sidebar that link), but it relies on creating a stub page for each tag; but that doesnāt seem too bad compared to having to run Jekyll locally and just publish the build output! I might blog the details of this once everything is fully working.</li>
<li>All of my posts (including those that have .html extensions) end up with trailing slashes in Jekyll. I canāt explain this, but they get 301 redirects due to the folder names, so I donāt think itās a huge problem.</li>
</ul>
<p>Iāve still got some tidying up to do, but maybe now that my blog posts can just be text files in a repo, Iāll blog a little more!</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2013/09/migrating-blog-to-github-pages-and-jekyll/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>
tag:blog.dantup.com,2013-06-11:/2013/06/fixing-duplicate-contacts-in-android-people-app-from-google-contacts-and-google/2013-06-11T00:00:00+00:002013-06-11T00:00:00+00:00Fixing Duplicate Contacts in Android People app from Google Contacts and Google+
<p>Now that I've swapped my Windows Phone for a Nexus 4, I've discovered what a mess my Google Contacts are in once synced to Android and enabling Google+ to supplement contact data. It didn't matter too much on the <a href="http://prodct.info/nexus7">Nexus 7</a> tablet, but on the phone it's more frequently used, so having the same contacts appear three times is a problem!</p>
<p>Note: The People app has a "join" feature that lets you merge contacts on the device; however this is stored on the device and not synced anywhere. With multiple devices (and the possibility of buying new devices/factory resets/etc.), this isn't an ideal solution, so I wanted to fix the problem at the source.</p>
<p>I did all the usual stuff (merging duplicates in the Google Contacts web app), but still found some contacts where I had 2, 3 or even 4 copies once they made it to Android! I spent many hours going through making tiny changes to contacts to figure out what was causing the duplicates, and I've listed all the things that bit me below.</p>
<ul>
<li>
<strong>"Email Contacts" in Google+ Circles</strong><br />
<small>I found a number of circles that had "email" contacts in. When Google+ first launched, I just put people suggested by Google into circles. These often turned out to be non-Gmail email addresses; and the person subsequently joined Google+, making two contacts with the same name/email. The fix is easy; just remove all email contacts from circles.</small>
</li>
<li>
<strong>People with multiple Google+ Profiles</strong><br />
<small>Because Google are pushing us into upgrading all Google accounts into Google+ Profiles (eg. you can't share pictures with Hangouts without this; despite it working in Google Talk), many people have two accounts (especially if they use Google Apps for work). To fix this; I just uncircled all but what appeared to be the "main" account for anybody with multiple.</small>
</li>
<li>
<strong>Google Contact records with different names to Google+</strong><br />
<small>This one is pretty stupid. I had my parents named "Mum" and "Dad" in Gooogle Contacts, and some contacts as Mike and Matt that were Michael and Matthew in Gooogle+. This seemed to confuse the Google+ sync, and showed them as separate contacts in the People app. I couldn't find a fix for this; so mum and dad now have their full names in People :(</small>
</li>
<li>
<strong>Google+ syncs stale data!</strong><br />
<small>This one bit me the hardest. Every time I thought I'd fixed the issue, dupes would reappear a few minutes later. I believe the problem was that the Google+ app cached circle data (so all my changes to circles for the above where stale) and then syncs it into People. To fix this, after each change, I did "Clear Data" on the Google+ app (in addition to the Contacts and Contacts Storage).</small>
</li>
<li>
<strong>Multiple profile URLs against a Google Contact</strong><br />
<small>This is similar to a contact having multiple Google+ accounts. I found that when I'd added some contacts to circles, it had copied their profile URL into the contact record (under URLs, with a type of Profile). When I'd merged contacts, I ended up with multiple of these against the same record. This also caused the contact to appear twice! Fixing this was easy; I just deleted all profile URLs against all contacts, unless it was the only piece of information tying them to their Google+ profile (in most cases, I had the email address, so this was not needed).</small>
</li>
</ul>
<p>After all these changes, I've cleared data/caches and forced syncs several times no both devices, and haven't (yet) seen any contacts duplicated again :-)</p>
<p>This post was served up via my RSS feed. Please <a href="https://blog.dantup.com/2013/06/fixing-duplicate-contacts-in-android-people-app-from-google-contacts-and-google/">visit the original article</a> to read/post comments. If you found this article interesting, why not <a href="http://www.twitter.com/DanTup">follow @DanTup on Twitter</a> for more? :)</p>