Trys Mudford's Blog

Everyday I'm Shuffling

Thu, 17 Dec 2020 00:00:00 +0000

I’ve been thinking about shuffling cards and arrays. That might sound odd, but having built a fair few spreadsheet games, and played an awful lot of cards this year, I guess it makes sense.

Let’s start with shuffling a deck of cards. This classic move is the riffle and bridge, or so I’m told:

Yes - I'm not very good at this shuffle yet...

It involves splitting the deck in two, and fanning the two halves back together from the bottom of each side, card by card. But I was wondering, assuming you did a perfect shuffle:

How long would it take to get the deck back to the first, initial state? Could you unshuffle the shuffle?

Now as you’ve seen from the above video, I’m not nearly a good enough shuffler to do this perfectly in real life, but I can code (replace shuffler with pretty much anything, and this statement sums up my life).

Let’s write a bit of JavaScript. A traditional deck is made of 52 cards, so we can create an array with numbers: 1-52 with the Array.from() method.

const cards = 52;
let deck = Array.from({ length: cards }, (_, i) => i + 1);

Next we have a method that splits the deck in two, empties the original pack, and pushes the cards back in, card by card.

function shuffleIn() {
  const half1 = deck.slice(0, cards / 2);
  const half2 = deck.slice(cards / 2, cards);
  deck = [];
  
  half2.forEach((card, index) => deck.push(card, half1[index]));
}

shuffleIn();

When we output the result, we can see what’s happening:

[26, 52, 25, 51, 24, 50, 23, 49, 22, 48, 21...]

The odd-position numbers are the original top half of the pack, and the even-positions are the bottom, and both positions are descending. Running it a second time on the newly created deck produces the following:

40, 27, 14, 1, 41, 28, 15, 2, 42, 29, 16, 3...]

It’s looking more random, but we still have a pattern: every fourth card is ascending. So, the big question, how many shuffles till we get back around to the same pack? 20, 40, 80? Let’s run it 100 times, and stop when it looks like the pattern is back to the beginning:

for (let i = 1; i < 100; i++) {
  shuffleIn();
  if (deck[0] === 1 && deck[1] === 2) {
    console.log(deck, `Shuffle no. ${i - 1}`);
    break;
  }
}

The answer? Shuffle no. 52.

That seems like a reasonable number given the number of cards. And it doesn’t seem like the sort of problem we’d run into in the real world, right?

But what if we made one small change. Let’s swap the halves round, so we push in the top deck first. All we’re changing is the final line here:

function shuffleOut() {
  const half1 = deck.slice(0, cards / 2);
  const half2 = deck.slice(cards / 2, cards);
  deck = [];
  
  half1.forEach((card, index) => deck.push(card, half2[index]));
}

The answer? Shuffle no. 8 😳

Goodness! I wasn’t expecting it to be that low! That seems like a similar number to a pretty ‘average’ shuffle in my experience. Interestingly, 7 is the number the Gilbert–Shannon–Reeds model arrives at to generate a thoroughly randomised deck.

So depending on which way you shuffle a pack, you could end up with very different results.

Other noteworthy findings

With the shuffleIn method, we arrive at the original order in reverse on shuffle no. 26:

Shuffle 26: [ 52, 51, 50, 49, 48, 47, 46, 45, ...]

You can even begin to see the convergent pattern as you get closer to 52, too:

Shuffle 49: [ 8, 16, 24, 32, 40, 48, 3, 11, ...]
Shuffle 50: [ 4, 8, 12, 16, 20, 24, 28, 32, ... ]
Shuffle 51: [ 2, 4, 6, 8, 10, 12, 14, 16, ...]
Shuffle 52: [ 1, 2, 3, 4, 5, 6, 7, 8, ...]

Shuffling arrays

Okay, onto shuffling arrays in JS. A cursory google with Bing for ‘randomly sort array js’ will return a bit of code a bit like this:

array.sort(() => Math.random() - 0.5);

If we un-minify it a little, we can see what’s happening:

array.sort(function (a, b) {
  return Math.random() -[]() 0.5;
});

array.sort() runs a comparison function that compares two items, normally items a and b. You return a positive or negative value to indicate whether item a is bigger or smaller than item b.

But in this ‘random’ function, instead of using the items themselves, it calls Math.random(); that returns a random number between 0 and 1, say: 0.7395929326168424. If you minus 0.5 from that, it’ll be a positive, or a negative number, achieving the same result.

In theory, random. In reality, flawed.

Let’s run the above in an experiment where we plot the distribution of the first card in a deck (call it the Ace of Hearts).

We’ll start by constructing a deck. It’s similar to before, but this time, we’re going to get a fresh deck every time we call it. We’ll also create a ‘distribution’ object with keys running from 0 - 51, all set with a starting point of 0.

const deck = () => Array.from({ length: 52 }, (_, i) => i + 1);
const distribution = {};
deck().forEach((l, i) => distribution[i] = 0);

Next we’ll pop in the shuffle function, and run it 10,000 times, plotting the indexOf position for card 0 every time we loop around. Finally, we’ll log the results. (For the record, this is running in Node 11).

const shuffle = (arr) => {
  const array = [...arr];
  array.sort(() => Math.random() - 0.5)
  return array;
}

for (let index = 0; index < 10000; index++) {
  const letters = shuffle(deck());
  distribution[letters.indexOf(1)]++;
}

console.log(distribution);

Okay, here goes:

Results

'0': 591,
'1': 506,
'2': 422,
'3': 405,
'4': 334,
'5': 313,
...
'45': 145,
'46': 164,
'47': 157,
'48': 144,
'49': 147,
'50': 109,
'51': 100

Woah there. The chances of the Ace of Hearts appearing in position 0 is six times more likely than it is appearing in position 51. That’s a huge bias!

Sidenote: Safari appears to have a reverse bias, in comparison to Chrome and Firefox.

To be totally honest, the only reason I even contemplated this being a problem was when I’d foolishly implemented this method of sorting on an online version of Nomination Whist I created in a spreadsheet. After playing for many weeks in lockdown, one of the players regularly seemed to pick up the Ace of Hearts when they were ‘randomly’ dealt cards, and this was why.

Depending on the sorting algorithm used in the browser/runtime, the items at either end of the array get compared less than those in the middle, and if the ‘random’ chance is effectively 50/50 (as Math.random() - 0.5 > 0 is), there’s a greater chance those items will stay nearer the ends.

To fix it, we need a better shuffle. The Fisher-Yates shuffle is the one to go for, as explained here by javascript.info.

const shuffle = (arr) => {
  const array = [...arr];
  for (let i = array.length - 1; i > 0; i--) {
    const j = Math.floor(Math.random() * (i + 1));
    [array[i], array[j]] = [array[j], array[i]]
  }

  return array;
}

Now when we run the test, the results are more far promising:

Results

'0': 187,
'1': 189,
'2': 174,
'3': 180,
'4': 198,
...
'46': 215,
'47': 207,
'48': 182,
'49': 171,
'50': 171,
'51': 181

Bingo.

Making a 'post-it game' PWA with mobile accelerometer API's

Thu, 02 Jan 2020 00:00:00 +0000

At the annual Clearleft Christmas Party, after a fabulous pub lunch and successful Secret Santa (courtesy of Basil), we ordered some drinks and settled in to play some games.

Charades was the first candidate, but after some truly questionable suggestions & acting, we thought it best to open the acting out to democracy and play the ‘Heads up’ game.

Originally known as the Rizla game or ‘post-it’ game, the aim is for one individual to attach a post-it/cigarette paper to their forehead, and for the rest of the group to act it out. When guessed, you move onto the next clue.

Nowadays there is, of course, an app for that. Instead of post-its, you use your phone. And instead of swapping clues/post-its, you droop your head down, and look back up. It’s a lot of fun, and generated a lot of laughter. But it got me thinking: why use an app, when you can use the web?! We are Clearleft, after all.

The next morning, while others were nursing slightly sore heads, I got to work on building just that. There were two main challenges to overcome:

Make the website work offline
Detect when the user looks down!

Building a PWA

I begun the project with the Preact CLI; this provided me with a progressive web app shell and a smart webpack build system.

Preact pre-caches routes and assets included in webpack, but I needed to get a few sound effects into the cache. I followed the documentation on how the service worker works, and added the following snippet:

workbox.routing.registerRoute(
  ({ url, event }) => url.pathname === '/assets/ding.mp3',
  workbox.strategies.cacheFirst()
);

Finally, it was important to get this to work without browser chrome, and launch in landscape mode. The web app manifest gives developers control over both of these things with the display and orientation properties.

{
  "display": "standalone",
  "orientation": "landscape"
}

Once deployed to Netlify, part one was complete!

Accelerometer access

Accessing device hardware sensors comes with some challenges. For one: security. In days of yore, you could gain access to this sensor without any user permission. This had massive security implications, namely that malicious code could infer PIN entries based on the angle of the device. Yikes.

This has quite rightly led to a big clampdown on accessing the API. Unfortunately, this means there’s a lot of conflicting documentation out there, making it hard to learn about the accelerometer today.

Additionally, hardware API’s spit out a huge number of events, and sifting through them can be a challenge. More on that in a bit.

DeviceOrientation vs. Accelerometer

I started off looking into the Accelerometer API. On first glance, it was perfect:

const accelerometer = new Accelerometer({ frequency: 30 });

accelerometer.addEventListener('reading', () => {
  const { x, y, z } = accelerometer;
  console.log(x, y, z);
});
accelerometer.start();

But poor browser support makes this a no-go for now. Furthermore, it’s wrapped up in the deeply confusing, and poorly supported Permissions API. This wasn’t going to work.

I looked into DeviceOrientation, an event on the window that emits the gyroscope position of the phone. But I initially struggled to get this hooked up. The MDN example seemed to work on my phone, but although the same simple example in my codebase and local server didn’t error, it also didn’t output any events.

A library called Gyronorm was linked to from the very same article, but that had all sorts of complications with linked third-party modules not playing ball. It was beginning to look like this wasn’t going to be possible.

I went back to the DeviceOrientation code and deployed it to Netlify. Loading that version DID work! The earlier issue was down to a lack of SSL on the local server. Progress!

Picking events

Wading through the deviceorientation events was a case of trial and error.

window.addEventListener('deviceorientation', e =>
  console.log(e.alpha, e.beta, e.gamma)
);

After deploying and running this, I held up my phone and started moving it around to get a feel of which value I should be focusing on. This game only requires the tracking of one angle: facing the phone downwards, and it appeared to be Gamma.

I wrote up some code to handle the movement and triggering of the next question:

// Convert all negative to positive integers
const angle = Math.abs(event.gamma);
// See if the angle is within our bounds of accuracy
const angleMatches = angle < DEGREES;
const { nexting, gameState } = this.state;

if (angleMatches && !nexting && gameState === 'running') {
  // Add the catch, call the next question method
  this.setState({ nexting: true }, this.next);
} else if (!angleMatches && nexting) {
  // Remove the catch
  this.setState({ nexting: false });
}

Apologies for the word nexting, the API throws out an awful lot of events, so it’s important to put a catch in place to stop the next question from being called multiple times before the user has lifted the phone back up to their head.

Debugging

I gave this a spin on my phone and iPad, and to my great surprise, it worked! Excited by this, I sent it to a friend with an iPhone. Surprise, surprise, yet more problems. The accelerometer did nothing.

I pondered for a while, and sent the site over to Jeremy. It also didn’t work on his phone. Our original theory was that the deviceorientation API hadn’t landed in time for his phone. But after checking the iOS versions of our respective devices, it became apparent it was the opposite problem. iOS 13 disabled the use of the API without a user opt-in.

After several minutes of Googling, I came across this issue on the Aframe repository. They had also had to overcome this problem to get VR to work in the browser. I delved into the PR fixing this issue, and updated my code to suit:

requestAccelerometer = () => {
  if (
    typeof DeviceOrientationEvent !== 'undefined' &&
    typeof DeviceOrientationEvent.requestPermission === 'function'
  ) {
    DeviceOrientationEvent.requestPermission()
      .then(response => {
        if (response === 'granted') {
          this.startAccelerometer();
        }
      })
      .catch(console.error);
  } else {
    this.startAccelerometer();
  }
};

startAccelerometer = () => {
  window.addEventListener('deviceorientation', this.move, true);
};

The DeviceOrientationEvent.requestPermission() method triggers a browser-based on-screen prompt for the user, asking if they are happy to use the accelerometer on this site. If they approve, we can start listening for those events.

With all that pieced together, hundreds of ideas collated, and a quick lick of visual paint, the site was ready to go! A good morning’s ‘work’! Below is a video of me playing the game, and here’s a link to the live game.

Basil: Secret Santa as a Service

Mon, 09 Dec 2019 00:00:00 +0000

I recently launched basil.christmas.trysmudford.com, a ‘Secret Santa as a service’. Basil has two modes. The first is a traditional list generator, letting you shuffle the participants and print off a nice, foldable list of names. The second is a little more involved, but considerably more exciting! A ‘head elf’ from your company can sign up and enter all participant names into the system. Basil will email everyone, letting them know who their giftee is.

But that’s only the beginning. We’ve all been there; you start in a new company and you get assigned the one person you’ve never spoken to. This is where Basil steps in. In each email, there’s a unique link that, when clicked, anonymously emails the giftee, asking for a little nudge in the right direction. They can respond, all without knowing who has asked for help! Basil-based encryption!

The legend of ‘Basil’ started many moons ago at Clearleft, when Kate Bulpitt discovered the mysterious elvish hero. ~~Kate~~ *ahem* Basil organised the whole affair, emailing each person in the team; acting as the encryption go-between. This worked great, but meant that individual knew all the Secret Santarers. It sounded to me like an opportunity for tech!

To be honest, it’s a bit of a silly side project, but there’s always ample opportunity for learning on websites like this. With no client restraints or budgets to consider, a side project is a great opportunity to try out new technologies and push ones design chops.

The stack

Vue.js is my go-to framework. When set up with Nuxt.js, I find it really quick and empowering to build in. Rather than spin up a Node.js server, I opted to use the generate mode, and host the site on Netlify. It gives me CDN hosting and a great devops experience with zero configuration.

The database & backend layer was an interesting choice, and where I focused my learning efforts. I opted for an avant-garde option of Airtable + Netlify Functions. Airtable is a lovechild of a spreadsheet and a database. The columns are typed, and the rows can be linked, so it’s possible to run it as a relational database. It has a very sensible API (and incredible live API docs). I used Airtable for our signature generator, but rather than use the HTTP REST API, this time I went for the npm module. Another learning opportunity.

The module still uses callbacks, so there was a bit of ‘promisification’ required to get it work nicely with async/await.

exports.updateElf = (rowId, payload = {}) => {
  return new Promise((resolve, reject) => {
    base('Elves').update(rowId, payload, function(err) {
      err ? reject(Errors.Generic) : resolve();
    });
  });
};

Netlify Functions are ‘serverless lambda functions’ that run as individual backends. When called, they spin up and make calls to the Airtable database. All API authentication is handled with environment variables stored on Netlify, so when developed with Netlify Dev, all your security is handled for you.

Emails were handled by Mailgun. The biggest hurdle was getting the first email to send. Their documentation doesn’t currently mention a different API URL for EU domains. As soon as I found this stackoverflow answer, I was away.

Classic form POSTs

Rather than use AJAX and JSON requests as is the usual approach in this decade, I ended up going old school and use form POSTs and redirects for the data exchange. This meant I could start from a base of solid HTML, without worrying about requests from JavaScript. It might not be quite as seamless having full page refreshes, but given most users will only see one form in the whole flow, it doesn’t harm the experience.

Deciding when to add complexity, and when to hold back, is another skill that’s worth honing. So regularly do we reach for the shiny tool, when the slightly dusty one will do just fine. Even in modern tools, like Vue.js, there’s still plenty of power in the humble

and 302 redirect.

Design

I’m not a designer, but I do love Christmas.

With those credentials out the way, I decided to have a crack at designing this site. Creative Market was my biggest friend on this project. There are so many über talented individuals on that platform. As soon as I stumbled upon these creatures, I fell in love.

The colour scheme and typography had to be suitably festive for such a project. Zeichen, and DM Sans provided a nice mix of conversational ‘basil tone’ and readable prose. A contrasting scheme of pink and dark blue, combined with lashings of noise, and topped with some wonderful snowflakes (created by Cassie), let to a suitably seasonable creation.

The initial design direction was decided in Sketch, but I quickly moved to the browser to roll it out. Working in Vue.js components, I was able to swiftly build out the various form-based pages with great ease.

Error codes

Deciding on an error structure is easily overlooked. As this project used redirects, rather than JSON responses, it was important to establish a format that could be interpreted by the frontend.

I came up with an ‘enum’ of possible error codes and shared it between the front- and backend. If the serverless function ever caught an error, it redirected the user to /error/${ErrorCode}/ where Nuxt rendered the appropriate message to the user. This was also an opportunity to play around with Basil’s tone of voice.

{
  TokenExpired: 'My dearest elf, it\'s time to log in again',
  AlreadyContacted: 'Have patience, child. You\'ve already contacted that elf!',
  FarTooMany: 'Humble apologies, you\'ve hit your elvish quota!'
}

Planning for ‘appropriate scale’

Basil was never going to take over the internet, but there was a chance a few others may like to use it. Rather than build it just for our internal use at Clearleft, I made sure the schema was set up to allow multiple groups to use the system. This did mean the added complication of putting in an authentication system, but that in itself was an opportunity to build a password-less authentication flow for the first time.

There was no need to prepare for greater scale than that. I think we’ve all been burned in the past worrying about whether the stack will cope with X users, with no basis for whether any users will arrive. More and more, I’m realising that building for myself, keeping in mind not to be exclusionary, nor paint myself into a corner, is the best bet for web things.

Give it a go!

If you’re in the market for a Secret Santa generator, managed or otherwise, please feel free to give Basil a whirl!

Visit Basil

Tiny lesson: rapid builds, email signatures and Airtable

Wed, 28 Aug 2019 00:00:00 +0000

We’re fortunate enough to have some rather snazzy email signatures, kindly created by Benjamin. He’s been lovingly crafting these by hand; diligently updating them each time an event concludes or a new Clearleftie joins. This seemed like a fun and helpful task to automate. After a morning of hackery, I had a working version of the signature generator deployed and ready for an internal test.

There are a couple of interesting technical decisions which we’ll delve into, but before that, let’s talk about building at pace more generally.

Quick web things, quick web wins

One of my passions is rapidly testing ideas on the web. Development pace is one of the biggest things the web has going for it over native applications. A HTML file, domain, server space and a couple of hours is all you need to get something online.

It’s worth saying upfront that rapid builds are definitely not for everything. In fact, they should be used sparingly for prototypes or side projects that aren’t business-critical. Like the good, fast, cheap venn diagram, quick builds are inherently flawed, but they do still have merit.

I really love a thorough plan and technical spec, but there’s something wonderful about rapidly spiking a problem and not worrying too much about the details. It’s how Sergey, JS Pedalboard, Javasnack and several marginally popular F1 parody websites came about.

The stack choice isn’t really important.

Spikes are a great way to try a new technology and learn by making mistakes. The only pre-requisite I’d suggest is to use a stack that you can deploy easily. There’s nothing worse than getting something working locally, then finding you can’t host it without an AWS degree or pricey hosting infrastructure.

In the past, PHP was my jam. It’s still so much easier to host than Node.js, and you can build quickly without getting too bogged down in implementation details. These days I’m tending to lean on static site generators like Hugo and Sergey (shameless plug), before deploying to Netlify.

For this project, I opted for Preact and a small vanilla Node build script. Preact CLI boilerplated the site very quickly, and after a few minutes of stripping back the extra cruft, I had an ES6, reactive and hot-loaded development environment ready. I then ran a quick ‘hello, world’ deploy to confirm it would all build on Netlify.

With that in place, it was time to actually build the darn thing.

From the very rough plan in my head, it appeared there were two main parts to this project:

The template generator - Preact
The data source - JSON & Airtable

Generating code with code

Email signatures are notoriously awful to code, but fortunately Benjamin had done the hard work already! I grabbed an existing signature and converted it into a little method that squirted the parts of a ‘person’ in, mixed it with some sensible defaults, and returned some HTML (well, JSX).

const person = {
  forename: 'Trys',
  surname: 'Mudford',
  team_name: 'trys-mudford',
  avatar_name: 'trys-mudford-small',
  role: 'Front end developer'
};

renderSignature = person => (
  <div style="min-height:50px;line-height:17px;color:#505050;min-width:350px;font-family: Arial, sans-serif; font-size: 10pt; line-height: 1.5;">
    <p>{person.forename}p>
    <p>---p>
    <a href={data.defaults.team_url + person.team_name}>
      <img
        style="float:left;margin:2px 6px 32px 0;width:90px"
        src={data.defaults.avatar_url + person.avatar_name + '.png'}
        alt={person.forename + ' Profile Pic'}
      />
    a>
    <p>
      <strong>{person.forename + ' ' + person.surname}strong>
      <br />
      {person.role} |{' '}
      <a
        style="color:#006ff5;text-decoration:none;font-weight:700;border-bottom:1px"
        href="https://clearleft.com/"
      >
        Clearleft
      a>
      <br />
      <a
        style="color:#505050;text-decoration:none"
        href={data.defaults.phone_url}
      >
        {data.defaults.phone_text}
      a>
    p>
  div>
);

The next step was to import the list of staff from a JSON file, pick out a selected team member and render the above template. Thanks to JS imports, this was nice and clean to achieve:

import { h, Component } from 'preact';
import data from './data';

class App extends Component {
  render() {
    const person = data.team.find(x => x.team_name === 'trys-mudford');

    return (
      <div>
        {person && (
          <section class="person">{this.renderSignature(person)}section>
        )}
      div>
    );
  }
}

Next I moved the hard-coded user identifier up into state, and added a