The Folly of 'Unimportant Data'

Having been involved in computers for nearly 4 decades now, I have a healthy view towards data backups. Both my Macs are backed up to a local Time Machine drive on my network, PLUS to Amazon Glacier. I also have important development & client files copied to Google Drive and DropBox on a regular basis. Onsite and offsite backup.

I backup nearly all my critical data that I work with daily. However, I also have a ton of data which (I thought) I consider unimportant. This comprises of:

• Old videos that I have edited and uploaded to Youtube etc.

• Copies of photos from my wife’s phone

• Images for all software installers I’ve used while setting up my PCs

• Interesting videos or lessons that I have downloaded from Youtube over the years

• Collections of interesting guitar tabs I’ve found online, etc.

Basically, stuff that I thought I could find again later if I really needed them.

Then, this week, the 3TB external drive that I had all this stuff on finally gave up the ghost and died. “No biggie” I thought to myself. “It was nice to have, but I’ll just get a new drive and start accumulating this unimportant stuff again from scratch”.

But then, in that same week:

• My wife asked about a photo from her old phone that she couldn’t find any longer, and was it in my backup?

• My son called to ask if I had a raw copy of a video I had edited for him some years ago as he needed another section (that I had edited out in the final render) for an audition he had to do.

• I found out I had to reinstall an audio plugin for one of my old recordings, and the original publisher no longer had the older version of the plugin (which I needed to match my audio project) available for download on their site any longer.

How coincidental was that? All of a sudden, I needed this “unimportant” data that I thought I could throw away.

So this week, I’ve sent the external drive away to a data recovery service to see what they can do about bringing it back to life. The offer a free inspection to examine the extent of the failure, and quoted my AUD$300 to recover the data if they can. I figure $300 is worth it to me to just have all that data at my fingertips again, and if I do get it back, I will immediately be setting up a Glacier backup for all of it.

The Disconnectivity of Remote Working

Photo by trail on Unsplash

Throughout the 30+ years of running my own business, I have explored all aspects of teamwork.  From having my own in house team, to having a totally remote team, to a combined mix of the two.

Which do I prefer? Now THAT is an interesting question.

I would consider myself an introvert, and I do prefer working by myself in my own home office a lot of the time.  However, some of my best working memories have been when I have been in an office situation and working alongside others.

There is something about the human connection of being in the same space as others.  A myriad of non verbal cues and communication that goes on, most at a sub conscious level, which lends itself to a better sense of being part of a community which is pulling in the same direction.

Case in point - my current startup is a fully remote setup.  For the past two years, it was really only myself and another co-founder, who worked in a small town literally on the other side of the world.

Now, my co-founder and I had a great working relationship, and we produced a ton of stuff together.  All communication was mainly via Slack and email, and we used to talk on a daily basis PLUS have a weekly web video catch up.

My co-founder left the startup about 2 months ago.  The first week was really challenging, as I directly missed talking to someone while working away on new ideas.

But by the end of the first month, I started to get used to working by myself again.  After all, I had run the startup by myself for about a year before my co-founder joined me.  So it felt basically the same as it did before.

By the end of the second month, I was actually struggling to recall even working with my former co-founder.  This concerned me, as I always considered myself a sensitive person who liked to reminisce about happy memories.  So why was it suddenly so difficult for me to recall any of those good times we had had?  My co-founder's departure was amicable, so this wasn't as a result of any ill feelings.  Rather it just seemed that those experiences and memories were just floating out of reach, and without anything to anchor them too, they just seemed to waft away whenever I tried to recall them.

Even when I would go back through a Slack conversation to find an old screenshot or idea, I would re-read some of our conversations - but I struggled to actually remember the emotions or personality behind those chats.  Re-reading them seemed somehow cold and impersonal and I couldn't tell if I was tired, or angry, excited or happy while typing those paragraphs.

As a direct contrast to that, I can still clearly recall events that happened in my office over 20 years ago when I worked only feet away from the rest of my team.

Tiny things like a shared look, collapsing on the floor laughing at an 'in house' joke, or the casual punch on the shoulder as someone congratulated you while walking past your desk - all those things just added so much to my working experience that I, even as a self confessed 'lone wolf', missed them terribly.

There is something about being around people who are experiencing the highs and lows of their lives (even outside of work) that is strangely enriching and bonding.

To extend this even further - I was looking through my Facebook feed just this week, and I realised that I have become close friends with a vast majority of people that I have worked with face to face over the decades.  Remote workers much less so.  For some reason when a former remote staff member posts about their family or holiday or other life event, I find myself a lot less engaged with their thoughts and feelings.  There is still an element of them being an unknown 'stranger' so that any such intimate details of their lives instills a sense of guilt that I tend to deliberately avoid seeming too familiar or presumptuous when reading their posts.

While my recently departed co-founder and I had discussed an actual company meetup where we (and potential future staff) could meet face to face, it never happened during our working time together.  And now that my co-founder has moved on, I have accepted that we will probably never, ever meet in real life.

I am in the process of building up a whole new remote team now though, and am looking at strategies to try and counter this feeling of disconnection with those that I will figuratively work alongside for the coming years.

Regular company face to face meetups are definitely on the cards.  But I am also thinking that we might need to put something else in place outside of those times.

But what could take the virtual place of those little moments like tossing a paper plane across the office to see whose desk it would land on, or the understanding look that I would share with a colleague across from me after hanging up from a talking to a difficult client, or the good natured group ribbing that would happen when a co-worker brought a delicious smelling lunch into the office?  I have yet to see a web or mobile app that can replicate this sort of interaction.

Perhaps I have to go and invent it?
 

Building a face recognition app in under an hour

Over the weekend, I was flicking through my Amazon AWS console, and I noticed a new service on there called 'Rekognition'.  I guess it was the mangled spelling that caught my attention, but I wondered what this service was? Amazon has a habit of adding new services to their platform with alarming regularity, and this one slipped past my radar somehow.

So I dived in and checked it out, and it turns out that in late 2016, Amazon released their own image recognition engine on their platform.  It not only does facial recognition, but general photo object identification too.  It is still fairly new, so the details were sketchy, but I was immediately excited to try it out.  Long story short, within an hour, I had knocked up a quick sample web page that could grab photos from my PC camera and perform basic facial recognition on it.  Want to know how to do the same? Read on...

I had dabbled in facial recognition technology before, using third party libraries, along with the Microsoft Face API, but the effort of putting together even a rudimentary prototype was fraught with complexity and a steep learning curve.  But while browsing the Rekognition docs (thin as they are), I realised that the AWS API was actually quite simple to use, while seemingly quite powerful.  I couldn't wait, and decided to jump in feet first to knock up a quick prototype.

The Objective

I wanted a 'quick and dirty' single web page that would allow me to grab a photo using my iMac camera, and perform some basic recognition on the photo - basically, I wanted to identify the user sitting in front of the camera.

The Amazon Rekognition service allows you to create one or more collections.  A collection is simply a, well, collection of facial vectors for sample photos that you tell it to save.  NOTE: The service doesn't store the actual photos, but a JSON representation of measurements obtained from a reference photo.

Once you have a collection on Amazon, you can then take a subject photo and have it compare the features of the subject to its reference collection, and return the closest match.  Sounds simply doesn't it?  And it is.  To be honest, coding the front end of this web page to get the camera data actually took longer than the back end to perform the recognition - by a factor of 3 to 1 !!

So, in short, the web page lets you (1) create or delete a collection of facial data on Amazon, (2) upload face data via a captured photo to your collection, and (3) compare new photos to the existing collection to find a match.

Oh, and as a tricky extra (4), I also added in the Amazon Polly service to this demo so that after recognising a photo, the page will broadcast a verbal, customised greeting to the person named in the photo!

The Front End

My first question was what library to use to capture the image using my iMac camera.  After a quick Google search, I found the amazing JPEG Camera library on GitHub by amw, which allows you to use a standard HTML5 canvas to perform the capture, or fallback to a Flash widget for older browsers.  I quickly grabbed the library, and modified the example javascript file for my needs.

The Back End

For the back end, I knocked up a quick Sinatra project, for a lightweight Ruby based framework that could do all the heavy lifting with AWS.  I actually used Sinatra extensively (well, Padrino actually) to build all my web apps, and highly recommend the platform.

Note: Amazon Rekognition example actually promote uploading the source photos used in their API to an Amazon S3 bucket first, then processing them.  I wanted to avoid this double step and send the image data directly to their API instead, which I managed to do.

I also managed to do a similar thing with their Polly greeting.  Instead of saving the audio to an MP3 file and playing that, I managed to encode the MP3 data directly into an <audio> tag on the page and play it from there!

The Code

I have placed all the code for this project on my GitHub page.  Feel free to grab it, fork it and improve it as you like.  I will endeavour to explain the code in more detail here.

The Steps

First things first, you will need an Amazon AWS account.  I won't go into the details of setting that up here, because there are many articles you can find on Google for doing so.

Creating an AWS IAM User

But once you are set up on AWS, the first thing we need to do is to create an Amazon IAM (Identity & Access Management) user which has the permissions to use the Rekognition service.  Oh, we will also set up permissions for Amazon's Polly service as well, because once I got started on these new services, I could not stop.

In the Amazon console, click on 'Services' in the top left corner, then choose 'IAM' from the vast list of Amazon services.  Then, on the left hand side menu, click on 'Users'.  This should show you a list of existing IAM users that you have created on the console, if you have done so in the past.

Click on the 'Add User' blue button on the top of this list to add a new IAM user.

Give the user a recognisable name (more for your own reference), and make sure you tick 'Programmatic Access' as you will be using this IAM in an API call.

Next is the permissions settings.  Make sure you click the THIRD box on the screen, that says 'Attach existing policies directly'.  Then, on the 'Filter: Policy Type' search box below that, type in 'rekognition' (note the Amazonian spelling) to filter only the Rekognition policies. Choose 'AmazonRekognitionFullAccess' from the list by placing a check mark next to it.

Next, change the search filter to 'polly', and place a check mark next to 'AmazonPollyFullAccess'.

Nearly there.  We now have full permission for this IAM for Amazon Rekognition and Amazon Polly.  Click on 'Next: Review' on the bottom right.

On the review page, you should see 2 Managed Policies giving you full access to Rekognition and Polly.  If you don't, go back and re-select the policies again as per the previous step.  If you do, then click 'Create User' on the bottom right.

Now this page is IMPORTANT.  Make a note of the AWS Key and Secret that you are given on this page, as we will need to incorporate it into our application below.  

This is the ONLY time that you will be shown the key/secret for this user, so please copy and paste the info somewhere safe, and download the CSV file from this page with the information in it and keep it safe as well.

Download the Code

Next step, is to download the sample code from my GitHub page so you can modify it as necessary.  Go to this link and either download the code as ZIP file, or perform a 'git clone' to clone it to your working folder.

First thing you need to do is to create a file called '.env' in your working folder, and enter these two lines, substituting your Amazon IAM Key and Secret in there (Note: These are NOT real key details below):

export AWS_KEY=A1B2C3D4E5J6K7L10
export AWS_SECRET=T/9rt344Ur+ln89we3552H5uKp901

You can also just run these two lines on your command shell (Linux and OSX) to set them as environment variable that the app can use.  Windows user can run them too, just replace the 'export' prefix with 'set'.

Now, if you have Ruby installed on your system (Note: No need for full Ruby on Rails, just the basic Ruby language is all you need), then you can run

bundle install

to install all the pre-requisites (Sinatra etc.), then you can type

ruby faceapp.rb

to actually run the app.  This should start up a web browser on port 4567, so you can fire up your browser and go to 

http://localhost:4567

to see the web page and begin testing.

Using the App

The web page itself is fairly simple.  You should see a live streaming image on the top center, which is the feed from your on board camera.

The first thing you will need to do is to create a collection by clicking the link at the very bottom left of the page.  This will create an empty collection on Amazon's servers to hold your image data.  Note that the default name for this collection is 'faceapp_test', but you can change that on the faceapp.rb ruby code (line 17).

Then, to begin adding faces to your collection, ask several people to sit down in front of your PC or table/phone, and make sure their face is in the photo frame ONLY (Multiple faces will make the scan fail).  Once ready, enter their name in the text input box and click the 'Add to collection' button.  You should see a message that their facial data has been added to the database.

Once you have built up several faces in your database, then you can get random people to sit down in front of the camera and click on 'Compare image'.  Hopefully for people who have been already added to the collection, you should get back their name on screen, as well as a verbal greeting personalised to their name.

Please note that the usual way for Amazon Rekognition to work is to upload the JPEG/PNG photo to an Amazon S3 Bucket, then run the processing from there, but I wanted to bypass that double step and actually send the photo data directly to Rekognition as a Base64 encoded byte stream.  Fortunately, the aws-sdk for Ruby allows you to do both methods.

Lets walk through the code now.

First of all, lets take a look at the we page raw HTML itself.

https://github.com/CyberFerret/FaceRekognition-Demo/blob/master/views/faceapp.erb

This is a really simple page that should be self explanatory to anyone familiar with HTML creation.  Just a series of names divs, as well as buttons and links.  Note that we are using jQuery, and also Moment.js for the custom greeting.  Of note is the faceapp.js code, which does all the tricky stuff, and the links to the JPEG camera library.

You may also notice the <audio> tags at the bottom of the file, and you may ask what this is all about - well, this is going to be the placeholder for the audio greeting we send to the user (see below).

Let's break down the main app js file.

https://github.com/CyberFerret/FaceRekognition-Demo/blob/master/public/js/faceapp.js

This sets up the JPEG Camera library to show the camera feed on screen, and process the upload of the images.

The add_to_collection() function is straightforward, in that it takes the captured image from the camera, then does a post to the /upload endpoint along with the user's name as the parameter.  The function will check that you have actually entered a name or it will not continue, as you need a short name as a unique identifier for this facial data.

The upload function simply checks that the call to /upload finished cleanly, and either displays a success message or the error if it doesn't.

The compare_image() function is what gets called when you click the, well, 'Compare image' button.  It simply grabs a frame from the camera, and POSTs the photo data to the /compare endpoint.  This endpoint will return either an error, or else a JSON structure containing the id (name) of the found face, as well as the percentage confidence.

If there is a successful face match, the function will then go ahead and send the name of the found face to the /speech endpoint.  This endpoint calls the Amazon Polly service to convert the custom greeting to an MP3 file that can be played back to the user.

The Amazon Polly service returns the greeting as a binary MP3 stream, and so we take this IO stream and BaseEncode64 it, and place it as an encoded source link in the <audio> placeholder tags on our web page, which we can then do a .play() on the element in order to play the MP3 through the user's speakers using the HTML5 Web Audio API.

This is also the first time I have placed encoded data in the audio src attribute, rather than a link to a physical MP3 file, and I am glad to report that it worked a treat!

Lastly on the app js file is the greetingTime() function.  All this does is work out whether to say 'good morning/afternoon/evening' depending on the user's time of day.  A lot of code for something so simple, but I wanted the custom greeting they hear to be tailored to their time of day.

Lastly, lets look at the Ruby code for the Sinatra app.

https://github.com/CyberFerret/FaceRekognition-Demo/blob/master/faceapp.rb

Pretty straightforward Sinatra stuff here.  The top is just the requires that we need for the various AWS SDK and other libraries.

Then there is a block setting up the AWS authentication configuration, and the default collection name that we will be using (which you can feel free to change).

Then, the rest of the code is simply the endpoints that Sinatra will listen out for.  It listens for a GET on '/' in order to display the actual web page to the end user, and it also listens out for POST calls to /upload, /compare and /speech which the javascript file above posts data to.  Only about 3 or 4 lines of code for each of these endpoints to actually carry out the facial recognition and speech tasks, all documented in the AWS SDK documentation.

That's about all that I can think of to share at this point.  Please have fun with the project, and let me know what you end up building with it.  Personally, I am using this project as a starting block for some amazing new features that I would love to have in our main web app HR Partner.

Good Luck, and enjoy your facial recognition/speech synthesis journey.

TopHN - A fun side project built with Vue.js and RethinkDB

TopHN running in a side window so I can see news bubbling up and down in real time while I work away... (Yes, what you see is some actual code from the project - don't laugh!).

Over the past couple of years, I have tried to push my ageing brain constantly, and one of the best ways I've found to do that is to try and learn a new programming language, framework or methodology every month or so, just to keep the skills sharp.

I've always had a love/hate relationship with NoSQL databases, having cut my teeth for many decades on pure SQL systems, so I wanted to get my hands dirty with that.  I've also struggled a little bit to get to grips with Javascript front end frameworks, and wanted to improve my skill sets in that area.

So this past weekend, I decided to get 'down and dirty' with Vue.js as well as RethinkDB.  There is a lot of good natured banter amongst programmers about React vs Vue vs Angular etc. and I wanted to see for myself which one would suit my programming style better.  I had already done a lot of work in Angular v1 with my mobile app development (using Cordova and Ionic), and wanted to see if Angular v2 and the other frameworks I mentioned would be an easy transition.

Long story short, I had a bit of trouble getting my head around Angular v2, as well as React.  At the end of the day, Vue.js just seemed more natural, and possibly closer to Angular v1 to me, and I found myself being able to understand concepts and start knocking together a basic app within short order.

RethinkDB has also been in the news lately, with their parent company shutting down, although the database itself looks like it will live on as open source.  I've always liked the look of the RethinkDB management console, as well as the ease of installation on various platforms, so I decided to install it on my development Mac and give it a go.

The Project

The big question is - what to build?  I wanted to build something actually useful, instead of just another throwaway project.  Then, one day last week while I was browsing around Hacker News, it hit me.

Now, I love browsing Hacker News, and catching up with the latest tech articles, but one of the things that I found myself repeatedly doing was (a) refreshing the main 'Top News' screen every few minutes to see what people were talking about, and what had made its way to the Top 30, and (b) checking the messages that I had personally posted recently, to see if there were any replies to them, and (c) constantly checking my Karma balance on the top of the screen to see if there had been a mass of up or downvotes to anything I had posted.

These three things seemed to be my primary activities on the site (apart from reading articles), so I decided to see if I could build a little side project to make it easier.  So TopHN was born!

What is TopHN in a nutshell? Well, it is basically a real time display of top news activity on your web screen.  To be fair, there are already a LOT of other Hacker News real time feeds available out there, many which are far better than mine - but I wanted my solution to be very specific.  Most of the others display comments and other details, but I wanted my solution to be just a 'dashboard' style view of the top, important stuff that was relevant to me (and hopefully most other users too).

First things first, I decided to take a look at the HackerNews API.  I was excited to see that this was based on Google's Firebase.  I had used Firebase in a couple of mobile programming jobs 2 years ago, and really loved the asynchronous 'push' system they used to publish changes.  I debated whether to use the Firebase feed directly, but decided that No, because I was going to be doing some other manipulation and polling of data, that I didn't want to clutter up the Firebase feed directly with more poll requests, but instead would try and replicate the HN data set in RethinkDB.

So I went ahead and set up a dedicated RethinkDB server in the cloud.  This was a piece of cake following their instructions.  One the same server, I built a small Node.js app (only about 30 lines of code), whose sole purpose was to listen to the HN API feed from Firebase, and grab the current data and save a snapshot of them in my RethinkDB database.

Hacker News actually publishes some really cool feeds - every 30 seconds or so, a list of the top 500 articles are pushed out to the world as a JSON string.  Also, they have a dedicated feed which pushes out a list of changes made every 20 to 30 seconds.  This includes a list of article and comment ids that have been changed or entered in their system, as well as the user ids of any users who had changed their status (i.e. made profile changes, or had their karma increased/decreased by someone, or posted a comment etc.).

I decided to use these two feeds as the basis for building my replicated data set.  Every time the 'Top 500' feed would be pushed out, I would grab the id's of the articles, have a quick look in RethinkDB to see if they already existed, and if they didn't, I would go and ask for the missing articles individually, and plop those in RethinkDB.  After a few days of doing this, I ended up with tens of thousands of articles in my database.

I would also sniff out the 'changes' feed, and scan the articles in there to see if I already had them, and copy them if not.  Same with the users.  Every time a user was mentioned in the 'changes' feed, I would grab their updated profile and save in RethinkDB.

The screenshot above shows the RethinkDB management console, a really cool tool for checking server performance, as well as testing queries and managing data tables and shards.

So far so good.  The replicated database was filling up with data every few seconds.  Now, the question was - What to do with it?

I was excited to see that RethinkDB also had a 'changes()' feature, which would publish data changes as they happened.  But unlike the Firebase tools, these weren't client side only tools, and needed some sort of server platform to engage the features.  So what I decided on, was to use another Node.js app as the server back end, and use Vue.js as the front end for the interface elements.

I would also need to build a connection between the two using socket.io.  I was a bit disappointed that there didn't seem to be any native way to push/pull the changes from server to client without it, but hey - we are all about learning new things, and building a socket driven app was certainly something I hadn't done before (at least not from scratch).

So, end of the day, this second Node.js app would sit on a different server, and and wait for a user to visit the site.  Now, users can do a couple of things.  They can simply visit the top level URL of the site, and just see the Top 30 feed in real time.  And I mean nearly real time.  As new articles are published, or they move up and down the Top 30, the page view will bubble them up and down and show the latest scores and comment counters.

If the user elected to enter in their HN username, the page would additionally also display the user's Karma balance in real time, along with a notation for how much it has changed in the last couple of minutes.  Nothing like vanity metrics to keep people excited!

Also, if their username is entered, the page will show their last 10 or so comments and stories they published, so they can keep an eye on any responses to comments etc.

The second Node.js server is essentially a push/pull server.  It will silently push Top 30 list changes to all web browsers connected to it.  AND it will also set up a custom push event handler for any browsers where the user has specified their username.  As you can expect, this take a bit of management, and server resources, so I hope I never get to experience the HackerNews 'hug of death' where a bunch of people log on at the same time, because I am not really sure of how far this will scale before it comes to a screaming halt.

The Vue.js components purely sit there and listen for JSON data packets from the server pushes, and then format them accordingly and display them on the web page without having to refresh.

I haven't gone into the nutty details of how I built this on here, but if there is any interest and I get lots of requests, then I am open to publishing some code snippets and going into deeper detail of how I built the various components.

All in all, I am pretty happy with what amounted to around 4 or 5 days of part time coding.  I think this is a useful tool, and as you can see from the header image, I tend to have a narrow Chrome window open off to the side so I can keep an eye on news happenings and watch them bubble up and down.  The web page is also totally responsive, and should work on most mobile browsers for portability.

Are you a Hacker News member? Why not check out https://tophn.info and let me know what you think?