It’s been a year of migrations. I moved my email to Fastmail. I set up a NAS to avoid Dropbox lock-in. I moved my Mastodon fediverse account to CoSocial. And now I’ve moved WordPress hosts.
There’s more I can do to be a more elbows-up Canadian and to embrace small tech. One step at a time, though.
This May 1, I joined Upgrade Inc as a Principal DevOps Engineer on the Infrastructure Performance Management (IPM) team. Technically this is a change of industries from automotive to financial. I was previously at Autonomic, a Ford subsidiary, as a Principal Site Reliability Engineer on the Observability team there. In practice, my clients continue to be my fellow engineers.
I learned a lot in my time at Autonomic from many wonderful people. Before Au, I had only a superficial knowledge of Kubernetes, and certainly had never touched the Go programming language. Nothing is a better teacher than many, many hands-on projects. Upon joining Upgrade, I discovered that they were already using something I had written and open sourced in Go, which feels like an unexpected compliment.
As I was finishing up at Au, one of my colleagues asked me — how am I so productive? To some extent that the illusion of the public record — we know our failings, but of others we mostly only know their successes. Still, an interesting question. It’s not because I’m immune to procrastination — I scroll through the internet as much as anyone else. It’s not because I put in more hours — overtime is for exceptional circumstances, not everyday, and it is not sustainable. I think I appear productive because I love solving the puzzles that come up in our profession, and completed projects have a way of accumulating.
A spreadsheet is the wrong tool for everything, but it’s also a very available tool. No results came up when I was searching for how to join data sets in different ranges in a Google Sheets or Excel worksheet, so here’s how you do it.
Suppose you have these three data sets in a Google Sheets worksheet:
| Key |
| Aardvark |
| Aardwolf |
| Aaron |
| Key |
| Aardvark |
| Koala |
| Zebra |
| Key |
| Aaron |
| Abdullah |
| Alice |
How do you join them and list the unique elements on a different sheet? You want something along the lines of this formula:
=index(sort(unique({sheet1!A$2:A$4; sheet2!A$2:A$24; sheet3!A$2:A$4; })),row()-1)
{range1; range2} joins the contents of multiple ranges into a union
unique eliminates duplicates, sort() is self explanatory, and index(range, i) gets an element
The output should be something like this:
| Key |
| Aardvark |
| Aardwolf |
| Aaron |
| Abdullah |
| Alice |
| Koala |
| Zebra |
You should never, ever post an @here, @channel, or @everyone in Slack, or Discord, or Teams, or HipChat, or Mattermost. When you do this, you turn an asynchronous communication tool into a synchronous one. You interrupt everyone’s day and night regardless of what time zone they are in.
It is completely incompatible with working Remote, as we all are. See It Doesn’t Have To Be Crazy At Work by DHH and Jason Fried — use asynchronous communication tools. I work 9 to 5 in Toronto, you work 10 to 6 in Fiji, don’t message me at 6pm Fiji time, and certainly don’t message everyone in the company at 6pm Fiji time to tell them that bran muffins contain bran.
A regular message is a sufficient unit of communication for virtually anything. Trust me — it will be duly considered when its time comes.
In economics there’s the concept of signalling, assuming that hasn’t been ruined by ignorant randos asserting that virtue is a bad thing to signal. What you signal when you @channel a trite quotidian message is that you are so much more busy than anyone else could possible be, and I couldn’t care less as to how busy you might be. Stop pretending to be busy, respect my time, and I’ll respect yours.
I recently saw a discussion about how very few people actually use the famous gitflow workflow. This makes sense to me. The situation where one might have multiple simultaneous supported releases is relatively rare in software as written, because most software is written with one customer or use case in mind and that use case will run the latest stable version. Your team is likely not working on Ruby, where you’ll want to release 2.5.5 and 2.6.2 at the same time. As a result, there won’t be any release branches the way Gitflow means it, and there won’t be any long-lived feature branches.
I want to discuss something much lower level — how to keep your git history clean, which makes meaningful commit messages possible and gives you a changelog for free. That’s what the git workflow on my team is focused on. We accomplish this by amending one or very few commits while we work on a branch and then rebasing for the Pull Request.
This is different from what I was accustomed to on my previous team, where I would continuously add new commits to a branch and then either merge or squash merge the Pull Request.
The typical sequence of steps is to create a branch:
git checkout -b <initials>/<branch-name>
Create or modify some files, and then indicate your intent to add the new files:
git add --intent-to-add
Examine the changes and add them interactively:
git add --patch
Initially create a new commit:
git commit
Later, amend the existing commit:
git commit --amend
Initially, push your branch to the repo:
git push --set-upstream origin HEAD
Later, push it again with the amended commits without overwriting other people’s changes:
git push --force-with-lease
Finally, Rebase and Merge your pull request for that squeaky clean git history. PRs created this way have the added benefit of being easy to review, so your teammates will get back to you with meaningful feedback faster.
I like to alias all of these commands for economy of typing:
git config --global alias.an 'add --intent-to-add'
git config --global alias.ap 'add --patch'
git config --global alias.ca 'commit --amend'
git config --global alias.co checkout
git config --global alias.pfwl 'push --force-with-lease'git config --global alias.pu 'push --set-upstream origin HEAD'
git config --global alias.st status
Autonomic sent us all to work from home for social distancing a week before the rest of the Silicon Valley companies did, and two weeks before my father’s Toronto bank did. Good on them. I very much appreciate the precaution.
It was a sharp transition, because it was hitherto an office-only environment. This was less because of what we do — all our infrastructure is on the public cloud, not any kind of intranet — and more because pair programming is fundamental to the work practice. I found myself enjoying pair programming since I started at Autonomic in January, and it helped me on-board rapidly and even open some pull requests on my very first day. Pair programming is excellent for knowledge transfer. In theory it’s possible to pair program remotely, but my team is still getting there.
Speaking of working remotely, I have Jason Fried‘s Remote on order. The Amazon Marketplace vendor promises that the book will get to me sometime in April. In the meantime, Jennifer Tarliuk has written up a nice summary of Remote.
While my team wasn’t remote, we are distributed. My manager and product manager are in California, while the rest of us are in Toronto and other cities. We already had the structure for remote planning, separate remote stand-ups for just Eastern time zone and for joint North America, and remote retrospectives in place.
New things we’ve put in place for work from home:
I’m lucky because I’m healthy so far, I don’t have kids to distract me, and my place is large enough for a home office. We should be kind to our coworkers who are caring for their kids, or who are living in a 300 sqft studio, or who are caring for sick family members. We should help them shoulder the load. My company has been smart and supportive, and I hope your place of work does the right things too.
I’m very excited to be joining Autonomic this coming Monday. They do self-driving car data. I’m joining as Senior Site Reliability Engineer to help grow the infrastructure by several degrees of magnitude. I expect to be doing a lot of Kubernetes and learning Golang as I go.
This draws to a close 14 years at IBM. My last few months at IBM have been spent recruiting some very strong team members, and I trust that the IBM Developer Skills Network and Cognitive Class are in great hands.
I joined IBM as co-op student. My university buddy Gerald had set his MSN Messenger status to “Message me if you want a job”. It was late in the semester and I was singularly bad at interviewing back then, so I jumped on the opportunity. I spent that year writing tutorials for using the IBM Db2 database and its exciting XML features with various programming languages. A few years later, that morphed into putting Db2 on the cloud, then Apache Hadoop, and then developing and operating an education site about all sorts of open source technologies with hands-on labs and a couple million students.
I’m leaving IBM as a Senior Software Developer Manager (or perhaps a Principal Engineer), but on a technical side what I do has been Site Reliability Engineering. Past a certain scale, you need to start thinking of operations in terms of acceptable error budgets and of instrumenting the right service level indicators to properly judge the health of many large, distributes services coming together to deliver an experience. I’m excited to have the opportunity to apply the same principles at ever larger scales at Autonomic.
I wish to officially register my complaint about the lack of a gnome emoji in Unicode. These folks exist:
Yet there is no gnome. If there had been, I could have said that I’m joining 🚗nomic in even fewer characters!
In 2019, my team embraced the philosophy that any library or package we create should be open sourced with a permissive license in a public source repo. This is not just the right thing to do — it’s also a more productive approach. It’s the right thing to do because we get to collaborate effectively with the larger development community and benefit from public tools like Github’s neat dependency update bot. It’s more productive because we avoid maintaining a shadow infrastructure of private package repos.
Some of these libraries are very minor, but the point is not to open source only perfect, polished, popular things. If you only open source perfection, you’ll open source nothing. Perfect is the enemy of good.
The thing that impresses me the most of the most of our open source projects is a dashboard for monitoring a Gluster storage cluster that some very brilliant people on my team put together.
We also open sourced a Ruby client for Kubernetes. Its distinguishing feature is being able to update among other things all of Pods, Services, Deployments, Endpoints, and Ingresses.
Another thing we released publicly was our extensible backup framework, Backwork. We deploy lots of little, inconsequential databases and files that we nevertheless need to be able to back up with one tool rather than memorizing the syntax of MySQL backup tool for one thing, the MongoDB backup tool for another, and so on.
We use JupyterLab a lot in our Skills Network Labs environment that lets learners learn Python, machine learning, and other technologies. To that end, we’ve put together several plugins.
I also open sourced an older Ruby on Rails library I put together when I needed to write several REST services in quick succession. It catches common Rails exceptions and transforms them into an appropriate RESTful status code (401, 404, 422, etc.)
I expect there are a lot more contributions coming in 2020, including several tools for the Open edX ecosystem for online courses.
The Internet Society is selling the .org registry to Ethos Capital, a private equity firm. Such .org domains are typically used by non-profits, the open source community, and — for ineffable reasons — my blog.
The function of a registry is to grant monopolies over specific domain names for a year. They can set a new price for renewal every year — say, instead of $10/year charge $10,000/year. If I were to not re-purchase my domain name when it expires, then all the existing links to my blog would instantly break.
When Ted Nelson invented hyperlinks in 1965 for the Xanadu hypertext system, they were bidirectional and impossible to break. For practicality, Tim Berners-Lee simplified the hypertext architecture to unidirectional links when adapting it to create the world wide web and the first web browser in the 1989, and broken links are the reality we live with today.
Consider insulin. The Canadian inventor of insulin, Frederick Banting, sold the patent for a dollar so that it would be freely accessible to everyone in the world. It costs a dollar to produce a dose. It retails for $30 a dose in Canada. It retails for $700 a dose in US. When you monetize someone’s lifeline, you can charge their life savings.
My blog depends on the stability of its domain name. How much is not breaking all incoming links worth to me?
Just to be safe, I’ve renewed my domain at the current reasonable price for the next nine years to postpone my decision point. In the meantime, I encourage everyone to sign EFF’s open letter to ISOC to block the sale of .ORG.
My team is hiring a new cohort of paid developer interns in Toronto (Markham) for January 2020:
We have a great, friendly, supportive environment. Everyone on the team gets two big monitors and the peripherals to go with them. IBM Canada Lab has free snacks and good coffee.
Our team operates at scale. Millions of learners use our education portals and containerized, microservice-driven hands-on labs. We develop with Python and Ruby, and we operate with Terraform and Kubernetes. Our course authors teach AI, machine learning, and other exciting technologies. There are a lot more details in the job descriptions above.
I think these are great opportunities. If you’re in school, I encourage you to apply and to refer your friends.