It’s been a year of migrations. I moved my email to Fastmail. I set up a NAS to avoid Dropbox lock-in. I moved my Mastodon fediverse account to CoSocial. And now I’ve moved WordPress hosts.
There’s more I can do to be a more elbows-up Canadian and to embrace small tech. One step at a time, though.
This May 1, I joined Upgrade Inc as a Principal DevOps Engineer on the Infrastructure Performance Management (IPM) team. Technically this is a change of industries from automotive to financial. I was previously at Autonomic, a Ford subsidiary, as a Principal Site Reliability Engineer on the Observability team there. In practice, my clients continue to be my fellow engineers.
I learned a lot in my time at Autonomic from many wonderful people. Before Au, I had only a superficial knowledge of Kubernetes, and certainly had never touched the Go programming language. Nothing is a better teacher than many, many hands-on projects. Upon joining Upgrade, I discovered that they were already using something I had written and open sourced in Go, which feels like an unexpected compliment.
As I was finishing up at Au, one of my colleagues asked me — how am I so productive? To some extent that the illusion of the public record — we know our failings, but of others we mostly only know their successes. Still, an interesting question. It’s not because I’m immune to procrastination — I scroll through the internet as much as anyone else. It’s not because I put in more hours — overtime is for exceptional circumstances, not everyday, and it is not sustainable. I think I appear productive because I love solving the puzzles that come up in our profession, and completed projects have a way of accumulating.
I’m very excited to be joining Autonomic this coming Monday. They do self-driving car data. I’m joining as Senior Site Reliability Engineer to help grow the infrastructure by several degrees of magnitude. I expect to be doing a lot of Kubernetes and learning Golang as I go.
This draws to a close 14 years at IBM. My last few months at IBM have been spent recruiting some very strong team members, and I trust that the IBM Developer Skills Network and Cognitive Class are in great hands.
I joined IBM as co-op student. My university buddy Gerald had set his MSN Messenger status to “Message me if you want a job”. It was late in the semester and I was singularly bad at interviewing back then, so I jumped on the opportunity. I spent that year writing tutorials for using the IBM Db2 database and its exciting XML features with various programming languages. A few years later, that morphed into putting Db2 on the cloud, then Apache Hadoop, and then developing and operating an education site about all sorts of open source technologies with hands-on labs and a couple million students.
I’m leaving IBM as a Senior Software Developer Manager (or perhaps a Principal Engineer), but on a technical side what I do has been Site Reliability Engineering. Past a certain scale, you need to start thinking of operations in terms of acceptable error budgets and of instrumenting the right service level indicators to properly judge the health of many large, distributes services coming together to deliver an experience. I’m excited to have the opportunity to apply the same principles at ever larger scales at Autonomic.
I wish to officially register my complaint about the lack of a gnome emoji in Unicode. These folks exist:
Yet there is no gnome. If there had been, I could have said that I’m joining 🚗nomic in even fewer characters!
Planet Db2 and Planet Big Data are two blog aggregators that I created when I first started at IBM. They were similar to Planet Debian or Planet Intertwingly but about the Db2 relational database server and about big data technologies like Hadoop. If you are interested in taking over the blogrolls, please contact me.
Blog aggregators are something from an earlier era of the web. We live in a closed off world of walled gardens and information silos like Facebook, Snapchat, and TikTok. In an earlier era, people ran their own blogs independently of any social network. The content was published in a machine-readable format that anyone or anything could pull to get notifications. Anyone could set up the equivalent of the Facebook Wall by using a tool to aggregate — that is, to federate — other people’s feeds.
I still inhabit that lost world to some extent because I use NewsBlur to subscribe to RSS and Atom feeds for news sources like Ars Technica, webcomics like XKCD, and so on. NewsBlur is one of many successors to the late Google Reader. It’s great because I get exactly the content that I have myself curated as something that I care about with none of the noise and none of the ads of Facebook or even Reddit.
The Planets are based on the Planet Venus software created by Sam Ruby. The architecture is that there’s a scheduled cron job that runs at a regular interval to pull updates from all the blogs in the blogroll. It then generates a static site that can be served by any webserver like Apache, Nginx, and so on.
I’m shutting the planets down because the current incarnation is too old to update in place, and I can’t justify spending the time myself to recreate the sites on new technology.
If you are member of the community who’s interested in taking over Planet Db2 or Planet Big Data, please contact me.
This weekend Brandon and I are at the Datathon for Diabetes in Boston. It starts tonight at 5 and goes all day Saturday. The goal is to use publicly available data to generate an insightful and innovative analysis of diabetes in United States and abroad.
We’re sponsoring a prize for the team that makes best use of Data Scientist Workbench in their solution. Novo Nordisk and Deloitte are also sponsoring a prize each.
Our prize consists of a FitBit Charge HR for each member of the winning team.
I think it’s worthwhile to learn and apply Spark as a tool to the problem of diabetes. Spark is an open source framework that lets you run your data analysis in parallel on multiple machines for speed and ability to work with large amounts of data.
Data Scientist Workbench has Spark ready to use with Python, Scala, and R in Jupyter, Zeppelin, and R Studio IDE.
If you run into trouble at the datathon, come up and ask me any question you like. I’ll be there for the duration as a mentor. As always, if you run into a Data Scientist Workbench issue, you should also open a support ticket.
May 11-12 is Datapalooza Beijing and May 19 is Datapalooza Denver. Also, Big Data University is now posting events on its Facebook page.
On Tuesday March 29, I’ll be demoing Data Scientist Workbench (DSWB) at Data Day 3.0 for the Carleton University Institute of Data Science.
I’m in Ottawa the weekend before, so feel free to ping me and connect. I’m on Twitter as @leonsp.
DataScientistWorkbench.com hosts open source data science tools for you for free. The tools include Jupyter and Zeppelin notebooks for developing and documenting your algorithms, R Studio IDE for focusing on your R code, and OpenRefine for cleaning your data.
The event is organized by the Carleton University Institute of Data Science in Ottawa. It runs from 8am to 3:30pm in the River building. You can find more details on their event page.
Next week I’ll be demoing Data Scientist Workbench at Spark Summit East (official site) in New York. Polong Lin will be there with me. Come by the expo floor next Wednesday and Thursday and chat with us.
Data Scientist Workbench is what my team builds. It hosts open source data science tools like Jupyter, OpenRefine, R Studio IDE, Zeppelin and others for you. There’s exciting stuff in the changelog every week.
I signed up in time to get into a training session at Spark Summit East, so I’ll be spending my Tuesday working with the Wikipedia data sets. In today’s industry jargon, I’m more of a data engineer than a data scientist, so I’m hoping my Spark skills are up to the level needed for the advanced course.
This week I’m at Datapalooza Seattle, which is a good opportunity to brush up and expand those same Spark skills. In fact, we just posted the Day 1 challenge for Datapalooza. If you’re following along at home, fire up your Data Scientist Workbench, open a Jupyter notebook, and give it a try.
On February 9 through 11, I’ll be mentoring hackers and budding data scientists at Galvanize during Datapalooza Seattle. It should be a great conference covering topics like things like machine learning, natural language processing, and data engineering infrastructure.
Last year’s Datapalooza in San Francisco was a fantastic event with lots of in-depth sessions. I was impressed with the range of material on data science and data engineering. The upcoming Datapalooza Seattle looks equally as fascinating.
My team at work runs Data Scientist Workbench which is free hosted suite of open source tools including Jupyter, Zeppelin, R Studio IDE, and OpenRefine. We also organize free data science education through Big Data University.
I’m expecting Antonio Cangiano, Polong Lin, and Leon Katsnelson to be at Datapalooza with me as fellow mentors.
Let me know if you’re in Seattle at the same time and we’ll connect.
Adobe was recently breached and 150,000,000 user accounts were stolen. Adobe was following the one of the worst practices of password storage — reversible encryption (rather than hashing with a salt using a good, slow algorithm like bcrypt). A very, very old throwaway password of mine was among those leaked.
XKCD has referred to this breach as The Greatest Crossword Puzzle in the History of the World!
![It was bound to happen eventually. This data theft will enable almost limitless [xkcd.com/792]-style password reuse attacks in the coming weeks. There's only one group that comes out of this looking smart: Everyone who pirated Photoshop.](http://i.imgur.com/mgVu9VR.png "It was bound to happen eventually. This data theft will enable almost limitless [xkcd.com/792]-style password reuse attacks in the coming weeks. There's only one group that comes out of this looking smart: Everyone who pirated Photoshop.")
With the help of LastPass’ Has Adobe Leaked My Password, let me illustrate why:
The following hints have been used by other people that share your password. This information could be used to determine your password as well.
Would you care to guess what password the naive, young me used for Adobe?
Next steps
On October 22, I’ll be climbing the CN Tower stairs for United Way. Any contribution is appreciated.
