2015 Year in Review

It’s that time of year again — time for a retrospective on how I did on my goals for the year. I had 5 main goals for 2015:

  • Job Hunting
  • Conferences
  • Blogging
  • Programming Language Design
  • Writing an Agile Book

Job Hunting

I got pretty lucky on this one. My main contract with Mercy got extended several times. Amos and I must have been doing a good job of keeping the customer happy. We even made it through a couple rounds of layoffs. I’m wrapping up the gig at Mercy now. I’m working one day a week there, as the project winds down.

I also started a new gig this month at CenturyLink. I’m working on a cloud development team. Our current project involves selling WordPress as a service. The manager had been courting me for most of the year. I’m excited about my new role; I’ll be writing about it in a blog post soon.

Conferences

I set a goal in 2014 to give my first conference talk. I accomplished that, giving an ambitious talk at RubyConf. I enjoyed having done that, and vowed to do more conference speaking.

I gave 3 conference talks in 2015. I gave a workshop on HTTP at RailsConf. I talked about immutable infrastructure at Madison+ Ruby. At RubyConf, I gave a talk on a micro-ORM I wrote. I also gave a lightning talk about Agile estimation (#noestimates).

I was an alternate speaker at Windy City Rails, but did not give my talk on Alternatives to ActiveRecord. I also went to Strange Loop, mainly to see several friends and acquaintances speak.

Blogging

I wrote 24 blog articles this year. That’s about one every other week. What really kept me going was participating in a writing pact. When the pact was going, I had a 75% blogging rate. That’s pretty good.

I’m not so sure about the quality of my blog writing though. I know that practicing writing is supposed to make you better. I know I wrote some really good articles over the past year, but I think I also wrote some articles that weren’t very good. I think sometimes the deadline has caused more harm than good. I’m not really sure what to do about that; perhaps just pushing on is the right answer.

Programming Language Design

I’ve taken a lot of notes on the design of my programming language. Any time I learn something interesting about another language, or come up with another idea, I write it down.

But I haven’t worked on the implementation. (I last worked on the implementation in 2014.) I should be experimenting with some ideas, implementing them to see how they work out. I’ve even kicked around the idea of starting with a Forth variant, just to get something working quickly.

I haven’t written any articles on my ideas this year either. My notes are pretty extensive, and it would be good to write some articles to help get my thoughts straight.

Writing an Agile Book

I’ve got some things to say about Agile, and want to write a book to express those ideas. I’ve made a start — I’ve got the chapters outlines, and have started on a few chapters. But I haven’t made as much progress as I’d like to. I shared what I’ve got with Amos, and he showed some interest in pairing with me on the writing. Hopefully we’ll work on it together in 2016 and publish it.

Other

There were a few other accomplishments that weren’t explicitly on my list, but I’d like to call attention to.

I’ve continued participating on the This Agile Life podcast. I was in 12 of the 33 episodes that were recorded in 2015. I hope to participate in more in 2016. We’re considering scheduling a standard recording night each week, which might help us record more regularly.

I recently took over as maintainer of Virtus, a library to declare attributes for Ruby model classes. I haven’t done a lot yet, since I’ve been busy with travel, vacation, and holidays. But I hope to catch up with all the pending pull requests and issues in the next month or so.

The accomplishment I’m most proud of is mentoring for the Roy Clay Sr. Tech Impact program. This is a program begun as a result of the Ferguson protest movement. We’re helping teach kids (from 14 to 25) web design and development. My personal goal was to give these kids an opportunity that they would not have otherwise had. But it turned out that some of them have actually started a business building web sites for small companies. I’m so proud of the progress they’ve made in such a short time; it’s a challenging program.

Conclusion

I’m pretty happy with my accomplishments this year. I made at least some progress on each of the goals I set. I’ve been thinking about my goals for next year; I’ll write that as a separate blog article next week.

Not Quite Callbacks

I’ve been working on application architectures based on Uncle Bob’s Ruby Midwest talk, following the hexagonal architectural pattern. I posted an article a couple months ago showing a way that works fairly well in Rails, and some accompanying Rails example code. But there was one thing I wasn’t quite happy with.

The problem is that we used callbacks (actually, a publish/subscribe mechanism) in a situation where they don’t seem to quite fit:

  def show
    interactor.on(:display) { |order| render order }
    interactor.on(:not_found) { |order_id| render status: 404 }
    interactor.get(params[:id])
  end

What we really want is to respond in different ways, depending on the result of the call to interactor.get(). There’s no good reason to define the responses before the call. It makes a lot more sense to define the responses after the call, because they’ll happen after the call. I’d much prefer that the code be written in the order that it will be run.

I discussed this problem with my friend and colleague, Amos King. We came up with a better solution, which puts things back in the right order:

  def show
    interactor.get(params[:id]) do |on|
      on.display { |order| render order }
      on.not_found { |order_id| render status: 404 }
    end
  end

He even wrote a small library to do this, which he called Riposte. I’m not sure what to call this pattern, but it seems to work pretty well in this situation. I suppose that they’re still technically callbacks, because they’re passed in in the block that’s passed in to the call to interactor.get(). But due to the magic of Ruby blocks, we get to put them in the order they should be.

Riposte also gives you the option of using the response object directly, instead of passing a block:

  def show
    on = interactor.get(params[:id])
    on.display { |order| render order }
    on.not_found { |order_id| render status: 404 }
  end

This shows that it’s just returning an object, with the twist that the response object has methods that take blocks. The nested blocks variant is really the same thing, except that it’s yielding to the response object instead of returning it.

I’ve decide that is the pattern I’d like to use for interactions and their callers within Ruby hexagonal architecture.

Architectural Thoughts

I’ve started working on my own framework in Ruby in the past couple days. It’s built upon my recent work at understanding Uncle Bob’s Ruby Midwest 2011 talk, and his article on Clean Architecture, and the resulting hexagonal architecture (AKA ports and adapters).

Somehow my research in that vein led me to Gary Bernhardt’s Boundaries talk. I’ve read a lot about the talk, and knew about the idea of “functional core / imperative shell”. And I’ve worked with a lot of similar ideas lately. But I believe this is the first time that I actually watched the whole video.

Even after having read a lot about similar ideas, it was pretty mind-expanding. Gary’s really good at presenting these kinds of ideas in a simple way.

OOP as usually taught includes encapsulation of data together with behavior, with mutable objects. Functional programming separates data and behavior, with mostly immutable data. From experience, encapsulating data and behavior together seems helpful. But experience also shows that immutability is useful. So it would be good to have both of those together. This is something I’ve been thinking for a few years — how best do we get both?

Gary calls the combination “FauxO”. Logic and data are still combined, but there’s no mutation. Anywhere OOP would normally have mutation would just generate a new object. There’s no language restriction involved in enforcing immutability — just discipline.

But without mutability, it’s hard to do IO and maintain state. So Gary’s solution is to encapsulate as much as possible into an immutable (functional or FauxO) core, and around that, use an imperative (traditional OOP) shell. The functional core contains the bulk of the logic, and the imperative shell is a glue layer that handles the real world, including disk, network, and other I/O.

The result of this is that the shell has fewer paths, but more dependencies. The core contains no dependencies, but encapsulates the different logic paths. So we’re encapsulating dependencies on one side, and business logic on the other side. Or put another way, the way to figure out the separation is by doing as much as you can without mutation, and then encapsulating the mutation separately.

I love how this naturally breaks things up, so that the core is all testable with unit tests, and the imperative shell is tested with integration tests. And since the shell has few or no logic paths, you get the testing pyramid, with more unit tests and fewer integration tests. The whole thing ends up being quite beautiful. Tests end up being very fast without any extra effort — not even stubbing or mocking. This tells us that things have been decomposed very well — an elegant design.

Gary makes the case that immutable objects can be treated as values, and passed across boundaries. Even process boundaries. This is something I’ve noticed as I’ve been working on my own Uncle Bob style hexagonal framework, but nobody in that camp ever mentioned that — they prefer DTOs or something more like hashes. I’m completely against hashes, because of the “stringly-typed” problem. And I don’t see much advantage in a DTO if I’ve got an immutable object; I’d be basically copying the object to an almost identical object. And I’d be losing any laziness possible for derived values within the original immutable object.

It’s striking to me how Gary’s image of an imperative shell around a functional core, plus Net, Disk, and State outside of the shell mirror’s Uncle Bob’s concentric circles. Uncle Bob has entities in the middle, surrounded by use cases, surrounded by Web, DB, and UI.

Another advantage that Gary shows is that breaking things up this way allows easy concurrency. In his example, he shows using the actor model — either just using threads and queues, or an actor library (or language feature).

After several years of thinking about the architectural issues seen in most large Rails apps, I’m starting to come to an understanding of how to combine all these ideas and come up with an architecture that will work better.

 

Hexagonal Rails Controllers

I’ve had a long love-hate relationship with Rails. I love the MVC framework and how it’s improved our speed of writing web apps. But I’ve never really been completely happy with it. I don’t generally agree with most of its opinions. I prefer models that follow the Data Mapper pattern, not the Active Record pattern. This includes separating the persistence layer from the models’ business logic. I prefer Slim or HAML to ERB. I prefer RSpec to Test::Unit or MiniTest. When Merb hit the scene, I was ready to make the jump, until Merb merged with Rails.

So inspired by PJ Hagerty’s recent article on alternative Ruby web frameworks, I started thinking about how I’d write a replacement for Rails. I’d definitely keep the basic MVC framework. But I’d also want to implement a more hexagonal architecture.

I started sketching out what this would look like, but I ended up starting with a Rails controller and finding the simplest way to make it hexagonal. I really don’t like callbacks, because they make tracing program execution difficult. But I didn’t see any other alternative. I found a simple pub/sub Ruby library called Wisper. It literally has only publish, subscribe, and on methods. (You use on to register single callbacks via blocks, and subscribe to register an object with method names corresponding to the callback names.)

The trick was figuring out how to break the controller into 2 pieces. What finally helped me was to find the single responsibilities of the 2 pieces. The Rails controller would remain in charge of managing the web interface, but would delegate to the other piece to handle any application-specific business logic. I decided to re-watch Uncle Bob Martin’s “Architecture The Lost Years” talk, which was the first time I was introduced to the ideas of Hexagonal Architecture. (He doesn’t name the architecture in the talk, but later calls it Clean Architecture.) He does a decent job of explaining how to break these 2 pieces apart. He used the term “interactor” in that talk, so I decided to go with that. He said that Jacobsen calls it a Control Object in Object Oriented Software Engineering, but that’s too close to Rails’s “controller”.

So here’s an example of what I ended up with:

class OrderController < ApplicationController
  def index
    interactor.on(:display) { |orders| render orders }
    interactor.list
  end

  def show
    interactor.on(:display) { |order| render order }
    interactor.on(:not_found) { |order_id| render status: 404 }
    interactor.get(params[:id])
  end

private

  def interactor
    @interactor ||= OrderInteractor.new
  end
end
require "wisper"
require "order"

class OrderInteractor
  include Wisper.publisher

  def list
    orders = Order.all
    publish(:display, orders)
  end

  def get(id)
    order = Order.find(id)
    publish(:display, order)
  rescue ActiveRecord::RecordNotFound
    publish(:not_found, id)
  end
end

I do have a few problems with this solution though. I’m not a fan of the name “interactor” for the business logic. I thought about calling it OrderOperator, or maybe OrderOperations, because it’s really a collection of operations. Perhaps it would be better to separate each operation into a separate class. Trailblazer does it that way. And for more complicated business logic, I would do that too, using the Method Object pattern. But like a Rails controller, there’s a lot in common among all the operations. I feel like a separate class for each operation for each would create too many coupled classes.

I’m also uncomfortable with the fact that the controller is delegating almost everything to the interactor. I guess this is OK, but it feels like there’s too little left when every line starts with interactor. I suppose extracting things some more would help mitigate this concern I’ll likely write a small gem to perform that extraction. I expect that that will allow a typical controller to be written in only a few lines. And maybe the same for the interactor side.

With the business logic extracted out of the controller, it was really easy for me to write a command-line version of the app. As Uncle Bob says, “the web is not particularly important to your application.”

I’ve put the code for this example on GitHub: https://github.com/boochtek/hexagonal-rails. I’ll likely experiment with it some more over the next few weeks and months.

Resolutions

January kept me pretty busy, so I’m a little late to this. But better late than never. And as an Agile practitioner, I don’t think personal retrospectives should be limited to one time of year.

Review of 2014

Last year I wrote a blog entry listing my goals for 2014. As far as New Year’s resolutions go, I was relatively successful — about 50% of my goals accomplished. Unfortunately, my Open Source contributions weren’t as strong as I had hoped; while I released some of my own work, I didn’t do much else. I did increase my blogging; getting in on a weekly blogging pact helped immensely. I also increased my participation on the This Agile Life podcast to a level that I’m happy with. But the accomplishment I’m most proud of was giving a presentation at RubyConf.

Plans for 2015

I’d like to keep things rolling from last year, but crank up a few things. My plans are quite ambitious, so I don’t expect to get everything done by any means. But I think by setting the bar high, I’ll end up with a lot I can be proud of.

Job Hunting

Late last year, I took the jump into independent consulting. So far, I’ve really enjoyed it, and I’m booked up through April. My wife graduates in May, so we’ve got the possibility of moving if that makes sense. So I’ll be looking for consulting projects in town, but I’ll also be looking at jobs in San Francisco and Chicago. The possibilities are exciting, and I’ll be taking my time to find something just right.

Conferences

I was incredibly nervous leading up to my RubyConf presentation. Part of that was just the common fear of public speaking. For me, that only kicks in at around 100 people, and this audience was around 250. I think another reason was that I chose a really ambitious topic, and I kept finding more that I wanted to talk about, but wasn’t prepared for. But I think I did a pretty good job presenting an advanced topic. And I was so pumped by the sense of accomplishment as soon as I finished. So I’m hoping to do it more. I’ve already submitted a couple proposals, and plan to submit several more.

Blogging

I believe that blogging is important for me to get my thoughts down — for myself and to share with others. I was really successful last year when I had a partner to keep me honest, via a pact. So I’ve started up another pact this year, which will hopefully ensure I’ll keep things going. I’ve got a really long backlog of topics, so as long as I keep at it, I’ll have plenty to write about.

I also want to move away from WordPress to a static system — probably Middleman. I’ve got 2 major problems with WordPress. First, I no longer trust its security, nor the security of any application written in PHP. Second, it generates HTML every time someone requests a page, instead of when the content is updated. I find that to be a waste of resources, and problematic from a security standpoint. The main problem with moving to a static blogging system is that I really want to allow comments, pingbacks, and tweetbacks. So I’ll have to find a way to integrate those.

Programming Language Design

Last year I started thinking about programming language design, and started implementing a language tentatively called Brilliant. I’ve done a lot more thinking on the topic, and have a lot of notes. But I haven’t implemented much more yet. This year, I’d like to get my thoughts more organized, and write a series of blog posts on various aspects of language design. The most interesting part seems to be the trade-offs involved in the ways that various language features interact. So I’d like to make some progress on the language implementation, but more importantly, I’d like to get a lot of my design ideas written down.

I’m also going to spend a lot of time learning a bunch more programming languages, so I have a better understanding of possible features, combinations of features, and their interactions. I’ve already start with Elixir, Clojure, and Racket. I’m hoping to also look at OCaml, Factor, and Haskell. I’ll probably also take a look at the 2 “Seven Languages in Seven Weeks” books.

Agile Book

I think people often have trouble getting started with Agile. I started on a book last year, and got down quite a lot of good ideas. But I realized that I’m going to have a hard time organizing all those ideas into something coherent. Still, I’d like to try to get something out there that lets people get started with Agile. My idea is to present a toolbox of practices to get started with and build on that foundation over time with additional practices. Sort of a playbook on how to get started over the first 6 to 12 months and be successful. I want to make some progress on the book, at least enough to decide whether it’s worth the effort to finish it and self-publish it.

 

Ruby Pattern: Parameterized Module Inclusion

I’ve run across a pattern in Ruby lately that I really like. It solves some problems that I’ve struggled with for several years. Let me start with the problems.

Let’s say you want to include an ORM in a model class, and want to tell it what database table to use. Typically, you’d do this:

class User
  include MyORM::Model
  table 'people'
end

But that table call is more like an option to the module inclusion than anything else. So what we’d really like is something like this:

class User
  include MyORM::Model, table: 'people'
end

But that’s not valid Ruby; include doesn’t let you pass anything other than a module.

So when I was learning about Virtus, I noticed that its example of how to include it is a bit different than the standard Ruby idiomatic include:

class User
  include Virtus.model
end

At first glance, it reads like the first example. But on closer inspection and consideration, it’s quite a bit different. Where MyORM::Model is a constant that refers to a module, Virtus.model is a method call. So there’s a method named model in the Virtus module. That method returns another module — which is exactly what’s needed in order to include it into our model class.

The easiest way to implement Virtus.model would be this:

module Virtus
  def model
    ::Virtus::Model
  end
end

module Virtus::Model
  # ...
end

If Virtus.model doesn’t need to take any arguments, that’s perfectly fine. In fact, I’ve started to use this implementation of the pattern for modules that don’t need parameters.

Because Virtus.model is a method, we can also call it with options:

class User
  include Virtus.model(constructor: false, mass_assignment: false)
end

We could even pass a block. But how do we process those options? There are a few different ways. However we do it, we have to be sure to return a module. And we can create modules in a few different ways.

Virtus uses the builder pattern. It takes the parameters passed in and builds a module dynamically. By that, I mean that it calls Module.new and then adds methods to that module. It does this by mixing in other modules, but it could do it by dynamically defining methods as well.

I’ve never seen this pattern in any other language. It’s obviously only possible because we can dynamically create modules.

The use of this idiom seems to be catching on a bit in the Ruby community. I’ve started using it myself, and will be adding it to my Includable::ActiveRecord gem soon.

 

 

Brilliant – My Very Own Programming Language

I’ve decided to design and implement my own programming language, and call it Brilliant.

I’ve been interested in programming languages and linguistics almost as long as I’ve been using computers. I’ve long thought that if I ever go back to college, it’s likely that I’ll concentrate on programming languages as a specialty.

My recent discovery and involvement with the Crystal programming language has gotten me excited about new language ideas. It’s also helped me realize that implementing a language myself is feasible, and that there’s no better time to start than now.

Implementation

For now, the Brilliant implementation doesn’t let you do much more than “Hello World”. But you gotta start somewhere. So this article isn’t really “Introducing Brilliant” so much as the start of a series of articles on its design and implementation.

I wanted to use a PEG (parsing expression grammar) parser for several reasons. For one, they seem to have gained a lot of popularity in the past few years. Also, PEGs cannot be ambiguous, which can solve a few difficult problems, such as the “dangling else“. Perhaps my favorite feature of PEG grammars is that you don’t need a separate tokenizer (lexer). This provides a nice advantage in that we can use keywords like “class” as variables, as long as they’re not used in a place where the keyword would make sense.

So knowing that I wanted to use a PEG, I had to find a PEG parser. I kind of wanted to use ANTLR, which has been a leading parser for many years. But the PEG seems to be new to version 4, and I couldn’t find any Ruby bindings for version 4. Treetop seems to be the most popular parser for Ruby, but I found the EBNF format that Rattler uses to be more to my taste. I think the fact that it’s newer also gives it a few advantages, having had a chance to learn some lessons from Treetop.

I thought about using the Rubinius VM, but decided to go with LLVM, mainly since it has slightly better docs for Ruby, and because it’s what Crystal uses. Also, it’s pretty easy to get it to compile to a binary executable or run in a JIT. In the future, I might consider switching to the Rubinius VM, the Erlang VM, or the Perl 6 VM (Parrot). But for now, I like the idea of being able to compile to a binary and easily interface with C, just like Crystal.

Goals

My main goal is to have fun playing around with language ideas.

I’ve found a really great language in Ruby, so I’ll be using it as a starting point. But Ruby does have its faults. In some ways, I want to answer the question “what would Ruby look like if we designed it today?”.

But I also want to explore other ideas. What if objects defaulted to immutable? What if functions and methods were assumed to be pure by default? Might it be possible to infer the purity of a function or method? (If so, we could automatically memoize them.) Can we make creating an actor as easy as creating an object?

I’ll also be looking at ideas from other programming languages. Could we get some of the benefits of Haskell’s purity without having to do somersaults to do IO? Could we mix Python’s indentation style scoping with brace style or begin/end style? Could we integrate Icon’s ideas about success and failure (goal-directed execution)? What interesting features can we pull from Ada, Io, CoffeeScript, Crystal, Rubinius, Perl 6, etc.?

I’m not so much as interested in cutting-edge features, as in features that can be easily used by the average programmer. More importantly, I’m interested in how features interact with each other, so they fit well together to create a whole that’s greater than the sum of the parts.

Naming

I wanted to name my programming language “Fermat”. I’ve long been intrigued by Fermat’s Last Theorem, and Fermat’s Little Theorem is important in number theory. Unfortunately, there’s already a computer algebra system with that name.

So I decided to find a name in the same vein as “Ruby” and “Crystal”. I looked at the Wikipedia page for “gemstones” for some inspiration. A lot of the names of gemstones are already taken. I considered some obscure gemstones, but saw the word “brilliant” and thought it was decent. It’s not the name of another gemstone, but still evokes some similarity to Ruby and Crystal.

So that’s the name for now. Perhaps I’ll decide to change it at some point in the future, but I needed a name for the project, as well as a file extension for source code files. I chose “bril” for that. I suppose “br” would be a better choice. Perhaps I’ll change that before the next article in this series.

Future

I hope to work on Brilliant every once in a while. I expect it’ll take a couple years before it’s really very useful.

When I do add a major feature, I’ll be certain to blog about it. I’ve got tons of ideas strewn about in various files. It would be great to get them organized and published —and even better to get them implemented.

Burying the Lede

Most of us don’t write very readable shell scripts. There are plenty of things we could do better, but today I want to talk about one in particular — burying the lede.

The term “burying the lede” comes from the field of journalism. Here’s the Wiktionary definition:

To begin a story with details of secondary importance to the reader while postponing more essential points or facts.

Like a good news article, code should tell a story. And the story should start with what’s most important. In the case of code, the most important information is the high-level functionality — a succinct summary of what the program does. In other words, write (and organize) the code top-down, as opposed to bottom-up.

Unfortunately, shell script doesn’t make this easy. Due to the way shell scripts are interpreted, you can’t call a function until after you’ve defined it. This leads to most of us structuring our code like this:

function do_something { ... }
function do_something_else { ... }

do_something
do_something_else

The problem with this is that the function definitions will likely take quite a few lines, and we won’t see what the top-level functionality is until we reach the end of the script.

I’d like to propose a standard way to structure shell scripts to mitigate this issue. (I’m really only talking about shell scripts that have function definitions within them.) I’m sure I’ve seen a few scripts do this, but it’s not very common at all.

My proposal is simple:

function main {
  do_something
  do_something_else
}
function do_something { ... }
function do_something_else { ... }

main

This structure lets us start with the lede. We describe the top-level functionality right away. Only then do we get to the secondary details. The name main makes it pretty clear that it contains the top-level functionality.

I’ve recently started writing my shell code like this, and I’m happy with the results. I’ve also started to use some other programming techniques in my shell scripts to improve readability: better naming, extracting more methods, and moving helper methods into separate files. It feels good to treat shell scripts like real code instead of just some stuff I’ve hacked together.

PS. The WordPress theme I’m currently using (Twenty Eleven) also buries the lede — I can barely even see the title of the blog post on my screen without scrolling. I’m going to have to change that soon.

Yak Shaving #1: Cursor Keys

I recently decided to start using Emacs again. I used it extensively from the early 1990s until the early 2000s. I pretty much stopped using it when I had a sysadmin job with no Emacs on the servers, and no ability to install it. With the rising popularity of tmux and tmate for remote pairing, and my dislike for vim’s modes, I decided to try going back to Emacs in the terminal.

One thing I really want in a text editor is good cursor key support. Shifted cursor keys
should select text, and Control+left and right should move by words. (Apple HIG says to use Option+left and right to move by words; most apps on Mac OS X seem to support both.) Things have worked this way on almost every text editor on every OS I’ve used — Amiga, DOS, Windows, NeXT, Mac, Motif, Gnome, KDE. It’s a part of the CUA standard that’s been in common usage on everything since the mid-1980s.

Enabling cursor keys in Emacs was pretty easy. I’ve decided to use Prelude to make getting started with Emacs easy. Emacs comes with the cursor keys enabled, but Prelude disables them. Undoing Prelude’s change is pretty easy:

(setq prelude-guru nil)

Trying to make shifted cursor keys work is where the trouble began. They work in the GUI version of Emacs, but not from the terminal. It turns out that the Mac Terminal doesn’t distinguish between cursor keys and shifted cursor keys in its default configuration. So I had to figure out how to configure key bindings in Terminal.

That’s easy enough — they’re in the preferences. But what should I set them to? This took a lot of research. Terminal emulation and ANSI code sequences are obscure, complex, and inconsistent. I eventually found the info I needed. For starters, Shift+Right, Shift+Left, Shift+Home, and Shift+End are defined in the terminfo. The rest I was able to piece together from various sources around the Internet.

I’m also trying to script my Mac configuration. So instead of manually adding all the keybindings in the Terminal preferences pane, I decided to write a script. Mac OS X does a decent job of allowing you to change preferences from the command line. For example, to always show the tab bar:

defaults write -app Terminal ShowTabBar 1

Easy enough, except for a couple problems. First, I had to figure out the obscure syntax used in the preferences for key codes. I was able to piece these together with some more Internet research. But the really big problem is that the keyboard bindings are 2 levels deep within a “dictionary” (hash map). And the defaults command doesn’t handle that. There are some obscure utilities that handle nested preferences, but they don’t work well with the preferences caching in Mac OS X 10.9 — a problem I ran into while testing.

So now I’m writing a utility in Python that does what the defaults command does, but that will handle nested dictionaries.

There’s a term for this process of having to solve one issue before you can solve another, and the issues get several layers deep. It’s called yak shaving.

Here’s another good example:

Father from Malcolm in the middle having to fix one thing before fixing another, ad infinitum.
Yak shaving – home repairs

I’m sure this will be the first of many posts about me yak shaving.

Includable ActiveRecord

I created a Ruby gem recently, called includable-activerecord. It’s pretty small, but I thought I might explain why I created it, and discuss its implementation.

Classical Inheritance

When you use ActiveRecord, you normally include it in your model like this:

class User < ActiveRecord::Base
  # ...
end

Your User class is inheriting from the ActiveRecord::Base class. This is class-based inheritance, also called “classical” inheritance. (That’s “classical” as in “class”, not as a synonym for “traditional”.) Class-based inheritance represents an “is-a” relationship. So we’re saying that a user is an ActiveRecord base. Another way to say this is that User is a subclass of ActiveRecord::Base.

There are a few problems with this. First, what is a “base”? The name was chosen because it’s a base class. But just like we don’t give factory classes names like UserFactory (at least not in Ruby), we shouldn’t name base classes Base.

I suppose that we’re trying to say that this is an ActiveRecord model. That sounds fine at first glance — this is our model for users. But what if we also want to say that a user is a person? Ruby doesn’t allow inheriting from multiple classes. Now we have to choose whether to inherit from ActiveRecord::Base or Person. Person makes more sense, because it fills the “is-a” rule better. Class inheritance is intended for a hierarchical “is-a” relationship, such as “a user is a person”, or “a circle is a shape”. But since ActiveRecord::Base is a base class, we have to use it as our base class.

We could work around this problem by subclassing Person from ActiveRecord::Base and then subclassing User from Person. That’s fine if Person is also a model that we store in the database. But if that’s not the case, then we have a problem.

Mixins

Ruby provides another way of implementing inheritance — mixins. We often don’t think of this as an inheritance model, but it really is. When we include a module, that module gets added to the class’s ancestor chain. We can mix in as many modules as we want.

Mixins indicate more of an “acts like” relationship than an “is-a” relationship. It’s for shared behavior between classes that don’t have a hierarchical relationship. For example, when we mix in the Enumerable module, we’re saying that we want our class to act like other classes that include Enumerable. That sounds more like what we want ActiveRecord to be. We want our user model to behave like other ActiveRecord models, in the way that they can persist to a database.

But ActiveRecord doesn’t support that. Almost all the other Ruby ORMs do; as we’ve shown above, this is for good reasons.

Implementation

So I decided to see if I could implement the equivalent of the ActiveRecord::Base class as a module that could be mixed into model classes. I decided to call my mixin module ActiveRecord::Model, because classes that mix it in will behave as ActiveRecord models.

It turns out that ActiveRecord::Base is a pretty complex class. It includes and extends a lot of other modules. Luckily, as of ActiveRecord 4.0, that’s all the code it includes.

The module only defines a single class method, included. This is one of Ruby’s many hook methods. It gets called when the module in question gets included in another module, and receives that other model as its argument. All we need to have this method do is to include everything that ActiveRecord::Base includes, and extend everything that ActiveRecord::Base extends. Ruby provides a method that’s defined on all classes, called included_modules, which we can use to get the list of everything that’s included in ActiveRecord::Base. Unfortunately, there’s no equivalent list of extended_modules. But a quick search on Stack Overflow found an implementation of extended_modules that we could use.

So with a bit of magic (i.e. hooks and meta-programming), we can get the lists of constituent modules from the ActiveRecord::Base class, and include them in our ActiveRecord::Model module.

So with all that, we can now include the includable-activerecord gem and mix it in, with all the advantages that provides:

class User
  include ActiveRecord::Model
  # ...
end

It was exciting to be able to make this work. Since I wrote it as a proof of concept, I haven’t written any tests yet. But it seems to be working just fine. The main thing I really need to look into is making sure that plugins that extend ActiveRecord::Base from their own code will still work. I’m pretty sure this will work out of the box, because the ActiveRecord::Model.included doesn’t run until the model class is loaded, and that happens after those plugins have initialized themselves.